HyPhy message board
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl
Datamonkey Server >> Datamonkey feedback >> Multiple hypothesis testing?
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl?num=1350919915

Message started by rss on Oct 22nd, 2012 at 8:31am

Title: Multiple hypothesis testing?
Post by rss on Oct 22nd, 2012 at 8:31am
Hi,
I was wondering if the p-values that are outputted from the datamonkey interface are corrected for multiple hypothesis testing.  Or, is it necessary to then apply a correction reflecting the fact that a significance test is performed at each polymorphic site in the alignment?
Many thanks!

Title: Re: Multiple hypothesis testing?
Post by konrad on Oct 22nd, 2012 at 4:27pm
Which analysis are you running?

Title: Re: Multiple hypothesis testing?
Post by rss on Oct 23rd, 2012 at 3:51am
SLAC and FEL.

Title: Re: Multiple hypothesis testing?
Post by konrad on Oct 23rd, 2012 at 9:50am
Those p-values are uncorrected. I.e. if you run the same test on a subset of the sites you should get the same values out.

Hope this helps,
Konrad

Title: Re: Multiple hypothesis testing?
Post by Sergei on Oct 23rd, 2012 at 10:51am
Without multiple testing correction, you would expect 5% of all sites (at p = 0.05) to be falsely detected as non-neutral under the strict null (i.e. every site is perfectly neutral). Both SLAC and FEL are quite conservative and real data are non-neutral, so this proportion is going to be much lower.

With multiple testing correction you require that the probability of calling at least one site a false positive under the strict null is 5% -- it is MUCH stricter.

Title: Re: Multiple hypothesis testing?
Post by konrad on Oct 24th, 2012 at 11:26am
Agreed; for most purposes I recommend reporting the list of sites thresholded on the uncorrected p-values, along with q-values (as estimated using the false discovery rate: see Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login) from which the expected number of false positives in the list can be obtained. E.g. as we did in tables 2 and S4-S19 of this paper: Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login.

It is not clear that thresholding on the family-wise error rate (e.g. the Holm- or Bonferroni-corrected p-values) is appropriate for selection analyses, because the aim is usually not to guarantee that the list is completely free of false positives.

HyPhy message board » Powered by YaBB 2.5.2!
YaBB Forum Software © 2000-2024. All Rights Reserved.