Welcome, Guest. Please Login
YaBB - Yet another Bulletin Board
 
  HomeHelpSearchLogin  
 
Multiple hypothesis testing? (Read 4496 times)
rss
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
Multiple hypothesis testing?
Oct 22nd, 2012 at 8:31am
 
Hi,
I was wondering if the p-values that are outputted from the datamonkey interface are corrected for multiple hypothesis testing.  Or, is it necessary to then apply a correction reflecting the fact that a significance test is performed at each polymorphic site in the alignment?
Many thanks!
Back to top
 
 
IP Logged
 
konrad
Junior Member
**
Offline


I love YaBB 1G - SP1!

Posts: 53
Re: Multiple hypothesis testing?
Reply #1 - Oct 22nd, 2012 at 4:27pm
 
Which analysis are you running?
Back to top
 
WWW WWW  
IP Logged
 
rss
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
Re: Multiple hypothesis testing?
Reply #2 - Oct 23rd, 2012 at 3:51am
 
SLAC and FEL.
Back to top
 
 
IP Logged
 
konrad
Junior Member
**
Offline


I love YaBB 1G - SP1!

Posts: 53
Re: Multiple hypothesis testing?
Reply #3 - Oct 23rd, 2012 at 9:50am
 
Those p-values are uncorrected. I.e. if you run the same test on a subset of the sites you should get the same values out.

Hope this helps,
Konrad
Back to top
 
WWW WWW  
IP Logged
 
Sergei
YaBB Administrator
*****
Offline


Datamonkeys are forever...

Posts: 1658
UCSD
Gender: male
Re: Multiple hypothesis testing?
Reply #4 - Oct 23rd, 2012 at 10:51am
 
Without multiple testing correction, you would expect 5% of all sites (at p = 0.05) to be falsely detected as non-neutral under the strict null (i.e. every site is perfectly neutral). Both SLAC and FEL are quite conservative and real data are non-neutral, so this proportion is going to be much lower.

With multiple testing correction you require that the probability of calling at least one site a false positive under the strict null is 5% -- it is MUCH stricter.
Back to top
 

Associate Professor
Division of Infectious Diseases
Division of Biomedical Informatics
School of Medicine
University of California San Diego
WWW WWW  
IP Logged
 
konrad
Junior Member
**
Offline


I love YaBB 1G - SP1!

Posts: 53
Re: Multiple hypothesis testing?
Reply #5 - Oct 24th, 2012 at 11:26am
 
Agreed; for most purposes I recommend reporting the list of sites thresholded on the uncorrected p-values, along with q-values (as estimated using the false discovery rate: see Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login) from which the expected number of false positives in the list can be obtained. E.g. as we did in tables 2 and S4-S19 of this paper: Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login.

It is not clear that thresholding on the family-wise error rate (e.g. the Holm- or Bonferroni-corrected p-values) is appropriate for selection analyses, because the aim is usually not to guarantee that the list is completely free of false positives.
Back to top
 
WWW WWW  
IP Logged