Welcome, Guest. Please Login
YaBB - Yet another Bulletin Board
 
  HomeHelpSearchLogin  
 
very slow analysis (Read 5770 times)
daisy
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
very slow analysis
Aug 2nd, 2012 at 7:14am
 
Hi.

I am having some trouble running the dNdSDistributionComparison.bf analysis. I was trying to run it on a linux sytem using multiple threads, and even when I used very short sequences the analysis would use ~2000% CPU and continue running for several days (still producing output, but VERY slowly!) until I eventually gave up and killed the process. The same analysis on the same sequences takes just a couple of minutes on my thoroughly mediocre windows laptop, using the GUI.

This occurred using the latest version of HYPHY. I have since installed a version from 2008, and that seems to run slightly quicker. However, I used the ‘MP2’ option when compiling it (MP gave me an error message), but the multiple threads don’t seem to work (ie % CPU = 100). And although quicker, it is still running too slowly for any lengthy analysis to be feasible.

I was wondering if you could help me either with any idea as to why the latest version is not working, or how to sort out the multiple threads in the 2008 version?

Cheers in advance

Daisy
Back to top
 
 
IP Logged
 
Sergei
YaBB Administrator
*****
Offline


Datamonkeys are forever...

Posts: 1658
UCSD
Gender: male
Re: very slow analysis
Reply #1 - Aug 2nd, 2012 at 8:20am
 
Hi Daisy,

This is indeed very odd. Can you possibly attach the files and the analysis options you had used so that I can try to recreate and debug the issue? Also, can you try running HYPHYMP with the CPU=1 flag (this should turn off multi-threading), to confirm that the processing issue is due to inefficient scheduling?

Sergei
Back to top
 

Associate Professor
Division of Infectious Diseases
Division of Biomedical Informatics
School of Medicine
University of California San Diego
WWW WWW  
IP Logged
 
daisy
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
Re: very slow analysis
Reply #2 - Aug 2nd, 2012 at 9:32am
 
Hi Segei,

The CPU=1 flag doesn’t seem to improve the performance, the analysis is still going at an extremely slow rate even for a very short dummy sequence (I did check the flag had the desired effect, CPU is down to 100%). So the old version of HYPHY from 2008 is still quicker.

I have attached the options I chose and the short dummy sequences I refer to (by the by, the analysis is just as slow/slower for actual genes, so it isn't due to the made-up alignments!). I have also attached the dNdSDistributionComparison batch file I have been using.

Thank you!

Daisy
Back to top
 
Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login (1 KB | 7 )
 
IP Logged
 
daisy
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
Re: very slow analysis
Reply #3 - Aug 2nd, 2012 at 9:34am
 
It is only letting me attach one file at a time.

Here is the batch file.
Back to top
 
Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login (28 KB | 4 )
 
IP Logged
 
Sergei
YaBB Administrator
*****
Offline


Datamonkeys are forever...

Posts: 1658
UCSD
Gender: male
Re: very slow analysis
Reply #4 - Aug 2nd, 2012 at 9:58am
 
Hi Daisy,

Thanks; can you also attach the test sequence/tree files?
You can attach a zip file to include multiple files at once.

Sergei
Back to top
 

Associate Professor
Division of Infectious Diseases
Division of Biomedical Informatics
School of Medicine
University of California San Diego
WWW WWW  
IP Logged
 
daisy
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
Re: very slow analysis
Reply #5 - Aug 2nd, 2012 at 12:00pm
 

Ha, so you can. Here are all the files together.

Daisy
Back to top
 
Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login Login (7 KB | 3 )
 
IP Logged
 
Sergei
YaBB Administrator
*****
Offline


Datamonkeys are forever...

Posts: 1658
UCSD
Gender: male
Re: very slow analysis
Reply #6 - Aug 2nd, 2012 at 8:29pm
 
Hi Daisy,

Please replace the line (around 735)

Code:
OPTIMIZATION_PRECISION = sop;
 



with

Code:
OPTIMIZATION_PRECISION = 0.001;
 



in dNdSDistributionComparison.bf.

In the newer version the previous code was setting OPTIMIZATION_PRECISION to 0 (i.e. infinite precision), causing the program to run the optimizer forever (until the hard limit on the total number of likelihood evaluations was hit).

Sergei
Back to top
 

Associate Professor
Division of Infectious Diseases
Division of Biomedical Informatics
School of Medicine
University of California San Diego
WWW WWW  
IP Logged
 
daisy
YaBB Newbies
*
Offline


Curious HyPhy user

Posts: 6
Re: very slow analysis
Reply #7 - Aug 3rd, 2012 at 4:26am
 
Hi Sergei,

Brilliant, that works much better, thank you!

Quick note though: it is still much faster for me to run the analysis using HYPHYSP as opposed to HYPHYMP. This is true for other analyses as well (eg PARRIS), so it’s not something wrong with this particular batch file. Have you any idea what could be going wrong with my multiple threads?

Thank you so much for your help,

Daisy
Back to top
 
 
IP Logged