HyPhy message board
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl
HYPHY Package >> HyPhy feedback >> very slow analysis
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl?num=1343916870

Message started by daisy on Aug 2nd, 2012 at 7:14am

Title: very slow analysis
Post by daisy on Aug 2nd, 2012 at 7:14am
Hi.

I am having some trouble running the dNdSDistributionComparison.bf analysis. I was trying to run it on a linux sytem using multiple threads, and even when I used very short sequences the analysis would use ~2000% CPU and continue running for several days (still producing output, but VERY slowly!) until I eventually gave up and killed the process. The same analysis on the same sequences takes just a couple of minutes on my thoroughly mediocre windows laptop, using the GUI.

This occurred using the latest version of HYPHY. I have since installed a version from 2008, and that seems to run slightly quicker. However, I used the ‘MP2’ option when compiling it (MP gave me an error message), but the multiple threads don’t seem to work (ie % CPU = 100). And although quicker, it is still running too slowly for any lengthy analysis to be feasible.

I was wondering if you could help me either with any idea as to why the latest version is not working, or how to sort out the multiple threads in the 2008 version?

Cheers in advance

Daisy

Title: Re: very slow analysis
Post by Sergei on Aug 2nd, 2012 at 8:20am
Hi Daisy,

This is indeed very odd. Can you possibly attach the files and the analysis options you had used so that I can try to recreate and debug the issue? Also, can you try running HYPHYMP with the CPU=1 flag (this should turn off multi-threading), to confirm that the processing issue is due to inefficient scheduling?

Sergei

Title: Re: very slow analysis
Post by daisy on Aug 2nd, 2012 at 9:32am
Hi Segei,

The CPU=1 flag doesn’t seem to improve the performance, the analysis is still going at an extremely slow rate even for a very short dummy sequence (I did check the flag had the desired effect, CPU is down to 100%). So the old version of HYPHY from 2008 is still quicker.

I have attached the options I chose and the short dummy sequences I refer to (by the by, the analysis is just as slow/slower for actual genes, so it isn't due to the made-up alignments!). I have also attached the dNdSDistributionComparison batch file I have been using.

Thank you!

Daisy
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl?action=downloadfile;file=comparison_dummy.bf (1 KB | 7 )

Title: Re: very slow analysis
Post by daisy on Aug 2nd, 2012 at 9:34am
It is only letting me attach one file at a time.

Here is the batch file.
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl?action=downloadfile;file=dNdSDistributionComparison.bf (28 KB | 4 )

Title: Re: very slow analysis
Post by Sergei on Aug 2nd, 2012 at 9:58am
Hi Daisy,

Thanks; can you also attach the test sequence/tree files?
You can attach a zip file to include multiple files at once.

Sergei

Title: Re: very slow analysis
Post by daisy on Aug 2nd, 2012 at 12:00pm

Ha, so you can. Here are all the files together.

Daisy
http://www.hyphy.org/cgi-bin/hyphy_forums/YaBB.pl?action=downloadfile;file=seq_and_trees.zip (7 KB | 3 )

Title: Re: very slow analysis
Post by Sergei on Aug 2nd, 2012 at 8:29pm
Hi Daisy,

Please replace the line (around 735)


Code (]
OPTIMIZATION_PRECISION = sop;
[/code):

with

[code]
OPTIMIZATION_PRECISION = 0.001;


in dNdSDistributionComparison.bf.

In the newer version the previous code was setting OPTIMIZATION_PRECISION to 0 (i.e. infinite precision), causing the program to run the optimizer forever (until the hard limit on the total number of likelihood evaluations was hit).

Sergei

Title: Re: very slow analysis
Post by daisy on Aug 3rd, 2012 at 4:26am
Hi Sergei,

Brilliant, that works much better, thank you!

Quick note though: it is still much faster for me to run the analysis using HYPHYSP as opposed to HYPHYMP. This is true for other analyses as well (eg PARRIS), so it’s not something wrong with this particular batch file. Have you any idea what could be going wrong with my multiple threads?

Thank you so much for your help,

Daisy

HyPhy message board » Powered by YaBB 2.5.2!
YaBB Forum Software © 2000-2024. All Rights Reserved.