Hello,
I'm now trying to compile with the MPI flag, and obtain this error:

Code (]
[tristan@babylon HYPHY_Source):

./build.sh MPI Linux Checking for curl Curl seems to be present COMPILER=g++, gcc COMPILER_FLAGS= -w -c -fsigned-char -O3 -fpermissive -I/home/tristan/bin/HYPHY_Source/Source -I/home/tristan/bin/HYPHY_Source/Source/SQLite +-----------------------------------------------------------+ |Building a single-threaded HYPHYKernelMPI for MPI          | +-----------------------------------------------------------+ Building baseobj.cpp In file included from category.h:33,                 from likefunc.h:33,                 from baseobj.cpp:44: batchlan.h:333:18: error: mpi.h: No such file or directory baseobj.cpp: In function ‘bool GlobalShutdown()’: baseobj.cpp:224: error: ‘MPI_COMM_WORLD’ was not declared in this scope baseobj.cpp:224: error: ‘MPI_Comm_rank’ was not declared in this scope baseobj.cpp:225: error: ‘MPI_Comm_size’ was not declared in this scope baseobj.cpp:301: error: ‘MPI_Finalize’ was not declared in this scope

I have lam install (I use it to run mrbayes with MPI, and it works fine). Librairies and binaries seem present:


Code (]
[tristan@babylon HYPHY_Source):

locate mpicc /etc/alternatives/mpicc /etc/alternatives/mpicc.1.gz /usr/bin/mpicc.lam /usr/bin/mpicc /usr/share/man/man1/mpicc.lam.1.gz /usr/share/man/man1/mpicc.1.gz [tristan@babylon HYPHY_Source] locate mpic++ /etc/alternatives/mpic++.1.gz /etc/alternatives/mpic++ /usr/bin/mpic++.lam /usr/bin/mpic++ /usr/share/man/man1/mpic++.lam.1.gz /usr/share/man/man1/mpic++.1.gz [tristan@babylon HYPHY_Source] locate mpi.h /home/tristan/tmp/mrbayes-3.1.2/macmpi.h /home/tristan/tmp/wise2.2.0/src/dyc/dbmpi.h /usr/include/lam/mpi.h

Is someone knows what's the problem?
(I did not found the solution for MP, and thus try to solve the problem using MPI, yes that's not nice...)
Thanks, Tristan

Dear Tristan,

Try editing the build.sh file to use 'mpic++' in place of 'g++' as the compiler and see what happens. Also, you may need to add a -I include path for wherever the mpi.h file is.

Cheers,
Sergei

thanks Sergei,
I changed "g++" by "mpic++" (lign 6 of build.sh), and I removed "-lmpich" from the lign 148, and the compilation worked. Great.

than what I did:

[code]
echo "localhost cpu=4" > lamhosts
lamboot lamhosts
mpirun -np 4 ./HYPHYMPI BatchFiles/speedtest.bf
[/code]

and it worked, the four cpus were used. Nevertheless the time needed is not that good: 31s, which is the same time used by a single cpus (which is already too slow)...

Dear Tristan,

speedtest.bf is not written to take advantage of MPI - only MP (it's actually very difficult to efficiently split up a single likelihood evaluation in a distributed environment). A number of standard analyses are MPI enables though, and you should see speed-ups there.

Try to run MPITest.bf to see that the MPI build is working properly.

I am still puzzled as to why gcc would produce such poor code for the Xeon - it is even slower than an old Athlon XP running at 1.33 GHz. Does your gcc version have a 'fast' flag? Perhaps try setting that in the MP (or SP) build and see if CPU specific tweaking will help.

Cheers,
Sergei

Here is the results with MPITest.bf:


Code (][tristan@babylon HYPHY_Source):

mpirun -np 4 ./HYPHYMPI BatchFiles/MPITest.bf Running a HYPHY-MPI test Detected 4 computational nodes Polling slave nodes... Polling node 2... OK Polling node 3... OK Polling node 4... OK Measuring simple job send/receieve throughput... Node     2 sent/received 8072.2 batch jobs per second Node     3 sent/received 9841 batch jobs per second Node     4 sent/received 10086.4 batch jobs per second Measuring relative computational performance... Master node reference index:    1304509 Slave node   1 index:    1367370.     104.82% relative to the master Slave node   2 index:    1368910.     104.94% relative to the master Slave node   3 index:    1352810.     103.70% relative to the master

Then I tried to compile with SP with the -fast and -m64 flags, but:

Code (][tristan@babylon HYPHY_Source):

./build_mod2.sh SP Linux Checking for curl Curl seems to be present COMPILER=g++, gcc COMPILER_FLAGS= -w -c -fast -m64 -fsigned-char -O3 -fpermissive -I/home/tristan/tmp/HYPHY_Source/Source -I/home/tristan/tmp/HYPHY_Source/Source/SQLite +--------------------------------------+ |Building a single threaded HYPHYKernel| +--------------------------------------+ Building baseobj.cpp cc1plus: error: unrecognized command line option "-fast" Error during compilation

Something interesting is that with MP, with the flag CPU=1, one cpu will be used at 100%, while with CPU>1, a single cpu is still used but this time at 120% (which speed up the computation: 31 -> 22s).

By the way, regarding MPI, why the 4 cpus are used at 100% with speedtest since only one is necessary?

I'am afraid I'm lost... thanks Sergei and sorry for all these questions. My aim is to convert a pipeline that test recombination and positive selection at the genome scale  to hyphy. Today the pipeline use many different programs (phyml, phypack, paml,...). It seems that with hyphy, most of the pipeline might be summarized in a single batch file and would furthermore be deployed easily on a cluster.
Tristan

Dear Tristan,

I still don't understand why gcc would generate such poorly optimized Xeon code, but things like that happen. I'd recommend playing with various optimization flags, especially Xeon (or Pentium) specific, and see if anything makes a difference. All of our Linux boxes run on AMD processors, so I can't easily test the Xeon problem. I think Intel provides its Linux c++ compilers (highly tuned) for free - you may want to check that out.

Re: genomic pipeline. It should indeed be possible to make a single batch file, possibly calling other batch files, to run your data through. We have selection and recombination analyses built in (standard analyses), and I am currently working on updating selection screens to allow for testing in the presence of recombination.

You may also want to take a look at Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login - a set of HyPhy scripts that Konrad Scheffler and colleagues developed - for a method to test for selection in the presence of recombination.

I'll be happy to answer any questions you may have along the way.

Cheers,
Sergei

Ouh, I found at least one answer few second before I give up.

On Xeon 64bit (= Xeon EM64T =  nocona cpu), you should compile with the following flags:
[code]
COMPILER_FLAGS=" -w -c -O2 -march=nocona -pipe -fsigned-char -fpermissive -I`pwd`/Source -I`pwd`/Source/SQLite ";
[/code]

now it takes 11s to perform speedtest.bf with SP.

(that does not fixed the MP problem)
Tristan

Dear Tristan,

Thanks so much for the tip! I'll put that into the build.sh file for the next update.

Does the MP build (with the new flags) take the same amount of time as the single build?

Cheers,
Sergei

Test with speedtest.bf
SP       -> 12s
MP CPU=1 -> 12s
MP CPU=2 -> 10s
MP CPU=4 -> 11s

Test with speedtest_2.bf:
SP       -> 84s
MP CPU=1 -> 85s
MP CPU=2 -> 85s
MP CPU=4 -> 88s

So, MP still does not work. To the opposite of speedtest.bf, the results with speedtest_2. are similar to the one obtained before modifying the flags (Multimedia File Viewing and Clickable Links are available for Registered Members only!!  You need to Login). (?)

If you want to make some test on a EM64T Xeon, I can open you a session here. (I will be out of reach for one week)
Tristan

Dear Tristan,


Quote:

If you want to make some test on a EM64T Xeon, I can open you a session here. (I will be out of reach for one week)

I'd like to try the Xeon - this odd behavior is a bit troubling.

Cheers,
Sergei