Dear Dan,

The branch-site test (when it works) can only test for DIVERSIFYING (i.e. elevated dN/dS) selection on a branch, not DIRECTIONAL selection. FEL can, likewise test only for diversifying selection. In some cases directional selection can be detected by both tests. YangNielsonBranchSite2005.bf should be equivalent to the PAML implementation (except HyPhy doesn't implement BEB). I have found branch site tests to be subject to serious issues due to model misspecification, i.e. they can tell you that selection happens on branch A, when in fact it is happening elsewhere in the tree, but I don't think there is a published reference for that. I am attaching an unpublished simulation study we did a few years ago - it has some background on the test as well.

The relative merit of REL-type tests (BranchSite) is that they are geared to tell you whether there was elevated dN/dS along branch A somewhere in the alignment (under strong model assumptions), but not so much to detect which SITES experienced said selection for small alignments.

Sergei