Well, even liberals admit that Jews have higher IQs due to genetics. The problem is what H. P. Lovecraft called their "culture-tradition." To paraphrase his point: if you were to have a bunch of people who had a different culture from you come to your country, become rich, and then use their money,
what sort of effect do you think this will have?
I'd suppose, for one thing, if there were a lot of rich Jews, then they'd make sure that anti-Semitism was widely seen as one of the most vile things possible, above other crimes. One can simply look at the page on the US State Department's website
Defining Anti-Semitism (yes, they seriously have this. No, they don't have this for other racial or ethnic groups). This page lists such things as "accusing Jewish citizens of being more loyal to Israel, or to the alleged priorities of Jews worldwide, than to the interests of their own nations" or "drawing comparisons of contemporary Israeli policy to that of the Nazis" or even saying that the Jews control pretty much anything as "anti-Semitism." They also include "double standards" regarding the state of Israel as being anti-Semitic.
It's far too broad a definition, in my honest opinion. Certainly, we should make it illegal for people to call for the genociding of Jews or whatever, but this definition seems to imply that criticizing the wrongdoing of individual Jews is racist. Somehow.