I disagree,
@Lord Sovereign has arrived at the correct conclusion-- and your own argument (esp. "they are worth nothing because they don't stand for the rights of people they disagree with") hints towards that same conclusion.
Why? Because the ADL (which is much worse than the ACLU, as far as I'm aware) doesn't just fail to
stand for the rights of certain others; they outright attempt to
destroy those rights.
And, as you almost certainly know, in a consistent moral framework,
one cannot claim rights that one denies to others. It's a performative contradiction (and, if applied in practice, it is evidently what one might call "the initiation of force").
As such, in your kindness, you have been too charitable to the ADL. Crushing their acts of violation isn't subsequently
also a violation of their rights, because they have
forfeited those rights by violating the rights of others. To compare: shooting someone who shoots at you isn't aggression, either. And that is why silencing someone who is trying to silence others (by force) also isn't aggression.
Destroying the ADL is an act in the defence of free speech.