Controlling Sponge Growth?

[Scales]

I am becoming plagued with what I believe is chicken liver sponge. It is just plain ugly IMO and has started covering too many rocks, pressing in on other more desirable inhabitants. I try to "toothbrush" it off during water changes, but am losing the battle, particularly in hard to get areas (where I have unintentionally fragged a few SPS corals trying to "solve" this problem).

Anyone have some advice? Is there a natural solution, something that eats this stuff?