There's not a problem with these, Josh. You could be growing worse things - at least you've provided the optimum environment for SOMEthing...
Lots of people have them in varying degrees - I had a few once, but they didn't turn into the white carpet some get. If you REALLY don't like them, look into some natural control - butterfly fish are notoriously hungry.
