The algae grows from the little spores in like around a circle. Sometimes I will see one of the little spores floating around, and I remove them. The fish don't eat the spores (maybe some type would) and wherever they land they turn all white and slimey. The spores are amber in color - algae appears colorless but has a slight orangey tint out of the aquarium in large quantities.
Should I take the rocks out and scrub them?
Some pics... In the bluer pic you can see some of the little brown spores on the tuft.
I'm not real concerned about it... I'm wondering if it may have just been something since tank start up and since I didn't really bother with much algae control or removal if it's just there. So far the stuff I removed from the back glass has not returned.