Patience is the key to this hobby, 3 months is too soon to be throwing in the towel. If it were my tank I would do large water changes (30-40% at least) once a week using RODI water, run some phosphate removers and carbon and keep the lights turned off. If you have no corals in the tank I would leave the lights off for the next month, the hermits won't mind
If you have what looks like brown mucous in your tank then it could be dinoflagelletes. Dinoflagellates can kill snails (it's also aptly named brown snot algae) .