I use the "Recharge Centre" for CO2 refills. I've also used Sprouse Fire&Safety and B.O.C. and Medigas in the past.
For local drygoods ('cuz I mail-order a fair amount too), I generally go to Elite and Gold. Elite and Wai's are more or less one and the same, so if Elite doesn't have something but Wai's does, I just ask him to grab it for me and come back in a couple of days.
I also get some stuff from Ocean Aquarium from time to time.
I find with my location being so far south, it's hard for me to get to the other stores on a regular basis. Elite and Gold win out for me based on location.. The other stores might have decent stuff too but I just don't get out there much.
I don't use Red Sea salt so I don't know that one. Elite has some stuff, Gold has some stuff.. the largest selection I've seen recently is Red Coral Aquarium up in the NW, they had at least 6 different kinds of salt when I was there a couple weeks ago. I ended up picking up a bucket of Aquamedic salt, I'll give it a try. Hopefully I don't end up hooked on it because that's a longggggggg drive for salt for me
PVC fittings - I think you'll find that most stores have about 95% of whatever you need at any particular moment, so you always have to go to "one more store" to get that last widget. Between Home Depot, Rona, Western Pump, and Plumbateria, I generally find whatever I'm looking for. Plumbateria and Western Pump will carry stuff in more sizes (eg., 1/4" pipe if you need- perfect for airline attachments to reactors and so on). Rona's gotten really better on their selection in the last year or two - they have things like unions now (used to be to find a union the only place to go was Western Pump).
I have not really found a good local source for good true-union fittings. Ie., a ball valve that has a union on both sides. Western pump will at least have the half-union fittings (union on one side). I have not yet found anywhere that has PVC gate valves. I have a vague memory of Kari telling me of a place up in the N.E. called "Canadian Plastics" or "Western Canadian Plastics" and that they had things like bulkheads and true union fittings but I haven't yet found this place... maybe someone else can pipe in if they know the place I'm trying to mention here ..
