How did you test your ground probe?

That's a bit of a 'loaded' question. It can vary depending on the person potentially getting zapped too. Don't want to open myself up for any lawsuits here by putting up numbers. Let's just say if you're getting readings higher than about 20 volts AC, you might wish to investigate to see if there's a component starting to break down in your system.

There are no absolute foolproof guarantees, even GFCI devices can fail or be faulty and everyone's wiring situation is a little different. You say you received a small shock, perhaps it was insufficient to trip the device. Not really sure why or how in your situation, adding the grounding probe suddenly caused the GFCI to trip. They are sensitive devices designed to operate on a very low amperage current differential between line and neutral.

I've had the GFCI trip on occasion simply from plugging in a small powerhead or air pump. Once the device is plugged in and I reset the GFCI, it's fine. I also have a T5HO set up on a digital timer power bar consistently tripping my GFCI. I have two of these set ups that are identical, same power bar, same ballast, same lamps. One trips the GFCI, the other doesn't. I simply don't use the set that trips the device any longer even though there's no indication of a safety issue.

No this was not the case at all, actually The G in gfci stands for ground which most of our equipment in our tanks does not have a ground in the plug-in therefore gfci is quite useless in half of the equipment we run, I was merely pointing out that this is why I would not run without a ground probe.

your right in that it stands for ground, and also right in that most of the equipment doesn't have a ground.

A little off topic but a gfci monitors current in and out on hot and neutral (simplistic terms, there is no in and out etc) if more goes out then comes back on neutral, usually do to a short to ground, then it trips. This explain why a gfci would trip upon the addition of a ground rod. but the problem existed before it. The more grounds the safer.

The F in GFCI stands for 'Fault'. GFCIs are designed to keep you safe irregardless of whether the equipment plugged into them has a ground prong on the plug or not. GFCIs are required by code in bathrooms and most electric shavers, hair dryers etc., only have two prong plugs. As mentioned, they monitor the current coming in and going out of the circuit they are installed in. As long as the device sees the same current on both the hot and neutral, all is well. If there is a differential of a few mA, not sure what the number is off hand, the device is designed to trip before serious harm comes to the individual using the appliance.

Just my speculation here and I could be totally wrong, but it's the only explanation that makes sense to me. I suspect that in the case of your powerhead there was some degree of deterioration of the insulation waiting to find a path to a lower potential (ground if you wish). Although you felt a shock, your body resistance was likely too high to cause enough current to flow to trip the device, i.e. it was not enough to harm you. By introducing the grounding probe (much less resistance to ground than your body), sufficient current was able to flow from the faulty powerhead to ground tripping the GFCI. It's great that you were able to isolate the faulty device by installing the ground probe before it became bad enough to trip the GFCI with your hand in the water.

The ground probe cannot be the source, it's only connected to the ground and if the receptacle didn't have a proper ground then removing or adding the probe would not make a difference. I see no way how a receptacle could be faulty in such a way that a ground probe would add voltage. The stray voltage is from the pump since unplugging the pump eliminates the voltage (also in the OP). Removing the ground probe simply prevents the flow of current, voltage can be created through induction in many ways from pumps and lights but it does not mean the equipment is faulty. Adding a ground probe creates a path for current to flow, typically the ground probe is a strong ground and a person won't feel any current because the ground probe is a much stronger ground but electricity takes the shortest easiest path so a poor ground is more likely the issue relating to the stray voltage but a better ground isn't the solution IMO.

Remove the ground probe, use a multimeter to measure the voltage between the tank water and removed ground probe. If you measure a voltage then the ground probe is functioning properly. The tank is obviously not grounded so voltage cannot flow from the ground probe to the tank, it must be the other way.

A GFCI is the best way to go, a ground probe won't really protect you, this example clearly tells you that. If the current was large enough to cause damage it still would go through you over the ground probe just as the small amount did.

I have found this artilce to be helpful.
http://angel-strike.com/aquarium/GroundingProbes.html

Don't pay too much attention to these types of internet articles/blogs or whatever you want to call them. You can find hundreds supporting both sides of any topic.