Hi Steve,
I would go to both. Butchart gardens are stunning. Amazing what money can do to a gravel pit. If you can make it for a Saturday night, they have a great fireworks show. The butterfly gardens are pretty cool as well, your kids will probably like that the best. ***Bonus*** Both of them are about 5 minutes apart too! Picking flowers.... what is a few out of millions! Take a picnic lunch, lots of great places to sit in the sun! Enjoy your stay in Paradise!!
Cheers,
Phil