Welcome to my world. I should show you a picture of the mold spot I've got on my ceiling in the front entry from the excessive humidity and the cold weather of the last few months. Yay hobby! The effect is probably worse with new houses such as mine because the R factors are much better and they're way more airtight.
Three suggestions for ya -
1- dehumidifier. I use one, and it helps a little. But it's noisy and having it run all the time when it's cold outside (when it's needed most) will put a little dent in your utility bill.
2- if you have a high efficiency furnace (or at least a low-cost DC drive fan), run the fan all the time. You can consider this without the high efficiency furnaces, but let me just say I tried this myself and was shocked at my utility bill the next month. With all the halides and stuff I run, that should tell ya something.
3- Consider an HRV (heat-recovery ventilator) installation. I'm having one put in myself. The people I've talked to who have HRV's swear by them. The idea is basically this, it's a heat exchanger. You ventilate your house (or a localized area) by venting out the air and using this air to preheat the air coming in. They have about 80% efficiency, which is pretty neat (i.e., 80% of the heat is recovered).
You can also look into a device called a humidex, which is sort of half of an HRV, from what I understand. It vents air to the outside, but relies on the non-airtightness of a house (or an open window) to replace the air coming in.
Incidentally if it's just a mildew buildup on your walls, I think you should be able to clean that up with bleach (rather than replace the drywall). It's just on the surface, right? If you have mildew on the INSIDE of your walls, that's another matter entirely (and quite serious), but it seems to me that shouldn't happen unless you've had an issue with flooding and so on (where the walls get actually wet through-and-through).
HTH..