You could make a relay buzzer out of some cheap parts and a wall adapter. Just put in a 9v batt and a car buzzer on a 12vdc relay on the normally closed position. When the power goes out, the relay will close and the buzzer will go off. maybe $10 in parts. provided you have a 12vdc adapter lying around your house.
Or you could always splurge for the NetGuardian System that will page, phone and email you when there is a problem too.
www.dpstele.com/products/ne/netguardian
Runs around $3000US ;