You can do one of three things:
1. get some thin but stiff wire and shove it down the last of the hole that you couldnt' destroy. Its a worm so making him mush will kill him
2. clip the coral off at the part where the worm lives, then clip the worm part off. now you have a frag you can grow and sell (if you're interested in that sort of thing) or another frag you can place somewhere in the tank.
3. (variation of #1) try either poking him with fine tweezers or destroy as much of his worm tube as you can and then get at him with the wire
just destroying his tube doesn't work, they'll just grow it back. you have to actually get to the worm inside and either pick him out or make a gory mess of him. I hate vermetids and had them spawn all over the place when I left one to hang out in my capricornis cuz I couldnt' get to him. At one point I had 6 on one of my clams.