All 3 types of fish you mentioned would possibly pick at, or generally bother the cowfish and or boxfish, as well as out compete them for food.