Seems like there would be more wizarding children than enough to fill three schools, wouldn't you say? Honestly, if their numbers are really that small, then Voldemort's whole shindig could be slapped down if the Muggles all banded together.