Well, even the Galactic Empire had a limit on manpower and resources. If forces are directed to a particular planet that means resources must be diverted from somewhere else. It's not just military forces- everything from technicians, mechanics, etc. must be present because enslaved beings may...