One theory / explanation I have found is that he is just asocial as fuck. Basically, he can organize everything and manipulate people, but he cannot ever gain true loyalty because he treats people like cogs in a machine. Guilliman spells it out after his rebirth IIRC., and even before that it is clear that Guilliman - probably one of the most loyal Primarchs - is loyal to humanity, and Emperor's idea of uniting humanity for its protection, not necessarily to the Emperor personally.
Malcador would indicate otherwise. As would the example of Horus, whose charisma was the product of basically playing a numbers game with conversations, body language, etc.
The Emperor might not have cared over much about gaining "true loyalty" but the very idea of true loyalty undercuts the entire point I am making.
The Imperium of the Emperor during the Great Crusade was not a state, or even a polity at the most base levels. It was the personal domain of a warlord just conquered and ruled by whim and decree without any attempt at integration or building any kind of stable entity.
The only domestic policies that the Emperor seemed to care about were 1) wiping out religions/spreading the Imperial Truth, 2) enforcing his rules regarding psykers and the Black Ships, 3) not rising in active rebellion/repudiation of his rule, and 4) forking over what military resources were demanded promptly.
Hell, the Imperium was on a barter economy and didn't even have any kind of tax policy until after Horus was made Warmaster!
The Emperor has been alive and active since literally before the Spartans fought the Persians. He lived through every imaginable form of government and society across a span of forty to fifty thousand years. That entire time he was superhumanly intelligent and had at least some interest in human politics. By sheer osmosis if nothing else, he would have learned how to make a stable empire if he wanted one.
This is a man who conquered and unified the entire Sol system in a period of a century or two and then built the force that would conquer the entire galaxy. The idea that he couldn't build the army of administrators and bureaucrats needed to secure that conquest at the same time is farcical. As is the claim that he thought he could build a stable empire without them.
But look at what the Emperor did as opposed to what he said or what others believed that he wanted. The consequence of his conquest of the galaxy was to 1) purge every noteworthy xenos power save the Orks and (to a lesser extent) the Eldar, 2) purge every pre-existing religion from humanity across the breadth of the galaxy, and 3) establish a common mythos for humanity.
Then you have the heresy which is just chalk full of religious connotations and symbolism; which ends with the Chaos Gods taking a major blow (they lost a good chunk of invested power when Horus fell) and established a savior myth so hard that Moses would squeal.
Very rapidly the Imperial Creed becomes a thing - a creed based on a text penned by a Primarch convinced every word of it was true and raised in a theocracy (that he would then lead). It would find fertile ground across the breadth of the galaxy as every competing faith had been purged and the supporting framework to support an aesthetic belief structure was both absent and impossible to create (at least so long as the tech-priests remained a thing).
The Custodes, despite the supposed hatred of the Emperor for religion and the absolute nature of the Imperial Truth did nothing as the Creed spread and even gained a seat on the High Twelve. Indeed, they would bring the first of the Sisters of Battle into the very presence of the Emperor to convince them that Vandire was a false prophet and they left even more convinced of the Emperor's divinity. The Living Saints are a thing as well.
All of this in a galaxy where belief shapes reality, faith creates gods, souls are real, souls have power, and souls can be dedicated to other entities via ritual. The entire Eldar pantheon was a deliberate creation, the Emperor was there to see the creation of Slaanesh as well.
Quadrillions of souls who pray to the God Emperor, perform their religious rituals to him, and dedicate their souls to him. Fanaticism is practically bred into the citizenry of the Imperium. "For the Emperor!" "The Emperor Protects!" - these aren't just catch phrases, the are catechisms.
Ten thousand years of effectively the entire human population of the galaxy dedicating their lives to the Emperor and worshipping him as a god before willing their souls to his protection. Ten thousand years of trillions of them dying in his name on the field of battle with a fanatics hatred of the xenos, the mutant, and the heretic and all dedicated to the Emperor. Ten thousand years of a thousand psykers per day being ritually sacrificed to the Emperor with their souls consumed. Ten thousand years of every* human psyker being soul bound to the Emperor.
The psychic power of the Orks goes to Gork and Mork and so is basically irrelevant in the game of Immaterial politics that the Emperor, Chaos, and the Eldar are playing. The Eldar trap their souls in Infinity Circuits and don't have the numbers anyways. The Necrons are soulless. Humanity is the ONLY source of souls/belief/warp power that exists at scale to contest. And for every heretic (which needs to be split four plus ways) there are a million ardent supporters of the God Emperor.
And note what two of the core tenets of the Imperial Creed are. The Emperor is anathema to the heretic and their masters, protecting all loyal humans from the predations of the xenos, the mutant, and the heretic. And that the xenos, mutant, and heretic are to be purged.
A religion purpose made to create an anathema and a belief structure virtually impossible for any non human power to suborn.
Losing? Every day that the Imperium of Man manages to hobble on is another day of all that belief and all those souls being fed to a pro-human warp entity that is anathema to chaos and will accept nothing less than total human domination of the galaxy.
The more reasonable scenario is that everything is basically going just as the Emperor planned and once he decides that he either has no choice or that his chances of victory are as good as its going to get, he unleashes all of that gathered power into the warp and basically scours it clean of anything but his power all at once, heals himself, and then restructures humanity into what he wants it to be.
Let the Eldar be steadily whittled down, let the eventual three way war between the Orks, the Tyrranids, and the Necrons occur and whittle all three down. Let Chaos remain both his foil and the force to temp away the less faithful. And then, purge.
Isn't it interesting how the second largest fleet in the galaxy (second only to the combined fleets of the Mechanicus) is the Black Fleet and yet it never does anything?
Let's not talk about Cawl and the Primaris. That was one tech-priest in secret using things he ill understood. What do you think that the Emperor could have squirreled away somewhere under the oversight of the Custodes or some of his Perpetual buddies?
Look at the Sisters of Silence. Blanks one and all and thus impossible to perceive in the warp. Spread the breadth of the Imperium via the Black Ships. Loyal only to the Emperor and his personal agents. Look at what happens when you have enough Pariahs in one place? It makes an active shadow in the Warp and that is why the Culuxes Temple was stuck out in the back of beyond.
Go to somewhere out of the way, perhaps with a rogue planetoid in interstellar space, put a ton of Pariahs on site to conceal the whole thing from Chaos, the Eldar, and anyone else with psyker powers, have any travel down with Black Ships crewed only by Blanks, and you could build an entire warmachine to put in stasis without anyone the wiser.
Use the Sisters as secure couriers and living data archives. All they need is cybernetics that allow them to establish a neural connection between one another on contact and you have an utterly secure communications conduit.
You think that the Custodes can't fake the seal of a Lord Inquisitor? Assuming that one of the perpetuals who was actually in the Emperor's inner circle and wasn't given such a seal basically from day one?
Or that the Grey Knights would hesitate even a moment if a Custodes gave them orders to do something and conceal the doing under the guise of fighting chaos?
---
I mean really, the Emperor being open and honest with his plans and intentions? Who would think that would be in character for him?
Now the entirety of the Great Crusade, Heresy, and Imperium being one grand gambit designed to serve the Emperor's true goals with everyone else playing the game set out as a distraction? Well that seems entirely in character for the Emperor.