The Prime Directive is an interesting example of how Liberal/Progressives take good ideas to far and then act morally superior where normal people go "what the actual fuck."
The Prime Directive, as presented in TOS, made a lot of sense as a lesson learned both from the Columbian Exchange, Clark's Third Law, and the even greater potential technological difference between Starfleet and civilizations they could encounter. Further TOS did a good job of showing why the Prime Directive HAD a place for instance, the episode "A Piece of the Action" showcased a planet that had suffered cultural contamination from Earth and suffered for it. Another example of why the Prime Directive was a good thing was "A Private Little War" where because the Klingons had no such directive they had intervened on a pre-war planet, uplifted a faction to favor them and so Starfleet got involved and uplifted a different faction to favor them... thus causing a planetary war where those natives were proxies for outsiders.
On the flip side, TOS also explored very well when the Prime Directive could be bent or broken. IE "For the World is Hollow and I have Touched the Sky" showcased a situation where ostensibly the Prime Directive applied (pre-warp civilization) but for the good of everyone involved they decided to intervene and fix things up so that the civilization had a chance to survive. You also had episodes like "The Devil in the Dark" where technically the Prime Directive was already broken and instead of just pulling out with the damage already done, Starfleet elected to take responsibility for the fuck up and basically uplifted/made a protectorate of the Horta (the fact we never saw Horta outside of the TOS novels and comics is a crime, but the TOS novels do give us Lt Naraht who was a reoccurring side character in numerous TOS novels by multiple authors and in comics).
But the underlying idea of the Prime Directive, non-interference with races that were not ready for it, took a turn for the religious some time in the 24th century. My personal opinion is that it was part of the Hubris Era of Starfleet. Starfleet had grown so assured of its power and rightness, had no peers that could challenge it either militarily OR ideologically (think about it, the Klingons were beaten and brought to heel not by military might, but by diplomacy... meanwhile the Romulans were keeping to themselves and had withdrawn), functionally a kind of ideological strictness had set in, the Prime Directive was just THAT a Directive and Starfleet and the Federation were the most successful civilization in the region and thus obviously their directives and ideals were objectively the best and not to be questioned. This attitude led to the strictness and unbendingness of the Prime Directive that we then saw in the TNG era. DS9, interestingly, was more willing to bend the Prime Directive as the Federation was faced with a peer opponent who challenged them militarily and philosophically... and arguably the Federation while it won the war, lost the philosophical argument underlying the war. It wasn't the Federation's ideals that won the Dominion War, but the two actions (and their consequences) that went against the ideals: Section 31 using a bioweapon against the Great Link and Sisko arranging the assassination of a Romulan Senator.
As to why Voyager clung to the Prime Directive so heavily despite their desperate situation? That actually makes more sense when you think about it: it was to keep their identity and cohesion together, and to maintain moral. It might sound counter-intuitive at first, but one of the reasons Janeway was successful in keeping Voyager together to get home was that she made sure her people kept their identity as Starfleet officers and members of the Federation. She further enforced that by make sure anyone they picked up along the way had to conform to those standards. Note that while they were not officers: Neelix and Kex were given explicit roles and duties on the ship, and were required to participate in normal ship training functions. Seven of Nine was the same way, as were any others they picked up. The only person on board who did not have a strict duty and job role was Naomi Wildman, a child, but notice that once she was old enough because of the circumstances of her upbringing she was seeking to have such a defined role on the ship because that was the expectation and culture (she wanted to be "Captain's Assistant"). Thus maintaining Starfleet order, including strict adherence to Starfleet's general directives, was crucial in maintaining that unified ship identity... even when you could argue it would have been better for them to compromise them.
As to Phlox and that episode... well... that was the writers seeking to retroactively justify the more religious Prime Directive that had crept into the show... and showcasing just how badly disconnected they'd become from normal morality and how twisted the Prime Directive had become.
The Prime Directive, as presented in TOS, made a lot of sense as a lesson learned both from the Columbian Exchange, Clark's Third Law, and the even greater potential technological difference between Starfleet and civilizations they could encounter. Further TOS did a good job of showing why the Prime Directive HAD a place for instance, the episode "A Piece of the Action" showcased a planet that had suffered cultural contamination from Earth and suffered for it. Another example of why the Prime Directive was a good thing was "A Private Little War" where because the Klingons had no such directive they had intervened on a pre-war planet, uplifted a faction to favor them and so Starfleet got involved and uplifted a different faction to favor them... thus causing a planetary war where those natives were proxies for outsiders.
On the flip side, TOS also explored very well when the Prime Directive could be bent or broken. IE "For the World is Hollow and I have Touched the Sky" showcased a situation where ostensibly the Prime Directive applied (pre-warp civilization) but for the good of everyone involved they decided to intervene and fix things up so that the civilization had a chance to survive. You also had episodes like "The Devil in the Dark" where technically the Prime Directive was already broken and instead of just pulling out with the damage already done, Starfleet elected to take responsibility for the fuck up and basically uplifted/made a protectorate of the Horta (the fact we never saw Horta outside of the TOS novels and comics is a crime, but the TOS novels do give us Lt Naraht who was a reoccurring side character in numerous TOS novels by multiple authors and in comics).
But the underlying idea of the Prime Directive, non-interference with races that were not ready for it, took a turn for the religious some time in the 24th century. My personal opinion is that it was part of the Hubris Era of Starfleet. Starfleet had grown so assured of its power and rightness, had no peers that could challenge it either militarily OR ideologically (think about it, the Klingons were beaten and brought to heel not by military might, but by diplomacy... meanwhile the Romulans were keeping to themselves and had withdrawn), functionally a kind of ideological strictness had set in, the Prime Directive was just THAT a Directive and Starfleet and the Federation were the most successful civilization in the region and thus obviously their directives and ideals were objectively the best and not to be questioned. This attitude led to the strictness and unbendingness of the Prime Directive that we then saw in the TNG era. DS9, interestingly, was more willing to bend the Prime Directive as the Federation was faced with a peer opponent who challenged them militarily and philosophically... and arguably the Federation while it won the war, lost the philosophical argument underlying the war. It wasn't the Federation's ideals that won the Dominion War, but the two actions (and their consequences) that went against the ideals: Section 31 using a bioweapon against the Great Link and Sisko arranging the assassination of a Romulan Senator.
As to why Voyager clung to the Prime Directive so heavily despite their desperate situation? That actually makes more sense when you think about it: it was to keep their identity and cohesion together, and to maintain moral. It might sound counter-intuitive at first, but one of the reasons Janeway was successful in keeping Voyager together to get home was that she made sure her people kept their identity as Starfleet officers and members of the Federation. She further enforced that by make sure anyone they picked up along the way had to conform to those standards. Note that while they were not officers: Neelix and Kex were given explicit roles and duties on the ship, and were required to participate in normal ship training functions. Seven of Nine was the same way, as were any others they picked up. The only person on board who did not have a strict duty and job role was Naomi Wildman, a child, but notice that once she was old enough because of the circumstances of her upbringing she was seeking to have such a defined role on the ship because that was the expectation and culture (she wanted to be "Captain's Assistant"). Thus maintaining Starfleet order, including strict adherence to Starfleet's general directives, was crucial in maintaining that unified ship identity... even when you could argue it would have been better for them to compromise them.
As to Phlox and that episode... well... that was the writers seeking to retroactively justify the more religious Prime Directive that had crept into the show... and showcasing just how badly disconnected they'd become from normal morality and how twisted the Prime Directive had become.