More detail
Faster gameplay
More interesting puzzles
Better weapons/armor/graphics/music
Interesting that you're using things that cannot be fairly measured due to the age difference between the games. Things like "more detail" are very arguably subjective. If you mean more detailed environments, then absolutely the Bethesda Fallout games have more detailed environments. They also are decades more recent than FO2 and are using a dedicated 3D engine that lends itself to fiddly details.
But if you mean overall worldbuilding and locations feeling like actual locations, I have to disagree. Many of the settlements and location in FO2 have a level of detail with logically placed outhouses, farmland, stores, and living quarters that FO4 lacks, plus FO3/4 suffers from Bethesda's tendency to compress settlements down to be minimalist looking, where in FO2 the locations felt much bigger and more expansive even though you were often limited to a fairly small play area.
"Faster gameplay" is a purely subjective matter, and considering that 2023's Game of the Year was a Turn based RPG much more in line with FO2 than FO4 it is clear even today people can and do enjoy a slower more tactical pace. The fact that you didn't find other ways to approach FO2's combat isn't because the game lacks those option, because there's numerous ways and builds in FO2 that are viable... heck, FO3/4 even explicitly made sure that certain strategies from FO2 could be replicated because they were that well known and popular (IE. the Sneak + Pickpocket + Planted Explosives combo was only included in FO3 because it was a humorous and popular strategy in FO2). Unarmed Combat, Sneak and snipe, charging in power armor with heavy weapons... all those strategies could be pursued in FO2.
As to puzzles... I'm honestly not sure FO4 even HAD puzzles? None that I really remember. FO2 wasn't really that puzzle heavy either.
As to better graphics, music, etc. Again, not a fair comparison and you well know it, decade and more of difference, completely different hardware expectations. Music is entirely a matter of technological progress. The entire Fallout 2 install, including music, is ~625 MB. FO4 is ~37 GB... so yeah, no shit is it going to have better music, even if they had the exact same song in both games FO4's would be less compressed and thus have higher audio fidelity because... that's the way audio compression works. A modern high quality, near lossless audio recording can easily come in at over 50 MB for a 3 minute song... not going to be including many of those on an old CD-ROM.
But let's get into some of the areas that you carefully avoided, firstly, the RP options. Yes, FO4 is fully voiced... and that makes it so there's much FEWER actual options to RP your character as you envision them, and in fact FO4 is notorious for really only being able to be "nice guy, smart-aleck, or meanish", and because everything is fully voiced. In comparison most FO2 dialogue trees had considerably more depth to them and more branches, since it was mostly a written game with only partial voicing.
Additionally FO4's "SPECIAL" system is a ghost of what it was in FO2, having dropped most everything that made the SPECIAL system, well, special and cool in favor of a watered down Skyrim esq. system that barely paid lip service to the PnP feel of the original Fallout games. FO3 and NV both managed to keep a better PnP feel with their version of the SPECIAL system than FO4 and FO4 clearly was dumbing down the system in order to appeal to FPS players who would be intimidated by the more complex PnP system that underlaid the older Fallout games.
As to better weapons, as someone who loved Small Guns in FO2, let me tell you FO3 and 4 did small guns dirty by dropping the best small guns and changing them into Energy weapons for... reasons? The Gauss Rifle and Gauss Pistol in FO2 used to be the best Small Guns for that skill and allowed small guns to remain competitive into endgame, in FO3 and 4 for some reason those weapons were moved to Energy Weapons and it actually left a massive hole in Small Guns that made it so they no longer had a high tech capstone weapon where Energy had both the Gauss Rifle AND Plasma rifles (which, going back to FO1 the Plasma rifles were the traditional top end capstone of the Energy Weapons).
This is before we get into the loot-shooter nonsense of randomized Legendary weapon drops that FO4 introduced...