A few observations:
1. In that time the central European classical music canon was far more dominant than it is today (later this month the NSO is doing Beethoven with William Grant Still, for instance). That made the dominance of a few figures such as von Karajan and Klemperer, who specialized in that repertoire, far more possible. In America, Germanic culture was more influential as well.
2. There was overall less conducting talent around at the time. Yes, I know the beloved status of your few favorites from back then, and their unique styles, but conductor #30 today, in terms of quality, is far better than before. Today it is harder for anyone to stand out.
3. The authoritarian and possibly abusive management style of von Karajan was far more acceptable back then. Without that style, he could not have honed such a unique sound.
4. Back then conductors actually could sell classical LPs and bring in revenue. This helped enable many of von Karajan's projects, including costly operas and symphonic cycles. Whether he would have done as well on YouTube, or other more contemporary media, is very much an open question, but probably not. He was very much a "whole package" sort of musical star.
4b. Radio really mattered too. His distant and forbidding but legendary personal style worked well in that medium, and the "always forward impetus whiplash" sonics cut through the poor sound quality.
5. I grew up with von Karajan's recordings in so many parts of the repertoire, but how many really have held up? His Bruckner's 8th and Mahler's 9th are incredible. His Cosi is amazing, though too rigidly controlled for my taste. His Verdi Aida. A big thumbs up to his Mozart #40 and #41. But the Wagner I don't listen to any more. Never loved his Beethoven cycle. Rarely is he the conductor in my favorite concerti performances, as he tended to blunt the styles of his accompanying soloists. Would I ever prefer him for Haydn, or for French music? No. Definitely some Strauss (the conductor most suited to him?), or perhaps his Tristan? I feel I could get 85% of his value with maybe five recordings? In a way that is quite impressive, but it does put matters in perspective.
7. In short, that was then, this is now.