Why do so many people have the impression that Oxford University is the world's best university?

Can you really judge a uni just on the fame of its alumni? I think a lot of people think of Cambridge as been Oxford's equal or even better. But the majority of us really don't care!