I'll reply, Swan my friend.
I'm only a student in the ph.d. program, but I have big ears, so I listen to the admissions group members (professors) around the coffee machine.
They don't talk much about the olympiads or gre scores and such. They seem to be generally interested in grades. But one person on the group can bring in a student. So if you won an olympiad or belong to some group that happens to impress just one person in the group, you're in -- for some schools.
Of course there are other schools (not mine, but friend's schools) that are so competitive that getting in means getting past all of the group members. For those schools, I think the top two most important factors are the grade in your physics courses (they don't pay nearly as much attention to averages as they do to the individual grades in, for instance, mechanics or E&M), and the other is your Physics GRE score. Some schools don't require a physics GRE, but if you have a good score on one, that will launch you through admissions groups, because the difficulty of that test is universally understood.
Many professors have no concern for the regular GRE, they consider it too easy.
I would say you have a very strong choice to get into at least one of your top five choices. Apply to a few different schools, and don't stress about the name of the school. It's better to do awesome work at a lesser-known school, than to be an also-ran at a famous school. Why? Because you'll get good jobs either way, but the former means you're probably a better physicist, and that's the whole point, right?
Finally, there are some programs that are nearly impossible to get into. they may have already chosen the top two physics students from this country, and the two three from that country, and the top U.S. students, and as many female candidates as they can get. And that leaves one or two open spots, which are tough or fairly impossible to get, no matter how good you are.
I go to the University of Alabama. There is no group in the area in which I work, (atmospheric physics) but there is one theorist (my advisor) that is totally awesome, always has time for me, knows everything, and is a good teacher. Finding this kind of position is good, I wouldn't switch to a school with a huge atmospheric program if it meant not having a great advisor.
So the moral of the story is, don't take the accepatance and rejections personally, it's just physics.