I think many things all of you are saying are correct, and that's what makes interpreting students' grades so confusing... However, a few disagreements,
I was saying that a 4.0 at a big state school was probably equivalent to like a 3.5 or so from a really tough physics program.
I think the best way to say it is that a 4.0 at a big state school is "ambiguous." There's no telling what the student would get at a more challenging school. While I doubt I'd have a 4.0 at a top undergrad physics program, I think I'd be closer to 4.0 than 3.5. On many occasions at my school, I'd cream all the exams and homework assignments just to see other students with much lower scores get the same A I did. The problem is that grades at a school usually just compare you to your peers, and since many of the students are just so disgracefully bad, getting an A basically only proves that you can add. A high GPA at an Ivy may be more convincing, but probably not to the extent that we'd think... Many of the students there are there because they're brilliant, but they're probably the minority. The rest are there because, as we've seen, they have rich, influential parents or they played corny high school games such as becoming president of 10 clubs, playing 5 varsity sports while pulling off the 3.98 or running community service marathons that brought in $175,000 to relocate the geese displaced when the new football field was built. Top schools are filled with these kinds of people, and most of them would be very unlikely to have strong physics GPAs anywhere.
How would schools like this necessarily consider students like this to be ideal even if their gpa was a perfect 4.0? After all, a perfect 4.0 gpa at schools like these could easily be the equivalent of a 3.5-3.7 range at a school known for tougher physics classes. (I actually go to Ohio State, which is known for at least in some classes like quantum mechanics being very tough, hence the comaprision).
I don't think anyone considers these 4.0 students at random places ideal based on grades alone, they get rejected all the time from grad schools and REU programs. But these students deserve just as much consideration as anyone else. If the 4.0 is all they have to show, then they're probably not so great but again, being at a big state school puts a ceiling on what a student can accomplish. It's like the quantitative section of the general GRE... it's simply useless because any decent student can get the maximum score. It's not to say a student who gets that score couldn't be a genius. Ohio state, to most people, is just as random a place as the other schools you mentioned. If Ohio State has a more stringent grading policy than its peers, then that's something you should have your recommenders include in their letters.
Some of you have said it all boils down to your letters of recommendation and who you know/who your recommenders know...
I can't say that's incorrect since I don't know, but in these instances we should no discredit the physics GRE so much. For everyone who says the physics GRE is an inaccurate measure of our knowledge and skill, even though it tests all of undergraduate physics, combines knowledge questions with tricky problems, and is given to every prospective physics grad student under nearly the same conditions, I'd like to ask how these "other factors," could be more reliable...
Is a student getting some wacky famous old prof to say that he's great, or being lucky enough to get his name on the prof's research, or having high grades from one of 500 programs that has wildly different grading systems really say more than the physics GRE?