GPA, PGRE, and Top 20 Admissions
Posted: Fri Jan 04, 2013 4:08 am
I have been curious for a while about just how "shallow" the physics admissions process really is. More specifically, I wanted to learn how well you can predict whether someone will get into a well-ranked program based strictly on their GPA and physics GRE score. So, using the phenomenal database this website has accumulated, I set about to find the (statistical) answer using all the applicant profiles from 2011 and 2012 (I was originally just going to do 2012, but I got to the end and was not convinced the sample size was large enough to be meaningful).
First some boring notes about methodology and disclaimers:
1. Due to a combination of selfishness and laziness, this analysis only considers domestic applicants.
2. I only considered applicants with a clear physics/astronomy focus. If an applicant was focused primarily on earth science/engineering/etc., I did not count them toward anything.
3. I have not controlled for gender/minority status.
4. There is potentially a large systematic error with my approach: an applicant with a fantastic GPA and PGRE score is very likely to have good research experience, great recommendation letters, personal statement writing skills, etc. Controlling for these factors would be difficult, and is not really in the spirit of the study. The point is to assess the predictive power of GPA together with PGRE score. How exactly you interpret that is up to you.
5. I define a "successful application experience," or SAE, as getting into any school ranked in the top 20 physics programs according to the US News list. Anyone getting into a top 20 program is said to have "had an SAE."
6. I define a "strong applicant," or SA, as anyone with an overall GPA at or above a 3.7 and a PGRE score at or above an 800 (if anyone repeated this for international applicants the PGRE number would obviously have to be higher). I observed surprisingly little variation between "major GPA" and "overall GPA" for most applicants, but this is something you could control for pretty easily if so inclined.
And, after reducing the 2011 and 2012 data, here are the results:
Total # of SAE: 80
# of SAE among those with PGRE at or above 800 and GPA at or above 3.7: 43
# of SAE among those with PGRE below 800 and GPA at or above 3.7: 24
# of SAE among those with PGRE above 800 and GPA below 3.7: 9
# of SAE among those with PGRE below 800 and GPA below 3.7: 4
Note that over half of SAEs went to SA-level profiles.
I also analyzed the results of all SA profiles:
Total # of SA Profiles: 55
# of SA Profiles with an SAE: 45
Discussion:
It is this last statistic I find most jarring. Almost every SA-level applicant got into a top 20 program. An anecdotal survey of the ten unlucky SA-level profiles tended to reveal other significant flaws (no research at all, been out of school for 15+ years, etc.). Most (virtually all) admissions councilors I have spoken with like to say that they give recommendation letters greater weight than GPA and PGRE score, but I find that somewhat hard to believe when confronted with a correlation coefficient above 0.8. You would have to believe that recommendation letter quality tracks extremely closely with GPA/PGRE to accept that argument. I tend to buy into the quasi-common wisdom that personal statements matter comparatively little, but I know of no way to test this.
It would be interesting if someone wanted to tweak the SAE/SA parameters and rerun the analysis. I conjecture that the results would change fairly little, since 3.7 is a pretty good GPA by almost any measure and an 800 is generally great for a domestic (I suspect anything above an 850 is overkill unless you are only interested in Stanford or Berkeley).
I do not know how interested the rest of the PhysicsGRE community is in my games, but here you go.
First some boring notes about methodology and disclaimers:
1. Due to a combination of selfishness and laziness, this analysis only considers domestic applicants.
2. I only considered applicants with a clear physics/astronomy focus. If an applicant was focused primarily on earth science/engineering/etc., I did not count them toward anything.
3. I have not controlled for gender/minority status.
4. There is potentially a large systematic error with my approach: an applicant with a fantastic GPA and PGRE score is very likely to have good research experience, great recommendation letters, personal statement writing skills, etc. Controlling for these factors would be difficult, and is not really in the spirit of the study. The point is to assess the predictive power of GPA together with PGRE score. How exactly you interpret that is up to you.
5. I define a "successful application experience," or SAE, as getting into any school ranked in the top 20 physics programs according to the US News list. Anyone getting into a top 20 program is said to have "had an SAE."
6. I define a "strong applicant," or SA, as anyone with an overall GPA at or above a 3.7 and a PGRE score at or above an 800 (if anyone repeated this for international applicants the PGRE number would obviously have to be higher). I observed surprisingly little variation between "major GPA" and "overall GPA" for most applicants, but this is something you could control for pretty easily if so inclined.
And, after reducing the 2011 and 2012 data, here are the results:
Total # of SAE: 80
# of SAE among those with PGRE at or above 800 and GPA at or above 3.7: 43
# of SAE among those with PGRE below 800 and GPA at or above 3.7: 24
# of SAE among those with PGRE above 800 and GPA below 3.7: 9
# of SAE among those with PGRE below 800 and GPA below 3.7: 4
Note that over half of SAEs went to SA-level profiles.
I also analyzed the results of all SA profiles:
Total # of SA Profiles: 55
# of SA Profiles with an SAE: 45
Discussion:
It is this last statistic I find most jarring. Almost every SA-level applicant got into a top 20 program. An anecdotal survey of the ten unlucky SA-level profiles tended to reveal other significant flaws (no research at all, been out of school for 15+ years, etc.). Most (virtually all) admissions councilors I have spoken with like to say that they give recommendation letters greater weight than GPA and PGRE score, but I find that somewhat hard to believe when confronted with a correlation coefficient above 0.8. You would have to believe that recommendation letter quality tracks extremely closely with GPA/PGRE to accept that argument. I tend to buy into the quasi-common wisdom that personal statements matter comparatively little, but I know of no way to test this.
It would be interesting if someone wanted to tweak the SAE/SA parameters and rerun the analysis. I conjecture that the results would change fairly little, since 3.7 is a pretty good GPA by almost any measure and an 800 is generally great for a domestic (I suspect anything above an 850 is overkill unless you are only interested in Stanford or Berkeley).
I do not know how interested the rest of the PhysicsGRE community is in my games, but here you go.