quizivex wrote:It's important to note that the "difficulty of the test" and "percentiles" are completely unrelated! You can't assume that because the "percentile vs. scaled score" plots for recent years line up with 2008, that the raw scores needed for each scaled score in upcoming tests will be similar to the 0877 practice test.
The "difficulty of the test" is measured by what raw score you must get to earn a given scaled score. The scaled scores are intended to measure a given level of performance independent of the difficulty of the particular version of the test, and independent of the performance of the other test takers.
On the other hand, the "percentile vs. scaled score" is determined entirely by the curve set by the pool of recent students taking the test. In the 0877 practice test booklet, they state "The table on page 91 contains, for each scaled score, the percentage of examinees tested between July 2007 and June 2010 who received lower scores.." So the percentiles are averages over a 3 year period, and will not substantially change from one year to the next.
HappyQuark's second graph shows that scaled scores have been increasing over the years. This is because the students have gotten better at the test over the years. The reason the "percentile vs. scaled scores" from 2011 and 2012 line up closest with the 2008 test is that 2008 is by far the most recent of all the years the practice tests were released.
The "difficulty of the test" is a function of one individual test only. There are several versions of the test given on every test date, and students in the same exam room have disagreed over how difficult the test was for that reason. The 0877 test is a sample size of one.
Incidentally, to measure difficulty of tests over the years, the scaled scores should be compared with raw scores. A plot of "raw score vs. percentile" is virtually meaningless because percentiles change over time.
Overall, we cannot use the 2008 test to estimate "which raw score we should be gunning for in the upcoming 2013 exams". It will be dangerous if PGRE.com users go into the test expecting it should be as "easy" as the 08 test and find that it isn't. One of the issues that makes the subject GRE experience difficult is that students won't know in advance how difficult their version of the test is. So if they're having an easy/hard time, they won't know if it's because they're doing well/bad, or if the test itself is easy/hard.... it's possible to mess yourself if you count on needing to get a certain number of questions right to reach a goal.
quizivex wrote:There are several versions of the test given on every test date, and students in the same exam room have disagreed over how difficult the test was for that reason. The 0877 test is a sample size of one.
No. The percentiles corresponding to each scaled score designate what percentage of students got below the given scaled score in the 3 previous years, regardless of what version of the test they all took. The raw score to scaled score table is a completely different stat which only applies to the particular test booklet... Such tables on the practice tests are valid for gauging your performance on that one test, even the very old practice tests.HappyQuark wrote:If I'm understanding your post, you are saying... that there are multiple versions of the test... but that each of these tests have significantly different levels of difficulty which, on average, is the cited value they give.
The scaled scores are intended to show the level of performance independent of what version of the test was taken. A student who gets a 720 with a raw score of 65 on a hard test would presumably get a higher raw score on an easier test, and end up with the same 720 scaled score in theory.HappyQuark wrote:The thing that doesn't make sense to me about your interpretation, If correct, is this makes the average values they cite completely useless. In fact the table they give at the end of each practice exam is completely meaningless. When I took the practice exams, it was my understanding that when I calculated my raw score on a practice test, I could look at the score conversions table in the back to see what my raw score and percentile rank would have been and, so far as I can tell, this is how everyone else has interpreted it.
quizivex wrote: In fact, some forum users have pointed out (myself included) that their percentiles for their tests have changed from the time they took the test to the time they applied to schools. That's because the 3-year pool of recent test takers changes each year.
Users browsing this forum: pistachion and 1 guest