The Surprising and Disappointing Predictors of Success in UCR's Philosophy PhD Program
Well, surprising and disappointing to me at least! You might be neither surprised or disappointed.
Here's the background: I'm on the UCR Philosophy PhD admissions committee again this year. I'm a fan of treating senior-year GPA in philosophy as a crucially important part of an application. Here's the kind of thing I tend to say to other members of the admissions committee: "For each A-minus, imagine an unwritten letter saying that this student isn't quite top notch and not really ready for a PhD program in philosophy". I'm also a fan of not putting too much weight on the reputation of students' undergrad institutions. I'm cheering for the Cal State underdogs. And the GRE I'm inclined to regard as having no predictive value once GPA, writing sample, and letters of recommendation are factored in. (I compare GRE to the bench press as a predictor of athletic performance. If you knew nothing else, it would be somewhat predictive, but once you've seen the person in the field, it really doesn't matter.)
Being an empirically-minded philosopher, though, I'm not happy with mere armchair plausibility. So I thought I could give extra weight to my arguments by looking at how current UCR grad-student performance relates to undergrad GPA, to undergrad institution of origin, and to GRE scores. The staff in the department office kindly provided me with data for all grad students from the entering class of 2007 to the present. This is only 37 students total (with some missing cells), but I thought I might at least pick up some trends. My two measures of academic success at UCR are GPA in the program (setting 3.5 as a floor because of two outliers with some Fs) and whether the student dropped out of the program.
It looks like everything I thought I knew is wrong.
I couldn't make undergrad GPA predictive. I tried several different ways. I tried including undergrad GPA for all students and I tried excluding those who had done some master's work before coming to UCR. When overall GPA didn't work I asked the staff to recode the data with just senior-year GPA in philosophy courses. Still no correlation. Maybe on a much larger sample GPA would show up as predictive, but on this sample it's not even close, not even close to close.
Two important caveats: First, all the students had excellent undergrad GPAs (except for one who repaired with a Master's degree). We're comparing 3.7's vs. 3.9's here, not 3.0's vs. 3.9's. In philosophy courses, their GPAs are even better: Median senior year philosophy GPA was 3.91. So surely this is a ceiling effect of some sort. Still, in my mind, 50% A-minuses in senior-year philosophy looks very different in a PhD application than does straight A's in senior-year philosophy. I would have thought the second sort of student much more promising overall. Second: The students with relatively lower GPA's who were nonetheless admitted (and thus the only students in this sample) presumably had especially excellent letters and writing samples, to help compensate for their disadvantage in GPA relative to other applicants. So maybe that's the explanation. (See, I just can't abandon my opinion!)
Equally annoyingly, given my biases, the verbal section of the GRE was highly predictive. (Math was not predictive.) Verbal GRE score correlates at .49 with graduate student GPA in Philosophy at UCR (p = .004). Our students' median verbal GRE score is 680. Those who scored above median have a mean UCR GPA of 3.91. Those who scored median or below have a mean UCR GPA of 3.76 (t test, p = .001). (Yes, we mostly give our PhD students grades ranging from B+ to A. If you're getting mostly A-minuses and B-pluses in our program, you're "struggling".) This shows up especially strikingly in a 2x2 split-half analysis. 11/14 (79%) of students with above-median verbal GREs have above-median GPAs in our program, while only 5/18 (28%) of students with at-or-below-median verbal GREs have above-median GPAs in our program (chi-square, p = .004).
There was also a trend for overall GRE score to predict sticking with the program: Dropouts had a mean GRE of 1243. Non-dropouts had a mean GRE of 1385. (This was not statistically significant, partly because the dropout group had much higher GRE variance, messing up straightforward application of the t test; p = .13, p = .01 assuming equal variances.)
And reputation of undergrad institution was predictive of GPA in our program. Students whose undergrad institution is a US News top-50 National University or top-25 National Liberal Arts College had a mean 3.91 GPA at UCR, while those not from those elite institutions had a mean GPA of 3.77 (t test, p = .01, equal variance not assumed).
The one bright spot for my preconceptions was this: Students with graduate-level training (usually an MA) before entering UCR tended to do well in the program. Their GPA was 3.87, compared to 3.77 for students with no prior graduate-level training (t test, p = .06); and they were much less likely to drop out: 1/18 (6%) vs. 8/19 (42%) (chi-square, p = .01).
I'm tempted to claim the philosopher's prerogative of rejecting empirical evidence I don't like and sticking with my armchair intuitions. Surely there's something in Kant I can use to prove a priori from principles of pure reason that undergrad GPA is more important than GRE! One thought is this: Since we don't take GRE very seriously in admissions and we do take undergrad GPA very seriously, whatever predictiveness GRE has isn't washed out through the admissions process in the way that the predictive value of GPA probably is to some extent washed out (per caveat 2 in the GPA discussion). If we started taking GRE very seriously in admissions, its predictive value among admitted students might vanish, since those with low GREs would have to be all the more excellent in the other dimensions of their application.
I'd be interested to hear if other professors at PhD have similar data, and whether they find similar results.