It’s Time To Revise STEM Doctoral Admissions Procedures

The selection of students for science, technology, engineering and mathematics (STEM) doctoral programs determines who will be leaders in higher education and fields that underpin the U.S. economy. For decades, this selection process has relied heavily on the Graduate Record Examination (GRE), a test originally developed to “level the playing field” for applicants from variously ranked educational institutions.

Unfortunately, women and students from underrepresented groups score less well on the GRE than other groups, even on the revised test instituted in 2011. This limits the pool of applicants from underrepresented groups who score above arbitrary “cut off” scores commonly used by admissions committees.

This is an impending problem because women make up more than 50% of the U.S. population but earn only 25% of STEM PhDs, and non-Asian minority groups comprise nearly 33% of the population but earn only 12% of these degrees (Fig. 1). Moreover, the U.S. Census Bureau projects that the U.S. will become a majority-minority nation by 2043. Thus, to maintain our preeminence in STEM, we need to engage citizens from all genders and all racial and ethnic groups in the STEM enterprise.

The Northeast Alliance for Graduate Education and the Professoriate (NEAGEP), originally formed in 1999 through grants from the National Science Foundation, is group of faculty and administrators from 15 institutions who work together to diversify the STEM doctoral populationreference^†. Leaders at each institution found that GRE scores of students from underrepresented groups, including women, constituted a sizeable obstacle to admission, thus serving as an obstacle to diversity. However, through novel initiatives, most convinced STEM graduate programs in their institutions to disregard GRE scores of minority applicants. They observed that students with lower than average GRE scores often excelled in these programs and went on to successful careers in industry, government and academic fields.

Based on this anecdotal evidence, four NEAGEP state flagship institutions conducted a study examining whether the GRE predicted STEM Ph.D. completion, first-year drop-out rate, or time to degree for U.S. citizens and Permanent Residents overall. With over 1800 subjects, this study is the largest to date focused on the ability of GRE scores to predict success in STEM doctoral programs. Results of this study were published recently in PLOS One.

The investigators found that GRE scores of women are lower than those of men, but neither completion rates, first-year drop-out rates, nor time to degree differed between genders. In addition, there were no differences in completion rates for women based on the quartile into which their GRE Verbal or GRE Quantitative scores fell.

Surprisingly, GRE scores for men were negatively correlated with completion rates. Particularly striking, men whose GRE Quantitative scores were in the lowest quartile (with a mean GRE percentile of 34) finished at a rate of 74%, while those in the highest quartile (mean of 91) finished at a rate of only 56.2% (Fig. 2).  The males with the lowest GRE Quantitative scores also had the lowest GRE Verbal scores. This pattern was seen in all of the four study institutions and in the field of engineering, where quantitative skills are often deemed especially important.

One reason given for using GRE scores in the admissions process is that scores are intended to predict preparedness for graduate coursework. Presumably, that would mean that students with low GRE scores would leave more often during the first year than those with high scores. In the PLOS One study, researchers found that neither GRE Verbal nor GRE Quantitative scores differed between men or women students who left during or after the first year and those who completed their STEM Ph.D. degrees (Fig. 3).

Similarly, if students with the lowest scores were not prepared for graduate work, it might be predicted that they would require a longer time to complete degrees. Again, the researchers found that this was not the case — regardless of the GRE scores, both male and female students completed STEM Ph.D. degrees in approximately 6 years (Fig. 4).

These findings show that GRE scores do not predict what are arguably the most important indices of preparedness for STEM Ph.D. studies — persistence beyond the first year, time to degree, or STEM Ph.D. completion. Not only are the GRE scores not predictive of success, but the GRE is also expensive to take, even with financial aid from the Educational Testing Service.

In addition, students are advised to study for the exam for several months and, to give themselves an advantage, they are offered a number of preparatory courses, with many costing more than $1000. These investments in time and money may be a competitive barrier for students who need to work outside of school to pay their college costs.

Even if students are admitted regardless of their scores, many define themselves by their “numbers.” Those with high scores may underestimate the difficulty of graduate school and those with low scores may feel that they are less capable than others, decreasing their confidence and performance. Those with low scores also are more likely to be assigned to lower level or “remedial” courses that could delay their time to graduation or limit opportunities for advanced learning. Combined with evidence that GREs are biased against women and students from underrepresented groups, it is hard to justify their continued use.

The authors of the PLOS One study suggest that one of their most important findings is that men in the lowest quartile of GRE scores finish at higher rates than students in any other group. They propose that it will be important to identify characteristics that motivated admissions committees to ignore the low GREs and to determine whether these characteristics contributed to the success of the low-scoring men.

The authors and colleagues from across the country have established a new organization, Beyond the GRE.Org. They recently launched a website (www.beyondthegre.org) that provides information for students who want to find programs that do not require GRE scores. The website also provides resources to help graduate programs design more inclusive and effective strategies for identifying talented STEM students who will thrive in their STEM Ph.D. programs.

These findings are described in the article entitled Multi-institutional study of GRE scores as predictors of STEM PhD degree completion: GRE gets a low mark, recently published in the journal PLOS One. This work was conducted by Sandra L. Petersen and Krista Gile from the University of Massachusetts Amherst, Evelyn S. Erenrich from The State University of New Jersey, Dovev L. Levine from the University of New Hampshire, and Jim Vigoreaux from the University of Vermont.

References

1. National Science Foundation, National Center for Science and Engineering Statistics; Special Report NSF 18-304. Doctorate REcipients from U.S. Universities: 2016 2018; Available from: www.nsf.gov/statistics/2018/nsf18304/.
2. Quick Facts, United States. 2017  [cited 2018; Available from: https://www.census.gov/quickfacts/fact/table/US/PST045217.

†: Bennett College, Boston University, Claflin University, Jackson State University, Lincoln University, Massachusetts Institute of Technology, Medgar Evers College, Pennsylvania State University, Rutgers the State University of New Jersey, and the Universities of Connecticut, Maine, Massachusetts Amherst, New Hampshire, Puerto Rico Mayagüez, Rhode Island and Vermont.