Nicholas Lemann’s engrossing history of the infamous SAT—once known as the Scholastic Aptitude Test, but now known only by its initials—raises a lot of questions to which none of us have very convincing answers. The Big Test begins with the bizarre tale of how the Educational Testing Service, a private company based on the outskirts of Princeton, a company with no official (that is to say, with no governmental) backing, came to determine which American students get into the best universities and colleges, and therefore who gets the most attractive jobs. Since its heyday in the 1950s, and with almost no public discussion, ETS has laid down a system that dictates the way high school students estimate their success, encourages the public to think of some ethnic groups as successes and others as failures, and imposes extraordinary social burdens on higher education. It is no small achievement for a not-for-profit company. Whether it is one to admire is another matter.
The second half of The Big Test follows the higher education system that the new testing culture created through the discovery of black educational disadvantage and the growth of affirmative action, and it goes on to discuss the backlash against affirmative action that spawned California’s Proposition 209. That measure, approved in a 1996 statewide referendum, made it illegal for public colleges to take race and ethnicity into account in their decisions on admissions. Its backers thought they had mortally wounded affirmative action; in fact, as Lemann shows, they merely demonstrated one more time that in American politics there are few simple answers.
In its method, The Big Test is very like The Promised Land (1991), Lemann’s moving account of the black migration from the Deep South into the industrial North. The narrative of The Big Test is carried along by a string of life stories, and if none of them is quite as interesting as those in The Promised Land, they once more display Nicholas Lemann’s talent for distilling social analysis out of personal history.
The author may be mildly frustrated that reaction to The Big Test has mostly ignored the extraordinary story of the rise of ETS, and has concentrated entirely on the virtues and vices of the SAT itself; but this is not to be wondered at. The Big Test raises innumerable questions of the sort that torment parents, teachers, students, and administrators—about the purpose of education, about its beneficiaries and casualties, and about our understanding of “merit.” Many of these questions are familiar, but some hit the reader with something of a shock. The most familiar question was given widespread publicity five years ago in The Bell Curve by Charles Murray and Richard J. Herrnstein. The question was: Why do black Americans score relatively badly on IQ tests when they are young, on the SAT in their teens, and on the LSAT and MCAT when applying to law and medical schools? It is not surprising that much of the reaction to The Big Test has again focused on the “Black-White Test Gap,” and therefore on the merits and deficiencies of affirmative action.
Another question is less familiar from the headlines, but must strike anyone who is curious about international differences in higher education. Why does the United States alone select students for college on the basis of tests that measure a narrow aptitude for taking that sort of test, which puts a premium on quick reading and quick problem-solving, rather than on the basis of demonstrated achievement in the subjects that students propose to study? Britain has an advanced school-leaver’s examination, the “A Level,” which tests students’ grasp of such subjects as history, English literature, and science. European countries have similar achievement tests, such as the baccalauréat, the Abitur, and the Laurea. Most English-speaking countries resemble Britain; and even Canada is more like Europe than it is like the US. When the SAT is attacked on educational grounds, it is the contrast with these European examinations that many critics have in mind.
Some unfamiliar questions are perhaps not asked in the US because the answers seem too obvious. Given the obvious unsatisfactoriness of the SAT, why is there no national school-leaving examination that could settle the question of whose students knew what? Why is there not even the outline of a national curriculum by which we might define a decently educated student? Why do we so rarely challenge the ideas that meritocracy is desirable and that the SAT and its pre-graduate school relatives define merit? Why do teachers and parents not ask more searching questions about the intellectual effects of “teaching to the test” (not only the SAT but other multiple-choice exams), which is the pedagogical style that runs throughout the American education system and even into graduate school? Lemann does not answer these questions in any detail, but they haunt the narrative of The Big Test and make the book much more than the story of how ETS came to dominate the world of college admissions testing.
Nicholas Lemann rightly begins by reminding us of the larger historical picture of which the rise of testing is part, and of the social consequences of that rise.
A thick line runs through the country, with people who have been to college on one side of it and people who haven’t on the other. This line gets brighter all the time. Whether a person is on one side or the other is now more indicative of income, of attitudes, and of political behavior than any other line one might draw: region, race, age, religion, sex, class.
Getting to college, or more exactly getting into an elite college, is the great aim of teenagers and their parents; and the great obstacle students have to get over is the assorted tests of success in school that lie in their way. Such a system does more than dictate who goes to which undergraduate school. It defines and selects an elite, the closest thing the United States possesses to a ruling class, “with its own mores and beliefs and tastes and folkways.”
The elite that has prospered in the world of the SAT, the LSAT for law school, and the MCAT for medical school is not the elite that the proselytizers for the testing movement hoped to create—that is to say, it is not an elite notable for its dispassionate concern for the public good. Nor is it an elite that everyone else accepts unquestioningly as the nation’s leaders in commerce, industry, law, politics, and learning. But it is an elite, and it is the creation of two men above all—James B. Conant, president of Harvard, and Henry Chauncey, a Harvard administrator who became the first president of ETS. It was Conant’s patronage that put Chauncey in a position to create a meritocracy, and after a fashion Chauncey did it. He could not have done it if there had not been powerful social pressures on his side, and those pressures would have resulted in some sort of examination-driven higher edu-cation system—but it would have been quite unpredictably different without Henry Chauncey. Nicholas Lemann has been given the run of Chauncey’s archive—Chauncey is still alive at a vigorous ninety-seven—and it is his remarkable career that is the subject of the first half of The Big Test.
Conant and Chauncey were not in fact of one mind about what merit consisted in. James Conant was what Marxists used to describe as a class traitor. He was a member of an elite whose political and economic power he deplored. He was president of Harvard for twenty years, but he thought badly of the faculty, believing that upper-class literary scholars were favored over world-class scientists. He thought even worse of the idle rich young men upon whom the literary scholars spread what cultivation they could.
The pre-war educational scene was, of course, almost unimaginably different from what it has become. Although the US sent more teenagers to high school than any other country in the world, fewer than half of them graduated. And although far more Americans went to some form of college than their peers in any other country—about a fifth of the age group—only one in five stayed long enough to get a degree. Most state colleges engaged in something close to remedial education. Open admissions was not a phrase but it was a fact. To the extent that there was selection of students, it was the result of the mass departure of unprepared or unenthusiastic entrants after a semester or two. Even at Ivy League schools, entry standards were hardly demanding, especially for graduates of established preparatory schools. Just as an upper-class Englishman could put his name down for Christ Church because his father and grandfather had been there before, the graduates of Groton could put their names down for Harvard, Yale, or Princeton. As Lemann says, they might be encouraged not to attend if they were “a little slow,” but great intelligence was neither expected nor encouraged.
It was essentially a WASP world, and a boy’s world, and Conant detested it. It was at odds with the social values that had made America the inspiration of liberals and democrats throughout the nineteenth century. He “took it as a given,” Lemann writes,
that the essence of American greatness was a quality that Alexis de Tocqueville had remarked upon early in the nineteenth century: social equality, of a kind that would be unthinkable in any other country. Because the United States didn’t have a rigid class system, it could take full advantage of its people’s talents and at the same time generate intense social cohesion across a range of physical space and a variety of ethnic origin that elsewhere would have been considered insuperable.
The closing of the frontier had created a WASP aristocracy at odds with the true America. What made the situation worse was that at the other end of the social spectrum, industrialization had created an urban proletariat whose instincts were defensive and collectivist and inimical to a culture of individual opportunity.
But Conant was also a passionate believer in the meritocratic views that had been the common coin of European progressives ever since the French Revolution. The new world needed a leadership different from anything the ancien régime had needed; it was to be scientifically trained and managerially effective; it could only be selected by an educational process that tested for brains and competence at every point. When Conant chose him, Henry Chauncey was merely an assistant dean at Harvard, but with Conant’s encouragement, he became an enthusiast for testing after Conant’s own heart. When America went into World War II, the US armed services got into intelligence testing as they had done in World War I. In 1943, Chauncey was recruited by his good friend John Stalnaker, the vice president of the College Entrance Examination Board, described by Lemann as a
tweedy, clubby association of a few dozen private schools and colleges that had been founded in 1900 to perfect the close fit between New England boarding schools and Ivy League colleges. The boarding schools wanted a uniform admissions test that all the colleges would accept, and the colleges wanted to impose some curricular order on the schools so their students would arrive reliably prepared.
Stalnaker undertook to develop and administer the “V-12” test—the armed services test that would sort out the cleverest draftees and send them off for education rather than straight to the front. Conant strongly recommended the College Entrance Examination Board to the government, and Henry Chauncey to the College Board. Chauncey became a hero to his future civilian employees. John Stalnaker left the College Board when it spun off ETS as a not-for-profit company after the war, and the presidency of ETS fell to Chauncey.
Chauncey was a much gentler spirit than Conant. He was a man with a mission—but it was a much cloudier one than Conant’s. He was sure that testing was essential, but none too sure just what we should be testing for. Lemann begins The Big Test with a vignette of Chauncey at prayer; or at any rate a vignette of Chauncey in February 1945 brooding during morning service on the need for a “Census of Abilities,” to be grafted onto the SAT, and to be administered by ETS on behalf of the entire nation. What he hoped for—a melancholy thought in retrospect—was a set of tests that would identify a wide range of abilities other than the narrowly academic. Conant, it is not unfair to say, was eager for tests that would keep the incompetent out, and had no interest in expanding higher education; Chauncey wanted to discover the good in everyone—not for nothing was he descended from a long line of ministers.
Chauncey’s hope was to discover everybody’s suitability for different sorts of employment—the other great dream, over the past two centuries, of social reformers who have persuaded themselves that individual opportunity, social efficiency, and general happiness can all be had if only we find the mechanism that will get everyone into the job that makes the best use of their abilities. Although it is not a uniquely American dream, one of its most striking (and chilling) expressions is in Edward Bellamy’s quintessentially American utopia, Looking Backward. The never-conducted Census of Abilities hangs over The Big Test as a silent reproach. What happened in practice was that Chauncey, whose early career as an assistant dean at Harvard had given no hint of what was to come, discovered on becoming the first president of ETS that he had an extraordinary business talent. It was a talent for selling the SAT and its progeny under often unpropitious conditions.
It was easier for him to exercise that talent because he never quite realized what he was doing in building up ETS. His numerous foes were entirely clear about what was happening: some complained that he was trying to establish a monopoly in the educational testing market; others that under the guise of running a not-for-profit enterprise, he was creating a vast business that extravagantly rewarded its employees. A small number of intellectual critics complained and continue to complain that the SAT wrecked American high schools by rewarding the mindless ability to take trivial tests, when the object ought to be to create really educated minds. Luckily for Chauncey, the person who might have caused him the most difficulty had died in 1943.
This was Carl Brigham, who had created the SAT back in the 1920s. Brigham had been a young assistant professor at Princeton when he was asked to conduct IQ tests on Army recruits during World War I. At that time, he held unabashedly racist views. His postwar tract, A Study of American Intelligence, declared that the popular view that Jews were smart was an illusion, and that the American IQ was doomed to decline if “Mediterranean” migrants were allowed to flood in. Brigham ran the College Board in the interwar years, and it was there that he created the SAT. But he came to think that his former views on race and intelligence were both false and pernicious, and he also changed his view of the SAT. He thought the test was useful for shortening the interview process in college admissions, but that it was very far from a measure of innate ability. Brigham resisted Conant’s pressure to establish a national educational testing service, and if he had not died in 1943, it seems likely that ETS would never have existed. In that case, the SAT would have continued as it began—as an Ivy League test for scholarship students. But Brigham died at the age of fifty-two, ETS was created, and Chauncey became its president.
Building up the business of ETS was a matter of persuading more and more colleges to use the SAT, and at the same time to ensure that no other organization got ahead of ETS by successfully marketing its own test to colleges in place of the SAT. Lemann’s account of how this was done makes a very good story. At almost every step along the way, ETS might have lost out. In its pre-war version the SAT was just used by Ivy League admissions deans to assess scholarship candidates; and the state universities might easily have gone off in other directions. (It is still true that more Midwestern schools and students use the test called the ACT—which is based on “public-school skills tests” and measures “achievement”—than the SAT.) ETS made a lot of money out of selling tests and charging fees, and might easily have had its nonprofit status removed by politicians responding to populist dislike of the whole testing ethos. In any other country in the world, the process would have fallen into the hands of semigovernmental examining boards or government agencies.
The intellectual deficiencies of the SAT might just as easily have undermined ETS. Among Nicholas Lemann’s particularly valuable services to the reader is his reminder of what the SAT actually is. It is not an IQ test, even though it sprang from the IQ tests administered for the Army during the First World War. It measures much more than innate intelligence, if indeed it measures innate intelligence at all. The private company called the Prince-ton Review, which prepares students to take tests, has flourished by working out that what the SAT really measures is the ability to read the minds of the people who set the test. When ETS is under attack, it reminds critics that the aim of the SAT is only to give admissions deans an idea of how students will perform in their freshman year. When the company is not under attack, the limitations of the SAT are less strongly emphasized.
The commercial excellence of the SAT lies in its reliability. If you took successive versions of the test, the results of the first would predict very accurately what you would get on another. The results do not predict with any precision how well you would do even in freshman year. But in the absence of the kind of national school-leaving examinations used in other countries, the SAT is a useful tool for admissions deans.
Its reliability has always been an important part of its mystique; but it is easy to misinterpret. It is true that, in the absence of highly specific coaching, students’ scores change very little from one test to another (and that after highly specific coaching, they settle down again at a higher level). Many people conclude from this that the test must reveal something deep and fundamental about a student’s abilities. Initially, ETS fostered that impression. For almost twenty years the company kept the contents of the tests a close secret. Spokesmen also said over and over that it was impossible to coach students for the test. If true, this would have made the secrecy pointless for any reason other than to spare ETS the effort of making up new questions for each test. In any case, ETS’s claims for the SAT’s uncoachability were destroyed by the test entrepreneur Stanley H. Kaplan. He gave parties for the kids who’d just taken the test and had them each recall one of the questions. Having worked out what the tests must look like, he taught high school students how to improve their scores, and made a thriving business out of it. Since then, outsmarting the compilers of the SAT has been taken to new levels of sophistication by John Katzman, the president of the Princeton Review.
Although Nicholas Lemann is very much taken with Henry Chauncey and his career, he is as aware as anyone else that what enabled ETS to prosper was mostly that the United States embarked on a period of rapid expansion in higher education, epitomized by the rise of what Clark Kerr called the “multiversity,” the huge, publicly funded institution that provided research for the government and the military, and turned out lawyers, doctors, teachers, and MBAs, and vast numbers of everyday graduates for everyday jobs. From the perspective of admissions deans, it would, in the absence of the SAT, have been too time-consuming to filter students into more and less selective colleges; on the other side of the fence, the SAT told students where they might hope to be accepted.
The effect of treating the SAT as the main indicator of academic merit was paradoxical. To Conant and Chauncey, merit was something they thought hard-working middle-class men possessed, and they hoped that the SAT would allow these young men to displace their idle and feckless social superiors. They did not wonder whether clever women would also displace less clever men; and black Americans were off the radar screen entirely. It is the plight of black Americans that occupies the second half of The Big Test, and has dominated the arguments of the past decade. Women have flourished under the new meritocracy, so much so that at colleges slightly below the most selective, a balanced freshman class is today only achieved by admitting men with slightly lower scores than their female peers.
Black Americans have done a lot less well. Forty-five years ago, when segregated education was outlawed, most social scientists assumed that low black scores reflected the miserable resources devoted to black education, and that throwing money at the problem would solve it. This belief was shattered in 1966 when the Coleman Report commissioned by President Johnson—the data for which was gathered by ETS—revealed that, contrary to what everyone had expected, the test score gap was uniform across the country. Black scores in the Northeast were not higher than white scores in the South. The effect of the findings was not good. They immediately undermined support for increased federal funding for black schools. In 1966, when Daniel Patrick Moynihan, then a Harvard professor, ran a year-long seminar to consider the policy implications of the Coleman Report—itself an austere and entirely nonprescriptive work of social science—he latched onto the finding that black students in integrated schools had higher scores than other black students; and so busing got federal encouragement.
Liberal university presidents thought it would be too long to wait until changes in schools produced black students with the same test scores as their white contemporaries, and they therefore lowered the required scores. So began the long battle over affirmative action. In a curious way, the SAT rendered affirmative action both necessary and impossible at the same time, a point that Nicholas Lemann makes with great acuteness. The argument for using the SAT in selecting students for higher education was always couched in terms of “opening opportunity” to the “talented.” Once the SAT was established as the gateway to higher education, and higher education was established as the gateway to prosperity, it became intolerable to imagine the top fifty colleges and universities with scarcely a black face on campus. Indeed, Nathan Glazer has recently argued that any serious attempt to abolish affirmative action would undermine the democratic legitimacy of the contemporary United States.
It might seem that since American higher education is so full of preferences—for athletes, alumni children, rich children, the relatives of local politicians, and the children of faculty—that adding another category for the sake of racial justice and social diversity would not be a problem. This is not true. Because the SAT—and its graduate school analogues—had become the badge of merit, letting in students whose SATs would otherwise have kept them out seemed improper in a way that the other cases did not. Had they been black and rich, or black athletes, or black alumni children, or black faculty children, little or no offense would have been given. If all they had going for them was their black and Hispanic ethnicity, the unfairness of the help they were given was impossible to ignore. From the other side of the Atlantic, of course, it is the advantages given to athletes, rich children, alumni children, and the rest that seem corrupt; affirmative action for groups long discriminated against might or might not work, but it does not seem corrupt in the same way.
Nicholas Lemann enlivens the paradoxical story of the rise and fall—and then the covert revival—of affirmative action in the state of California by following the careers of some of the Asian-Americans and black protagonists who were involved in the struggle against Proposition 209. He describes how Asian-Americans who had prospered from the meritocratic regime of the SAT went on eventually to fight for the preservation of an admissions system that disadvantaged their own kin. Both in California and in Texas, legislators who were opposed to affirmative action have perceived how politi-cally and socially dangerous such opposition can be and have now voted for new entitlements to admission to undergraduate programs that give a deliberate and substantial boost to the chances of low-SAT ethnic minority students. Minority enrollments at Berkeley are still down from pre- Proposition 209 numbers; but it is a safe bet that further tinkering will take place until some rough-and-ready norm is reestablished. Graduate professional schools present more problems for legislative sleight of hand, and thus far minority enrollments in the top law and medical schools in Texas and California have stayed down. It would be unwise to believe that admissions officers will not think of clever ways of getting them to where they used to be without actually breaking the law.
When it comes to black-white differences in test scores the raw data seem striking, but their interpretation is highly contested. From the per-spective of elite colleges, the unhappiest statistic is that if they were to take the top 5 percent of the age group as measured by their test scores, only 2 percent of the students they accept would be black. Interestingly, black children brought up by white foster parents score like white children until their early teens and then their scores drift toward the black norm. The children of the military defy these generalizations: white army children score much the same as their civilian peers, while black army children score a lot higher. Both white and black scores are heavily biased by social class. It helps a lot to have a father earning more than $70,000, and not only because a well-off parent can afford sophisticated coaching for the SAT. It helps to be rich even more if you are black; the black-white gap narrows strikingly at higher income levels. The public, so far as surveys can be trusted, do not think that black-white differences are biologically determined; but that is little help if they think that black Americans are culturally doomed to be more idle, less good in school, more prone to violence, and less willing to help themselves than their white or Asian peers.
So far, discussion of The Big Test has concentrated heavily on affirmative action, but Americans ought to move off that terrain if they can. For there are three large issues, economic, educational, and sociopolitical, that lie behind arguments over affirmative action, and they only accidentally involve race. The Big Test is generally very sensitive to the changes in American society that have occurred alongside the rise of the testing culture; but Nicholas Lemann perhaps underplays the impact of the economic changes of the past twenty-five years on ordinary white Americans. The numbers are familiar enough. The economic gains of the past quarter-century have very largely gone to the top 5 percent of the population, and to them alone; the bottom fifth have lost ground in real terms, and the members of almost every segment between them and the top fifth have seen their real wages stagnate until the past five years or so. Even within the top 20 percent, gains in real income are unequally spread, with the top 5 percent taking most of them. This is in sharp contrast with the first quarter-century after the end of World War II: until the early 1970s the economy grew faster than during the next twenty years and everyone benefited. But the top 5 percent gained less than the rest, while the bottom fifth gained most.
William Julius Wilson’s The Bridge over the Racial Divide1 argues that against this background, affirmative action is a distraction because it pits black Americans against less-well-off white Americans. Nobody benefits except the well-heeled and their political allies. It may perhaps be true that the new knowledge-based economy is inherently likely to create more inequality than what went before—though the effect seems to be confined to the United States and Britain. What matters is that the other institutions of society should compensate for any such effect—that education, health care, a healthy environment, and physical safety should not be too much at the mercy of the market. Wilson argues convincingly that only a transracial political coalition can achieve that.
Nonetheless, there remains a problem with black education performance on the conventional tests. The only less than cheerful note struck by the Mellon Foundation’s study The Shape of the River2 last year was its admission that black students at highly selective colleges did very well by almost every measure other than their “grade-point average”—i.e., their overall performance in the courses they took. This suggests that it will be a long time before the black-white test score gap vanishes from high schools too. In view of the public dislike for anything that smacks of racial quotas, what is needed is what William Julius Wilson calls “affirmative opportunity” as opposed to affirmative action. What he proposes is eminently sensible on educational grounds as well as grounds of social justice. He wants college admissions officers to look at the complete record of students and try to anticipate what they might achieve in four years with decent instruction. So unsurprising is this idea that ETS itself has recently introduced the idea of “strivers,” the students who do much better than their social background and educational opportunities suggest they should. ETS, as one might expect, proposes to acknowledge “striving” by tinkering with the scores.
It is one of the curiosities of American higher education that faculty take a great interest in graduate admissions, but almost none in undergraduate admissions, other than to complain about the dumb jocks reading the newspapers in the back of the lecture theater. An admissions system that set out to discover whether students were fit for higher education—whether they could read attentively, write with precision, and analyze cogently material related to academic disciplines—would do everyone a lot of good.
We come, then, to the large question underlying The Big Test. What should we think about the ideal of a meritocracy? Nicholas Lemann has no doubt that we do not want it. Like any rational person, he agrees that technical competence matters very much where it is at issue. That we want our brains operated on by competent brain surgeons and by nobody else is a trite observation because it is too true to deny. But not every important position in our social and political life is a matter of technical skill; we want also public and private positions of responsibility held by people who are of good character, who empathize readily with the hopes and fears of ordinary people, who are calm in a crisis and optimistic and energetic in good times.
In any case, merit—as distinct from competence—is a dubious notion. To suppose that merit can be defined by a highly artificial test taken by young people at the age of sixteen or seventeen is really very odd. Indeed, it makes sense only on the basis of just those beliefs that ETS has officially renounced—the beliefs, that is, that there is some general intellectual capacity that can be turned to any number of different tasks, and that a test along the lines of the SAT detects it. It is hard not to wish that Nicholas Lemann had written more about the values that should replace the current assumptions about meritocracy even if this meant saying rather less about the struggle to defeat Proposition 209. But we still should be very grateful for what The Big Test reveals about theirrationality of the educational system we have now.
November 18, 1999
University of California Press, 1999. ↩
William G. Bowen and Derek Bok, The Shape of the River: Long-Term Consequences of Considering Race in College and University Admissions (Princeton University Press, 1998). See Ronald Dworkin, “Affirming Affirmative Action,” The New York Review, October 22, 1998. ↩