Following the procedure of my first compilation on batting averages, I computed both the league fielding average for all regular players and the mean score of the five best for each year since the beginning of major league play in 1876. Figure 20, showing decadal averages for the National League through time, confirms the predictions in a striking manner. Not only does improvement decelerate strongly with time, but the decrease is continuous and entirely unreversed, even for the tiny increments of the last few decades, as averages reach a plateau so near the right wall.
For the first half of baseball (the fifty-five years from 1876 to 1930), decadal fielding averages rose from 0.9622 to 0.9925 for the best players, for a total gain of 0.0303; and from 0.8872 to 0.9685 for average performers, for a total gain of 0.0813. (For a good sense of total improvement, note that the average player of the 1920s did a tiny bit better than the very best fielders of the 1870s.) For baseball's second half (the fifty years from 1931 to 1980), the increase slowed substantially, but never stopped. Decadal averages for the best players rose from 0.9940 to 0.9968. for a small total gain of 0.0028—or less than 10 percent of the recorded rise of baseball's first halt. Over the same fifty years, values for league averages rose from 0.971 for the 1930s to 0.9774 for the 1970s, a total gain of 0.0064—again less than 10 percent of the improvement recorded during the same number of years during baseball's first half.
These data continue to excite me. As stated before, I have spent a professional lifetime compiling statistical data of this sort for the growth of organisms and the evolution of lineages. I have a sense of the patterns expected from such data, and have learned to pay special attention to noise and inevitable departures from expectations. I am just not used to the exceptionless data produced over and over again by the history of baseball. I would have thought that any human institution must be more sensitive than natural systems to the vagaries of accident and history, and that baseball would therefore yield more exceptions and a fuzzier signal (if any at all). And yet, here again—as with the decline of standard deviations in batting averages (see page 106)—I find absolute regularity of change, even when the total accumulation is so small that one would expect some exceptions just from the inevitable statistical errors of life and computation. Again, I get the eerie feeling that I must be calculating something quite general about the nature of systems, and not just compiling the individualized numbers of a particular and idiosyncratic institution (yes, I know, it's just a feeling, not a proof). Baseball is a truly remarkable system for statisticians, manifesting two properties devoutly to be wished, but not often encountered, in actual data: an institution that has worked by the same rules for a century, and has compiled complete data (nothing major missing) on all measurable aspects of its history.
For example, as decadal averages for the five best reach their plateau in baseball s second half, improvement slows markedly, but never reverses—the total rise of only 0.0028 occurs in a steady climb by tiny increments: 0.9940. 0.9953, 0.9958, and 0.9968. Lest one consider these gains too small to be anything but accidental, the first achievements of individual yearly values also show the same pattern. Who would have thought that the rise from 0.990 to 0.991 to 0.992. and so on, could mean anything at all: An increment of one in the third decimal place can't possibly be measuring anything significant about actual play. And yet 0.990 is first reached in 1907, 0.991 in 1909, 0.992 in 1914, 0.993 in 1915, 0.994 in 1922. 0.995 in 1930. Then, thank goodness, I find one tiny break in pattern (for I was beginning to think that baseball's God had decided to mock me; the natural world is supposed to contain exceptions). The first value of 0.996 occurs in 1948, but the sharp fielders of 1946 got to 0.997 first! Then we are back on track and do not reach 0.998 until 1972.
This remarkable regularity can occur only because, as my hypothesis requires in its major contention, variation declines so powerfully through time and becomes so restricted in later years. (With such limited variation from year to year, any general signal, however weak, should be more easily detected.) For example, yearly values during the 1930s range only from 0.992 to 0.995 for best scores, and from 0.968 to 0.973 for average scores. By contrast, during baseball's first full decade of the 1880s, the yearly best ranged from 0.966 to 0.981. and the average from 0.891 to 0.927.
This regularity may be affirmed with parallel data for the American League (shown with the National League in Table 3). Again, we find unreversed decline, though this time with one exception as American League values fall slightly during the 1970s—and I have no idea why (if one can properly even ask such a question for such a minuscule effect). Note the remarkable similarity between the leagues in rates of improvement across decades. We are not, of course, observing two independent systems, for styles of play do alter roughly in parallel as both leagues form a single institution (with some minor exceptions, as the National League's blessed refusal to adopt the designated hitter rule indicates in our times). But nearly identical behavior in two cases does show that we are probably picking up a true signal and not a statistical accident.
TABLE 3
DECADAL FIELDING AVERAGES FOR FIVE BEST PLAYERS AND FOR ALL PLAYERS IN MAJOR LEAGUE BASEBALL
NATIONAL LEAGUE AMERICAN LEAGUE
All Players Five Best All Players Five Best
1870s .8872 .9622
1880s .9103 .9740
1890s .9347 .9852
1900s .9540 .9874 .9543 .9868
1910s .9626 .9912 .9606 .9899
1920s .9685 .9925 .9681 .9940
1930s .9711 .9940 .9704 .9946
1940s .9736 .9953 .9740 .9946
1950s .9763 .9955 .9772 .9960
1960s .9765 .9958 .9781 .9968
1970s .9774 .9968 .9776 .9967
Data on fielding averages are particularly well suited to illustrate the focal concept of right walls—the key notion behind my second explanation for viewing the disappearance of 0.400 hitting as a sign of general improvement in play. Fielding averages have an absolute, natural, and logical right wall of 1.000—for 1.000 represents errorless play, and you cannot make a negative number of errors! Today's best fielders are standing with toes already grazing the right wall—0.998 is about an error per year, and nobody can be absolutely perfect. (Outfielders, pitchers, and catchers occasionally turn in seasons of 1.000 fielding, but only one infielder has ever done so for a full season's regular play—Steve Garvey at first base in 1984.)
If you doubted my explanation for shrinking variation at the upper end of the bell curve for batting averages—that as the mean moves toward the right wall, variation scrunches up into an ever smaller available space, and must therefore decrease—you will surely grant me the argument for fielding averages so close to an absolute wall. Even the 1870s didn't provide much space, but fielders had a bit of breathing room for improvement between their first decadal best of 0.962 and the wall. And improve they did, and steadily. But now, with the five best averaging 0.9968, there just isn't any more space, barring the construction of truly errorless robotic fielding machines.
As the mean moves toward the wall, variation must decrease. For absolute measures of fielding, high numbers persist and low values get axed. But for relative measures of hitting, the wall itself bears no number. The advancing mean retains the same value (as a balance between hitting and pitching), while both hitting and pitching move in lockstep toward their right walls of human limitation. Thus, 0.400 hitting disappears as the league mean of 0.260 inarches steadily toward the wall. But the 0.400 hitters of yore are alive and well, probably more numerous than ever, and standing where they always have resided—just inches from the right wall. But their current best does not measure 0.400 anymore, because everyone else has improved so much, raising average play to a level where an unchanged (or even slightly improved) best can no longer soar so tar above the norm.
The best hitters of early baseball could compile 0.400 averages by taking advantage of a standard in average play much lower than today's premier batters encounter. Wade Boggs would hit
0.400 every year against the pitching and fielding of the 1890s, while Wee Willie Keeler would be lucky to crack 0.320 today. Since pitching and batting both feature relative records, and presumably exist in effective balance throughout the history of baseball, we should be able to detect similar phenomena in the statistics of pitching through time. The best pitchers of the past, legendary figures like Christy Mathewson, Cy Young, Walter Johnson, Three Finger Brown, and Grover Cleveland Alexander, should be no better than their modern counterparts Sandy Koufax, Bob Gibson, Tom Seaver, and Nolan Ryan. But the old pitchers, standing next to their own right wall and facing much poorer average batting, should have racked up numbers that modern hurlers just can't equal.
The fascinating and well-known history of minimal earned run averages provides our best illustration of symmetry between batting and pitching—another indication that these statistics record the general behavior of systems, not just a peculiarity of batting in baseball. As the best hatters sacrificed their 0.400 averages because variation declined while average play improved, the best pitchers lost their earned run averages below 1.50 because ordinary hitters became too good.
The list of the hundred best seasonal ERAs shows a remarkable imbalance. More than 90 percent of the entries were achieved before 1920. Since then, only nine pitchers have obtained an earned run average in the top one hundred (and remember that the number of pitchers, hence the number of opportunities, has expanded dramatically, first with the introduction of the American League and later with expansion from an original eight to our current roster of fourteen teams per league). Moreover, of these nine modern values, seven rank in the lower halt. It we consider the modern achievements, from the bottom up, we get a good sense of the obstacles that must face our superb contemporary pitchers.
Tied at number 100 are Sandy Koufax (1.74 in 1964) and Ron Guidry (1.74 in 1978). Koufax was, well, Koufax—by general agreement the greatest of modern pitchers, perhaps of all pitchers anywhere, anytime (he also holds the ninety-seventh spot at 1.73 for 1966). Guidry, a wonderful Yankee pitcher for a few years, compiled a stellar season in 1978 (with an unmatched combination of total victories and winning percentage of 25-5, for 0.893), and then threw his arm out. Nolan Ryan occupies eighty-seventh place at 1.69 for 1981. And Ryan was, well, Ryan. Nothing else need be said. Carl Hubbell, perhaps the premier pitcher of the 1950s (Lefty Grove was no slouch, either) turned in 1.66 in 1933 for seventy-sixth place and the only entry for his high-hitting decade. Dean Chance, a strictly okay pitcher of the last generation, posted an anomalous 1.65 for seventy-first place in 1964—and I can't figure this one at all. Spud Chandler holds sixty-sixth place at 1.64 for 1943—a fine (if not fabulous) pitcher during the war years, when all decent hitters were blasting away at Germany or Japan instead. Luis Tiant, a damned fine pitcher but not among the greatest, holds sixtieth place at 1.60 for 1968—and I'll return to him in a moment. Dwight Gooden had a fabulous sophomore season in 1985, with a 1.55 ERA that puts him in forty-second place as one of only two modern pitchers in the first half-hundred. He then fell victim to what the newspapers politely call "substance abuse."
We then come to what may be the finest record in modern sports—Bob Gibson's truly incredible 1.12 ERA of 1968, for fourth place, surrounded by forty old-timers before we meet Doc Gooden at number forty-two. Gibson's only superiors are Tim Keefe with 0.86 in 1880, Dutch Leonard at 0.96 for 1914, and Three Finger Brown at 1.04 for 1906. How could Gibson compile such a record—the only post-1920 value below 1.50, and way, way below at that—in our modern era of greatly improved average hitting?
I don't want to take a thing away from Bob Gibson, who absolutely terrified me in the 1967 World Series, when he almost single-handedly beat the Red Sox by winning three games and casting a pall of inevitability over the whole proceedings. But, in slight mitigation, 1968 was a really funny year, as mentioned previously (see page 104). For some set of reasons that no one understands, pitching took a dramatic upper hand that year, capping a trend of several years' duration. (As explained before, the rule-makers then restored the usual order by lowering the pitching mound and decreasing the strike zone; batting averages and ERAs rose appropriately in the 1969 season and have remained in balance ever since.) The 1968 season didn't just belong to Gibson; in that year, low ERAs sprouted like dandelions in my garden. In most years of modern baseball, no pitcher in either league has posted an ERA lower than 2.00. Uniquely in 1968, all five leading American League pitchers bettered this mark, as Yastrzemski won the batting title with a paltry 0.301 (Tiant at 1.60, McDowell at 1.81, McNally at 1.95, McLain at 1.96—a banner year for Scotland—and John at 1.98. As I said, Tiant was a terrific pitcher and great fun to watch, but not one of the game's greatest. If he could post 1.60 for 1968, baseball was really out of whack that year.) So Gibson certainly took maximal advantage of a weird year, but let's not take anything away from him. No one, no matter how good, had any statistical right to post a value so much better than anything achieved for sixty years, especially when general improvement in play should have made such low ERAs effectively unobtainable. Gibson had one helluva year!
In quick summary of a long and detailed argument, symmetrically shrinking variation in batting averages must record general improvement of play (including hitting, of course) for two reasons—the first (expressed in terms of the history of institutions) because systems manned by best performers in competition, and working under the same rules through time, slowly discover optimal procedures and reduce their variation as all personnel learn and master the best ways; the second (expressed in terms of performers and human limits) because the mean moves toward the right wall, thus leaving less space for the spread of variation. Hitting 0.400 is not a thing, but the right tail of the full house for variation in batting averages. As variation shrinks because general play improves, 0.400 hitting disappears as a consequence of increasing excellence in play.
11
A Philosophical Conclusion
Some people regard this explanation as a sad story. One can scarcely decry a general improvement in play, but the increasing standardization thus engendered does seem to remove much of the fun and drama from sports. The "play" in play diminishes as activities become ever more "scientific" in the pejorative sense of operating like optimized clockwork. Perhaps no giants inhabited the earth during baseball's early days, but the best then soared so far above the norm that their numbers seemed truly heroic and otherworldly, while our current champions cannot rise nearly so far above the vastly improved average.
But I suggest that we should rejoice in the shrinkage of variation and consequent elimination of 0.400 hitting. Yes, excellence in play does imply increasing precision and standardization, but what complaint can we lodge against repeated maximal beauty? I have now been a fan for fifty years. I have seen hundreds of perfectly executed double plays and brilliant pegs from outfield to home (that may or may not have beaten the runner charging from third)—the kind of beautifully orchestrated precision that probably occurred only rarely in baseball's early years. I do nor thrill any less with each repetition. The pinnacle of excellence is so rare, its productions so exquisite. Did we ever get bored with Caruso or Pavarotti in their prime? I would much rather have my expectation of excellence affirmed when I go to the ballpark or the opera house than to take potluck and hope for a rare glimpse of glory in a sea of mediocrity.
Moreover, the rise in general excellence and consequent shrinkage of variation does not remove the possibility of transcendence. In fact, I would argue that transcendence becomes all the more intriguing and exciting for the smaller space now allocated to such a possibility, and for the consequently greater struggle that must attend the achievement. When the norm stood miles from the right wall, records could be broken with relative ease. But when the average player can almost touch the wall, then transcendence of the mean marks a true outer limit for conceivable human achievement. (Again, I would make an analogy to musical performance. Do we not rejoice when every string in a symphony
orchestra plays with exquisite beauty and consummate professionalism? And do we not thrill all the more when, in this context of superb general performance, a great soloist does something so special that only angels in heaven could have contemplated the possibility?) I would carry the argument even further and point out that a norm near the right wall pushes the very best to seek levels of greater accomplishment that otherwise might never have been conceptualized. I will speak in the final chapter about the heroic efforts, often with attendant accident and loss of life, that such "pushing of the envelope" imposes in the almost holy mania that infects the greatest performers in the circus arts and other dangerous activities. Call it foolish (and swear up and down that you would never so act yourself), but acknowledge that human greatness often forms a strange partnership with human obsession, and that the mix sometimes spells glory—or death.