A.C. Thomas, Scientist

Elsevier Bought Mendeley; Internet Freaks Out; I'm Barely Surprised

2013-04-09T23:07:42Z

I love it when my nerdiest pastime and professional interest -- bibliometrics and academic paper management -- makes the news in a big way. I like it more when it's direct evidence of all the issues that academia faces as a public good.

Mendeley is a "freemium" service for managing collections of academic papers, offering a cloud-based storage service for personal libraries. Its users have considerable affection for the service, whose management team has proclaimed their dedication to the Open Access movement. In the process, and in contrast, the company has built an impressively large database on user activity, one that was kept to itself rather than being available to its users.

Which is why the backlash to its purchase by Elsevier, a company that takes advantage of our public good for its private enirchment, strikes me as extremely naive. Mendeley's supposed commitment to an open access movement was already betrayed by their Facebook-like business model.

I'm less shocked since this is only the latest in a series of "betrayals" by companies supposedly behind principles of openness:

Combine this with the recent rise of "predatory" journals, and you can see why my worry has less to do with any individual companies and much more about the need to solidify the process of scientific communication as a public good.

Resigned To Change

2013-02-04T19:41:42Z

What follows: I resign from two editorial boards on principle. I don't feel heroic about it, but it had to be done.

Last year, I signed the Elsevier boycott as soon as it was announced. I firmly believed at the time that the principles of the boycott were sound: this was a company that had historically charged obscene prices, and made extreme profits, by selling other people's work with cartel-like levels of market control. I knew how this made sense in the past -- as both a filter and a distribution source, academics had little choice but to work with for-profit publishing companies. But now, the situation borders on the absurd. To make an example out of one of the biggest publishers seemed almost automatic, and I joined the official boycott without hesitation, in addition to years of avoiding Elsevier journals to publish my own work.

All that's needed for the system to work without big publishing companies is an environment of open publication, and so I've enthusiastically submitted my work to society journals and others with principles of openness. One of these was the Berkeley Electronic Press (bepress), which as a non-profit electronic publisher, committed to open access, promised a way forward: with the Internet as the ultimate distribution venue, all that would be needed is an editorial structure, handled as it has been by academics, the vast majority of whom work pro bono.

And so I joined two such efforts; first, the nascent journal Statistics, Politics and Policy, still in its infancy, in 2010; and second, the slightly more venerable Journal of Quantitative Analysis in Sports, which (to my delight, as a long time author and reader) I was asked to join roughly a year ago. Both have sterling editorial boards (aside from me) and I've enjoyed my time and efforts with both groups. But things got complicated in September 2011, when for-profit publisher De Gruyter announced that it was buying many bepress journals, including both SPP and JQAS. Originally it seemed as though little would change; my back-channel inquiries suggested that the new bosses wanted to change very little from the original bepress setup, which is why I was comfortable joining JQAS after the transition.
]]>
The sticking point for me came in July when we discussed the company's policy on preprints at the editorial board meetings. If the main contribution of a journal today is to improve the appearance and readability of an article, then it makes sense that a journal should allow preprints of work to be held by sites like arxiv.org, since that increases the incentive to get a fresh, improved copy from a journal website. (In fact, I rather like the idea that copyright in the content should be separate from copyright of the presentation -- it makes it clear who did what work.)

I was surprised to learn the de Gruyter policy on this: that they would prefer no traces of the article as preprint to be in the wild, and that they would permit a personal copy of the final proof to be on an individual's website -- but not on any public archive sites. In today's academic climate, restricting the preprint market like this is detrimental to new science. Individual websites are rough and poorly indexed; public archive sites, claiming only the barest of rights, can handle much of this burden at a low cost. The rise in importance of arxiv.org as a home for department tech reports (including my current employer) accentuates this.

Helping de Gruyter make money off my volunteer work might have advantages for my career, but it's bad for science to keep this kind of power in the hands of for-profit companies when the alternatives are so compelling, and when so much of the funding for our work comes from public sources. I know that it has to be a lot cheaper than the current system; we just need to figure out how to make it work on a grander scale.

In the end, as an early-career scientist, there's little I can do to change the course of the journals from within. I'd rather focus my efforts onto matters that can push academic publishing towards more open publication -- keeping more rights for authors and the people who pay for the research, and. And so I decided to resign from the boards of both SPP and JQAS as of the end of last year, not because of any of the people involved -- I have warm feelings for both editorial boards, and I personally like the de Gruyter reps we've worked with -- but because it's counterproductive for me to continue in that capacity.

Update: To my astonishment, I received notice this week that the senior editors of SPP are all resigning from the journal and are trying to found a new, similar journal under the auspices of the American Statistical Association. I'm fairly certain that my resignation didn't push anyone on this, but it was very comforting to know that they had similar misgivings to the arrangement as I did.

The Statistical Properties of the Electoral College Are Perfectly Bearable

2013-01-28T17:17:08Z

What follows: I give a not-so-ringing endorsement of the Electoral College, by showing that the current mode has reasonable partisan symmetry. I'd still prefer a scheme with the national popular vote, but what we've got ain't so broke.

Andrew Gelman, Gary King, Jonathan Katz and I published an article on the Electoral College just in time to miss the 2012 US Presidentlal election (here from SSRN and here from the journal website) but apparently just in time to catch the reactions of people complaining about how the election went. Last week, news broke that a group of Virginia politicians wanted to reapportion their state's electoral votes by congressional district, echoing similar attempts in Pennsylvania in 2012 and California in 2008, making it clear that the issue isn't going away any time soon.

In brief, we quantified how much partisan bias there has been in the electoral college system as it stands today (essentially none), if certain states reapportioned in this matter (it depends on the state), and if all states did so (it would have been substantially biased towards the Republicans). In extending the analysis for this post, we find that the Electoral College had no meaningful partisan bias in the 2012 election either.
]]> of course it is) but what its effect would be on the entire system if more states did this. More to the point, we wanted to check the state of the system as it was at each election, according to the simple question of partisan symmetry:

If one party in a two-party system receives X% of the vote and Y% of the seats, then in the hypothetical situation that the other candidate receives X% of the vote, they should also receive Y% of the seats.

Replace "party" with "presidential candidate" and "seats" with "electoral votes" and you get to the heart of it; see the paper for details on how we estimate partisan bias. The easiest application of this is if the overall popular vote is tied, then they should expect to receive an equal number of electoral votes. Is this condition present now? Would it be if California or other states split their votes by district? What if every state did it that way?

The paper contains an analysis for each election between 1956 and 2008; for this post, I re-ran the analysis adding preliminary data from 2012 (with a little imputation for as-yet unreported districts) and calculated the effective partisan bias for the election. Zero indicates a symmetric system; a bias of 1 (or -1) would indicate that if the vote were split evenly, the Democratic (or Republican) candidate would win all of the electoral votes available.

As the system stands right now, things are basically fair, and have been from 1980 onwards -- closest to zero is fairest:

If California's electoral votes had been split by congressional district, there would have been some interesting consequences -- not nearly enough of a bump in 1980 to re-elect Jimmy Carter, but a consistent Republican edge ever since.

Suppose we counterweighed this by changing historically Republican Texas to Congressional district apportionment. It would have helped, but not nearly enough to balance the scale:

Now, suppose every state split their electoral votes by Congressional district. The edge is consistently Republican, even today:

In the end, even if other states counter-balanced each other to try and even things out, it would probably make things worse. As things stand, the status quo of the Electoral College is adequate without any kind of large-scale modification, so far as we can predict.

Digital Publishing Isn't Harming Science, It's Liberating It

2012-11-27T04:20:48Z

It's somewhat appropriate that a complaint from a scientific authority on the decay of scientific publishing should be circulated on the Huffington Post, whose legions of unpaid bloggers gain only exposure for their efforts; how closely it parallels the history of scientists, working without pay, as both content producers and vetters, and what it means for the future. Douglas Fields' comment on scientific publishing (thanks, Simply Statistics!) has the facts right, but the conclusions he draws are contradicted by the very nature of the system he's trying to assault.

The key to it all is the nature of peer review:
]]>

A scientific discovery is useless if it is not communicated with authority to the scientific community. For centuries scientists submitted their research findings for publication in scientific journals that were run by the leading scientists with expertise in a specialized field who served as journal editors. The editors evaluated the submission, and if the findings appeared to be important and technically sound, they sought out other scientists around the world with recognized expertise in the area to read the manuscript critically and advise the editor and authors (anonymously) on its suitability for publication.
This process is essential to root out poor science and pseudoscience, and to prevent bogging down the advancement of science by cluttering the literature with contradictory and erroneous findings. The expert peer reviewers evaluated the potential strengths, weaknesses, technical flaws, significance and novelty of the finding, and they suggested the need for further experiments. If the study failed to be accepted for publication by the editor, the authors benefited from the editorial review process, and they revised their work for submission to another journal.

I'm with you, sir! This is the beauty of the peer review system, and the source of it isn't the paper it's printed on; it's the stamp of approval of the editorial board that matters. A quality board is a collection of distinguished members with noteworthy professional experience, combined with their past record of approving meaningful publications.

Recent government-mandated changes in scientific publishing are undermining this critical process of validation in scientific publication.

And now he's lost me. Scientific validation is carried out by the editorial board and its referees -- the vast majority of whom are unpaid volunteers -- and abetted by a publisher, not controlled by one. The sharp drop in publishing costs from online publishing will only put more control in the hands of the academics who decide what's truly important.

The first change to which he speaks -- a mandate that all papers should be openly accessible for all readers, if their research was funded by federal grants -- affects publishers, not editorial boards. While Fields defends the necessity of the publisher as the producer, editor and disseminator of research, he seems to underplay his own role as the editor-in-chief of a journal, one with the responsibility of seeking out the editorial board, ensuring the quality of the process, and so forth. It's true he doesn't copy-edit or type-set, but these tasks are getting cheaper all the time, and arguably, current publishers don't do that great a job of it.

Fields is also conflating the two major models of Open Access publishing: "Green OA", which says that authors should archive their preprints on public sites (at little cost) is the PubMed approach, and doesn't take away whatever value that copy-editing, type-setting and large-scale printing adds."Gold OA", in which authors pay the publishers for the dissemination of their work, is the model pursued both by top-quality outfits (including CUP) and spamming bottom-feeders. That's why his second point -- that electronic publishing decreases the cost barriers to entry -- is on the mark. But I'm baffled by what follows in his personal testimonial:

Neuron Glia Biology was a scientific journal that was launched in 2004 by me and like-minded scientists to advance scientific research on neuron-glia interactions, and it was published by Cambridge University Press until this year. Neuron Glia Biology provided the opportunity for 1,400 authors to introduce their new research on neuron-glia interactions into the scientific literature, and it helped advance a new field of science, but no longer.

Again, I say: this commentary was published on the Huffington Post. For free. Whether or not it was more visible because of this service, the real stamp of approval comes not from being on this website, but from your peers in the community who judge your work. And those 1,400 authors will not stop writing, the editorial board of Neuron Glia Biology will still believe in their mission, and if it comes to it, finding an online-only home for a format won't change that -- I know it's easier in the mathematical sciences, but biology isn't far behind. The success of the enterprise comes down to the acceptance of the community first.

Vanity journals might be going for a money grab, but so are Elsevier and Nature, both of which are hideously profitable thanks to their monopolistic tactics and reliance on free labour -- not to mention that CUP seems to be doing all right for itself. The pressure from the community is exactly why I doubt that most scholars will fall for bad articles in true vanity journals -- and thanks to the exact peer forces that propel academia, if they do get any attention, the end result will be a humiliated, slightly poorer academic, not the end of the discipline as we know it.

I sympathize with Dr. Fields' anxieties about the state of academic publication today, but I'm far more excited by the premise of technology to keep things fresh than I am about a corporate/government takeover of science. We just have to remember that we're still the ones in charge.

538's Uncertainty Estimates Are As Good As They Get

2012-11-07T13:01:35Z

(or, in which I finally do an analysis of some 2012 election data)

Many are celebrating the success of the poll aggregators who forecasted the states won by each candidate -- many called all 50 right, including FiveThirtyEight. No doubt Nate Silver will continue to be the world's most famous meta-analyst given this accomplishment -- even though several of his peers, such as the Princeton Election Consortium, Votamatic and Simon Jackman's projections for the Huffington Post, seemed to do equally well. The strength and depth of the number of polls in swing states no doubt had a lot to do with all their successes.

How much of an accomplishment this is, of course, depends on context; the winner in most states was easily predicted ahead of time with the barest minimum of polling. Consider instead a related question: how close were the vote shares in each state to the prediction, as a function of the margin of error?

The simplest way to check this is to calculate a p-value for each prediction: for each prediction and its associated uncertainty, calculate the probability that the observed value (vote share) is greater than a simulated draw from this distribution. The key is that for a large number of independent prediction-uncertainty pairs, we should see a uniform distribution of p-values between 0 and 1.

I grabbed the estimates from FiveThirtyEight and Votamatic (at this time, I have only estimates, not uncertainties, for PEC or HuffPost) and calculated the respective p-values assuming a normal distribution in each case. Media coverage suggested that Nate Silver's intervals were too conservative; if this were the case, we would expect a higher concentration of p-values around 50%. (If too anti-conservative, the p-values would be more extreme, towards 0 or 1.)

On the contrary, the 538 distribution is nearly uniform. The closer the points are to the diagonal, the better the fit to the uniform:

Repeating the process for Votamatic:

The values are pushed towards zero and one, so the confidence intervals are far too tight: the Votamatic predictions turned out to be too overly precise.

The data I used are here. (I read the Votamatic intervals directly off the graphs; if I can get a more precise value, I'll repeat the analysis.) I'm very curious to know how the other meta-pollsters did, so if anyone has put together that data, please send it my way.

The Journal System and Statistical Publishing

2012-10-11T05:35:55Z

David Banks has some notions about how to evolve the peer review system, specifically for publishing in statistics. Not surprisingly, I agree with him about most things, namely the rise of the Internet as giving rise to many more creative options for outlet.

One of the trickier things to figure out is whether or not article quality would be upheld under a new system. Quoth Banks:

Article quality can be signaled in multiple ways, either by conventional review or by ungameable rating systems, similar to page-ranking algorithms.

Conventional review has its benefits, but I'm not sure we have a good way of instituting this yet. And no system is ungameable, even PageRank (think "miserable failure"), but as long as there's effort put into it by the community, there's hope.

Scrabble Cheating

2012-08-16T05:53:51Z

News of a cheating scandal in Scrabble has rippled through the community, after a competitor (proverbially) hid the blanks up his sleeve during matches, leading to his subsequent disqualification. As he is a minor, his name is not being shared, thereby preventing us from asking why if he was going to cheat, why he couldn't have done a better job of it.

Let me take the opportunity to remind tournament organizers everywhere that the latent tile order design mechanism could have prevented this travesty from happening. And all they would have had to do was spend tens of thousands of dollars to design and build the physical apparatus to make it happen, and tens of thousands more to outfit the entire tournament with them. But in the long run, shouldn't we do everything we can for the children?

Our New Hockey Modelling Paper: How Much Better Are Some Players Than Others?

2012-08-09T04:40:14Z

Following Sam's presentation at this year's JSM, we are proud to release our preprint for consumption:

A.C. Thomas, Samuel L. Ventura, Shane Jensen, Stephen Ma, "Competing Process Hazard Function Models for Player Ratings in Ice Hockey", available from arXiv.

Abstract: Evaluating the overall ability of players in the National Hockey League (NHL) is a difficult task. Existing methods such as the famous "plus/minus" statistic have many shortcomings. Standard linear regression methods work well when player substitutions are relatively uncommon and scoring events are relatively common, such as in basketball, but as neither of these conditions exists for hockey, we use an approach that embraces these characteristics. We model the scoring rate for each team as its own semi-Markov process, with hazard functions for each process that depend on the players on the ice. This method yields offensive and defensive player ability ratings which take into account quality of teammates and opponents, the game situation, and other desired factors, that themselves have a meaningful interpretation in terms of game outcomes. Additionally, since the number of parameters in this model can be quite large, we make use of two different shrinkage methods depending on the question of interest: full Bayesian hierarchical models that partially pool parameters according to player position, and penalized maximum likelihood estimation to select a smaller number of parameters that stand out as being substantially different from average. We demonstrate this on games through five NHL seasons.

Our ultimate goal for this project was to first come up with a mathematically rigorous method for determining how players affected the outcomes of hockey games. As we are stochastic modellers by training, this to us meant finding a generative probability model for how these games may come to be. (I am a fan of this approach in hockey for several reasons.) We took the Rosenbaum/Macdonald approach of dividing the game into shifts, so that no players substitute for each other during each observational unit. We then took the outcome of each event to be whether or not one team scored a goal (or changed off some of their players) and automatically factored in how much time had elapsed. We also adjust for the fact that some players play much of their time together, and that some players play very little.

There are a lot of things we can actually put in this model beyond player identifiers, like teams or pairs of players together -- so long as we're willing to wait for the solution to compute, which for all players over 5 seasons, can be on the order of a day using our current code. We discovered a few things that are of interest to hockey fans as well as statisticians and probabilists, but two jump out especially to me:

Defencemen as a group are far more interchangeable than goalies or forwards are, at even strength. This is likely because they share most of their prime duty -- defence -- with the goaltender, who show much more variety in ability, whereas most of the burden on scoring belongs to the forwards.
There are a few player pair combinations that are just plain awful together (rather than when they play apart) such as when Sidney Crosby and Evgeni Malkin played on the same line. The additional deficit to team defence was so big compared to any extra gain in offensive ability that it would be much more worth playing them separately.

Further results -- and plenty of tables! -- are available in the paper.

Well, This Is Embarrassing...

2012-07-11T02:04:21Z

I posted a preprint earlier today for a comment on a discussion paper on social network outcomes, but found out shortly afterward that the comments for this journal are to be embargoed until publication. So I've complied, removed the paper from the preprint site and taken my blog post down for now. Once the embargo is lifted, I will re-post both along with other reactions.

My sincerest apologies to those who linked to the post to now find it dead -- particularly those people I sent it to for their reaction.

Prediction, The Big Discovery and Heartbreak

2012-04-08T19:23:19Z

It's a year old, but I only just heard the story of what happened to a baseball researcher I first read about 10 years ago:

http://espn.go.com/blog/sweetspot/post/_/id/6835/voros-mccracken-changed-the-game
http://www.thepostgame.com/features/201101/sabermetrician-exile

"Voros" McCracken had a particular insight about pitcher ability in the late 1990s, namely that a subset of the data -- "defense-independent" pitching statistics-- was not only an excellent predictor of the runs allowed by a pitcher, but is also highly persistent from year to year. Others discovered the same principle around the same time, but it was the publication of this work that got the attention: the strong claim that the differences between pitchers, on batted balls in play, were so small as to be ignored. While it wasn't quite correct -- there is some predictive power in the remaining information -- it was enough to change people's ideas about how the game works.

And now, McCracken is out of baseball, applying analytical methods to undisclosed professional soccer clubs, having made a meager living while working for the Boston Red Sox in the early 2000s. As one of the figures I read about before going to grad school in statistics, it definitely got my attention when I heard this news.

There are two things I take away from this whole story:

1) After 12 years of exposure to real data and methods, I feel like it should have been *screamingly obvious* to find the best set of predictors of success, and an assignment I could give to an undergrad with existing databases. But would it be obvious enough to construct a narrative around it, and convince the public? I can think of a good explanation after the fact, that there's a lot of general uncertainty once the bat hits the ball, like the angle of attack, but I've had years to think about it.

2) Any research I do on sports is for the sake of teaching, or just as a hobby, and the value it brings me to share something with the world is a nice bonus. I'm not under pressure to find a brilliant discovery to keep my job -- at least, not when it comes to sports.

Gelman has said that the most noteworthy discoveries these days aren't the small effects that come out with more data, but the big ones that everyone else just missed. This definitely qualified as one of those. Whatever the next big discovery, I'm sure we'll all think it was obvious years later, even if it wasn't.

Statistics and Scrabble, Together At Last

2011-07-14T14:48:43Z

Sitting on my to-do list for a while now has been an exploration of Scrabble from an experimental design point of view; how to better design a tournament to make the variance as small as possible while still preserving the appearance of the home game to its players. One goal was to figure out a way to carry out a true "duplicate" version of Scrabble so that multiple pairs could have access to the same tiles, rather than the currently popular version in Europe that has no defensive element to it.

I'm proud (relieved?) to say that I've finally finished the first draft of this work for two-player head-to-head games, with a duplication method that ensures that if the game were repeated, each player would receive tiles from the reserve in the same sequence: think of the tiles being laid out in order (but unseen to the players), so that one player draws from the front and the other draws from the back. Like Lady and the Tramp with spaghetti:

I modified the Scrabble simulator Quackle to accept a predetermined tile order, then simulated over one million matches between Quackle's "Speedy Player"s using each of 10,600 tile orders 100 times. One goal of this was to figure out how much of the variance in score comes from the tile order and how much comes from the board, given that a tile order would be expected. It turns out to be about half-bag, half-board, so that if this scheme could be used in tournaments, it would visibly cut down the number of matches needed to figure out the best player (though it would need a Goldbergian apparatus to implement in live games.)

Some other findings from the simulations:

The blank is worth about 30 points to a good player, each S about 10.
The Q is a burden to whichever player receives it, effectively serving as a 5 point penalty for having to deal with it due to its effect in reducing bingo opportunities, needing either a U or a blank for a chance at a bingo and a 50-point bonus.
The J is essentially neutral pointwise.
The X and the Z are each worth about 3-5 extra points to the player who receives them. Their difficulty in playing in bingoes is mitigated by their usefulness in other short words.

I have yet to make any other conclusions about how I think the game should be modified, mainly because it's premature without testing these ideas out on human players. Any volunteers?

Geek talk: Running R processes remotely through ESS

2011-04-17T04:47:14Z

I've been wondering for a while how to use the convenience of Emacs on a local machine, while running R processors on a server remotely. According to http://www.xemacs.org/Documentation/packages/html/ess_3.html, this can be done with the ess-remote environment.

1) Use M-x shell to open a shell environment.

2) Use ssh to connect to the server of interest. If desired, use "screen" to make the process resilient to disconnection.

3) Run R.

4) Use M-x ess-remote to enable the shell as an ESS process.

The advantages of this process are the added convenience of an environment that allows instant execution of lines of R code. Files still must be loaded on the remote server, rather than referenced on the local machine, but it's still an improvement.

Big chances to change redistricting this year...

2011-02-06T20:20:43Z

...assuming we can take advantage of them.

State legislatures are preparing to conduct their decennial redistricting processes, now that the data from the last census has been processed. In states where the legislature does it directly, the result is known to be far less than fair for its constituents, in that either one party tries to take control of an "unfair" number of seats (remember the Oklahoma exodus!) or the incumbents on both sides work to protect their own re-election prospects (some people have all the luck).

There is hope that a bipartisan commission, whose members cannot run for office in their newly drawn districts, would be able to break the easily recognizable incumbency advantage and, from there, create a map that would be fairer to all parties. But how likely is this? In fact, there seems to have been very little difference in the maps produced by commissions and by legislatures in terms of the absolute performance of electoral systems, or in the change in their performance after redistricting has taken place.

Read all about it in my "editorial" paper that proposes questions about the redistricting process that commissions should be asking to guide their work this year. Not easy to publish a paper of questions unless you're near or past retirement age, let alone when those questions come from null results!

Catcher Spotting Data Now Available

2011-01-03T18:22:20Z

Thanks to all those who took part in my trial of Catcher Spotting utilities. As promised, I've posted the data for all those who want to work with it.

The paper is posted here and is in submission for the JQAS special issue on the NCSSORS conference.

Joe And Ted's Excellent Accomplishments: Streaks and the Evolving Sport of Baseball

2010-10-18T15:19:51Z

Summarizing my latest sports piece, and first peer-reviewed publication on baseball. Get it for free here.

Even after nearly a century and a half of major league play, baseball still has no shortage of great questions and puzzles, one of them being the phenomenon of the streak. Just how remarkable was Joe DiMaggio's 56-game hitting streak, and Ted Williams's less celebrated 84 consecutive games reaching base? If we could rerun the past 139 seasons of baseball, how likely would we be to see a streak of that length (or longer) again? And how can we trust that the answer we get back is in any way a reliable one, without the use of a time machine?

]]> streak was somehow magical, other lesser streaks would be expected. After all, while it might not be all that likely that Ty Cobb, Paul Molitor and Jimmy Rollins would run up hitting streaks of 40, 39 and 38 games respectively as their career bests, it isn't all that unusual to say that in these hypothetical do-overs, someone would have done as well in each person's place, and that streaks of roughly these lengths would end up as numbers 6, 7 and 8 on the all-time list.

To test these outcomes, I started with a publicly available database on yearly player outcomes, and assumed that each player would have the same number of games and plate appearances in each year. I then produced several plate appearances for each player in each hypothetical game by, essentially, the Strat-O-Matic or "Wheel of Fortune" method - spin a (computer-based) wheel marked with "hit", "walk" and "out", where each wedge on the wheel has as much space as the probability of each event - and noted whether a hit (or a walk) was recorded in the game. (This, of course, assumes that there is no "hot hand" effect that would cause players on a streak to keep performing above their expected ability.) This is done repeatedly to get a large number of histories to compare to the real McCoy.

For hitting streaks, it turns out that this method does a great job for creating those lesser streaks for 1950 and onwards, but overpredicts how long these high-ranked streaks would be from 1900 until 1950, and is even more wildly high for the corresponding streaks in the 19th century. An easy fix for this is to allow the hit probability to vary from day to day in these early eras - on some days the batter has a higher average than normal, on some days a lower average - and by choosing the right variabilities, those lesser streaks in simulation for each era line up with their "real" counterparts. This produces a highly educated guess about how rarely we would see a 56-game streak or more: a little less than 5% of the time since 1901, a streak of the same length as DiMaggio's would be observed. In my eyes, that's certainly a remarkable record, whatever factors led to it.

But even this easy tweak opens up a lot of questions. If this extra fluctuation is enough to make the model look like the real world, what real-world factors would produce it? One possibility is that games pre-1940 were called due to darkness more often; another is that while on the road, players would encounter many different types of ballparks; most likely of all, the quality of opposing pitching was far more variable than today, and that a few good (or lucky) pitchers were far better at stopping streaks than others.

If this explanation is true, then we have a new issue to contend with: on the aggregate, the past 60 years of streaks needed no extra variability in day-to-day hitting to produce a streak list that was comparable to reality. Does that mean that the modern use of relief pitching has created a game where a player's opposing pitching is virtually indistinguishable from day to day? Or, more extremely, could it mean that on the whole, any two given major league pitchers are effectively indistinguishable in stopping hits from occurring? That's one philosophy of baseball research, first suggested by Robert "Voros" McCracken a little over a decade ago and still a hot topic for debate, but it also has a taste of Stephen Jay Gould, who pointed out that the variability between hitters has been decreasing over time. While Gould suggested that this would mean the .400 hitter was a thing of the past, the implication here is that this lower variability would encourage longer hitting streaks: without pitchers that are dramatically different from their counterparts, a player on a streak would be less likely to be shut down by an opposing ace today than 100 years ago.

This method isn't nearly as successful for figuring out on-base streaks; in fact, the same machinery that was just used for hits alone grossly overestimates how long history's on-base streaks would last, no matter how much reasonable difference in ability we estimate between pitchers. While this certainly implies that the opposing pitchers still vary widely in their ability to prevent bases-on-balls (the flip-side to McCracken's argument), it gives us no shortage of alternative explanations to consider as the season begins, and just as much continuing mystery about this era of the game that we are privileged to witness.