Reporting A Negative Result in Crowd
It’s well known that negative results are rarely if ever published in the sciences and social sciences. It may well be true in the humanities as well, though there tend to be more ways to interpret results creatively in the humanities. As one of my advisers once said of the humanities: “We don’t have findings. We have readings.”
Well, I’m here to report a negative result in an experiment in crowdsourcing for humanities research that I conducted with colleagues at Stanford University and Historypin, a web platform that enables people to share photographs, documents, sounds, and moving images by “pinning” them to an online map.
They say that success has many fathers, and failure is an orphan. I’d say that our failure had many parents, and I’ve come to the conclusion that a fundamental difference between two of those parents stands at the root of our problems. And I hope it is useful to other historians to call this out.
That fundamental difference is between history and heritage. I’m not the first to focus on this difference, of course. David Lowenthal devoted a book, The Heritage Crusade and the Spoils of History (1996), to understanding the distinctions between those two enterprises, which are nevertheless, as he wrote, “inextricably conjoined.” It’s worth revisiting briefly how Lowenthal underscores their dissimilar intents:
“The historian, however blinkered and presentist and self-deceived, seeks to convey a past consensually known, open to inspection and proof, continually revised and eroded as time and hindsight outdate its truths. The heritage fashioner, however historically scrupulous, seeks to design a past that will fix the identity and enhance the well-being of some chosen individual or folk. History cannot be wholly dispassionate, or it will not be felt worth learning or conveying; heritage cannot totally disregard history, or it will seem too incredible to command fealty. But the aims that animate these two enterprises, and their modes of persuasion, are contrary to each other. To avoid confusion and unwarranted censure, it is vital to bear that opposition in mind.”
It turned out that crowdsourcing, as we attempted it in a project called Year of the Bay, may be very well suited for heritage. But history? Not so much.
Some of my colleagues have come to a different conclusion about our experiment. They emphasize the importance of building community and partnerships with museums, libraries, neighborhood groups, and heritage organizations in order for crowdsourcing to succeed. I don’t disagree. But in my view, that’s mostly what Lowenthal called heritage work, not history. What we call “public history” in the United States today often strives to have it both ways, and sometimes succeeds, but it is good to keep Lowenthal’s cautions in mind, especially, I think, when sourcing from the crowd. Let me explain.
The Year of the Bay experiment (yearofthebay.org) was one of three experiments in crowdsourcing for humanities research conducted at Stanford’s Center for Spatial and Textual Analysis (cesta.stanford.edu) with support from the Andrew W. Mellon Foundation over the past few years. The experiment was designed to test crowdsourcing in conjunction with a major public event and widespread media coverage in order to reach and engage a large audience.
My criteria for judging the success of crowdsourcing in our experiment was whether it could produce new material that could contribute to new historical scholarship. As an environmental historian, my research objective was to see whether a large, public history crowdsourcing project could be used to generate a useful new research archive that, in this case, would help us better understand the environmental history of the San Francisco Bay. My hypothesis was that new archival materials contributed by the crowd could enrich and complicate the dominant narrative of the bay’s environmental history, especially by surfacing culturally diverse sources from the Bay Area’s ethnically diverse communities. I also predicted that participants would help generate useful, accurate, and meaningful metadata for archival sources lacking metadata, such as thousands of photos in the San Francisco Public Library.
“Our hook,” as my colleagues Jon Voss, Gabriel Wolfenstein, and Kerri Young have explained elsewhere, “the Year of the Bay, was 2013, a year that brought the America’s Cup races to the San Francisco Bay, along with completion of a new span of the historic San Francisco–Oakland Bay Bridge (one of the largest public works projects in America at this time), the 150th anniversary of the Port of San Francisco, the opening of a new home for the Exploratorium on the Bay in San Francisco, and exhibitions at the California Historical Society and the Oakland Museum of California focused on the bay.”
We executed the experiment just as planned. We got great media coverage, including a regular feature in the Sunday edition of the San Francisco Chronicle. We mounted an exhibition at the California Historical Society, which I curated, that brought thousands of visitors to the museum, where the first thing they saw was a work table with a laptop, scanner, and projector, where they could contribute their own materials and see them immediately online. We reached out to organizations around the Bay Area, and a Historypin organizer visited many of those organizations to help them with the process of contributing to the new archive. And we got good results: some 7,950 items were contributed, along with 2,126 metadata entries.
But none of this helped answer my historical questions, because it turns out the crowd was not interested in helping me answer my questions about diverse cultural views and practices related to the bay. Over the course of the experiment, it became more and more obvious that they were interested in asking their own questions, debating the answers, and telling their own stories. That is fine. We expected and hoped for it. But the result was a large bucket of mostly isolated moments without historical context—such as the two photographs of one participant’s grandmother as a young woman motorcycle racing on the salt flats by the bay.
I expected to find more sources like these showing the wide variety of perceptions and uses of the bay and bay shore. Such sources would have allowed me to test my hypothesis that different cultural perceptions of the environment shaped attitudes and uses and likely persisted throughout the twentieth century and even into the twenty-first century, even while a powerful environmental narrative about saving the bay came to dominate public discourse. Metadata similar to the information that accompanied the motorcycle-racing photographs—such as the grandmother’s name, the name of the motorcycle club, the names of other people in the photograph, the location, and the date—could have enabled me to begin to connect sources, put them in context, and construct a historical narrative.
It is the historian’s job, of course, to construct historical context from sources that don’t always have an immediately apparent connection. After reviewing the thousands of photographs and bits of metadata contributed to Year of the Bay, however, I came to the reluctant conclusion that—the motorcycle-racing grandmother notwithstanding—we had harvested very few new sources and very little new information that could contribute to constructing a more complex and culturally nuanced understanding of the environmental history of the bay. We did, however, harvest a lot of other material.
The crowd contributed metadata about date ranges and locations for some of the thousands of uncataloged photographs contributed by libraries and archives. But the photographs and other metadata that participants contributed on their own were often associated with individual memories and stories. That’s not surprising. Constructing memory is a crucial personal form of heritage, of telling stories about ourselves, that helps us make sense of our lives, our times, and places that matter to us to create a sense of place and belonging—all hallmarks of heritage rather than history.
Neighborhood groups also contributed some notable photographs and discussions about buildings and changes in their neighborhoods. There is a vibrant interest in neighborhood-scale historical changes in San Francisco, which is also true elsewhere. When people look at maps, the first place they often try to find is home. And many people are interested in what was there before. But that history usually leans toward heritage, too, and it is aimed at fashioning an identity and enhancing the well-being of the community, which we saw when residents of the rapidly gentrifying Bernal Hill neighborhood of San Francisco enthusiastically participated in the Year of the Bay project.
Year of the Bay also saw some notable engagement and contributions related to San Francisco’s LGBTQ history, Chinese history, Jewish history, and its history of transportation and buildings. And it may be that historians specializing in those areas will someday find something new and useful in this archive. I hope this is true, though I am mildly skeptical of the prospect.
That is not to say that a concerted effort couldn’t be made to fill an important historical, archival lacuna using some of the crowdsourcing methods we used in Year of the Bay. And the methods we used could, perhaps, be used to collect new materials and information from more narrowly focused communities, as another experiment at Stanford is seeking to find out by testing whether useful historical information about railroads in the American West can be gathered by engaging railroad buffs.
I have come to the reluctant conclusion that for a historian, this kind of intensive “community sourcing” is likely to be very similar to good old-fashioned historical research, in which a historian uncovers and assembles an archive. And it will likely involve quite similar work: visiting archives to search for uncataloged material, reaching out to organizations that and individuals who might have archives of their own, meeting with them to assess the value of the materials and the feasibility of adding them to an organized archive, and perhaps using the material to write a new history. In this kind of scenario, the added work of mounting a crowdsourcing campaign, in my judgment, is beside the point for history.
It’s telling that in our project as well as the railroad project, my colleagues and I shifted from calling what we did “crowdsourcing” to calling it “community sourcing.” In fact, it’s not that different from good old community organizing. It took a full-time community organizer, Kerri Young, working for Historypin—and a proverbial village—to pull off the Year of the Bay experiment. We found that “community sourcing”—reaching out to individuals and organizations, organizing meetings and “pinning” sessions, identifying “history mysteries” and promoting them on social media especially targeted to interested and influential groups—consistently outperformed mainstream media in garnering engagement. We saw large spikes in traffic after every mainstream media mention of Year of the Bay, but Young’s community organizing work consistently generated more contributions to the project.
Year of the Bay, it turned out, had all the hallmarks of an effective public history project, and it showed that crowdsourcing can be a useful tool for heritage organizations. The California Historical Society (disclaimer: I serve on the society’s board of trustees) is currently using Historypin as a digital platform for mapping the archives of organizations celebrating the centenary of the 1915 Panama-Pacific International Exposition in San Francisco and for encouraging people to contribute their own photographs and other sources. But we are using it as a platform for engagement rather than a vehicle for historical research.
As a public history experiment, then, Year of the Bay was a success. But the project was designed as an experiment in using crowdsourcing for historical research. And on that count, it should go down in history as a negative result.
JON CHRISTENSEN is an adjunct assistant professor in the Department of History, Institute of the Environment and Sustainability, and Center for Digital Humanities at the University of California, Los Angeles, and he is a partner in Stamen Design, a data visualization, mapping, and communications firm in San Francisco.