The theme of DH2020 is “carrefours/intersections,” a place where paths cross. Our conference will explore three paths in particular: First Nations, Native American, and Indigenous Studies; public digital humanities; and the open data movement. As part of our focus on open data and open scholarship, the DH2020 conference will have an open peer-review process.
In the open peer-review workflow, submissions will be visible to assigned reviewers and the program committee. For the first time, though, the names and affiliations of the submitters will be included with the submissions. Likewise, the names of the reviewers will be available to the program committee, others who reviewed the same proposal after they complete their review, and the proposal’s author(s) after the initial reviews are completed. The submissions, reviews, and responses will not be made available to the public. If accepted, revised submission abstracts and author names will be published in the online program; reviewer names will be published as an alphabetical list unassociated with the abstract that they reviewed. The reviews and responses themselves remain confidential.
Open peer-review being adopted for DH2020 has been part of a multi-year experiment. Prior to 2018, the annual DH conference used single-blind peer review, which meant that reviewers could see author’s names and affiliations, but reviewer identities were kept confidential. It was determined by the ADHO Steering Committee in consultation with the Conference Coordinating Committee that other forms of peer-review might be beneficial to the conference. Without exploring each of the options, our community of digital humanists would not be able to chart the best way forward. For DH2018 (Mexico City) and DH2019 (Utrecht), the conference utilized a double-blind review process where neither author nor review names and affiliations were disclosed.
The results of two years of double-blind peer review highlighted challenges including inconsistent anonymization and public-facing scholarship that made anonymization problematic. In 2018 and 2019, the DH conference received a number of submissions that prominently featured author names; others failed to anonymize thoroughly; and yet others attempted to anonymize in ways that only highlighted identities. For instance, some people anonymized self-citations, which then meant the reviewer had only to Google an article title to determine someone’s identity. Many digital humanists know who is working on what projects. Roopika Risam suggests that, among the many challenges facing peer review in digital humanities, the public nature of much of our work is one of them. It also means that even when a submission has been anonymized, it is often not functionally anonymous. Knowing who wrote a paper can invite cronyism; yet even in double-blind peer reviews, it is possible to identify that a friend or colleague wrote a paper.
Additionally both single- and double-blind peer review struggled with the “reviewer 2” problem. Reviewer 2, the well-known meme in academic culture, is notorious for being lazy or downright mean. In practice this ranges from not reading the submission to including ad hominem attacks in their review.
The move to include an open-peer review process for DH2020 is not undertaken lightly. Kathleen Fitzpatrick has repeatedly made the case for the value of open peer review (see here for an example from the Modern Language Association’s Profession journal, and here for a podcast interview with Sara Benson, for instance). Fitzpatrick contends that “open review produces a wide range of comprehensible, if not traditional, markers of authority, which we must learn to read alongside digitally published texts.” Some studies have found that the quality of the reviews goes up because the reviewer knows their name will be attached to their work.
That said, we recognize that open peer review could place scholars in precarious, untenured, or problematic situations in the awkward position of being asked to be critical of a submission by a more established scholar. It can, likewise, bring out the worst parts of subconscious bias that plague academia. DH2020 will face these challenges by working to assign reviews fairly and by encouraging submitters and reviewers to educate themselves on subconscious biases, which can only strengthen our community. Of course, one year of open peer review is no panacea: we must continually assess our biases and privileges. Open peer review, as well as a conference survey that will be launched post DH2020, supports these self-reflections and field assessments.
For DH2020, we have provided guidelines and models for submitters to help strengthen submissions. Our guidelines and models for reviewers, furthermore, will help reviewers create more useful reviews, which will in turn, offer feedback to revise and improve submissions. We will also work, when assigning reviews, to be attentive to known issues around conflicts of rank. As always, authors and reviewers will be able to note conflicts of interest that will nullify a review assignment. Neither nullification of reviewer or author assignment will be made public to the opposite party. We encourage you in your comments to the program committee to note if you have concerns about a specific reviewer(s). We will attempt to accommodate those notations.
Ultimately, no matter what kind of peer review you are undertaking, we encourage people to “Follow the golden rule of reviewing: review others as you would want to be reviewed: that is, fairly and constructively.” We hope that having an open peer-review process for 2020 will lead to a strong slate of presentations, posters, and workshops. We look forward to joining you in Ottawa!
Laura Estill and Jennifer Guiliano
DH2020 Program Committee Chairs