The role of the reviewer is to assist the Program Committee (PC) in assessing the quality of proposals submitted. Reviews do not wholly determine which proposals will be accepted or rejected. They provide expert information that the PC uses in making its decisions. Reviewers also provide helpful, constructive feedback to authors, which can strengthen the quality and intellectual rigor of the conference.
A good review will suggest concrete ways in which the proposal may be strengthened. This feedback is important whether you are recommending that the submission be rejected or accepted. If the former, it will enable the author to submit a stronger proposal next year (and may encourage and benefit a new member or a young scholar); if the latter, it will result in a stronger paper being presented. In either case, constructive criticism projects collegiality and an interest in others’ work.
Whatever you may think privately of the proposal or project, and whatever you may know (or think you know) about the author, it is essential that you be uncompromisingly professional and courteous in reviewing all submissions. Rudeness of any sort is destructive to the morale of the community and is absolutely unacceptable in a review. Comments which are purely negative should be addressed solely to the program committee.
Representing the conference theme of “open access”, DH2020 will use the open peer review process, which requires submitter and reviewer identities to be disclosed to one another. Reviews themselves will remain confidential to the submitter, reviewers, and the members of the Program Committee. We ask discretion on the part of all involved to ensure that reviewing meets the conference code of conduct.
Reviewers will be able to read comments by other reviewers who have been assigned to assess the same submission. These peer reviews will only be visible after you have submitted your own review. Reviewers are expected to assess each contribution independently, on its own merits, and are asked not to address other reviewers’ comments directly.
Submitters will have the opportunity to provide a written response to reviewer concerns regarding their submission. Reviewers are expected to engage with this additional material following the first round of review and make any changes to their scoring as a result in a timely manner. Additional comments will be welcomed at this time regarding any changes the reviewer might wish to make.
Please note that we encourage all reviewers to be aware of both explicit and implicit bias in the review process. Biases are developed through our background, cultural environment and personal experiences. Importantly, the program committee for DH2020 reminds all reviewers that explicit or implicit bias can negatively shape the reviewing process, the conference program, and the conference as a whole. We reserve the right to reassign a review should we feel that it has been shaped by explicit or implicit bias.
Submissions will be evaluated based on:
- Overall organization and clarity of proposed submission (20%)
- Explicit engagement with relevant scholarship, with references and justifications displaying knowledge of the current state of appropriate fields (30%)
- Thematic relevance to “carrefours/intersection”; Native American, Indigenous, and First Nations Studies; public digital humanities; or the open data movement (10%)
- Clear theoretical, methodological, or pedagogical framework and explicit statement of purpose (20%)
- Applicability, significance, and value of the theoretical, methodological, and/or practical contribution to the digital humanities generally (20%)
Reviewing Submissions by Type
Please review the submission types before completing your reviews. Reviewers are welcome to suggest alternative presentation formats (for instance, considering long paper proposals for short paper presentations, etc).
Posters
Posters are an appropriate venue for project descriptions and updates. Poster proposals should be evaluated based on the clarity of the information being conveyed.
Lightning talks
Lightning talks are an appropriate venue for project descriptions and updates. Lightning talk proposals should clearly focus on a single project, pedagogical method, idea, technology, or problem. It is intended to either solicit feedback from peers or to advertise the release of a new project, dataset, or tool. Reviewers should also assess whether the submission can be accomplished in the time allotted (5 minutes).
Short presentations
Short papers are appropriate for reporting on works in progress, limited scholarly interventions, or for describing a singular tool or project. Reviewers should also assess whether the submission can be accomplished in the time allotted (10 minutes).
Long presentations
Proposals for long papers should deal with substantial completed research, report the development of new methodologies; or present rigorous theoretical, speculative, or critical discussions. Long-paper submissions seek substantive feedback and discussion of the submission’s relationship to other scholarship in the field. Reviewers should also assess whether the submission can be accomplished in the time allotted (15 minutes).
Panels
Panels should focus on a single theme and be inherently coherent in presenting a substantial body of research or a research question. A panel should be conceived as a 90-minute session of four to six speakers. Reviewers are advised to consider whether the constitution of the panel reflects the constitution of the field and/or research topic that is being addressed and ADHO’s expressed commitment to diversity or to explicitly address problems in those areas. If not, suggestions to submitters are welcomed.
Forums
Forums should focus on a single thematic or methodological challenge and be designed to facilitate a conversation at large with the digital humanities community. The forum should be conceived as a 90-minute discussion with at least three facilitators. Reviewers should evaluate how the proposal explicitly addresses attendee contributions. Reviewers are advised to consider whether the constitution of the forum reflects the constitution of the field and/or research topic that is being addressed and ADHO’s expressed commitment to diversity or to explicitly address problems in those areas. If not, suggestions to submitters are welcomed.
Pre-conference workshops and tutorials
Workshops and tutorials should be evaluated based on their syllabus and/or planned workshop materials. Workshops and tutorials may take the form of hands-on experiences, skills development, as well as group discussions around shared readings. Reviewers are encouraged to assess how, based on their CV(s), the proposer(s) are suited to lead the particular session. Reviewers should also assess whether the submission can be accomplished in the time allotted (either 2 hours or 4 hours).
Ten Tips for Conference Reviewing
- Be aware of the Review Criteria
- Offer constructive feedback that gives submitters direct suggestions for improvement
- Use positive language and affirmative statements rather than negative statements (e.g. “This submission could be improved by considering the following:…” rather than “You are missing X.”)
- Include full citations or links to information that you feel would benefit the submission
- Use specific examples when evaluating writing style: feel free to quote from the submission.
- If you disagree with the entire premise of the submission, be considerate of the work completed by the submitter when outlining your rationale. It is not appropriate to disagree without providing evidence to support one’s position.
- If you find a proposal wholly compelling, please still give one or two sentences complimenting its strengths
- Reviewing is a dialogue between the submitter, reviewers, and the program committee. Be aware that you may be asked to revise or update your review with additional information.
- Reviewing is a service to the digital humanities. Submitting and reviewing deserves to be recognized: list your contribution(s) on your CV or share your accomplishments.
- Follow the golden rule of reviewing: review others as you would want to be reviewed: that is, fairly and constructively.