Hi @may20-cohort — thanks to those of you who could make it yesterday! We had a very good session about review and feedback, with some thoughtful discussions about inclusive review processes. Take a look at the recap below.
Questions and Discussion
@poritzj shared his perspective on the general impression of peer review of commercial textbooks versus OER. We should be more vocal and deliberately highlight when our OER has gone through different kinds of review!
- Jonathan also cautioned about whether anonymous review is really completely anonymous — since in the scholarly publishing world, it’s possible to zero in on the identity of authors based on topic, method, keywords, etc. Does this vary with OER?
@mbranson asked about implicit bias and reviewers, which is a concern in the Math field. How can this bias be reduced or removed? Bias can present itself along many lines - race, ethnicity, gender, institution, language, other social/cultural lines. I’ve compiled a list of some of the resources that I’ve come across that explore the issue of bias and peer review (particularly):
Leveraging anonymous review, removing author identifying details from content, training your reviewers, providing etiquette instructions in the review guide proved most common suggestions. Resources may also contain links to other studies on this topic, which you may be interested in further reading.
Chat Transcript and Resources
Most of the discussion took place during the call rather than in the chat. I’ve saved the transcript, and this was what was shared:
Feedback and review is a chance for you to share your book with subject experts and ensure that the content is appropriate, accurate, and adequately covers the material. Collaborators will read through content and provide critical suggestions to improve the resource for its intended audience. The presence of review on your OER signals to a prospective adopter that the work has passed through rigorous quality control, and that its content is suitable for use in the classroom. In so doing, it acts as good advertising for the efficacy of your resource.
We’ve identified different types of review processes that can be carried out on your book:
- Peer review (pre-publication): It is conducted by subject-matter experts and is a common marker that ensures quality of educational content. It can be a prerequisite for adoption or even submission to repositories. We group peer review into anonymous (most traditional, has prestige value to some groups), non-anonymous (lets you credit reviewers and set up a dialogue with authors, and open (most transparent & collaborative, letting others join in).
- Accessibility review: All creators should ideally find an accessibility practitioner to conduct a set of final checks on the books to see what accessibility standards are met, and to identify areas for improvement.
- Classroom review: Think of this as a trial run of your resource in the classroom setting, and an opportunity to include student voices in the publishing process. Feedback can be gathered both from the instructor using the book to teach as well as the students using the book to learn.
- Feedback via project discussion: All resources should contain a pathway for anyone to provide feedback about the book. These openings may even help recruit other collaborators to work on revisions and other post-publication tasks.
- Post-publication review: These can be collected in repositories, journals, from adopters, and are important to show that the book is being used and having an impact. They are also extremely helpful to promote further improvement of the book and more adoptions, especially if posted on a public site.
With each type of review, you should adequately prepare for the process prior to recruiting reviewers. Collaborate with your existing team to confirm what you would like to get out of each type of review. Once you have your reviewers confirmed, managing the process is fairly straightforward. The review process doesn’t end once reviewers hand in their feedback — we suggest you thank reviewers and keep them engaged and enthusiastic about the resource (take a look at the slides for more details). At heart, review is about bringing more hands on deck to invest in your resource and help it grow with their feedback and input.