The University of Maryland is rewarding faculty members in the department of psychology who perform and disseminate research in accordance with open science practices. In April, the department adopted new guidelines that explicitly codify open science as a core criteria in tenure and promotion review.
The change was several years in the making and championed by Michael Dougherty, chair of the department. “When you think about the goal and purpose of higher education and why we take these positions, it’s because we felt there would be some good that we could impart on the world,” Dougherty said. “The traditional markers of impact are how many times you’ve been cited [in a journal]. That’s not the type of impact that is valuable to the broader society.”
The new policy was necessary, he said, so incentives for advancement reflect the values of scientists and their institutions.
“The land grant institution is really founded on giving back to the community what the community is investing in,” he said. “Making our work as accessible as possible, with as few barriers as possible, has to be a cornerstone component. You can’t conceive impact without access.”
Throughout his career, Dougherty has advocated for leveraging open practices to enhance scientific integrity. He uses the Open Science Framework for documenting and sharing data, and requires his students to use this platform as well. Once named department chair five years ago, Dougherty said he was committed to rewarding work that was made broadly available without barriers, but he recognized it would be a culture change that required time. He started by sharing information with his colleagues and talking about the main issues of transitioning to open science over two to three years.
On the College Park campus, Dougherty assembled a small working group of faculty members (an assistant professor, associate professor, and full professor) to join him in rewriting the review guidelines. The last time they were touched was 2006, so revisions were overdue, he said. Informed by the San Francisco Declaration on Research Assessment (DORA) and other resources, they started fresh — tossing out old criteria and redrafting the policy.
Dougherty co-authored an article in 2019 that underscored the importance of making research evaluation more transparent and in service to the public good. Soon after the piece was published, he connected with the National Academies of Sciences, Engineering and Medicine Roundtable on Aligning Incentives for Open Science where he met like-minded scholars pushing for changes in the incentive structure.
“It gave me some hope,” Dougherty said of the group. “When the Roundtable was launched, we talked about accelerating change. The National Academies brings with it some cachet. If we can leverage that cachet to really do something that’s going to institute change, we have a shot.”
Engaging with the Roundtable and, more recently, the Roundtable’s spin-off Higher Education Leadership Initiative for Open Scholarship (HELIOS) was a chance to push for systemic change, Dougherty said. While there is often talk about a desire to transform the tenure and promotion process, it’s difficult for individual institutions to do alone. Since advancement has traditionally been tied to prestige of journal publication, ushering in a new approach involved many scholars acting together, he said.
It’s difficult for an institution to make a policy change when there is uncertainty whether peers, colleagues and funders outside the institution share the same values. “The institution is still too small of a unit for the changes to be able to cascade into everyone’s behavior,” said Juan Pablo Alperin, associate professor in the publishing program at Simon Fraser University in Canada who has studied the issue.
Between 2017 and 2022, Alperin and the Open Research Funders Group’s Eric McKiernan were among the scholars who conducted a multi-year research project analyzing more than 850 review, promotion, and tenure (RPT) guidelines and 338 surveys, with scholars from 129 research institutions across Canada and the United States. They found that academics perceive others to care more about prestige than they do themselves, suggesting an appetite for a shift in evaluation criteria—if a transformation could be coordinated.
Alperin said the new guidelines at UMD and elsewhere are encouraging and create momentum.
“We learned from our research that these guidelines create a signal–asserting values by departments and institutions that can be effective,” said Alperin. “As more institutions make this explicit in their guidelines, that starts to create the conversations needed for a widespread desire for change.”
The UMD department of psychology document begins with a statement of overarching principles that lay the groundwork for what the new approach was trying to accomplish.
The evaluative criteria includes a commitment to providing equitable access to scholarly articles through open access publications and preprint servers (in accordance with UMD’s Equitable Access policy). The department now places a premium on team science and embraces diverse approaches to scholarship. It acknowledges the difficulty doing work with hard-to-reach populations and the importance of community engaged work and open science. The guidelines also draw explicitly from the Roundtable’s Toolkit for Fostering Open Science Practices in the evaluative criteria used in merit review.
There were multiple faculty meetings that included group editing of the guidelines before they were adopted in the spring of 2022.
“The level of faculty engagement was really healthy,” Dougherty said. “By the time we were finalizing it, people had bought into it. They knew what the right thing to do was. It was not a hard sell.”
For other institutions considering adopting new evaluation guidelines, Dougherty suggests the key is reframing impact as accessibility. The new approach is a way to empower people to do the research they want to do, he said, and translates into scholars feeling their work is meaningful.
Geeta Swamy, Associate Vice President for Research and Vice Dean for Scientific Integrity at Duke University and HELIOS strategic lead, praised the UMD psychology department’s approach as a model that other institutions can replicate. “A core part of the HELIOS collaboration is identifying real-world solutions that can effectively move the needle toward open scholarship, then working to tailor and scale them across scores of institutions. Our Institutional & Departmental Policy Language Working Group is keen to leverage and amplify Maryland’s work.”
Adds Dougherty: “When I think about what it is that we need to do as scientists to really solve the world’s problems, it’s all hands on deck. And in order for that to happen, we have to be able to make our research products, not just the articles, but the data, the analysis, code, everything available.”
Posted in collaboration with HELIOS.