How openRxiv Labs Operates
How it Works¶
openRxiv Labs will run a select number of large-scale experiments each year with value-aligned partners that focus on priority opportunity areas that we have defined with our community and stakeholders. Partners and experiments will be selected based on criteria below, initially by conversation with the openRxiv team.
Each selected experiment will run for a pre-defined duration and include hypotheses, goals, and success metrics. Experiments will be hosted on the openRxiv Labs website and, where appropriate, linked to from the bioRxiv and medRxiv preprint servers so that existing users can read them. Users can give feedback on experiments, and explore applicable data and code generated by them, in the openRxiv Labs GitHub organization.
Successful experiments may be integrated into the core openRxiv platform. But all experiments will publish results openly on the openRxiv Labs blog so that we and the community can learn from them.
openRxiv Labs itself is an experiment, so elements of Labs are likely to evolve as we start the work and continue to learn more. We’re approaching the work with a spirit of curiosity and collaboration. These experiments are meant to provide us and partners the opportunities to try out ideas and innovations on real-world content, to show, not just tell, what’s possible. We look forward to building openRxiv Labs together with the researchers, institutions, funders, and innovators who share our excitement about the future of preprinting.
Experiment Criteria¶
To ensure openRxiv Labs is as impactful as possible, we evaluate potential experiments against the following criteria:
Mission and value alignment. Experiments must attempt to advance the mission of openRxiv and the broader open science ecosystem in ways that align with our core values.
Collaborative. Experiments must be conducted with at least one values-aligned partner, which could be organizational or institution partners or be funders providing support for work that our team will do with the community.
Leverage open research outputs. Experiments must build on top of outputs that openRxiv provides, such that, if successful, they would add to the “constellation” of research outputs and signals that openRxiv aspires to catalyze the creation of.
High impact potential. Experiments must have the potential to drive transformative change within the ecosystem or meaningfully impact openRxiv’s organization-level goals, often by being integrated into openRxiv’s core offering if successful.
Measurable success and failure criteria. Experiments must be true experiments, in that before being accepted they must establish a measurable hypothesis and include both ways of gathering the data required to test the hypothesis and criteria and a plan for gracefully sunsetting in the case of failure.
Openly publishable. As part of their output, experiments must produce at least one blog post, preprint, dataset, or other openly licensed output, so that openRxiv and the open research community more broadly can learn from it, regardless of the outcome.
Partner Criteria¶
Alignment with partners is a key part of the criteria for evaluating if an experiment is a fit for openRxiv Labs. We are looking for collaborative partners whose sustainability/business models are designed such that their success adds value to the entire open science ecosystem, not just themselves. Specifically, we are looking for partners who meet the following criteria:
Have a meaningful stake in the result. Partners must be willing to dedicate significant time, resources, and brand/name recognition to the experiment such that they are meaningfully invested in its success or failure. Labs experiments should not be mere side projects or small tests for partners, but truly impactful projects.
Demonstrate a core commitment to open science principles and practices. Partners must be interested in advancing open science as part of their work or mission, not just neutral parties whose tools happen to be useful to open science.
Contribute to open-source or other open, community resources. Partners’ Labs projects must result in some set of openly licensed and available outputs — whether code, documentation, data, etc. They should not use Labs to develop or advertise closed or inaccessible resources.
Use fair and transparent business/sustainability models. Partners could be a for-profit or non-profit organization, but they must not sustain themselves by pursuing extractive, abusive, or other non-aligned business models such as gating content, displaying behaviorally targeted advertising, collecting and selling user data, selling closed-source software, etc.
Have the capacity to sustain the experiment through to completion. Some Labs experiments may come with funding, and others may be funded entirely or partially by openRxiv. In all cases, we expect partners to be operationally and financially capable of running the experiment without needing the experiment to result in additional revenue.
Operating Principles¶
To protect the hard-won trust openRxiv has built amongst its community, experiments will be subject to the following principles, which will be encoded in Labs contracts as much as is possible:
Data collection and ownership. Data collected during experiments should be used exclusively to evaluate its success and failure and make iterative improvements. All data collected will be wholly owned by openRxiv, and cannot be sold to third parties or used for behavioral targeting, training AI, or any other purpose other than improving the experiment. Where legal and ethical, we may license anonymized data back to partners so they can use experiments to improve their work, but only if it benefits the experiment and ecosystem as a whole, and does not violate openRxiv’s terms and policies.
Privacy & hosting. Experiments should be accessible exclusively via an openRxiv-owned domain or subdomain, so that they are subject to openRxiv’s privacy policy and terms. They should be hosted on server providers that are compatible with openRxiv’s policies.
Opting in and out. Generally, Labs experiments will involve content that we have been given the explicit permission to modify and experiment with. As a result, experiments will default to applying to all content. However, prior to running an experiment we will evaluate its proposed content use to make sure that it does not violate any explicit or implicit rights or agreements with our users, particularly when it comes to handling user information, automatically notifying users, or altering or processing the content in ways that could be seen as changing its type or purpose. In some cases, we may add the ability for authors to opt in or out from having their content used.
Experiment Submission and Selection Process¶
Partners interested in working with openRxiv Labs can reach out directly by emailing hello@openrxiv.org with a summary of the experiment they wish to run. The openRxiv team will evaluate the experiments based on their alignment with the priority opportunity areas and the criteria below, and invite collaborators whose projects seem like a potential fit to discuss it further with the team. If we decide to move forward, invited collaborators will be asked to submit a proposal, which the team, and key stakeholders, will evaluate in a multi-stage process, culminating in a contractual agreement to run the experiment together.
We view openRxiv Labs itself as an ongoing experiment. As we learn more about the ideal process for openRxiv Labs, we will iterate on, formalize, and publish more details about the selection process to provide potential collaborators, and the community at large, with more concrete pathways for getting involved, as well as lessons learned that others can use to run their own Labs programs.
Interested in Collaborating?¶
We’d love to hear from you. Email us directly at hello@openrxiv.org with:
An overview of you and your organization, including a statement on why you believe you meet the above partner criteria
Details about the experiment you’re interested in running
An overview of why you believe your idea meets the above experiment criteria
The best way to reach you