A pre‑analysis plan is a step‑by‑step plan. It sets out how evaluators will conduct a randomised controlled trial (RCT). Evaluators prepare these plans before the trial begins.
Preregistration means registering an RCT in a public registry before starting the trial. The preregistration may also have a pre‑analysis plan. If so, both the preregistration and the analysis plan are time‑stamped when they are added to the registry.
This page provides guidance about preregistration and pre‑analysis plans.
We explain why they are important and also provide a pre‑analysis plan template, and some examples to help you plan your trial.
Background
Preregistration and pre‑analysis plans for RCTs are becoming more common in programs and policy evaluations.
It has been a standard practice to document the research protocols in natural and physical sciences. This is especially true when developing new drugs, treatments, or technologies.
Since 2005, medical research journals have required that clinical trials be registered in a public trial registry. This must be done at or before patient enrolment (International Committee of Medical Journal Editors, Sept 2004). Advocates within the social sciences, including in psychology, international development, and experimental economics, also pushed for preregistration and pre‑analysis plans since the early 2010’s.
Pre‑analysis plans became popular within the social sciences and economics with the launch of the Open Science Framework (OSF) Registry in 2012 and the American Economic Association (AEA) RCT Registry in 2013. These registries provide databases of RCTs in the social sciences, similar to Clinicaltrials.gov and the Australian New Zealand Clinical Trials Registry for medicine.
Both registries have experienced rapid growth in preregistrations over the past decade. For example, over its first decade, the AEA RCT Registry grew to include over 7,400 randomised experiments, ‘with no sign of plateaued growth yet’ (Goyal and Cavanagh, 26 Jan 2024).
Preregistering your RCT
Preregistration is the practice of registering an RCT before it starts. Preregistering an RCT creates a record in a public trial registry. It includes the details of the planned trial, hypotheses, and method of analysis.
Advantages
There are several advantages of preregistering an RCT on a public registry.
Preregistration is a commitment to publish a study, regardless of the results. This can help to combat ‘publication bias’, which occurs when only exciting, positive results are published. This is also known as the ‘file drawer problem’. It is also important to know about 'null’ results because we can learn from studies that do not show an impact, as well as the ones that do.
Preregistration improves the reproducibility and replicability of your trial. It does this by publicly sharing information on the trial’s design, hypotheses, data collection, and analysis.
Preregistration tells other evaluators that you are working on a topic. It allows for collaboration and reduces duplication of effort.
Public registries can track all studies in each field of research, which can help with comprehensive meta‑analysis. This also lets other researchers see gaps in the literature, and future research needs.
Timing
Ideally, evaluators will register a trial before it begins. This gives the greatest assurance that the trial analysis was not inappropriately changed in response to 'inconvenient' results. However, evaluators can preregister a trial at any time before data collection occurs.
You can often use much of the information from your ethics application for the preregistration of your trial. So, you may find it convenient to register your trial soon after getting ethics approval. If needed, you can also make changes to a trial registration after you have submitted it to the registry.
Public registries
Several public registries accept RCT registrations. Each has different eligibility requirements.
The American Economic Association (AEA) RCT Registry is widely used by evaluators in economics and the social sciences. You can preregister any RCT here, and there are no restrictions on the location, or the topic of study. Registration is free, and you do not have to be a member of the association to preregister your trial.
Visit the American Economic Association (AEA) RCT Registry
You also can register your study with the Open Science Framework (OSF) Registry. This is a free and open platform that is designed to support evaluation research and collaboration.
Visit the Open Science Framework (OSF) Registry
You can register any causal inference studies, like RCTs and quasi‑experimental methods, in education and related fields at the Registry of Efficacy and Effectiveness Studies (REES). Registrations are free however you must register using an Inter‑university Consortium for Political and Social Research (ICPSR) Researcher Passport.
Visit the Registry of Efficacy and Effectiveness Studies (REES)
Requirements
What you include in your preregistration can vary depending on the public registry you choose. However, in most cases you will need:
- trial title
- the location or country where you will undertake the trial
- any keywords for your trial
- an abstract
- trial and intervention start and end dates
- primary outcomes
- experimental design
- sample size
- ethics approval.
Preparing a pre-analysis plan for your RCT
A pre‑analysis plan has more detail on a trial’s design and method of analysis than a preregistration alone. Even if you do not publicly preregister your RCT, the Australian Centre for Evaluation still encourages you to prepare a pre‑analysis plan.
Advantages
Pre‑analysis plans have several advantages.
Developing a pre‑analysis plan is just part of good planning! For example, it helps make sure that you think through the data you need for the analysis. It also helps you to assess the options for how to collect this data.
A pre‑analysis plan helps with knowledge management. It is an important reference point if there are any changes in the trial team personnel.
A pre‑analysis plan increases the credibility of the findings. By pre‑specifying the analysis, a pre‑analysis plan can address concerns of ‘cherry picking’ the results (also known as ‘p‑hacking’ or ‘specification searching’).
While preregistration can also help address these concerns, a pre‑analysis plan is best for trials where the evaluator must make many choices in their analysis. For example, your trial’s outcomes could be measured in many ways, or you could analyse treatment effects for many sub‑groups of the population. Exploring these options is still worthwhile but should be clearly labelled as ‘exploratory’, not ‘confirmatory’.
Having a pre‑analysis plan can help address publication bias. It improves the chances that inconclusive or null results are still reported.
Preparation
A pre‑analysis plan is a valuable guide to your evaluation research design. However, setting out all the details can be time consuming, so start planning early.
A pre‑analysis plan should have a time‑stamp or be registered publicly before collecting and seeing the data. Ideally this should happen before the intervention starts. If necessary, you can change your pre‑analysis plan after it has been time‑stamped. Deviations from your pre‑analysis plan are acceptable if they can be justified.
If you preregister your trial, you can submit your pre‑analysis plan at the same time, or you can upload it later. Either way, the pre‑analysis plan will be time‑stamped on the registry. Instead, if you choose not to preregister your trial at a public registry, you can still time‑stamp your pre‑analysis plan by simply emailing the document to a third party.
Requirements
The Evaluation Toolkit includes a pre‑analysis plan template. This template follows the structure outlined below:
- policy context
- trial aim
- interventions
- outcome measures
- population of interest and sample selection
- sample size calculations
- hypotheses
- randomisation
- method of analysis
- trial threats
- interpretation of results
- pre‑analysis plan commitments.
You can vary the level of detail in your pre‑analysis plan. Please use the sections of this template that are relevant to your trial and remove those that are not.
Download the pre-analysis plan template [DOCX 397KB]
If you are in the Australian Government and need help with preparing a pre‑analysis plan for your trial, please contact the Australian Centre for Evaluation at evaluation@treasury.gov.au. We are also available to conduct a peer review of your pre‑analysis plan, if desired.
Examples
You can find pre‑analysis plans on the American Economic Association (AEA) RCT Registry or the Open Science Framework (OSF) Registry.
Below are examples of past pre‑analysis plans for RCTs from the Behavioural Economics Team of the Australian Government (BETA). These examples used a similar template to the one on the Evaluation Toolkit.
Example 1 – Strengthening students’ resilience. This study evaluated the impact of an app on university student completion rates. This was a clustered, multi‑site two‑arm RCT in two Australian universities.
Example 2 – Workplace giving. This study evaluated the impact of different email messages (give now, or give later) on signing up for workplace giving. This was a simple, two‑arm RCT.