How to compare two groups that differed at the start of a study

"I'm comparing two groups for my psychology dissertation. Everyone completed the same scale before and after a workshop. The workshop group had a higher score to begin with. Should I compare final scores, calculate changes, or include baseline in a model?"

First step. Write down how participants ended up in each group. Then make one row per person with their participant code, group, baseline score and follow-up score. For each group, record how many people started, how many have a complete pair of scores, and the mean at each time point for the same people. Use that summary to choose a comparison and decide what you can conclude.

To describe scores after the workshop, start with the follow-up means. To describe change, subtract each person's baseline from their follow-up score and compare mean changes. To compare follow-up at a shared baseline, use a regression model with group and baseline score. This is called analysis of covariance, or ANCOVA. In a randomised study where baseline helps predict follow-up, it is a common choice for the primary analysis. If participants signed up for the workshop themselves, the same model gives a comparison adjusted for baseline. What you can conclude depends on how the groups were formed, how well the outcome was measured and what else differed between groups.

What to establish before you calculate a difference

Record who decided which group each person joined

In the methods section, explain what determined group allocation. "After the baseline measurement, participants were randomly allocated to one of two groups" fits a study that used random allocation. Say what was randomised, when and how you recorded it. "Participants signed up for a session that suited them" describes a choice linked to availability and possibly to motivation or the severity of their difficulties. "One seminar group attended the workshop and the other received reading materials" describes groups that existed before the study.

Random sampling and random allocation to a condition are separate decisions. You can recruit volunteers from one degree programme and then randomly allocate some of them to the workshop. That helps you compare conditions among the participants you recruited. To generalise to other degree programmes or universities in Poland, explain how well your participants and study conditions represent that wider population. You could also randomly select people to take part in a survey and then let them choose whether to attend a workshop. In that case, any claim about what the workshop did still faces the problem of non-random allocation.

Record when the first measurement took place in relation to sign-up and the sessions. If one group received instructions for the exercises before the first questionnaire, its baseline may already reflect part of the intervention. If participants knew their allocation before answering, their expectations may have shaped their responses. Record the dates of recruitment, measurement, allocation and the first session so the reader can reconstruct the sequence.

Check that the two scores mean the same thing

The baseline score, or pretest, comes first. The follow-up score, or posttest, is measured after a defined period. Scale points must mean the same thing at both times. Check that both measurements use the same scale or versions with evidence of comparability, the same scoring rules and the same direction. If higher scores indicate greater tension, a lower score means less reported tension. A move from 22 to 19 gives -3 points when change is defined as "follow-up minus baseline".

Open the scoring instructions. Check whether the same items were reverse-scored and the same rules applied to missing responses at both times. One file may contain a sum of the items while another contains their mean, despite similar column names. Check what the questionnaire asked about as well. Asking about tension "right now" and asking about it "over the past week" can produce numbers in the same range that refer to different periods. Start by deciding whether both scores can reasonably be treated as measures of the same outcome.

Check how many people contribute to each comparison

A pair consists of two observations from the same person. Use verified participant codes and retain one score per person per time point under your study's rules. Count complete pairs, baseline-only scores and follow-up-only scores separately. Record the primary analysis sample size. If a code appears twice, find out why. Details such as a participant restarting an interrupted questionnaire can affect which response you retain.

The pre-workshop mean among people who completed the workshop and answered twice may differ from the mean among everyone recruited. If you subtract the baseline mean from one subset from the follow-up mean of another, your comparison also reflects a change in who is included. The mean of individual changes equals the difference between the baseline and follow-up means when both means use exactly the same people with equal weights.

To check why the methods give different numbers, first run all three on the same complete pairs. In a real study, who enters the primary analysis depends on the design and the missing data. An analysis of all available follow-up scores may include more people than a simple change-score analysis or a basic ANCOVA, which use both values. Report any difference in sample size so the reader can see whether the results differ because of the calculation, the participants included, or both.

Three comparisons you can make in a spreadsheet and a model

Follow-up scores show where the groups ended up

Start by calculating the mean follow-up score for each group. Subtract the mean for the comparison group from the mean for the workshop group. The result is a difference in scale points at a specified time. It is useful when your question concerns the average level of tension at the end of the sessions. You can say which group reported more tension and how many points separated the means.

When groups start at different levels, the follow-up result may carry forward a difference that existed before the study. A group with higher baseline tension can show a clear decline and still end up with a higher mean than the other group. A group with a lower follow-up mean may have ended there partly because its baseline mean was already lower. To describe where the groups finished, you need the scores from that time point. To explain how they got there, you need the first measurement and information about what happened in between.

To draw an inference from follow-up scores alone in two independent groups, choose a method suited to the variability and design. Scores from people in the same workshop group can resemble one another because of shared experiences. Labelling two conditions in a spreadsheet does not make each participant a fully independent replicate. A test of follow-up scores answers a question about follow-up. To compare observed changes, you need the change scores or a model that represents both measurements appropriately.

Mean change describes what happened among people with paired scores

Add a "change" column and enter the follow-up score minus the baseline score for each person. Use the same order throughout. On the tension scale, a negative value indicates a decline and a positive value indicates a rise. Report the mean change in each group and subtract the two means in the agreed order, such as workshop minus comparison. If the workshop group has a mean change of -3 and the comparison group has a mean change of 0, the difference in mean change is -3 points.

Running a separate before-and-after test for each group answers two questions about change within groups. One test can produce a small p-value and the other a large one even when mean changes are similar, if the groups have different sample sizes or variability. To find out whether changes differ between groups, estimate that difference directly and quantify its uncertainty. With two independent groups, you can compare participants' change scores across groups or use the relevant contrast from a model that includes both time points. Pairing here refers to the two scores from the same person.

Simple subtraction gives baseline a particular role. When you compare change, you subtract the entire baseline difference from the follow-up difference. In the example below, the baseline difference is 4 points and the follow-up difference is 1 point. The difference in change is therefore 1 – 4, or -3 points. The definition of change gives you the arithmetic. A claim about the workshop requires knowing how the group means would have moved without it.

Analysis of covariance compares predicted scores at a shared baseline

In a basic ANCOVA, follow-up score is the outcome. Group and baseline score are the predictors used to estimate it. Baseline score is the covariate, an additional variable taken into account in the comparison. With a common slope, one coefficient describes the baseline-to-follow-up relationship in both groups. Picture two parallel lines, one at a different height for each group.

Code the workshop group as 1 and the comparison group as 0. You can also subtract a reference value, such as 20 points, from the baseline score. With that coding, the intercept is the predicted mean in the comparison group for someone starting at 20 points. The group coefficient is the estimated difference between the predicted means, workshop minus comparison, at a common baseline. With parallel lines, that difference is constant across baseline scores. In practice, interpret it within the range where you have data from both groups.

The baseline coefficient tells you how much the model's predicted follow-up score changes for a one-point difference at baseline within the same group. In the example the coefficient is 0.5. Two people in the same group who start 2 points apart therefore have predicted follow-up scores 1 point apart. In your own study you will estimate this coefficient from the data. It helps you break down the observed follow-up difference into the part associated with the groups' baseline scores and the remaining difference the model predicts at a common starting level.

A psychology example. A workshop before a presentation

Imagine a psychology project about the tension people feel before giving a presentation. Group A attends a short workshop and group B receives materials to read on their own. Each person completes the same scale before either condition begins and again one week later. For this example, the scale runs from 0 to 40, with higher scores indicating greater reported tension. The numbers below are made up so you can practise the calculations. The scale, participants and sessions are all part of the fictional example.

Enter the data under "code, group, baseline, follow-up". The numbers in brackets give each participant's baseline and follow-up scores in that order.

• Group A (workshop). A1 (18, 18), A2 (20, 16), A3 (22, 21), A4 (24, 18), A5 (26, 22).
• Group B (materials). B1 (14, 15), B2 (16, 19), B3 (18, 16), B4 (20, 21), B5 (22, 19).

Each group has five complete pairs. Every between-group difference is calculated as A minus B. A positive follow-up difference means greater tension in A. A negative difference in change means that A's scores fell further or rose less than B's.

Step 1. Compare the two time points in the same groups

The mean baseline score in A is (18 + 20 + 22 + 24 + 26) / 5 = 22. In B it is (14 + 16 + 18 + 20 + 22) / 5 = 18. The baseline difference, A minus B, is 4 points.

The mean follow-up score in A is (18 + 16 + 21 + 18 + 22) / 5 = 19. In B it is (15 + 19 + 16 + 21 + 19) / 5 = 18. The follow-up difference, A minus B, is 1 point. After one week, the workshop group in this example averaged one point higher than the group that received reading materials. The group means also show that the gap narrowed from 4 points to 1 point.

Step 2. Calculate the change in each row

For A1, the change is 18 – 18 = 0. For A2, it is 16 – 20 = -4. Do this for every pair.

• Group A. 0, -4, -1, -6, -4. The sum is -15 and the mean is -15 / 5 = -3 points.
• Group B. +1, +3, -2, +1, -3. The sum is 0 and the mean is 0 / 5 = 0 points.

Subtract B's mean change from A's to get -3 – 0 = -3 points. In these example data, A's mean score fell by 3 points more than B's. The zero for B describes the group mean. B2 reported a rise of 3 points and B5 a fall of 3 points. These two changes cancel each other out, even though they go in opposite directions.

You can check the arithmetic using the means. The calculation (19 – 22) – (18 – 18) gives -3. You also get -3 by subtracting the baseline difference of 4 from the follow-up difference of 1. If your study has missing data, first check that both means cover the same people. With different sets of people, this calculation no longer compares the means of individual change scores.

Step 3. Estimate the difference at a shared baseline

For this exercise, use a linear model with a separate intercept for each group and a common slope. The baseline scores overlap from 18 to 22 points. A baseline score of 20 occurs in both groups, so use it as the reference value. With all ten observations included, the baseline coefficient is 0.5.

You can reproduce the calculation by hand. Within each group, subtract its own baseline mean from each baseline score and its own follow-up mean from each follow-up score. Both groups have the same sequence of baseline deviations, from -4 and -2 through 0 to +2 and +4. Their squared values add up to 40 in A and 40 in B.

• In A, the deviations from the follow-up mean of 19 are -1, -3, +2, -1 and +3. Multiply each by the corresponding baseline deviation and add the products. The sum is 4 + 6 + 0 – 2 + 12 = 20.
• In B, the deviations from the follow-up mean of 18 are -3, +1, -2, +3 and +1. The sum of the corresponding products is 12 – 2 + 0 + 6 + 4 = 20.

The slope is the sum of the products divided by the sum of the squared baseline deviations. These data give (20 + 20) / (40 + 40) = 0.5. You use each group's own means because you are estimating the relationship between baseline and follow-up within groups. If you calculated that relationship for the pooled data without a group indicator, the gap between the group means would also influence the result.

At a baseline of 20, A is 2 points below its own baseline mean. Its predicted follow-up mean is 19 + 0.5 × (20 – 22) = 18. For B, a baseline of 20 is 2 points above its own baseline mean, so its predicted follow-up mean is 18 + 0.5 × (20 – 18) = 19. The adjusted difference, A minus B, is 18 – 19 = -1 point. The calculation 1 – 0.5 × 4 = -1 gives the same result. The model subtracts 0.5 points from the follow-up difference for each point of baseline difference, a total of 2 points.

Step 4. Interpret the results in scale points

• +1 point at follow-up. In the observed sample, A has a higher mean follow-up score than B.
• -3 points in the difference in change. In the observed sample, the mean for A fell by 3 points more than the mean for B.
• -1 point in the model with baseline. At the same baseline score of 20, the model predicts a follow-up score 1 point lower for A than for B.

The coefficient of 0.5 applies to these data and this model. In other data it may be closer to 0 or 1, it may be above 1, or a common slope may fit poorly. In the basic model using the same people, comparing follow-up alone amounts to giving the baseline difference a weight of 0 in the adjustment. Comparing mean change gives it a weight of 1. ANCOVA estimates the weight from the data. When the baseline means are equal in the sample, the baseline difference is 0 and all three point estimates coincide in this setup. Their standard errors can still differ because the methods use different information to estimate uncertainty.

With this small dataset you can check each step. At five people per group, one unusual score can shift a mean or a fitted line. This exercise lets you describe the calculations. Evaluating how well the workshop worked requires a study design, a measure of uncertainty and an assessment of what the difference means on the scale. The illustrative one-point difference has no predetermined practical meaning.

Why group allocation matters for interpretation

Randomisation means the groups can differ at baseline by chance

First, suppose all ten people signed up for the same study, completed the baseline measure and were randomly allocated to A or B. One allocation can give a mean of 22 in A and 18 in B. If you repeatedly allocated the same participants at random to two equally sized groups, the chance difference would vary across allocations. The expected baseline difference is zero in this design. Any one allocation can produce a gap of 4 points or another value.

If baseline helps predict follow-up, including it in a prespecified model can improve the precision of the estimated difference between conditions (Vickers and Altman, 2001). It can reduce the variability left unexplained after accounting for baseline. In a randomised study, choose the primary method before seeing the follow-up results, taking into account the allocation procedure, endpoint and important predictors. Choosing whichever comparison produces the smallest p-value makes that p-value and its confidence interval harder to interpret.

A significance test for the baseline difference cannot tell you how participants were allocated. A small p-value can occur by chance in a properly randomised study. A large p-value in self-selected groups may reflect a small sample, wide variation or similar means despite differences in the distributions. Record group sizes, means and variability at baseline. If the difference surprises you, check the group codes, measurement sequence and whether the randomisation procedure was followed. Base the planned analysis on the question and the design.

Consider what happens after allocation. If some participants allocated to A miss the workshop, grouping people by attendance can lose the benefit of randomisation. Decide whether you are asking about the effect of being offered the workshop or the effect of attending it. The first question generally calls for analysis by original allocation, with appropriate handling of missing data. The second may require additional methods and assumptions, since people who attend can differ from those who drop out. Report how many people were allocated, how many took part and how many provided a follow-up score.

If participants chose their group, consider what else differed

In the second scenario, the same numbers come from a study where people signed up for the workshop or chose the reading materials. People feeling more tense may have been more likely to choose the workshop. They may also have practised more, sought additional support or had a presentation scheduled closer to the first questionnaire. A model with baseline accounts for the score in your spreadsheet. The remaining differences may still be related to what you observe after a week.

The same issue can arise when A and B are existing seminar groups. One group may meet before the presentations and the other afterwards. They may have different lecturers, class sizes or proportions of students in each year of study. If each condition is represented by only one seminar group, the condition and the group are intertwined. Adding students to those two groups does not create additional independent replications of the condition allocation. Describe the observed comparison and the circumstances of each class. To evaluate the intervention at group level, the study design needs a way to separate the condition from the characteristics of the individual groups.

Allocation based on a baseline score deserves separate consideration. A participant might be referred to the workshop once their tension score passes a threshold. In some designs, accounting for the variable used for allocation is essential to a suitable model. Under a strict threshold, people with the same baseline score may have access to only one condition. A standard model comparison then relies heavily on assumptions about what the fitted line does outside the overlapping data. Record the allocation rule, inspect the plot and restrict the conclusion to what the design supports. Estimating an effect under threshold-based allocation may require a specific design and analysis around the threshold, with assumptions of its own.

Consider how the groups would have changed under comparable conditions

In a non-randomised study, a simple difference in change can describe how the groups moved between measurements. To interpret it as an effect of the workshop, you would need to explain why the groups would have changed by the same amount on average over that period without it. A single measurement before the sessions tells you where the groups started. It cannot show what their previous trends were. Even similar trends before the sessions are only part of the justification, because something may happen later that affects one of the groups.

With a baseline-adjusted model, ask whether people from the two groups with comparable baseline scores would have similar expected follow-up scores under the same condition. If they differ in how much they prepared for the presentation and preparation was not measured, adjustment for baseline tension may leave an important difference. This interpretation also depends on comparable measurements, access to both conditions for comparable participants, and how missing data were handled.

Van Breukelen (2006) and Senn (2006) show how assumptions about allocation and outcomes without the intervention affect the assessment of both methods. In the pre-existing groups Van Breukelen analysed, differences between the populations and the way they were recruited mattered. Senn points out that unequal baseline means alone do not determine whether adjustment produces a biased estimate. In a dissertation with self-selected groups, you can report both calculations as statistical comparisons. Justify any causal claim from the design and assumptions, independently of which number fits the hypothesis better.

How to check whether the model fits your data

Plot the two measurements first

Put the baseline score on the horizontal axis and the follow-up score on the vertical axis. Each point represents one person. Use different colours or symbols for each group. Check where scores overlap and how the relationship looks within each group. The example has shared baseline scores of 18, 20 and 22. If one group's scores range from 10 to 20 and the other's from 25 to 35, a comparison "at the same score" will rely largely on model assumptions. The software can extend a line into an empty range, but that adds no observations.

If the fitted line rises much faster in one group, the difference between predicted scores may depend on baseline. The model can account for this through an interaction between group and baseline score. With an interaction you can estimate the difference and its uncertainty at selected baseline scores in the shared range, such as 18 and 22 points. In a small sample, the interval for the interaction may be very wide. Use what you know about the scale, what you see in the plot and what the analysis is meant to answer. A large p-value for the interaction does not establish that the slopes are equal.

Inspect the residuals, the differences between observed and predicted scores. If the model tends to miss in the same direction at low and high scores, the relationship may be curved. Greater residual variability in one group affects how you calculate uncertainty. If one observation strongly affects the estimate, check where it came from and whether it was scored correctly. Keep a valid unusual score in line with your analysis rules and show how it affects the result if necessary. Independence at the level of allocation and data collection matters as well.

Consider the limits of the scale and regression to the mean

Someone who starts at 39 on a 40-point scale can rise by at most 1 point. Someone starting at 2 can fall by at most 2 points. If many people in one group start near a limit of the scale, the groups have different room to change. Count how many people score near the minimum or maximum at both time points. Use the instrument's documentation to check what a point means in different parts of the scale. A linear model may fit poorly when scores pile up at a boundary or when the scale has only a few ordered categories.

Regression to the mean is a separate phenomenon. An individual's score depends partly on temporary circumstances and measurement error. If you select people with unusually high scores for the workshop, some will get less extreme scores later even if their average level of tension is stable. Bąbel (2008) discusses this risk when participants are selected according to symptom severity. Brzeziński (1989) also points out that a pretest itself can sensitise participants and affect how they respond at later stages of an experiment. In a study of tension before a presentation, the initial questions may draw attention to particular reactions that participants will later rate again.

Check who did not provide a follow-up score

List participants with complete pairs and those known to be missing a follow-up score. Count them in each condition and check whether their baseline scores differ. Suppose most of the people missing from A had high baseline tension. The follow-up mean among those with complete pairs may then describe a subset that differs from everyone assigned to A. Record any known reasons, such as a scheduling clash or withdrawal after the first session. Use zero for a genuinely observed score of zero. A blank cell should preserve the information that the observation is missing.

When more data are missing, the choice of analysis depends on the information available and what you can reasonably assume about missing scores. A repeated-measures model or appropriately implemented multiple imputation can use information that a complete-pair analysis leaves out. Multiple imputation creates several completed versions of the dataset and combines the results while accounting for the uncertainty of the imputed values. When missingness is related to unobserved outcomes, you may need additional assumptions and a sensitivity analysis. If the design only supports a description of complete pairs, state that scope in the methods, results and discussion.

How to report the methods, results and limitations

Methods in the example with random allocation. "After the baseline measurement, participants were randomly allocated to the workshop group or the group receiving reading materials. Tension before a presentation was measured again one week later. The primary analysis modelled follow-up score as a function of allocation and baseline score. The difference was expressed as the predicted score in the workshop group minus the predicted score in the materials group at the same baseline level." In your own methods section, add how you randomised participants and how you handled missing data, where relevant.

Results for the example data. "The analysis included five complete pairs in each group. The mean score in group A fell from 22 to 19 points. In group B it was 18 points at both time points. In the linear model, at a baseline of 20 points, the predicted follow-up scores were 18 points in A and 19 in B. The adjusted difference, A minus B, was -1 point." In a real dissertation, report a measure of variability and a confidence interval for the primary difference. If you report a p-value, attach it to the relevant comparison. A wider interval leaves more uncertainty about the difference. If it covers values with different practical implications, describe that uncertainty.

Limitation when participants choose their condition. "Participants chose their own condition. The model accounted for baseline tension. Previous presentation experience and timing may also have influenced follow-up scores. The analysis did not account for those factors. We interpret the estimate as a comparison adjusted for baseline." Tailor the possible explanations to how you recruited participants and what information you actually collected.

Before submitting the chapter, check that the reference group is consistent across the methods, results and figure caption. State the order of subtraction, the follow-up time and the number of people included in the analysis. With parallel lines, give the common baseline score used to present predicted means. If you included an interaction, state the baseline values to which each reported difference applies. If you also report a comparison of changes, present its estimate and interval alongside the comparison it belongs to. If the methods disagree, revisit the range shared by both groups, allocation, missing data and the assumptions that connect the calculation to your conclusion.

References

• Bąbel, P. (2008). Efekt placebo: fakt czy artefakt? Roczniki Psychologiczne, 11(1), 59-76. Regression to the mean, pp. 66-67.
• Brzeziński, J. (1989). Eksperymentalne badanie zmian – metodologiczne problemy stosowania pretestu zmiennej zależnej. Przegląd Socjologiczny, 37, 113-139.
• Senn, S. (2006). Change from baseline and analysis of covariance revisited. Statistics in Medicine, 25(24), 4334-4344. DOI 10.1002/sim.2682.
• Van Breukelen, G. J. P. (2006). ANCOVA versus change from baseline: more power in randomized studies, more bias in nonrandomized studies. Journal of Clinical Epidemiology, 59(9), 920-925. DOI 10.1016/j.jclinepi.2006.02.007. PMID 16895814.
• Vickers, A. J., and Altman, D. G. (2001). Analysing controlled trials with baseline and follow up measurements. BMJ, 323(7321), 1123-1124. DOI 10.1136/bmj.323.7321.1123. PMCID PMC1121605.