Simpson’s paradox in survey results

Published: 09.09.2026 Updated: 09.11.2026 About 6 minutes to read

A student sorts two sets of survey forms beside a calculator.

Your survey suggests that students are more likely to recommend workshop B. Separate the full-time and part-time students and A comes out ahead in both groups. If you are writing up the survey for a thesis, start by checking how many people each percentage represents. That is enough to reconstruct the result and explain it to your reader.

What changed? A leads in both groups, and B leads once you combine their responses. This reversal is called Simpson’s paradox. Each percentage can be correct because the overall figures reflect the different mix of students who attended each workshop.

Count how many people would recommend each workshop

Imagine a thesis about library workshops at a university in Poland. In this fictional study, each student attended one of two formats, A or B. Afterwards, everyone reported their mode of study and said whether they would recommend the workshop. For any group, divide the number of “yes” responses by the total number of responses in that group.

Full-time students. Nine of 10 would recommend A (90%), and 32 of 40 would recommend B (80%). A leads by 10 percentage points.
Part-time students. Twenty-four of 40 would recommend A (60%), and five of 10 would recommend B (50%). A again leads by 10 percentage points.
All participants. For A, there are 9 + 24 “yes” responses, or 33 out of 50 (66%). For B, there are 32 + 5, or 37 out of 50 (74%). Overall, B leads by 8 percentage points.

Look at who filled in the survey. In both formats, full-time students were more likely to answer “yes”. They accounted for 20% of participants in A and 80% in B. So A’s overall figure is closer to 60%, the rate for its larger part-time group. B’s is closer to 80%, the rate for its larger full-time group. Smoluk and Szmigiel discuss this role of group sizes in their Polish-language chapter on Simpson’s paradox.

A tempting shortcut produces a different number. The average of 90% and 60% is 75%, which is what A would get if it had equal numbers of full-time and part-time students. The survey groups had 10 and 40 people, so add the “yes” counts, add the group sizes and divide 33 by 50.

Recreate the comparison in your own spreadsheet

Check who you are counting. In this example, one row represents one person and the three columns you need are workshop format, mode of study and response. Filter for one format and one mode of study. Count all valid responses and then the “yes” responses. Repeat for the other groups. Check for duplicates and make sure A and B used the same question and counting rules.
Record how you handle missing values. Mark a blank response separately. Someone who answered the recommendation question but left out their mode of study can still appear in the overall summary. Also calculate an overall figure using only the people in the subgroup comparison. Report how many responses were left out and why.
Explain why you split the sample. Mode of study was established before the workshop and may be related to scheduling and perceived accessibility. Check that each subgroup contains both formats. In a group of 10 people, changing one answer moves the result by 10 percentage points. Say whether you planned the split or decided on it while reviewing the responses.

Choose which result to present first

If your question is “Which workshop were its participants more likely to recommend?”, start with 66% and 74%. If you want to compare students with the same mode of study, start with the two pairs of subgroup percentages. When the comparisons point in different directions, show both and make clear who is in each denominator.

The study by Bickel and colleagues also shows why grouping matters. When they looked at graduate admissions within individual units at Berkeley, they found that women had applied more often to units with lower admission rates. Use the way your own study was organised to justify any split. For example, satisfaction measured after the workshop may itself depend on the format, so grouping by satisfaction has a different interpretation from grouping by mode of study.

In the workshop example, students were not assigned to formats at random. The differences might also reflect who signed up, when the sessions took place and who led them. Splitting by mode of study lets you compare responses within two more similar groups. To estimate the effect of the format itself, you would need a study design that can separate it from those other factors.

Example wording for your results chapter

In the sample, 33 of 50 students (66%) would recommend workshop A and 37 of 50 (74%) would recommend workshop B. Among full-time students, the recommendation rate was higher for A (90%, 9/10) than for B (80%, 32/40). The same pattern held among part-time students (60%, 24/40 for A and 50%, 5/10 for B). The higher overall recommendation rate for B reflected its larger share of full-time students (80% compared with 20% in A). The comparison describes what the participants reported.

In the methods section, explain how participants ended up in each format and how you handled missing values. Report group sizes in the results and consider the limitations of group selection in the discussion. If you correct a denominator, update the charts and summaries that use the same sample. For more on working with the spreadsheet, see “How to analyse survey results for a thesis: Excel and examples”.

Read through the write-up before submitting

Check that both formats use responses to the same question, from the same period, with the same rules for missing values.
Report the number of “yes” responses and all valid responses for each percentage, especially in small groups.
Explain which group accounts for a larger share of each format and what that does to the overall figure.
Identify what describes the sample and what would require a separate study of the effect of format.

Questions about presenting and interpreting results

Is a description needed for every chart?

The result should be discussed in the text, but the commentary does not need to repeat every value. Highlight the key finding and its connection to the research question. Avoid including a series of charts without explaining why they are shown.

How can I connect a limitation to its effect on the conclusions?

Identify a specific feature of the procedure, the finding or claim it affects, and the consequence. Then explain what you actually did to address it and what remains uncertain. A generic list mentioning sample size and time pressure is not enough.

References

Simpson, E. H. (1951). The Interpretation of Interaction in Contingency Tables. Journal of the Royal Statistical Society: Series B (Methodological), 13(2), 238-241. DOI 10.1111/j.2517-6161.1951.tb00088.x.
Bickel, P. J., Hammel, E. A., & O’Connell, J. W. (1975). Sex Bias in Graduate Admissions: Data from Berkeley. Science, 187(4175), 398-404. DOI 10.1126/science.187.4175.398.
Smoluk, A., & Szmigiel, C. (2022). Objaśnienie i uogólnienie paradoksu Simpsona. W A. Smoluk (red.), Ianua scientiae (s. 18-22). Wydawnictwo Uniwersytetu Ekonomicznego we Wrocławiu. DOI tomu 10.15611/2022.07.7.

Let’s discuss your comparison

Tell us which groups you are comparing and where you are in the process. We can check the calculations, help edit the results chapter or do both.

Statistical analysis for bachelor’s and master’s theses Analysis and presentation of research results for your thesis

Tell us about your survey results

2026-10-19

How to write survey conclusions for a thesis: worked examples

A survey conclusion answers a research question using the responses actually collected. A useful conclusion states what the study found, whom that finding concerns and what the evidence cannot establish. When interpreting survey results, a conclusion connects a research question to what the responses actually establish. State the finding, identify its numerical basis, and define whom it concerns and what was measured. A percentage supplies evidence; it does not answer every thesis question on its own. The worked example follows one dataset: 80 students rating instructions for booking consultations. Using the same data, we move from single responses and a rating item through missing answers and multiple responses to a complete conclusions paragraph and proportionate recommendations. Every participant, answer and number is fictional. This is a teaching example, not a client project or an actual university study. Reports an observation: 48 of 80 people selected an agreement response. Identify the question, category and respondents involved. Explain what the result means for the research question: positive ratings of the instructions’ clarity predominated in this group. That remains a self-report, not a test of booking performance. Relates the findings to other research, possible explanations and limitations. Attaching an author’s name to a similar percentage is insufficient: compare questions, samples and measurement. Closes the thesis aim and brings together its main findings. It may contain conclusions, but is not automatically a copy of the last results subsection. Follow the structure required for your own programme. The 2024 Faculty of Health Sciences guidance from the Medical University of Lublin distinguishes results, discussion and conclusions arising from the study. It illustrates distinct functions; it is not a universal chapter plan for every degree. This guide focuses on moving from questionnaire responses to a defensible conclusion. For comparing your findings with published research, use the separate discussion guide. If you are closing the whole thesis, check the guide to its final chapter. The fictional project has a specific aim: find out how students on one programme rate online consultation-booking instructions and what difficulties they report. The link was shared in student groups. Participants were not randomly...

Read article

2026-11-04

Statistical analysis for a psychology master’s thesis - what to do with collected data

From collected data to clear results: scale scoring, missing values, two worked analyses and effect reporting for a psychology master’s thesis. You have collected responses for your psychology master’s thesis, but the spreadsheet still does not look like a results chapter? The next step is to check what the columns mean, calculate scores using the instrument’s instructions and connect each analysis to a specific question. Only then can the numbers become an answer you can explain clearly in your thesis. Perhaps you have a form export, a statistical software file and a message from your supervisor asking you to prepare the results. Do not start by running one test after another. First reconstruct the route from a person’s responses to the score you intend to analyse. This guide covers the stage after data collection: checking the material, working through two different analyses and writing the chapter, rather than designing another questionnaire. My name is Wioletta Boćkowska and I have run StudentSupport since 2018. In the subject area , I help connect questions, instruments, calculations and writing. If you already have data, you can request within an agreed scope, from preparing the dataset to calculations and reporting. I agree that scope after reviewing the material, before work begins. Keep an unchanged source export and work on a copy. Separate the responses, variable dictionary and decision record. In the dictionary, explain each column, its allowed values, unit and missing-value code. A code of 9 might be a valid response to one question and a missing-value marker in another. The software cannot recognise that distinction for you. Check the unit of observation. If one person completed the questionnaire twice, two rows do not mean two independent participants. You need a way to match the measurements and an analysis that accounts for that structure. Responses from people in the same class or team may also require attention to their shared context. Column names alone cannot establish independence. Your Polish university’s requirements and agreements with your supervisor still matter. The UKW Faculty of Psychology’s 2025 guide distinguishes the method, analysis of results and discussion. This...