You have test results for 100 pupils in four classes. Two classes used concept maps and two used lists of examples. The spreadsheet shows 50 pupils for each method. Before choosing a test, add the class identifier to each result and ask one question. Who could have received a different method independently of the others? If whole classes were assigned a method, there were four classes involved in the assignment. All 100 results still help you describe what happened in each class. They represent four class-level assignments.
In a thesis in the “Pedagogy” programme, first establish the level at which you measured the outcome, the level at which you assigned the condition and the group your conclusion is meant to describe. In the first fictional example, you will calculate means for pupils and for classes. You will see why ranking the two methods can give different results, how to report the four classes in your results chapter and how the design changes if tasks are assigned to individual pupils within each class.
Identify the units of measurement and assignment and the scope of your conclusion
The unit of measurement tells you who or what an observation belongs to. In a spreadsheet of individual test results, it is the pupil. The unit of assignment is whoever or whatever was assigned a method, material or other condition. It could be a pupil, a class, a teacher or a school. Your conclusion states who the result is meant to describe. You might describe four particular classes or try to estimate the difference you would expect in a wider population. Before choosing a test, write down all three answers in plain language.
How do you tell at what level the method was assigned?
Go back to the timetable or project plan. If all pupils in class A1 were assigned concept maps and all pupils in class B1 were assigned lists of examples, the class was the unit of assignment. If each teacher was to use one method across all their classes, assignment took place at teacher level. If the school leadership chose one method for the entire school, you collect outcomes at pupil level while the choice of method takes place at school level. In that case, the number of classes does not tell you how many independent decisions were made about which method to use.
If you only have the completed spreadsheet, ask the project lead to reconstruct the sequence. “Everyone in class A1 used method A” describes the data. The class might have been assigned as a whole, the pupils might have been randomly assigned individually and happened to get the same method, or the class might have been formed after assignment. These are different designs. Record whether assignment followed a random procedure, a school decision, the timetable or the participants' choices. If you cannot recover a detail, record it as a limitation and base the analysis on what you can confirm.
Why might results from pupils in the same class be related?
Pupils in a class often share a teacher, a timetable, instructions and the conditions in which they sit the test. These shared experiences can make their results more similar. In an analysis, this is called clustering and each class is a cluster. Each pupil is a participant with their own score. They also share some learning and measurement conditions with their classmates. Recognising this dependence affects how you assess the precision of the difference between methods.
Pick a class in your dataset and ask what might have affected all its pupils at once. If one class sat the test just after a break and another near the end of a long day, the timing is a class-level characteristic. Pupils in a class often share the method too. Entering the same label beside 25 pupils links their results to the project plan. Observing more pupils exposed to the same decision leaves the number of independent assignments unchanged.
Does randomly sampling classes mean the method was assigned at random?
The Polish ICCS 2022 report from the Educational Research Institute shows two-stage sampling. Schools were sampled first, followed by classes in schools with more than one. In many schools, one or two classes took part depending on the number of pupils in Year 8 of Polish primary school. The chapter on factors associated with the results also shows a multilevel analysis that separates differences between schools from differences between pupils within schools. It illustrates how to collect and analyse data on pupils in groups. Random selection to participate did not assign a teaching method.
Random sampling and random assignment to a method answer different questions. Sampling determines who enters a study and under what conditions the results may describe a wider population. Assignment concerns comparisons between people or groups given different conditions. Even if four classes at one school are randomly assigned, what you can conclude about other schools depends on how the school was selected and on conditions in those other schools. In the CKE handbook Badając egzaminy, Jakubowski and Pokropek (2009) describe how to distinguish questions and conclusions at school and pupil level in educational research. Use that principle to check whether your data can answer the question you set for your thesis.
Calculate the results from four classes step by step
Suppose four classes at one school spent two weeks practising how to organise information from short texts. Fictional method A is “Concept Map” and fictional method B is “List of Examples”. The school assigned the methods to whole classes according to the timetable, without randomisation. Pupils in every class took the same test, scored from 0 to 20. Results are available for all 100 pupils in the exercise. The class identifiers and figures are fictional. The aim is to see what the results support and how the calculation changes the answer.
Record the class sizes and scores before pooling the classes
- A1, “Concept Map”. 30 pupils. Six pupils received each of the scores 14, 15, 16, 17 and 18. The total is
6 × (14 + 15 + 16 + 17 + 18) = 480, and the mean is480 / 30 = 16. - A2, “Concept Map”. 20 pupils. Four pupils received each of the scores 8, 9, 10, 11 and 12. The total is
4 × (8 + 9 + 10 + 11 + 12) = 200, and the mean is200 / 20 = 10. - B1, “List of Examples”. 20 pupils. Four pupils received each of the scores 13, 14, 15, 16 and 17. The total is
4 × (13 + 14 + 15 + 16 + 17) = 300, and the mean is300 / 20 = 15. - B2, “List of Examples”. 30 pupils. Six pupils received each of the scores 10, 11, 12, 13 and 14. The total is
6 × (10 + 11 + 12 + 13 + 14) = 360, and the mean is360 / 30 = 12.
These four summaries retain more information than two means. Under method A, the class means differ by 6 points. Under method B, they differ by 3 points. Both classes using method B fall between the two classes using method A. Check whether you could still see that pattern if the class labels were removed. If your thesis also uses results from before the lessons, include those next to the means for each class. This example has post-lesson scores only, so it cannot be used to calculate gains in skills.
The mean score per pupil in the classes observed
For the mean pupil score under each method, add up the scores in both classes and divide by the number of pupils in those classes. For A, this gives (480 + 200) / 50 = 13.6. For B, it gives (300 + 360) / 50 = 13.2. By this measure, the mean for A is 0.4 points higher. Each pupil's score gets the same weight, so the higher mean from A1, with 30 pupils, contributes more to A's mean than the lower mean from A2, with 20 pupils. For method B, B2 is the larger class and has the lower mean, which brings down the pooled result.
You could write, “Pupils using concept maps scored a mean of 13.6 points.” Add “50 pupils from two classes” to show how the group was defined. If any score were missing, the number of pupils in the denominator would change. Distinguish how many pupils took part in the project from how many took the test, and state which classes had missing scores.
The mean score per class in the classes observed
If your question concerns the mean class score and each class is weighted equally, calculate the four class means first. Then average them by method. For A, this is (16 + 10) / 2 = 13. For B, it is (15 + 12) / 2 = 13.5. The mean across classes using A is 0.5 points lower. The level of the description and the weights have changed. In the previous calculation, the class with 30 pupils carried the most weight under each method. Here each of the two classes contributes half. The scores are the same. The mean answers a different question.
Before analysing your data, decide whether the pupil mean or class mean is your priority. If a team lead wants to describe whole classes and give each class equal weight, the class mean fits that aim. If you want to describe all participants, the pupil mean fits. You can report both and explain their weights. Because the direction of the difference changes in this exercise, show all four classes. A conclusion based only on whichever mean suits your argument would conceal part of the pattern.
Does turning scores into two categories help you choose a test?
For this exercise, set a descriptive threshold of 14 points before doing the calculation, then count who scored at least that much. This applies to 30 out of 30 pupils in A1, zero out of 20 in A2, 16 out of 20 in B1 and 6 out of 30 in B2. Pooling pupils gives 30 / 50 = 60% for A and 22 / 50 = 44% for B. Averaging the percentages with equal weight per class gives (100% + 0%) / 2 = 50% and (80% + 20%) / 2 = 50%. Turning scores into two categories leaves the assignment structure intact and shows how the pooled result depends on class sizes.
If you are considering a chi-square test, the guide with a worked example using two categories shows how to prepare a standard cross-tabulation and interpret the result when the assumptions hold. With four classes, return to the question of independence first. The counts in a simple cross-tabulation may let the software run the calculation. Treating each pupil as an independent assignment is unsupported by this design. The same applies if you switch to an exact or rank-based test. Changing the test leaves the shared classroom conditions in place.
Check what the number of classes lets you conclude
A mean describes the scores you observed. A test or a confidence interval aims to say something about the uncertainty of a difference under a particular study design. Those steps require different information. You can have every pupil's test score for four classes and still have very little basis for saying what to expect in other classes. When methods are assigned to classes, comparisons between methods depend on contrasts between those classes. Within a class, more scores help estimate the class mean. To learn how classes using the same method differ, you need to observe more classes.
What changes when you have one class per method?
Imagine that only A1 and B2 are left in the exercise. Both have 30 pupils. Their means are 16 and 12 points. You can describe a four-point difference between those classes. Each pupil with a score of 17 or 18 gives you more information about the distribution in A1. Adding pupils to A1 tells you nothing about what another class would score using concept maps. In these data, the class and the method divide up the pupils in exactly the same way. The difference may reflect the method, characteristics of the classes, how the lessons were taught or a combination of these factors. The data cannot distinguish those possibilities.
The US What Works Clearinghouse handbook for education research describes a similar problem when there is a single class, school or teacher in one condition. It illustrates how a characteristic of the unit can coincide with the condition. In a Polish thesis, describe the mechanism in your own words and explain how it applies to your study. “Each method was used in one class, so we reported the difference between those classes” is more precise than a note that says “small sample”. If one teacher taught every class in one condition and none in the other, adding another class taught by that teacher would still leave their teaching associated with only one condition.
What do you gain with two classes per method?
With two classes per condition, you can show how similar the results were within each method. In the example, you can compare means of 16 and 10 with 15 and 12. Ordering the class means does not produce two separate groups by method. You also know that the pooled pupil mean depends on which class was larger. Two classes per condition give you little information about how much classes usually vary. The timetable may also link a method to the time of the lesson or who is in the class. Report all four classes first. In the discussion, explain how the methods were assigned and whether you can distinguish the condition from the characteristics of the classes.
If the four classes were randomly assigned without further restrictions, with two receiving each method, there would be only six possible pairs of classes for method A. The other two classes would receive method B in each assignment. Collecting more results from the same classes would leave the number of possible assignments unchanged. This shows why a comparison based on randomisation has limited resolution with so few units. In our fictional study, the methods were assigned by timetable. Those six random allocations cannot be used as the basis for a test on these data.
Does a measure of similarity within classes give you an answer?
In an analysis of clustered data, you may encounter the intraclass correlation coefficient, or ICC. In a simple model with a class-level component, it describes the proportion of outcome variance attributable to differences between classes. Two pupils in the same class share the same class component. A higher ICC usually means that, for some comparisons, pupils in one class contribute less independent information than their number suggests. The ICC depends on the outcome, how it is measured, the sample and the model. Record what the value you are citing refers to.
When planning a simple study, you can illustrate the idea with equally sized classes and a common ICC. For 25 pupils per class and an assumed ICC of 0.10, a simple design effect is 1 + (25 – 1) × 0.10 = 3.4. This shows how clustering can increase variance compared with an appropriate reference design that assumes independent observations. The calculation relies on its own assumptions, while the four-class example has unequal class sizes and non-random assignment. Do not use 3.4 to adjust those results. An ICC from another study cannot supply additional classes. An ICC estimated from only a few classes can be very imprecise. An estimate near zero leaves the assignment procedure unchanged.
When are multilevel models and class-level analyses useful?
Given an appropriate design and number of classes, you can consider comparing class means, fitting a model that describes pupil-level and class-level data, or using methods that account for clustering when estimating uncertainty. In a class-level analysis, there is one summary per class. If class sizes differ, decide what the weights should represent before you calculate the contrast. A multilevel model uses individual results and also describes variation associated with the class. A larger structure can include another level, such as the school. To choose an approach, establish who received the method and which units shared conditions.
In a simulation study, Leyrat and colleagues (2018) showed that with few groups, test results depend on the method and the assumptions of the study design. They studied trials in which groups were randomly assigned. Their work shows why standard software output should not be assumed to give the right precision when there are few classes. If you have four classes with non-random assignment and no baseline measurement, clear descriptive results fit the question in this example better than a test that treats 100 pupils as independent assignments. With more classes, the choice of model, any adjustment for the small number of classes and the assumptions all depend on the actual design.
If the method is constant within each class, including a separate indicator for each class leaves no separate method effect to estimate. The class indicators already determine which method each pupil received, so the model cannot estimate an additional coefficient for the method. This differs from a design with both methods in each class. A model with class random effects uses model assumptions and between-class comparisons to estimate the difference. With few classes, choosing that model alone does little to resolve a problem with the design.
Consider another design – assigning tasks within each class
Keep the same fictional method names and four classes of 30, 20, 20 and 30 pupils. In this new design, every class received both types of material in the same lesson. Within each class, half the pupils were randomly assigned to “Concept Map” and half to “List of Examples”. The assignment procedure is different, and the class means from the first exercise belong to that earlier design. The totals below are new fictional results from a test with the same range of 0 to 20 points. See how the calculation changes when both methods are used in every class.
Calculate the difference within the same classroom context
- Class 1: 15 pupils per method. The totals are 225 for A and 210 for B. The means are 15 and 14, so A minus B is 1 point.
- Class 2: 10 pupils per method. The totals of 110 and 120 give means of 11 and 12, a difference of minus 1 point.
- Class 3: 10 pupils per method. The totals of 140 and 130 give means of 14 and 13, a difference of 1 point.
- Class 4: 15 pupils per method. The totals of 195 and 180 give means of 13 and 12, a difference of 1 point.
With equal weights for classes, the mean difference is (1 – 1 + 1 + 1) / 4 = 0.5 points. With equal weights for pupils, the mean for A is (225 + 110 + 140 + 195) / 50 = 13.4, and the mean for B is (210 + 120 + 130 + 180) / 50 = 12.8. The difference is 0.6 points. Assigning half the pupils in each class to each method gives the same weight to a given class in both pooled pupil means. You can therefore get the same difference of 0.6 by weighting the within-class differences by class size: (30 × 1 + 20 × (-1) + 20 × 1 + 30 × 1) / 100 = 0.6.
This design lets you compare methods A and B within the same classes. In class 2, you can compare two groups of 10 pupils who shared the classroom context. The analysis can make use of the fact that assignment was carried out separately within each class. If you use a randomisation-based procedure or a model that accounts for classes, check whether it reflects the actual 50-50 split. The exercise gives fictional totals only. To estimate uncertainty from pupil-level data, you would need individual scores or suitable additional statistics for your chosen analysis.
Could pupils in the same class affect one another?
If pupils studied on their own from a worksheet, both materials could be used in the same room. If the method relied on a whole-class discussion, pupils might have heard the explanations or shared materials. In that case, pupils assigned different methods may have had similar experiences. Ask how the lesson actually ran and describe what happened where it matters to the interpretation. A plan that says “pupils work individually” states an assumption. Classroom observations, a record from the teacher or project documentation can show whether the conditions stayed separate in practice.
If every pupil completed both tasks, the structure changes again. Both scores then belong to the same person, and you may want to compare paired results. Which method came first, whether the tasks were equally difficult and whether the first task affected the second all matter. Two rows can describe the same pupil. First identify the pupil, class and task order. Then decide whether the question concerns a difference within a pupil or a comparison between pupils assigned to different tasks.
What can an actual study tell you about how conditions were assigned?
In the evaluation report for ReflectED, Motteram and colleagues (2016) describe randomising classes within schools. In the methods, they explain that one reason was that the programme was intended for whole-class teaching. This shows how the way lessons are delivered determines what can be assigned. In your thesis, a single specific sentence may be enough. “Each class completed one shared task” explains why the condition was assigned to the whole class. “Each pupil received their own material, and the versions were assigned separately within each class” describes the second design. In either case, say whether assignment was random and how it was done.
If you are still planning the study, sketch assignment and measurement on one page. Show which classes could receive each method and who would teach them. Check whether the time of the lesson, the teacher or the school is tied to just one condition from the start. If you have few classes, discuss with your supervisor whether the realistic aim is to describe a few classes, assess feasibility or estimate an effect with a given level of precision. These aims can require different data. Agreeing the aim before assignment helps justify the number of classes and choose a suitable outcome.
Describe the study so readers can trace the conclusion
State the unit in three places in the thesis. In the methods, explain who received the method. In the results, report the numbers of classes and pupils. In the discussion, explain why these data support the conclusion you draw. The CONSORT extension for cluster randomised trials calls for reporting the numbers of clusters and participants in the analysis and explaining how clustering was accounted for. In a student thesis, showing both levels helps readers follow the analysis even if there was no randomisation or if you are describing only the selected classes.
What should you record in the methods and data dictionary?
In the data file or its documentation, record the pupil identifier, class, teaching method and score as separate fields. If the classes come from several schools, add a school identifier. Two schools may both have a class called “8A”. Record when you collected the data, how the condition was assigned and how many classes were actually meant to use each method. If you measured outcomes twice, record whether the pairs of results belong to the same pupils and how you linked them. A spreadsheet may have a thousand rows for multiple tasks. The data dictionary should help you establish how many people, classes and independent assignment decisions those rows represent.
The descriptive comparison included four classes from one school. Two used “Concept Map” and two used “List of Examples”. The method was assigned to the whole class according to the timetable, without randomisation. Test scores from 0 to 20 were recorded for 100 pupils. A mean was calculated for each class. For each method, we also calculated the pooled pupil mean and the mean across classes with equal weight for each class.
This describes the first example. With your own data, report how many classes entered the study and how many were included in each calculation. If one class did not sit the test, the missing scores affect an entire cluster. If individual pupils missed the test, give the number of available scores for each class. If you compare baseline and follow-up measurements, say whether the means concern the same pupils. Keep the original score scale and assign zeros to missing values only if the measurement rules call for it.
How do you report both means without making them look contradictory?
In the two classes using “Concept Map”, the means were 16 and 10 points. In the two classes using “List of Examples”, they were 15 and 12 points. The pooled pupil means were 13.6 and 13.2 points, respectively, with 50 pupils per method. The equally weighted class means were 13.0 and 13.5 points, respectively, with two classes per method. Class sizes differed, so the two summaries weighted class results differently.
With the first figure, readers can see that the mean covers all pupils. With the second, they can see that each class contributes once. Both calculations are correct and answer different questions. If your supervisor asks for one main result, choose it to match your aim. Report the other as additional information if it helps explain the distribution. In this example, the descriptive conclusion could focus on differences between the four classes. You can show class sizes, scores and methods on four study cards so readers can see the whole pattern.
What should you write in the discussion if your aim was to evaluate effectiveness?
The comparison included two classes per method. The methods were assigned by timetable and outcomes were measured only after the lessons. The data do not show whether the classes already differed at the start or allow the difference in scores to be separated from other classroom factors. The means describe the groups studied and show how the summary depends on the weighting of classes. Assessing the effect of the method would require a design that supports a comparison while accounting for differences between classes.
If your study includes a baseline measure, you can show starting differences for each class. Even if you have these results, check whether both groups had comparable measurement procedures, whether the numbers of observations were similar and whether other events could account for the change. If you have no baseline scores, use “post-lesson score”. A statement such as “pupils improved their scores” requires an earlier score for comparison. Naming the reference point helps you frame an answer grounded in the data.
What should you check before submitting the chapter?
- Count pupils, classes, schools and teachers in each condition. State the level at which the method was assigned and whether any unit was assigned to only one condition.
- Compare planned and observed numbers. Check whether you lost individual pupils or entire classes and how this affected the denominators.
- State the unit and weighting for every mean or proportion. Reproduce one figure from the class summaries.
- Check the protocol, assignment rules and order of measurements. Use these details to frame the question, estimate uncertainty and word the conclusions.
- If you are considering a new analysis, record when you chose the method and why. Determine whether you need an additional measurement or more classes to answer the original question.
Questions about class results and statistical analysis
How many people are needed for statistical analysis in a master's thesis?
There is no single number suitable for every thesis. The planned test, the number and size of groups, the expected effect size, and the required power and precision of the analysis are important. A small sample limits inference. It is best to discuss the sample size before starting data collection.
Does every survey or thesis require statistical tests?
No. The scope of the analysis depends on the aim of the work and the research questions. Sometimes counts, percentages, and descriptive statistics are enough. If the study concerns differences or relationships, I check which methods are appropriate for the data and whether the conditions for their application are met.
A thesis based on interpreting regulations, a literature review, qualitative analysis of interviews, or an individual case study may not require testing statistical hypotheses. In qualitative research, the analysis may focus on the meaning of the statements. In that case, the appropriate support may be methodology, material analysis, or text editing. I do not add calculations just to make the thesis look more scientific or advanced.
How does thesis scope differ from study limitations?
Scope defines the deliberately chosen problem, evidence, population and period. Limitations explain how design features or difficulties restrict interpretation. An omitted topic is not automatically a flaw; the important question is whether the conclusions stay within the stated scope.
How can I connect a limitation to its effect on the conclusions?
Identify a specific feature of the procedure, the finding or claim it affects, and the consequence. Then explain what you actually did to address it and what remains uncertain. A generic list mentioning sample size and time pressure is not enough.
How much does statistical analysis for a degree thesis cost?
I will prepare a quote after reviewing the database and research questions. The cost depends on data cleaning, the number and complexity of analyses, the scope of tables and description, and the deadline. I do not price the service based solely on the number of pages in the thesis.
What will I receive after the statistical analysis is completed?
I agree the scope before starting. It may include a prepared dataset, descriptive statistics, test results, tables, charts, a description of the methods and interpretation in relation to the research questions. I agree the format of editable files and any materials from the software when I prepare the quote.
Sources
- Wasilewska, O. (ed.). (2023). Młodzi w demokracji. Wyniki Międzynarodowego Badania Kompetencji Obywatelskich ICCS 2022. Instytut Badań Edukacyjnych. DOI
10.24131/9788367385534. - Jakubowski, M., Pokropek, A. (2009). Badając egzaminy. Podejście ilościowe w badaniach edukacyjnych. Centralna Komisja Egzaminacyjna. ISBN
978-83-7400-265-3.https://www.ptde.org/pluginfile.php/1433/mod_page/content/5/Badaj%C4%85c%20egzaminy.pdf - What Works Clearinghouse (2022). What Works Clearinghouse Procedures and Standards Handbook, Version 5.0. U.S. Department of Education, Institute of Education Sciences.
https://ies.ed.gov/ncee/wwc/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf - Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., CONSORT Group (2012). CONSORT 2010 statement: extension to cluster randomised trials. BMJ, 345, e5661. DOI
10.1136/bmj.e5661. - Leyrat, C., Morgan, K. E., Leurent, B., Kahan, B. C. (2018). Cluster randomized trials with a small number of clusters: which analyses should be used? International Journal of Epidemiology, 47(1), 321-331. DOI
10.1093/ije/dyx169. Corrigendum: DOI10.1093/ije/dyy057. - Motteram, G., Choudry, S., Kalambouka, A., Hutcheson, G., Barton, A. (2016). ReflectED. Evaluation report and executive summary. Education Endowment Foundation.
https://d2tic4wvo1iusb.cloudfront.net/production/documents/projects/EEF_Project_Report_ReflectED.pdf
Would you like to choose an analysis that fits your classroom study?
Share your thesis topic, research questions, how the methods were assigned, the class sizes and an anonymised extract of the data. If you have measurements before and after the lessons or have already run a test, explain how the scores were linked and what units went into the earlier calculation. Indicate whether you need a descriptive comparison, a check of the assumptions or a new analysis.
Contact StudentSupport


