Before-and-after surveys for a nursing thesis - what can you calculate if responses cannot be linked?

After a class on patient education, you have two survey files for your nursing thesis. One is from before the class and the other from afterwards. If you have no way to link both answers from the same person, calculate the results for each time point separately. You can describe the difference between percentages, but the files cannot tell you whose answer changed. Start by checking who could complete each survey and whether the same code appears in both files.

Check what you can link

• You have two records for the same person. If a code was built into the study, check that it appears only once at each time point and refers to the same person. A pair consists of two results assigned to one participant. With those data, you can see how that person’s answer changed.
• Only some records form verified pairs. Keep verified pairs separate from records from only one time point and records with ambiguous codes. A participant known to have missed the second measurement is different from two forms you cannot match with confidence.
• The forms have no shared code. Row number, submission order and similar scores are insufficient for linking records. Equal numbers do not guarantee that the same people responded. When both surveys drew from the same group, some people may appear in both. Missing codes alone do not justify a test designed for independent groups.

Teaching materials from Kielce University of Technology use a before-and-after measurement as an example of paired data when each person has two recorded values. In a school study, Agley and colleagues tested a way to match responses that had been planned before data collection. Even there, some forms could not be linked with confidence. You cannot apply those rules retrospectively if the information they rely on was never collected.

Calculate the percentages at both time points

In a fictional nursing thesis study, 40 students took part in a workshop on talking with a simulated patient. Before and immediately after the class, students received the same short question. They had to select the wording that asks a patient to explain an instruction in their own words. There were three options. “Is everything clear?”, “Do you have the instructions?” and “Could you tell me in your own words how you understand these instructions?” The last answer was correct at both time points. The class and results are fictional. In a Polish pilot study on communication with simulated patients, Bodys-Cupak and colleagues used questionnaires for the teacher, simulated patient and student, alongside checklists for the procedures performed. Our example counts answers to one question and does not assess performance in the conversation.

All 40 participants answered before the class. Afterwards, only people from that workshop could complete the form, but 30 answered. A person could submit one form at each time point. Every submitted form included a valid answer to the question. No common code was recorded, so it is impossible to identify which 30 people answered after the class.

See what the same totals can conceal

• Nobody who answered after the class changed their answer. All 24 students who had answered correctly could have responded again, along with six of the 16 who had answered incorrectly. Each person gave the same answer as before.
• Six students switched from an incorrect answer to the correct one. Eighteen students who had answered correctly could have responded again, along with 12 who had answered incorrectly. All 18 who had answered correctly did so again. Of the other 12, six chose the correct answer and six repeated their mistake. The result after the class was still 24 out of 30.

Both scenarios fit the same files. Without pairs, you cannot tell which occurred or whether some people switched to an incorrect answer. The difference in percentages may depend on who responded the second time. Even with complete pairs, before-and-after measurement alone does not establish that the class caused the change. That requires an appropriate study design.

Separate methods, results and limitations

Methods. The same question was asked before the workshop and immediately afterwards. The percentage of correct answers was calculated separately among those who responded at each time point. The forms had no code for linking responses from the same person.

Results. Before the workshop, 24 of 40 respondents chose the correct answer (60%). Afterwards, 24 of 30 respondents did so (80%). The percentage was 20 percentage points higher in the second set of responses.

Limitation. The result describes two sets of forms because individual responses could not be linked. The number of people who changed their answer could not be determined. Any change in individual answers could not be separated from a change in who responded.

If you find some verified pairs, report the number of pairs and the number of forms outside them at each time point. You can then describe change among people with two verified answers. Their results may differ from those of the full group. An analysis that combines paired and unpaired records requires a method suited to that structure. Derrick and colleagues investigated this problem for comparisons of two proportions.

Check that the chart caption gives 40 responses before the class and 30 after it. If the study plan called for measuring individual change, explain why, in this case, you can only report descriptive counts and percentages separately for each time point and discuss this with your supervisor. For the broader process, see Nursing master's thesis: from research questions to discussion.

Questions about comparing measurements and testing a hypothesis

Sources

• Nowakowska, M. (n.d.). Testy istotności różnic między parametrami cechy dla dwóch populacji. Politechnika Świętokrzyska. Section on two paired samples.
• Agley, J., Tidd, D., Jun, M., Eldridge, L., Xiao, Y., Sussman, S., Jayawardene, W., Agley, D., Gassman, R., & Dickinson, S. L. (2021). Developing and validating a novel anonymous method for matching longitudinal school-based data. Educational and Psychological Measurement, 81(1), 90-109. DOI 10.1177/0013164420938457.
• Bodys-Cupak, I., Łatka, J., Ziarko, E., Majda, A., & Zalewska-Puchała, J. (2020). Medical simulation with simulated patients in the education of Polish nursing students – pilot study. Pielęgniarstwo Polskie, 77(3), 166-173. DOI 10.20883/pielpol.2020.19.
• Derrick, B., Dobson-McKittrick, A., Toher, D., & White, P. (2015). Test statistics for comparing two proportions with partially overlapping samples. Journal of Applied Quantitative Methods, 10(3), 1-14.