Reverse-scored questionnaire items - how to calculate a score in a psychology thesis

Your spreadsheet has a column labelled “total score”. All the values fall within the allowable range, but you wonder whether the correct items were reverse-scored before the responses were added. Start with one complete row from the original export. Open the scoring instructions for the exact version of the questionnaire used in your study. Mark the items specified in the scoring key, recalculate their points by hand and add up the scores for all required items. Compare each recalculated value and the total with the spreadsheet. For your first check, choose a row where at least one response to be reversed is away from the midpoint.

A reverse-scored item here is a question or statement that the key instructs you to score in the opposite direction to the raw response code. On a scale from 1 to 5, a response of 5 becomes 1, and a response of 1 becomes 5. This aligns the direction in which each item contributes to the total score. If you are writing a master's thesis in the “Psychology” programme, you can run this check before your first analysis or revisit it after receiving feedback from your thesis supervisor. The steps below take you from the scoring key and the export to recalculating the score by hand and revising your results chapter if it relied on the earlier total.

Identify the items whose scoring needs to change

First, find the form the participants saw, the response file and the scoring instructions. Record the version and language of the instrument, the numbers of the items to reverse-score and how the final score is calculated. Check whether you need a sum, a mean or several separate scores. Write down what a higher value means according to the key. A short note such as “higher scores mean more difficulties” gives you a concrete question to ask about any row. Do the points added for this response move the score towards more or fewer difficulties?

Why can’t you use the word “not” to identify reverse-scored items?

Strong agreement with “I found it easy to get back to the task after a break” points to fewer difficulties. Equally strong agreement with “I couldn't concentrate after the break” may point to more difficulties. If a higher score means more difficulties, the first statement would need to be reverse-scored. For an existing instrument, follow the scoring key for the specific version.

If you are developing your own set of items, record the scoring rule when you design the study. If you have already used the items, check the study plan and earlier notes. If scoring was never agreed, consult your thesis supervisor about what the score should mean and how to construct it before making further analytical decisions. Record when you adopted the rule and what justifies combining the responses into one score.

How do you match the key to spreadsheet columns?

If the key says “item 4”, find the fourth question on that scale in the version used in the study, then locate its field in the export. The consent and demographic questions may have appeared before the scale, so item 4 could be in the twelfth column. The form may also have presented items in random order. In that case, use the question ID or verify which wording corresponds to which column. Check whether the file contains raw responses or scores already calculated by the survey platform.

Record the item's identifier in the key, the source column name, the allowable codes and their labels, and the scoring rule in your data dictionary. If you collected responses at several time points, record the wave. Identical headers in two files could refer to the same questions or be names assigned afresh by a different form. If the study design allows you to link someone's responses across waves, use the participant ID and the wave identifier. Sorting changes the order of the rows.

When should you use a formula and when do you need a response map?

For a straightforward reversal on a scale with evenly spaced response codes, use minimum + maximum – response. Take the minimum and maximum from the key and the coding for that specific item. The rule still uses the allowable endpoints if nobody in your sample selected them. For scores from 1 to 5, use 6 – response. For scores from 0 to 4, use 4 – response. In both examples the midpoint stays the same. Checking the midpoint alone will miss whether the values were actually reversed.

You may receive a file in which responses to a scale scored from 0 to 4 appear as words or as codes from 1 to 5. Check both endpoints and at least one middle category, then record the full mapping to the points specified in the instructions. If the form numbers the options starting at 1, the number of the selected option and the points assigned to it are separate pieces of information. Map the values first, then reverse the items identified in the key. If the instructions use specific weights or an item-specific mapping between responses and scores, record it in full.

Recalculate six responses and see what a scoring error changes

This example uses an entirely fictional set of items called “Difficulties Returning to Study After a Break”. It illustrates how to check a spreadsheet. The statements and records are invented. Imagine participants answering questions about resuming independent study after an unplanned break. The scoring key for the exercise assigns 1 to 5 points per response. Here 1 means “strongly disagree”, 2 “somewhat disagree”, 3 “neither agree nor disagree”, 4 “somewhat agree”, and 5 “strongly agree”. A higher total is intended to reflect more self-reported difficulty returning to study.

How does the scoring key work in this example?

1. P1. “I put off getting back to the task I had started.” Score as answered.
2. P2. “I immediately knew where I had left off.” Reverse-score this item.
3. P3. “I started another task before returning to the previous one.” Score as answered.
4. P4. “I found it hard to concentrate again when I returned.” Score as answered.
5. P5. “I easily moved on to the next step in the task after the break.” Reverse-score this item.
6. P6. “I went over the material again to find where I had stopped studying.” Score as answered.

Someone who strongly agrees with P2 or P5 is describing an easy return at that stage. In this exercise, that response contributes few points to the difficulty score. The map for P2 and P5 is 1 → 5, 2 → 4, 3 → 3, 4 → 2, 5 → 1. To obtain a score, use all six responses and reverse P2 and P5 once each. Every item contributes 1 to 5 points. The lowest possible total is 6 and the highest is 30. The exercise uses no cut-off scores. Rows A01 to A04 trace different effects of the same error.

From the raw responses for A01 to a score of 16

Read the values in order from P1 to P6. The export shows 1, 5, 3, 5, 4, 4 for A01. Adding the raw values gives 1 + 5 + 3 + 5 + 4 + 4 = 22. The key specifies 6 - 5 = 1 for P2, where the response was 5. For P5, where the response was 4, use 6 - 4 = 2. P1, P3, P4 and P6 contribute 1, 3, 5 and 4 points respectively. The total after reverse-scoring is 1 + 1 + 3 + 5 + 2 + 4 = 16. This is 6 points lower than the incorrectly calculated score. P2 accounts for a decrease of 4 points and P5 for a decrease of 2.

Compare both figures with the range of 6-30. Both 22 and 16 fall within it. A range check can catch some errors, such as an unexpected 99 in one of the columns. To verify scoring direction, trace how the raw code becomes an item score. Do the same for P4. A raw response of 5 correctly gives 5 points because strong agreement with P4 describes considerable difficulty. Comparing P2 and P4 shows why the same answer, “strongly agree”, earns different points in these two columns.

The same error can change participants' ranking or leave a total unchanged

• A02 has responses of 4, 2, 2, 3, 1, 3. The raw responses add up to 15. After reversing P2 from 2 to 4 and P5 from 1 to 5, the total is 4 + 4 + 2 + 3 + 5 + 3 = 21. The score increases by 6. With the incorrect scoring, A01 has more points than A02 (22 versus 15). Correcting the scores reverses their order (16 versus 21). Across these two rows, the mean of the incorrect totals is 18.5 and the mean of the corrected totals is also 18.5. The identical means hide the change in the order of the individual scores.
• A03 has responses of 1, 4, 3, 5, 2, 4. The raw total is 19. Changing P2 from 4 to 2 and P5 from 2 to 4 gives 1 + 2 + 3 + 5 + 4 + 4 = 19. P2 loses two points and P5 gains two. Checking both components is what establishes that the unchanged total agrees with the key.
• A04 has responses of 2, missing, 4, 3, 2, 1. P5 changes from 2 to 4 and the score for P2 stays missing. Since the exercise requires all six responses, the total score stays blank. Substituting zero for the missing P2 response would give 6 - 0 = 6, which is outside the range for a single item. The resulting total of 2 + 6 + 4 + 3 + 4 + 1 = 20 would look allowable.

Before applying the rule to the full file, you can create two test rows from scratch. In the first, enter 1, 5, 1, 1, 5, 1. After scoring, each item should give one point, for a total of 6. In the second, use 5, 1, 5, 5, 1, 5. Each item should give five points, for a total of 30. Using the wrong minimum or maximum for the reversal will quickly become visible in these extreme values. Then test a row with a missing response. Together the three tests show whether the formula handles both endpoints and can leave a total blank.

Correct the scoring rule across the dataset and make it easy to check again

Work on a copy of the export or build a new version of the file with a command you can run again. In this exercise, keep P2_surowa and P5_surowa for the raw values and create P2_punkty and P5_punkty alongside them for the scored values. Calculate wynik_v2 from the score columns. For the four items scored as answered, use consistent names for the score columns or have the formula point to the four verified source columns. Either way, record where each component comes from. One formula covering all data collected with the same questionnaire version and coding gives you a clear way to check the results after importing the data again.

Identify the response before assigning points

For P2 and P5, first identify any missing values, then calculate 6 – response only for values of 1, 2, 3, 4 or 5. Keep any response marked as missing as a missing value. Flag a code outside the agreed list for review and leave the total blank until you know what the entry means. This matters if the software reads a cell containing 5 as text, for example, or if an invisible space makes a blank field look like a response. Check the column summary to see which values the software recognised as numbers and which it read as text.

The condition for wynik_v2 is separate from the condition for an individual item. First check that each required item has an allowable score. Then add the scores. For A04, you can score P5, but the total also depends on P2. The standard sum function in many spreadsheets skips blank cells. In this example it would give 2 + 4 + 3 + 4 + 1 = 14, the sum of five scored responses. To apply the process to your own file, set up the reversal and follow the documented rule for missing responses for your scale. Here the key for the fictional example requires all six responses.

On a 0-4 scale, 0 can be a valid response and can become 4 after reversal. A blank cell has a different meaning. Treat a separate code such as 99 as missing only after confirming that meaning in the export settings, the form or the data dictionary. If you put all numbers through 4 – response first, 99 will become -95. Without reconstructing how responses were coded, it can be difficult to tell later whether the number came from using the wrong key or from an unhandled code. Record confirmed missing values and unexpected values separately. This lets you investigate a specific problem and preserve missing values where participants gave no response.

How do you tell a range error from double reversal?

Return to the row you checked by hand and see which column supplies the values. Suppose you see 5 in one P2 column for A01 and 1 in another, and the formula for the score subtracts 1 from 6. You get 5, which is the original value. The formula has reversed an already scored column a second time. Using the wrong range leaves a different trace. If you put the example's response of 5 for P2 into 4 – response, you get minus one, which falls outside the scoring range of 1-5. In a spreadsheet, trace the formula back to the source column. In a script, check which variable it uses. A column name ending in R can be ambiguous. It can mean that values still need to be reversed or that reversal has already taken place.

If the form calculated the total automatically, export the item-level responses and compare at least a few of the scores shown by the platform with scores calculated from the manual or instructions. Look for rows with different values on a reverse-scored item. A03 has the same total before and after correct scoring, which makes it easy to miss a problem. Checking A01 and A02 shows whether the scores change in both directions. If all you have left is a total and you have no access to item responses or scores, you generally cannot reconstruct how many points came from the items that needed to be reversed. Look for the original export or a backup of the project. If neither is available, record the limits of the available data and discuss with your supervisor what can be used for further analysis.

What should you check after filling the formula down the sheet?

Check that the rule works across columns and across the full sample. If A01 and A02 are different people with different responses in the source data, but P2 is 1 in every scored row after you fill the formula down, check the cell references. A fixed reference to one cell can score every row using the first person's value. An incorrect join can have a similar effect if the same participant ID appears in both waves and there is no separate field to distinguish them. Check the row count and the number of repeated IDs. If you joined the files, verify that each record received data from the same participant and the correct wave of measurement.

For a reverse-scored column, the count of valid responses coded 1 before scoring should match the count of values scored 5 afterwards. Compare 2 with 4, 3 with 3, and missing values with missing values in the same way. Expect this pattern for P2 and P5. Items scored as answered should keep their values and stay matched to the original records. Matching category counts is an additional check. If someone sorted a column on its own, the counts could still be correct even though responses are now assigned to different IDs. Compare selected records as well and verify that each score comes from the correct response.

PSS-10 as an example of using the scoring key for a specific version

Responses on the Perceived Stress Scale (PSS-10) are scored from 0 to 4. The instructions from the laboratory at Carnegie Mellon University identify four reverse-scored items in the 10-item version, 4, 5, 7 and 8. Reverse each using 4 – response. This gives 0 → 4, 1 → 3, 2 → 2, 3 → 1, 4 → 0. Keep the scores for the other items as they are and add all ten values. With complete responses, possible raw scores range from 0 to 40. A higher score indicates more perceived stress. Item numbering matters because the same CMU instructions specify different items for the four-item and 14-item versions. In the export, follow the numbering for the version you used.

In Poland, Pracownia Testów Psychologicznych PTP describes PSS-10 in Narzędzia pomiaru stresu i radzenia sobie ze stresem by Zygfryd Juczyński and Nina Ogińska-Bulik, who developed the Polish adaptation. The publisher describes a measure of stress perceived during the past month. Domagalska and colleagues (2021) give an example of using this adaptation with secondary school teachers in Poland. Their methods describe the number of items, five response categories, four reverse-scored items and the sum. Base your own account on the instructions for the edition you used. The published study shows which details you can report so that readers understand how you arrived at the score.

Scoring produces the raw score. Interpreting it against the norms for a particular adaptation is a further step that requires the correct manual and a reason to apply the relevant norms. The cited study also describes conversion to sten scores, which are standardised units for expressing a score in relation to norms. The CMU laboratory explains that the scale measures perceived stress and provides no clinical cut-off for diagnosis. For your own analysis, decide whether to use the raw score or another form justified by the research question and the instructions. Check which one your software is actually reading.

The same coding issue can arise here as in the exercise. If the platform uses 1 and 5 as identifiers for the endpoint options, and the key expects 0 and 4 as scores, first recreate the coding that matches the response labels. Then use 4 – response for the four items specified above. To spot this, look at the export settings and compare a chosen response label with its stored value. If you encounter a missing response in your study, check the Polish version's documentation for instructions. The rules for the six-item exercise apply to that example alone.

Decide what you need to check beyond the formula

Imagine returning to the project after a week. The spreadsheet contains two versions of the score, the data file used by your software contains one, and the methods chapter says “the responses were summed”. If the check by hand revealed an error, find where the expected and actual scores first diverged. This could have happened when assigning codes to responses, when referring to an item with the same name, or when calculating the total. Record which sheet, waves or groups the error affects. If some responses are coded differently, check how the raw responses become scores in that subset separately.

What does a negative correlation between an item and the remaining items tell you?

In a reliability analysis, you may see a correlation between an item and the combined score for the other items on the scale. The latter score is calculated without the item you are checking. If the correlation is negative, people who score higher on the other items tend to have a lower value on the item under review. For an item worded in the opposite direction and still in its raw form, this may indicate that it was never reversed. Return to the key and compare responses with scores for at least a few participants. Also check that you selected the scored column for the reliability analysis. The software can calculate correlations using a different column from the one used to create the total score.

If the coding matches the key, investigate the data and how the measure was used. Make sure the correlation was calculated against the remaining items in the same scale or subscale and on the correct records. Check the sample size, variation in responses and distribution across categories. In a small sample, adding further observations can change the correlation. If almost everyone chose the same answer, the item provides little information about differences between people in this sample. Discuss any change to the items that make up the score with your supervisor and give a methodological reason for it. Changing the key to obtain a higher alpha would change how you measure the construct after seeing the data.

Does correct coding resolve problems with item wording?

Reverse-scoring aligns the direction of the scores. How participants read a question is a separate issue. Van Sonderen, Sanderman and Coyne (2013) analysed responses from 700 people to the MFI-20 fatigue questionnaire. In this sample, they found no evidence that reverse-worded items prevented response bias. The authors linked the patterns they saw to possible inattention and confusion. Their findings concern a particular instrument and sample. For your thesis, once you have confirmed that you scored the items as instructed, check the item wording, the form and the way the study was conducted for clues to the response pattern.

If you have feedback from participants, notes from a pilot study or a record of changes to the wording, revisit them. In the exercise, P6 describes going over the material again. Participants could see this as a difficulty or as a deliberate study strategy. While designing the items, you could test this by asking pilot participants to explain what the statement means to them. After data collection, changing the wording in your description of the instrument does not change what participants answered. Use the form they saw. If the ambiguity remains, describe it as a limitation and refer to what you were able to establish.

When should you recalculate reliability?

After correcting the scoring direction, recalculate the reliability statistic you use for the score under analysis, where the data and your analysis plan allow it. If you have several subscales, do this separately for each one whose items or scoring changed. Check how many participants were included and how the software handled missing values. Cronbach's alpha describes one aspect of score consistency under a particular scoring procedure. Check that the key was applied correctly by inspecting the individual scores directly. To justify your interpretation, also consider how the score was constructed and the available evidence of validity. A higher alpha after the change is insufficient to confirm either point.

The reporting standards for quantitative research in psychology by Appelbaum and colleagues (2018) call for reporting reliability estimates for the scores analysed in the study sample where possible. In your thesis, specify which score you assessed and which records were used. If you quote a reliability coefficient from the manual, identify the source and distinguish it from any coefficient estimated from your own data. Rows A01-A04 in the exercise were chosen to check the arithmetic. They let you verify scoring, and their number and selection provide no basis for conclusions about the psychometric properties of the set of items.

How do you correct results if you have already run the analysis?

First find out where you used the incorrect column. You may have used it to calculate descriptive statistics, assign participants to groups, compute differences between measurements or select rows for a model. Check the filename and the variable in the analysis command or project settings. Renaming a column in the spreadsheet may leave the software using data it loaded earlier.

From spotting the spreadsheet error to rerunning the analysis

1. Record the starting point. Keep the original export, the previous analysis file, the source of the scoring key and the draft results. Record which items and files were affected and when you found the error.
2. Recreate the scores from the responses. Build the corrected column from the export. Compare the rows you recalculated by hand and check the item scores, range, missing values and record counts. Before analysis, confirm that the software uses the corrected column.
3. Identify the results affected by the correction. Record the analysis numbers, hypotheses, figure titles and passages that use the score. Include analyses that use it indirectly, such as a comparison of groups defined by the score in your study plan.
4. Rerun the relevant analyses. For a scoring error, follow the plan you agreed for selecting statistical tests. If the correction changes the sample size or reveals a problem with an assumption, document the reason for any change to the analysis and distinguish it from fixing the calculation. For repeated measurements, also recalculate any variable that depends on both measurements if either needed correction.
5. Update the thesis using the new results. Review descriptive statistics, reliability, test results, effect sizes, confidence intervals and sample sizes if you report them in the chapter. Update the figures, abstract, discussion and conclusions accordingly. If your supervisor has already seen the results, include a brief note on what changed and which results were affected.

Whether an association with another variable changes depends on whose scores changed, by how much and how those participants scored on the other variable. Recalculate the association and compare the estimates and sample sizes. If you agree with your supervisor to keep the results from before the correction in your records, mark them as based on incorrect scoring. Base the final report on calculations that follow the instructions.

How do you describe scoring in your methods chapter and record the correction?

Give readers enough information to reproduce the score. Include the name and source of the version, the number of items, how points were assigned, how scores were summed or averaged, the possible range and what a higher score means. Describe the rule for missing responses as you actually applied it. If you use an existing instrument, check what information you can cite from the manual and how to refer to the scoring key without reproducing protected material. The guide to describing a research instrument in your thesis gives a full explanation and a model structure. The paragraph below uses the fictional items from the exercise.

The score for “Difficulties Returning to Study After a Break” was calculated as the sum of the scores for six items. Responses were coded from 1 to 5. Items P2 and P5 were reverse-scored by subtracting the response value from 6. A total score was calculated where all six responses were available. Possible totals ranged from 6 to 30 points, with higher values indicating greater self-reported difficulty returning to study.

You can keep a separate short note with your calculations. For the exercise, it could read “The old column added the raw values of P2 and P5. The corrected column summed the scores after these items were reverse-scored. A01 (22 before correction, 16 after) and A02 (15 before correction, 21 after) were checked by hand, along with A03, where the total stayed at 19 even though both item scores changed. The total for A04 was left blank because P2 was missing.” In a real project, add which analyses you reran, where the files are stored and whether the basis for any conclusions changed. Record the IDs of the rows you audited in your working documentation, following the project's data storage rules.

If you later copy results from the software into a Polish text, check the conventions for writing numbers. In the Polish adaptation of the APA 7 style rules, Skimina, Harasimczuk and Cieciuch (2022) recommend a decimal comma and a leading zero even for values that cannot exceed one, such as correlation coefficients. Follow your own thesis seminar's requirements in the report. Round values as needed for the chapter and keep the original precision in the calculation file. This ensures you can rerun the analysis with the values computed from the data, and anyone reviewing your work can compare the chapter with the software output.

What should you have to hand before submitting the chapter?

• The form used in the study and the source of the scoring instructions. Mark the page that explains how to calculate the score you are analysing.
• A data dictionary that maps form fields to spreadsheet columns. If you switched versions, record the wave in which the wording or codes changed.
• A copy of the raw export and the commands or formulas that reproduce the calculations. Record the filename of the version you imported into the software.
• The results of the manual checks, with row identifiers and a record of where each row was checked. Someone repeating the checks should be able to find the same item values before and after reversal.
• The updated analysis results and a record of which parts of the thesis changed. If sample sizes differ before and after the correction, identify which cases were added or excluded and why.

Sources

• Carnegie Mellon University, Laboratory for the Study of Stress, Immunity, and Disease. PSS Scoring. https://www.cmu.edu/dietrich/psychology/stress-immunity-disease-lab/scales/html/pssscoring.html
• Carnegie Mellon University, Laboratory for the Study of Stress, Immunity, and Disease. Scales. Perceived Stress Scale. https://www.cmu.edu/dietrich/psychology/stress-immunity-disease-lab/scales/
• Juczyński, Z., Ogińska-Bulik, N. (2009). Narzędzia pomiaru stresu i radzenia sobie ze stresem. Pracownia Testów Psychologicznych PTP. https://www.practest.com.pl/sklep/test/NPSR
• Domagalska, J., Rusin, M., Razzaghi, M., Nowak, P. (2021). Personality Type D, Level of Perceived Stress, Insomnia, and Depression Among High School Teachers in Poland. Frontiers in Psychology, 12, 626945. DOI 10.3389/fpsyg.2021.626945.
• van Sonderen, E., Sanderman, R., Coyne, J. C. (2013). Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain. PLOS ONE, 8(7), e68967. DOI 10.1371/journal.pone.0068967. Publisher correction DOI 10.1371/annotation/af78b324-7b44-4f89-b932-e851fe04a8e5.
• Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board Task Force report. American Psychologist, 73(1), 3-25. DOI 10.1037/amp0000191.
• Skimina, E., Harasimczuk, J., Cieciuch, J. (2022). Podstawowe standardy edytorskie naukowych tekstów psychologicznych w języku polskim na podstawie reguł APA 7. Wydawnictwo Liberi Libri. DOI 10.47943/lib.9788363487560.