Quantitative Reasoning Indirect Assessment Report

1 Study

1.1 Background

A survey, Indirect Assessment in Quantitative Reasoning, was used to complement the Quantitative Reasoning Rubric for direct assessment.

1.2 Method and Sample

The survey was distributed through Google Forms to assess quantitative reasoning skills and engagement using the “Standard Form.”

The data was collected during the last two weeks of the Spring 2024 semester, resulting in 469 complete student responses.

1.3 Variables

Variable names, full wording of the questions, short labels used in analyses, and the response sets are as follows.

Variable Name	[Short label] Full Wording of the Question and the Response Set
q1_methodology	[Methodology] Throughout this course, how often have you done each of the following?... Describe the methods that you chose to solve a quantitative problem and why they were appropriate. Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q2_calculation	[Calculation]... Carried out calculations, either by hand or using software. Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q3_visual_representation	[Visual Representation]... Created a visual representation of numerical information, such as a graph, table, chart, or diagram. Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q4_interpretation	[Interpretation]... Described numerical information (numbers, graphs, statistics, etc.) by writing in words. Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q5_evaluation	[Evaluation]... Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.) Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q6_coherence_purpose	[Coherence/Purpose]... Constructed a single, coherent argument using several conclusions that were each based on quantitative evidence. Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q7_reasonableness	[Reasonableness]... Decided when a solution or inference was reasonable or not reasonable in the context of a problem. Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q8_used_qr	[QR in World Issues]... Used quantitative reasoning to examine a real-world problem or issue (unemployment, climate change, public health, etc.) Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q9_improve_general	[Improved General QR] Describe how much this course has improved your general quantitative/numerical skills. None (1) / Minor improvement (2) / Moderate improvement (3) / Major improvement (4)
q10_improve_realworld	[Improved Applied QR] Describe how much this course has improved your ability to apply your quantitative/numerical skills to real-world problems. None (1) / Minor improvement (2) / Moderate improvement (3) / Major improvement (4)
q11_confidence	[Confidence] Describe how confident you feel using numbers in your everyday life. None (1) / Minor improvement (2) / Moderate improvement (3) / Major improvement (4)
q12_example_qr_class	[Example class] Please give an example of a quantitative skill that you developed in this course that you feel will be important in the future (such as in a future class, workplace, or daily life). Open-ended question
q13_freq_calculations	[Frequency of Calculations] How frequently do you do calculations in your everyday life? Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q14_freq_graphs_tables	[Frequency of Graph Use] How frequently do you encounter graphs and tables in your everyday life? Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q15_freq_course_situations	[Frequency of Encounters] How frequently in your daily life do you encounter situations like those you have learned about in this course? Never (1) / Infrequently (2) / Sometimes (3) / Frequently (4)
q16_example_qr_life	[Example life] Please give an example of a time when you used "quantitative reasoning" in your daily life. Open-ended question

2 Results

2.1 Quantitative Reasoning Skills: Summary

2.1.1 Table 1. Frequencies of Core Quantitative Reasoning Skills

Core QR Skills - Items	N = 4691
[Methodology] Described the methods used to solve a quantitative problem and why they were appropriate
Never	19 (4.1%)
Infrequently	57 (12%)
Sometimes	202 (43%)
Frequently	188 (40%)
[Calculation] Carried out calculations, either by hand or using software
Never	9 (1.9%)
Infrequently	30 (6.4%)
Sometimes	125 (27%)
Frequently	304 (65%)
[Visual Representation] Created a visual representation of numerical information (graph, table, chart, etc.)
Never	14 (3.0%)
Infrequently	33 (7.0%)
Sometimes	157 (33%)
Frequently	265 (57%)
[Interpretation] Described numerical information (numbers, graphs, statistics, etc.) by writing in words
Never	12 (2.6%)
Infrequently	51 (11%)
Sometimes	151 (32%)
Frequently	252 (54%)
[Evaluation] Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.)
Never	10 (2.1%)
Infrequently	42 (9.0%)
Sometimes	175 (38%)
Frequently	239 (51%)
[Coherence/Purpose] Constructed a single, coherent argument using several conclusions that were each based on quantitative evidence
Never	27 (5.8%)
Infrequently	89 (19%)
Sometimes	190 (41%)
Frequently	159 (34%)
[Reasonableness] Decided when a solution or inference was reasonable or not reasonable in the context of a problem
Never	17 (3.6%)
Infrequently	53 (11%)
Sometimes	219 (47%)
Frequently	179 (38%)
1n (%)

Table 1 presents the frequencies of student engagement in various quantitative activities within the class, measured through seven different questions.

“Carried out calculations, either by hand or using software,” shows that a significant majority of students, 65%, frequently engage in calculations. “Created a visual representation, like a graph, chart, or diagram,” indicates that 56.5% of students frequently create visual representations. “Interpreted numerical information by writing in words,” shows 54.1% of students frequently interpret numerical data in written form.

The last three items received the least “frequently” responses display more advanced quantitative reasoning and analysis.”Described the methods used to solve a quantitative problem and why they were appropriate” shows 40.3% of students frequently describe methodology. “Decided when a solution or inference was reasonable or not reasonable in the context of a problem” indicates that 38.2% of students frequently assess reasonableness. Finally, the item “Constructed a single, coherent argument using several conclusions that were each based on quantitative evidence” has the lowest “frequently” response at 34.2%, which is less common among students.

2.1.2 Figure 1. Frequencies of Core Quantitative Reasoning Skills

2.1.3 Table 2. Descriptive Statistics of Core Quantitative Reasoning Skills

Core QR Skills - Items	N = 4691
[Calculation] Carried out calculations, either by hand or using software	3.55 (0.70)
[Visual Representation] Created a visual representation of numerical information (graph, table, chart, etc.)	3.43 (0.75)
[Interpretation] Described numerical information (numbers, graphs, statistics, etc.) by writing in words	3.38 (0.78)
[Evaluation] Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.)	3.38 (0.74)
[Methodology] Described the methods used to solve a quantitative problem and why they were appropriate	3.20 (0.81)
[Reasonableness] Decided when a solution or inference was reasonable or not reasonable in the context of a problem	3.20 (0.78)
[Coherence/Purpose] Constructed a single, coherent argument using several conclusions that were each based on quantitative evidence	3.03 (0.88)
1Mean (SD)

Table 2 presents the descriptive statistics of core quantitative reasoning skills. The highest mean score is for “Calculation” at 3.55 out of four. “Visual representation” has a mean score of 3.43. “Interpretation” and “Evaluation” both have a mean score of 3.38. Students describe numerical information in written form and reach conclusions based on analysis at similar frequencies.

“Reasonableness” and “Methodology” both score 3.20. Students decide when solutions are reasonable in context and describe methods used to solve quantitative problems at moderate frequencies. “Coherence/Purpose,” which involves constructing coherent arguments using multiple quantitative conclusions, has a mean score of 3.03. Students construct coherent arguments using several conclusions based on quantitative evidence less frequently than other skills.

The standard deviations range from 0.70 to 0.88, with “Coherence/Purpose” showing the highest variability among students (SD = 0.88). In contrast, “Calculation” (SD = 0.70) and “Visual Representation” (SD = 0.75) show more consistent performance across students.

2.1.4 Figure 2. Descriptive Statistics of Quantitative Reasoning Skills

2.2 Quantitative Reasoning Improvement and Confidence: Summary

2.2.1 Table 3. Frequencies of Quantitative Reasoning Improvement and Confidence

QR Improvement and Confidence - Items	N = 4691
[Improved General QR] How much has this course improved your general quantitative/numerical skills?
None	32 (6.8%)
Minor improvement	42 (9.0%)
Moderate improvement	266 (57%)
Major improvement	128 (27%)
[Improved Applied QR] How much has this course improved your ability to apply QR to real-world problems?
None	16 (3.4%)
Minor improvement	92 (20%)
Moderate improvement	208 (44%)
Major improvement	153 (33%)
[Confidence] How confident do you feel using numbers in your everyday life?
Not at all confident	12 (2.6%)
Not confident	27 (5.8%)
Confident	280 (60%)
Very confident	147 (32%)
1n (%)

Table 3 presents the frequencies of student responses regarding course improvement and confidence in quantitative reasoning.

For general quantitative/numerical skills improvement, most students (57%) report moderate improvement, with 27% reporting major improvement. Only 6.8% of students indicate no improvement in their quantitative skills from the course.

Similarly, for applied QR skills improvement, 56% of students report moderate improvement and 27% report major improvement in their ability to apply quantitative reasoning to real-world problems. A small percentage (8.5%) report no improvement in this area.

Regarding confidence in using numbers in everyday life, the majority of students feel confident (60%) or very confident (32%). Only 2.6% of students report feeling not at all confident, with 5.8% feeling not confident in using numbers day-to-day.

2.2.2 Figure 3. Frequencies of Quantitative Reasoning Improvement and Confidence

2.2.3 Table 4. Descriptive Statistics of Quantitative Reasoning Improvement and Confidence

QR Improvement and Confidence - Items	N = 4691
[Confidence] How confident do you feel using numbers in your everyday life?	3.21 (0.66)
[Improved Applied QR] How much has this course improved your ability to apply QR to real-world problems?	3.06 (0.81)
[Improved General QR] How much has this course improved your general quantitative/numerical skills?	3.05 (0.80)
1Mean (SD)

Table 4 presents the descriptive statistics of student improvement and confidence measures. Students report high confidence in using numbers in everyday life (M = 3.21). This score is the highest among the three measures.

Applied quantitative skill improvement shows a mean of 3.06. Students report similar levels of improvement in applying quantitative reasoning to real-world problems, with a mean of 3.05.

2.2.4 Figure 4. Descriptive Statistics of Quantitative Reasoning Improvement and Confidence

Note 1: Alternative of the above table.
Note 2: Different response scales are used: QR Improvement (1-4: None to Major improvement) and Confidence (1-4: Not at all confident to Very confident).

2.3 Quantitative Reasoning Engagement: Summary

2.3.1 Table 5. Frequencies of Quantitative Reasoning Engagement

Frequency of QR Engagement - Items	N = 4691
[QR in World Issues] Used quantitative reasoning to examine a real-world problem or issue
Never	39 (8.3%)
Infrequently	78 (17%)
Sometimes	178 (38%)
Frequently	174 (37%)
[Frequency of Calculations] How frequently do you do calculations in your everyday life?
Never	33 (7.1%)
Infrequently	109 (23%)
Sometimes	197 (42%)
Frequently	127 (27%)
[Frequency of Graph Use] How frequently do you encounter graphs and tables in your everyday life?
Never	78 (17%)
Infrequently	179 (38%)
Sometimes	141 (30%)
Frequently	70 (15%)
[Frequency of Encounters] How frequently in your daily life do you encounter situations like those you have learned about in this course?
Never	80 (17%)
Infrequently	180 (38%)
Sometimes	158 (34%)
Frequently	51 (11%)
1n (%)

Table 5 presents the frequencies of student engagement with quantitative reasoning in various contexts.

For using quantitative reasoning to examine world issues, 37% of students do this frequently. For calculations in everyday life, 27% of students perform these frequently. Graph and table usage shows 15% of students encounter these frequently. Regarding course-related situations in daily life, 11% of students encounter these frequently.

These results show that quantitative reasoning skills are most frequently applied when examining world issues, followed by performing everyday calculations.

2.3.2 Figure 5. Frequencies of Quantitative Reasoning Engagement

2.3.3 Table 6. Descriptive Statistics of Quantitative Reasoning Engagement

Frequency of QR Engagement - Items	N = 4691
[QR in World Issues] Used quantitative reasoning to examine a real-world problem or issue	3.04 (0.93)
[Frequency of Calculations] How frequently do you do calculations in your everyday life?	2.90 (0.88)
[Frequency of Graph Use] How frequently do you encounter graphs and tables in your everyday life?	2.43 (0.94)
[Frequency of Encounters] How frequently in your daily life do you encounter situations like those you have learned about in this course?	2.38 (0.89)
1Mean (SD)

Table 6 presents the descriptive statistics of quantitative reasoning engagement measures.

Students most frequently use quantitative reasoning to examine world issues (M = 3.04). Frequency of calculations in everyday life shows a mean of 2.90. Graph and table usage displays a mean of 2.43.

Frequency of encountering course-related situations in daily life has the lowest mean at 2.38. Students experience situations similar to those taught in the course less frequently than other forms of quantitative engagement.

2.3.4 Figure 6. Descriptive Statistics of Quantitative Reasoning Engagement

2.4 Consolidated Results: Summary

2.4.1 Table 7. Consolidated Frequency Analysis of All Items

All Items	N = 4691
[Methodology] Described the methods used to solve a quantitative problem and why they were appropriate
Never	19 (4.1%)
Infrequently	57 (12%)
Sometimes	202 (43%)
Frequently	188 (40%)
[Calculation] Carried out calculations, either by hand or using software
Never	9 (1.9%)
Infrequently	30 (6.4%)
Sometimes	125 (27%)
Frequently	304 (65%)
[Visual Representation] Created a visual representation of numerical information (graph, table, chart, etc.)
Never	14 (3.0%)
Infrequently	33 (7.0%)
Sometimes	157 (33%)
Frequently	265 (57%)
[Interpretation] Described numerical information (numbers, graphs, statistics, etc.) by writing in words
Never	12 (2.6%)
Infrequently	51 (11%)
Sometimes	151 (32%)
Frequently	252 (54%)
[Evaluation] Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.)
Never	10 (2.1%)
Infrequently	42 (9.0%)
Sometimes	175 (38%)
Frequently	239 (51%)
[Coherence/Purpose] Constructed a single, coherent argument using several conclusions that were each based on quantitative evidence
Never	27 (5.8%)
Infrequently	89 (19%)
Sometimes	190 (41%)
Frequently	159 (34%)
[Reasonableness] Decided when a solution or inference was reasonable or not reasonable in the context of a problem
Never	17 (3.6%)
Infrequently	53 (11%)
Sometimes	219 (47%)
Frequently	179 (38%)
[Improved General QR] How much has this course improved your general quantitative/numerical skills?
None	32 (6.8%)
Minor improvement	42 (9.0%)
Moderate improvement	266 (57%)
Major improvement	128 (27%)
[Improved Applied QR] How much has this course improved your ability to apply QR to real-world problems?
None	16 (3.4%)
Minor improvement	92 (20%)
Moderate improvement	208 (44%)
Major improvement	153 (33%)
[Confidence] How confident do you feel using numbers in your everyday life?
Not at all confident	12 (2.6%)
Not confident	27 (5.8%)
Confident	280 (60%)
Very confident	147 (32%)
[QR in World Issues] Used quantitative reasoning to examine a real-world problem or issue
Never	39 (8.3%)
Infrequently	78 (17%)
Sometimes	178 (38%)
Frequently	174 (37%)
[Frequency of Calculations] How frequently do you do calculations in your everyday life?
Never	33 (7.1%)
Infrequently	109 (23%)
Sometimes	197 (42%)
Frequently	127 (27%)
[Frequency of Graph Use] How frequently do you encounter graphs and tables in your everyday life?
Never	78 (17%)
Infrequently	179 (38%)
Sometimes	141 (30%)
Frequently	70 (15%)
[Frequency of Encounters] How frequently in your daily life do you encounter situations like those you have learned about in this course?
Never	80 (17%)
Infrequently	180 (38%)
Sometimes	158 (34%)
Frequently	51 (11%)
1n (%)

Note: Alternative of the above frequency tables.

2.4.2 Figure 7. Consolidated Frequency Analysis of All Items

Note 1: Alternative of the above frequency graphs.
Note 2: Different response scales are used: QR Core Skills & QR Engagement (1-4: Never to Frequently), QR Improvement (1-4: None to Major improvement), and Confidence (1-4: Not at all confident to Very confident).

2.4.3 Table 8. Consolidated Descriptive Statistics of All Items

All Items	N = 4691
[Calculation] Carried out calculations, either by hand or using software	3.55 (0.70)
[Visual Representation] Created a visual representation of numerical information (graph, table, chart, etc.)	3.43 (0.75)
[Interpretation] Described numerical information (numbers, graphs, statistics, etc.) by writing in words	3.38 (0.78)
[Evaluation] Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.)	3.38 (0.74)
[Confidence] How confident do you feel using numbers in your everyday life?	3.21 (0.66)
[Methodology] Described the methods used to solve a quantitative problem and why they were appropriate	3.20 (0.81)
[Reasonableness] Decided when a solution or inference was reasonable or not reasonable in the context of a problem	3.20 (0.78)
[Improved Applied QR] How much has this course improved your ability to apply QR to real-world problems?	3.06 (0.81)
[Improved General QR] How much has this course improved your general quantitative/numerical skills?	3.05 (0.80)
[QR in World Issues] Used quantitative reasoning to examine a real-world problem or issue	3.04 (0.93)
[Coherence/Purpose] Constructed a single, coherent argument using several conclusions that were each based on quantitative evidence	3.03 (0.88)
[Frequency of Calculations] How frequently do you do calculations in your everyday life?	2.90 (0.88)
[Frequency of Graph Use] How frequently do you encounter graphs and tables in your everyday life?	2.43 (0.94)
[Frequency of Encounters] How frequently in your daily life do you encounter situations like those you have learned about in this course?	2.38 (0.89)
1Mean (SD)

Note: Alternative of the above descriptive statistics tables.

2.4.4 Figure 8. Consolidated Descriptive Statistics of All Items

Note 1: Alternative of the above descriptive statistics graphs.
Note 2: Different response scales are used: Core QR Skills & QR Engagement (1-4: Never to Frequently), QR Improvement (1-4: None to Major improvement), and Confidence (1-4: Not at all confident to Very confident).

3 Correlation Analyses

3.1 Figure 9. Correlation Matrix of All Variables

Note: Bottom-left cells show scatterplots; diagonal cells show variable distributions. Top-right values are Pearson r coefficients. Correlation strength: weak (|r| < 0.3), moderate (0.3 < |r| < 0.5), strong (|r| > 0.5). * p < .05, ** p < .01, *** p < .001. Plot may be simplified, or variables/smoothers removed, due to limited sample size.

Figure 9 presents the correlation matrix of all variables.

The strongest correlations within the core quantitative reasoning skills are observed between:

Coherence/Purpose and Reasonableness (r = 0.68, p<0.001)
Evaluation and Interpretation (r = 0.64, p<0.001)
Interpretation and Visual Representation (r = 0.61, p<0.001)

These strong correlations may suggest that students who build coherent arguments backed by quantitative evidence also tend to judge the reasonableness of solutions. Students with strong evaluation skills typically have strong interpretation abilities, and those who interpret numerical information also create visual representations of data.

The correlation between “How much has this course improved your general quantitative/numerical skills?” and “How much has this course improved your ability to apply QR to real-world problems?” (r = 0.90, p<0.001). Notably, “Confidence in using numbers” shows no significant correlations with any variables, which may suggest that confidence operates somewhat independently from the reported quantitative reasoning skills, improvement, and engagement.

The “Using quantitative reasoning to examine real-world problems” shows moderate correlations with core quantitative reasoning skills, particularly with Methodology (r = 0.55, p<0.001) and Reasonableness (r = 0.51, p<0.001). This suggests relationships between classroom-based skills and their application to real-world issues.

Within the qualitative reasoning engagement variables, the analysis reveals a statistically significant correlation (r = 0.68, p<0.001) between students’ reported frequency of encountering graphs and tables in everyday life and their experience with course-related situations in daily contexts.

The weakest correlations involve “Used quantitative reasoning to examine a real-world problem or issue,” which shows moderate correlations with “How frequently do you do calculations in your everyday life?” (r = 0.24, p<0.001), “How frequently do you encounter graphs and tables in your everyday life?” (r= 0.26, p<0.001), and “How frequently in your daily life do you encounter situations like those you have learned about in this course?” (r = 0.34, p<0.001).

3.2 Figure 10. Correlogram of All Variables

Note 1: Alternative of the above graph.
Note 2: Cells represent the correlation coefficient between two variables. Blue shows positive correlations, red shows negative correlations, and circle size reflects correlation magnitude. Correlation strength: weak (|r| < 0.3), moderate (0.3 < |r| < 0.5), strong (|r| > 0.5). Insignificant correlations are invisible (p > .05).

4 Regression Analyses

4.1 Table 9. Linear Regression Model Predicting Coherence/Purpose

Linear Regression Model Predicting Coherence/Purpose
Predictor	Estimate	SE	95% CI	p-value
Intercept	-0.01	(0.03)	[-0.06, 0.05]	0.837
Methodology	0.21***	(0.05)	[0.11, 0.30]	0.000
Calculation	-0.11†	(0.06)	[-0.22, 0.00]	0.053
Visual Representation	-0.05	(0.05)	[-0.16, 0.05]	0.310
Interpretation	0.07	(0.06)	[-0.04, 0.18]	0.209
Evaluation	0.24***	(0.06)	[0.13, 0.35]	0.000
Reasonableness	0.53***	(0.05)	[0.43, 0.63]	0.000
SE = Standard Error, CI = Confidence Interval † p < .10, * p < .05, p < .01, * p < .001 R² = 0.511; Adjusted R² = 0.505

A linear regression was conducted to identify the factors that influence students’ ability to construct a single, coherent argument using several conclusions based on quantitative evidence—referred to as “Coherence/Purpose.” This item received the lowest “frequently” responses at 34.2%, as noted in prior sections. Note that all variables were mean-centered prior to analyses to minimize multicollinearity.

The results reveal that four skills significantly influence the ability to construct coherent quantitative arguments. Specifically, the skill of describing the methods used to solve a quantitative problem and justifying their appropriateness increases the “Coherence/Purpose” score by 0.21 points on a 1-to-4 scale. Similarly, the ability to reach conclusions based on one’s own analysis of numerical information, such as numbers, graphs, or statistics, increases the score by 0.24 points. The most substantial impact arises from deciding when a solution or inference is reasonable within a problem’s context, which increases the Coherence/Purpose score by 0.53 points. Finally, “Carried out calculations, either by hand or using software” shows a negative effect, decreasing the Coherence/Purpose score by 0.11 points. This effect is significant at a marginal level (p < 0.10), a less reliable but noteworthy influence that warrants further exploration. The model explains approximately 51.1% of the variance in “Coherence/Purpose” (R² = 0.511).

4.2 Table 11. Linear Regression Model Predicting Core Quantitative Reasoning Improvement

Linear Regression Model Predicting QR Improvement
Predictor	Estimate	SE	95% CI	p-value
Intercept	0.01	(0.03)	[-0.04, 0.06]	0.717
QR Core Skills	0.55***	(0.05)	[0.46, 0.65]	0.000
Confidence	-0.03	(0.04)	[-0.10, 0.05]	0.467
QR Engagement	0.29***	(0.04)	[0.21, 0.37]	0.000
SE = Standard Error, CI = Confidence Interval † p < .10, * p < .05, p < .01, * p < .001 R² = 0.430; Adjusted R² = 0.426

A linear regression was conducted to identify the factors that influence students’ self-reported improvement. For this analysis, composite indexes were used.

The results reveal that quantitative reasoning core skills increases the improvement score by 0.55 points on a 1-to-4 scale. Similarly, qualitative reasoning engagement increases the score by 0.29 points. However, confidence has no effect on the improvement score. The model explains approximately 43% of the variance in “Coherence/Purpose” (R² = 0.430).