CIVIL ENGINEERING × GENERATIVE AIA MULTI-SOURCE STUDY / 2025–2026
Starting perceptionsPRE surveyi
Where students started
Before the course, respondents rated AI familiarity, potential usefulness, and participation potential. These starting distributions put later changes in context.
Optional AI consultation accompanied students' own reasoning, calculations, and discussion. The session explorer summarizes 1,562 feedback records about use, helpful tasks, and remaining questions.
These submissions were not linked to individual PRE/POST respondents.
Matched PRE/POST responses describe changes in the same respondents' appraisals. The findings distinguish observed change from baseline-conditioned POST comparisons.
Neither comparison establishes a causal effect of GenAI.
Students first explained what civil engineering entails, then compared their explanation with additional explanations from course materials or optional AI consultation. Peer and class discussion supported refinement.
10 weeks · one three-hour session per week.
G2Spring 2026
Construction Project Scheduling
Upper-level / senior-level
Activity, time, and resource reasoning, supported by conventional calculations and scheduling software.
87enrolled students
59matched respondents
3sections · 28 / 29 / 30
Work breakdown structuresActivity relationships & durationsProductivity & resourcesVerification of generated suggestions
Inside an instructional activity
Students developed a residential-townhouse activity list or WBS and examined formwork, reinforcing steel, and concrete work. They compared optional AI suggestions with durations, productivity information, crew assumptions, and their own calculations.
15 weeks · 20 sessions per section. Five weeks of twice-weekly meetings, staggered across the three sections.
01Student reasoning
02Optional GenAI consultation
03Critical evaluation
04Discussion and refinement
Instructional sequence, not a measured individual pathway. Consultation was encouraged, optional, and ungraded; it was not a standardized or randomized treatment.
02THE LONGITUDINAL EVIDENCE
A change in perception. A question of interpretation.
Starting distributions shape the story. Explore the responses before interpreting the differences.
Three indicators. Two comparison questions. No causal claim.
01G1 showed upward mean responses across all three indicators.
02G2 began with higher familiarity; some perceptions were relatively stable.
03Observed-change differences attenuated in baseline-conditioned comparisons.
FIGURE 01 / RESPONSE DISTRIBUTIONS
Where students started. Where they ended.
Identical complete pairs within each item
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G1n = 38 complete pairs
G2n = 58 complete pairs
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
G2n = 59 complete pairs
Perceived participation potential (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G1
PRE
1
1 / 38
2.63%
familiarity
G1
PRE
2
14 / 38
36.84%
familiarity
G1
PRE
3
16 / 38
42.11%
familiarity
G1
PRE
4
7 / 38
18.42%
familiarity
G1
POST
1
1 / 38
2.63%
familiarity
G1
POST
2
0 / 38
0.00%
familiarity
G1
POST
3
13 / 38
34.21%
familiarity
G1
POST
4
24 / 38
63.16%
familiarity
G2
PRE
1
1 / 58
1.72%
familiarity
G2
PRE
2
8 / 58
13.79%
familiarity
G2
PRE
3
25 / 58
43.10%
familiarity
G2
PRE
4
24 / 58
41.38%
familiarity
G2
POST
1
3 / 58
5.17%
familiarity
G2
POST
2
3 / 58
5.17%
familiarity
G2
POST
3
24 / 58
41.38%
familiarity
G2
POST
4
28 / 58
48.28%
usefulness
G1
PRE
1
0 / 38
0.00%
usefulness
G1
PRE
2
0 / 38
0.00%
usefulness
G1
PRE
3
12 / 38
31.58%
usefulness
G1
PRE
4
20 / 38
52.63%
usefulness
G1
PRE
5
6 / 38
15.79%
usefulness
G1
POST
1
0 / 38
0.00%
usefulness
G1
POST
2
0 / 38
0.00%
usefulness
G1
POST
3
4 / 38
10.53%
usefulness
G1
POST
4
22 / 38
57.89%
usefulness
G1
POST
5
12 / 38
31.58%
usefulness
G2
PRE
1
0 / 59
0.00%
usefulness
G2
PRE
2
2 / 59
3.39%
usefulness
G2
PRE
3
12 / 59
20.34%
usefulness
G2
PRE
4
35 / 59
59.32%
usefulness
G2
PRE
5
10 / 59
16.95%
usefulness
G2
POST
1
1 / 59
1.69%
usefulness
G2
POST
2
1 / 59
1.69%
usefulness
G2
POST
3
6 / 59
10.17%
usefulness
G2
POST
4
34 / 59
57.63%
usefulness
G2
POST
5
17 / 59
28.81%
participation
G1
PRE
1
0 / 38
0.00%
participation
G1
PRE
2
0 / 38
0.00%
participation
G1
PRE
3
12 / 38
31.58%
participation
G1
PRE
4
19 / 38
50.00%
participation
G1
PRE
5
7 / 38
18.42%
participation
G1
POST
1
1 / 38
2.63%
participation
G1
POST
2
0 / 38
0.00%
participation
G1
POST
3
2 / 38
5.26%
participation
G1
POST
4
22 / 38
57.89%
participation
G1
POST
5
13 / 38
34.21%
participation
G2
PRE
1
0 / 58
0.00%
participation
G2
PRE
2
0 / 58
0.00%
participation
G2
PRE
3
7 / 58
12.07%
participation
G2
PRE
4
33 / 58
56.90%
participation
G2
PRE
5
18 / 58
31.03%
participation
G2
POST
1
3 / 58
5.17%
participation
G2
POST
2
0 / 58
0.00%
participation
G2
POST
3
7 / 58
12.07%
participation
G2
POST
4
32 / 58
55.17%
participation
G2
POST
5
16 / 58
27.59%
Familiarity measures broad AI familiarity. Usefulness wording differed slightly between G1 PRE and POST; cross-course and temporal measurement equivalence was not established. Participation potential is a belief about potential support, not observed participation.
FIGURE 02 / TWO DIFFERENT QUESTIONS
The comparison depends on the question.
Difference in observed change
How much more did G1’s matched responses move from PRE to POST than G2’s?
Baseline-conditioned POST contrast
How do later responses differ when the model conditions on the measured initial response?
G1 minus G2 · estimates and marginal 95% confidence intervals
● ━ Observed-change difference■ ┄ Baseline-conditioned POST
Solid circles: Welch–Satterthwaite change comparisons. Dashed squares: numeric-PRE model with HC3 standard errors. Zero indicates no estimated contrast. Intervals are exploratory and not section-cluster adjusted. Neither analysis identifies a causal effect.
Read exact comparative estimates
G1 minus G2, rounded to three decimals
Indicator
Comparison
Estimate
95% CI
familiarity
Observed change
0.730
[0.318, 1.141]
familiarity
Conditional POST
0.326
[-0.016, 0.667]
usefulness
Observed change
0.165
[-0.144, 0.474]
usefulness
Conditional POST
0.132
[-0.134, 0.397]
participation
Observed change
0.532
[0.159, 0.905]
participation
Conditional POST
0.328
[-0.006, 0.661]
INDIVIDUAL RESPONSE MOVEMENTS
Averages contain different experiences.
Familiarityn = 38Mean 2.763 3.579
26102
↑ Increased 26= Unchanged 10↓ Decreased 2
Usefulnessn = 38Mean 3.842 4.211
16193
↑ Increased 16= Unchanged 19↓ Decreased 3
Participation potentialn = 38Mean 3.868 4.211
18173
↑ Increased 18= Unchanged 17↓ Decreased 3
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G1: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
4
70
9 / 61 / 0
70
4.343
70
5
67
49 / 18 / 0
67
4.567
67
6
72
66 / 6 / 0
72
4.681
72
8
76
64 / 12 / 0
76
4.711
76
9
74
61 / 12 / 1
74
4.500
74
10
84
73 / 10 / 1
84
4.774
84
THE DISCIPLINARY DETAIL / G1
What did the assistance refer to?
Civil engineering, infrastructure and construction systems
54
Project concepts, lifecycle and management
40
Quality, safety and professional ethics
31
Communication, disciplinary language and study support
21
Software operations and graphical representations
6
Other named academic subject
4
Scheduling relationships, durations and calendars
3
Resources, productivity and costs
3
Work decomposition and construction operations
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
04HOW THE STUDY WAS CONDUCTED
Two streams. Distinct evidence.
A transparent design keeps each observation connected to the claim it can support.
Complementary interpretationAnalytically distinct · no individual linkage between streams
01 / Study design & analytical samples
An observational, two-course, two-wave study with complementary session feedback. One instructor taught all four sections. Courses differed in semester, curriculum, composition, and schedule; there was no random assignment or non-AI comparison. The matched sample was 38 in G1 and 59 in G2. Complete pairs were 38/58 for familiarity, 38/59 for usefulness, and 38/58 for participation.
02 / Questionnaires & measurement wording
Three distinct single ordered items measured AI familiarity (1–4), potential usefulness (1–5), and participation potential (1–5). Familiarity referred to AI broadly. G1 usefulness wording shifted from improving understanding to course learning; G2 referred to construction-planning concepts. Temporal and cross-course measurement equivalence was not established. These items were analyzed separately, without constructing a composite learning score.
03 / Longitudinal matching & missing responses
Identifier formatting was standardized without inferring missing characters or using outcome similarity. G1 used unique exact matches and identifier-to-email links in private author materials; only chronological one-to-one pairs were retained. G2 required a unique exact identifier at both waves and chronological order. Duplicate or unresolved candidates were excluded. Two different G2 respondents had one missing item; no values were imputed. Self-generated codes do not independently verify identity.
04 / Statistical analyses & baseline adjustment
Within-course mean POST-minus-PRE changes used paired t-based 95% confidence intervals. G1-minus-G2 observed-change differences used Welch–Satterthwaite intervals. Conditional POST models included course and numeric PRE, with HC3 standard errors and normal-reference 95% intervals. These are exploratory, marginal intervals; Holm-adjusted tests were also reported within three-indicator families. G2 sections were pooled. HC3 addresses heteroskedasticity and does not adjust for section clustering.
Checks used categorical baseline coding, prior educational chatbot users, and the main G1 response window (37 rather than 38). The participation contrast was sensitive to one valid G1 record. Linked and unmatched PRE responses differed; legal-category worst-case bounds for unique-PRE records allowed either sign of mean change. POST-only Spearman associations between reported use and perceived learning/confidence included whole-record bootstrap and leave-one-out checks; they do not establish time ordering or dose response.
06 / Session feedback & topic indexing
All nonempty submissions were retained with item-specific denominators: 443 records across six G1 forms and 1,119 across sixteen G2 forms. Topic labels came from dictionary-based descriptive indexing, with AI assistance during dictionary development. Categories overlap. There was no independent human double coding or interrater statistic. The notebook reproduces coded summaries, not classification of the private original text. Missing or nonspecific text does not mean that no uncertainty existed.
07 / Limitations & scope of claims
Optional AI use was encouraged but neither standardized nor measured as an individual treatment. Underlying model versions and verification accuracy were not recorded. Outcomes were self-reported, bounded single items, with selective matching and unavailable section labels. Session records may repeat respondents and were not linked to longitudinal records. G1 submission counts sometimes exceeded enrollment. These data do not demonstrate disciplinary performance, actual participation, causal effects, or population-wide learning gains.
A SHARED VOCABULARY
Read the evidence with confidence.
Five concepts for interpreting the figures.
Longitudinal linkage
Connecting the same respondent’s PRE and POST records, allowing within-person comparisons. Session submissions were not linked this way.
Confidence interval
A range reflecting sampling uncertainty under the analysis assumptions. The displayed 95% intervals are marginal and exploratory, and do not account for section clustering.
Ceiling effect
Limited room for upward movement when responses start near the top of a bounded scale. A high starting distribution does not prove that a ceiling mechanism caused the observed pattern.
Observed change
The numerical POST response minus the PRE response for the same respondent. Differences in average change compare how much the two matched groups moved.
Baseline adjustment
A model comparing later responses while conditioning on the measured initial response. It answers a different question from observed change and does not remove all confounding or establish causality.
The research connects engineering education, civil and construction practice, and longitudinal methodology. The search strategies support further inquiry across these areas.
15SEARCH STRATEGIES
0 strategies
Select a theme or research group to explore the Scopus search strategies.
E1Applications
GenAI applications in engineering education
The search covers empirical studies of GenAI applications, instructional activities, and student perceptions in engineering education.
RESEARCH RELEVANCEEngineering education context and interpretation
View full Scopus query
TITLE-ABS-KEY(
("generative artificial intelligence" OR
"generative AI" OR GenAI OR ChatGPT OR
"large language model*" OR "AI chatbot*")
AND
("engineering education" OR
"engineering student*" OR
"engineering curriculum" OR
"engineering course*")
)
AND PUBYEAR > 2021
E2Longitudinal evaluation
Longitudinal and pretest–posttest GenAI evaluation
Longitudinal and pretest-posttest studies provide a basis for interpreting matched surveys, repeated measures, and baseline comparisons.
RESEARCH RELEVANCEEngineering education context and interpretation
View full Scopus query
TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"large language model*" OR "AI chatbot*")
AND
(engineer* OR "higher education" OR
universit* OR undergraduate*)
AND
(longitudinal OR pretest OR posttest OR
"pre-test" OR "post-test" OR
"repeated measures" OR
"quasi-experimental" OR
"within-subject*")
)
AND PUBYEAR > 2021
E3Learning design
Scaffolding and instructional design for GenAI-supported learning
Scaffolding and instructional design research situates GenAI consultation within broader learning activities.
RESEARCH RELEVANCEEngineering education context and interpretation
View full Scopus query
TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"large language model*" OR
"educational chatbot*")
AND
(scaffold* OR "instructional design" OR
"guided learning" OR "guided instruction" OR
"pedagogical integration" OR
"learning activit*" OR "AI tutor*")
AND
(engineer* OR "higher education" OR
universit* OR undergraduate*)
)
AND PUBYEAR > 2021
E4Educational theory
Prior knowledge, expertise, and instructional readiness
Research on prior knowledge, expertise, and readiness informs the interpretation of different course contexts and curricular levels.
RESEARCH RELEVANCEEngineering education context and interpretation
View full Scopus query
TITLE-ABS-KEY(
("prior knowledge" OR "prior experience" OR
"expertise reversal" OR
"learner readiness" OR
"aptitude treatment interaction" OR
"prior expertise")
AND
(scaffold* OR instruction* OR
"technology-enhanced learning" OR
"adaptive learning" OR
"intelligent tutoring" OR
"engineering education")
)
E5AI readiness
AI familiarity, AI literacy, and student readiness
AI familiarity is considered alongside AI literacy, self-efficacy, and technology readiness, while retaining their conceptual distinctions.
RESEARCH RELEVANCEEngineering education context and interpretation
View full Scopus query
TITLE-ABS-KEY(
("AI literacy" OR "AI familiarity" OR
"AI self-efficacy" OR
"digital competenc*" OR
"technology readiness" OR
"AI readiness")
AND
(student* OR undergraduate* OR
"engineering education" OR
"higher education")
AND
("generative AI" OR ChatGPT OR
"artificial intelligence" OR
"large language model*")
)
AND PUBYEAR > 2020
E6Evidence and outcomes
Perceived learning benefits versus demonstrated performance
Studies comparing perceived benefits with measured performance clarify the educational claims supported by each type of evidence.
RESEARCH RELEVANCEEngineering education context and interpretation
View full Scopus query
TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"large language model*" OR
"AI-assisted learning")
AND
("engineering education" OR
"engineering student*" OR
"higher education")
AND
("learning outcome*" OR
"academic performance" OR
"student perception*" OR
"learning gain*" OR
"self-reported learning" OR
"learning achievement")
)
AND PUBYEAR > 2021
M1Baseline comparisons
Baseline adjustment, change scores, and Lord's paradox
Baseline adjustment and change-score research clarifies the distinct questions addressed by observed change and conditional POST comparisons.
RESEARCH RELEVANCEAnalytical design, measurement, and limits of inference
View full Scopus query
TITLE-ABS-KEY(
("Lord's paradox" OR "baseline adjustment" OR
"baseline-adjusted" OR ANCOVA OR
"change score*" OR "regression to the mean")
AND
(nonrandom* OR "pretest-posttest" OR
"pretest posttest" OR
"preexisting group*" OR
"pre-existing group*" OR
"longitudinal comparison")
)
AND PUBYEAR > 1999
M2Measurement
Measurement validity, response shift, and ceiling effects
Measurement research addresses the validity of single items, bounded scales, response shifts, and ceiling effects in repeated self-reports.
RESEARCH RELEVANCEAnalytical design, measurement, and limits of inference
View full Scopus query
TITLE-ABS-KEY(
("response shift" OR "ceiling effect*" OR
"single-item measure*" OR
"measurement invariance" OR
"self-report bias" OR
"scale responsiveness")
AND
(pretest OR posttest OR longitudinal OR
"educational evaluation")
AND
(student* OR education* OR training)
)
M3Longitudinal linkage
Longitudinal linkage, missing data, and sample selection
Longitudinal linkage research informs the handling of matching, duplicated identifiers, missing follow-up responses, and sample selection.
RESEARCH RELEVANCEAnalytical design, measurement, and limits of inference
View full Scopus query
TITLE-ABS-KEY(
("self-generated identifier*" OR
"anonymous identifier*" OR
"participant linkage" OR
"record linkage" OR
"longitudinal matching" OR
"survey linkage")
AND
(survey* OR longitudinal OR
cohort* OR respondent*)
)
AND PUBYEAR > 2004
M4Evidence and outcomes
AI-use frequency, engagement, and self-reported outcomes
Research on reported AI use examines associations with engagement, perceived learning benefits, and confidence.
RESEARCH RELEVANCEAnalytical design, measurement, and limits of inference
View full Scopus query
TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"large language model*" OR "AI chatbot*")
AND
("usage frequency" OR "frequency of use" OR
"student engagement" OR "AI engagement")
AND
("perceived learning" OR confidence OR
"learning outcome*" OR "self-efficacy" OR
"academic performance")
)
AND PUBYEAR > 2021
C1Applications
GenAI in civil engineering and construction education
The disciplinary literature places GenAI within civil engineering, construction engineering, and construction management education.
RESEARCH RELEVANCECivil and construction disciplinary context
View full Scopus query
TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"generative artificial intelligence" OR
"large language model*" OR
"AI chatbot*")
AND
("civil engineering education" OR
"construction education" OR
"construction management education" OR
"construction engineering education" OR
"built environment education" OR
"civil engineering student*" OR
"construction management student*")
)
AND PUBYEAR > 2021
C2Scheduling
GenAI in construction scheduling and professional tasks
Scheduling research connects GenAI-supported activities with work breakdown structures, activity relationships, durations, resources, and technical verification.
RESEARCH RELEVANCECivil and construction disciplinary context
View full Scopus query
TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"large language model*" OR
"AI chatbot*")
AND
("construction scheduling" OR
"project scheduling" OR
"construction planning" OR
"work breakdown structure*" OR
"critical path method" OR
"construction project management")
AND
(educat* OR teach* OR learn* OR
student* OR curriculum)
)
AND PUBYEAR > 2021
C3Professional judgment
Professional competencies, verification, and responsible AI use
Research on professional competencies addresses judgment, critical evaluation, verification, and responsible AI use in engineering practice.
RESEARCH RELEVANCECivil and construction disciplinary context
View full Scopus query
TITLE-ABS-KEY(
("engineering education" OR
"civil engineering education" OR
"construction education" OR
"construction management education")
AND
("generative AI" OR ChatGPT OR
"large language model*" OR
"artificial intelligence")
AND
("professional competenc*" OR
"AI literacy" OR
verificat* OR "critical thinking" OR
"professional judgment" OR
"human-AI collaboration" OR
"responsible AI")
)
AND PUBYEAR > 2021
C4Curriculum
AI education across academic levels and the curriculum
Curricular research considers differences in AI-related learning needs and assessment across academic levels.
RESEARCH RELEVANCECivil and construction disciplinary context
View full Scopus query
TITLE-ABS-KEY(
("engineering education" OR
"civil engineering education" OR
"construction education" OR
"engineering student*")
AND
("first-year" OR freshman* OR
"upper-level" OR "final-year" OR
senior* OR "academic level*" OR
"curriculum integration" OR
"curriculum progression")
AND
("generative AI" OR ChatGPT OR
"AI literacy" OR "artificial intelligence")
)
AND PUBYEAR > 2021
J1Journal positioning
GenAI educational research in CAEE, EJEE, JCEE, and IJCER
The journal search covers GenAI educational research in CAEE, EJEE, JCEE, and IJCER.
RESEARCH RELEVANCEGenAI studies in engineering and construction education journals
View full Scopus query
(
EXACTSRCTITLE("Computer Applications in Engineering Education")
OR EXACTSRCTITLE("European Journal of Engineering Education")
OR EXACTSRCTITLE("Journal of Civil Engineering Education")
OR EXACTSRCTITLE("International Journal of Construction Education and Research")
)
AND TITLE-ABS-KEY(
("generative AI" OR ChatGPT OR
"large language model*" OR
"AI chatbot*" OR "artificial intelligence")
AND
(student* OR educat* OR teach* OR
learn* OR pedagog*)
)
AND PUBYEAR > 2021
No strategies match these filters. Try another theme or research group.
THE RESEARCH BIBLIOGRAPHY
The scholarship behind the study.
21 RESEARCH REFERENCES
The bibliography connects the study with engineering education, civil and construction practice, educational theory, and methodology.
Select a theme to explore the research bibliography.
0 references
01
2025Computer Applications in Engineering Education
J. L. Jooste, K. E. Wolff, and J. G. Joubert. Engineering Students' Perceptions and Use of Generative Artificial Intelligence. Computer Applications in Engineering Education 33(4) (2025): e70064. https://doi.org/10.1002/cae.70064
N. Matzakos and M. Moundridou. Exploring Large Language Models Integration in Higher Education: A Case Study in a Mathematics Laboratory for Civil Engineering Students. Computer Applications in Engineering Education 33(3) (2025): e70049. https://doi.org/10.1002/cae.70049
C. Vieira, J. L. De la Hoz, A. J. Magana, and D. Restrepo
Engineering education
Full citation & DOI
C. Vieira, J. L. De la Hoz, A. J. Magana, and D. Restrepo. Engineering Students' Experiences With ChatGPT to Generate Code for Disciplinary Programming. Computer Applications in Engineering Education 33(6) (2025): e70090. https://doi.org/10.1002/cae.70090
S. M. J. Uddin, A. Albert, M. Tamanna, A. Ovid, and A. Alsharef
Engineering educationCivil/construction education
Full citation & DOI
S. M. J. Uddin, A. Albert, M. Tamanna, A. Ovid, and A. Alsharef. ChatGPT as an educational resource for civil engineering students. Computer Applications in Engineering Education 32(4) (2024): e22747. https://doi.org/10.1002/cae.22747
V. G. AlHakim, N. A. Paiman, and M. H. S. Rahman. Genie-on-demand: A custom AI chatbot for enhancing learning performance, self-efficacy, and technology acceptance in occupational health and safety for engineering education. Computer Applications in Engineering Education 32(6) (2024): e22800. https://doi.org/10.1002/cae.22800
N. Chaudhary, S. M. J. Uddin, and Z. Jin. desAIner: Design, Implementation, and Evaluation of an AI Tool for Construction Hazard Prevention Through Design (CHPtD) Education. Computer Applications in Engineering Education 34(4) (2026): e70231. https://doi.org/10.1002/cae.70231
P. Chakraborty. Effect of Using Generative Artificial Intelligence in Different Roles in a Course on Computer Programming. Computer Applications in Engineering Education 34(4) (2026): e70223. https://doi.org/10.1002/cae.70223
R. Sajja, Y. Sermet, B. Fodale, and I. Demir. Evaluating AI-powered learning assistants in engineering higher education with implications for student engagement, ethics, and policy. Scientific Reports 16 (2026): 7565. https://doi.org/10.1038/s41598-026-39237-5
M. T. Kane. Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement 50(1) (2013): 1-73. https://doi.org/10.1111/jedm.12000
R. J. Mislevy, L. S. Steinberg, and R. G. Almond. Focus Article: On the Structure of Educational Assessments. Measurement: Interdisciplinary Research and Perspectives 1(1) (2003): 3-62. https://doi.org/10.1207/S15366359MEA0101_02
D. Long and B. Magerko. What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020): 1-16. https://doi.org/10.1145/3313831.3376727
J. A. Fredricks, P. C. Blumenfeld, and A. H. Paris
Educational theory
Full citation & DOI
J. A. Fredricks, P. C. Blumenfeld, and A. H. Paris. School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research 74(1) (2004): 59-109. https://doi.org/10.3102/00346543074001059
G. J. P. van Breukelen. ANCOVA Versus CHANGE From Baseline in Nonrandomized Studies: The Difference. Multivariate Behavioral Research 48(6) (2013): 895-922. https://doi.org/10.1080/00273171.2013.831743
A. G. Barnett, J. C. van der Pols, and A. J. Dobson
Methodology
Full citation & DOI
A. G. Barnett, J. C. van der Pols, and A. J. Dobson. Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology 34(1) (2005): 215-220. https://doi.org/10.1093/ije/dyh299
T. M. Liddell and J. K. Kruschke. Analyzing ordinal data with metric models: What could possibly go wrong?. Journal of Experimental Social Psychology 79 (2018): 328-348. https://doi.org/10.1016/j.jesp.2018.08.009
L. A. Yurek, J. Vasey, and D. S. Havens. The Use of Self-Generated Identification Codes in Longitudinal Research. Evaluation Review 32(5) (2008): 435-452. https://doi.org/10.1177/0193841X08316676
J. G. MacKinnon and H. White. Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties. Journal of Econometrics 29(3) (1985): 305-325. https://doi.org/10.1016/0304-4076(85)90158-7
S. Holm. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics 6(2) (1979): 65-70. https://www.jstor.org/stable/4615733
H.-F. Hsieh and S. E. Shannon. Three Approaches to Qualitative Content Analysis. Qualitative Health Research 15(9) (2005): 1277-1288. https://doi.org/10.1177/1049732305276687
A. Nieto-Leal, C. Vieira, R. Chacón, and H. Murzi. Exploring the Use of Prompts and AI-Generated Images to Elicit Explanations in Engineering Education. Computer Applications in Engineering Education 34(2) (2026): e70173. https://doi.org/10.1002/cae.70173
Participant-level data and personal identifiers are not available for download. Materials awaiting public release remain unavailable.
07RESEARCH INSIGHTS & IMPLICATIONS
Better questions. More useful evidence.
Three interpretations for engineering educators, grounded in the study and bounded by its observational design.
01
Starting conditions matter.
Students may enter courses with different familiarity levels and expectations. Report those distributions and define the comparison question before interpreting change.
02
Experience is multidimensional.
Favorable ratings and unresolved technical questions can coexist. Pair appraisals with task-specific follow-up, such as explaining activity logic or justifying a resource assumption.
03
Evidence must match educational claims.
Perceptions, observed behavior, and disciplinary performance answer different questions. Assess explanations, calculations, schedules, or later unaided work when claiming competence.
These interpretations draw on alignment and validity perspectives; they do not establish causal mechanisms.
○ ─── ◉ ─── ○
Better AI-supported education begins with better evidence about how students learn, use, and evaluate the technology.
Students first explained what civil engineering entails, then compared their explanation with additional explanations from course materials or optional AI consultation. Peer and class discussion supported refinement.
10 weeks · one three-hour session per week.
G2Spring 2026
Construction Project Scheduling
Upper-level / senior-level
Activity, time, and resource reasoning, supported by conventional calculations and scheduling software.
87enrolled students
59matched respondents
3sections · 28 / 29 / 30
Work breakdown structuresActivity relationships & durationsProductivity & resourcesVerification of generated suggestions
Inside an instructional activity
Students developed a residential-townhouse activity list or WBS and examined formwork, reinforcing steel, and concrete work. They compared optional AI suggestions with durations, productivity information, crew assumptions, and their own calculations.
15 weeks · 20 sessions per section. Five weeks of twice-weekly meetings, staggered across the three sections.
G1Fall 2025
Introduction to Civil Engineering
Introductory / freshman-level
A foundation in the profession, infrastructure, construction projects, professional skills, and ethics.
Students first explained what civil engineering entails, then compared their explanation with additional explanations from course materials or optional AI consultation. Peer and class discussion supported refinement.
10 weeks · one three-hour session per week.
G2Spring 2026
Construction Project Scheduling
Upper-level / senior-level
Activity, time, and resource reasoning, supported by conventional calculations and scheduling software.
87enrolled students
59matched respondents
3sections · 28 / 29 / 30
Work breakdown structuresActivity relationships & durationsProductivity & resourcesVerification of generated suggestions
Inside an instructional activity
Students developed a residential-townhouse activity list or WBS and examined formwork, reinforcing steel, and concrete work. They compared optional AI suggestions with durations, productivity information, crew assumptions, and their own calculations.
15 weeks · 20 sessions per section. Five weeks of twice-weekly meetings, staggered across the three sections.
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G1n = 38 complete pairs
G2n = 58 complete pairs
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
G2n = 59 complete pairs
Perceived participation potential (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G1
PRE
1
1 / 38
2.63%
familiarity
G1
PRE
2
14 / 38
36.84%
familiarity
G1
PRE
3
16 / 38
42.11%
familiarity
G1
PRE
4
7 / 38
18.42%
familiarity
G1
POST
1
1 / 38
2.63%
familiarity
G1
POST
2
0 / 38
0.00%
familiarity
G1
POST
3
13 / 38
34.21%
familiarity
G1
POST
4
24 / 38
63.16%
familiarity
G2
PRE
1
1 / 58
1.72%
familiarity
G2
PRE
2
8 / 58
13.79%
familiarity
G2
PRE
3
25 / 58
43.10%
familiarity
G2
PRE
4
24 / 58
41.38%
familiarity
G2
POST
1
3 / 58
5.17%
familiarity
G2
POST
2
3 / 58
5.17%
familiarity
G2
POST
3
24 / 58
41.38%
familiarity
G2
POST
4
28 / 58
48.28%
usefulness
G1
PRE
1
0 / 38
0.00%
usefulness
G1
PRE
2
0 / 38
0.00%
usefulness
G1
PRE
3
12 / 38
31.58%
usefulness
G1
PRE
4
20 / 38
52.63%
usefulness
G1
PRE
5
6 / 38
15.79%
usefulness
G1
POST
1
0 / 38
0.00%
usefulness
G1
POST
2
0 / 38
0.00%
usefulness
G1
POST
3
4 / 38
10.53%
usefulness
G1
POST
4
22 / 38
57.89%
usefulness
G1
POST
5
12 / 38
31.58%
usefulness
G2
PRE
1
0 / 59
0.00%
usefulness
G2
PRE
2
2 / 59
3.39%
usefulness
G2
PRE
3
12 / 59
20.34%
usefulness
G2
PRE
4
35 / 59
59.32%
usefulness
G2
PRE
5
10 / 59
16.95%
usefulness
G2
POST
1
1 / 59
1.69%
usefulness
G2
POST
2
1 / 59
1.69%
usefulness
G2
POST
3
6 / 59
10.17%
usefulness
G2
POST
4
34 / 59
57.63%
usefulness
G2
POST
5
17 / 59
28.81%
participation
G1
PRE
1
0 / 38
0.00%
participation
G1
PRE
2
0 / 38
0.00%
participation
G1
PRE
3
12 / 38
31.58%
participation
G1
PRE
4
19 / 38
50.00%
participation
G1
PRE
5
7 / 38
18.42%
participation
G1
POST
1
1 / 38
2.63%
participation
G1
POST
2
0 / 38
0.00%
participation
G1
POST
3
2 / 38
5.26%
participation
G1
POST
4
22 / 38
57.89%
participation
G1
POST
5
13 / 38
34.21%
participation
G2
PRE
1
0 / 58
0.00%
participation
G2
PRE
2
0 / 58
0.00%
participation
G2
PRE
3
7 / 58
12.07%
participation
G2
PRE
4
33 / 58
56.90%
participation
G2
PRE
5
18 / 58
31.03%
participation
G2
POST
1
3 / 58
5.17%
participation
G2
POST
2
0 / 58
0.00%
participation
G2
POST
3
7 / 58
12.07%
participation
G2
POST
4
32 / 58
55.17%
participation
G2
POST
5
16 / 58
27.59%
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G1n = 38 complete pairs
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G1
PRE
1
1 / 38
2.63%
familiarity
G1
PRE
2
14 / 38
36.84%
familiarity
G1
PRE
3
16 / 38
42.11%
familiarity
G1
PRE
4
7 / 38
18.42%
familiarity
G1
POST
1
1 / 38
2.63%
familiarity
G1
POST
2
0 / 38
0.00%
familiarity
G1
POST
3
13 / 38
34.21%
familiarity
G1
POST
4
24 / 38
63.16%
familiarity
G2
PRE
1
1 / 58
1.72%
familiarity
G2
PRE
2
8 / 58
13.79%
familiarity
G2
PRE
3
25 / 58
43.10%
familiarity
G2
PRE
4
24 / 58
41.38%
familiarity
G2
POST
1
3 / 58
5.17%
familiarity
G2
POST
2
3 / 58
5.17%
familiarity
G2
POST
3
24 / 58
41.38%
familiarity
G2
POST
4
28 / 58
48.28%
Complete-pair PRE / POST response distributions
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
G2n = 59 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
usefulness
G1
PRE
1
0 / 38
0.00%
usefulness
G1
PRE
2
0 / 38
0.00%
usefulness
G1
PRE
3
12 / 38
31.58%
usefulness
G1
PRE
4
20 / 38
52.63%
usefulness
G1
PRE
5
6 / 38
15.79%
usefulness
G1
POST
1
0 / 38
0.00%
usefulness
G1
POST
2
0 / 38
0.00%
usefulness
G1
POST
3
4 / 38
10.53%
usefulness
G1
POST
4
22 / 38
57.89%
usefulness
G1
POST
5
12 / 38
31.58%
usefulness
G2
PRE
1
0 / 59
0.00%
usefulness
G2
PRE
2
2 / 59
3.39%
usefulness
G2
PRE
3
12 / 59
20.34%
usefulness
G2
PRE
4
35 / 59
59.32%
usefulness
G2
PRE
5
10 / 59
16.95%
usefulness
G2
POST
1
1 / 59
1.69%
usefulness
G2
POST
2
1 / 59
1.69%
usefulness
G2
POST
3
6 / 59
10.17%
usefulness
G2
POST
4
34 / 59
57.63%
usefulness
G2
POST
5
17 / 59
28.81%
Complete-pair PRE / POST response distributions
Perceived participation potential (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
participation
G1
PRE
1
0 / 38
0.00%
participation
G1
PRE
2
0 / 38
0.00%
participation
G1
PRE
3
12 / 38
31.58%
participation
G1
PRE
4
19 / 38
50.00%
participation
G1
PRE
5
7 / 38
18.42%
participation
G1
POST
1
1 / 38
2.63%
participation
G1
POST
2
0 / 38
0.00%
participation
G1
POST
3
2 / 38
5.26%
participation
G1
POST
4
22 / 38
57.89%
participation
G1
POST
5
13 / 38
34.21%
participation
G2
PRE
1
0 / 58
0.00%
participation
G2
PRE
2
0 / 58
0.00%
participation
G2
PRE
3
7 / 58
12.07%
participation
G2
PRE
4
33 / 58
56.90%
participation
G2
PRE
5
18 / 58
31.03%
participation
G2
POST
1
3 / 58
5.17%
participation
G2
POST
2
0 / 58
0.00%
participation
G2
POST
3
7 / 58
12.07%
participation
G2
POST
4
32 / 58
55.17%
participation
G2
POST
5
16 / 58
27.59%
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G1n = 38 complete pairs
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
Perceived participation potential (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G1
PRE
1
1 / 38
2.63%
familiarity
G1
PRE
2
14 / 38
36.84%
familiarity
G1
PRE
3
16 / 38
42.11%
familiarity
G1
PRE
4
7 / 38
18.42%
familiarity
G1
POST
1
1 / 38
2.63%
familiarity
G1
POST
2
0 / 38
0.00%
familiarity
G1
POST
3
13 / 38
34.21%
familiarity
G1
POST
4
24 / 38
63.16%
usefulness
G1
PRE
1
0 / 38
0.00%
usefulness
G1
PRE
2
0 / 38
0.00%
usefulness
G1
PRE
3
12 / 38
31.58%
usefulness
G1
PRE
4
20 / 38
52.63%
usefulness
G1
PRE
5
6 / 38
15.79%
usefulness
G1
POST
1
0 / 38
0.00%
usefulness
G1
POST
2
0 / 38
0.00%
usefulness
G1
POST
3
4 / 38
10.53%
usefulness
G1
POST
4
22 / 38
57.89%
usefulness
G1
POST
5
12 / 38
31.58%
participation
G1
PRE
1
0 / 38
0.00%
participation
G1
PRE
2
0 / 38
0.00%
participation
G1
PRE
3
12 / 38
31.58%
participation
G1
PRE
4
19 / 38
50.00%
participation
G1
PRE
5
7 / 38
18.42%
participation
G1
POST
1
1 / 38
2.63%
participation
G1
POST
2
0 / 38
0.00%
participation
G1
POST
3
2 / 38
5.26%
participation
G1
POST
4
22 / 38
57.89%
participation
G1
POST
5
13 / 38
34.21%
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G1n = 38 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G1
PRE
1
1 / 38
2.63%
familiarity
G1
PRE
2
14 / 38
36.84%
familiarity
G1
PRE
3
16 / 38
42.11%
familiarity
G1
PRE
4
7 / 38
18.42%
familiarity
G1
POST
1
1 / 38
2.63%
familiarity
G1
POST
2
0 / 38
0.00%
familiarity
G1
POST
3
13 / 38
34.21%
familiarity
G1
POST
4
24 / 38
63.16%
Complete-pair PRE / POST response distributions
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
usefulness
G1
PRE
1
0 / 38
0.00%
usefulness
G1
PRE
2
0 / 38
0.00%
usefulness
G1
PRE
3
12 / 38
31.58%
usefulness
G1
PRE
4
20 / 38
52.63%
usefulness
G1
PRE
5
6 / 38
15.79%
usefulness
G1
POST
1
0 / 38
0.00%
usefulness
G1
POST
2
0 / 38
0.00%
usefulness
G1
POST
3
4 / 38
10.53%
usefulness
G1
POST
4
22 / 38
57.89%
usefulness
G1
POST
5
12 / 38
31.58%
Complete-pair PRE / POST response distributions
Perceived participation potential (1–5)
LOWER12345HIGHER
G1n = 38 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
participation
G1
PRE
1
0 / 38
0.00%
participation
G1
PRE
2
0 / 38
0.00%
participation
G1
PRE
3
12 / 38
31.58%
participation
G1
PRE
4
19 / 38
50.00%
participation
G1
PRE
5
7 / 38
18.42%
participation
G1
POST
1
1 / 38
2.63%
participation
G1
POST
2
0 / 38
0.00%
participation
G1
POST
3
2 / 38
5.26%
participation
G1
POST
4
22 / 38
57.89%
participation
G1
POST
5
13 / 38
34.21%
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G2n = 58 complete pairs
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G2n = 59 complete pairs
Perceived participation potential (1–5)
LOWER12345HIGHER
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G2
PRE
1
1 / 58
1.72%
familiarity
G2
PRE
2
8 / 58
13.79%
familiarity
G2
PRE
3
25 / 58
43.10%
familiarity
G2
PRE
4
24 / 58
41.38%
familiarity
G2
POST
1
3 / 58
5.17%
familiarity
G2
POST
2
3 / 58
5.17%
familiarity
G2
POST
3
24 / 58
41.38%
familiarity
G2
POST
4
28 / 58
48.28%
usefulness
G2
PRE
1
0 / 59
0.00%
usefulness
G2
PRE
2
2 / 59
3.39%
usefulness
G2
PRE
3
12 / 59
20.34%
usefulness
G2
PRE
4
35 / 59
59.32%
usefulness
G2
PRE
5
10 / 59
16.95%
usefulness
G2
POST
1
1 / 59
1.69%
usefulness
G2
POST
2
1 / 59
1.69%
usefulness
G2
POST
3
6 / 59
10.17%
usefulness
G2
POST
4
34 / 59
57.63%
usefulness
G2
POST
5
17 / 59
28.81%
participation
G2
PRE
1
0 / 58
0.00%
participation
G2
PRE
2
0 / 58
0.00%
participation
G2
PRE
3
7 / 58
12.07%
participation
G2
PRE
4
33 / 58
56.90%
participation
G2
PRE
5
18 / 58
31.03%
participation
G2
POST
1
3 / 58
5.17%
participation
G2
POST
2
0 / 58
0.00%
participation
G2
POST
3
7 / 58
12.07%
participation
G2
POST
4
32 / 58
55.17%
participation
G2
POST
5
16 / 58
27.59%
Complete-pair PRE / POST response distributions
AI familiarity (1–4)
LOWER1234HIGHER
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
familiarity
G2
PRE
1
1 / 58
1.72%
familiarity
G2
PRE
2
8 / 58
13.79%
familiarity
G2
PRE
3
25 / 58
43.10%
familiarity
G2
PRE
4
24 / 58
41.38%
familiarity
G2
POST
1
3 / 58
5.17%
familiarity
G2
POST
2
3 / 58
5.17%
familiarity
G2
POST
3
24 / 58
41.38%
familiarity
G2
POST
4
28 / 58
48.28%
Complete-pair PRE / POST response distributions
Perceived potential usefulness (1–5)
LOWER12345HIGHER
G2n = 59 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
usefulness
G2
PRE
1
0 / 59
0.00%
usefulness
G2
PRE
2
2 / 59
3.39%
usefulness
G2
PRE
3
12 / 59
20.34%
usefulness
G2
PRE
4
35 / 59
59.32%
usefulness
G2
PRE
5
10 / 59
16.95%
usefulness
G2
POST
1
1 / 59
1.69%
usefulness
G2
POST
2
1 / 59
1.69%
usefulness
G2
POST
3
6 / 59
10.17%
usefulness
G2
POST
4
34 / 59
57.63%
usefulness
G2
POST
5
17 / 59
28.81%
Complete-pair PRE / POST response distributions
Perceived participation potential (1–5)
LOWER12345HIGHER
G2n = 58 complete pairs
Bars sum to 100%; printed segment labels are rounded. G1/G2 complete pairs: familiarity 38/58, usefulness 38/59, participation 38/58.
Read exact response counts
Response counts for the selected courses and indicators
Indicator
Course
Wave
Category
Count / n
Percent
participation
G2
PRE
1
0 / 58
0.00%
participation
G2
PRE
2
0 / 58
0.00%
participation
G2
PRE
3
7 / 58
12.07%
participation
G2
PRE
4
33 / 58
56.90%
participation
G2
PRE
5
18 / 58
31.03%
participation
G2
POST
1
3 / 58
5.17%
participation
G2
POST
2
0 / 58
0.00%
participation
G2
POST
3
7 / 58
12.07%
participation
G2
POST
4
32 / 58
55.17%
participation
G2
POST
5
16 / 58
27.59%
INDIVIDUAL RESPONSE MOVEMENTS
Averages contain different experiences.
Familiarityn = 38Mean 2.763 3.579
26102
↑ Increased 26= Unchanged 10↓ Decreased 2
Usefulnessn = 38Mean 3.842 4.211
16193
↑ Increased 16= Unchanged 19↓ Decreased 3
Participation potentialn = 38Mean 3.868 4.211
18173
↑ Increased 18= Unchanged 17↓ Decreased 3
INDIVIDUAL RESPONSE MOVEMENTS
Averages contain different experiences.
Familiarityn = 58Mean 3.241 3.328
173011
↑ Increased 17= Unchanged 30↓ Decreased 11
Usefulnessn = 59Mean 3.898 4.102
19328
↑ Increased 19= Unchanged 32↓ Decreased 8
Participation potentialn = 58Mean 4.190 4.000
113314
↑ Increased 11= Unchanged 33↓ Decreased 14
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G1: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
4
70
9 / 61 / 0
70
4.343
70
5
67
49 / 18 / 0
67
4.567
67
6
72
66 / 6 / 0
72
4.681
72
8
76
64 / 12 / 0
76
4.711
76
9
74
61 / 12 / 1
74
4.500
74
10
84
73 / 10 / 1
84
4.774
84
THE DISCIPLINARY DETAIL / G1
What did the assistance refer to?
Civil engineering, infrastructure and construction systems
54
Project concepts, lifecycle and management
40
Quality, safety and professional ethics
31
Communication, disciplinary language and study support
21
Software operations and graphical representations
6
Other named academic subject
4
Scheduling relationships, durations and calendars
3
Resources, productivity and costs
3
Work decomposition and construction operations
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G1: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
4
70
9 / 61 / 0
70
4.343
70
5
67
49 / 18 / 0
67
4.567
67
6
72
66 / 6 / 0
72
4.681
72
8
76
64 / 12 / 0
76
4.711
76
9
74
61 / 12 / 1
74
4.500
74
10
84
73 / 10 / 1
84
4.774
84
THE DISCIPLINARY DETAIL / G1
What did the assistance refer to?
Civil engineering, infrastructure and construction systems
13
Project concepts, lifecycle and management
7
Quality, safety and professional ethics
6
Scheduling relationships, durations and calendars
2
Resources, productivity and costs
1
Other named academic subject
1
Communication, disciplinary language and study support
0
Work decomposition and construction operations
0
Software operations and graphical representations
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G1: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
4
70
9 / 61 / 0
70
4.343
70
5
67
49 / 18 / 0
67
4.567
67
6
72
66 / 6 / 0
72
4.681
72
8
76
64 / 12 / 0
76
4.711
76
9
74
61 / 12 / 1
74
4.500
74
10
84
73 / 10 / 1
84
4.774
84
THE DISCIPLINARY DETAIL / G1
What did the assistance refer to?
Civil engineering, infrastructure and construction systems
54
Project concepts, lifecycle and management
40
Quality, safety and professional ethics
31
Communication, disciplinary language and study support
21
Software operations and graphical representations
6
Other named academic subject
4
Scheduling relationships, durations and calendars
3
Resources, productivity and costs
3
Work decomposition and construction operations
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G1: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
4
70
9 / 61 / 0
70
4.343
70
5
67
49 / 18 / 0
67
4.567
67
6
72
66 / 6 / 0
72
4.681
72
8
76
64 / 12 / 0
76
4.711
76
9
74
61 / 12 / 1
74
4.500
74
10
84
73 / 10 / 1
84
4.774
84
THE DISCIPLINARY DETAIL / G1
What did the assistance refer to?
Civil engineering, infrastructure and construction systems
13
Project concepts, lifecycle and management
7
Quality, safety and professional ethics
6
Scheduling relationships, durations and calendars
2
Resources, productivity and costs
1
Other named academic subject
1
Communication, disciplinary language and study support
0
Work decomposition and construction operations
0
Software operations and graphical representations
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G2: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
1
68
61 / 7 / 0
68
4.463
67
2
81
76 / 3 / 2
81
4.407
81
3
77
70 / 7 / 0
77
4.623
77
4
73
66 / 6 / 1
73
4.589
73
5
77
64 / 8 / 4
76
4.600
75
6
75
65 / 9 / 1
75
4.653
75
7
76
66 / 9 / 1
76
4.711
76
8
64
56 / 8 / 0
64
4.762
63
9
60
53 / 4 / 1
58
4.644
59
10
70
60 / 6 / 2
68
4.681
69
11
69
58 / 7 / 3
68
4.642
67
12
67
55 / 7 / 4
66
4.727
66
13
72
60 / 7 / 4
71
4.704
71
14
67
58 / 6 / 2
66
4.708
65
15
75
61 / 12 / 1
74
4.736
72
16
48
41 / 3 / 1
45
4.761
46
THE DISCIPLINARY DETAIL / G2
What did the assistance refer to?
Scheduling relationships, durations and calendars
140
Resources, productivity and costs
112
Software operations and graphical representations
81
Work decomposition and construction operations
68
Project concepts, lifecycle and management
10
Communication, disciplinary language and study support
6
Civil engineering, infrastructure and construction systems
1
Quality, safety and professional ethics
0
Other named academic subject
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G2: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
1
68
61 / 7 / 0
68
4.463
67
2
81
76 / 3 / 2
81
4.407
81
3
77
70 / 7 / 0
77
4.623
77
4
73
66 / 6 / 1
73
4.589
73
5
77
64 / 8 / 4
76
4.600
75
6
75
65 / 9 / 1
75
4.653
75
7
76
66 / 9 / 1
76
4.711
76
8
64
56 / 8 / 0
64
4.762
63
9
60
53 / 4 / 1
58
4.644
59
10
70
60 / 6 / 2
68
4.681
69
11
69
58 / 7 / 3
68
4.642
67
12
67
55 / 7 / 4
66
4.727
66
13
72
60 / 7 / 4
71
4.704
71
14
67
58 / 6 / 2
66
4.708
65
15
75
61 / 12 / 1
74
4.736
72
16
48
41 / 3 / 1
45
4.761
46
THE DISCIPLINARY DETAIL / G2
What did the assistance refer to?
Scheduling relationships, durations and calendars
21
Software operations and graphical representations
17
Resources, productivity and costs
7
Work decomposition and construction operations
2
Project concepts, lifecycle and management
1
Communication, disciplinary language and study support
1
Civil engineering, infrastructure and construction systems
0
Quality, safety and professional ethics
0
Other named academic subject
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G2: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
1
68
61 / 7 / 0
68
4.463
67
2
81
76 / 3 / 2
81
4.407
81
3
77
70 / 7 / 0
77
4.623
77
4
73
66 / 6 / 1
73
4.589
73
5
77
64 / 8 / 4
76
4.600
75
6
75
65 / 9 / 1
75
4.653
75
7
76
66 / 9 / 1
76
4.711
76
8
64
56 / 8 / 0
64
4.762
63
9
60
53 / 4 / 1
58
4.644
59
10
70
60 / 6 / 2
68
4.681
69
11
69
58 / 7 / 3
68
4.642
67
12
67
55 / 7 / 4
66
4.727
66
13
72
60 / 7 / 4
71
4.704
71
14
67
58 / 6 / 2
66
4.708
65
15
75
61 / 12 / 1
74
4.736
72
16
48
41 / 3 / 1
45
4.761
46
THE DISCIPLINARY DETAIL / G2
What did the assistance refer to?
Scheduling relationships, durations and calendars
140
Resources, productivity and costs
112
Software operations and graphical representations
81
Work decomposition and construction operations
68
Project concepts, lifecycle and management
10
Communication, disciplinary language and study support
6
Civil engineering, infrastructure and construction systems
1
Quality, safety and professional ethics
0
Other named academic subject
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.
03THE SESSION-LEVEL EVIDENCE
Useful assistance. Unfinished questions.
A second evidence stream brings the classroom tasks into focus: what respondents used, what helped, and what remained unclear.
Missing resource choices are excluded from stacked bars. G1 available weeks: 4, 5, 6, 8, 9, 10; Week 7 was unavailable. G2 labels identify instructional occasions, not common calendar weeks, and pool three sections. Sixteen forms cover part of the 20-session schedule.
Read the session data
G2: all available session forms
Session
Records
Course / general / none
Valid choices
Mean rating
Valid ratings
1
68
61 / 7 / 0
68
4.463
67
2
81
76 / 3 / 2
81
4.407
81
3
77
70 / 7 / 0
77
4.623
77
4
73
66 / 6 / 1
73
4.589
73
5
77
64 / 8 / 4
76
4.600
75
6
75
65 / 9 / 1
75
4.653
75
7
76
66 / 9 / 1
76
4.711
76
8
64
56 / 8 / 0
64
4.762
63
9
60
53 / 4 / 1
58
4.644
59
10
70
60 / 6 / 2
68
4.681
69
11
69
58 / 7 / 3
68
4.642
67
12
67
55 / 7 / 4
66
4.727
66
13
72
60 / 7 / 4
71
4.704
71
14
67
58 / 6 / 2
66
4.708
65
15
75
61 / 12 / 1
74
4.736
72
16
48
41 / 3 / 1
45
4.761
46
THE DISCIPLINARY DETAIL / G2
What did the assistance refer to?
Scheduling relationships, durations and calendars
21
Software operations and graphical representations
17
Resources, productivity and costs
7
Work decomposition and construction operations
2
Project concepts, lifecycle and management
1
Communication, disciplinary language and study support
1
Civil engineering, infrastructure and construction systems
0
Quality, safety and professional ethics
0
Other named academic subject
0
Number of submissions mentioning each topic; categories overlap. Dictionary-based indexing, not a validated thematic structure. Blank, nonspecific, non-use, and no-topic statuses remain in the corpus. “Other” is excluded from the course-related unresolved-topic comparison below.
FAVORABLE, YET UNRESOLVED64 / 70
Each mark represents one selected submission.
A positive rating can sit beside an open question.
Of 70 submissions naming a course-related unresolved topic, 64 also gave one of the two highest effectiveness ratings: quite effective or very effective.
G1: 27/29. G2: 37/41. Remaining questions concerned topics such as parent-task durations, parallel activities, calendars, and resource units.
A tool can help without resolving every difficulty. The unclear concept may concern material not discussed with the chatbot. This is co-occurrence within selected response records, not the prevalence of uncertainty among students or evidence of learning.