Developing and validating a productive thinking assessment for sustainable university science learning

Assessing productive thinking in sustainable science learning requires instruments that capture coordinated cognitive and metacognitive processes during authentic problem-solving. Existing higher-order thinking assessments often examine critical thinking, creativity, or metacognition separately and provide limited validity evidence in performance-based sustainability contexts. This study developed and evaluated a performance-based Productive Thinking Assessment (PTA) measuring three interrelated dimensions: critical–analytical, creative thinking, and thinking process management within microplastic and bioplastic contexts.

A multi-method instrument development design was used to obtain complementary validity evidence. Content validation involved seven experts, while psychometric evaluation involved 200 pre-service elementary science teachers from three Indonesian universities. Response-process evidence was examined using Think-Aloud Protocols with six students.

Content validity was evaluated using Aiken’s V, internal structure using Principal Component Analysis (PCA) and PLS-SEM, item functioning using Rasch analysis, and scoring reliability using intraclass correlation coefficients. Aiken’s V coefficients ranged from 0.81 to 0.95. PCA supported the proposed three-dimensional structure, and PLS-SEM indicated high indicator reliability, internal consistency, and convergent validity.

Rasch analysis showed acceptable item fit, with item difficulties ranging from − 0.829 to 1.289 logits and ordered, estimable rating-category thresholds. Inter-rater reliability ranged from moderate to good for individual ratings and from good to excellent for average ratings. The think-aloud findings illustrated evidence-based reasoning, solution generation, and metacognitive regulation during task completion.

The findings provide complementary validity evidence supporting the interpretation of PTA scores for assessing productive thinking within sustainability-oriented science learning, while indicating the need for further validation across broader populations and science contexts. This research was supported by the Indonesia Endowment Fund for Education (Lembaga Pengelola Dana Pendidikan, LPDP), Ministry of Finance of the Republic of Indonesia (Registration No. 0004008/TRP/D/ASN-2023). The funding agency had no role in the study design, collection, analysis, or interpretation of data, writing of the manuscript, or decision to submit the manuscript for publication.

Doctoral Program of Science Education, Faculty of Teacher Training and Education, Universitas Sebelas Maret, Surakarta, Indonesia Department of Biology Education, Faculty of Teacher Training and Education, Universitas Sebelas Maret, Surakarta, Indonesia The authors declare no competing interests. All authors have read and approved the final version of the manuscript and consented to its publication in this journal. This study was conducted with research permits officially approved by the Universitas Syiah Kuala (Approval No. 8550/UN11.F6/PK.03.08/2025), Universitas Samudra (Approval No. 1216/UN54.4/PT.01.04/2025) and Universitas Serambi Mekkah (Approval No. 160/FKIP-USM/VI/2025).