Construction of a Competency-Based Assessment Tool for Microstructural Analysis in Practical Metallurgical (CAT-M)

Abstract

The Metallurgy Laboratory course emphasizes learning outcomes in microstructural analysis and heat-treatment processes; however, existing assessments rely predominantly on knowledge tests and lack comprehensive tools for evaluating practical skills and professional attributes. This limitation restricts students from identifying their competency levels and hinders effective learning development. To address this gap, the present study developed and validated the Competency-Based Assessment Tool for Microstructural Analysis (CAT-M) to provide a comprehensive evaluation of student performance in practical metallurgy. The study involved 30 third-year students enrolled in the Industrial Education Program at King Mongkut’s University of Technology Thonburi during the second semester of the 2023 academic year. CAT-M comprises three components: a multiple-choice knowledge test, a rubric-based practical skill assessment, and a behavioural rating scale that assesses responsibility, discipline, and safety awareness. Content validity was examined using the Item-Objective Congruence (IOC) method, while test quality was evaluated through item analysis and reliability indices. IOC values ranged from 0.67 to 1.00. The knowledge test demonstrated item difficulty ranging from 0.20 to 1.00, with discrimination indices of 0.20–1.00 (pre-test) and 0.00–1.00 (post-test). Reliability indices were acceptable, with KR-20 values of 1.12 (pre-test) and 1.19 (post-test), and Cronbach’s alpha coefficients of 0.63 and 0.53. The rubric and behavioural scale also demonstrated acceptable validity with IOC values above 0.50. The findings indicate that CAT-M is a valid, structured, and appropriate instrument for assessing knowledge, skills, and attributes. By integrating multiple domains, CAT-M enhances evaluation practices, supports student self-awareness, and strengthens competency-based curriculum development in technical and vocational education.