PERTANIKA JOURNAL OF SOCIAL SCIENCES AND HUMANITIES

This research is designed to develop a comprehensive assessment framework to study junior high school students’ transformative competencies in three dimensions: creating new value, reconciling tensions and dilemmas, and taking responsibility. Researchers developed and verified the quality of the assessment framework among the 120 junior high school students purposively selected from four schools in the northeastern and central regions of Thailand as the test-takers. The project is segmented into four phases: creation of the construct map, item design, outcome space, and Wright map. The framework’s efficacy is analyzed using the Rasch model and the Maximum Likelihood Estimation method. Validity is rigorously evaluated through empirical evidence derived from the Wright map and cognitive interviews. Reliability is assessed via the standard error of measurement, test information function, expected-a-posteriori reliability, and Infit Mean Square value. Key findings included the development of a five-level construct map and the formulation of scaled open-ended questions based on the test takers’ responses. A cut-off point is determined by dividing the threshold level by the number of tests at corresponding levels, adhering to Wright map criteria for each dimension. Subsequent item analysis and modeling confirmed the internal structure’s validity and reliability. This framework equips them with skills to navigate and effectively address complex real-world challenges, enhancing targeted educational strategies, promoting superior learning outcomes, and preparing students for advanced academic and professional environments.

• Adams, R. J. (2005). Reliability as a measurement design effect. Studies in Educational Evaluation, 31(2-3), 162-172. https://doi.org/10.1016/j.stueduc.2005.05.008

• Adams, R. J., Wu, M. L., Cloney, D., Berezner, A., & Wilson, M. (2020). ACER ConQuest: Generalised Item Response Modelling Software (Version 5.29) [Computer software]. Australian Council for Educational Research. https://www.acer.org/au/conquest

• Adams, R., & Khoo, S. (1996). Quest: Interactive test analysis system version 2.1. The Australian council for educational research. https://eric.ed.gov/?d=ED362553

• Al-Alawneh, M. K., Hawamleh, M. S., Al-Jamal, D. A., & Sasa, G. S. (2019). Communication skills in practice. International Journal of Learning, Teaching and Educational Research, 18(6), 1-19. https://doi.org/10.26803/ijlter.18.6.1

• American Educational Research Association. (2014). Standards for educational and psychological testing (6th ed). https://www.testingstandards.net/uploads/7/6/6/4/76643089/9780935302356.pdf

• Anakttototy, K., Sekawael, M., Latief, M. R. A., & Bin-Hady, W. R. A. (2023). Beyond linguistics: Exploring the cognitive and motivational barriers to essay writing for tertiary students. International Journal of Language Education, 7(3), 447-468. https://doi.org/10.26858/ijole.v7i3.37070

• Baker, F. B., & Kim, S. (2017). The basics of item response theory using R. Springer. https://link.springer.com/book/10.1007/978-3-319-54205-8

• Canto-Sperber, M., & Depuy, J. P. (2001). Competencies for good life and good society. In D. Rychen & L. Salganik (Eds.), Defining and selecting competencies (pp. 67-92). Hogrefe Publishing.

• DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195377033.001.0001

• Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-357. https://doi.org/10.1037/h0061470

• Grayling, A. (2017). Future education and skills: Education 2030: Reflections on transformative competencies 2030. Organization for Economic Co-operation and Development. https://www.oecd.org/education/2030-project/contact/EDU-EDPC(2017)16-ANN5.pdf

• Halpern, D. F., & Dunn, D. S. (2021). Critical thinking: A model of intelligence for solving real-world problems. The Journal of Intelligence, 9(2), Article 22. https://doi.org/10.3390/jintelligence9020022

• Haste, H. (2001). Ambiguity, autonomy, and agency: Psychological challenges to new competence. In D. Rychen & L. Salganik (Eds.), Defining and selecting competencies (pp. 93-120). Hogrefe Publishing. https://api.semanticscholar.org/CorpusID:142385997

• Hipkins, R. (2012). The engaging nature of teaching for competency development. In S. L. Christenson, A. L. Reschly & C. Wylie (Eds.), Handbook of research on student engagement (pp. 441-456). Springer. https://doi.org/10.1007/978-1-4614-2018-7_21

• Junpeng, P., Marwiang, M., Chiajunthuk, S., Suwannatrai, P., Chanayota, K., Pongboriboon, K., Tang, K. N., & Wilson, M. (2020). Validation of a digital tool for diagnosing mathematical proficiency. International Journal of Evaluation and Research in Education, 9(3), 665-674. https://doi.org/10.11591/ijere.v9i3.20503

• Kesorn, N., Junpeng, P., Marwiang, M., Pongboriboon, K., Tang, K. N., Bathia, S., & Wilson, M. (2020). Development of an assessment tool for mathematical reading, analytical thinking and mathematical writing. International Journal of Evaluation and Research in Education, 9(4), 955-962. https://doi.org/10.11591/ijere.v9i4.20505

• Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1964). Taxonomy of educational objectives: Handbook II: The affective domain. David McKay Publications https://scholar.google.com/scholar_lookup?title=Taxonomy+of+educational+objectives%3A+The+classification+of+educational+goals.+Handbook+II%3A+Affective+domain&author=DR+Krathwohl&author=BS+Bloom&author=BB+Masia&publication_year=1964

• Laukonnen, R., Biddel, H., & Gallagher, R. (2018). Preparing humanity for change and artificial intelligence: Learning to learn as a safeguard against volatility, uncertainty, complexity and ambiguity. Organization for Economic Co-operation and Development. https://doi.org/10.31234/osf.io/g5qwc

• Lohse-Bossenz, H., Kunina-Habenicht, O., & Kunter, M. (2013). The role of educational psychology in teacher education: Expert opinions on what teachers should know about learning, development, and assessment. European Journal of Psychology of Education, 28(4), 1543-1565. https://doi.org/10.1007/s10212-013-0181-6

• Masters, G. N., & Wright, B. D. (1997). The partial credit model. In W. J. van der Linden & R. K. Hambleton (Eds.). Handbook of modern item response theory. Springer. https://doi.org/10.1007/978-1-4757-2691-6_6

• Mezirow, J. (2003). Transformative learning as discourse. Journal of Transformative Education, 1(1), 58-63. https://doi.org/10.1177/1541344603252172

• Nägele, C., & Stalder, B. E. (2017). Competence and the need for transferable skills. In M. Mulde (Ed.), Competence-based vocational and professional education: Bridging the worlds of work and education (pp.739-753). Springer. https://doi.org/10.1007/978-3-319-41713-4

• Nussbaum, M. (1997). Cultivating humanity: A classical defense of reform in liberal education. Harvard University Press. https://doi.org/10.2307/j.ctvjghth8

• Organization for Economic Cooperation and Development (OECD). (2017). Education 2030 - Conceptual learning framework: Background papers. The future of education and skills: Education 2030. https://one.oecd.org/document/EDU/EDPC(2017)25/ANN3/en/pdf

• Organization for Economic Cooperation and Development. (2018). OECD future of education and skills 2030 https://www.oecd.org/education/2030?E2030%20Position%20Paper%(05.04.2018).pdf.

• Organization for Economic Cooperation and Development. (n. d.). Evaluation and assessment frameworks for improving school outcomes: Common policy challenges. https://www.oecd.org/education/school/46927511.pdf

• Padilla, J. L., & Leighton, J. P. (2017). Cognitive interviewing and think aloud methods. In B. Zumbo & A. Hubley (Eds.), Understanding and investigating response processes in validation research (pp 11-228). Springer. https://doi.org/10.1007/978-3-319-56129-5_12

• Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danmarks Paedagogiske Institut. https://books.google.co.th/books?id=aB9qLgEACAAJ

• Reeves, T. C. (2006). Design research from a technology perspective. In J. V. D. Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research (pp. 52-66). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203088364-13/design-research-technology-perspective-thomas-reeves

• Stodden, D. F., Pesce, C., Zarrett, N., Tomporowski, P., Ben-Soussan, T. D., Brian, A., Abrams, T. C., & Weist, M. D. (2023). Holistic functioning from a developmental perspective: A new synthesis with a focus on a multi-tiered system support structure. Clinical Child and Family Psychology Review, 26(2), 343-361. https://doi.org/10.1007/s10567-023-00428-5

• Vongvanich, S. (2020). Design research in education. Chulalongkorn University Printing House. https://www.chulabook.com/education/102930

• Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. National Institute for Science Education. https://eric.ed.gov/?id=ED414305

• Wilson, M. R. (2005). Constructing measures: An item response modeling approach. Routledge. https://doi.org/10.4324/9781410611697

• Wilson, M., Allen, D. D., & Li, J. C. (2006). Improving measurement in health education and health behavior research using item response modelling: Comparison with the classical test theory approach. Health Education Research, 2(1), i19-i32. https://doi.org/10.1093/her/cyl053

• Wolff, J. (2015). Political philosophy and the real world of the welfare state. Journal of Applied Philosophy, 32(4), 360-372.

• Yazon, A. D., Ang-Manaig, K., Buama, C. A. C., & Tesoro, J. F. B. (2019). Digital literacy, digital competence and research productivity of educators. Universal Journal of Educational Research, 7(8), 1734-1743. https://doi.org/10.13189/ujer.2019.070812

• Zajda, J. (2021). Constructivist learning theory and creating effective learning environments. In J. Zajda (Ed.), Globalisation and education reforms: Creating effective learning environments (pp. 35-50). Springer. https://doi.org/10.1007/978-3-030-71575-5_3

• Adams, R. J. (2005). Reliability as a measurement design effect. Studies in Educational Evaluation, 31(2-3), 162-172. https://doi.org/10.1016/j.stueduc.2005.05.008

• Adams, R. J., Wu, M. L., Cloney, D., Berezner, A., & Wilson, M. (2020). ACER ConQuest: Generalised Item Response Modelling Software (Version 5.29) [Computer software]. Australian Council for Educational Research. https://www.acer.org/au/conquest

• Adams, R., & Khoo, S. (1996). Quest: Interactive test analysis system version 2.1. The Australian council for educational research. https://eric.ed.gov/?d=ED362553

• Al-Alawneh, M. K., Hawamleh, M. S., Al-Jamal, D. A., & Sasa, G. S. (2019). Communication skills in practice. International Journal of Learning, Teaching and Educational Research, 18(6), 1-19. https://doi.org/10.26803/ijlter.18.6.1

• American Educational Research Association. (2014). Standards for educational and psychological testing (6th ed). https://www.testingstandards.net/uploads/7/6/6/4/76643089/9780935302356.pdf

• Anakttototy, K., Sekawael, M., Latief, M. R. A., & Bin-Hady, W. R. A. (2023). Beyond linguistics: Exploring the cognitive and motivational barriers to essay writing for tertiary students. International Journal of Language Education, 7(3), 447-468. https://doi.org/10.26858/ijole.v7i3.37070

• Baker, F. B., & Kim, S. (2017). The basics of item response theory using R. Springer. https://link.springer.com/book/10.1007/978-3-319-54205-8

• Canto-Sperber, M., & Depuy, J. P. (2001). Competencies for good life and good society. In D. Rychen & L. Salganik (Eds.), Defining and selecting competencies (pp. 67-92). Hogrefe Publishing.

• DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195377033.001.0001

• Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-357. https://doi.org/10.1037/h0061470

• Grayling, A. (2017). Future education and skills: Education 2030: Reflections on transformative competencies 2030. Organization for Economic Co-operation and Development. https://www.oecd.org/education/2030-project/contact/EDU-EDPC(2017)16-ANN5.pdf

• Halpern, D. F., & Dunn, D. S. (2021). Critical thinking: A model of intelligence for solving real-world problems. The Journal of Intelligence, 9(2), Article 22. https://doi.org/10.3390/jintelligence9020022

• Haste, H. (2001). Ambiguity, autonomy, and agency: Psychological challenges to new competence. In D. Rychen & L. Salganik (Eds.), Defining and selecting competencies (pp. 93-120). Hogrefe Publishing. https://api.semanticscholar.org/CorpusID:142385997

• Hipkins, R. (2012). The engaging nature of teaching for competency development. In S. L. Christenson, A. L. Reschly & C. Wylie (Eds.), Handbook of research on student engagement (pp. 441-456). Springer. https://doi.org/10.1007/978-1-4614-2018-7_21

• Junpeng, P., Marwiang, M., Chiajunthuk, S., Suwannatrai, P., Chanayota, K., Pongboriboon, K., Tang, K. N., & Wilson, M. (2020). Validation of a digital tool for diagnosing mathematical proficiency. International Journal of Evaluation and Research in Education, 9(3), 665-674. https://doi.org/10.11591/ijere.v9i3.20503

• Kesorn, N., Junpeng, P., Marwiang, M., Pongboriboon, K., Tang, K. N., Bathia, S., & Wilson, M. (2020). Development of an assessment tool for mathematical reading, analytical thinking and mathematical writing. International Journal of Evaluation and Research in Education, 9(4), 955-962. https://doi.org/10.11591/ijere.v9i4.20505

• Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1964). Taxonomy of educational objectives: Handbook II: The affective domain. David McKay Publications https://scholar.google.com/scholar_lookup?title=Taxonomy+of+educational+objectives%3A+The+classification+of+educational+goals.+Handbook+II%3A+Affective+domain&author=DR+Krathwohl&author=BS+Bloom&author=BB+Masia&publication_year=1964

• Laukonnen, R., Biddel, H., & Gallagher, R. (2018). Preparing humanity for change and artificial intelligence: Learning to learn as a safeguard against volatility, uncertainty, complexity and ambiguity. Organization for Economic Co-operation and Development. https://doi.org/10.31234/osf.io/g5qwc

• Lohse-Bossenz, H., Kunina-Habenicht, O., & Kunter, M. (2013). The role of educational psychology in teacher education: Expert opinions on what teachers should know about learning, development, and assessment. European Journal of Psychology of Education, 28(4), 1543-1565. https://doi.org/10.1007/s10212-013-0181-6

• Masters, G. N., & Wright, B. D. (1997). The partial credit model. In W. J. van der Linden & R. K. Hambleton (Eds.). Handbook of modern item response theory. Springer. https://doi.org/10.1007/978-1-4757-2691-6_6

• Mezirow, J. (2003). Transformative learning as discourse. Journal of Transformative Education, 1(1), 58-63. https://doi.org/10.1177/1541344603252172

• Nägele, C., & Stalder, B. E. (2017). Competence and the need for transferable skills. In M. Mulde (Ed.), Competence-based vocational and professional education: Bridging the worlds of work and education (pp.739-753). Springer. https://doi.org/10.1007/978-3-319-41713-4

• Nussbaum, M. (1997). Cultivating humanity: A classical defense of reform in liberal education. Harvard University Press. https://doi.org/10.2307/j.ctvjghth8

• Organization for Economic Cooperation and Development (OECD). (2017). Education 2030 - Conceptual learning framework: Background papers. The future of education and skills: Education 2030. https://one.oecd.org/document/EDU/EDPC(2017)25/ANN3/en/pdf

• Organization for Economic Cooperation and Development. (2018). OECD future of education and skills 2030 https://www.oecd.org/education/2030?E2030%20Position%20Paper%(05.04.2018).pdf.

• Organization for Economic Cooperation and Development. (n. d.). Evaluation and assessment frameworks for improving school outcomes: Common policy challenges. https://www.oecd.org/education/school/46927511.pdf

• Padilla, J. L., & Leighton, J. P. (2017). Cognitive interviewing and think aloud methods. In B. Zumbo & A. Hubley (Eds.), Understanding and investigating response processes in validation research (pp 11-228). Springer. https://doi.org/10.1007/978-3-319-56129-5_12

• Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danmarks Paedagogiske Institut. https://books.google.co.th/books?id=aB9qLgEACAAJ

• Reeves, T. C. (2006). Design research from a technology perspective. In J. V. D. Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research (pp. 52-66). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203088364-13/design-research-technology-perspective-thomas-reeves

• Stodden, D. F., Pesce, C., Zarrett, N., Tomporowski, P., Ben-Soussan, T. D., Brian, A., Abrams, T. C., & Weist, M. D. (2023). Holistic functioning from a developmental perspective: A new synthesis with a focus on a multi-tiered system support structure. Clinical Child and Family Psychology Review, 26(2), 343-361. https://doi.org/10.1007/s10567-023-00428-5

• Vongvanich, S. (2020). Design research in education. Chulalongkorn University Printing House. https://www.chulabook.com/education/102930

• Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. National Institute for Science Education. https://eric.ed.gov/?id=ED414305

• Wilson, M. R. (2005). Constructing measures: An item response modeling approach. Routledge. https://doi.org/10.4324/9781410611697

• Wilson, M., Allen, D. D., & Li, J. C. (2006). Improving measurement in health education and health behavior research using item response modelling: Comparison with the classical test theory approach. Health Education Research, 2(1), i19-i32. https://doi.org/10.1093/her/cyl053

• Wolff, J. (2015). Political philosophy and the real world of the welfare state. Journal of Applied Philosophy, 32(4), 360-372.

• Yazon, A. D., Ang-Manaig, K., Buama, C. A. C., & Tesoro, J. F. B. (2019). Digital literacy, digital competence and research productivity of educators. Universal Journal of Educational Research, 7(8), 1734-1743. https://doi.org/10.13189/ujer.2019.070812

• Zajda, J. (2021). Constructivist learning theory and creating effective learning environments. In J. Zajda (Ed.), Globalisation and education reforms: Creating effective learning environments (pp. 35-50). Springer. https://doi.org/10.1007/978-3-030-71575-5_3