Vol. 15 No. 1 (2024): MEDIA EDUCATION – Studi, ricerche e buone pratiche
Articles

Assessing the assessments: toward a multidimensional approach to AI literacy

PDF
  • Gabriele Biagini
Gabriele Biagini
University of Florence, Florence
DOI: https://doi.org/10.36253/me-15831

Published 2024-06-12

Keywords

  • AI Literacy,
  • Self-Perception,
  • Scale,
  • Questionnaire

Copyright (c) 2024 Gabriele Biagini

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

This scoping review explores the field of artificial intelligence (AI) literacy, focusing on the tools available for evaluating individuals’ self-perception of their AI literacy. In an era where AI technologies increasingly infiltrate various aspect of daily life, from healthcare diagnostics to personalized digital platforms, the need for a comprehensive understanding of AI literacy has never been more critical. This literacy extends beyond mere technical competence to include ethical considerations, critical thinking, and socio-emotional skills, reflecting the complex interplay between AI technologies and societal norms. The review synthesizes findings from diverse studies, highlighting the development and validation processes of several key instruments designed to measure AI literacy across different dimensions. These tools – ranging from the Artificial Intelligence Literacy Questionnaire (AILQ) to the General Attitudes towards Artificial Intelligence Scale (GAAIS) – embody the nature of AI literacy, encompassing affective, behavioral, cognitive, and ethical components. Each instrument offers unique insights into how individuals perceive their abilities to understand, engage with, and ethically apply AI technologies. By examining these assessment tools, the review sheds light on the current landscape of AI literacy measurement, underscoring the importance of self-perception in educational strategies, personal growth, and ethical decision-making. The findings suggest a critical need for educational interventions and policy formulations that address the gaps between perceived and actual AI literacy, promoting a more inclusive, critically aware, and competent engagement with AI technologies.

PDF

References

  1. Bandura, A. (1997) Self-Efficacy: The Exercise of Control. W.H. Freeman and Company, New York.
  2. Biagini, G., Cuomo, S., & Ranieri, M. (2023). Developing and validating a multidimensional AI literacy questionnaire: Operationalizing AI literacy for higher education. Proceedings of the First International Workshop on High-performance Artificial Intelligence Systems in Education Rome, Italy, Novembre 6, 2023.
  3. Calvani, A., Fini, A., & Ranieri, M. (2009). Assessing Digital Competence in Secondary Education. Issues, Models and Instruments. Issues in Information and Media Literacy: Education, Practice and Pedagogy, 153–172.
  4. Carolus, A., Koch, M.J., Straka, S., Latoschik, M.C., Wienrich, C. (2023). MAILS – Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Computers in Human Behavior: Artificial Humans, Volume 1, Issue 2, pp. 1-10. DOI: https://doi.org/10.1016/j.chbah.2023.100014
  5. Chai, C. S., Lin, P. Y., Jong, M. S. Y., Dai, Y., Chiu, T. K., & Huang, B. (2020). Factors influencing students’ behavioral intention to continue artificial intelligence learning. In 2020 International Symposium on Educational Technology (ISET), pp. 147–150. DOI: https://doi.org/10.1109/ISET49818.2020.00040
  6. Chiu, T. K., & Chai, C. S. (2020). Sustainable curriculum planning for artificial intelligence education: A self-determination theory perspective. Sustainability, 12(14), 1-18. DOI: https://doi.org/10.3390/su12145568
  7. Chiu, T. K., Meng, H., Chai, C. S., King, I., Wong, S., & Yam, Y. (2021). Creation and evaluation of a pretertiary artificial intelligence (AI) curriculum. IEEE Transactions on Education, 65(1), 30–39. https://doi.org/10.1109/ TE.2021.3085878 DOI: https://doi.org/10.1109/TE.2021.3085878
  8. Cuomo, S., Biagini, G., & Ranieri, M. (2022). Artificial Intelligence Literacy, che cos’è e come promuoverla. Dall’analisi della letteratura ad una proposta di Framework. Media Education. https://doi.org/10.36253/me-13374. DOI: https://doi.org/10.36253/me-13374
  9. Cooper, H., Hedges, L.V., & Valentine, J.C. (2019). The Handbook of Research Synthesis and Meta-Analysis. (3rd edition. ed.). Russell Sage Foundation. DOI: https://doi.org/10.7758/9781610448864
  10. Dai, Y., Chai, C. S., Lin, P. Y., Jong, M. S. Y., Guo, Y., & Qin, J. (2020). Promoting students’ well-being by developing their readiness for the artificial intelligence age. Sustainability, 12(16). https://doi.org/10.3390/su12166597. DOI: https://doi.org/10.3390/su12166597
  11. DiSalvo, B., Yip, J., Bonsignore, E., & Carl, D. (2017). Participatory design for learning. In Participatory design for learning, pp. 3-6. Routledge. DOI: https://doi.org/10.4324/9781315630830-2
  12. Druga, S., Vu, S. T., Likhith, E., & Qiu, T. (2019). Inclusive AI literacy for kids around the world. Proceedings of FabLearn 2019, pp. 104–111. https://doi.org/10.1145/3311890.3311904 DOI: https://doi.org/10.1145/3311890.3311904
  13. Druga, S., Christoph, F. L., & Ko, A. J. (2022). Family as a Third Space for AI Literacies: How do children and parents learn about AI together? CHI Conference on Human Factors in Computing Systems, pp. 1–17. https://doi.org/10.1145/3491102.3502031 DOI: https://doi.org/10.1145/3491102.3502031
  14. Kandlhofer, M., Steinbauer, G., Hirschmugl-Gaisch, S., & Huber, P. (2016). Artificial Intelligence and Computer Science in Education: From Kindergarten to University. 2016 IEEE Frontiers in Education Conference (FIE). DOI: https://doi.org/10.1109/FIE.2016.7757570
  15. Kong, S.-C., Cheung, W. M.-Y., & Tsang, O. (2022). Evaluating an artificial intelligence literacy programme for empowering and developing concepts, literacy and ethical awareness in senior secondary students. Education and Information Technologies. https://doi.org/10.1007/s10639-022-11408-7. DOI: https://doi.org/10.1007/s10639-022-11408-7
  16. Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2022). Evaluating artificial intelligence literacy courses for fostering conceptual learning, literacy and empowerment in university students: Refocusing to conceptual building. Computers in Human Behavior Reports, 7, 100223. https://doi.org/10.1016/j.caeai.2021.100026 DOI: https://doi.org/10.1016/j.chbr.2022.100223
  17. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121. DOI: https://doi.org/10.1037//0022-3514.77.6.1121
  18. Laupichler, M. C., Aster, A., & Raupach, T. (2023). Delphi study for the development and preliminary validation of an item set for the assessment of non-experts’ AI literacy. Computers and Education: Artificial Intelligence, 4, 100126. https://doi.org/10.1016/j.caeai.2023.100126. DOI: https://doi.org/10.1016/j.caeai.2023.100126
  19. Lin, C.-H., Yu, C.-C., Shih, P.-K., & Wu, L. Y. (2021). STEM-based Artificial Intelligence Learning in General Education for Non- Engineering Undergraduate Students. 15.
  20. Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–16. https://doi.org/10.1145/3313831.3376727. DOI: https://doi.org/10.1145/3313831.3376727
  21. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. & The PRISMA Group. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. DOI: https://doi.org/10.1371/journal.pmed.1000097
  22. Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041. DOI: https://doi.org/10.1016/j.caeai.2021.100041
  23. Ng, D. T. K., Leung, J. K. L., Su, M. J., Yim, I. H. Y., Qiao, M. S., & Chu, S. K. W. (2022). AI Literacy in K-16 Classrooms. Springer International Publishing. https://doi.org/10.1007/978-3-031-18880-0 DOI: https://doi.org/10.1007/978-3-031-18880-0
  24. Ng, D. T. K., Wu, W., Leung, J. K. L., & Chu, S. K. W. (2023). Artificial Intelligence (AI) literacy questionnaire with confirmatory factor analysis. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT), pp. 233–235. IEEE. https://doi.org/10.1109/ICALT58122.2023.00074
  25. Pinski, M., & Benlian, A. (2023). Ai literacy – towards measuring human competency in artificial intelligence. In Proceedings of the 56th Hawaii international conference on system sciences, pp. 165–174. Maui, USA: IEEE Computer Society Press. DOI: https://doi.org/10.24251/HICSS.2023.021
  26. Saracco, R. (2020). Understanding Digital Transformation: The Key Concepts. IEEE Communications Society.
  27. Schepman, A., & Rodway, P. (2020). Initial validation of the general attitudes towards Artificial Intelligence Scale. Computers in Human Behavior Reports, 1, 100014. https://doi.org/10.1016/j.chbr.2020.100014. DOI: https://doi.org/10.1016/j.chbr.2020.100014
  28. Sindermann, C., Sha, P., Zhou, M., Wernicke, J., Schmitt, H. S., Li, M., & Montag, C. (2021). Assessing the attitude towards artificial intelligence: Introduction of a short measure in German, Chinese, and English Language. KI – Künstliche Intelligenz, 35(1), 109–118. DOI: https://doi.org/10.1007/s13218-020-00689-0
  29. Southworth, J., Migliaccio, K., Glover, J., Reed, D., McCarty, C., Brendemuhl, J., & Thomas, A. (2023). Developing a model for AI across the curriculum: Transforming the higher education landscape via innovation in AI literacy. Computers and Education: Artificial Intelligence, 4, 100127. https://doi.org/10.1016/j.caeai.2023.100127 DOI: https://doi.org/10.1016/j.caeai.2023.100127
  30. United Nations Educational, Scientific and Cultural Organization. (2021). AI and education: Guidance for policy makers. https://unesdoc.unesco.org/ark: /48223/pf0000376709.
  31. Wang, B., Rau, P. L. P., & Yuan, T. (2022). In Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour and Information Technology. https://doi.org/10.1080/0144929X.2022.2072768. DOI: https://doi.org/10.1080/0144929X.2022.2072768
  32. Wang, Y. Y., & Wang, Y. S. (2022). Development and validation of an artificial intelligence anxiety scale: An initial application in predicting motivated learning behavior. Interactive Learning Environments, 30(4), 619–634. https://doi.org/10.1080/10494820.2019.1674887. DOI: https://doi.org/10.1080/10494820.2019.1674887
  33. Zhang, H., Lee, I., Ali, S., DiPaola, D., Cheng, Y., & Breazeal, C. (2022). Integrating Ethics and Career Futures with Technical Learning to Promote AI Literacy for Middle School Students: An Exploratory Study. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-022-00293-3 DOI: https://doi.org/10.1007/s40593-022-00293-3