Journal of New Advances in Educational Management

Journal of New Advances in Educational Management

Identifying factors affecting digital formative evaluation in e-learning and teaching based on the transcendental method

Document Type : Original Article

Authors
1 PhD student, Department of Educational Management, Faculty of Humanities, Islamic Azad University, Pardisan Branch, Qom
2 Associate Professor, Department of Educational Management, Faculty of Management and Accounting, Farabi College, Qom, Iran
3 Department of Educational Management, Faculty of Humanities, Islamic Azad University of Qom, Qom, Iran
4 Assistant Professor, Department of Educational Management, Faculty of Humanities, Islamic Azad University of Qom, Qom, Iran
Abstract
The lack of correct mechanisms of assessment and evaluation of education in e-learning has caused managers to implement fleeting strategies and programs with additional costs.   This is the result of the lack of proper identification of evaluation components in e-learning and training, and the digital formative evaluation process helps e-learning and training management systems to adopt appropriate strategies and apply the digital formative evaluation process to meet the main objective of learning, despite various models and frameworks in the field of e-learning and training so far no comprehensive model has been presented. It is necessary to consider the multiple dimensions of academic assessment and evaluation as an integrated one for achieving the same truly evaluation for learning.  In this paper, in order to explain a comprehensive model, by systematic review of literature and using meta-classical qualitative research method, all dimensions of digital formative evaluation have been identified, then the level of importance and firstly, The nature of each of the effective factors has been determined based on the method of content analysis using Shannon's quantitative entropy method, so that this research has innovation in both methodology and results
Keywords

Aldon, G., Cusi, A., Morselli, F., Panero, M., & Sabena, C. (2015). Which support technology can give to mathematics formative assessment? The FaSMEd project in Italy and France. Quaderni di Ricerca in Didattica (Mathematics), 25, 631-641.
Alexander, C., Ishikawa, S. & Silverstein, M. (1977). A pattern language: towns, buildings, construction. New York, NY: Oxford University Press.
Alizadeh, Shahnaz; Salehi, Keyvan; Moghadamzadeh, Ali (2016). Analyzing the quality of classroom assessment of teachers; A mixed research study. Quarterly journal of research in school and virtual learning, number 17, year 5, page 63-84.
American Educational Research Association; Joint Committee on Standards for Educational and Psychological Testing (U.S.); American Psychological Association; National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
ARG (Assessment Reform Group). (2002). Assessment for learning: Ten principles. http://www.hkeaa.edu.hk/DocLibrary/ SBA/HKDSE/Eng_DVD/doc/Afl_principles.pdf Accessed 4 February, 2020.
ATC21S. (2014). Assessment and teaching of 21st century skills: Collaborative problem solving empirical progressions.http://www.atc21s.org/uploads/3/7/0/0/37007163/collaborative_problem_solving_emprical_progressions_v1.1.pdf Accessed 13 February, 2020.
Baker, R.S., Goldstein, A.B., & Heffernan, N.T. (2011). Detecting learning moment by moment. International Journal of Artificial Intelligence in Education, 21(1-2), 5-25.
Barrett, B.S., Moran, A.L., & Woods, J.E. (2014). Meteorology meets engineering: An interdisciplinary STEM module for middle and early secondary school students. International Journal of STEM Education, 1(6).
Beesley, A.D., Clark, T.F., Dempsey, K., & Tweed, A. (2018). Enhancing formative assessment practice and encouraging middle school mathematics engagement and persistence. School Science & Mathematics, 118(1), 4-16.
Bennett, R.E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy, and Practice, 18(1), 5-25.
Bergin, J., Kohls, C., Köppe, C., Mor, Y., Portier, M., Schümmer, T., & Warburton, S. (2015). Assessment-Driven Course Design - Fair Play Patterns. In Proceedings of the 22nd Conference on Pattern Languages of Programs. The Hillside Group.
Bicer, A., Capraro, R.M., & Capraro, M.M. (2017). Integrated STEM assessment model. EURASIA Journal of Mathematics Science and Technology Education, 13(7), 3959-3968.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: principles, policy & practice, 5(1), 7-74.
Black, P., & Wiliam, D. (2018). Classroom assessment and pedagogy. Assessment in Education: principles, policy, & practice, 25(6), 551-575.
Burkhardt, H., & Schoenfeld, A. (2019). Formative assessment in mathematics. In H. Andrade, R.E. Bennett, & G. Cizek (Eds.), Handbook of formative assessment in the disciplines (pp. 35-67). New York, NY: Routledge.
Burns, M.K., Klingbeil, DA., & Ysseldyke, J. (2010). The effects of technology-enhanced formative evaluation on student performance on state accountability math tests. Psychology in the Schools, 47(6), 582-591.
Canty, D., Seery, N., Hartell, E., & Doyle, A. (2017). Integrating peer assessment in technology education through adaptive comparative judgement. Paper presented at Pupils’ Attitudes Toward Technology (PATT-34) Conference. Philadelphia, PA.
Care, E., Griffin, P., & Wilson, M. (Eds.). (2018). Assessment and teaching of 21st century skills. Research and applications. Dordrecht: Springer.
Cherner, T., & Smith, D. (2017). Reconceptualizing TPACK to meet the needs of twenty-first-century education. The New Educator, 13(4), 329-349.
Cowie, B., Moreland, J., & Otrel-Cass, K. (2013). Expanding notions of assessment for learning inside science and technology primary classrooms. Rotterdam, The Netherlands: Sense Publishers.
Crompton, H., Burke, D., & Lin, Y. (2019). Mobile learning and student cognition: A systematic review of PK-12 research using Bloom’s Taxonomy. British Journal of Educational Technology, 50(2), 684–701.
Czaja, S. J., Charness, N., Fisk, A. D., Hertzog, C., Nair, S. N., Rogers, W. A., & Sharit, J. (2006). Factors predicting the use of technology: findings from the Center for Research and Education on Aging and Technology Enhancement (CREATE). Psychology and Aging, 21(2), 333–352.
Decristan, J., Klieme, E., Kunter, M., Hochweber, J. Büttner , G., Fauth, B., Hondrich, A.L., Rieser, S., & Hardy, I. (2015). Embedded formative assessment and classroom process quality: How do they interact in promoting science understanding? American Educational Research Journal, 52(6), 1133-1159.
Dolin J., Black P., Harlen W., & Tiberghien A. (2018). Exploring relations between formative and summative assessment. In: Dolin J., Evans R. (Eds.), Transforming assessment through an interplay between practice, research and policy. Contributions from science education research (pp. 53-80). Cham, Switzerland: Springer.
Dolin, J., & Evans, R. (Eds.). (2018). Transforming assessment through an interplay between practice, research and policy.Contributions from science education research. Cham, Switzerland: Springer.
Dukuzumuremyi, S., & Siklander, P. (2018). Interactions between pupils and their teachers in collaborative and technologyenhanced learning settings in the inclusive classroom. Teaching and Teacher Education, 76, 165-174.
Dunn, K.,. & Mulvenon, S. (2009). A critical review of research on formative assessment: The limited scientific evidence of the impact of formative assessment in education. Practical Assessment, Research, & Evaluation, 14(7), 1-11.
Duschl, R. (2019). Learning progressions: framing and designing coherent sequences for STEM Education. Disciplinary and Interdisciplinary Science Education Research, 1(4). https://doi.org/10.1186/s43031-019-0005-x Accessed 4 February, 2020.
Economou, A. (2018). ATS2020-Assessment of transversal skills: Reflections and policy recommendations on transversal skills development and assessment. http://www.ats2020.eu/images/documents/ATS2020-Reflections.pdf Accessed 13 February, 2020.
ETS (Educational Testing Service) (nd.). What are learning progressions and why are they important? https://news.ets.org/ stories/what-are-learning-progressions-and-why-are-they-important/ Accessed 4 February, 2020.
European Commission. (2016). FaSMEd summary report. https://cordis.europa.eu/docs/results/612/612337/final1-finalfasmed- summary-report-final.pdf Accessed 4 February, 2020.
Faber, J.M., Luyten, H., & Visscher, A.J. (2017). The effects of a digital formative assessment tool on mathematics achievement and student motivation: Results of a randomized experiment. Computers & Education, 106, 83-96.
Feldman, A., & Capobianco, B.M. (2008). Teacher learning of technology enhanced formative assessment. Journal of Science Education Technology, 17, 82-99.
Finlayson ,O., & McLoughlin, E. (2017) Building teacher confidence in inquiry and assessment: Experiences from a Pan- European Collaboration In M. Peters, B. Cowie, & I. Menter (Eds.). A companion to research in teacher education (pp. 825-838). Singapore: Springer.
Furtak, E.M., Heredia, S.C., & Morrison, D. (2019). Formative assessment in science education: Mapping a shifting terrain. In H. Andrade, R.E. Bennett, & G. Cizek (Eds.). Handbook of formative assessment in the disciplines (pp. 97-125). New York, NY: Routledge.
Gail Morreim, J. (2016). How Digital Formative Assessment Increases student Achievment and Motivation. Saint Paul.Minnesota: Hamline University.
Geer, R., White, B., Zeegers, Y., Au, W., & Barnes, A. (2017). Emerging pedagogies for the use of iPads in schools. British Journal of Educational Technology, 48(2), 490-498.
Gholtash, Abbas; Ojinejad, Ahmadreza; Dehghan Mangabadi, Alireza (2014). Pathology of the descriptive evaluation model in order to provide a suitable model in the elementary school. Quarterly Journal of Research in Educational and Virtual Learning, 10th issue, 3rd year, page 7-16.
Griffin, P., & Care, E. (Eds.). (2015). Assessment and teaching of 21st century skills: Methods and approaches. Dordrecht: Springer.
Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st century skills. Dordrecht: Springer.
Grundschober, I., Ghoneim, A., Baumgartner, P., & Gruber-Mücke, T. (2018). A pattern language remix for ATS2020. Using existing pedagogical patterns to create a new language for formative assessment within the ATS2020 learning model.
Harris, J., Phillips, M., Koehler, M., & Rosenberg, J. (2017). TPCK/TPACK research and development: Past, present, and future directions. Australasian Journal of Educational Technology, 33(3), i-viii.
Haßler, B., Major, L., & Hennessy, S. (2016). Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32(2), 139-156.
Hatlevik, O. E., Guðmundsdóttir, G. B., & Loi, M. (2015). Examining factors predicting students’ digital competence. Journal of  Information Technology Education: Research, 14, 123-137.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.
Hayward, L. (2015). Assessment is learning: the preposition vanishes. Assessment in Education: principles, policy & practice, 22(1), 27-43.
Hickey, D.T., Taasoobshirazi, G., & Cross, D. (2012). Assessment as learning: enhancing discourse, understanding, and achievement in innovative science curricula. Journal of Research in Science Teaching, 49(10), 1240-1270.
Hmelo-Silver, C.E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235-266.
Hondrich, A.L., Decristan, J., Hertel, S., & Klieme, E. (2018). Formative assessment and intrinsic motivation: The mediating role of perceived competence. Zeitschrift für Erziehungswissenschaft, 21, 717-734.
Hooker, T. (2017). Transforming teachers’ formative assessment practices through ePortfolios. Teaching and Teacher Education, 67, 440-453.
http:// www.hillside.net/plop/2015/papers/panthers/4.pdf Accessed 13 February, 2020.
In R. Sickinger, P. Baumgartner, & T. Gruber-Mücke (Eds.). Pursuit of pattern languages for societal change. A comprehensive perspective of current pattern research and practice (pp. 288–317). Krems: Tredition.
Jennings, L.B. (2010). Inquiry-based learning. In T.C. Hunt, J.C. Carper, T.J. Lasley II, & D. Raisch. Encyclopedia of Educational Reform and Dissent. Thousand Oaks, CA: SAGE Publications.
Jönsson, A. (2020). Definitions of formative assessment need to make a distinction between a psychometric understanding of assessment and “evaluate judgements.” Frontiers In Education, 5(2), 1-4.
Khanifar, Hossein; (Spring 2015). Designing an entrepreneurial process model in Iran's food industry. Entrepreneurship Development, 9th, 219-237.
Khodadahosseini, Seyyed Hamid. (Spring 2013). Designing an entrepreneurial branding process model in small and medium businesses in the food industry. Brand Management Quarterly, 1, 13-45.
Kimbell, R. (2012). Evolving project e-scape for national assessment. International Journal of Technology and Design Education, 22(2), 135-155.
Kimbell, R., Wheeler, T., Miller, S., & Pollitt, A. (2007). E-scape portfolio assessment: Phase 2 report. Goldsmiths, University of London.
Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and call for research. Educational Measurement: Issues and Practice, 30(4), 28-37.
Kippers, W.B., Wolterinck, C.H.D., Schildkamp, K., Poortman, C.L., & Visscher, A.J. (2018). Teachers’ views on the use of assessment for learning and data-based decision making in classroom practice. Teaching and Teacher Education, 75, 199-213.
Kirschner P.A., Sweller, J., & Clark, R.E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86.
Kirschner, P.A., & De Bruyckere, P. (2017). The myths of the digital natives and the multitasker. Teaching and Teacher Education, 67, 135-142.
Knowles, T.R., & Kelley, J.G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3(11), 1-11.
Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70.
Laal, M. (2013). Collaborative learning; elements. Social and Behavioral Sciences, 83, 814-818.
Lai, C. (2019). Trends of mobile learning: A review of the top 100 highly cited papers. British Journal of Educational Technology, 50. doi:10.1111/bjet.12884 Accessed 4 February, 2020
Larson, K.A., Trees, F.P., & Weaver, D.S. (2008). Continuous Feedback Pedagogical Patterns. In PLoP’08 Proceedings of the 15th Conference on Pattern Languages of Programs. New York, NY: ACM. Retrieved from: http://hillside.net/plop/2008/ papers/PLoP2008_26_Larson+Trees+Weaver.pdf
Lashkar-Blouki, (fall and winter 2011). Designing a sustainable strategy process model using a hybrid approach. Strategic Management Thought, 6th, 121-151.
Lee, H., Feldman, A., & Beatty, I.D. (2012). Factors that affect science and mathematics teachers’ initial implementation of technology-enhanced formative assessment using a classroom response system. Journal of Science Education Technology, 21, 523-539.
Looney, J. (2019). Digital formative assessment: A review of the literature.
Looney, J. (2019). Digital formative assessment: A review of the literature. http://www.eun.org/documents/411753/817341/ Assess%40Learning+Literature+Review/be02d527-8c2f-45e3-9f75-2c5cd596261d Accessed 4 December, 2019.
Lysaght, Z., & O’Leary, M. (2017). Scaling up, writ small: using an assessment for learning audit instrument to stimulate site-based professional development, one school at a time. Assessment in Education: Principles, Policy & Practice, 24(2), 271-289.
Maier, W., Wolfe, N., & Randler, C. (2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education, 95, 85-98.
Marjan Faber, J. (2020). Effect of digital formative assessment tools on teaching quality and student achievement. PhD Thesis, University of Twente.
Martin, M.O., Mullis, I.V.S., & Foy, P. (2017). TIMSS 2019 Assessment Design. In I.V.S. Mullis & M. O. Martin (Eds.), TIMSS 2019 Assessment Frameworks (pp. 79-91).
McGarr, O., & McDonagh, A. (2019). Digital competence in teacher education, Output 1 of the Erasmus+ funded Developing Student Teachers’ Digital Competence (DICTE) project. https://dicte.oslomet.no/ Accessed 13 February, 2020.
McMillan, J; Cauley, K. (2010). Formative assessment techniques to support student motivation and achievement. Educational strategies, 83(1), 1-6.
Meusen-Beekman, K., Joosten-ten Brinke, D., & Boshuizen, H.P.A. (2015). Developing young adolescents’ self-regulation by means of formative assessment: A theoretical perspective. Cogent Education, 2(1), 1071233.
Mishra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108(6), 1017-1054.
Molenaar, I., Horvers, A., & Baker, R. S. (2019). What can moment-by-moment learning curves tell about students’ selfregulated learning?. Learning and Instruction, 101206. https://doi.org/10.1016/j.learninstruc.2019.05.003. Accessed 4 February, 2020.
Moss, C.M., & Brookhart, S. (2019). Advancing formative assessment in every classroom: A guide for instructional leaders.Alexandria, VA: ASCD Publications.
Ng, W. (2012). Empowering scientific literacy through digital literacy and multiliteracies. Hauppauge, NY: Nova Science Publishers, Inc.
Nikou, S.A., & Economides, A.A. (2018). Mobile-based assessment: A literature review of publications in major refered journals from 2009 to 2018. Computers & Education, 125, 101-119.
Nikou, S.A., & Economides, A.A. (2019). Factors that influence behavioral intention to use mobile-based assessment: A STEM teacher’s perspective. British Journal of Educational Technology, 50(2), 587-600.
O’Leary, M., Scully, D., Karakolidis, A., & Pitsia, V. (2018). The state of the art in digital technology based assessment. European Journal of Education, 53(2), 160-175.
Orr, G. (2010). A review of literature in mobile learning: Affordances and constraints. Paper presented at the 6th IEEE International Conference on Wireless, Mobile, and Ubiquitous Technologies in Education, Kaohsiung, Taiwan. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5476544 Accessed 13 February, 2020.
Panero, M., & Aldon, G. (2016). How teachers evolve their formative assessment practices when digital tools are involved in the classroom. Digital Experiences in Mathematics Education, 2, 70-86.
Pinger, P., Rakoczy, K., Besser, M., & Klieme, E. (2018). Implementation of formative assessment – effects of quality of programme delivery on students’ mathematics achievement and interest. Assessment in Education: Principles, Policy & Practice, 25(2), 160-182.
promising practices for enhancing accessibility for students with disabilities. Journal of Special Education Technology, 33(4), 219-236.
Quellmalz, E.S., Davenport, J.L., Timms, M.J., DeBoer, G.E., Jordan, K.A., Huang, C., & Buckley, B.C. (2013). Next-generation environments for assessing and promoting complex science learning. Journal of Educational Psychology, 105(4), 1100- 1114.
Reynolds, Katherine;et al. (2020). Digital Formative Assessment of Transversal Skills STEM. Dublin City University: ISBN: 978-1-911669-05-0.
Ruiz-Primo, M.A., & Furtak, E.M. (2006). Informal formative assessment and scientific inquiry: Exploring teachers’ practices and student learning. Educational Assessment, 11(3), 205-235.
Ryoo, K., & Linn, M.C. (2016). Designing automated guidance for concept diagrams in inquiry instruction. Journal of Research in Science Teaching, 53(7), 1003-1035.
Sadler, D.R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144.
Scalise, K., Irvin, P.S., Alresheed, F., Zvoch, K., Yim-Dockery, H., Park, S., Landis, B., Meng, P., Kleinfelder, B., Halladay, L., & Partsafas, A. (2018). Accommodations in digital interactive STEM assessment tasks: Current accommodations and
Seery, N., Canty, D., & Phelan, P. (2012). The validity and value of peer assessment using adaptive comparative judgement in design driven practical education. International Journal of Technology Design Education, 22, 205-226.
Shavelson, R.J., Young, D.B., Ayala, C.C., Brandon, P.R., Furtak, E.M., Ruiz-Primo, M.A., Tomita, M.K., & Yin,Y. (2008). On the impact of curriculum-embedded formative assessment on learning: A collaboration between curriculum and assessment developers. Applied Measurement in Education, 21(4), 295-314.
Shute, V.J., & Rahimi, S. (2017). Review of computer-based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33, 1-19.
Shute, V.J., Leighton, J.P., Jang, E.E., & Chu, M. (2016). Advances in the science of assessment. Educational Assessment, 21(1), 34-59.
Sireci, S.G., & Zenisky, A.L. (2006). Innovative item formats in computer-based testing: In pursuit of improved construct representation. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 329-348). New York, NY: Routledge.
Spante, M., Hashemi, S., Lundin, M., & Algers, A. (2018). Digital competence and digital literacy in higher education research: Systematic review of concept use. Cogent Education, 5, 1-21.
Spector, J.M., Ifenthaler, D., Sampson, D., Yang, L., Mukama, E., Warusavitarana, A., Dona, K.L., Eichorn, K., Fluck, A., Huang, R., Bridges, S., Lu, J., Ren, Y., Gui, X., Deenan, C.C., San Diego, J., & Gibson, D.C. (2016). Technology enhanced formative assessment for 21st century learning. Educational Technology and Society, 19(3), 58-71.
Stobart, G. (2006). The validity of formative assessment. In Garder, J. (Ed.), Assessment and Learning. Thousand Oaks, CA: SAGE Publications Ltd.
Szendey, olivia;et al. (2020). Virtual Learning Environments and Digital tools impementing Formative Assessment and Transversal Slills in STEM. Dublin City University: ISBN: 978-1-911669-14-2.
Timmers, C.F., Walraven, A., & Veldkamp, B.P. (2015). The effect of regulation feedback in computer-based formative assessment on information problem solving. Computers & Education, 87, 1-9.
Van De Ven, A. (1992). Suggestions for Studying Strategy Process: A Research Note. Strategic Management Journal, 13(S1), 169-188.
van Dijk, A.M., & Lazonder, A. (2016). Scaffolding students’ use of learner-generated content in a technology-enhanced inquiry learning environment. Interactive Learning Environments, 24(1), 194-204.
Vogelzang, J., & Admiraal, W.F. (2017). Classroom action research on formative assessment in a context-based chemistry course. ducational Action Research, 25(1), 155-166.
Wiliam, D. (2016). The secret of effective feedback. Educational Leadership, 73(7), 10-15.
Wiliam, D. (2019). Why formative assessment is always both domain-general and domain-specific and what matters is the balance between the two. In H. Andrade, R.E. Bennett, & G. Cizek (Eds.), Handbook of formative assessment in the disciplines (pp. 243-264). New York, NY: Routledge.
Wiliam, D., & Black, P. (1996). Meanings and consequences: A basis for distinguishing formative and summative functions of assessment? British Educational Research Journal, 22(5), 537-548.
Wiliam, D., & Thompson, M. (2007). Integrating assessment with learning: What will it take to make it work? In C.A. Dwyer (Ed.), The future of assessment: Shaping teaching and learning. New York, NY: Lawrence Erlbaum Associates.
Wiliam, D., Lee, C., Harrison, C., & Black, P. (2004). Teachers developing assessment for learning: impact on student achievement. Assessment in Education, 11(1), 49-65.
Wiliam, D; Thompson.M;. (2007). Integrating assessment with learning: What will it take to make it work? New York: In C.A. Dwyer(Ed.), The future of assessment: Shaping teaching and learning

  • Receive Date 21 April 2023
  • Revise Date 22 May 2023
  • Accept Date 26 May 2023
  • Publish Date 21 March 2023