I. Introduction

The reality of education today is characterized by the proliferation of methodologies that are highly adaptable to the needs of the current learner (^{Llanos & Bravo-Agapito, 2017}). Under this premise, flipped learning is a methodological alternative to fulfill the demands of an education in which both information and communication technologies (ICTs) and students themselves stand as protagonists of the teaching-learning process (^{Froehlich, 2018}; ^{Zainuddin et al., 2019}). Flipped learning began in the United States thanks to the training actions of teachers Jonathan Bergmann and Aaron Sams, aimed at providing digital audio-visual material to those students who could not attend class regularly (^{Bergmann & Sams, 2012}). Since then, flipped learning has evolved and adapted to achieve a high degree of effectiveness and establish itself as a methodology with a growing number of followers (^{Fuentes-Cabrera et al., 2020}; ^{He et al., 2016}; ^{Pozo et al., 2019}; ^{Segura-Robles et al., 2020}).

The flipped learning methodology consists in reversing the traditional learning processes (^{Bauer-Ramazani et al., 2016}). Students use the after-school period to work with the theoretical and conceptual elements through digital tools (^{Long et al., 2017}; ^{Schmidt & Ralph, 2016}) and allocate the time in the classroom to carrying out practical and dynamic activities under supervision by the teacher (^{El Miedany, 2019}). This enhances the protagonist role of the students (^{Touron & Santiago, 2015}), who self-regulate and adapt their own educational process individually (^{Boelens et al., 2018}; ^{López et al., 2019}), allowing them to learn at times when they are more available, better able to access material, and more motivated (^{Shih & Tsai, 2017}).

The main research on flipped learning has found that its implementation fosters student motivation and participation in training activities (^{Chyr et al., 2017}; ^{Huang et al., 2018}; ^{Tse et al., 2019}) that are configured under a collaborative approach (^{DeLozier & Rhodes, 2017}) that promotes socialization within the peer group (^{MacLeod et al., 2018}). This cooperative climate allows a higher rate of development in problem solving (^{Bognar et al., 2019}; ^{Pozo et al., 2020}), in the assimilation of curricular content (^{Karabulut et al., 2018}), in the achievement of didactic objectives (^{Awidi & Paynter, 2019}), and thus in the grades attained by the students (^{Hinojo-Lucena et al., 2018}; ^{Sola et al., 2019}).

However, not all these potentialities of the flipped classroom are without limitations. Some authors have mentioned the need for high levels of abstraction in students in order to carry out activities (^{Hwang et al., 2015}), the inability shown by some students to solve problems (^{Bognar et al., 2019}; ^{Mengual-Andres et al., 2020}), and the difficulties of accessing and managing learning platforms (^{Hwang & Lai, 2017}; ^{López et al., 2019}; ^{López-Belmonte et al., 2019}; ^{Yilmaz, 2017}). In reference to this last limitation, one of the different tools that can be used in implementing flipped learning is the learning management system (LMS), one of the educational technology systems best able to adapt to this type of methodology to overcome problems of accessibility and usability (^{Almarashdeh, 2016}; ^{Rhode et al., 2017}).

We understand that LMSs are tools intended for the administration of high-volume information banks. These tools are used together in digital frameworks that allow the generation of content, practical activities, and assessment tools (^{Gilhooly, 2001}). These types of tools allow multiple courses to be managed at once with a multitude of students, who can register and log in to the platform to complete the different courses and carry out flexible assessment processes (^{Gallagher, 2007}), all based on a functional, intuitive, and simple interface (^{Hall, 2002}) with a high level of digital security (^{Biech, 2008}).

This produces a dynamic educational context that takes advantage of the learning management capacity offered by technology to improve the academic performance of students (^{Mwalumbwe & Mtebe, 2017}; ^{Han & Sug-Shin, 2016}) and their satisfaction during the teaching-learning process (^{Ramírez-Correa et al., 2017}), and bring about positive effects on interaction and collaboration (^{Dias et al., 2017}; ^{Hadjileontiadou et al., 2015}). ^{Watson and Watson (2007)} affirm that the defining characteristics of the LMS are based on the ability to provide individualized teaching units incorporated into a standardized curriculum. Courses are expanded taking into account the progression of levels and degrees of difficulty, individual compilations of student results, and finally, a teaching-learning process configured with a focus on the particularities of each student.

^{Santiago and Bergman (2018)} explain the use of the LMS as a cycle based on learning analytics:

Currently, there are many LMSs available for use in education. Moodle, one of the most relevant platforms, focuses on the social constructivist perspective of learning, allowing it to integrate tools for individualized creation of H5P content, while also being compatible with PHP7 (^{Kerimbayev et al., 2017}). Google Classroom, meanwhile, has an increasing number of users within the educational field thanks to the effectiveness of the compendium of Google Apps for Education applications, all organized as a cloud-based educational management tool (^{Ramadhani et al., 2019}). On the other hand, Chamilo is a tool based on the use of resources in the Sharable Content Object Reference Model (SCORM) format and is much more focused on interactive and social aspects (^{Ardila et al., 2015}). Also worth noting is WordPress, a digital tool that is not an LMS itself but can combine with certain utilities allowing them to be used as such, in a simple and intuitive way (^{Harris & Rea, 2019}).

In recent years, research has emerged that analyzes the specific effects of using LMSs on flipped learning methodology, showing that LMSs are a permanent, key factor in optimizing flipped learning. ^{Nouri et al. (2019)} explain that these systems allow the students’ tasks to be monitored while allowing individualized comments to be made to support their learning progress. In this way, the flipped classroom takes advantage of these management tools to insert didactic sessions in video and digital content, enabling increased student engagement with content and materials and enhancing student self-regulation and the teacher’s capacity for intervention.

In a similar way, ^{Andujar et al. (2020)} place special emphasis on the insight provided to teachers as to the student’s current position in the teaching-learning process, in addition to the fact that teachers are able to exercise control specific to the actions carried out by students within the management platform when working with the curricular materials or performing tasks. ^{Karaca and Akif (2017)} affirm that the increase in teacher interaction becomes especially important during corrections of the evaluation tests, since the management platforms allow the valid answers to be disclosed and make it possible to insert video links with specific explanations or redirect students to various materials individually to plug any gaps in their learning.

Likewise, other authors who have analyzed the effects of using LMSs in flipped learning have found improvements in student engagement and participation, all based on a self-reflective and self-evaluative learning culture that favors the performance of activities based on problem solving (^{Wang, 2017}). This pedagogical combination allows students to become involved and motivated in the teaching-learning process based on a varied and dynamic interaction with the assessment tests (multiple choice, true and false, short answer, open-ended answer, etc.) and with the curricular content and learning materials (^{Karaca & Akif, 2017}).

II. Methodology

The scope of the objectives and the answers to the research questions were pursued through a post-test quasi-experimental design with a descriptive and correlational scope, based on a quantitative research methodology. To perform this research adequately, we followed the guidelines of experts in this type of study (^{Hernandez et al., 2014}; ^{Rodriguez, 2011}). Likewise, we employed the analytical structure of studies in recent years in order to implement a study model validated by experts (^{Hinojo et al., 2020}; ^{Pozo et al., 2019}). Two study groups - a control group (CG) and an experimental group (EG) - were established in this research. The CG engaged in flipped learning with audio-visual content hosted on a non-interactive website. The EG, on the other hand, used flipped learning with an LMS to house the ready-made teaching materials. This design gives rise to two types of variables: the medium where the contents are housed as the independent variable, and the efficacy obtained in the academic indicators as the dependent variable.

Basic statistics such as the mean (M) and standard deviation (SD) were used. The distribution trend was determined with the skewness (Skw) and kurtosis (Kme) tests. The means between the analysis groups were compared with Student’s t test (tn1 + n2 - 2). Furthermore, the strength of association and the size of the effect were obtained with Cohen’s d and the biserial correlation (rxy). The data were analyzed with Statistical Package for the Social Sciences (SPSS) v.25, with a significance level of p < 0.05.

A total of 231 Spanish secondary school students participated in this didactic learning experience. These subjects were chosen through purposive sampling, due to the researchers’ close contact with the school in question. The sample size in this type of research does not influence, alter, or bias the results; this has been previously verified by experts (^{Chou & Feng, 2019}; ^{Yılmaz & Soyer, 2018}).

The selected sample was made up of 41.9% boys and the rest girls, with an average age of 13 years (SD = 1.48), in the first year of secondary education in the Spanish education system.

Eight groups of students with the aforementioned characteristics were taken. The composition of the groups was natural, that is, they corresponded exactly to the groups taught in the school at that level (n = 8). This configuration gave rise to two group types. A total of four groups were formed as control groups (CG1, n = 30; CG2, n = 30; CG3, n = 29; CG4, n = 27), while four others were experimental groups (EG1, n = 30; EG2, n = 29; EG3, n = 29; EG4, n = 27). Students were randomly assigned to each type of group. The experience was carried out over the last three months of the 2019-2020 school year.

An ad hoc questionnaire was designed and used to collect data, with various instruments widely validated by the scientific literature taken as a reference (^{den Brok et al., 2006}; ^{Kyrgiridis et al., 2014}; ^{Watson & Glaser, 2002}). Ad hoc questionnaires are specifically designed for research, based on other validated questionnaires from the scientific literature (^{Achilli, 2005}).

The final questionnaire is made up of two well-differentiated blocks. The first block encompasses all the classic sociodemographic variables in educational research (gender, age, educational stage). The second block covers other items, ordered according to the different dimensions of the study objective, notably the ability of flipped learning to develop:

To select the various items that were to comprise the final questionnaire, we followed a number of inclusion criteria. On the one hand, we included items that allowed us to collect data on the different dimensions that made up the objectives of the study. At the same time, these items should be concise and short, to avoid tiring students. Exclusion criteria were also established, eliminating, for example, any items that may have led to confusion or which were discarded by the expert panel that we consulted, thus avoiding an excessive number of questions.

The expert panel gave a few recommendations to improve the questionnaire, like reducing the number of questions or rephrasing some items. Feedback was analyzed with Kendall’s W to determine the concordance and relevance of the judgments made (W = 0.83). This is a classic statistic employed to evaluate concordance between experts (^{Craigie et al., 2002}).

The questionnaire employs a 4-point Likert scale. A 5-point scale was not used to prevent participants from tending to respond centrally (^{Matas, 2018}). Finally, Table 1 presents the reliability and validity scores obtained.

After participants were assigned to the study groups, the different educational methodologies were applied. The CG students carried out a didactic unit based on flipped learning with audio-visual content that did not allow interaction between students or with the teacher. They could only view the teaching materials hosted on a web server. The EG students carried out a didactic unit based on flipped learning with instructional material on an LMS platform that allowed all kinds of interactions (chat messaging, thematic discussion forums, private messages to the teacher, private messages between students, and public messages in each of the sections that contained the videos).

The same type of class activities were carried out in all the groups. The face-to-face educational tasks focused on inquiry, collaboration, and problem solving by students. The audio-visual educational material was prepared by two teachers from the school in question, experts in this type of active learning methodology. These same teachers implemented the necessary methodology to carry out the research.

Once the didactic unit was completed, the questionnaire was administered in order to collect measurements. The data collected was statistically processed to achieve precise results to achieve the proposed objectives and answer each of the research questions.

This research was framed in accordance with the ethical principles expressed in the Declaration of Helsinki. Prior to involving the students, the research project was approved by the academic committee of the school itself, and informed consent was obtained from the participants.

III. Results

The results found in the descriptive analysis show that the means achieved by the experimental group are superior to those of the control group in all the dimensions analyzed. The averages achieved in all dimensions, both in the control group and in the experimental group, are above 2.5, which demonstrates a medium-high evaluation by students of the general application of flipped learning. The distribution of the sample is normal, since the values of asymmetry and kurtosis are between -1.96 and +1.96 (^{Jöreskog & Moustaki, 2001}). The standard deviation of both the control and experimental group dimensions shows an equal impact among the students. All cases are platikurtic, except for the students’ decision-making about their learning, which is mesokurtic (Table 2).

The relationship among students in the experimental group shows significant differences in all the dimensions of the study. Students in the experimental group obtained higher values than the control group in almost all variables. However, similar results were observed between the two groups in the variables “Decisions”, “Interests-strengths”, and “Student knowledge” (Figure 1).

Figure 1 Comparison between Control Group and Experimental Group 

In this study, we analyzed the independence between classic flipped classrooms and LMS-associated flipped classrooms, applying Student’s t tests for independent samples. The data shown in the analysis demonstrate a very significant impact from student-teacher interaction and access to learning content being enhanced by the use of an LMS in conjunction with the flipped learning method. This shows the strength of association between media.

The other dimensions show significant associations, except in students’ possibilities to make decisions about their learning and in teachers’ knowledge of the interests and strengths of students, where there is no significant relationship. The strength of association in this case is low. The size of the effect, according to Cohen’s d, is very low in all the dimensions analyzed (Table 3).

IV. Discussion

The use of LMSs to apply the flipped learning methodology is increasingly widespread among the teaching community as a way to optimize the results of this methodological approach (^{Andujar et al., 2020}). In addition, an LMS overcomes the limitation posed by flipped learning in terms of access and usability (^{Almarashdeh, 2016}; ^{Rhode et al., 2017}). The scientific literature has been nourished - especially in recent years - by various studies that have analyzed the impact of flipping learning moments together with these management platforms (^{Harris & Rea, 2019}; ^{Kerimbayev et al., 2017}; ^{Ramadhani et al., 2019}).

As far as the interactive dimension is concerned, the results of this study show that the possibilities of working with other colleagues and the degree of interaction between colleagues are high in a flipped classroom, but this interaction becomes even more frequent if management tools are used during the teaching-learning process. These results are in line with other studies that have analyzed this aspect (^{DeLozier & Rhodes, 2017}; ^{MacLeod et al., 2018}).

Regarding the cognitive aspect, and along the same lines as other studies, using an LMS during the application of flipped learning yields higher levels of optimization in dimensions related to critical thinking (^{Wang, 2017}) and the degree of interaction with the teaching staff (^{Karaca & Akif, 2017}; ^{Santiago & Bergman, 2018}). Likewise, positive results have also been obtained regarding access to content, the quality of learning materials and resources, and the variability of forms of knowledge acquisition, in line with findings reported by other authors (^{Karabulut et al., 2018}; (^{Karaca & Akif, 2017}; ^{Santiago & Bergman, 2018}).

On the other hand, it is pertinent to highlight that, of all the dimensions analyzed, those related to motivation reflected the lowest level of significance. This study obtained relevant results from the application of an LMS in dimensions related to the adaptability of the methodology to student needs and in relation to the effectiveness and practicality of learning. These results are similar to those found in the scientific literature (^{López et al., 2019}; ^{Santiago & Bergman, 2018}). However, it has not been possible to verify that the use of an LMS in flipped learning affects dimensions related to decision-making, nor were we able to confirm that using an LMS helps to boost teachers’ knowledge of their students or teachers’ capacity to take into account the interests and strengths of students, contrary to the results of other similar studies (^{Bognar et al., 2019}; ^{Mengual-Andres et al., 2020}; ^{Santiago & Bergman, 2018}).

Regarding academic performance, the scores obtained by students who have used an LMS are slightly higher than those of students who have used digital media in the flipped classroom. This is consistent with results found in the scientific literature (^{Hinojo-Lucena et al., 2018}; ^{Sola et al., 2019}).

V. Conclusions

This research has analyzed the impact of the use of an LMS or other digital media in the application of the flipped learning methodology at an interactive, cognitive, motivational, and academic performance level. At all these levels, the results allow us to conclude that using an LMS while applying the flipped learning methodology contributes to an optimization of the general teaching-learning process, particularly in interactive, cognitive, motivational, and academic performance aspects. The use of an LMS also makes it possible to overcome limitations related to student access to didactic contents.

The main limitations encountered in implementing this research lie in the fact that the selected sampling technique was purposive. Thus, no sampling was carried out of the total population, so we recommend generalizing the results found in the study or extrapolating them to population samples with significant differences. On the other hand, access to the different schools was complex due to an excess of bureaucracy and poor training of teachers in some schools that had accepted the proposed intervention but lacked qualified staff to put it into practice.

Despite the aforementioned limitations, this study aims to show and offer the scientific community and the teaching community an effective pedagogical approach in flipped learning, which can be greatly benefited and enhanced if combined with the use of an LMS. Therefore, our intention is to offer a forward-looking analysis that disseminates the benefits of using this methodology and encourages the scientific community to carry out contrasting analyses with respect to the data obtained in this study, from population samples of different characteristics. For this reason, as a future line of research, the team of researchers is developing a pilot study on the application of the flipped learning method with an LMS as part of a multi-stage analysis that considers the particularities of applying the methodology based on educational stage.

This study has a series of implications, both theoretical and practical. On a theoretical level, this study contributes to the increase in scientific literature on the use of flipped learning, as a methodology, through virtual teaching platforms. This type of study provides more data to the scientific community. In addition, the research method employed, with a control group and an experimental group, may in the future enable meta-analyses with similar studies. Also, the theoretical framework developed may serve as a reference and a basis for other researchers who wish to carry out research on this line of study.

On a practical level, this study may have various implications in education. The teaching method analyzed can be applied in various educational contexts, particularly at present, considering that the COVID-19 pandemic has led to changes in teaching-learning processes. For this reason, the teaching method analyzed can be a very efficient resource for developing the educational process with confidence. In addition, using virtual teaching platforms with the flipped learning method facilitates interaction between all those involved, which will allow for a more individualized and personalized knowledge of students. This brings with it more appropriate and individualized teaching. Finally, the data obtained in this study can serve as a basis for institutions and educational bodies to establish pedagogical training plans, and for adapting and adjusting techno-pedagogical resources for future development, with a guarantee of success.