How do you collect your data for analysis and how do you evaluate it?

Pedro Jose Munoz Merino: Most current online learning platforms already collect data about students’ interactions. LMSs (learning-management systems), MOOCs (massive open online courses) platforms, and ITSs (intelligent tutoring systems) usually provide an option to enable or disable this data-collection feature. In platforms or learning environments that do not make it possible to enable the collection of data, we can implement software tools that collect data. For instance, when some event or interaction occurs, some software function is invoked and the correspondent data is stored in a database. The issue of collecting data in online learning platforms is technologically solved, and the possibility of collecting all the data generated in these systems currently exists.

The data collected about user interactions is usually stored as a set of events. Each event usually has a form similar to the following: "User U makes a specific type of action A on a resource R at time stamp T on application A". This raw data is meaningless for stakeholders in this form, but has the potential to be very powerful when it is transformed.

First, the data needs to be cleaned (e.g. removing outliers). Next, we use pedagogical knowledge, expert knowledge, semantic knowledge, or machine-learning techniques in order to transform the raw data into intelligent information that can be understood by stakeholders. For example, we might apply Bayesian Networks (a type of machine-learning technique) for the evaluation of the students’ level in various skills in combination with expert knowledge about the topic (to make the various skills, the relationship among skills, or relationships among exercises and skills).

Some examples of intelligent information that we can infer from raw data are the learning effectiveness, the learning efficiency (e.g. two students can learn in an effective way, but one of them might learn more quickly), interest (e.g. related to the level of interaction with the resources), emotions, behaviors, meta-cognitive skills, or preferences. 

Where do you see concrete improvements in the learning process?

Pedro Jose Munoz Merino: The application of learning analytics provides different improvements in the learning process:

• Adaptive learning - The learning resources can be adapted to the particular needs of the students based on their learning profiles, personalizing the learning experience.
• Advanced feedback - Advanced feedback can be provided to students using the intelligent information obtained.
• More precise evaluation of students - A more precise evaluation of students is made possible. For example, student performance in different skills can be evaluated in a more accurate way, taking into account the difficulty of exercises based on real data of previous students’ interactions. Or for example, new indicators might be included that are not traditionally used such as the efficiency or the students’ interest.
• Improvement of educational resources. For example, portions of video resources with problems can be detected depending on the number of repetitions of these fragments by students.
• Improvement of educational methodologies. Various methodological approaches can be tested and compared based on some indicators, and we can select the best methodologies depending on each specific context.

What are the advantages for learners and teachers?

Pedro Jose Munoz Merino: I think there are three main advantages for learners.

• First, the application of learning analytics enables learners to learn more and in a more efficient way. This is a consequence of being able to have better methodologies, better contents, adaptive learning, or advanced feedback.
• Second, students are able to apply a self-regulated learning approach in which they can receive, e.g. dashboards with useful visualizations so that they can be aware of their learning process and make their own decisions.
• Third, students can extend their knowledge about themselves, having information not only about their learning process but also about their meta-cognitive skills (e.g. if they know when to request for help when they need it) or other traversal abilities such as their communication skills or their perseverance. This additional knowledge can make students improve in other areas besides learning.

Learning analytics makes it easier for teachers to detect which contents or parts of the contents should be improved. In addition, teachers can get to know detailed information about the whole class and the different students and can make interventions to improve learning based on this information or to make learning more engaging. For example, the automatic detection of students motivated by gamification can be used to provide them with gamification activities. When the number of students is very high, teachers cannot provide personalized feedback and adaptive resources to them. Learning analytic tools can facilitate these tasks by providing automatic feedback and personalized resources. In addition, the early prediction of learning outcomes, social activity, etc. can be used to enable teachers to take actions to improve the learning process.