Learning Analytics

Ok, hello #eci834 classmates! It’s been a little bit since our last flurry of posts, hasn’t it? As we are all fully engaged with building and helping one another design online learning experiences, it’s nice to get a breather and shift focus i a different direction. To add to this , Alec has left us with as much leeway as we could ask for in determining our own area of interest this week.

I didn’t have any particular direction in mind, apart from wanting to look at something I hadn’t really put much thought into beforehand. In reading a summary of emerging trends in online education for the year 2018, moderated by Tony Bates, I came across a the use of analytics in education. Interesting! The use of data in schools is definitely not new. I’ve been a part of developing strategic plans that respond to collected data for years. But to see it as an emerging field on its own is interesting to me!

The concept of analytics is not new to me in and of itself. But my exposure and engagement with meta analytical statistics is mainly confined to sports. The movie Moneyball is the glorified, Brad Pitt delivered story of a small market, perpetual underdog Oakland Athletics baseball team using analytics to punch well above their weight in league play. They ended up coming close to winning a World Series with a team of relatively unknown baseball players.

Tony Bates mention of education analytics was mainly to point out its perceived limitations and that it is not, as of yet, widely used. But as he points out so well … “actual usage today does not necessarily predict future usage.”

Good enough for me! I’m hooked!

so what is it?

To briefly introduce Learning analytics, I am relying on a 2014 article written by  Dirk T Tempelaar, Bart Rienteis, and Bas Giesbers:

In search for the most informative data for feedback generation: Learning analytics in a data-rich context

In their study they describe learning analytics as using large pools of data to increasingly personalize education. Learning analytics uses data from sources like user clicks, survey feedback and participation online.Specifically, this trio of researchers were trying to model a practical use of learning analytics on a large scale. They collected learner data (self-report surveys) and learning data (from learning management systems).

Not surprisingly, for something as labour intensive as collecting and interpreting large sets of data, industry is ready to step in!

Yetanalytics, Civitas Learning, Brightbytes, Panorama Education, and Education Analytics are but a few examples of many companies that are out there and ready to support our educators and teaching practice. For a modest profit, of course.

ANALYTICS at work: EXAMPLE

To familiarize myself with analytics in an education context, I followed Tony Bates’ link to a recorded keynote address by Bart Rienteis, professor of Learning Analytics with the Open University in the United Kingdom. It is a 40 minute address, to I found it well worth the time to watch., found here. Rienteis  engagingly relates observations made from  study on student satisfaction with online courses by pooling data from over 100 000 post-secondary student surveys, through 151 separate online learning modules. Through dissemination of this vast data, there were some surprising revelations!

For one, while common sense might lead one to believe (myself included) that there would be a correlation between student success with and achievement in online modules, results look like this:

“There are amazing modules, with tremendously high pass rates, that get relatively low satisfaction scores by students.” Rienteis

This kind of larger scale inference, that student surveys showing relative satisfaction with a course may have little to do with how students are actually performing, would have not really been easily possible without that vast pool of data. I certainly can’t see how that inference can be drawn from the results of a survey from a single online course… As Rientais says,  there are clearly other factors at play, beyond academic success, that are determining whether a student feels satisfied with an online course!

I’m sure we’ll be hearing more from learning analytics as our respective ministries and school boards continue to build strategic plans based on data that filter down to our schools and classroom planning.

Thanks for reading!

2 comments to “Learning Analytics”
2 comments to “Learning Analytics”
  1. Hi Hosna and thanks for the post. I oved the movie Moneyball and there are a lot of inferences that can be made with this movie. New approaches often experience a lot of resistance but how can we evolve if change is never embraced or even considered? Analytics have been around a long time and I have used them as well and question the results that they apply at times. I use them when I evaluate test question responses to analyze if my question was understood, relevant and if the answers provided were also appropriate. Analytics is also involved in our LMS Brightspace and I will refer to it to see how engaged students are in the material I am providing. Once I incorrectly assumed a students engagement in the course as she downloaded all of the material ahead of time and reviewed it offline. The analytics showed she was not online very often and sent me a different message than what was actually happening. Analytics is often relied on for community dentistry as well. Again, the numbers can lead reviewers astray as sometimes the questions/answers are misleading or are referencing a certain portion of the population that may not be an accurate representation of the whole group. A lot of government programming is decided upon if the actions can be quantitatively measured. For example, fluoride programs in schools. They can measure how many kids have had fluoride treatment, but it has not lead to much lower decay rates. What about proper brushing and flossing, annual dental visits or diet considerations? Far too often we rely on quantifiable evidence and do not think about qualitative data? The reasons why people choose what they do is as important. You highlight this with the example above. The students did well in the class but they were dissatisfied with taking it. What is more important? The result, process or both? Finally, relying too much on analytics can cause you to make generalizations . I do see value in analytics but we must consider other factors when deciding how effective an approach or strategy is. Thanks for your insights!

  2. Thanks Dean! 🙂 My name is Joe though!

    Indeed, I can see it being very inappropriate to make specific inferences from the use of aggregated data. There are always stories behind the data and we need to dig into them!

Leave a Reply

Your email address will not be published. Required fields are marked *