In July 2018 I got a chance to attend International Educational Data Mining (EDM) conference in Buffalo, New York to present a paper on ‘Contextual Derivation of Stable BKT parameters for Analyzing Content Efficacy.’ The paper was a result of a breakthrough we had achieved in the implementation of Bayesian Knowledge Tracing model which reduced its tendency to overfit the underlying data.

EDM is a well-known conference and boasts of participants from many top-tier universities. This was Ph.D. opportunity to meet the experts in the field of EDM and rub the proverbial shoulder with them. So I had started planning well in advance to make the most out of this conference as EDM is still a nascent field in India and lacks a vibrant community around EDM and learning analytics.

As a stroke of luck, the venue for this year conference was New York which is the hub of the domain and has many universities in the vicinity which are doing cutting-edge work. I reached out to these universities and was able to set up discussions with some of the leading experts’ thanks to introductions by Ryan Baker who has been helping us with our EDM efforts here at EI. Below is a short account of the trip –  

Visit to Carnegie Mellon University, Pittsburgh –

CMU hosts Human Computer Interaction Institute (HCII) which is doing a lot of work in developing research that can be used to inform the design, development, and evaluation of learning tools. HCII is one of the pioneers of the field and has had key figures such as Albert Corbett, Robert Anderson, Ken Koedinger, Bruce Mclaren and others on its rolls. I had the opportunity to meet Ken Koedinger, Bruce Mclaren and Vincent Aleven during my visit to the CMU and was able to spend significant time discussing their take on the field of Educational Data Mining and the key drivers which are going to lead the growth of Edtech.

Ken and Vincent work closely, and my discussion with them revolved mainly around design and development of Intelligent Tutoring Systems as Ken and Vincent played a key role in the development of Cognitive Tutor, one of the earliest ITS which is still being used by more than 1500+ schools in the USA.

It was really amazing to get the first-hand account of some of the stories/thought process which leads to the design of Cognitive Tutor, especially the part about how much rigor was put into developing the content and how the team went about it. We also touched off other relevant topics such as the use of hints & solved examples as a means to encourage students to develop metacognitive process, strategies to enhance student engagement and the student knowledge using machine learning models.

The discussion with Bruce involved the entirely different topic of using games as a means to teach kids. Bruce’ areas of interest include Educational games, Collaborative learning and E-learning principles. He shared his experience of developing DecimAliens (a game on Decimal Numbers). Along with Educational games, Bruce is also exploring how solved, and erroneous examples can be used in effective learning.  

International Educational Data Mining Conference, Buffalo –

The conference was a 4-day affair and was hosted in Buffalo, New York. The first day of the conference was dedicated to workshops and the other 3 days consisted of paper presentations with 3 parallel tracks – a Main track which represented full-length Journal papers, Industry track and Post-doctoral track. Around 120 to 140 participants attended the conference spread across the 4 days.

The paper presentations broadly revolved around 3 themes – 1) Online learning courses (MOOCs), 2) Models to track student knowledge level in an ITS, 3) Models to detect student affective and emotional states in an ITS. There were many interesting insights that have application for Mindspark, but it will be difficult to summarize them all here. Sharing the 3 or 4 key takeaways for me –

  • There is a continued emphasis on the ‘Doer effect.’
  • Researchers are experimenting a lot with hints and worked examples to figure out the best combination and see what works best for students of different proficiency levels
  • A lot of rigor is put into content development through student interviews, data collection etc.
  • Focus on designing ITS features which helps students self-regulate their learning and increase student engagement

Apart from the sessions, the conference was useful in connecting with other experts as expected. It was also great to meet a lot of researchers from India who are pursuing their PhD thesis or are faculty at universities in the USA.

The only limitation which I felt was that Industrial representation was somewhat limited. I could spot representatives from Aleks, ETS, Carnegie Learning, IBM, Accenture but it seemed lesser than what I had expected. This may be because companies do not like to reveal their research -initiatives to protect their IP/ competitive advantage.

The conference also involved a trip to Niagra Falls which is very close to Buffalo. The trip was really fun and a must for anyone who visits New York. Though I had seen Niagra Falls around 2 years back during my earlier visit to New York, the experience was no less enjoyable this time and cannot be captured in words!!

Visit to Worcester Polytechnic Institute (WPI) –

Learning Sciences and Technologies department at Worcester polytechnic institute drives most of the research in EdTech. I got an opportunity to meet Neil Heffernan (and his team) and Ivon Arroyo at WPI. Neil Heffernan is the founder of Assistments (an ITS) which is being used by 100,000 students across the US and is free for US schools.

Assistments focus on providing hints, scaffolding, and feedback as a means of teaching students and allows teachers to put and use their own content through Assistments. Neil’s focus on developing Assistments as a self-sustainable ITS which is able to crowdsource content and is the tools of choice of self-motivated teachers/students.

The discussions with Ivon happened around the ITS, Wayang Outpost she has developed in-house with her team. The ITS has an extensive array of features along with a pedagogical agent. It can assess a student’s math skills, affective states, and metacognitive states in a comprehensive manner.

Her work also includes another tool Animalwatch which is an Intelligent Tutoring System for basic arithmetic and fractions and offers word problems about endangered species. This ITS was used in research to understand different hint strategies and their effectiveness.

Closing thoughts

It was an exhaustive trip involving travel to 4 cities in 9 days, but the effort was worthwhile. I hope that our team will publish many more papers in the future and my colleagues will get the opportunity to attend similar conferences as it will be a rich experience for them and also give them a more in-depth understanding of the domain and its potential which is immense!!

Deepak Agarwal

Deepak Agarwal

Deepak works with the Data Science team and has been with EI for past 5+ years.

His interests lie at the intersection of pedagogy, analytics and machine learning to develop models/insights in the domain of intelligent tutoring systems like Mindspark.

Along with data science, he has extensive experience in Assessments and has worked in the past with LSEP team at EI.
Deepak Agarwal

Latest posts by Deepak Agarwal (see all)