While the use of technology in education is becoming increasingly popular, successfully implementing EdTech is challenging, especially in resource-scarce settings. Concerns over infrastructure, security, and engagement are common and must be overcome for any intervention to be effective. Educational Initiatives (Ei) has had extensive experience taking Mindspark, its Personalized Adaptive Learning software, to students in government schools. Any organization attempting EdTech interventions in capacity-constrained environments will likely face similar obstacles and may benefit from insights from Ei’s efforts.
As part of the Quality Education India Development Impact Bond (QEI DIB), Educational Initiatives partnered with Pratham Infotech Foundation (PIF) to provide access to Ei Mindspark to approximately 12,500 students in and around Lucknow, Uttar Pradesh. The project began in 2019 and is scheduled to run till 2022.
A Development Impact Bond is a type of results-based financing scheme. Metrics for ‘successful implementation’ were agreed upon at the outset by all parties, and annual evaluations would judge whether targets had been met. The QEI DIB set a target of 20 hours of Mindspark usage per year in both language and mathematics. Additionally, learning gains of up to 0.2 standard deviations (SDs) were to be achieved relative to control groups in the first year. Only if these targets were achieved, would the project continue in the next year and implementing parties be fully compensated.
In the picture, you can see Team members setting up labs in schools.
Establishing computer labs in schools was difficult given the resource constraints most schools faced, both in terms of infrastructure and time. School administrators and teachers alike were hesitant to partake since they would need to allocate slots from the schools’ timetable for Mindspark usage. To solve this, extensive orientation sessions were held within schools to help administrators and teachers understand the benefits that their students would gain from using Ei Mindspark. Additionally, teachers were also told how they could use Mindspark usage data to identify individual and collective student weaknesses, which could help improve in-class instruction. Pratham Infotech Foundation also provided a Sancharak (lab-in-charge) to support the operation of Mindspark labs by teachers and students and minimise their burden.
In addition to securing cooperation from schools, certain infrastructural requirements had to be met to establish Mindspark labs. On average, 15-20 Chromebooks were placed in each lab, depending on the number of students that would use them at a single time. A room of about 15 feet x 20 feet was needed for this. These rooms needed good flooring, walls, and roofs to prevent water damage to computers during rains. To prevent theft of equipment, grills also had to be placed on windows where needed, and doors had strong locks placed on them. Regular electricity supply was necessary to run these labs, as was mobile network so servers could be periodically updated.
Identifying schools that fulfilled the above criteria took about 1.5 months. Over 100 schools were surveyed in all, of which 55 were chosen. Finally, approximately 12,500 students from classes 1-10 were given access to Mindspark in these schools.
Despite this preparation, challenges arose in day-to-day programme implementation of the programme. Our implementation partners for this project, for example, while extremely skilled at outreach on the ground, were not familiar working with a Personalized Adaptive Learning software like Mindspark. Working in labs and engaging with teachers and students daily, therefore, proved challenging for them. To mitigate this, an intensive two-day workshop was held wherein Mindspark’s functioning was explained in-depth to them. This helped them understand the particularities of the programme, its benefits, basic troubleshooting, generating usage reports for teachers, etc. Additionally, daily and weekly usage targets for students were set in consultation with them to ensure that the end-of-year goals would be met. Following this, the team members were better equipped to take the programme forward.
Students using Mindspark in labs in Lucknow
Certain issues arose with schools as well. Teachers remained hesitant, initially, to support this. Since students used Mindspark during school hours, teachers were worried that this would hinder their ability to finish the set grade-level curriculum. Teachers’ cooperation was doubly important due to high rates of student absenteeism; they remained a key lever to convince students to attend and ensure adequate Mindspark usage. Team members continued reaching out to teachers to convince them. Crucially, the teachers also saw students benefit from Mindspark usage over time and eventually, most of them lent their support to the programme.
During the year, non-payment of electricity dues by certain schools caused their electricity connection to be suspended. Because of this, Mindspark labs were inoperable for a few weeks. Implementation teams had to consistently engage with school authorities and local officials to remedy this and, in certain cases, pay part of the dues as well.
Through regular innovation, intervention and outreach, implementation teams were able to ensure that usage targets were met between October 2019 – March 2020. Crucially, students evaluated showed that Ei Mindspark had helped them achieve significant learning improvements. Independent evaluations showed learning gains amounting to 0.23 standard deviations among users relative to students from other schools who did not use Mindspark, which was above the benchmark of 0.2 SDs stipulated in the agreement.
This experience implementing Mindspark in Lucknow has shown that having a good product alone does not guarantee success for a programme. While Mindspark’s efficacy had been proven earlier, actually improving learning outcomes depends on far more. Training of field teams for effective implementation, outreach and support to convince stakeholders at the last mile, and constant innovation to tackle challenges that arise in capacity-constrained settings is key. This is especially true in a country like India where such interventions remain a radical departure from conventional teaching methods, and hence, creating the conditions for adoption on the ground is crucial.