How observability helps Quill in its mission to help kids write better

Profile picture for user pwainewright By Phil Wainewright November 6, 2020
Summary:
Quill, which helps low income school students improve their writing skills, discusses the impact of observability tools provided under New Relic's donation program

Fleming County students learning with Quill
Fleming County students using Quill (via Quill.org)

One of the concerns as schools closed at the height of the COVID-19 lockdown earlier this year was its impact on the progress of already disadvantaged pupils. Denied in-person attention as they grappled with remote learning, would they fall even further behind? The response from many teachers across the US was to turn to Quill, a non-profit organization dedicated to helping low-income students improve their writing skills, which in less than six weeks saw over a million new students sign up for its online service.

With just 22 members of staff, including a six-person software engineering team, the sudden demand was a test of the organization's resilience, driving its total user population above three million. Executive Director Peter Gault recalls:

For us, that was a huge spike in new users. And it was about five times our normal usage compared to last year ...

As you can imagine, having a million new students joining in a month, that's an incredible amount of traffic for a team of six engineers.

Quill stayed on top of the traffic surge using application monitoring software from New Relic, which it first deployed six years ago under the vendor's non-profit program. Quill has since transitioned to the full-stack New Relic One product under the vendor's recently launched Observability for Good program, which offers non-profits up to five users and 1TB of data ingest free of charge, along with discounts for AI functionality and access to pro bono services.

Why observability is better than Slack

Gault believes the move to observability will help Quill's team do more with less. Its service runs on Heroku's Ruby on Rails infrastructure, and the team has had to use a range of tools such as Rollbar, Sentry and Skylight alongside New Relic to monitor various aspects of performance. The team was using Slack as an ad-hoc platform to bring all of this information together, he explains:

All these services had slightly different strengths. But it meant that all of our information was spread out across all of these different SaaS tools. That to us obviously created a high risk in terms of not being able to see the key information in one place.

So people, I think, treated Slack as their way of solving that problem — just send everything to Slack and you'll have all of your information. But Slack, of course, can get very, very noisy, when you have all of these integrations, and then all this data.

So I see observability as being this next way forward of trying to have consolidation, of having one source of truth around where all of your data lives, instead of trying to have it split over a number of different sources ...

We're a small team, we're not available 24/7. Just getting too much low-quality information really prevented us from being efficient in building and supporting students.

Instead of having to bookmark different pages and tools to check on metrics such as back-end traffic, front-end request times, database response times and so on, engineers can now bring all their key information into a single view. Gault sums up:

[It] enables us to really pull together all of the different tools, having one dashboard where we can see all of our really key performance metrics in one place ... to be able to very quickly solve bottlenecks and issues so that we can rapidly scale the platform [and] support all of our students and teachers.

Quill gives students instant feedback

Performance is crucial because the Quill service revolves around giving instant feedback to students, focused on improving their language syntax and grammar skills. An initial online assessment uses AI to build a personalized pathway for each student to practice their skills. Then, as they go through the exercises and type in responses, it automatically analyzes their writing and gives immediate feedback to help them revise and improve their work, based on the errors or issues detected. The process is designed to emulate the one-to-one attention of the best in-person teaching, as Gault explains:

It's an incredibly labor-intensive process. As a result, a lot of students — particularly low income students — really don't get enough practice and feedback that they need to become strong writers. That's really what Quill's mission is trying to address, trying to help 10 million low-income students become strong writers. By using AI to automatically grade student writing and provide feedback, the students can revise and improve their work.

One of the most important metrics to track is browser performance, since that can significantly impact the user experience. Gault explains:

That's a big challenge for us. A lot of our students are using low-cost devices ... Combined with schools having poor wifi potentially, and weak devices, being able to look not just at back-end performance, but also on-device performance is really critical for us.

Value of full stack observability

Reducing the time and resource that the engineering team has to spend on keeping the core infrastructure running has a direct impact on its ability to tackle issues like the user experience and adding new capabilities to the Quill service. This is the real value of the full-stack observability platform. Gault says:

The more time that we're spending on things like trying to use all these different tools to be able to see all this data, the less time that we're spending on building the platform.

We have to be very careful with our time and our engineers' time. So anything that we can do to make them more efficient just has a huge, exponential impact in terms of the development of the product and our ability to serve students.

New Relic formally launched the Observability for Good non-profit donation program at its Nerd Days virtual event last month. It also unveiled a partnership with Code for America, a national network of volunteers who put technology to work to help state county and local governments better serve their communities.

Code for America's ninety volunteer brigades have delivered applications to achieve goals such as improving access to the social safety net, helping clear eligible criminal records, letting voters check eligibility or look up campaign finance transparency, making tax filings simple so people can get tax credits and stimulus checks, publishing online resource maps that help people find local food banks, and resource planning tools for hospitals.

My take

The digital world puts a premium on literacy, so it's fitting to see technologists using digital tools to help raise literacy standards among those who might otherwise miss out. In return for its help, New Relic's non-profit partners provide strong use cases for the impact of its technology on their operations.