Case study ·
CS-02
·
Minerva
AI on pedagogy’s terms: grading efficiency up 40% in Minerva’s internal AI pilot
I led Minerva’s end-to-end AI exploration for CourseBook, working with a PM and an engineer: a stakeholder workshop, five AI principles, four prototyped concepts, and three shipped to a limited partner pilot. Five weeks, end to end.
01
Three pressures: hype, shadow AI, and the risk of standing still
CourseBook had consolidated curriculum creation, delivery, and assessment into a single block-based system, and v1 was taking shape when three pressures converged on it. The hype pull: every roadmap wanted AI bolted on, features shipped for their own sake. Shadow AI: students were already using outside tools, ungoverned and off-platform, so the real question was how, not whether. And doing nothing carried its own risk. Standing still wouldn’t pause AI; it would just shape learning without us.
The stakes are specific in education. AI touches grades, academic integrity, and student data, and a careless feature can quietly damage how people learn — while Minerva’s whole pitch to partners is pedagogy you can trust. Whatever we shipped would set the precedent for every AI decision after it.
So “add AI everywhere” became my brief to do the opposite: move us from “AI could do everything” to a focused strategy, going wide across students, instructors, and curriculum developers before narrowing to a few concepts that each earned their place.
Field note
“AI could do everything” is this decade’s “the homepage should pop.” The job is the same: turn appetite into a decision.
02
First a workshop, then principles, then pixels
Instead of jumping to concepts, I designed and facilitated a one-day workshop that put the key stakeholders in one room. Three activities did the work. Opportunity framing asked which jobs are painful or impossible without AI. A directed-activities library collected student activities where AI is the object of learning, not the shortcut: spot the hallucinations, debate an AI opponent, interview a historical figure. And a risk-and-guardrails pass tagged every idea with its risks — privacy, bias, academic integrity — and the UX countermeasures that answer them.
The most durable output was five CourseBook AI principles that every later decision was tested against:
Transparency without friction. Named assistants, not “AI” banners — we learned that loud labels made people hesitate, so trust lives in behavior.
Human-in-the-loop for consequential tasks. Grades, due dates, assessments: the AI drafts, a person decides.
Pedagogy first, novelty second. Scaffolding, retrieval practice, and metacognition, grounded in course content.
Opinionated, on evidence. Smart defaults and a point of view, guiding students and instructors toward good AI habits instead of staying neutral.
Auditability and oversight. Every AI interaction logged, reviewable by instructors and students.
The principles went straight to work as a filter, and the filter had teeth. We never litigated a screen; we checked it against the list.
03
Three calls: study not solve, drafts not grades, a bar with names
Three calls shaped the work. Here they are the way I keep them: the call, the alternative it beat, why, and the cost.
04
Four concepts, models grounded in Minerva’s own grading history, one limited pilot
Scaffolds, never shortcuts. Drafts, never decides. Coaches, never dictates.
Those kickers, one per tool, were the trust mechanics the pilot shipped on. I paired with an engineer to build working prototypes of each concept, and we refined the weightings, prompts, use cases, and output quality with stakeholders through each iteration; the AI Blocks presets were shaped closely with the academic team, so debate, explain, and peer-review activities encoded real pedagogy. The grading assistant wasn’t a generic model with a rubric bolted on, either: we grounded it in Minerva University’s own grading history — real, rubric-scored assessments — so its drafts carried the institution’s standards rather than the internet’s. Alongside the builds I ran quick usability tests and interviews with internal instructors and students, iterating on copy, entry points at the page and question level, and edit-and-accept flows. That’s where the transparency-without-friction principle earned its keep: loudly labeling features “AI” made people hesitate in our usability rounds, so the assistants got names and behaviors instead of banners. The difference between an AI feature people use and one they resent is mostly in how it asks permission.
The AI Blocks, student tutor, and grading assistant went to a limited partner pilot; the lesson-plan assistant stayed behind, and I won’t retrofit a tidy rationale onto that call. Instructors said the tools felt integrated, not gimmicky.
05
Grading efficiency up 40% in Minerva’s internal AI pilot
06
What changed beyond the interface
The screens were the fast part. The durable changes were how the company decides about AI, and they sort honestly:
Product direction I changed
An “AI everywhere” appetite became a four-concept strategy with named rejects. The impact-effort bar, not seniority, settled what shipped.
Team practice I established
Principles before pixels. The workshop’s five principles became the standing filter for every AI proposal after this project, and the risk-tagging pass traveled with them.
Influence, shared with partners
When partners asked for AI-detection tooling, I pushed back — I argued detection wasn’t reliable enough for this use — and we redirected that energy into guiding good AI habits and instructor-visible logs instead.
Execution I owned
The workshop design, the concept design, and the trust-and-disclosure testing that produced the named-assistants call.
07
The guardrails were the design work
The design work that mattered happened upstream: getting a room to agree on principles before anyone drew a pixel, then letting those principles say no. “Study, not solve” and “drafts, never decides” cost us demo appeal and speed, and they’re why instructors described the tools as integrated rather than bolted on. Trust is the design, not a disclaimer. Five weeks was enough because the hard arguments were settled in week one.
What I’d change: instrument the tools for impact from day one, so the next version of this page trades a directional number for a measured one. And pilot the fourth concept sooner — the pedagogy coach is the idea I most want real data on.
Closing note
Pilot numbers get re-measured before they get repeated. This page will carry the updated figures as the pilot grows.