Case study Side project · AI agents

SME Agent: from a rigid script to an agent that follows the conversation

Subject-matter experts publish practice sets and previous-year questions to a question bank through a long, fiddly process. My first automation was a script with exactly one right path. The agent lets people work the way they actually talk.

Role
Solo builder
Context
Side project · ed-tech content ops
Stack
Python · LangGraph · SQLite
Integrations
Question-bank API · Google Drive · Excel
Workflows: daily practice problems (DPP) and previous-year questions (PYQ)
2
Systems the agent drives: question bank, Google Drive and an Excel tracker
3
Every thread is checkpointed — stop mid-flow, pick it up tomorrow
Resumable
A PYQ thread pauses for the expert to confirm the answer key, then finishes the upload. Close the terminal at any point and the thread resumes where it stopped.
TL;DR A LangGraph agent that helps subject-matter experts turn question papers into structured question-bank entries. It parses the source, builds question payloads, updates the Excel tracker, syncs files with Google Drive and pushes to the question-bank API — pausing for human review where judgment is needed. Every conversation is a thread checkpointed to SQLite, so work can stop mid-flow and resume later.

01 · The problemOne right path, and nobody walks it

Publishing a question set is a chain of small, careful steps: pull the questions and answers out of a document, shape them into the question bank's schema, attach files, record everything in a tracker, upload.

My first automation scripted that chain end to end. It worked — as long as the user did exactly what the script expected, in exactly that order. Asked for one question, then wanted to add another before uploading? Asked something mid-way? Changed their mind about a step? The script had no idea what to do with any of it.

People don't follow flowcharts. They have goals, and they change their minds.

02 · The intuitionKnow the workflows, follow the person

An experienced colleague knows the process cold — which steps exist and what "done" looks like — but takes instructions in whatever order they come. That's the behaviour I wanted: the agent should understand the workflows and decide the next step from what the person just asked.

Two things make that safe. State has to be explicit, so the agent always knows where the task stands. And it has to be durable, so a session can stop and resume without replaying an ever-growing chat history — which also keeps prompts small.

03 · The solutionGraphs, tools, checkpoints and a human

  1. Workflows as graphs. DPP and PYQ are LangGraph workflows with typed state for the questions, the progress and the thread itself.
  2. Tools for the real systems. A question-bank client and payload builder, a Google Drive client and an Excel updater do the actual work; fuzzy matching helps line up names and entries.
  3. Human in the loop. The graph interrupts where judgment is needed — like confirming an answer key — and resumes with the expert's input.
  4. Durable threads. A SQLite checkpointer saves every thread, and each thread tracks its status — running, waiting for the user or completed — so work survives restarts.

04 · ImplementationMaking every thread survivable

Threads that survive Ctrl-C

The entry point compiles the chosen workflow with a checkpointer and wraps the run in thread-status bookkeeping. Interrupting isn't a failure — it's a pause.

graph = pyq_graph_builder.compile(checkpointer=get_checkpointer())
config = {"configurable": {"thread_id": thread_id}}

update_thread_status(thread_id, WorkflowStatus.RUNNING)
try:
    _run(graph, config)
    update_thread_status(thread_id, WorkflowStatus.COMPLETED)
except KeyboardInterrupt:
    print("Interrupted. Thread saved — resume it next time.")
    update_thread_status(thread_id, WorkflowStatus.WAITING_USER)

From the entry point, lightly trimmed.

State over transcript

Structured state keeps decisions grounded and prompts lean. The graph knows which questions exist, which steps are done and what's waiting on the user, so the next decision doesn't depend on replaying everything that was ever said.

Picking up where you left off

Starting the agent lists existing threads by workflow and status, so an expert can resume yesterday's half-finished paper instead of starting over.

05 · ResultsFrom babysitting a script to having a conversation

Scripted flow

One fixed order of steps. Any detour — an extra question, a question mid-way, a change of mind — breaks the run.

SME Agent

Requests in any order, review exactly where judgment is needed, and threads that pause and resume on demand.
  • Off-script requests no longer break the run — the agent routes them to the right step.
  • Expert review stays in the loop exactly where it matters, instead of everywhere.
  • No lost progress: a closed terminal is just a paused thread.

06 · TakeawaysWhat I'd carry into the next agent

  • Scripts encode a process; agents encode a goal. Pick the one that matches how your users actually behave.
  • Make state explicit and durable before making the agent clever.
  • Human-in-the-loop is a feature: put review where judgment lives, and nowhere else.

Building agents that keep a human in the loop? Let's compare notes.