Education & EdTech// diagnostic

Applicants ask where their application is, and the assistant quotes the handbook

In short

An assistant that answers 'where is my application' with the admissions process is doing the only thing it was built to do: read documents. The question is record-specific, so no document can resolve it, and every one of those conversations escalates. The fix is an authenticated record lookup with a session-bound identity check in front of it — not more corpus.

Key takeaways

  • Classify the escalations before building anything. Record-specific questions are usually the largest slice.
  • No corpus answers a record question. The missing piece is a lookup, and tuning cannot substitute.
  • Identity must ride on a session the portal already authenticated. Never verify with knowledge questions.
  • Internal stage names are not answers. Map each to a sentence naming the next action, its owner and a date.
  • Every status answer needs an as-at time, because accurate and current are 2 different claims.

The assistant quotes the handbook because the handbook is all it has. "Where is my application" is a question about 1 row in the admissions system, and no policy document, prospectus or FAQ page contains that row. The model is behaving correctly against a corpus that cannot contain the answer, which is why this failure survives 3 rounds of content improvement and a change of model.

The cost is concentrated. Status chasing is usually the largest category of inbound contact between application and enrolment, it repeats per applicant rather than per topic, and a generic answer guarantees a second contact. Deflecting it into a process explanation adds a step in front of the queue rather than shortening it.

Classify what the escalated conversations were actually asking

The confirming check is a read, not a build. It establishes what share of escalations are record-specific, and that number decides whether a lookup is the next thing worth building.

  1. Take 200 conversations that escalated or ended without resolution, from a period that includes at least 1 peak week. Fewer than 200 and the tail categories are noise.
  2. Label each by what the person needed, not by what the assistant said. Use 4 buckets: record-specific, policy or process, decision or judgement, and other.
  3. Split the record-specific bucket into what it wanted: current stage, what is outstanding from me, what happens next and when, and confirm you received something.
  4. Count how many of those conversations contained an identifier the applicant volunteered — a reference number, an email address, a date of birth. That number tells you how much identity work the applicant is already trying to do.
  5. Report the record-specific share as a proportion of total conversations as well as of escalations. Leadership will ask for both, and only the first sizes the opportunity.

Four reasons a status question gets a process answer

CauseWhat you observeThe checkWhere the fix lives
No authenticated lookup existsEvery record question resolves to process prose, however it is phrasedAsk whether any read path to the application record exists at all. Usually the answer is a portal login and nothing elseAn integration plus an identity step
Stage names are internal jargonThe lookup works and the applicant asks a follow-up question about the words in the answerShow 10 raw stage values to somebody outside admissions and ask what each meansA published stage-to-language map
Status is accurate but staleThe answer matches the record and contradicts what an advisor said this morningCompare the answer's timestamp with the last sync and with the last human action on the fileRead from the source, or state an as-at time
Correct but not actionableThe applicant is told a stage and immediately asks what they should doRead 20 delivered answers and count how many name an action, an owner and a dateThe answer template, not the data
Ranked by frequency, with the observable that identifies each

Rows 2 and 4 get skipped because the integration feels like the hard part and the wording feels like polish. A technically correct lookup reporting "Referred — Faculty" produces almost as many follow-ups as no lookup at all.

A status answer that names a stage has told the applicant where the institution is. It has not told them what to do, which is the only thing they asked.

Identity has to arrive with the session, not be asked for

The lookup is the easy half. Everything dangerous here is in deciding that the person in the chat is the person on the record — 1 design works, and a whole family of designs quietly does not.

  • Session-bound is the answer. Put the assistant inside the applicant portal, behind the login that already exists, and let it read the authenticated identity from the session — an SSO session through SAML or OIDC, or the portal's own session, passed to the assistant as a short-lived signed token scoped to 1 applicant.
  • Public-channel conversations get a code, not a quiz. On a public web widget or a messaging channel there is no session, so send a one-time code to the email address or phone number already on the record and bind the conversation to it for a stated window.
  • Bind the token to the record, not the conversation. The assistant should be structurally incapable of reading another applicant's row, not instructed not to.
  • Log the identity basis with every answer. Which method authenticated this conversation, at what time, and which record it was scoped to — that log is what a later complaint is answered from.
  • Time-box it. A session authenticated for hours on a shared device is a disclosure waiting to happen, and portals are routinely opened on a parent's laptop.

There is a second, subtler pressure on the same boundary. A determined person will not attack the identity check; they will walk the assistant towards the information in small, individually reasonable steps — confirm the programme, confirm the campus, confirm whether a decision has been made. That incremental-extraction shape is the same one the tutoring side of this vertical deals with when students hint their way to the final answer, and it is defeated the same way: by making the data unreachable rather than by making the refusals better worded.

Every stage needs a sentence, an owner and a next action

Pipeline stage names are written for the people who move applications between them. Published unmodified they are meaningless or alarming. The map below is the artefact to write and have admissions sign off — 1 row per stage, agreed once, rendered identically everywhere.

Internal stageWhat the applicant is toldWhose move it isWhat they can do now
SubmittedYour application has arrived and is queued for checkingOursNothing. Expect the next update by a stated date
Awaiting documentsWe are waiting for 1 named document from youYoursUpload the named document. Nothing moves until it arrives
In assessmentAn assessor is reading your applicationOursNothing, and adding information now may reset the queue position
Referred to facultyA subject specialist is reviewing your applicationOursNothing. This stage is normal and not a warning sign
Decision pending approvalA decision has been made and is being checked before releaseOursNothing. The decision is not disclosed before release
The shape of a stage map, with illustrative internal labels

Two rules keep the map safe. A stage never implies an outcome — "decision pending approval" must not hint at which decision — and the map is versioned, because a stage renamed in the CRM with no matching entry puts raw jargon in front of applicants. Translate the map as a unit rather than phrase by phrase, for the reason in the translated answer that contradicts the English policy page.

Accurate and current are 2 different claims

A status answer read from a nightly copy is accurate as at last night, and an applicant who was telephoned this morning knows more than the assistant does. Both of you are then wrong in public. There are only 3 honest positions, and they cost different amounts.

  1. Read live from the source system on each request, and accept the latency and the rate limits that come with it. This is the only option that is simply current.
  2. Read from a replica and state the as-at time in the answer itself. Cheap, honest, and adequate for most stages — but it must be the actual sync time, not the time the answer was generated.
  3. Read from a replica and suppress the answer when the record was touched after the last sync. Requires a change feed, and worth it only where the stage carries consequences.

The deeper staleness is human rather than technical: an offer agreed verbally in a call and not yet entered is invisible to all 3 options. Name that with admissions as a process gap rather than engineering around it.

The 4 things the lookup must never return

  • Internal free-text notes. Assessor comments are written for colleagues, and a lookup that returns a whole record eventually returns one.
  • An unreleased decision, in any form, including the absence of one. "I can see a decision" is a disclosure.
  • A predicted date. "Usually about 2 weeks" becomes a commitment the moment it is said, thousands of times.
  • Anything about another person. Sibling applications, a parent's account and an agent's other candidates share the same tables, and only query scope separates them.

When the answer is not available, the escalation has to carry the applicant's identity, their reference and their own words to the advisor — a transfer payload rather than a fresh chat window, which is the discipline in the handoff from assistant to advisor as a protocol. That same write is where duplicate contacts get created, covered separately in every escalation creating a second enquirer record.

What a working status lookup still will not answer

A lookup resolves where and what next. It does not resolve why: why was I referred, why is this slower than my friend's, why does the portal say one thing and the letter another. Those are advisor conversations, and a good lookup raises their share of the queue while cutting its volume — a win that reads as a regression on any dashboard measuring escalation rate alone.

Scope it as an integration with an identity design and a stage map, not as a chatbot feature: 1 read path, 1 authentication method, 1 signed-off map, 1 audit log. That is the shape of work we scope as AI agents and automation, and the questions worth asking a supplier who proposes something larger are in choosing an AI development partner. This page sits under admissions, enrolment and student-services assistants, within our education and edtech work.

Frequently asked questions

Short answers to the follow-ups this page tends to raise.

Why can't our chatbot answer application status questions?

Because it has no read path to the application record. A retrieval assistant answers from documents, and the status of one application is not in any document — so the closest match it can find is the process description, which is what it returns. Adding pages, tuning ranking or changing model makes no difference; the missing component is an authenticated lookup against the admissions system.

How should an assistant verify who it is talking to before showing an application status?

Inherit the identity from a session the institution already authenticated. Put the assistant inside the applicant portal and read the authenticated user from the SSO or portal session, passed in as a short-lived token scoped to that 1 record. On a public channel where no session exists, send a one-time code to the email or phone number already on the record. Do not verify with knowledge questions.

Is asking for date of birth and application reference enough to identify an applicant?

No. Both are printed on correspondence that applicants routinely forward to family, schools and recruitment agents, and a date of birth is widely known. Anyone holding a copy of the offer letter would pass. Use a check where the institution controls delivery — a live authenticated session, or a code sent to the contact details already held on the record.

Should the status answer come from the CRM live or from a copy?

Live where you can afford the latency and the rate limits, and from a replica with a stated as-at time where you cannot. The unacceptable option is a replica presented as current, because an applicant who spoke to an advisor this morning will catch it. Put the as-at time in the answer rather than in a tooltip.

  • admissions
  • record lookup
  • identity
  • diagnosis
// shipped work

The work behind this page

Builds from our portfolio that this page draws on.

Read next

Working on something in this space?

Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.

Start the conversation