Education & EdTech// diagnostic

The Hindi answer and the English policy page say different things

In short

An assistant that answers in one language and contradicts the published policy in another is usually translating its own output rather than retrieving from a translated corpus. Back-translate a fixed set of high-stakes answers and compare against the source: numbers, dates, conditions and hedges are where divergence lands. A disclaimer does not fix this. Moving translation earlier does.

Key takeaways

  • Translating the answer puts a generation step between the applicant and the policy. Translate the corpus instead.
  • Back-translation is the only cheap test: compare numbers, dates, conditions and hedges, not general meaning.
  • Hedged wording is what gets lost first. 'May be considered' becomes 'will be accepted' and the school owns the result.
  • Qualification names map, they do not translate — and the mapping is an editorial decision, not a language one.
  • 1,00,000 and 100,000 are the same number written two ways, and one of them parses wrongly almost everywhere.

When an applicant acts on an answer in their own language that does not match the published English requirement, the pipeline is almost always translating the answer rather than retrieving from a translated corpus. A generation step has been inserted between the applicant and the policy, and it is free to paraphrase a condition, drop a qualifier, or convert a number it should have quoted. Back-translate a fixed set of high-stakes answers and compare them to the source page field by field — the divergence is nearly always in the same four places.

This matters more than an ordinary retrieval bug because of who absorbs the consequence. An applicant who reads that a qualification is accepted, and applies on that basis, has spent time and money on the institution's statement. The categories where this happens are predictable: entry requirements, deadlines, document lists and anything touching immigration — which is exactly why visa and sponsorship questions get refused and routed rather than answered in any language.

Back-translate the twenty answers that cost money to get wrong

  1. Fix a set of 20 to 40 questions, not a random sample. Entry requirements per programme, application and payment deadlines, required documents, English-language evidence, deferral and refund conditions.
  2. Ask each one in every language you support, and capture the answer verbatim along with the source it cited.
  3. Translate each answer back into the source language with a translator that had no part in producing it, so the errors do not cancel out.
  4. Compare the back-translation against the cited source page field by field: every number, every date, every condition, every hedge. Not general meaning — general meaning is where this always looks fine.
  5. Record which sections were correct and which were wrong. Patchiness by section is diagnostic; uniform wrongness is a different fault.
  6. Repeat for one applicant profile that is not the default — a qualification from another system, a name in a non-Latin script — because that is where mapping failures surface.

The output is a table of answers by language and by field, and it is worth keeping as a regression set. Every corpus update and every model change should re-run it, because this class of failure returns quietly.

Five causes, and what each leaves in the diff

PatternCauseFirst move
Meaning preserved, conditions and hedges softenedThe answer is being translated after generationTranslate the corpus; quote policy text verbatim
One language consistently states an older ruleA human translation from a previous cycle treated as authoritativeVersion translations by academic year, like the source
Qualification and programme names inconsistent between answersTerminology drift — no glossary, so each answer coins its ownA locked term list, applied before generation
Names, grades or numbers mangled; some answers refuse to match a recordScript, transliteration or digit-grouping handling on the way inNormalise at intake; store the original script alongside
Answer is confident where the source is conditionalHedging lost in translation, or never retrievedTreat conditional language as a quote-only field
What the back-translation pattern points at

The second row is the one institutions least expect, because the translation was done properly — by a person, carefully, two years ago. Nothing marked it as belonging to an intake cycle, so it kept being served after the English page moved. Academic-year versioning is the single most load-bearing property of an institutional corpus, and it is covered in preparing a catalogue and handbook set an assistant can cite.

Translating the answer is the decision that produced this

There are three places translation can sit, and they are not equivalent. Translate the answer, and every reply passes through an unreviewed generation step that can restate a policy. Translate the query, retrieve from the source-language corpus and answer in the source language, and the applicant is reading a language they may not have. Translate the corpus, and every answer is retrieved from text somebody approved — with the cost that the corpus must now be maintained in every language you offer.

For anything that changes what an applicant does, translate the corpus. For everything else — opening hours, campus questions, how to reset a password — translating the answer is fine and cheap. That split is the same one that decides where automation is safe at all, argued in which student questions are safe to answer without a person. Where a translated variant does not exist, answer in the source language, say plainly that it is the published version, and offer a human — the trigger and payload for that are in the handoff protocol from assistant to advisor.

A qualification name is a mapping, not a translation

Board and qualification names carry institutional meaning that no translator can supply. A school-leaving certificate from one system does not have an equivalent in another; it has an admissions decision attached to it, made by the institution and written down somewhere. When an assistant translates the name, it invents an equivalence the institution never asserted — and the applicant reads it as a ruling.

Grades behave the same way. A percentage, a cumulative grade point average and a divisional classification are three different scales, and converting between them is policy rather than arithmetic. Hold qualification names and grade scales in a term list with an approved rendering per language, and have the assistant refuse a conversion it has no mapping for. Refusing is a correct answer here. Guessing is the failure this whole page is about.

Where 'may be considered' becomes 'will be accepted'

Admissions prose is deliberately conditional, and conditionality is the first thing lost in translation and summarisation alike. A source that reads normally requires, may be considered, or is subject to approval becomes requires, is accepted, and is approved. The applicant then acts on a promise that nobody made. Test for this specifically in the back-translation: read the modal verbs, not the nouns.

The same pipeline has a matching problem on the way in, which is worth fixing at the same time because it produces different symptoms with one root cause.

  • Scripts and transliteration. One name has several valid Latin spellings, and a record lookup that compares them literally will miss. Store the original script, normalise Unicode consistently at write time, and match on a normalised form rather than the display string.
  • Digit grouping. A figure written 1,00,000 in Indian grouping and 100,000 elsewhere is the same number, and a parser that splits on commas will read one of them wrongly. Parse with an explicit locale rather than a default.
  • Dates. Day-first and month-first order are both in use, and a deadline is exactly the field where a silent misread is expensive. Store an unambiguous form internally and render per locale on the way out.
  • Language tags. Record the applicant's language as a proper tag — the hi-IN form — on the enquiry record, so every later message goes out in the language the conversation started in rather than being guessed again each time.

A disclaimer saying the English version prevails does not help someone who cannot read the English version. It only decides who is blamed.

What to change, in the order that pays

Start with the approved-strings list for requirements, deadlines and document lists in every language you serve, versioned by academic year. Then the term list for qualifications and grade scales. Then the regression set from the back-translation exercise, run on every corpus or model change. Then coverage reporting per field per language, so a missing variant is a number somebody sees rather than something an applicant discovers.

One quality note that is easy to miss in all this: an answer that is technically correct but reads as machine-translated gets abandoned, and abandonment looks like success in a containment metric. The turn-level view that makes that visible is the same one used in students who open the tutor, take two turns and leave. Work of this kind — corpus versioning, approved strings, coverage reporting — is what we scope under internal tools and ops. This page sits inside admissions, enrolment and student-services assistants, part of our education and edtech work.

Frequently asked questions

Short answers to the follow-ups this page tends to raise.

Why does our multilingual assistant give different answers in different languages?

Because it is most likely translating its own answer after generating it, which puts an unreviewed rewriting step between the applicant and the policy. Paraphrase drops conditions and hardens hedges, so the second language ends up stating something the source page does not. Retrieving from a corpus that was translated and approved in advance removes the step entirely.

How do you test an assistant's answers in a language you do not read?

Back-translate them with a translator that had no part in producing the answer, then compare against the cited source field by field — every number, date, condition and hedge. Do not assess general meaning, which almost always looks acceptable. Keep the question set fixed at 20 to 40 high-stakes items so the comparison is repeatable on every corpus or model change.

Should we translate the documents or translate the answers?

Translate the documents for anything that changes what an applicant does — entry requirements, deadlines, document lists, fee schedules and refund conditions. Translating answers is acceptable for low-stakes questions where a paraphrase costs nothing. The dividing line is what a wrong answer costs the student, not how hard the content is to translate.

Is a disclaimer that the English version prevails enough?

No. It allocates blame rather than preventing harm, and it is meaningless to someone who cannot read the prevailing version. The workable version is narrower and more honest: quote approved text per language, cite the page it came from, mark clearly where a translated variant does not exist, and offer a named human route for anything conditional.

  • multilingual
  • admissions
  • retrieval
  • assistants
// shipped work

The work behind this page

Builds from our portfolio that this page draws on.

Read next

Working on something in this space?

Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.

Start the conversation