Home/Languages/Aliases and Normalisation ENУКРРУС API Reference (ReDoc) ↗

Aliases and Normalisation

How any spelling of a code lands on the right language — including the ones that are broken.

Codes reach us from TMS exports, customer APIs, spreadsheets and a decade of accumulated integrations. They do not agree on much. Normalisation is what makes them comparable.

Two forms of a code

The lookup key — lowercase, - separated, whitespace stripped. pt_BR, PT-BR, pt-br and pt - br all become pt-br. Aliases are stored in this form and matched by exact equality, so the hot path is one indexed comparison and never a LIKE scan.

The canonical form — BCP-47 casing: primary subtag lowercase, script Titlecase, region uppercase. pt-BR, zh-Hans, sr-Latn-RS. This is what the API returns and what you should store.

Canonicalisation also repairs subtag order. The console holds both sr-Latn-RS and sr-RS-Latn for the same language; BCP-47 puts script before region, so both parse to the same tag and render as sr-Latn-RS.

Homoglyph repair

Before either form is computed, codes go through a homoglyph pass.

This is not hypothetical. The console database contains сeb for Cebuano whose first character is U+0441 CYRILLIC SMALL LETTER ES, not ASCII c. It renders identically in every font and can never match anything a caller sends, which is how Cebuano ended up effectively unreachable for several providers.

The repair maps the Cyrillic letters that are visually identical to ASCII ones — а е о р с х у і ј — and only when the result becomes pure ASCII, so a genuinely Cyrillic-scripted value is left alone rather than mangled. Full-width Latin (ｅｎ) is folded by NFKC in the same pass.

Where aliases come from

Source Examples Why
canonical the language's own code always present
console es-la, ES-LA for es-419 every spelling the legacy database held, including losers of ambiguous pairs
curated deu, ger, zho, iw, in, ji, mo ISO 639-2/3 and deprecated ISO 639-1 codes TMS exports still emit
provider spellings a provider used that differ from the winner keeps the losing side of a collapsed pair reachable
manual whatever a new integration turns up with added from the [[Admin-Panel

Uniqueness

An alias belongs to exactly one language, globally. This is enforced by the database, not by convention.

It matters because inbound resolution must be a function. The console let zh-CN sit on both "Chinese (PRC)" and "Chinese (Simplified)", fa-AF on both "Dari" and "Farsi (Afghanistan)", bn-BD on two Bengali rows. get_language_by_code resolved these with .first() — an unordered query.

The seed builder resolves each contest once, by:

  1. the language whose canonical code is that alias,
  2. then the one the console marked is_default,
  3. then the one with more provider mappings — i.e. the one production actually uses,
  4. then the lowest legacy id, which is what .first() returns today.

Every decision is listed in data/seed/REPORT.md, so the ones worth arguing about are visible rather than buried.

What you do not need an alias for

Case and separator variants. pt_BR already matches the alias pt-br; adding pt_br as a separate row would be redundant.

Checking a code without resolving it

/v1/normalize answers the provider-independent question — which language is this, and what do we call it?

curl "https://languages.service.custom.mt/v1/normalize?language=ZH_HANS"

{ "query": "ZH_HANS", "known": true, "matched_alias": "zh-hans",
  "language": { "code": "zh-CN", "name": "Chinese (Simplified)", "aliases": ["chs", "zh-hans", …] } }

Unlike the other endpoints, an unknown code is not an error here — known: false comes back with 200. That lets a caller classify a whole list in one pass instead of catching 404s. See Normalise incoming codes.

Next