Home/Cookbook/Normalise Incoming Codes ENУКРРУС API Reference (ReDoc) ↗

Normalise Incoming Codes

Turning a pile of inconsistent codes from an integration into canonical ones.

Goal: an integration sends you pt_BR, POR-BR, zh_hans, iw, xx-YY. You want canonical codes and a list of what you could not place — without a provider in the picture.

/v1/normalize answers exactly that, and an unknown code is not an error:

curl "https://languages.service.custom.mt/v1/normalize?language=ZH_HANS"

{ "query": "ZH_HANS", "known": true, "matched_alias": "zh-hans",
  "language": { "code": "zh-CN", "name": "Chinese (Simplified)",
                "kind": "regional", "base_language_code": "zh", … } }

curl "https://languages.service.custom.mt/v1/normalize?language=xx-YY"

{ "query": "xx-YY", "language": null, "matched_alias": null, "known": false }

200 both times. That is deliberate: classifying a list in one pass beats catching 404s in a loop.

Cleaning a whole list

def classify(codes: list[str]) -> tuple[dict[str, str], list[str]]:
    """Return {input: canonical} and the list of codes we could not place."""
    canonical, unknown = {}, []
    for code in codes:
        body = httpx.get(f"{BASE}/v1/normalize", params={"language": code}).json()
        if body["known"]:
            canonical[code] = body["language"]["code"]
        else:
            unknown.append(code)
    return canonical, unknown

The unknown list is the actionable output: each entry is either a typo on their side or an alias missing on ours. Add the real ones from the admin panel — see Add a new language.

Storing the canonical code

Store language.code, not the raw input. This is what makes two records comparable: a project stored as pt_BR and one stored as por-br are the same language, but nothing short of normalisation will tell you so.

Keep the raw value too if you need to echo it back to the integration.

Detecting duplicates you already have

canonical, _ = classify(existing_codes)
from collections import Counter

for code, count in Counter(canonical.values()).items():
    if count > 1:
        print(f"{code}: {[k for k, v in canonical.items() if v == code]}")
# zh-CN: ['zh_hans', 'zh-CN', 'CHS']

Three rows in your database, one language.

Next