Standards

Computers do not store letters; they store numbers. A character encoding is simply the agreement about which number means which character, so that the byte saved by one program is read back as the same letter by another. When that agreement holds, text just works; when it breaks, the results are the familiar garbled mess that has plagued computing since its earliest days.

When a document is written with one encoding and read with another, you get mojibake — the garbled accents, mystery boxes and question marks that everyone has seen at some point. A café becomes 'café', a curly quote turns into a string of symbols, and an entire page of another language can dissolve into nonsense. The underlying cause is always the same: the writer and the reader disagreed about which number stands for which character.

From ASCII to a patchwork of encodings

The roots of the problem lie in the limited encodings of early computing. ASCII, one of the foundational schemes, covered only English — the basic Latin letters, digits and a handful of symbols — because it used a small range of numbers. That was adequate for early American computing but left no room for accented letters, let alone the scripts of most of the world's languages.

What followed was a patchwork of incompatible 8-bit encodings, each squeezing a different set of additional characters into the same limited space. One encoding covered Western European accents, another Cyrillic, another Greek, and so on, with the same number meaning different characters in each. A document only made sense if you knew exactly which of these encodings it used, and getting it wrong produced mojibake — a fragile, error-prone state of affairs.

How Unicode fixed the root problem

Unicode solved the underlying problem at its root. Rather than carving up a small number space, it assigns every character in every script — Latin, Cyrillic, Arabic, Chinese, emoji and far more — its own unique code point. Unicode is the universal catalogue: a single, agreed-upon identity for every character humanity writes, removing the ambiguity that doomed the old patchwork of encodings.

It is worth separating two ideas that are easy to conflate, because the distinction is the key to understanding the topic. Unicode defines the code points — the abstract numbers assigned to characters — but it does not by itself say how those numbers are turned into bytes on disk or on the wire. That second job, mapping code points to actual bytes, is the role of an encoding, and UTF-8 is the encoding that does it.

Why UTF-8 won the web

UTF-8 won out over the alternatives for several concrete reasons. It is backward-compatible with ASCII, so any plain English text is already valid UTF-8 with no changes. It is space-efficient for common text, using a single byte for the most frequent characters and more only when needed. And it can represent every Unicode character, so a single encoding finally suffices for every language at once.

Those properties combined to make UTF-8 the overwhelming default of the modern web, and the HTML standard recommends declaring it explicitly. The convention is to place a <meta charset="utf-8"> declaration near the top of every document, which tells the browser unambiguously how to interpret the bytes that follow. Declaring it removes any guesswork and prevents the browser from falling back on a wrong assumption.

Where bugs still creep in

Skipping that declaration, or letting the layers disagree, is exactly where problems still creep in today. If a file is saved in one encoding but served with a header claiming another, or rendered without any declaration at all, a browser may guess incorrectly and reintroduce the very mojibake Unicode was meant to eliminate. The mistakes are nearly always a mismatch between layers, not a flaw in UTF-8 itself.

The practical advice is therefore reassuringly simple: save your files as UTF-8, serve them as UTF-8, and declare UTF-8. Get those three to agree and an entire category of frustrating, hard-to-trace encoding bugs simply disappears. UTF-8 won precisely because it makes the right behaviour the easy default, and aligning your whole pipeline behind it is one of the cheapest reliability wins in web development.