Control Characters: What LF, CR and NUL Really Do
Dev Tools

Control Characters: What LF, CR and NUL Really Do

A Machine That Had to Be Told Everything

Open any text editor today and pressing Enter just works: the cursor drops to a new line, no further instructions required. That behavior is a modern convenience layered on top of a much older, much dumber machine. ASCII's lowest 32 codes, 0 through 31, were not designed for editors at all. They were designed for teleprinters — electromechanical typewriters that received a stream of bits over a wire and had to be told, explicitly, every physical action to take. There was no screen, no cursor, no concept of "the next line" happening automatically. If you wanted the print head to move, you sent a code that meant move it. That's the whole reason a "control character" doesn't print a glyph: it isn't text, it's a command aimed at a piece of hardware that no longer exists in most workplaces, sent down a channel that still does.

Our companion piece on Unicode planes, normalization and sort order follows the standard past those 128 slots and into the code space that grew out of it. This one stays inside the bottom 32 and a handful of stowaways from later in Unicode, and asks a narrower question: what job was each of these commands hired to do, and where does that old machinery still leak into software running today? Paste any of the codes below into this site's ASCII & Unicode converter and check the numeric value for yourself, instead of guessing from how a character renders.

Two Separate Motions Pretending to Be One Keystroke

A typewriter carriage does two distinct mechanical jobs when you finish a line: it slides the paper sideways back to the left margin, and it rolls the platen so the next strike lands one row down. On paper these read as a single gesture. On a teleprinter they were two separate commands, because the two motors doing the work were separate and neither one implied the other. Code 13, carriage return (CR), meant "send the print head back to column one" — a horizontal move that changes where you are on the current row without touching which row you're on. Code 10, line feed (LF), meant "advance the paper one row" — a vertical move that never touches the horizontal position by itself. A terminal that received only an LF would print the next character directly beneath where the last one left off, staircasing text diagonally down the page, because nothing had told it to go back to the margin.

That's not a hypothetical: teleprinters genuinely required both codes, sent together, to reproduce what a mechanical typewriter's return lever did in one hand motion. Software written for those machines just forwarded the pair straight through, and once operating systems started storing text as files instead of driving a physical roller, the pairing stuck around as a convention rather than a mechanical necessity.

Why Three Platforms Ended Up With Three Answers

Once line endings moved from hardware into stored files, each operating system lineage kept a different piece of that teleprinter habit. Windows, tracing its ancestry through MS-DOS back to CP/M, kept both codes together as CRLF (bytes 13 then 10) — the literal, unmodified pairing the teleprinter era used. Unix, arriving from a research environment that wanted the smallest workable representation, dropped the CR entirely and let a bare LF stand for the whole line ending, trusting the terminal driver to supply the visual carriage return on output rather than storing it in the file. Classic Mac OS, before Mac OS X, went the third way and kept only the CR, dropping the LF instead — a choice that had nothing to do with either of the other two and everything to do with how its own terminal drivers were built. When Apple rebuilt Mac OS X on a Unix foundation, it adopted the Unix convention, and the old CR-only files became mostly a historical curiosity you now meet only in decades-old archives.

The practical fallout of three answers to one question is visible the moment a file crosses platforms without translation. A script edited on Windows and moved to a Unix build server can fail with an error pointing at the very first line, because the shebang line #!/bin/bash is followed by an invisible carriage return that Unix treats as part of the command name, turning bash into a program called bash^M that doesn't exist. A diff tool comparing a Windows-edited file against its Unix original can report every single line as changed, even when the visible text is identical, because the line-ending bytes themselves differ. Git's core.autocrlf setting exists specifically to translate between the two conventions at checkout and commit time so a shared repository doesn't accumulate a mix of both.

TAB: A Command to Jump, Not a Fixed Number of Spaces

Code 9 is horizontal tabulation — TAB. Like CR and LF, it began as an instruction to a physical machine: typewriters equipped with tab stops let an operator set fixed columns in advance, then press a single lever to jump the carriage straight to the next one, instead of tapping the space bar repeatedly and hoping the count came out even. That's the origin of a detail that trips up anyone who thinks of indentation as "some number of spaces": a tab is one instruction, one byte, one code point, regardless of how many columns it ends up spanning on the page. How far it jumps was never fixed by the character itself — it depends on where the tab stops are set, in the terminal, the editor, or the printer reading it.

That indirection is exactly why tabs versus spaces is a live argument in programming and not a settled question: a file indented with actual tab characters renders at whatever width each viewer's tab stops are configured to, so the same source file can look properly aligned in one editor and visibly ragged in another, while a file indented with literal space characters renders identically everywhere because a space is always exactly one column, in every viewer, with no configuration involved. Mixing the two inside one block of code is the classic failure mode: a line that looks aligned in the editor that produced it can appear shifted by several columns in a reviewer's editor set to a different tab width, purely because two different rendering rules were combined in the same file without either side realizing it.

NUL: The Byte That Means "Stop Reading"

Code 0, NUL, has no printed form and predates even the idea of "empty" in most programming contexts — it's a byte whose entire job is to signal absence or termination, never to represent a symbol. Its most consequential role today is inside the C programming language, where a string is not stored with a separate length field the way many higher-level languages store one; instead, the string is just a run of bytes in memory, and the convention that tells any function where that run ends is a single NUL byte placed immediately after the last real character. A function like strlen does not know how long a string is going to be until it starts at the beginning and reads forward one byte at a time, counting, until it hits that NUL and stops.

That convention is efficient, and it is also the direct ancestor of an entire category of security bug. If an attacker can smuggle a NUL byte into the middle of data a program expected to be a clean filename or path, some layers of the system will read past it, expecting more characters, while other layers stop dead at the NUL and treat everything after it as if it doesn't exist — the two halves of the system disagreeing about where the string actually ends. That mismatch, sometimes called a NUL byte injection, has been used to make a validation check look at one string while the code that eventually opens a file looks at a shorter, truncated version of the same bytes. The fix isn't clever parsing; it's rejecting embedded NUL bytes outright in any field that isn't supposed to contain them, since a legitimate filename or user-entered string has no honest reason to carry one.

Control Characters: What LF, CR and NUL Really Do

The Invisible Characters That Are Not Control Codes At All

Everything above lives in the C0 range, U+0000 through U+001F, plus DEL at U+007F. It's worth being precise about that boundary, because a second, unrelated family of troublemakers gets lumped in with "control characters" in casual conversation and isn't part of that range at all — these are ordinary Unicode characters, formally classified as spacing or formatting characters, that simply happen to render as nothing or as something indistinguishable from a plain space. Confusing the two groups leads to the wrong fix: you can't strip these the way you'd strip a stray CR, because tools that filter "control characters" by checking the C0 range will pass every one of them straight through untouched.

  • U+00A0 NO-BREAK SPACE. Visually identical to an ordinary space, but it explicitly tells a line-wrapping algorithm not to break the line at that point — the exact behavior you want between a number and its unit, so "10 km" never splits across two lines. It becomes a bug the moment it lands somewhere by accident, usually pasted in from a word processor or a web page that used it for layout spacing: a search for a plain space fails to match it, a CSV parser that splits fields on whitespace may or may not treat it as a delimiter depending on the library, and a dictionary lookup or exact-string comparison against user input silently fails because "café" typed with a real space and "café" pasted with a no-break space are, byte for byte, two different strings that look printed identically.
  • U+200B ZERO WIDTH SPACE. Renders as literally nothing — no width, no glyph, invisible in every normal font. It exists to mark a legal word-break point inside a long unbroken run of characters, such as a URL or a compound word in a script that doesn't use spaces between words, so a browser knows where it's allowed to wrap without inserting a visible mark. Loose in the wrong place, it's a diff tool's nightmare: two lines of source code that render as pixel-identical in every editor can differ by an invisible zero-width space, so a code review shows no visible change while the diff tool insists the line was edited, and a build script comparing file contents for a cache key gets a different hash for what looks, on screen, like the exact same file.
  • U+FEFF, read as the byte order mark. This one code point does two different jobs depending on context. At the very start of a file, sequences of bytes decoded a specific way signal which byte order a UTF-16 file uses, or simply mark a file as UTF-8 by convention — that's the byte order mark, or BOM, and some editors and export tools prepend it automatically. Read back by a parser that isn't expecting it, that leading sequence lands as a handful of extra, invisible bytes glued onto the very first line of the file. A JSON parser fed a file that opens with a BOM followed by { can reject the entire document as malformed, because the very first character it sees isn't the opening brace it required — it's three bytes it doesn't recognize as valid JSON syntax at all. A shell script with a BOM at the top can fail its shebang check the same way a stray carriage return does, because the interpreter path lookup doesn't start where the script's first visible character appears to be.

Finding Them: Read the Code, Don't Trust the Rendering

None of the characters above announce themselves visually — that's the entire reason they cause trouble. The reliable way to confirm one is present is the same principle used for encoding mismatches generally: stop reading the rendered text and inspect the underlying numbers instead. On a command line, od -An -tx1 file_to_check.txt | head -20 or xxd file_to_check.txt | head -20 dumps the file's raw bytes in hex regardless of how any editor chooses to display them, and a run of 0d 0a pairs where you expected bare 0a tells you immediately that CRLF line endings snuck into a file a Unix tool expects to see LF-only. Grep has a quieter trick for the same job: grep -U -c $'\r' filename counts lines containing a literal carriage return without letting the terminal reinterpret it first.

For characters further out in Unicode — a no-break space, a zero-width space, a stray BOM — a byte dump still works but gets harder to read by eye once the file has any non-ASCII text at all, because those characters are two or three bytes wide in UTF-8 rather than one. Run the suspect text through this site's ASCII & Unicode converter instead and that step disappears: it reports the code point behind every character you paste, so a space that measures out as U+00A0 rather than the ordinary U+0020 is visible immediately, without decoding hex by hand. The same lookup habit is worth applying to any character that looks perfectly ordinary but behaves strangely — a search that won't match, a diff that won't stay quiet, a parser that rejects input that looks fine to your eyes. If instead the trouble is a whole word turning into different letters rather than an invisible gap appearing where none should be, that's a distinct failure with its own mechanism — read the breakdown in why your ñ turns into ñ and how to fix it.

A Note on the C1 Range, and a Quick Reference

Unicode reserves a second, less commonly seen block of control codes at U+0080 through U+009F, called the C1 controls. The Unicode Character Database itself leaves this block formally unnamed; the names attached to individual C1 codes in most references — things like NEXT LINE or PARTIAL LINE FORWARD — come from ISO 6429 (equivalently ECMA-48), not from Unicode's own naming authority. In practice this range rarely appears in ordinary text files; it's more likely to be misread bytes from a different encoding than a deliberate control sequence, which is itself worth knowing when a byte dump turns up a value in that range unexpectedly.

  • NUL (0) — string terminator in C; strip if found in fields that shouldn't carry it.
  • TAB (9) — one character; width depends on the viewer's configured tab stops, not the byte itself.
  • LF (10) — the whole line ending on Unix and in modern Mac OS X files.
  • CR (13) — the whole line ending in pre-OS X classic Mac files; paired with LF as CRLF on Windows.
  • ESC (27) — introduces a longer escape sequence in terminal control codes, not a complete command by itself.
  • DEL (127) — outside the C0 block by numbering, but grouped with it by behavior: no glyph, and every one of its seven bits reads 1 — the only code point where that's true, which is exactly why punching all seven holes on paper tape was the era's way to void a mistake.
  • U+00A0, U+200B, U+FEFF — ordinary Unicode characters, not C0 controls, that render as nothing or as a plain space and break diffs, parsers, and lookups precisely because they look like nothing happened.

Every one of these is a byte or two you can hunt down deliberately once you stop trusting how the text looks on screen and start reading what it actually contains.

← Back to Blog