I Wrote a Text Editor, and the Language Got Fixed Instead

Mere is a small language I write for myself. It has an interpreter and four compiled backends. This is about two days spent writing a text editor in it.

The editor came out unfinished. There is no search. No selection, no copy, no paste. Six keys and the arrows. And those two days still made the language better in six places — five of which live where the existing test suite structurally cannot look.

This is a measured piece. Every number below came off this machine or out of CI, and the unflattering ones are in the same tables as the flattering ones. So are the three times I was wrong.

Why an editor

I wrote a kilo-style editor in this language once before, in July. It was 223 lines, and the point of it was deliberately to add nothing to the language — to find out whether it could be written with what already existed, and what hurt while writing it.

This time I aimed at the opposite question. What does an editor need that this language does not have? I narrowed it to two axes: very large files and a language server.

Everything was measured before anything was written

I did the investigation first, and three items fell off the plan because they turned out to be unnecessary.

Expected	Measured
The language server needs a new long-lived pipe capability	It does not. A socketpair makes the child process look exactly like an already-accepted TCP connection
The event loop needs a new capability	It does not. A poll(2) wrapper already existed, and it takes any fd
Newline scanning needs SIMD	It does not. clang vectorises the scalar loop well enough that the difference is unmeasurable

The socketpair one was the prettiest. Mere’s tcp_read and tcp_write are in fact read(2) and write(2) — nothing socket-specific about them. So attaching a child to one end of a socketpair means the existing socket calls work on it unchanged. Zero compiler change; twenty-five lines of C.

And in place of the three that were not needed, three bugs nobody had planned for came out.

1. The positioned read was reading one byte at a time

The editor is a piece table. The file is never loaded; the document is a list of pieces, each naming a stretch of the original. Opening a 3 GB file allocates one piece.

That needs a way to read part of a file at an offset. Mere had file_pread — except it returned a vector of ints, one boxed integer per byte, and the C backend built it one fgetc at a time.

Pushing 208 MB through in 256 KiB pages:

	Wall time	Peak RSS
file_pread	3.64 s	10.1 MB
read_bytes (whole file)	0.33 s	210 MB

You could have memory or speed, not both.

The write side already had a byte-string version. Another dogfood added it eight months ago, having said the same thing about packing one integer per byte. Only the read half was missing.

After adding it:

	Wall time	Peak RSS
file_pread_bytes	0.36 s	1.8 MB

Ten times faster and a fifth of the memory, and the choice is gone.

Why did eight months pass? The dogfood that first asked for file_pread was a B-tree, and a B-tree wants a page it is about to index as numbers — the vector was right for it. It takes a second user, one that streams the file, before the shape turns out to be wrong.

On the WebAssembly backend the opposite was true. The host import already handed back a byte pointer, and file_pread was adding one conversion after it. The cheap thing had been underneath all along.

2. The region freed the memory and never reused it

This was the one that could not be found by reading.

Mere has region R { ... }. Whatever the block allocates is released together when it exits. An editor rebuilds its screen on every keystroke, so without one that is 5.9 GB over 20,000 redraws. With one it is 2.0 MB. Both measured.

But a loop that built a large value inside a block behaved oddly. Forty iterations of a 5 MiB value reached 216 MB.

My first reading was “the block is not reclaiming it.” That was wrong.

I put counters into the generated C and ran it:

block_release=40  freed_chain=40  big_allocs=40

It released forty times. The bookkeeping was correct. And the process grew anyway.

What was not happening was reuse. The release threw the region away whole and re-seeded it at 1 MiB, so the next iteration asked malloc for those megabytes again — and malloc does not hand the same pages back.

Keeping the largest block made the same loop 13.7 MB and flat.

Value size	Before	After
4 MiB	8.6 MB	11.3 MB
5 MiB	107 MB	13.4 MB
8 MiB	167 MB	19.5 MB
16 MiB	327 MB	35.9 MB

Resident memory now scales with one iteration rather than with the iteration count. The 4 MiB row is the cost: a region that grew no longer shrinks back.

The lesson here is not about the implementation. Peak RSS does not answer “was it reclaimed?” It answers “how much was resident at once”, and “held and reused” and “freed and re-obtained” have the same peak. What answered the question was a counter.

And the hypothesis was tested by patching the already-generated C directly and comparing — the 16x showed up before the compiler was touched at all.

3. Raw mode was not delivering the keys a program asked for

This is the embarrassing one.

There is a function that puts the terminal in “raw mode”: no echo, no line buffering. Every editor and every game calls it first.

It cleared ICANON and ECHO and stopped there. IXON stayed on.

With IXON on, Ctrl-S is XOFF and Ctrl-Q is XON. The line discipline eats both, and the program never sees either byte.

And the editor I published in July documents Ctrl-S as save and Ctrl-Q as quit.

Driven under a real pty:

Bytes drawn after Ctrl-S	0 (the terminal has stopped)
Bytes drawn after Ctrl-Q	0, process alive, file never written

For two months there was a published editor that could not be saved or quit from in a real terminal.

Why did nobody notice? Because the tests ran through a pipe, and a pipe has no line discipline. 0x13 and 0x11 both arrive, and everything looks finished.

Fixing the compiler and rebuilding — without changing one line of the editor’s own source — made it save and quit.

There was a second one of the same family: ISIG. With it set, Ctrl-Z is SUSP, so an undo bound to it silently does nothing.

But I did not treat it the same way. Clearing ISIG costs you Ctrl-C. An editor can pay that; a game that quits on q has no reason to, and folding it into raw mode would take the escape hatch away from every existing TUI to serve the one that asked. It went in as a separate call.

4. A library function cannot return a container

Drawing Japanese correctly means counting in grapheme clusters — what a reader calls one character. 👩‍👩‍👦 is seven code points, one character, two columns wide.

The clustering library already existed. Using it leaked about 60 KB per frame: 127.8 MB over 2,000 frames, perfectly linear, inside a region block.

The reason is in the language’s design. A container goes to the program-lifetime region when its allocation is not lexically inside the caller’s block — and a library function never is. So one buffer per cluster was one buffer that never came back.

There were two ways to fix it: change the type system, or stop the library using a container.

Measuring said the second was enough. A cluster is one to a handful of code points, so the quadratic that naive concatenation would pay is bounded by the length of one cluster, not of the text. The buffer was buying nothing.

2,000 frames × 40 lines of Japanese	Peak RSS	Wall time
Mutable buffer	127.8 MB (linear)	0.84 s
Plain strings	1.6 MB (flat)	0.45 s

78 times the memory, and 1.9 times faster. It still agrees with ICU on all 8,509 conformance inputs.

I left the type system alone, because looking into it showed the conservative behaviour was the correct one. An allocation that does not appear in a function’s type could be internal or could be shared with something that outlives the call, and the type alone cannot tell them apart. Something else makes that distinction, and it decides that what it cannot tell must be assumed to outlive.

While there I found a comment in the compiler claiming the opposite — “what is invisible cannot escape on its own, so binding it costs nothing.” I built a witness. It escaped. The comment is fixed.

The three times I was wrong

This is the part of a retrospective worth writing down.

I wrote that the region was not reclaiming. It was (above). I read peak RSS as an answer to “was it reclaimed”. My own notes contain an entry saying exactly that peak RSS does not answer that question, and I walked into it anyway.

I wrote that there was no display-width function anywhere. There was. It had been in the standard prelude for over a year and was in the documentation. I had searched the contrib directory and the builtin table and never the prelude.

The new one still earned its place, and measuring is what settled that: over 17,661 code points the two disagree on 2,083 (11.8%). The old one is fourteen hand-written ranges and misses 1,488 combining and format characters, U+200B ZERO WIDTH SPACE among them. Fine for lining up a table column; not fine for putting a cursor where a glyph ends, because there the error does not stay in one cell.

“There was none, so I made one” was the wrong description. “The one there was is not accurate enough” is the right one, and both documents now cross-reference each other with the number.

I fabricated a regression twice by rebuilding the compiler while the test suite was running against it. Reproducing the failure on its own is what showed it was mine. Also an entry in my own notes.

And then CI fixed one of my gates

I pushed the fixes and CI went red, where everything had been green locally.

What failed was a gate I had written. To check that a region gives back a block above the size cap, it compared peak RSS across iteration counts.

It passed on macOS. It failed on glibc.

The reason is the same as section 2. “Held and reused” and “freed and re-obtained” have the same peak. macOS hands large frees straight back to the OS, so the accumulating case happened to show up. glibc reuses the block, so it reads flat. My gate was measuring the allocator, not the compiler.

I changed the instrument rather than the expectation. The runtime now reports how many bytes the cached regions are holding — a function of the program, not of the machine.

	Bytes retained
Under the cap (5 MiB value)	8,388,608 (the grown block is kept)
Over the cap (32 MiB value)	1,048,576 (given back and re-seeded)

The numbers

Elapsed	2 days
Compiler changes	511 lines
Measurement and gates	970 lines
Documentation	307 lines
The editor	1,560 lines of my own (979 Mere / 115 C / 466 test)

Nearly twice as much went into “make this go red next time” as into the fix itself.

That looked inefficient at first. But five of the six findings lived where the tests structurally could not look, so it is the ratio you would expect. The defect was not in the subject. It was in the instrument.

What got built: a gate that drives a program under a real pty and asks whether the bytes it was sent arrived; a gate that compares the generated width table against a second implementation across 18,226 code points; and a probe that asks the terminal itself with ESC[6n. The last one is necessary because the width of an “ambiguous” character is a property of the terminal rather than of Unicode, and only the terminal can answer.

All of them were poisoned to confirm they go red. One did not, at first — I had not noticed that the language server’s JSON parser tolerates a raw newline inside a string. Changing the test input to contain a quote is what made it discriminate.

So — is there an editor?

No.

Six keys and the arrows. The language server offers nine methods and the editor uses one of them, for diagnostics. And there is no search. You can open a 208 MB log in half a second and then not look for anything in it.

Here is what there is:

208 MB opens in 0.5 s at 24.7 MB, one piece	Better than most editors
Undo is O(1) regardless of file size	The piece list has one entry per edit, not per line
Column-correct Japanese, cursor by cluster	Also better than most editors

I wrote at the start that the axis of “best” had to be chosen first — speed, Japanese, the language server, extensibility — because the answer changes what you build. Choosing two of them meant giving up on completeness as an editor, and it did.

Why it was still worth writing

July’s probe aimed to add nothing to the language and returned one item. This one returned ten of eleven. The same subject yields differently depending on what you aim it at.

And July’s editor was both a casualty and a beneficiary of this arc. Something that had not worked for two months started working without a line of its own source changing.

I had thought of a dogfood as an instrument for measuring the language. It is also the thing that receives the language’s changes. Thirteen of the three dozen dogfoods are asked, from the language’s own CI, whether their code is still a program. This made it fourteen — because the surface the editor goes through, an external function declaration the compiler does not implement, had no downstream witness at all.

If I continue, it should be search. An editor that is the only one able to open a huge file, and cannot look inside it, is the strangest thing on the list. Though what the language would gain from building it is a separate question.