Capra: Silicon Compiler from Scratch¶
About the animals in this picture and where they are standing
- Alpine ibex (Capra ibex) lives in the European Alps, usually above the tree line on steep rock, and is a famously sure-footed climber. Males carry horns that can reach about a metre. Hunting left only a small remnant, around the Gran Paradiso in Italy, by the early 1800s; reintroductions have since restored it across the Alps.
- Markhor (Capra falconeri) lives in the mountains of Pakistan, Afghanistan and neighbouring regions. The males' horns twist in a corkscrew; the name is often explained as Persian for 'snake-eater'. It is Pakistan's national animal, and community-based conservation has helped some populations recover.
- Nubian ibex (Capra nubiana) lives in the rocky, arid hills of north-east Africa and the Arabian Peninsula, from the Sinai and Egypt south to Sudan and Ethiopia. Its coat matches the desert rock, and the males' ridged horns sweep back like a sickle.
- Bezoar ibex (wild goat) (Capra aegagrus) lives from Turkey and the Caucasus through Iran to Pakistan. It is the wild ancestor of the domestic goat, which people began keeping some ten thousand years ago.
- Domestic goat (Capra hircus) descends from the bezoar ibex and is among the oldest domesticated animals. There are on the order of a billion of them, kept for milk, meat, fibre and as pack animals.
The scene is imaginary: it is set on Earth, with its Moon in the sky. The animals are stylised drawings, shown where they could not live, for effect.
An open-source LLM-inference accelerator, from Verilog to the compiler that targets it.
This book builds a small neural-network accelerator the way a chip team would, in the order a chip team would, and then builds the compiler that turns a model into programs for it. Everything runs on a laptop with three open-source tools: Icarus Verilog and Verilator (two independent simulators) and Yosys (synthesis). No vendor tools, no FPGA board, no licences.
Where to start: the Introduction is a bird's-eye view of the whole book (motivation, objectives, scope, roadmap, how to read it). Then Getting Started installs the tools and Background explains Verilog and what each tool does.
What you will build
| Part | Chapters | What you end up with |
|---|---|---|
| 1. Foundations | 1-3 | The build-simulate-synthesize-mutate loop on one small circuit; an int8 multiply-accumulate unit with its overflow behaviour; the quantization arithmetic and the requantizer that connects layers |
| 2. The Compute Engine | 4-6 | A systolic array; a scratchpad, a DMA engine and double buffering; exp, reciprocal and softmax hardware |
| 3. The Whole Chip | 7-9 | An instruction set, a sequencer and a chip that runs programs; attention with a KV cache; batching and a roofline analysis of why decoding is slow |
| 4. From Verilog to Gates | 10 | Synthesis to a cell library, a timing analyzer, gate-level simulation, fault grading and equivalence checking |
| 5. Compilation | 11-12 | Capra, a compiler from a model graph to the chip's instructions; a tiny transformer decoding tokens on the simulated silicon |
| 6. Case Studies | 13-17 | Industrial techniques built and tested on the stack: int4 weight-only quantization, grouped-query attention, paged KV cache with a sliding window, speculative decoding, mixture-of-experts routing |
| Appendices | A-H | Primers (A-G): digital logic, Verilog, number formats, linear algebra for ML, transformers and LLM inference, memory systems, the EDA flow; and (H) the answers to every chapter's and appendix's self-check questions |
The rules of the book
- Every circuit has a golden model written in Python that shares no code with the Verilog. A testbench compares the two, and the same testbench runs in two simulators.
- Every test is tested. A mutation script breaks the Verilog one line at a time; a test that does not notice has a gap, and the book says what it found.
- Every number is measured or derived, and says which. Cycle counts come from simulation, areas from Yosys, and where a figure is an assumption (a clock speed, a memory bandwidth) the page says so.
- This is a model of a chip, not a chip. Nothing here has been placed, routed, timed against a real process, taped out or run on silicon. Each chapter says what its numbers can and cannot show.
This book is a companion to Unix OS from Scratch and IR Forge: A Compiler from Scratch; Chapters 40 to 43 of the compiler book built GA-1, a cycle-counting software model of a matrix accelerator. This book builds a hardware description of one, and the compiler for it.
Under development.