Mountain Goat: A Short Tour of the Language

This page is a short introduction to the Mountain Goat language: what a program looks like, what values and types exist, how to declare things, and what each operator does. It is the page to read first if you have not seen the language before. Chapters 20 and 21 build and extend the compiler for it; this page only describes what you can write.

Everything on this page was run for real. Each example is a file in docs/tour/code/, and each output below is what the compiler and the program printed.

Compile and run

cd docs/part22/code && ./build.sh      # once: builds the compiler's mg-opt tool
cd ../../tour/code
../../part22/code/mgc run 01_first_program.mg    # compile to a native executable and run it
../../part22/code/mgc mlir 01_first_program.mg   # show the MLIR the front end produces
../../part22/code/mgc ptx  05_dynamic_types.mg   # compile to GPU code (PTX); nothing is launched
./run_tour.sh > tour_out.txt                      # run examples 1-6 and the first nine errors
./run_gallery.sh > gallery_out.txt                # run the gallery (examples 7-16) and three more errors
./run_more.sh > more_out.txt                      # run the extra examples (17-22)

mgc run needs LLVM 18's MLIR tools (mlir-opt-18, mlir-translate-18, clang-18) installed; see Getting Started. Chapter 20 explains each stage mgc runs.

The mental model: think in shapes

Before any syntax, one idea carries the whole language: every value is a rectangle of numbers, and every operation is a rule about rectangles. An RxC matrix has R rows and C columns. If you can say the shape of each line of your program, you can say whether it is legal, and the compiler does exactly that check, before it generates any code.

Four rules cover almost everything you will write:

Rule	Shapes in	Shape out	Example
Same shape (+ - * /)	RxC and RxC	RxC	a + b adds matching elements
Size 1 stretches	RxC and 1xC (or Rx1, 1x1)	RxC	data - col_mean(data)
Inner sizes meet (@)	RxK and KxC	RxC	the K's must match, then vanish
Reductions squash one axis	RxC	1xC (col_) or Rx1 (row_)	col_sum(a)

Example 7 follows one pair of matrices through all four rules, with the shape written beside every line. It is the same exercise you should do in your head for your own programs:

################ mgc run 07_shapes_flow.mg
  | # Think in shapes first. Each line below says what shape it has; the compiler checks every one of these claims.
  | let a = [[1, 2, 3], [4, 5, 6]]          # 2x3
  | let b = [[1, 0, 2, 1], [0, 1, 1, 2], [1, 1, 0, 0]]   # 3x4
  | print a @ b                              # (2x3) @ (3x4): the inner 3s meet and vanish -> 2x4
  | print transpose(a @ b)                   # 2x4 turned on its side -> 4x2
  | print col_sum(a @ b)                     # one total per column -> 1x4
  | print row_sum(a @ b)                     # one total per row    -> 2x1
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 4] strides = [4, 1] data = 
[[4,   5,   4,   5], 
 [10,   11,   13,   14]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [4, 2] strides = [2, 1] data = 
[[4,   10], 
 [5,   11], 
 [4,   13], 
 [5,   14]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 4] strides = [4, 1] data = 
[[14,   16,   17,   19]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 1] strides = [1, 1] data = 
[[18], 
 [48]]
  ----- exit status 0

Read the comments beside each print: (2x3) @ (3x4) gives 2x4 (the two 3s meet and vanish); transpose swaps the two sizes; col_sum leaves one total per column (1x4); row_sum leaves one per row (2x1). The printed sizes = [...] line is the shape you predicted. When the prediction and the output disagree, you have found a misunderstanding, which is the cheapest time to find it.

Maths on paper, Mountain Goat on the screen

You would write	Mountain Goat	Shape rule
A + B	a + b	same shape (or a size-1 stretch)
A ∘ B (Hadamard, elementwise)	a * b	same shape
A B (matrix product)	a @ b	inner sizes meet
Aᵀ	transpose(a)	RxC becomes CxR
−A	-a	same shape
2A + 1	a * 2 + 1	scalars are numbers known when compiling
Σ over rows, per column	col_sum(a)	RxC becomes 1xC
Σ over columns, per row	row_sum(a)	RxC becomes Rx1
mean of each column	col_mean(a)	1xC; needs a known row count
max(0, x)	relu(a)	same shape
f(x) = W x + b	def f(x: tensor[2x1], w: tensor[2x2], b: tensor[2x1]) = w @ x + b	checked at every call

The two most common slips are in the first rows: * is not the matrix product (that is @), and a ? dimension never stretches. Both have worked examples below.

What a program looks like

A Mountain Goat program is a file ending in .mg with one statement per line. A # starts a comment that runs to the end of the line. Blank lines are ignored. There are three kinds of statement: let, print and def.

################ mgc run 01_first_program.mg
  | # A Mountain Goat program is a list of lines. This is a comment: everything after '#' is ignored.
  | let m = [[1, 2, 3], [4, 5, 6]]      # a 2 by 3 matrix
  | print m                             # show it
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 3] strides = [3, 1] data = 
[[1,   2,   3], 
 [4,   5,   6]]
  ----- exit status 0

The first line is a comment. let m = ... declares a name m and gives it a value: a two-by-three matrix. print m shows it. The output is MLIR's own runtime printer, not something Mountain Goat designed: sizes = [2, 3] is the matrix shape, strides says how the data is laid out in memory, and the numbers follow.

Values and types

Mountain Goat has exactly one kind of value that can be printed or passed around: a two-dimensional matrix of 64-bit floating-point numbers (f64, the same as C's double). There are no integers, strings, booleans, lists or records. Every other piece of the language exists to build, combine or name matrices.

The type of a matrix is its shape. You write a type as tensor[R x C] where R is the number of rows and C the number of columns, with no spaces: tensor[2x3] is a matrix of two rows and three columns. Types appear only in one place, the parameters of a def (below); everywhere else the compiler works the type out for you.

Each dimension is either:

• a positive whole number, a static dimension known when the program is compiled: tensor[2x3];
• ?, a dynamic dimension known only when the program runs: tensor[?x3] is "any number of rows, exactly 3 columns", and tensor[?x?] is "any shape at all".

A tensor is the general word for an array of numbers with any number of dimensions; Mountain Goat only has the two-dimensional kind (a matrix), but keeps the word in the type because that is what the compiler's internal form calls it.

There is one more kind of value, which cannot be printed: the scalar, a plain number with no shape, like 4 or 0.5. A scalar exists only to combine with a matrix (m * 4); it is known when the program is compiled, so there is nothing to compute at run time.

Literals

A matrix literal is rows in brackets inside an outer pair of brackets: [[1, 2, 3], [4, 5, 6]] is a 2x3 matrix. All rows must have the same length. Numbers may be whole (2), decimal (1.5), negative (-2) or scientific (3e2, which is 300).

################ mgc run 02_literals.mg
  | # Matrix literals: rows in brackets, numbers may be integers, decimals, negative, or scientific.
  | let a = [[1.5, -2, 3e2]]            # one row, three columns (1x3)
  | let b = [[1], [2], [3]]             # three rows, one column (3x1)
  | print a
  | print b
  | # A bare number is a scalar. It has no shape and can only be combined with a matrix.
  | let k = 4
  | print a * k
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 3] strides = [3, 1] data = 
[[1.5,   -2,   300]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 1] strides = [1, 1] data = 
[[1], 
 [2], 
 [3]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 3] strides = [3, 1] data = 
[[6,   -8,   1200]]
  ----- exit status 0

a is 1x3 and b is 3x1: the shape is simply how the literal is bracketed. let k = 4 declares a scalar. a * k multiplies every element of a by 4, so 1.5, -2, 300 becomes 6, -8, 1200.

Declaring names with let

let name = expression gives a name to the value of an expression. The name can then be used in later lines. There are no variables that change: a let does not modify anything. A later let of the same name declares a new binding that replaces the old one from that point on; the right-hand side of the new let still sees the old value.

################ mgc run 03_let_and_shadowing.mg
  | # 'let' binds a name to a value. A later 'let' of the same name replaces it (the old value is untouched).
  | let a = [[1, 2]]
  | let a = a * 10                      # the right-hand side still sees the OLD a
  | let a = a + 1
  | print a
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[11,   21]]
  ----- exit status 0

Start with [[1, 2]]; the second line makes [[10, 20]] (using the first a); the third makes [[11, 21]]. The output is 11, 21.

A let can bind a scalar too (let k = 2 + 3 * 4 is the scalar 14, computed by the compiler).

Declaring functions with def

def name(parameters) = expression declares a function. Each parameter has a name and a type, written name: tensor[RxC]. The body is a single expression: there are no statements inside a function, so no let and no print in a body. The function's result type is worked out from the body; you do not write it. A function is used by calling it, name(arguments), anywhere an expression can appear.

################ mgc run 04_functions.mg
  | # 'def' declares a function: typed parameters, one expression as the body, the result type is worked out.
  | def scale(a: tensor[2x2]) = a * 3
  | def shift_and_scale(a: tensor[2x2]) = scale(a + 1)    # may call functions defined ABOVE it
  | print shift_and_scale([[1, 2], [3, 4]])
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[6,   9], 
 [12,   15]]
  ----- exit status 0

Two rules to know. A function may only call functions defined above it (so there is no recursion and no forward reference; the error for breaking this is shown below). And the arguments of a call must fit the parameter types: a matrix of known shape fits a parameter with the same static shape, or a ?.

Dynamic dimensions: one function, many shapes

A parameter type with ? makes the function work for many shapes. The compiler produces one compiled function, and the shapes are looked at when it runs.

################ mgc run 05_dynamic_types.mg
  | # '?' in a type means "any size, known only when the program runs". One function, many shapes.
  | def twice(a: tensor[?x?]) = a + a                      # any rows, any columns
  | def row_sums(a: tensor[?x3]) = a @ [[1], [1], [1]]     # any rows, but exactly 3 columns
  | print twice([[1, 2]])
  | print twice([[1], [2], [3]])
  | print row_sums([[1, 2, 3], [4, 5, 6]])
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[2,   4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 1] strides = [1, 1] data = 
[[2], 
 [4], 
 [6]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 1] strides = [1, 1] data = 
[[6], 
 [15]]
  ----- exit status 0

twice is called with a 1x2 matrix and then with a 3x1 matrix. row_sums accepts any number of rows but exactly 3 columns, and multiplies by a column of ones, so it adds up each row: 1+2+3 = 6 and 4+5+6 = 15.

When a dimension is ?, a mismatch cannot be seen while compiling. If two ? dimensions turn out different at run time, the program stops with an error message on standard error (Chapters 14 and 19); it does not print a wrong answer.

Expressions and operators

An expression computes a value: a name, a literal, a function call, transpose(e), a parenthesized expression, or two expressions joined by an operator.

Operator	Meaning	Shape rule
a + b, a - b	elementwise add, subtract	same shape (or ? that agree at run time)
a * b	elementwise product (the Hadamard product)	same shape
a / b	elementwise division	same shape
a @ b	matrix product	a is m by k, b is k by n, result is m by n
transpose(a)	rows become columns	m by n becomes n by m
-a	negate every element	same shape
matrix op scalar, scalar op matrix	combine every element with the number (+ - * /)	same shape as the matrix
relu(a)	max(x, 0) for every element	same shape
row_sum(a), row_max(a), row_mean(a)	one value per row	m by n gives m by 1
col_sum(a), col_max(a), col_mean(a)	one value per column	m by n gives 1 by n

Broadcasting. For + - * /, if two shapes differ but each pair of dimensions is equal or has a static 1, the size-1 dimension is repeated to fit: [[1, 2, 3], [4, 5, 6]] + [[10, 20, 30]] adds the row to both rows. A ? dimension is never broadcast, and row_mean/col_mean need a static size along the averaged axis. Chapter 22 explains all of it.

From tightest to loosest binding: unary -; then *, / and @ (left to right); then + and - (left to right). Parentheses override. So a + b * c means a + (b * c).

################ mgc run 06_operators.mg
  | # Every operator on two 2x2 matrices (and a scalar).
  | let a = [[1, 2], [3, 4]]
  | let b = [[10, 20], [30, 40]]
  | print a + b
  | print b - a
  | print a * b                        # elementwise (Hadamard)
  | print b / a
  | print a @ b                        # matrix product
  | print transpose(a)
  | print -a
  | print a * 2 + 1
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[11,   22], 
 [33,   44]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[9,   18], 
 [27,   36]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[10,   40], 
 [90,   160]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[10,   10], 
 [10,   10]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[70,   100], 
 [150,   220]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[1,   3], 
 [2,   4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[-1,   -2], 
 [-3,   -4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[3,   5], 
 [7,   9]]
  ----- exit status 0

With a = [[1, 2], [3, 4]] and b = [[10, 20], [30, 40]] the eight outputs are, in order: a + b, b - a, a * b (elementwise: 10, 40, 90, 160), b / a, a @ b (matrix product: 70, 100, 150, 220), transpose(a), -a, and a * 2 + 1. Compare the third and fifth: same inputs, different operators, different answers.

A gallery of small programs

The examples above show one construct at a time. These show constructs working together on problems you might recognise. Each is a complete file in docs/tour/code/; the output is what mgc run printed (base@ = 0x… replaces a memory address that changes from run to run). Run them all with ./run_gallery.sh > gallery_out.txt. For each one, try to predict the output before reading it.

8. Statistics of a table without loops

Rows are samples and columns are measurements. col_mean gives a 1x2; subtracting it from the 4x2 data stretches it down every row; squaring (* with itself) and averaging gives the variance of each column.

################ mgc run 08_column_statistics.mg
  | # Mean and variance of every column of a data table (rows are samples, columns are measurements), with no loops.
  | let data = [[1, 10], [2, 20], [3, 30], [4, 40]]    # 4 samples, 2 measurements
  | let mean = col_mean(data)                           # 1x2: one mean per column
  | let centered = data - mean                          # 4x2: the 1x2 mean is broadcast down all 4 rows
  | print mean
  | print centered
  | print col_mean(centered * centered)                 # 1x2: the variance of each column (mean of squares of the centered data)
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[2.5,   25]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [4, 2] strides = [2, 1] data = 
[[-1.5,   -15], 
 [-0.5,   -5], 
 [0.5,   5], 
 [1.5,   15]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[1.25,   125]]
  ----- exit status 0

The means are 2.5 and 25; the centered data has columns that sum to zero; the variances are 1.25 and 125 (the second column is ten times larger, so its variance is a hundred times larger).

9. A polynomial at many points, with one product

Row i of the Vandermonde matrix is [1, x, x²] for the i-th point, so multiplying by the coefficient column evaluates 1 + 2x + 3x² at all three points at once.

################ mgc run 09_polynomial.mg
  | # Evaluate the polynomial 1 + 2x + 3x^2 at x = 1, 2, 3 with ONE matrix product.
  | # Row i of the "Vandermonde" matrix is [1, x, x^2] for the i-th x; multiplying by the coefficient column [1, 2, 3] evaluates all of them at once.
  | let vandermonde = [[1, 1, 1], [1, 2, 4], [1, 3, 9]]
  | let coefficients = [[1], [2], [3]]
  | print vandermonde @ coefficients        # 1+2+3 = 6,  1+4+12 = 17,  1+6+27 = 34
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 1] strides = [1, 1] data = 
[[6], 
 [17], 
 [34]]
  ----- exit status 0

Check one by hand: at x = 3, 1 + 6 + 27 = 34.

10. Rotating points

Each column of p is a point. The matrix [[0, -1], [1, 0]] turns every point a quarter turn; applying it twice negates everything.

################ mgc run 10_rotation.mg
  | # Rotating points by 90 degrees is a matrix product. Each COLUMN of p is one point (x on the top row, y below).
  | let rotate90 = [[0, -1], [1, 0]]
  | let p = [[1, 0, -1], [0, 1, 0]]          # three points: (1,0), (0,1), (-1,0)
  | print rotate90 @ p                       # each point turned a quarter turn counter-clockwise
  | print rotate90 @ rotate90 @ p            # turned twice = a half turn = every coordinate negated
  | print -p                                 # the same numbers (a "-0" in the output is negative zero: it equals 0, and the sign of a zero is how floating-point arithmetic remembers which side it came from)
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 3] strides = [3, 1] data = 
[[0,   -1,   0], 
 [1,   0,   -1]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 3] strides = [3, 1] data = 
[[-1,   0,   1], 
 [0,   -1,   0]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 3] strides = [3, 1] data = 
[[-1,   -0,   1], 
 [-0,   -1,   -0]]
  ----- exit status 0

Look at the third result: -0. That is negative zero. It compares equal to 0; floating-point arithmetic keeps the sign of a zero, and negating 0 produces -0. Nothing is wrong: it is how IEEE floating point works, and -0 == 0 is true. (Whether a given zero prints as 0 or -0 depends on the sign of the operands that produced it, so do not rely on it.)

11. A Markov chain

today is a 1x2 row of probabilities and t is the transition matrix. today @ t is tomorrow; squaring t three times gives eight steps with three products instead of seven.

################ mgc run 11_markov_chain.mg
  | # A two-state weather model. Row i of t says: if today is state i, the chance tomorrow is each state. Each row adds up to 1.
  | let t = [[0.9, 0.1], [0.5, 0.5]]         # state 0 = sunny, state 1 = rainy
  | let today = [[1, 0]]                     # it is sunny today (a 1x2 row of probabilities)
  | print today @ t                          # tomorrow:        [0.9, 0.1]
  | print today @ t @ t                      # the day after:   [0.86, 0.14]
  | let t2 = t @ t
  | let t4 = t2 @ t2
  | let t8 = t4 @ t4                         # eight steps at once, by repeated squaring: three products instead of seven
  | print today @ t8                         # drifting toward the long-run mix of 5/6 sunny, 1/6 rainy
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[0.9,   0.1]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[0.86,   0.14]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[0.833443,   0.166557]]
  ----- exit status 0

The long-run answer is 5/6 sunny, 1/6 rainy (0.8333…, 0.1667…). After eight steps we are at 0.833443: the distance from 5/6 shrinks by a factor of 0.4 each day, and 0.4⁸ ≈ 0.00066.

12. Counting walks in a graph

Entry [i][j] of the adjacency matrix a is 1 when there is an edge from i to j. Matrix powers count walks: a @ a counts the two-step ones.

################ mgc run 12_graph_walks.mg
  | # Counting walks in a directed graph. Entry [i][j] of a is 1 if there is an edge from node i to node j.
  | # Entry [i][j] of (a @ a) counts the 2-step walks from i to j; of (a @ a @ a), the 3-step walks.
  | let a = [[0, 1, 1, 0], [0, 0, 1, 1], [0, 0, 0, 1], [1, 0, 0, 0]]
  | print a @ a
  | print a @ a @ a
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [4, 4] strides = [4, 1] data = 
[[0,   0,   1,   2], 
 [1,   0,   0,   1], 
 [1,   0,   0,   0], 
 [0,   1,   1,   0]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [4, 4] strides = [4, 1] data = 
[[2,   0,   0,   1], 
 [1,   1,   1,   0], 
 [0,   1,   1,   0], 
 [0,   0,   1,   2]]
  ----- exit status 0

In the first result, [0][3] is 2: node 0 reaches node 3 in two steps by two different routes (0→1→3 and 0→2→3).

13. Blurring an image

Each row of the image is a row of pixels; the blur matrix replaces every pixel with a weighted average of itself and its two neighbours.

################ mgc run 13_blur.mg
  | # Blurring a tiny image, one row at a time, is a matrix product: the blur matrix averages each pixel with its neighbours.
  | let image = [[0, 0, 9, 0, 0], [0, 9, 9, 9, 0], [9, 9, 9, 9, 9]]     # 3 rows of 5 pixels
  | let blur = [[0.5, 0.5, 0, 0, 0], [0.25, 0.5, 0.25, 0, 0], [0, 0.25, 0.5, 0.25, 0], [0, 0, 0.25, 0.5, 0.25], [0, 0, 0, 0.5, 0.5]]
  | print image @ blur                      # (3x5) @ (5x5): every pixel becomes a weighted average of itself and its left and right neighbours
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 5] strides = [5, 1] data = 
[[0,   2.25,   4.5,   2.25,   0], 
 [2.25,   6.75,   9,   6.75,   2.25], 
 [6.75,   11.25,   9,   11.25,   6.75]]
  ----- exit status 0

The sharp 9 in the middle of the first row spreads into 2.25, 4.5, 2.25; the total brightness of a row is preserved (9 before, 9 after).

14. A function applied twice

affine takes a point, a matrix and a shift. The parameter types fix the shapes, so a call with a wrong-shaped argument is rejected at compile time (example 7 in the error list above).

################ mgc run 14_affine_layers.mg
  | # A function that maps a point by a matrix and a shift, applied twice. Shapes are in the types, so the compiler checks every call.
  | def affine(x: tensor[2x1], w: tensor[2x2], b: tensor[2x1]) = w @ x + b
  | let x = [[1], [2]]
  | let w = [[2, 0], [0, 3]]               # stretch x by 2 and y by 3
  | let b = [[1], [1]]                     # then shift by (1, 1)
  | print affine(x, w, b)                  # (2*1+1, 3*2+1) = (3, 7)
  | print affine(affine(x, w, b), w, b)    # again: (2*3+1, 3*7+1) = (7, 22)
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 1] strides = [1, 1] data = 
[[3], 
 [7]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 1] strides = [1, 1] data = 
[[7], 
 [22]]
  ----- exit status 0

15. Any number of samples (and how to add a bias)

? in the row count means "any batch size", so scale compiles once and serves a batch of one and a batch of three. Adding a bias to every row needs the bias repeated once per sample, and since a ? size is never stretched, the program does the repeating with a matrix product: a column of ones (?x1) times the 1x2 bias is a ?x2 matrix of copies.

################ mgc run 15_batches.mg
  | # One function, any number of samples: rows are samples, so a '?' in the row count means "any batch size".
  | def scale(batch: tensor[?x2], w: tensor[2x2]) = batch @ w
  | print scale([[1, 1]], [[1, 0], [0, 2]])                                  # a batch of one
  | print scale([[1, 1], [2, 2], [3, 3]], [[1, 0], [0, 2]])                  # a batch of three: the same compiled function
  | 
  | # Adding a bias to every row needs the bias repeated once per sample. A '?' size is never broadcast (it is not known when compiling),
  | # but a matrix product can do the repeating: a column of ones (one 1 per sample) times the 1x2 bias is a ?x2 matrix of copies of the bias.
  | def scale_and_shift(batch: tensor[?x2], ones: tensor[?x1], w: tensor[2x2], b: tensor[1x2]) = batch @ w + ones @ b
  | print scale_and_shift([[1, 1], [2, 2], [3, 3]], [[1], [1], [1]], [[1, 0], [0, 2]], [[10, 20]])
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[1,   2]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 2] strides = [2, 1] data = 
[[1,   2], 
 [2,   4], 
 [3,   6]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 2] strides = [2, 1] data = 
[[11,   22], 
 [12,   24], 
 [13,   26]]
  ----- exit status 0

16. The ? gotcha: a program that stops by design

What happens if you skip the ones-column and write batch @ w + b with a 1x2 bias b? It compiles, because a ? might turn out to be 1. With one sample the shapes happen to match and the answer is right; with three, the program aborts with a message, not a wrong answer:

################ mgc run 16_dynamic_bias_gotcha.mg
  | # THE GOTCHA: adding a fixed 1x2 bias directly to a result with a '?' row count compiles, because '?' might turn out to be 1,
  | # but a '?' is never broadcast. With one sample the sizes happen to match; with three the program stops with an error message.
  | def scale_and_shift(batch: tensor[?x2], w: tensor[2x2], b: tensor[1x2]) = batch @ w + b
  | print scale_and_shift([[1, 1]], [[1, 0], [0, 2]], [[10, 20]])                        # fine: one row against one row
  | print scale_and_shift([[1, 1], [2, 2], [3, 3]], [[1, 0], [0, 2]], [[10, 20]])        # three rows against one: aborts at run time
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 2] strides = [2, 1] data = 
[[11,   22]]
mg.add: operand shapes differ at runtime in dimension 0
Aborted
  ----- exit status 134

The first call printed [[11, 22]]; the second stopped with mg.add: operand shapes differ at runtime in dimension 0 and exit status 134 (the shell's code for an abort). This is the run-time check from Chapter 14 working as intended. The "FAIL"-looking ending is the expected result of this example, and the fix is example 15.

More examples of each construct

The first six examples introduced one idea each. These six go back over the same ideas (literals, let, def, ? types, operators) with more cases, including the ones that surprise people. Run them with ./run_more.sh > more_out.txt; again, predict first.

17. More about literals

A literal is always a matrix. A single number in brackets is a 1x1 matrix, which is not the same thing as the bare scalar 5. A row is 1xN and a column is Nx1, and transpose turns one into the other.

################ mgc run 17_more_literals.mg
  | # More about literals. Every literal is a matrix with at least one row and column; the shape is the count of rows by the count in each row.
  | print [[5]]                                    # one number is a 1x1 matrix, not a scalar (a bare 5 is the scalar)
  | print [[0.25, -0.5, 1e-3]]                     # decimals, negatives and scientific notation can mix in one row
  | print [[1, 2, 3, 4, 5, 6]]                     # a row: 1x6
  | print [[1], [2], [3], [4], [5], [6]]           # a column: 6x1
  | print transpose([[1, 2, 3, 4, 5, 6]])          # transposing the row gives the same column
  | print [[2, 4], [6, 8]] / 2                     # a scalar divides every element: [[1, 2], [3, 4]]
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 1] strides = [1, 1] data = 
[[5]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 3] strides = [3, 1] data = 
[[0.25,   -0.5,   0.001]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 6] strides = [6, 1] data = 
[[1,   2,   3,   4,   5,   6]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [6, 1] strides = [1, 1] data = 
[[1], 
 [2], 
 [3], 
 [4], 
 [5], 
 [6]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [6, 1] strides = [1, 1] data = 
[[1], 
 [2], 
 [3], 
 [4], 
 [5], 
 [6]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[1,   2], 
 [3,   4]]
  ----- exit status 0

18. Building a calculation in named steps

let is how a longer calculation is written readably: the mean of a row, then each number's distance from it, then the average squared distance (the variance). The 1x1 mean is stretched across all eight numbers. At the end, let variance is written a second time, which shadows the first: the old value is untouched, and every earlier line already used it.

################ mgc run 18_let_chains.mg
  | # Names let a calculation be built in steps, and each step can be printed. Here: the mean and the spread of one row of numbers.
  | let x = [[2, 4, 4, 4, 5, 5, 7, 9]]           # 1x8
  | let mean = row_mean(x)                         # 1x1: 40/8 = 5
  | let centered = x - mean                        # the 1x1 mean is stretched across all eight
  | let variance = row_mean(centered * centered)   # 1x1: the mean of the squared distances from the mean = 4
  | print mean
  | print centered
  | print variance
  | # Shadowing: a new let of an old name does not change the old value; any earlier line already used it.
  | let variance = variance * variance
  | print variance                                 # 16: this is the new binding
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 1] strides = [1, 1] data = 
[[5]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 8] strides = [8, 1] data = 
[[-3,   -1,   -1,   -1,   0,   0,   2,   4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 1] strides = [1, 1] data = 
[[4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 1] strides = [1, 1] data = 
[[16]]
  ----- exit status 0

The mean is 5, the distances are -3 -1 -1 -1 0 0 2 4 and their mean square is (9+1+1+1+0+0+4+16)/8 = 4. Squaring the variance (a new let) gives 16.

19. Functions built from functions

A function may call any function defined above it. The shapes flow through every call, and each call is checked.

################ mgc run 19_function_composition.mg
  | # Functions are built from other functions defined ABOVE them. The shapes flow through every call and are checked at each one.
  | def double(a: tensor[2x2]) = a * 2
  | def add_identity(a: tensor[2x2]) = a + [[1, 0], [0, 1]]
  | def double_then_add_identity(a: tensor[2x2]) = add_identity(double(a))
  | def square(a: tensor[2x2]) = a @ a
  | let m = [[1, 2], [3, 4]]
  | print double_then_add_identity(m)               # 2m + I = [[3, 4], [6, 9]]
  | print square(double_then_add_identity(m))       # [[3,4],[6,9]] squared as a matrix product: [[33, 48], [72, 105]]
  | print double(square(m)) - square(double(m))     # (2*m^2) - (2m)^2 = 2m^2 - 4m^2 = -2m^2 = [[-14, -20], [-30, -44]]
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[3,   4], 
 [6,   9]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[33,   48], 
 [72,   105]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[-14,   -20], 
 [-30,   -44]]
  ----- exit status 0

The first result is 2m + I. The second squares it with @ (a matrix product, not an elementwise square). The third is a small identity check done by computing both sides: 2m² − (2m)² = −2m², and the output is minus twice m @ m (m @ m is [[7, 10], [15, 22]]).

20. One function, many shapes

The ? type means one compiled function serves any size. Each result's shape follows from the arguments' shapes.

################ mgc run 20_one_function_many_shapes.mg
  | # '?' in a type: ONE compiled function serves any size. The shape of each result follows from the shapes of the arguments.
  | def gram(a: tensor[?x?]) = a @ transpose(a)                 # a times its own transpose: rows x rows
  | def total_per_column(a: tensor[?x?]) = col_sum(a)           # one total per column: 1 x columns
  | print gram([[1, 2]])                                         # 1x2 times 2x1 = 1x1: [[5]]
  | print gram([[1, 0], [0, 1], [1, 1]])                         # 3x2 times 2x3 = 3x3
  | print total_per_column([[1, 2, 3], [4, 5, 6]])               # [[5, 7, 9]]
  | print total_per_column([[1], [2], [3], [4]])                 # [[10]]
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 1] strides = [1, 1] data = 
[[5]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [3, 3] strides = [3, 1] data = 
[[1,   0,   1], 
 [0,   1,   1], 
 [1,   1,   2]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 3] strides = [3, 1] data = 
[[5,   7,   9]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [1, 1] strides = [1, 1] data = 
[[10]]
  ----- exit status 0

gram multiplies a matrix by its own transpose, so the result is rows-by-rows: 1x1 for one row, 3x3 for three. total_per_column has a different result width for each call.

21. A scalar on either side

For + and * the order does not matter, but for - and / it does: 10 - a is not a - 10.

################ mgc run 21_scalars_on_either_side.mg
  | # A scalar can stand on either side of an operator, and the order matters for - and /.
  | let a = [[1, 2], [4, 8]]
  | print a - 1                  # each element minus 1
  | print 10 - a                 # 10 minus each element (the scalar is on the LEFT)
  | print a / 2                  # each element over 2
  | print 16 / a                 # 16 over each element
  | print 2 * a + 1              # scalars on the left work for * and + too
  | print -(a - 4)               # unary minus applies to a whole expression
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[0,   1], 
 [3,   7]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[9,   8], 
 [6,   2]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[0.5,   1], 
 [2,   4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[16,   8], 
 [4,   2]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[3,   5], 
 [9,   17]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[3,   2], 
 [-0,   -4]]
  ----- exit status 0

a - 1 takes one off every element; 10 - a takes every element off ten; a / 2 halves; 16 / a divides sixteen by each element. The last result shows -0: negating a 0 gives negative zero (see example 10), which compares equal to 0.

22. Which operator goes first

@, * and / bind more tightly than + and -, unary minus binds tightest, parentheses override, and operators of the same level go from left to right.

################ mgc run 22_precedence.mg
  | # Which operator goes first? '@', '*' and '/' bind tighter than '+' and '-'; unary minus binds tightest; parentheses override. Same level goes left to right.
  | let a = [[1, 2], [3, 4]]
  | let b = [[1, 0], [0, 2]]
  | print a + a * b              # a * b first (elementwise), then add: [[1,2],[3,4]] + [[1,0],[0,8]] = [[2, 2], [3, 12]]
  | print (a + a) * b            # parentheses first: [[2,4],[6,8]] * [[1,0],[0,2]] = [[2, 0], [0, 16]]
  | print a @ b + a              # the product first: [[1,4],[3,8]] + [[1,2],[3,4]] = [[2, 6], [6, 12]]
  | print a @ (b + a)            # [[1,2],[3,4]] @ [[2,2],[3,6]] = [[8, 14], [18, 30]]
  | print a - a - a              # left to right: (a - a) - a = -a
  | print -a * b                 # unary minus first: (-a) * b = [[-1, 0], [0, -8]]
  | print a @ b @ a              # left to right: (a @ b) @ a = [[1,4],[3,8]] @ [[1,2],[3,4]] = [[13, 18], [27, 38]]
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[2,   2], 
 [3,   12]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[2,   0], 
 [0,   16]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[2,   6], 
 [6,   12]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[8,   14], 
 [18,   30]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[-1,   -2], 
 [-3,   -4]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[-1,   -0], 
 [-0,   -8]]
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[13,   18], 
 [27,   38]]
  ----- exit status 0

The comments in the file work each one out. Two are worth a second look. a - a - a is (a - a) - a = -a, not a - (a - a) = a. -a * b is (-a) * b, not -(a * b); here the two happen to print the same numbers, which is why the file also gives a case, a @ b + a against a @ (b + a), where the grouping visibly matters.

Mistakes the compiler reports

The compiler checks what it can before producing any code, and reports the file and line. Each of these is one real example from docs/tour/code/errors/:

################ mgc run errors/e1_bad_statement.mg
  | # a line must start with let, print or def
  | show [[1]]
  ----- output
errors/e1_bad_statement.mg:2: error: a statement must start with 'let', 'print' or 'def', not 'show'
  ----- exit status 1

################ mgc run errors/e2_unknown_name.mg
  | let a = [[1, 2]]
  | print b
  ----- output
errors/e2_unknown_name.mg:2: error: unknown name 'b'
  ----- exit status 1

################ mgc run errors/e3_shape_mismatch.mg
  | # 2x3 and 3x2 cannot be added: the dimensions differ and neither is 1, so broadcasting cannot help
  | let a = [[1, 2, 3], [4, 5, 6]]
  | let b = [[1, 2], [3, 4], [5, 6]]
  | print a + b
  ----- output
errors/e3_shape_mismatch.mg:4: error: cannot add shapes 2x3 and 3x2: dimensions 2 and 3 differ and neither is 1
  ----- exit status 1

################ mgc run errors/e4_function_order.mg
  | # f is used before it is defined
  | print f([[1]])
  | def f(a: tensor[1x1]) = a
  ----- output
errors/e4_function_order.mg:2: error: unknown function 'f'
  ----- exit status 1

################ mgc run errors/e5_arity.mg
  | def f(a: tensor[1x1]) = a
  | print f([[1]], [[2]])
  ----- output
errors/e5_arity.mg:2: error: f takes 1 arguments, got 2
  ----- exit status 1

################ mgc run errors/e6_scalar_print.mg
  | let k = 3
  | print k
  ----- output
errors/e6_scalar_print.mg:2: error: print needs a matrix; a scalar has no shape to print
  ----- exit status 1

################ mgc run errors/e7_param_shape.mg
  | def f(a: tensor[2x2]) = a
  | print f([[1, 2, 3]])
  ----- output
errors/e7_param_shape.mg:2: error: argument shape 1x3 does not fit parameter 2x2
  ----- exit status 1

################ mgc run errors/e8_zero_dimension.mg
  | def f(a: tensor[0x3]) = a
  ----- output
errors/e8_zero_dimension.mg:1: error: bad dimension '0': use a positive integer or '?'
  ----- exit status 1

################ mgc run errors/e9_let_in_def.mg
  | def f(a: tensor[1x1]) =
  |   let b = a
  ----- output
errors/e9_let_in_def.mg:1: error: expected an expression, found the end of the line
  ----- exit status 1

In order: a line that starts with something other than let, print or def; a name that was never declared; adding a 2x3 to a 3x2 (neither differing dimension is 1, so not even broadcasting can fix it); calling a function before it is defined; giving a function the wrong number of arguments; trying to print a scalar; passing a matrix whose shape does not fit the parameter; a zero-size dimension; and a def with no expression after the =. Every one of these exits with status 1 and produces no program.

What the compiler cannot check is a mismatch between two ? dimensions, since the sizes do not exist until the program runs. That check happens at run time (Chapter 14); see Chapter 20's example 4 and Chapter 21's example 12.

Three more mistakes, all about * and @, the pair beginners mix up most:

################ mgc run errors/e10_star_vs_at.mg
  | # '*' multiplies matching elements; it does not do the matrix product. 2x3 and 3x2 do not match, so this is an error: you meant '@'.
  | let a = [[1, 2, 3], [4, 5, 6]]
  | let b = [[1, 2], [3, 4], [5, 6]]
  | print a * b
  ----- output
errors/e10_star_vs_at.mg:4: error: cannot multiply shapes 2x3 and 3x2: dimensions 2 and 3 differ and neither is 1
  ----- exit status 1

################ mgc run errors/e11_at_needs_inner_match.mg
  | # '@' needs the inner sizes to match: (2x3) @ (2x3) has inner sizes 3 and 2.
  | let a = [[1, 2, 3], [4, 5, 6]]
  | print a @ a
  ----- output
errors/e11_at_needs_inner_match.mg:3: error: cannot multiply (matmul) shapes 2x3 and 2x3: inner dimensions differ
  ----- exit status 1

################ mgc run errors/e12_transpose_fixes_it.mg
  | # The usual fix for the previous mistake: transpose one side so the inner sizes meet. This one is NOT an error: (2x3) @ (3x2) is 2x2.
  | let a = [[1, 2, 3], [4, 5, 6]]
  | print a @ transpose(a)
  ----- output
Unranked Memref base@ = 0x… rank = 2 offset = 0 sizes = [2, 2] strides = [2, 1] data = 
[[14,   32], 
 [32,   77]]
  ----- exit status 0

The first is a * b on a 2x3 and a 3x2: * pairs up matching elements, and these do not match. You probably meant @. The second is a @ a on a 2x3: the inner sizes are 3 and 2. The third is the usual cure, transposing one side so the inner sizes meet; it is not an error (the output is a times its own transpose, a 2x2).

Exercises: predict the shape

Before running anything, say the shape of each expression. Then check with mgc run. Let a be 2x3, b be 3x4, c be 2x4.

1. a @ b
2. a @ b + c
3. b @ a
4. transpose(b) @ transpose(a)
5. col_sum(a @ b)
6. a * a
7. a @ a
8. row_mean(a) + a

Answers

1. 2x4: the inner 3s meet and vanish.
2. 2x4: a @ b is 2x4, the same shape as c.
3. Error. 3x4 @ 2x3 has inner sizes 4 and 2. The order of @ matters.
4. 4x2: (3x4)ᵀ is 4x3, (2x3)ᵀ is 3x2, and 4x3 @ 3x2 is 4x2. It is exactly transpose(a @ b), a fact you can confirm with example 7.
5. 1x4: one total per column.
6. 2x3: elementwise, so the shape is unchanged (it is not a matrix product).
7. Error. Inner sizes 3 and 2, shown in the third error example above.
8. 2x3: row_mean(a) is 2x1, and a size-1 column stretches across the three columns.

Next, three small design questions.

Q1. Why does the compiler reject a * b for a 2x3 and a 3x2 instead of quietly doing something sensible?

Answer

Because there is no single sensible meaning. * is defined as "same shape, element by element"; the only stretching allowed is a size of exactly 1. Guessing (say, a matrix product) would hide a bug rather than report it. The error names the shapes so you can see which side to fix.

Q2. Why does example 16 compile but abort, when example 7's mismatch is rejected before it ever runs?

Answer

Example 7's dimensions are numbers in the source, so the compiler can compare them. In example 16 the row count of batch @ w is ?, which could be 1 at run time (and would then match the bias). The compiler cannot know, so it lets the program through and the generated code checks the sizes when it runs. Everything the compiler can check statically it does; the rest becomes a run-time check with a message.

Q3. Why does ones @ b work as a stand-in for "repeat b once per row"?

Answer

A ?x1 column of ones times a 1x2 row is ?x2; entry [i][j] is 1 · b[j], so every row is a copy of b. The matrix product does the repeating, and its shape rule (inner sizes 1 and 1 meet) is satisfied for any number of rows.

The full grammar

The language's complete grammar is in the header comment of the front end's source, shown here exactly as in mgfront.py:

"""mgfront: the Mountain Goat surface-syntax front end. Reads .mg source, writes MLIR (mg dialect) to stdout.

Built-in functions (Chapter 22): relu(m); row_sum, row_max, row_mean (one value per row, an Mx1 result) and col_sum, col_max, col_mean
(one value per column, 1xN). Elementwise operators broadcast a static dimension of size 1 (e.g. a 2x3 plus a 1x3).
Operators (Chapter 21): + - * / are elementwise on matrices ('*' is the Hadamard product) and also combine a matrix
with a scalar; '@' is the matrix product; unary '-' negates. Scalars are numbers known at compile time.
Grammar (one statement per line, '#' starts a comment):
  program := (def | stmt)*
  def     := 'def' NAME '(' [NAME ':' type (',' NAME ':' type)*] ')' '=' expr      -- a function of tensors
  stmt    := 'let' NAME '=' expr | 'print' expr
  type    := 'tensor' '[' dim 'x' dim ']'      dim := INT | '?'
  expr    := product (('+'|'-') product)*
  product := unary (('*'|'/'|'@') unary)*     unary := '-' unary | term
  term    := NAME | NUMBER | matrix | 'transpose' '(' expr ')' | NAME '(' expr (',' expr)* ')' | '(' expr ')'
  matrix  := '[' row (',' row)* ']'   row := '[' NUM (',' NUM)* ']'
Every value is a rank-2 f64 tensor. A dimension is an int or None (dynamic, written '?').
"""

(INT, NUM and NAME are the usual: digits, numbers, and letters-digits-underscores starting with a letter or underscore.)

What the language does not have

No integers, strings or booleans; no loops, conditionals or recursion; no indexing into a matrix or reading a single element; no matrices of other ranks (no vectors that are one-dimensional, no three-dimensional arrays); broadcasting only for static size-1 dimensions (a ? dimension is never broadcast); no variables that change; no input from files or the keyboard; no let or print inside a function; no way to import another file. These are not oversights to apologize for: the language is the size of the compiler this book builds, and every feature has to be lowered, verified and tested.

Where each construct goes in the compiler

You write	The front end emits (Chapters 3, 20, 21, 22)	Lowered by
[[1, 2], [3, 4]]	mg.constant	Chapter 4
a + b, a - b, a * b, a / b	mg.add, mg.sub, mg.mul, mg.div	Chapters 4, 14, 21
a @ b	mg.matmul	Chapter 21
transpose(a)	mg.transpose	Chapter 4
-a	mg.neg	Chapter 21
a + 3, 10 - a, ...	mg.scalar	Chapter 21
relu(a)	mg.relu	Chapter 22
row_sum(a), col_max(a), ...	mg.reduce (mean adds a mg.scalar divide)	Chapter 22
a + bias with a size-1 dimension	mg.broadcast then the elementwise op	Chapter 22
print a	mg.print	Chapter 13
def f(...) = ...	a func.func	Chapters 4 and 5
a call f(x)	func.call, with a tensor.cast if a static shape meets a ?	Chapter 20
a ? dimension	a ? in the tensor type; a run-time cf.assert	Chapters 13, 14, 19

Quick reference

# comment
let name = expr                       declare a matrix or scalar
def f(a: tensor[RxC], ...) = expr     declare a function (R, C: positive integer or ?)
print expr                            print a matrix
[[1, 2], [3, 4]]                      matrix literal      3.5  -2  1e3   scalar literal
+  -   elementwise            *  /   elementwise (or matrix with scalar)
@      matrix product         -a     negate        transpose(a)