Blok uses only the cookies needed to run the site and remember your preferences. See our Privacy Policy.

Week 1

Lecture 1

Week 1 · Geometry in n dimensions

Length, Orthogonality and Unit Vectors

Everything in this course is built from one operation: the dot product. It measures length, it decides when two vectors are at right angles, and it is the only tool you need to do geometry in dimensions you cannot picture.

In $R^{2}$ and $R^{3}$ you can see what "length" and "right angle" mean. In $R^{7}$ you cannot.

So we do the reverse. We take one algebraic operation, check that it gives the right answers in the two dimensions we can see, and then DEFINE length and angle by it everywhere else.

Definition

The dot product (inner product)

For $x, y \in R^{n}$ ,

⟨ x, y ⟩ = i = 1 \sum n x_{i} y_{i} = x^{T} y

Multiply matching entries, add them up. The result is a single number, not a vector.

Source: Lecture 3, p. 2

Blok explainer

Two notations, one object. Bretscher writes $x \cdot y$ ; the lectures write $⟨ x, y ⟩$ .

They mean exactly the same thing for real vectors. This guide uses $⟨ x, y ⟩$ , because that is the notation the exam is written in — but do not be thrown when the textbook uses a dot.

Worked exampleBlok explainer

Compute $⟨ x, y ⟩$ for $x = (1, - 1, 2, - 2)$ and $y = (2, 3, 4, 5)$ .

Solution

Pair off the entries and multiply:

1 \cdot 2 + (- 1) \cdot 3 + 2 \cdot 4 + (- 2) \cdot 5

That is:

2 - 3 + 8 - 10 = - 3

Answer

$⟨ x, y ⟩ = - 3$

One number out. A negative one, which will turn out to mean the two vectors point somewhat away from each other.

Source: Bretscher §5.1 Exercise 6

Definition

Orthogonality, length, unit vectors

Let $v, w \in R^{n}$ .

• $v$ and $w$ are PERPENDICULAR (or ORTHOGONAL) if $⟨ v, w ⟩ = 0$ .
• The LENGTH (magnitude, norm) of $v$ is $∥ v ∥ = ⟨ v, v ⟩$ .
• $u$ is a UNIT VECTOR if $∥ u ∥ = 1$ , equivalently $⟨ u, u ⟩ = 1$ .

Source: Bretscher §5.1.1

Blok explainer

Why is $∥ v ∥ = ⟨ v, v ⟩$ the right definition of length?

Because in $R^{2}$ it IS Pythagoras. With $v = (v_{1}, v_{2})$ :

⟨ v, v ⟩ = v_{1}^{2} + v_{2}^{2} ⟹ ∥ v ∥ = v_{1}^{2} + v_{2}^{2}

That is the hypotenuse of the right triangle with legs $v_{1}$ and $v_{2}$ . The definition in $R^{n}$ is the same formula with more terms.

Memory tipBlok explainer

The square root is a nuisance in almost every proof, so almost nobody works with $∥ v ∥$ directly. They work with $∥ v ∥^{2} = ⟨ v, v ⟩$ and take the root at the very end.

See a norm in a proof? Square it immediately. $∥ v ∥^{2} = ⟨ v, v ⟩$ turns geometry into algebra you can expand.

Worked example

Find the length of $v = (2, 3, 4, 5)$ .

Solution

∥ v ∥ = ⟨ v, v ⟩ = 2^{2} + 3^{2} + 4^{2} + 5^{2}

= 4 + 9 + 16 + 25 = 54

And $54 = 9 \cdot 6$ , so:

∥ v ∥ = 36

Answer

$∥ v ∥ = 54 = 36$

Source: Bretscher §5.1 Exercise 3

Theorem

Scaling a vector scales its length

For any $v \in R^{n}$ and any scalar $k$ ,

∥ k v ∥ = ∣ k ∣ ∥ v ∥

Source: Bretscher §5.1 Exercise 25a

Proof

$∥ k v ∥ = ∣ k ∣ ∥ v ∥$ .

Square the left-hand side:
$∥ k v ∥^{2} = ⟨ k v, k v ⟩$
Why we’re allowed to: That is the definition of the norm, rearranged. Squaring is safe because both sides are $\geq 0$ .
Pull both scalars out:
$⟨ k v, k v ⟩ = k^{2} ⟨ v, v ⟩ = k^{2} ∥ v ∥^{2}$
Why we’re allowed to:
Take the square root:
$∥ k v ∥ = k^{2} ∥ v ∥ = ∣ k ∣ ∥ v ∥$

That theorem is what makes NORMALISING possible. If $v \neq = 0$ , then

u = \frac{1}{∥ v ∥} v

is a unit vector pointing the same way as $v$ .

Check:

∥ u ∥ = \frac{1}{∥ v ∥} \cdot ∥ v ∥ = 1

Source: Bretscher §5.1.1

Memory tipBlok explainer

You will normalise a vector on almost every page of this course — Gram-Schmidt, QR, the spectral theorem and the SVD all end with it.

Normalise = divide by your own length. Direction survives, size becomes 1.

Common mistakeBlok explainer

Dividing a vector by its length means dividing EVERY entry by that one number. It does not mean dividing entry $i$ by entry $i$ .

For $v = (2, 1, - 2)$ we have $∥ v ∥ = 4 + 1 + 4 = 3$ , so

u = \frac{1}{3} 21 - 2 = 2/3 1/3 - 2/3

Keep the $\frac{1}{3}$ out front rather than writing thirds inside — it is less to write and much less to get wrong later.

Orthogonality extends from vectors to whole subspaces.

A vector $x \in R^{n}$ is orthogonal to a subspace $V$ if it is orthogonal to EVERY vector in $V$ : $⟨ x, v ⟩ = 0$ for all $v \in V$ .

Source: Bretscher §5.1

Theorem

Checking orthogonality to a subspace

Let $v_{1}, \dots, v_{m}$ be a basis of a subspace $V$ of $R^{n}$ . Then $x$ is orthogonal to $V$ if and only if $x$ is orthogonal to each of $v_{1}, \dots, v_{m}$ .

Source: Bretscher §5.1 Exercise 22

Blok explainer

This is the theorem that makes the definition usable. "Orthogonal to every one of infinitely many vectors" is not something you can check. "Orthogonal to these three" is.

The seminar sheet flags this exercise as one worth remembering. It is used silently in nearly every later computation.

Source: Seminar 1

Proof

$x ⊥ V$ if and only if $x ⊥ v_{i}$ for every basis vector $v_{i}$ of $V$ .

( $\Rightarrow$ ) Suppose $x ⊥ V$ . Each $v_{i}$ is itself a vector in $V$ , so
$⟨ x, v_{i} ⟩ = 0$
Why we’re allowed to: This direction is free: a basis vector is a member of the subspace it spans.
( $\Leftarrow$ ) Suppose $⟨ x, v_{i} ⟩ = 0$ for all $i$ . Take any $v \in V$ and write it in the basis:
$v = c_{1} v_{1} + \dots + c_{m} v_{m}$
Then every term dies:
$⟨ x, v ⟩ = c_{1} ⟨ x, v_{1} ⟩ + \dots + c_{m} ⟨ x, v_{m} ⟩ = c_{1} \cdot 0 + \dots + c_{m} \cdot 0 = 0$
Since $v$ was an arbitrary vector of $V$ , $x$ is orthogonal to all of $V$ .
Why we’re allowed to: ARBITRARY is doing real work: nothing in the calculation used a specific $v$ .

Worked example

For which value of the constant $k$ are $u = (2, 3, 4)$ and $v = (1, k, 1)$ perpendicular?

Solution

Perpendicular means the inner product is zero, so set it to zero and solve:

⟨ u, v ⟩ = 2 \cdot 1 + 3 \cdot k + 4 \cdot 1 = 3 k + 6

Setting this equal to $0$ :

3 k + 6 = 0 ⟹ k = - 2

Answer

$k = - 2$ .

Check: $⟨(2, 3, 4), (1, - 2, 1)⟩ = 2 - 6 + 4 = 0$ . ✓

Source: Bretscher §5.1 Exercise 10

Check yourselfBlok explainer

Is the zero vector orthogonal to every vector in

R^{n}

? Is it a unit vector?Click to reveal the answer

Orthogonal to everything: yes. $⟨ 0, v ⟩ = 0$ for every $v$ , including $v = 0$ itself.

A unit vector: no. $∥ 0 ∥ = 0 \neq = 1$ .

This is why the normalising formula carries the condition $v \neq = 0$ — you cannot divide by a length of zero.

Week 1 · Orthonormal sets

Orthonormal Sets and Linear Independence

A set of vectors that are all unit length and all at right angles to one another. Two conditions, and they buy you linear independence for free — no row reduction, no determinant.

Take a set $B = {u_{1}, u_{2}, \dots, u_{k}} \subseteq R^{n}$ and ask two things of it.

1. Every vector has length 1: $∥ u_{i} ∥ = 1$ for $i = 1, \dots, k$ . They are all NORMALISED. 2. Every pair is orthogonal: $⟨ u_{i}, u_{j} ⟩ = 0$ whenever .

Source: Lecture 1, p. 2

Definition

Orthonormal vectors

The vectors $u_{1}, \dots, u_{k}$ in $R^{n}$ are ORTHONORMAL if

⟨ u_{i}, u_{j} ⟩ = {10 if i = j if i \neq = j

Blok explainer

That one formula packs both conditions in.

Setting $i = j$ gives $⟨ u_{i}, u_{i} ⟩ = ∥ u_{i} ∥^{2} = 1$ , which is condition 1. Setting gives condition 2.

Worked example

Show that the standard basis $e_{1}, e_{2}, \dots, e_{n}$ of $R^{n}$ is orthonormal.

Worked example

Show that for any angle $θ$ , the vectors $[cos θ sin θ]$ and $[- sin θ cos θ]$ are orthonormal.

Theorem

Properties of orthonormal vectors

(a) Orthonormal vectors are linearly independent.

(b) Orthonormal vectors $u_{1}, \dots, u_{n}$ in $R^{n}$ form a basis of $R^{n}$ .

Blok explainer

Part (a) is the payoff. Linear independence is normally something you WORK OUT — row reduce, look for a pivot in every column.

For an orthonormal set you get it for free, from the definition alone.

Proof

Orthonormal vectors $u_{1}, \dots, u_{k}$ are linearly independent.

Suppose a combination vanishes:
$c_{1} u_{1} + c_{2} u_{2} + \dots + c_{k} u_{k} = 0$

Memory tipBlok explainer

The move in step 2 — take the inner product of an equation with one particular $u_{i}$ — is the single most reused trick in this course. It appears again in the projection formula, in Gram-Schmidt and in the spectral theorem.

Want to isolate the $i$ -th coefficient? Dot the whole equation with $u_{i}$ . Orthonormality deletes every other term.

Part (b) follows from part (a) with no extra work.

$n$ orthonormal vectors in $R^{n}$ are linearly independent by (a), and any $n$ linearly independent vectors in $R^{n}$ form a basis. So they are a basis — an ORTHONORMAL BASIS.

Source: Bretscher §5.1.3

Common mistakeBlok explainer

The converse of (a) is false. Linearly independent vectors need not be orthonormal — they need not even be orthogonal.

$(1, 0)$ and $(1, 1)$ are linearly independent, but $⟨(1, 0), (1, 1)⟩ = 1 \neq = 0$ .

Exam walkthrough

The vectors $u_{1} = \frac{1}{2} (1, 1, 1, 1)$ , $u_{2} = \frac{1}{2} (1, 1, - 1, - 1)$ and in are orthonormal. Can you find a vector making all four orthonormal? If so, how many are there?

Check yourselfBlok explainer

Can you have five orthonormal vectors in

R^{4}

?Click to reveal the answer

No. They would be five linearly independent vectors in a 4-dimensional space, and no such set exists.

So an orthonormal set in $R^{n}$ has at most $n$ members — and when it has exactly $n$ , it is a basis.

Week 1 · Orthogonal projection

The Orthogonal Projection onto a Subspace

Split any vector into the part that lies inside a subspace and the part that sticks out at right angles. The split always exists, and there is only one of them — which is what makes the whole of Chapter 5 possible.

The goal. Given a linear subspace $V \subseteq R^{n}$ and a vector $x \in R^{n}$ , find a decomposition

x = x^{∥} + x^{⊥}

where $x^{∥} \in V$ , and $x^{⊥} ⊥ v$ for every $v \in V$ .

Source: Lecture 1, p. 4

The lecture labels the picture: $x$ is DATA, $V$ is the space of MODELS.

That is not decoration. $x^{∥}$ is the best your models can do; $x^{⊥}$ is what they miss. Lecture 4 turns exactly this picture into the method of least squares.

Source: Lecture 1, p. 4

Blok explainer

Start in a case you can see. $V$ is a line $L$ in $R^{2}$ , and $x$ is an arrow that does not lie along it.

Drop a perpendicular from the tip of $x$ onto $L$ . The foot of that perpendicular is $x^{∥}$ ; the dropped segment is $x^{⊥}$ . Everything below is that picture, in dimensions, done with algebra instead of a ruler.

FigureBlok explainer

Coefficient

⟨ x, u ⟩ = 2.121

Lengths

∥ proj_{V} x ∥ = 2.121 < 3.000 = ∥ x ∥

the angle of x

70°

The solid arrow along the line is $x^{∥} = proj_{V} x$ ; the dashed one is $x^{⊥}$ . The small square marks the right angle, and it never opens up however far you drag — that is the whole content of the decomposition theorem. Watch the second readout too: the projection is never longer than $x$ , with equality only when $x$ lies along the line.

The line $V$ is fixed; drag $x$ around it.

The solid arrow along $V$ is $proj_{V} x$ , the dashed one is $x^{⊥}$ , and the small square is the right angle between them. It never opens up, however far you drag — that is the existence half of the theorem, holding for every at once.

Theorem

Orthogonal projection

Let $V$ be a subspace of $R^{n}$ and $x \in R^{n}$ . Then $x$ can be written as

x = x^{∥} + x^{⊥}

Blok explainer

Two separate claims are hiding in there, and a proof has to earn both.

EXISTENCE — such a split can be found. UNIQUENESS — there is no second one.

Existence is a construction; uniqueness is a short argument that needs no construction at all.

Proof

Existence: with an orthonormal basis of $V$ in hand, the split can be written down.

Let ${u_{1}, \dots, u_{k}}$ be an orthonormal basis of $V$ , and suppose the split exists. Since $x^{∥}$ lies in , it is a combination of the basis:

Proof

Uniqueness: there is only one such split.

Suppose the vector splits two ways:
$x = x^{∥} + x^{⊥} = y^{∥} + y^{⊥}$

Memory tipBlok explainer

The uniqueness argument is three lines and reappears constantly: if a vector lies in $V$ and in $V^{⊥}$ at once, it is zero.

$V \cap V^{⊥} = {0}$ , because the only vector perpendicular to itself is the zero vector.

Blok explainer

Notice what the existence proof needed and what it did not.

It needed an ORTHONORMAL basis of $V$ — orthonormality is what collapsed the sum in step 4. Any old basis would have left $k$ simultaneous equations to solve.

It did not need $V$ to be anything special. Any subspace of any $R^{n}$ , of any dimension.

Common mistakeBlok explainer

$x^{⊥}$ is not "the part of $x$ outside $V$ " in a loose sense — it is orthogonal to ALL of $V$ , not merely different from $x^{∥}$ .

Splitting $x = a + b$ with can be done in infinitely many ways: pick any and set .

Check yourselfBlok explainer

What is

proj_{V} x

when

x

already lies in

V

? And when

x ⊥ V

?Click to reveal the answer

If $x \in V$ : take and . That is a valid split, and by uniqueness it is THE split. So .

Week 1 · The projection formula

The Projection Formula and Orthonormal Coordinates

One formula computes the projection, and the same formula hands you coordinates. In a general basis, finding coordinates means solving a linear system. In an orthonormal one, it means taking inner products — and that is the entire reason this course cares about orthonormal bases.

Theorem

Formula for the orthogonal projection

If $V$ is a subspace of $R^{n}$ with ORTHONORMAL basis $u_{1}, \dots, u_{m}$ , then

proj_{V} x = ⟨ u_{1}, x ⟩ u_{1} + \dots + ⟨ u_{m}, x ⟩ u_{m}

for every $x \in R^{n}$ .

Source: Bretscher §5.1.5

Blok explainer

Read it as an instruction, not a formula.

For each basis direction $u_{i}$ : measure how much of $x$ points along it — that is the number $⟨ u_{i}, x ⟩$ — then lay off that much in the direction $u_{i}$ . Add up the results.

Each term $⟨ u_{i}, x ⟩ u_{i}$ is itself the projection of $x$ onto the LINE spanned by $u_{i}$ . So projecting onto a subspace is nothing more than projecting onto each of its basis lines and adding.

In $R^{3}$ : projecting onto the -plane is the same as projecting onto the -axis, projecting onto the -axis, and adding the two results.

Common mistakeBlok explainer

The formula is only valid for an ORTHONORMAL basis. Feed it a merely-orthogonal basis, or an arbitrary one, and it returns the wrong vector.

If your basis is orthogonal but not normalised, either divide each vector by its length first, or use the version that carries the correction factor:

proj_{V} x = i \sum \frac{⟨ v _{i} , x ⟩}{⟨ v _{i} , v _{i} ⟩} v_{i}

Worked example

Let $V = im (A)$ in $R^{4}$ , where $A$ has columns $(1, 1, 1, 1)$ and $(1, - 1, - 1, 1)$ . Find for .

Memory tipBlok explainer

Always verify a projection by checking $x - proj_{V} x$ against the basis vectors. Two inner products that come out zero is near-proof you are right; one that does not is instant proof you are wrong.

Projection done? Subtract it from $x$ and dot the leftover with every basis vector. All zeros or start again.

Blok explainer

Now push the formula to its limit. Take $V$ to be the whole of $R^{n}$ .

Projecting a vector onto the entire space cannot change it, so $proj_{V} x = x$ . Put that into the formula and you get an identity that holds for every $x$ .

Theorem

Expansion in an orthonormal basis

Let $u_{1}, \dots, u_{n}$ be an orthonormal basis of $R^{n}$ . Then

x = ⟨ u_{1}, x ⟩ u_{1} + \dots + ⟨ u_{n}, x ⟩ u_{n}

In coordinate language: the coordinate vector of $x$ with respect to the orthonormal basis $B = {u_{1}, \dots, u_{n}}$ is

[x]_{B} = ⟨ x, u_{1} ⟩ ⟨ x, u_{2} ⟩ ⋮ ⟨ x, u_{n} ⟩

Blok explainer

This is the practical point of the whole lecture.

Given ANY basis $v_{1}, \dots, v_{n}$ , writing $x = c_{1} v_{1} + \dots + c_{n} v_{n}$ means solving an linear system for the . That is real work, and it grows fast with .

The lecture derives the same fact directly. Write $x = c_{1} u_{1} + \dots + c_{n} u_{n}$ and take the inner product with :

Memory tipBlok explainer

Same trick as the independence proof in Topic 2, used for a different purpose. Dot the expansion with $u_{i}$ ; orthonormality deletes everything else; the $i$ -th coefficient falls out.

Orthonormal basis $\Rightarrow$ coordinates are just inner products.

Worked exampleBlok explainer

Write $x = (3, 1)$ in the orthonormal basis $u_{1} = \frac{1}{2} (1, 1)$ , of .

Check yourselfBlok explainer

Why does the formula in Theorem 5.1.5 have

m

terms while the one in Theorem 5.1.6 has

n

?Click to reveal the answer

Because $m = dim V$ and $n = dim R^{n}$ . The projection formula sums over a basis of the SUBSPACE; the expansion formula sums over a basis of the whole space.

Week 1 · Orthogonal complement

The Orthogonal Complement

Every subspace has a partner: the set of all vectors perpendicular to it. Their dimensions add up to n, they meet only at the origin, and each is the other's partner — which lets you compute a hard projection by doing an easy one instead.

Consider the projection $T (x) = proj_{V} x$ and ask what its kernel is.

$T (x) = 0$ means $x^{∥} = 0$ , so $x = x^{⊥}$ . The kernel is exactly the set of vectors perpendicular to $V$ . That set deserves a name.

Source: Bretscher §5.1 Example 5

Definition

Orthogonal complement

Let $V$ be a subspace of $R^{n}$ . The ORTHOGONAL COMPLEMENT $V^{⊥}$ of $V$ is

V^{⊥} = {x \in R^{n} : ⟨ x, v ⟩ = 0 for all v \in V}

Blok explainer

Say it out loud: " $V$ perp is the set of $x$ in $R^{n}$ such that $x$ is orthogonal to $v$ , for every $v$ in $V$ ."

Picture it in $R^{3}$ . If is a plane through the origin, is the line through the origin normal to it. If is a line, is the plane through the origin perpendicular to it.

Worked example

Find $L^{⊥}$ for the line $L = im 123$ in .

Theorem

Properties of the orthogonal complement

Let $V$ be a subspace of $R^{n}$ . Then:

(a) $V^{⊥}$ is a subspace of $R^{n}$ . (b) $V \cap V^{⊥} = {0}$ . (c) . (d) .

Proof

Parts (a), (b) and (c).

(a) $V^{⊥}$ is the kernel of the linear map $T (x) = proj_{V} x$ , and the kernel of a linear transformation is always a subspace.
Why we’re allowed to: Theorem 5.1.4 states that the projection is linear; the kernel of a linear map is a subspace by the standard result from Mathematics 2. No axiom-checking is needed.
(b) Suppose $x$ is in and also in . Being in means is orthogonal to every vector of — and is itself one of those vectors.

Proof

Part (d): $(V^{⊥})^{⊥} = V$ .

First show $V \subseteq (V^{⊥})^{⊥}$ . Take . Every satisfies , so is orthogonal to everything in — which is what belonging to means.

Memory tipBlok explainer

Part (c) is the workhorse. It turns "find $V^{⊥}$ " into a problem where you already know the answer's SIZE before you start — and a dimension that comes out wrong tells you instantly that you have made a mistake.

$dim V + dim V^{⊥} = n$ . Plane in $R^{3}$ ? Complement is a line. Line in ? Complement is 3-dimensional.

Exam walkthrough

Let $A = [11 - 1 - 2 23 - 2 - 4]$ . Find and verify the correctness of your solution.

Blok explainer

The complement also gives a shortcut for projections.

Since $x = x^{∥} + x^{⊥}$ , you can compute EITHER piece and subtract:

proj_{V} x = x - proj_{V^{⊥}} x

Exam walkthrough

Let $A = [4 - 4 2]$ and $V = ker (A)$ . Find the orthogonal projection of onto , and verify your answer.

Check yourselfBlok explainer

What are

{0}^{⊥}

and

(R^{n})^{⊥}

?Click to reveal the answer

${0}^{⊥} = R^{n}$ : every vector is orthogonal to the zero vector, so nothing is excluded.

Week 1 · Pythagoras

Pythagoras and the Size of a Projection

Pythagoras' theorem, stated in n dimensions and proved in two lines. It immediately gives the fact that a shadow is never longer than the object casting it — and that innocent-looking inequality is what the Cauchy-Schwarz inequality is built on.

Blok explainer

In $R^{2}$ , Pythagoras is about right triangles. In $R^{n}$ we have no triangles to draw — but we do have the definitions of length and orthogonality, and that turns out to be enough.

Theorem

Pythagorean theorem

Let $x, y \in R^{n}$ . Then

∥ x + y ∥^{2} = ∥ x ∥^{2} + ∥ y ∥^{2}

holds IF AND ONLY IF $x$ and $y$ are orthogonal.

Source: Bretscher §5.1.9

Proof

$∥ x + y ∥^{2} = ∥ x ∥^{2} + ∥ y ∥^{2}$ exactly when $⟨ x, y ⟩ = 0$ .

Blok explainer

The expansion in step 2 is worth memorising on its own:

∥ x + y ∥^{2} = ∥ x ∥^{2} + 2 ⟨ x, y ⟩ + ∥ y ∥^{2}

Worked example

Consider a line $L$ in $R^{3}$ and a vector $x \in R^{3}$ . What is the relationship between $∥ x ∥$ and $∥ proj_{L} x ∥$ ?

Theorem

An inequality for the size of a projection

Let $V$ be a subspace of $R^{n}$ and $x \in R^{n}$ . Then

∥ proj_{V} x ∥ \leq ∥ x ∥

The proof is the example above with $L$ replaced by $V$ — nothing about the argument used the dimension.

∥ x ∥^{2} = ∥ proj_{V} x ∥^{2} + ∥ x^{⊥} ∥^{2} \geq ∥ proj_{V} x ∥^{2}

Memory tipBlok explainer

A shadow is never longer than the object casting it, and it is exactly as long only when the object is lying flat on the ground.

Projecting can only shorten. $∥ proj_{V} x ∥ \leq ∥ x ∥$ , equality iff $x$ was already in $V$ .

Exam walkthrough

Give an algebraic proof of the parallelogram identity

∥ v + w ∥^{2} + ∥ v - w ∥^{2} = 2 ∥ v ∥^{2} + 2 ∥ w ∥^{2}

Check yourselfBlok explainer

R^{2}

with

x = (3, 4)

and

y = (- 4, 3)

, verify Pythagoras numerically.Click to reveal the answer

$⟨ x, y ⟩ = - 12 + 12 = 0$ , so they are orthogonal and the theorem should apply.

Week 1 · Cauchy-Schwarz

Cauchy-Schwarz and the Angle Between Vectors

One inequality bounds the inner product by the product of the lengths. It is what lets us define an angle in dimensions we cannot picture, it gives the triangle inequality for free, and it turns up in exam questions that look nothing like linear algebra.

Theorem

Cauchy-Schwarz inequality

For all $x, y \in R^{n}$ ,

∣ ⟨ x, y ⟩ ∣ \leq ∥ x ∥ ∥ y ∥

with equality if and only if $x$ and $y$ are parallel.

Source: Bretscher §5.1.11

Blok explainer

Absolute value bars on the left, because the inner product can be negative while the right-hand side never is.

The content: the inner product cannot be large unless BOTH vectors are long. Two short vectors can never have a big inner product, however well aligned they are.

Blok explainer

The proof is a surprise. It uses nothing but the projection inequality from Topic 6 — no clever algebra, no discriminant trick.

The idea is to project $x$ onto the line spanned by $y$ , and then read off what "the shadow is shorter than the object" says.

Proof

$∣ ⟨ x, y ⟩ ∣ \leq ∥ x ∥ ∥ y ∥$ , assuming $y \neq = 0$ .

Set $u = y / ∥ y ∥$ , a unit vector, and let $V = span {u}$ — the line through .

Two remarks the lecture flags explicitly.

The proof assumed $∥ y ∥ \neq = 0$ . If $y = 0$ , both sides of the inequality are $0$ , so it holds trivially — the theorem is still true, the proof just needs the separate line.

Equality holds when $x$ is parallel to $y$ , because then $x^{⊥} = 0$ and step 2's inequality becomes an equality.

Common mistakeBlok explainer

Writing "take $u = y / ∥ y ∥$ " without saying $y \neq = 0$ is the standard way to lose a mark on a proof question.

Open with: "If $y = 0$ both sides are zero and the inequality holds. So assume $y \neq = 0$ ."

Theorem

Triangle inequality

For all $v, w \in R^{n}$ ,

∥ v + w ∥ \leq ∥ v ∥ + ∥ w ∥

Blok explainer

The seminar sheet singles this exercise out as one to remember. The picture is the reason for the name: going straight from $A$ to $C$ is never longer than detouring via $B$ .

Source: Seminar 1

Proof

$∥ v + w ∥ \leq ∥ v ∥ + ∥ w ∥$ .

Square the left side and expand:
$∥ v + w ∥^{2} = ∥ v ∥^{2} + 2 ⟨ v, w ⟩ + ∥ w ∥^{2}$

Blok explainer

Now the payoff Cauchy-Schwarz was really for.

In $R^{3}$ you may already know that $⟨ x, y ⟩ = ∥ x ∥ ∥ y ∥ cos θ$ , where $θ$ is the angle between the vectors. Rearranged:

cos θ = \frac{⟨ x , y ⟩}{∥ x ∥ ∥ y ∥}

Definition

Angle between two vectors

For nonzero $x, y \in R^{n}$ , the angle $θ$ between them is

θ = arccos (\frac{⟨ x , y ⟩}{∥ x ∥ ∥ y ∥})

This definition needs Cauchy-Schwarz to be legal at all.

$arccos$ only accepts inputs between $- 1$ and $1$ . Dividing Cauchy-Schwarz through by $∥ x ∥ ∥ y ∥$ gives exactly

\frac{⟨ x , y ⟩}{∥ x ∥ ∥ y ∥} \leq 1

Memory tipBlok explainer

The sign of $⟨ x, y ⟩$ already tells you the shape of the angle without computing an arccos at all.

$⟨ x, y ⟩ > 0$ : acute. $= 0$ : right angle. $< 0$ : obtuse. Cosine is positive below $90°$ and negative above.

Worked example

Find the angle between $x = (1, 0, 0, 0)$ and $y = (1, 1, 1, 1)$ in $R^{4}$ .

Solution

Three quantities, then one arccos.

⟨ x, y ⟩ = 1 + 0 + 0 + 0 = 1

Exam walkthrough

Let $x_{1}, x_{2}, \dots, x_{n} \in R$ with $x_{1} + \dots + x_{n} = 1$ . Use the Cauchy-Schwarz inequality to prove that

Check yourselfBlok explainer

The triangle inequality is an equality when? Use the proof to answer, not a picture.Click to reveal the answer

Two inequalities were used, so both must be tight.

Step 3 (Cauchy-Schwarz) is tight when $v$ and $w$ are parallel. Step 2 ( $2 ⟨ v, w ⟩ \leq 2∣ ⟨ v, w ⟩ ∣$ ) is tight when $⟨ v, w ⟩ \geq 0$ .

Leave it as a surd. Decimals lose exactness and gain nothing in an exam.

Each entry of

k v

k \cdot v_{i}

, so every term of the sum picks up a factor

k

twice.

Why we’re allowed to: k2​=∣k∣, not k — the absolute value is the whole point. A length can never come out negative.

Why we’re allowed to: Every vector of V is a linear combination of a basis of V — that is what a basis means.

Why we’re allowed to: The dot product is linear in each argument, so it distributes over the sum and lets the scalars out front.

Source: Bretscher §5.1.2

The word itself is the two conditions glued together: ORTHO-gonal plus NORMAL-ised.

$e_{i}$ is the vector with a $1$ in slot $i$ and zeros everywhere else.

For $⟨ e_{i}, e_{i} ⟩$ : every term of the sum is $0 \cdot 0$ except slot $i$ , which is $1 \cdot 1$ . So $⟨ e_{i}, e_{i} ⟩ = 1$ .

For $⟨ e_{i}, e_{j} ⟩$ with $i \neq = j$ : in every slot, at least one of the two factors is $0$ . So every term is $0$ , and

Answer

Orthonormal. The standard basis is the model every other orthonormal basis is built to imitate: a set of perpendicular directions, each one unit long.

Source: Bretscher §5.1 Example 1

Lengths first:

cos^{2} θ + sin^{2} θ = 1 and (- sin θ)^{2} + cos^{2} θ = 1

So both are unit vectors.

Now the inner product:

cos θ \cdot (- sin θ) + sin θ \cdot cos θ = - cos θ sin θ + sin θ cos θ = 0

Answer

Orthonormal for every $θ$ .

Geometrically this is the standard basis rotated by $θ$ — turning a pair of perpendicular unit arrows does not change either their length or the right angle between them. That observation becomes a theorem in Lecture 3.

Source: Bretscher §5.1 Example 2

Source: Bretscher §5.1.3

with each $c_{i} \in R$ and $0$ the zero vector of $R^{n}$ .

Why we’re allowed to: This is the definition of linear independence: assume a combination equals zero, then show every coefficient must be zero.

Fix one index $i$ and take the inner product of BOTH sides with $u_{i}$ :

⟨ c_{1} u_{1} + \dots + c_{k} u_{k}, u_{i} ⟩ = ⟨ 0, u_{i} ⟩

Why we’re allowed to: Equal vectors have equal inner products with anything. Choosing ui​ as the partner is the whole trick.

Expand the left side:

c_{1} ⟨ u_{1}, u_{i} ⟩ + \dots + c_{i} ⟨ u_{i}, u_{i} ⟩ + \dots + c_{k} ⟨ u_{k}, u_{i} ⟩

Why we’re allowed to: The inner product is linear in its first argument, so it distributes over the sum and the scalars come out front.

Every $⟨ u_{j}, u_{i} ⟩$ with $j \neq = i$ is $0$ , and $⟨ u_{i}, u_{i} ⟩$ is $1$ . The whole sum collapses, and the right side $⟨ 0, u_{i} ⟩$ is $0$ :

c_{i} = 0

Why we’re allowed to: This is exactly the definition of orthonormal — the one line of the proof that uses the hypothesis at all.

The index $i$ was arbitrary, so every coefficient is zero, and the vectors are linearly independent.

Why we’re allowed to: The argument never used a specific i, so it holds for i=1,…,k at once.

Source: Lecture 1, p. 2

Orthonormal is a much stronger condition than independent. Turning an independent set into an orthonormal one is exactly the job of the Gram-Schmidt process in Lecture 2.

u_{3} = \frac{1}{2} (1, - 1, 1, - 1)

How to think about it — before any algebra

Three orthonormal vectors in $R^{4}$ . By part (a) they span a 3-dimensional subspace $V$ , so there is exactly one dimension left over.

So I am looking for a unit vector in the orthogonal complement of $V$ , and that complement is a LINE (dimension $4 - 3 = 1$ ). A line through the origin contains exactly two unit vectors, pointing opposite ways.

That already answers "how many" before I compute anything: two.

To find them, I need $x$ with $⟨ u_{i}, x ⟩ = 0$ for $i = 1, 2, 3$ . The $\frac{1}{2}$ factors do not affect whether an inner product is zero, so I can drop them and solve with the raw sign patterns.

Write $x = (x_{1}, x_{2}, x_{3}, x_{4})$ and set each inner product to zero:

x_{1} + x_{2} + x_{3} + x_{4} = 0

x_{1} + x_{2} - x_{3} - x_{4} = 0

x_{1} - x_{2} + x_{3} - x_{4} = 0

Add the first two: $2 x_{1} + 2 x_{2} = 0$ , so $x_{2} = - x_{1}$ . Subtract the second from the first: , so .

Put those into the third equation:

x_{1} - (- x_{1}) + x_{3} - (- x_{3}) = 2 x_{1} + 2 x_{3} = 0 ⟹ x_{3} = - x_{1}

So $x = x_{1} (1, - 1, - 1, 1)$ : the solution space is a line, as predicted.

Normalise. $∥(1, - 1, - 1, 1)∥ = 4 = 2$ , so

u_{4} = \pm \frac{1}{2} (1, - 1, - 1, 1)

u_{4} = \pm \frac{1}{2} (1, - 1, - 1, 1) two such vectors

Marks: setting up the three orthogonality equations, solving them to a one-dimensional solution space, normalising, and stating BOTH signs.

The $\pm$ is the half people lose. "How many such vectors are there" is asking for it explicitly.

A dimension count ( $4 - 3 = 1$ , and a line holds two unit vectors) earns the counting mark even if the arithmetic goes wrong.

Source: Bretscher §5.1 Exercise 16

Watch the length readout too: $∥ proj_{V} x ∥ \leq ∥ x ∥$ , with equality exactly when $x$ lands on the line. That is Topic 6's theorem, three topics early.

with $x^{∥} \in V$ and $x^{⊥}$ perpendicular to $V$ , and this representation is UNIQUE.

The vector $x^{∥}$ is called the ORTHOGONAL PROJECTION of $x$ onto $V$ , written $proj_{V} x$ . The map $T (x) = proj_{V} x$ from $R^{n}$ to $R^{n}$ is linear.

Source: Bretscher §5.1.4

x^{∥} = c_{1} u_{1} + \dots + c_{k} u_{k}

for coefficients $c_{i}$ we have yet to find.

Why we’re allowed to: Every vector of V is a linear combination of a basis of V. Gram-Schmidt (Lecture 2) guarantees an orthonormal basis exists for any subspace, so assuming one is safe.

Then $x^{⊥} = x - x^{∥}$ must be orthogonal to $V$ , and by the basis test that means

⟨ x^{⊥}, u_{i} ⟩ = 0 for each i

Why we’re allowed to: Orthogonal to V is equivalent to orthogonal to each vector of one basis of V.

Expand that condition:

0 = ⟨ x - c_{1} u_{1} - \dots - c_{k} u_{k}, u_{i} ⟩ = ⟨ x, u_{i} ⟩ - c_{1} ⟨ u_{1}, u_{i} ⟩ - \dots - c_{k} ⟨ u_{k}, u_{i} ⟩

Why we’re allowed to: Linearity of the inner product in its first argument.

Orthonormality kills every term except the $i$ -th, where $⟨ u_{i}, u_{i} ⟩ = 1$ :

0 = ⟨ x, u_{i} ⟩ - c_{i} ⟹ c_{i} = ⟨ x, u_{i} ⟩

Why we’re allowed to: The definition of orthonormal. The coefficients were not chosen — they were FORCED.

So define

x^{∥} = ⟨ x, u_{1} ⟩ u_{1} + \dots + ⟨ x, u_{k} ⟩ u_{k}, x^{⊥} = x - x^{∥}

The first lies in $V$ , and by construction $⟨ x^{⊥}, u_{i} ⟩ = 0$ for every $i$ , so the second is orthogonal to $V$ .

Why we’re allowed to: The calculation above run backwards. Having derived what the coefficients must be, we check that this choice does work — which is what turns a derivation into an existence proof.

Source: Lecture 1, p. 4

with $x^{∥}, y^{∥} \in V$ and $x^{⊥}, y^{⊥} \in V^{⊥}$ .

Why we’re allowed to: The standard way to prove uniqueness: assume two of them, then prove they are the same one.

Rearrange to put the $V$ -parts on one side:

x^{∥} - y^{∥} = y^{⊥} - x^{⊥}

Why we’re allowed to: Ordinary vector algebra on the equation x∥+x⊥=y∥+y⊥.

The left side lies in $V$ and the right side lies in $V^{⊥}$ . They are the same vector, call it $w$ — so $w$ is in BOTH.

Why we’re allowed to: V and V⊥ are each closed under subtraction: both are subspaces.

Being in $V^{⊥}$ means $w$ is orthogonal to everything in $V$ , and $w$ itself is in $V$ . So

⟨ w, w ⟩ = ∥ w ∥^{2} = 0 ⟹ w = 0

Why we’re allowed to: A vector orthogonal to itself has zero length, and the zero vector is the only vector of zero length.

Hence $x^{∥} - y^{∥} = 0$ and $y^{⊥} - x^{⊥} = 0$ : the two splits were the same split.

Why we’re allowed to: w=0 applied to both sides of step 2.

Source: Lecture 1, p. 4

Exactly one of those splits has $b ⊥ V$ . That is the content of the uniqueness half, and it is why the theorem is worth stating at all.

If $x ⊥ V$ : take $x^{∥} = 0$ and $x^{⊥} = x$ . So $proj_{V} x = 0$ .

These two cases say the projection map fixes $V$ and annihilates $V^{⊥}$ — a fact used again in Lecture 7 to read off its eigenvalues, which turn out to be just $1$ and $0$ .

Source: Bretscher §5.1.5

If the basis is not even orthogonal, the formula does not apply at all — run Gram-Schmidt first (Lecture 2), or use the matrix formula $A (A^{T} A)^{- 1} A^{T}$ from Lecture 4.

First check the two columns are orthogonal:

1 - 1 - 1 + 1 = 0 ✓

They are independent, so they are a basis of $V$ . Each has length $4 = 2$ , so normalise:

u_{1} = \frac{1}{2} (1, 1, 1, 1), u_{2} = \frac{1}{2} (1, - 1, - 1, 1)

Now the two coefficients:

⟨ u_{1}, x ⟩ = \frac{1}{2} (1 + 3 + 1 + 7) = 6

Assemble:

proj_{V} x = 6 u_{1} + 2 u_{2} = 3 (1, 1, 1, 1) + (1, - 1, - 1, 1)

Answer

$proj_{V} x = (4, 2, 2, 4)$ .

Check it. $x - proj_{V} x = (- 3, 1, - 1, 3)$ , and

⟨(- 3, 1, - 1, 3), (1, 1, 1, 1)⟩ = 0, ⟨(- 3, 1, - 1, 3), (1, - 1, - 1, 1)⟩ = 0

The remainder is perpendicular to both basis vectors, so it is perpendicular to $V$ . That check costs ten seconds and catches almost every arithmetic slip.

Source: Bretscher §5.1 Example 4

for every $x \in R^{n}$ .

Source: Bretscher §5.1.6

Source: Lecture 1, p. 3

Given an ORTHONORMAL basis, $c_{i} = ⟨ x, u_{i} ⟩$ . One inner product each. No system, no elimination.

Source: Bretscher §5.1.6

⟨ x, u_{i} ⟩ = c_{1} ⟨ u_{1}, u_{i} ⟩ + \dots + c_{i} ⟨ u_{i}, u_{i} ⟩ + \dots + c_{n} ⟨ u_{n}, u_{i} ⟩

Every term dies except the $i$ -th, where the inner product is $1$ :

⟨ x, u_{i} ⟩ = c_{i}

Source: Lecture 1, p. 3

u_{2} = \frac{1}{2} (- 1, 1)

Two inner products, and nothing else:

⟨ x, u_{1} ⟩ = \frac{1}{2} (3 + 1) = \frac{4}{2} = 22

⟨ x, u_{2} ⟩ = \frac{1}{2} (- 3 + 1) = \frac{- 2}{2} = - 2

[x]_{B} = [22 - 2]

Check by reassembling:

22 \cdot \frac{1}{2} (1, 1) - 2 \cdot \frac{1}{2} (- 1, 1) = 2 (1, 1) - (- 1, 1) = (3, 1)

Answer

$[x]_{B} = (22, - 2)$ .

Compare the effort with solving $[11 - 1 1] c = [31]$ — doable in $R^{2}$ , unpleasant in .

Source: Bretscher §5.1.6

Theorem 5.1.6 is the special case

V = R^{n}

, where

m = n

and the projection does nothing.

Equivalently, $V^{⊥}$ is the kernel of the orthogonal projection onto $V$ .

Source: Bretscher §5.1.7

$L$ is spanned by the single vector $(1, 2, 3)$ , so by the basis test a vector $x$ lies in $L^{⊥}$ exactly when it is orthogonal to that one vector:

⟨(1, 2, 3), x ⟩ = 0

Written out:

x_{1} + 2 x_{2} + 3 x_{3} = 0

That is one linear equation in three unknowns — a plane through the origin. In matrix language,

L^{⊥} = ker [123]

Answer

$L^{⊥}$ is the plane $x_{1} + 2 x_{2} + 3 x_{3} = 0$ .

Note the pattern: the SPAN of a vector and the KERNEL of that same vector written as a row. It is the shape of Theorem 5.4.1 in Lecture 4, $(im A)^{⊥} = ker (A^{T})$ , in its smallest case.

Source: Lecture 4, p. 1

dim (V) + dim (V^{⊥}) = n

(V^{⊥})^{⊥} = V

Source: Bretscher §5.1.8

Why we’re allowed to: The definition of V⊥, applied with the particular choice v=x.

So the vector is orthogonal to itself:

⟨ x, x ⟩ = ∥ x ∥^{2} = 0 ⟹ x = 0

Why we’re allowed to: The norm is zero only for the zero vector — the definiteness of the inner product.

n = dim (im T) + dim (ker T)

Why we’re allowed to: Rank-nullity holds for any linear map Rn→Rn, and T is linear.

The image of $T$ is $V$ and the kernel of $T$ is $V^{⊥}$ :

n = dim (V) + dim (V^{⊥})

Why we’re allowed to: The projection lands exactly on V and fixes it, so imT=V; kerT=V⊥ is the definition above.

Source: Bretscher §5.1.8

Why we’re allowed to: Straight from the definition of V⊥, read in the other direction. Orthogonality is symmetric: ⟨x,v⟩=⟨v,x⟩.

Now count dimensions. By (c) applied to $V^{⊥}$ :

dim ((V^{⊥})^{⊥}) = n - dim (V^{⊥})

Why we’re allowed to: Part (c) holds for ANY subspace, and V⊥ is one by part (a).

By (c) applied to $V$ , $dim (V^{⊥}) = n - dim (V)$ . Substituting:

dim ((V^{⊥})^{⊥}) = n - (n - dim V) = dim (V)

Why we’re allowed to: Arithmetic on the two dimension counts.

A subspace contained in another of the SAME dimension must equal it. So $(V^{⊥})^{⊥} = V$ .

Why we’re allowed to: Standard: if W⊆U and dimW=dimU, a basis of W is an independent set of dimU vectors in U, hence a basis of U.

Source: Bretscher §5.1 Exercise 23

How to think about it — before any algebra

Two steps, in the obvious order: find $ker A$ , then find the complement of what I find.

Before any arithmetic, predict the sizes. $A$ has rank $2$ (the rows are clearly not multiples of each other), so by rank-nullity $dim ker A = 4 - 2 = 2$ . Then part (c) gives $dim (ker A)^{⊥} = 4 - 2 = 2$ as well.

So I am looking for two independent vectors at the end. If I get one or three, something has gone wrong.

For the second step I use the basis test: a vector is in $(ker A)^{⊥}$ exactly when it is orthogonal to each of the two basis vectors of $ker A$ — two equations, not infinitely many.

The question says "verify", so the verification is worth marks. I will check dimensions add to $4$ , and check every basis vector of one space against every basis vector of the other.

STEP 1 — find $ker A$ . Row-reduce: subtracting row 1 from row 2 gives

[10 - 1 - 1 21 - 2 - 2]

From the second row, $x_{2} = x_{3} - 2 x_{4}$ . Substituting into the first row:

x_{1} - (x_{3} - 2 x_{4}) + 2 x_{3} - 2 x_{4} = 0 ⟹ x_{1} + x_{3} = 0

So $x_{1} = - x_{3}$ , with $x_{3}$ and $x_{4}$ free. Taking and :

STEP 2 — find the complement. A vector $y$ is in $(ker A)^{⊥}$ iff it is orthogonal to both:

- y_{1} + y_{2} + y_{3} = 0

The second gives $y_{4} = 2 y_{2}$ ; the first gives $y_{1} = y_{2} + y_{3}$ , with free. Taking and :

STEP 3 — verify. Dimensions: $2 + 2 = 4$ . ✓

All four pairings:

⟨(1, 1, 0, 2), (- 1, 1, 1, 0)⟩ = - 1 + 1 + 0 + 0 = 0 ✓

⟨(1, 1, 0, 2), (0, - 2, 0, 1)⟩ = 0 - 2 + 0 + 2 = 0 ✓

(ker A)^{⊥} = span {(1, 1, 0, 2), (1, 0, 1, 0)}

Marks: correctly computing $ker A$ , setting up the orthogonality conditions against a BASIS of it rather than against all of it, solving, and verifying.

Any basis of the same plane scores full marks — the span is what is being asked for, not one particular pair of vectors.

Worth noticing for later: this answer is the span of the two ROWS of $A$ . Check it — $(1, - 1, 2, - 2) = - (1, 1, 0, 2) + 2 (1, 0, 1, 0)$ . That is Theorem 5.4.1 of Lecture 4, $(ker A)^{⊥} = im (A^{T})$ , and once you have met it this whole question collapses to "write down the row space".

Source: Seminar 1, Q1

When $V$ is big and $V^{⊥}$ is small — a plane in $R^{3}$ , say, whose complement is a single line — projecting onto the complement is far less work.

How to think about it — before any algebra

$V = ker A$ is the plane $4 x_{1} - 4 x_{2} + 2 x_{3} = 0$ in $R^{3}$ — a 2-dimensional space. Finding an orthonormal basis for it would mean two vectors and a Gram-Schmidt step.

But $V^{⊥}$ is only 1-dimensional, and I can write it down instantly: the plane's normal direction is the coefficient vector $(4, - 4, 2)$ .

So project onto the LINE instead and subtract. One inner product rather than a Gram-Schmidt.

I will simplify $(4, - 4, 2)$ to $(2, - 2, 1)$ first — a line does not care about scaling, and the smaller numbers make the arithmetic clean.

$V^{⊥}$ is spanned by $n = (2, - 2, 1)$ , with

∥ n ∥^{2} = 4 + 4 + 1 = 9

Project $x$ onto that line. Using the un-normalised form of the formula:

proj_{V^{⊥}} x = \frac{⟨ x , n ⟩}{⟨ n , n ⟩} n

Compute the numerator:

⟨ x, n ⟩ = 2 \cdot 2 + 2 \cdot (- 2) + 9 \cdot 1 = 4 - 4 + 9 = 9

proj_{V^{⊥}} x = \frac{9}{9} (2, - 2, 1) = (2, - 2, 1)

Subtract:

proj_{V} x = x - proj_{V^{⊥}} x = (2, 2, 9) - (2, - 2, 1) = (0, 4, 8)

proj_{V} x = (0, 4, 8)

Verification, both halves of which earn marks:

• $(0, 4, 8)$ must lie in $V$ : $4 (0) - 4 (4) + 2 (8) = - 16 + 16 = 0$ . ✓
• $x - proj_{V} x = (2, - 2, 1)$ must be perpendicular to $V$ : it is a scalar multiple of the normal $n$ , so it is. ✓

Marks: identifying $V^{⊥}$ as the span of the coefficient vector, projecting onto it correctly, subtracting, and verifying both conditions.

The method mark is for choosing to project onto the complement. Doing it the long way — Gram-Schmidt on a basis of the plane — is not wrong and scores the same, but it takes three times as long under time pressure.

Source: Seminar 1, Q4

$(R^{n})^{⊥} = {0}$ : a vector orthogonal to everything is orthogonal to itself, so it is zero.

Both obey part (c): $0 + n = n$ and $n + 0 = n$ . And they are each other's complements, as part (d) requires.

Start from the squared norm:
$∥ x + y ∥^{2} = ⟨ x + y, x + y ⟩$
Why we’re allowed to: The definition of the norm, squared — the standard first move for any statement about lengths.
Expand:
$⟨ x, x ⟩ + ⟨ x, y ⟩ + ⟨ y, x ⟩ + ⟨ y, y ⟩$
Why we’re allowed to: The inner product is linear in each argument, so it distributes exactly like ordinary multiplication over a sum.
The two middle terms are equal, so this is
$∥ x ∥^{2} + 2 ⟨ x, y ⟩ + ∥ y ∥^{2}$
Why we’re allowed to: For real vectors $⟨ x, y ⟩ = ⟨ y, x ⟩$ — the sum does not care about the order.
That equals $∥ x ∥^{2} + ∥ y ∥^{2}$ exactly when the middle term vanishes:
$2 ⟨ x, y ⟩ = 0 ⟺ ⟨ x, y ⟩ = 0$

Source: Lecture 1, p. 6

It is $(a + b)^{2} = a^{2} + 2 ab + b^{2}$ for vectors. It will be the first line of half a dozen later proofs, including the triangle inequality and the parallelogram identity.

Split $x$ into its two pieces: $x = proj_{L} x + x^{⊥}$ , and those two pieces are orthogonal.

So Pythagoras applies:

∥ x ∥^{2} = ∥ proj_{L} x ∥^{2} + ∥ x^{⊥} ∥^{2}

Since $∥ x^{⊥} ∥^{2} \geq 0$ , dropping it can only decrease the right side:

∥ x ∥^{2} \geq ∥ proj_{L} x ∥^{2}

Answer

$∥ proj_{L} x ∥ \leq ∥ x ∥$ , with equality exactly when $x$ lies on $L$ .

Equality forces $∥ x^{⊥} ∥ = 0$ , so $x^{⊥} = 0$ and $x = proj_{L} x \in L$ .

Source: Bretscher §5.1 Example 6

with equality if and only if $x \in V$ .

Source: Bretscher §5.1.10

Source: Lecture 1, p. 6

and interpret the result geometrically using a sketch.

How to think about it — before any algebra

"Algebraic proof" means: do not draw anything, expand.

There are two squared norms on the left. I know how to expand each — square the norm into an inner product and distribute. The only difference between them is the sign on $w$ .

I expect the cross terms $\pm 2 ⟨ v, w ⟩$ to cancel, because that is the only way an identity like this could hold for ALL $v$ and $w$ : the right side has no $⟨ v, w ⟩$ in it at all.

For the geometric half, I need to know what the four objects are. Draw the parallelogram with sides $v$ and $w$ from a common corner. Then $v + w$ is one diagonal and $v - w$ is the other. So the identity is a statement about diagonals versus sides.

Expand the first term:

∥ v + w ∥^{2} = ⟨ v + w, v + w ⟩ = ∥ v ∥^{2} + 2 ⟨ v, w ⟩ + ∥ w ∥^{2}

Expand the second, which differs only in the sign of $w$ :

∥ v - w ∥^{2} = ⟨ v - w, v - w ⟩ = ∥ v ∥^{2} - 2 ⟨ v, w ⟩ + ∥ w ∥^{2}

Add them. The cross terms $+ 2 ⟨ v, w ⟩$ and $- 2 ⟨ v, w ⟩$ cancel:

∥ v + w ∥^{2} + ∥ v - w ∥^{2} = 2 ∥ v ∥^{2} + 2 ∥ w ∥^{2}

GEOMETRIC READING. Draw the parallelogram whose sides from one corner are $v$ and $w$ . Its two diagonals are the vectors $v + w$ (the long way across) and $v - w$ (from the tip of $w$ to the tip of $v$ ).

The parallelogram has four sides: two of length $∥ v ∥$ and two of length $∥ w ∥$ . So the right-hand side is the sum of the squares of all four sides, and the left-hand side is the sum of the squares of the two diagonals.

∥ v + w ∥^{2} + ∥ v - w ∥^{2} = 2 ∥ v ∥^{2} + 2 ∥ w ∥^{2}

In a parallelogram, the sum of the squares of the two diagonals equals the sum of the squares of the four sides.

Marks: expanding both norms via the inner product, observing that the cross terms cancel, and identifying the diagonals as $v + w$ and $v - w$ in a labelled sketch.

The sketch is not decoration here — the question asks for it, and the geometric statement is unmarkable without labelling which vector is which diagonal.

Sanity check with a rectangle: $v = (a, 0)$ , $w = (0, b)$ . Both diagonals have length $a^{2} + b^{2}$ , so the left side is , and the right side is . ✓

Source: Seminar 1, Q2

$∥ x ∥^{2} = 9 + 16 = 25$ and $∥ y ∥^{2} = 16 + 9 = 25$ , summing to $50$ .

$x + y = (- 1, 7)$ , so $∥ x + y ∥^{2} = 1 + 49 = 50$ . ✓

Now break orthogonality: take $y = (4, 3)$ instead. Then $⟨ x, y ⟩ = 24$ , $x + y = (7, 7)$ and $∥ x + y ∥^{2} = 98$ , while $∥ x ∥^{2} + ∥ y ∥^{2} = 50$ . The gap is exactly $2 ⟨ x, y ⟩ = 48$ — the cross term the proof identified.

Why we’re allowed to: Normalising needs y=0, which is exactly the assumption. The line through y is the same as the line through u.

Apply the projection inequality:

∥ x ∥ \geq ∥ proj_{V} x ∥ = ∥⟨ x, u ⟩ u ∥

Why we’re allowed to: Theorem 5.1.10 for the inequality, and the projection formula onto a line spanned by a UNIT vector for the equality.

Pull the scalar out of the norm:

∥⟨ x, u ⟩ u ∥ = ∣ ⟨ x, u ⟩ ∣ ∥ u ∥ = ∣ ⟨ x, u ⟩ ∣

The last step is $∥ u ∥ = 1$ .

Why we’re allowed to: ∥kv∥=∣k∣∥v∥ from Topic 1. The absolute value appears here, and it is where the one in the theorem comes from.

Substitute $u$ back:

∣ ⟨ x, u ⟩ ∣ = ⟨ x, \frac{y}{∥ y ∥} ⟩ = \frac{1}{∥ y ∥} ∣ ⟨ x, y ⟩ ∣

Why we’re allowed to: 1/∥y∥ is a positive scalar, so it comes out of the inner product and straight out of the absolute value unchanged.

So $∥ x ∥ \geq \frac{1}{∥ y ∥} ∣ ⟨ x, y ⟩ ∣$ . Multiply both sides by $∥ y ∥ > 0$ :

∣ ⟨ x, y ⟩ ∣ \leq ∥ x ∥ ∥ y ∥

Why we’re allowed to: Multiplying an inequality by a positive number preserves its direction.

Source: Lecture 1, p. 7

One sentence, and the proof is complete rather than nearly complete. The lecturer wrote this remark in the margin in a different colour, which is a fair signal about how often it is forgotten.

Source: Lecture 1, p. 7

Source: Bretscher §5.1 Exercise 12

Why we’re allowed to: The expansion from Topic 6. Squaring is legitimate because both sides of the target inequality are ≥0.

Bound the middle term above by its absolute value:

2 ⟨ v, w ⟩ \leq 2∣ ⟨ v, w ⟩ ∣

Why we’re allowed to: Any real number is at most its own absolute value. This is where a possibly-negative cross term is made harmless.

Apply Cauchy-Schwarz:

2∣ ⟨ v, w ⟩ ∣ \leq 2 ∥ v ∥ ∥ w ∥

Why we’re allowed to: The theorem just proved — this is the only step that uses it, and the whole proof is built around getting here.

Chain the two bounds:

∥ v + w ∥^{2} \leq ∥ v ∥^{2} + 2 ∥ v ∥ ∥ w ∥ + ∥ w ∥^{2} = (∥ v ∥ + ∥ w ∥)^{2}

Why we’re allowed to: The right-hand side is a perfect square, recognised in the other direction: a2+2ab+b2=(a+b)2.

Take square roots of both sides:

∥ v + w ∥ \leq ∥ v ∥ + ∥ w ∥

Why we’re allowed to: The square root is increasing on the non-negatives, and both sides here are non-negative, so the inequality survives.

Source: Bretscher §5.1 Exercise 12

In $R^{n}$ there is no angle to measure. So we turn the formula round and use it as the DEFINITION.

By the range of $arccos$ , this $θ$ lies between $0$ and $π$ .

Source: Bretscher §5.1.12

so the input is always in range. Without the inequality, the definition would be meaningless.

Source: Bretscher §5.1.12

∥ x ∥ = 1, ∥ y ∥ = 1 + 1 + 1 + 1 = 2

θ = arccos (\frac{1}{1 \cdot 2}) = arccos (\frac{1}{2}) = \frac{π}{3}

Answer

$θ = π /3$ , that is $60°$ .

A clean answer in $R^{4}$ , where there is nothing to draw. That is the point of the definition.

Source: Bretscher §5.1 Example 7

x_{1}^{2} + \dots + x_{n}^{2} \geq \frac{1}{n}

When does equality occur?

How to think about it — before any algebra

Nothing here mentions vectors, so the first job is to MAKE two vectors whose inner product and norms produce the three quantities in the question.

What do I have? A sum of the $x_{i}$ , and a sum of the $x_{i}^{2}$ . The second is $∥ x ∥^{2}$ for $x = (x_{1}, \dots, x_{n})$ — that is the quantity I want to bound below.

The first is $\sum x_{i}$ , which is $⟨ x, y ⟩$ with $y = (1, 1, \dots, 1)$ . That fixes both vectors.

Now check the direction. Cauchy-Schwarz bounds $∣ ⟨ x, y ⟩ ∣$ from ABOVE by $∥ x ∥ ∥ y ∥$ . I want a LOWER bound on $∥ x ∥$ — but the inner product here is a fixed number, $1$ , so the upper bound on the left is really a lower bound on the right. It works out.

For the equality case: Cauchy-Schwarz is an equality exactly when the two vectors are parallel. $x$ parallel to $(1, \dots, 1)$ means all the $x_{i}$ are equal, and the constraint then pins their common value.

Set

x = (x_{1}, \dots, x_{n}), y = (1, 1, \dots, 1)

Then:

⟨ x, y ⟩ = x_{1} + \dots + x_{n} = 1

∥ x ∥^{2} = x_{1}^{2} + \dots + x_{n}^{2}, ∥ y ∥ = n

Cauchy-Schwarz gives

∣ ⟨ x, y ⟩ ∣ \leq ∥ x ∥ ∥ y ∥ ⟹ 1 \leq ∥ x ∥ n

Divide by $n > 0$ :

∥ x ∥ \geq \frac{1}{n}

Both sides are non-negative, so squaring preserves the direction:

x_{1}^{2} + \dots + x_{n}^{2} = ∥ x ∥^{2} \geq \frac{1}{n}

EQUALITY. Cauchy-Schwarz is an equality exactly when $x$ and $y$ are parallel, i.e. $x = c (1, 1, \dots, 1)$ for some scalar $c$ — every $x_{i}$ equal to $c$ .

The constraint then forces $n c = 1$ , so $c = 1/ n$ :

x_{1} = x_{2} = \dots = x_{n} = \frac{1}{n}

Check: $\sum x_{i}^{2} = n \cdot (1/ n)^{2} = 1/ n$ . ✓

x_{1}^{2} + \dots + x_{n}^{2} \geq \frac{1}{n}, equality iff x_{i} = \frac{1}{n} for all i

Marks: choosing $y = (1, \dots, 1)$ , identifying all three of $⟨ x, y ⟩ = 1$ , $∥ y ∥ = n$ and , applying the inequality, and handling the equality case.

The choice of $y$ is the whole question. Everything after it is substitution.

The equality case needs BOTH halves: parallel to $(1, \dots, 1)$ makes the entries equal, and only then does the constraint $\sum x_{i} = 1$ pin the common value to $1/ n$ . Stating "all equal" without using the constraint is half the mark.

Source: Seminar 1, Q3

Together: $v$ and $w$ point in the SAME direction, not merely along the same line. Anti-parallel vectors satisfy Cauchy-Schwarz with equality but fail step 2 — which matches the picture, since walking out and back is strictly longer than not moving.

⟨ e_{i}, e_{j} ⟩ = 0

2 x_{3} + 2 x_{4} = 0

⟨ u_{2}, x ⟩ = \frac{1}{2} (1 - 3 - 1 + 7) = 2

(x_{3}, x_{4}) = (1, 0)

ker A = span {(- 1, 1, 1, 0), (0, - 2, 0, 1)}

- 2 y_{2} + y_{4} = 0

(y_{2}, y_{3}) = (1, 0)

(ker A)^{⊥} = span {(1, 1, 0, 2), (1, 0, 1, 0)}

⟨(1, 0, 1, 0), (- 1, 1, 1, 0)⟩ = - 1 + 0 + 1 + 0 = 0 ✓

⟨(1, 0, 1, 0), (0, - 2, 0, 1)⟩ = 0 + 0 + 0 + 0 = 0 ✓

Why we’re allowed to: Cancel the two terms common to both sides. Both directions come out at once, which is why the theorem is an if-and-only-if.

∥ x ∥^{2} = \sum x_{i}^{2}