Inner Product Spaces: Functions as Vectors

Generalizing the dot product

An inner product on a vector space $V$ is a function $\langle \cdot, \cdot \rangle : V \times V \to \mathbb{R}$ satisfying four axioms for all $\mathbf{u}, \mathbf{v}, \mathbf{w} \in V$ and scalars $c$:

1. Symmetric: $\langle \mathbf{u}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{u} \rangle$.
2. Linear in first slot: $\langle c \mathbf{u} + \mathbf{w}, \mathbf{v} \rangle = c \langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{w}, \mathbf{v} \rangle$.
3. Positive: $\langle \mathbf{u}, \mathbf{u} \rangle \ge 0$.
4. Definite: $\langle \mathbf{u}, \mathbf{u} \rangle = 0 \implies \mathbf{u} = \mathbf{0}$.

A vector space with an inner product is an inner product space. All the geometric concepts that work in $\mathbb{R}^n$ (length, angle, orthogonality, projections) generalize immediately.

The standard inner products

• Euclidean on $\mathbb{R}^n$: $\langle \mathbf{u}, \mathbf{v} \rangle = \sum u_i v_i$ — the ordinary dot product.
• $L^2$ on continuous functions: $\langle f, g \rangle = \int_a^b f(x) g(x) \, dx$.
• Weighted: $\langle \mathbf{u}, \mathbf{v} \rangle = \mathbf{u}^T W \mathbf{v}$ for positive-definite $W$.
• Matrix (Frobenius): $\langle A, B \rangle = \operatorname{tr}(A^T B) = \sum A_{ij} B_{ij}$.

Step-by-step

$f(x) = x$, $g(x) = x^2$ on $[0, 1]$.

$\langle f, g \rangle = \int_0^1 x \cdot x^2 \, dx = \int_0^1 x^3 \, dx = \left[\dfrac{x^4}{4}\right]_0^1 = \dfrac{1}{4}$

$\boxed{\langle f, g \rangle = \tfrac{1}{4}}$

Norms and angles

Once you have an inner product, you get a norm $\|\cdot\|$ and an angle $\theta$ via: $\|\mathbf{u}\| = \sqrt{\langle \mathbf{u}, \mathbf{u} \rangle}, \quad \cos \theta = \dfrac{\langle \mathbf{u}, \mathbf{v} \rangle}{\|\mathbf{u}\| \, \|\mathbf{v}\|}$

For our example: $\|f\|^2 = \int_0^1 x^2 \, dx = \tfrac{1}{3} \implies \|f\| = \tfrac{1}{\sqrt{3}}$ $\|g\|^2 = \int_0^1 x^4 \, dx = \tfrac{1}{5} \implies \|g\| = \tfrac{1}{\sqrt{5}}$

$\cos \theta = \dfrac{1/4}{\tfrac{1}{\sqrt{3}} \cdot \tfrac{1}{\sqrt{5}}} = \dfrac{1}{4} \cdot \sqrt{15} = \dfrac{\sqrt{15}}{4} \approx 0.968$

So the "angle" between $f$ and $g$ is about $\arccos(0.968) \approx 14.5°$. They're almost parallel as functions on $[0,1]$ — both increasing and positive on the interval.

Cauchy–Schwarz inequality

For any $\mathbf{u}, \mathbf{v}$ in an inner product space: $|\langle \mathbf{u}, \mathbf{v} \rangle| \le \|\mathbf{u}\| \, \|\mathbf{v}\|$

Equality iff $\mathbf{u}$ and $\mathbf{v}$ are linearly dependent. This is the most important inequality in inner product spaces — it's what lets you define an angle at all.

For our $f, g$: $|1/4| = 0.25 \le (1/\sqrt{3})(1/\sqrt{5}) = 1/\sqrt{15} \approx 0.258$ ✓

Orthogonality in function spaces

Two functions are orthogonal under $\int_a^b f g \, dx$ when their product integrates to zero. Classic examples:

• $\sin(nx)$ and $\cos(mx)$ on $[-\pi, \pi]$ for any $m, n$: $\int_{-\pi}^{\pi} \sin(nx) \cos(mx) \, dx = 0$.
• $\sin(nx)$ and $\sin(mx)$ for $m \ne n$: orthogonal.
• $\cos(nx)$ and $\cos(mx)$ for $m \ne n$: orthogonal.

This is the foundation of Fourier series — functions are decomposed onto the orthogonal basis $\{1, \sin(nx), \cos(nx)\}$.

Where inner product spaces matter

• Fourier series and transforms: expand signals onto orthonormal function bases.
• Quantum mechanics: states are vectors in an inner product (Hilbert) space, observables are Hermitian operators.
• Statistics: variance is a squared norm, correlation is essentially $\cos \theta$ on centered random variables.
• Machine learning: kernel methods define inner products in infinite-dimensional feature spaces without computing them directly.

Common mistakes

• Forgetting the axioms. Not every bilinear form is a valid inner product; it must be symmetric and positive-definite.
• Treating inner products of functions like of vectors. The computation now involves an integral, not a sum, but the geometric intuition transfers directly.
• Missing the weight. In weighted inner products $\int_a^b f g w \, dx$, forgetting $w$ gives the wrong answer.

Try it in the visualization

Two functions are drawn on $[0, 1]$. Their product is shaded; the area under the product curve equals the inner product. Drag function parameters and watch how the area (and hence $\langle f, g \rangle$) changes.