Positive Definite Matrices

Definition

A symmetric (or Hermitian) $n \times n$ matrix $A$ is positive definite (PD) if $\mathbf{x}^T A \mathbf{x} > 0 \quad \text{for every nonzero } \mathbf{x} \in \mathbb{R}^n$

Positive semidefinite (PSD): the inequality is $\ge 0$, with possible equality when $\mathbf{x} \ne \mathbf{0}$.

Equivalent conditions — pick whichever is easiest

For a symmetric $A$, all of the following are equivalent:

1. $A$ is positive definite.
2. All eigenvalues are strictly positive: $\lambda_i > 0$.
3. All leading principal minors are positive (Sylvester's criterion): $\det$ of top-left $k \times k$ submatrices $> 0$ for every $k = 1, \ldots, n$.
4. Cholesky exists: $A = L L^T$ with $L$ lower triangular and positive diagonal entries.
5. $A = B^T B$ for some invertible $B$.

Step-by-step — verify $A = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix}$ is PD

$A$ is symmetric ✓ (necessary precondition).

Check 1 — Principal minors (Sylvester).

• $\det(A_1) = A_{11} = 2 > 0$ ✓
• $\det(A_2) = \det A = (2)(3) - (1)(1) = 5 > 0$ ✓

All positive → $A$ is PD.

Check 2 — Eigenvalues.

Characteristic polynomial: $(2 - \lambda)(3 - \lambda) - 1 = \lambda^2 - 5\lambda + 5 = 0$.

$\lambda = \dfrac{5 \pm \sqrt{25 - 20}}{2} = \dfrac{5 \pm \sqrt{5}}{2} \approx \{3.618, 1.382\}$

Both $> 0$ → $A$ is PD. ✓ (Consistent with Sylvester.)

Check 3 — Quadratic form.

$\mathbf{x}^T A \mathbf{x} = 2 x_1^2 + 2 x_1 x_2 + 3 x_2^2$.

Complete the square: $2(x_1^2 + x_1 x_2) + 3 x_2^2 = 2(x_1 + \tfrac{1}{2} x_2)^2 - \tfrac{1}{2} x_2^2 + 3 x_2^2$ $= 2(x_1 + \tfrac{1}{2} x_2)^2 + \tfrac{5}{2} x_2^2$

Sum of squares with positive coefficients — strictly positive for any $\mathbf{x} \ne \mathbf{0}$ ✓

Geometric picture

The level sets of $\mathbf{x}^T A \mathbf{x} = c$ for PD matrices are ellipsoids centered at the origin. Their axes align with $A$'s eigenvectors; the semi-axis lengths are $\sqrt{c / \lambda_i}$.

For our $A$: two axes at the eigenvector directions, lengths proportional to $1/\sqrt{\lambda_i}$. Since $\lambda_1 \approx 3.62$ (shorter axis) and $\lambda_2 \approx 1.38$ (longer axis).

PSD (but not PD) produces degenerate "ellipsoids" — cylinders or half-spaces.

Non-PD indefinite matrices produce hyperboloids or saddle-like surfaces.

Why PD matters

• Minimization. A quadratic $f(\mathbf{x}) = \tfrac{1}{2} \mathbf{x}^T A \mathbf{x} - \mathbf{b}^T \mathbf{x} + c$ has a unique minimum iff $A$ is PD.
• Covariance. Any valid covariance matrix is PSD; PD when no linear dependency among variables.
• Kernel methods, Gaussian processes. Require PSD kernel matrices.
• Physics. Stable equilibria have PD Hessians.
• Cholesky factorization is the fastest solver for $A \mathbf{x} = \mathbf{b}$ with $A$ PD — half the cost of LU.

Cholesky decomposition

For $A = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix}$: $L = \begin{pmatrix} \sqrt{2} & 0 \\ 1/\sqrt{2} & \sqrt{5/2} \end{pmatrix}$

Verify: $L L^T = \begin{pmatrix} 2 & 1 \\ 1 & 1/2 + 5/2 \end{pmatrix} = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix}$ ✓

Exists uniquely (with positive diagonal) iff $A$ is PD.

Common mistakes

• Forgetting symmetry. PD requires $A^T = A$. A non-symmetric $A$ may satisfy $\mathbf{x}^T A \mathbf{x} > 0$ for all $\mathbf{x}$, but we still call it "positive" only when symmetric.
• Checking only one eigenvalue. All must be positive.
• Mixing leading minors with arbitrary minors. Sylvester's criterion uses the specific top-left submatrices.

Try it in the visualization

Level curves of $\mathbf{x}^T A \mathbf{x}$ are drawn for several $c$ values. Sliders change $A$'s entries; you see ellipses morph continuously and turn into hyperbolas the instant PD is lost. Eigenvalue signs display live.