Quadratics in linear algebra


A quadratic or quadratic function is a polynomial whose highest-order term is a square. For instance $f(x) = 3x^2 + x -3$ is a quadratic.

Quadratics of more than one variable are polynomial functions in which each term is the product of at most two of the inputs, possibly the same input twice. For instance, in the quadratic

$f(x, y) =\kc{2}xy + \kc{3}x^2 - \kc{4}y^2 + \kc{3}x + \kc{8}$

the first term contains the product of both inputs, and the second and third term contain one of the inputs twice.

Quadratics are of particular interest in linear algebra, because they can be modeled by a matrix. We already know how a linear function of multiple variables can be modeled by a matrix $\bc{\A}$. We combine the inputs into a vector $\x$ and compute $\bc{\A}\x$ giving us a vector of all the output variables.

In the case of a quadratic function, we create $\x$ in the same way, and we require $\bc{\A}$ to be a square matrix. We then compute the value

$\x^T\bc{\A}\x \p$

This may look a little mysterious, but it’s just a concise way of writing second-order polynomials in multiple variables. For instance,

$$ \x^T\begin{pmatrix}\rc{2}& \bc{3}\\ \gc{4}& \oc{5} \end{pmatrix}\x = \rc{2}{x_1}^2 + \bc{3}x_2 x_1 + \gc{4} x_1x_2 + \oc{5}{x_2}^2 \p $$

Note the following about this definition:

We are used to thinking about the operation of a matrix on a vector. How does the quadratic form relate to this operation? One way to think about it is to realize that the quadratic form is the dot product of the input of this operation $\x$, with the output $\bc{\A}\x$.

One thing the dot product measures is how much two vectors point in the same direction. If a vector $\x$ leads to a low value for the quadratic $\x^T\bc{\A}\x$, then it may be because $\bc{\A}$ rotates it to poin t in a different direction.

To help you understanding, consider the following quadratic forms

Visualization: the unit circle

Ellipses and matrix square roots