Matrix (mathematics)

From Wikipedia, the free encyclopedia

In mathematics, a matrix (plural matrices, or less commonly matrixes) is a rectangular array of numbers, such as

Entries of a matrix are often denoted by a variable with two subscripts, as shown on the right. Matrices of the same size can be added and subtracted entrywise and matrices of compatible size can be multiplied. These operations have many of the properties of ordinary arithmetic, except that matrix multiplication is not commutative, that is, AB and BA are not equal in general. Matrices consisting of only one column or row are called vectors, while higher-dimensional, e.g. three-dimensional, arrays of numbers are called tensors. Matrices with entries in other fields or rings are also studied.

Matrices are a key tool in linear algebra. One use of matrices is to represent linear transformations, which are higher-dimensional analogs of linear functions of the form f(x) = cx, where c is a constant; matrix multiplication corresponds to composition of linear transformations. Matrices can also keep track of the coefficients in a system of linear equations. For a square matrix, the determinant and inverse matrix (when it exists) govern the behavior of solutions to the corresponding system of linear equations, and eigenvalues and eigenvectors provide insight into the geometry of the associated linear transformation.

Matrices find many applications. Physics makes use of matrices in various domains, for example in geometrical optics and matrix mechanics; the latter led to studying in more detail matrices with an infinite number of rows and columns. Graph theory uses matrices to keep track of distances between pairs of vertices in a graph. Computer graphics uses matrices to project 3-dimensional space onto a 2-dimensional screen. Matrix calculus generalizes classical analytical notions such as derivatives of functions or exponentials to matrices. The latter is a recurring need in solving ordinary differential equations. Serialism and dodecaphonism are musical movements of the 20th century that utilize a square mathematical matrix to determine the pattern of music intervals.

A major branch of numerical analysis is devoted to the development of efficient algorithms for matrix computations, a subject that is centuries old but still an active area of research. Matrix decomposition methods simplify computations, both theoretically and practically. For sparse matrices, specifically tailored algorithms can provide speedups; such matrices arise in the finite element method, for example.

[edit] Definition

A matrix is a rectangular arrangement of numbers.^[1] For example,

An alternative notation uses large parentheses instead of box brackets:

The horizontal and vertical lines in a matrix are called rows and columns, respectively. The numbers in the matrix are called its entries or its elements. To specify a matrix's size, a matrix with m rows and n columns is called an m-by-n matrix or m × n matrix, while m and n are called its dimensions. The above is a 4-by-3 matrix.

A matrix where one of the dimensions equals one is also called a vector, and may be interpreted as an element of real coordinate space. An m × 1 matrix (one column and m rows) is called a column vector and a 1 × n matrix (one row and n columns) is called a row vector. For example, the second row vector of the above matrix A is

Most of this article focuses on real and complex matrices, i.e., matrices whose entries are real or complex numbers. More general types of entries are discussed below.

[edit] Notation

The entry that lies in the i-th row and the j-th column of a matrix is typically referred to as the i,j, (i,j), or (i,j)^th entry of the matrix. For example, (2,3) entry of the above matrix A is 7. Matrices are usually denoted using upper-case letters, while the corresponding lower-case letters, with two subscript indices, represent the entries. For example, the (i, j)^th entry of a matrix A is most commonly written as a_i,j. Alternative notations for that entry are A[i,j] or A_i,j. In addition to using upper-case letters to symbolize matrices, many authors use a special typographical style, commonly boldface upright (non-italic), to further distinguish matrices from other variables. An asterisk is commonly used to refer to all of the rows or columns in a matrix. For example, a_i,∗ refers to the i^th row of A, and a_∗,j refers to the j^th column of A. The set of all m-by-n matrices is denoted M(m, n).

A common shorthand is

A = [a_i,j]_{i=1,...,m; j=1,...,n} or more briefly A = [a_i,j]_m×n

to define an m × n matrix A. In this case, the entries a_i,j are defined separately for all integers 1 ≤ i ≤ m and 1 ≤ j ≤ n; for example the above matrix A is specified by

A = [i , j]_{i=1,...,4; j=1,...,3}.

Some programming languages start the numbering of rows and columns at zero, in which case the entries of an m-by-n matrix are indexed by 0 ≤ i ≤ m − 1 and 0 ≤ j ≤ n − 1.^[2] This article follows the more common convention in mathematical writing where enumeration starts from 1.

[edit] Basic operations

There are a number of operations that can be applied to modify matrices called matrix addition, scalar multiplication and transposition.^[3] These form the basic techniques to deal with matrices.

Familiar properties of numbers extend to these operations of matrices: for example, addition is commutative, i.e. the matrix sum does not depend on the order of the summands: A + B = B + A.^[4] The transpose is compatible with addition and scalar multiplication, as expressed by (cA)^T = c(A^T) and (A + B)^T = A^T + B^T. Finally, (A^T)^T = A.

Row operations are ways to change matrices. There are three types of row operations: row switching, that is interchanging two rows of a matrix, row multiplication, multiplying all entries of a row by a non-zero constant and finally row addition which means adding a multiple of a row to another row. These row operations are used in a number of ways including solving linear equations and finding inverses.

[edit] Matrix multiplication, linear equations and linear transformations

Multiplication of two matrices is defined only if the number of columns of the left matrix is the same as the number of rows of the right matrix. If A is an m-by-n matrix and B is an n-by-p matrix, then their matrix product AB is the m-by-p matrix whose entries are given by

where 1 ≤ i ≤ m and 1 ≤ j ≤ p.^[5] For example (the underlined entry 1 in the product is calculated as the product 1 · 1 + 0 · 1 + 2 · 0 = 1):

Matrix multiplication satisfies the rules (AB)C = A(BC) (associativity), and (A+B)C = AC+BC as well as C(A+B) = CA+CB (left and right distributivity), whenever the size of the matrices is such that the various products are defined.^[6] The product AB may be defined without BA being defined, namely if A and B are m-by-n and n-by-k matrices, respectively, and m ≠ k. Even if both products are defined, they need not be equal, i.e. generally one has

AB ≠ BA,

i.e., matrix multiplication is not commutative, in marked contrast to (rational, real, or complex) numbers whose product is independent of the order of the factors. An example of two matrices not commuting with each other is:

whereas

The identity matrix I_n of size n is the n-by-n matrix in which all the elements on the main diagonal are equal to 1 and all other elements are equal to 0, e.g.

It is called identity matrix because multiplication with it leaves a matrix unchanged: MI_n = I_mM = M for any m-by-n matrix M.

Besides the ordinary matrix multiplication just described, there exist other less frequently used operations on matrices that can be considered forms of multiplication, such as the Hadamard product and the Kronecker product.^[7] They arise in solving matrix equations such as the Sylvester equation.

[edit] Linear equations

A particular case of matrix multiplication is tightly linked to linear equations: if x designates a column vector (i.e. n×1-matrix) of n variables x₁, x₂, ..., x_n, and A is an m-by-n matrix, then the matrix equation

Ax = b,

where b is some m×1-column vector, is equivalent to the system of linear equations

A_1,1x₁ + A_1,2x₂ + ... + A_1,nx_n = b₁

...

A_m,1x₁ + A_m,2x₂ + ... + A_m,nx_n = b_m .^[8]

This way, matrices can be used to compactly write and deal with multiple linear equations, i.e. systems of linear equations.

[edit] Linear transformations

Matrices and matrix multiplication reveal their essential features when related to linear transformations, also known as linear maps. A real m-by-n matrix A gives rise to a linear transformation Rⁿ → R^m mapping each vector x in Rⁿ to the (matrix) product Ax, which is a vector in R^m. Conversely, each linear transformation f: Rⁿ → R^m arises from a unique m-by-n matrix A: explicitly, the (i, j)-entry of A is the i^th coordinate of f(e_j), where e_j = (0,...,0,1,0,...,0) is the unit vector with 1 in the j^th position and 0 elsewhere. The matrix A is said to represent the linear map f, and A is called the transformation matrix of f.

The following table shows a number of 2-by-2 matrices with the associated linear maps of R². The blue original is mapped to the green grid and shapes, the origin (0,0) is marked with a black point.

Under the 1-to-1 correspondence between matrices and linear maps, matrix multiplication corresponds to composition of maps^[9]: if a k-by-m matrix B represents another linear map g : R^m → R^k, then the composition g ∘ f is represented by BA since

(g ∘ f)(x) = g(f(x)) = g(Ax) = B(Ax) = (BA)x.

The last equality follows from the above-mentioned associativity of matrix multiplication.

The rank of a matrix A is the maximum number of linearly independent row vectors of the matrix, which is the same as the maximum number of linearly independent column vectors.^[10] Equivalently it is the dimension of the image of the linear map represented by A.^[11] The rank-nullity theorem states that the dimension of the kernel of a matrix plus the rank equals the number of columns of the matrix.^[12]

[edit] Square matrices

A square matrix is a matrix which has the same number of rows and columns. A n-by-n matrix is known as a square matrix of order n. Any two square matrices of the same order can be added and multiplied. A square matrix A is called invertible or non-singular if there exists a matrix B such that

AB = I_n.^[13]

This is equivalent to BA = I_n.^[14] Moreover, if B exists, it is unique and is called the inverse matrix of A, denoted A⁻¹.

The entries A_i,i form the main diagonal of a matrix. The trace, tr(A) of a square matrix A is the sum of its diagonal entries. While, as mentioned above, matrix multiplication is not commutative, the trace of the product of two matrices is independent of the order of the factors: tr(AB) = tr(BA).^[15]

If all entries outside the main diagonal are zero, A is called a diagonal matrix. If only all entries above (below) the main diagonal are zero, A is called a lower triangular matrix (upper triangular matrix, respectively). For example, if n = 3, they look like

(diagonal), (lower) and (upper triangular matrix).

[edit] Determinant

The determinant det(A) or |A| of a square matrix A is a number encoding certain properties of the matrix. A matrix is invertible if and only if its determinant is nonzero. Its absolute value equals the area (in R²) or volume (in R³) of the image of the unit square (or cube), while its sign corresponds to the orientation of the corresponding linear map: the determinant is positive if and only if the orientation is preserved.

The determinant of 2-by-2 matrices is given by

the determinant of 3-by-3 matrices involves 6 terms (rule of Sarrus). The more lengthy Leibniz formula generalises these two formulae to all dimensions.^[16]

The determinant of a product of square matrices equals the product of their determinants: det(AB) = det(A) · det(B).^[17] Adding a multiple of any row to another row, or a multiple of any column to another column, does not change the determinant. Interchanging two rows or two columns affects the determinant by multiplying it by −1.^[18] Using these operations, any matrix can be transformed to a lower (or upper) triangular matrix, and for such matrices the determinant equals the product of the entries on the main diagonal; this provides a method to calculate the determinant of any matrix. Finally, the Laplace expansion expresses the determinant in terms of minors, i.e., determinants of smaller matrices.^[19] This expansion can be used for a recursive definition of determinants (taking as starting case the determinant of a 1-by-1 matrix, which is its unique entry, or even the determinant of a 0-by-0 matrix, which is 1), that can be seen to be equivalent to the Leibniz formula. Determinants can be used to solve linear systems using Cramer's rule, where the division of the determinants of two related square matrices equates to the value of each of the system's variables.^[20]

[edit] Eigenvalues and eigenvectors

A number λ and a non-zero vector v satisfying

Av = λv

are called an eigenvalue and an eigenvector of A, respectively.^{[nb 1][21]} The number λ is an eigenvalue of an n×n-matrix A if and only if A−λI_n is not invertible, which is equivalent to

det(A−λI) = 0.^[22]

The function p_A(t) = det(A−tI) is called the characteristic polynomial of A, its degree is n. Therefore p_A(t) has at most n different roots, i.e., eigenvalues of the matrix.^[23] They may be complex even if the entries of A are real. According to the Cayley-Hamilton theorem, p_A(A) = 0, that is to say, the characteristic polynomial applied to the matrix itself yields the zero matrix.

[edit] Symmetry

A square matrix A that is equal to its transpose, i.e. A = A^T, is a symmetric matrix; if it is equal to the negative of its transpose, i.e. A = −A^T, then it is a skew-symmetric matrix. In complex matrices, symmetry is often replaced by the concept of Hermitian matrices, which satisfy A^∗ = A, where the star denotes the conjugate transpose of the matrix, i.e. the transpose of the complex conjugate of A.

By the spectral theorem, real symmetric matrices and complex Hermitian matrices have an eigenbasis; i.e., every vector is expressible as a linear combination of eigenvectors. In both cases, all eigenvalues are real.^[24] This theorem can be generalized to infinite-dimensional situations related to matrices with infinitely many rows and columns, see below.

[edit] Definiteness

A symmetric n×n-matrix is called positive definite (negative definite, indefinite, resp.), if for all nonzero vectors x ∈ Rⁿ the associated quadratic form given by

Q(x) = x^TAx

takes only positive values (negative, both negative and positive values, respectively).^[25] Allowing as input two different vectors instead yields the bilinear form associated to A:

B_A (x, y) = x^TAy.^[26]

A symmetric matrix is positive definite if and only if all its eigenvalues are positive.^[27] The table at the right shows two possibilities for 2-by-2 matrices.

[edit] Computational aspects

In addition to theoretical knowledge of properties of matrices and their relation to other fields, it is important for practical purposes to perform matrix calculations effectively and precisely. The domain studying these matters is called numerical linear algebra.^[28] As with other numerical situations, two main aspects are the complexity of algorithms and their numerical stability. Many problems can be solved by both direct algorithms or iterative approaches. For example, finding eigenvectors can be done by finding a sequence of vectors x_n converging to an eigenvector when n tends to infinity.^[29]

Determining the complexity of an algorithm means finding upper bounds or estimates of how many elementary operations such as additions and multiplications of scalars are necessary to perform some algorithm, e.g. multiplication of matrices. For example, calculating the matrix product of two n-by-n matrix using the definition given above needs n³ multiplications, since for any of the n² entries of the product, n multiplications are necessary. The Strassen algorithm outperforms this "naive" algorithm; it needs only n^2.807 multiplications.^[30] A refined approach also incorporates specific features of the computing devices.

In many practical situations additional information about the matrices involved is known. An important case are sparse matrices, i.e. matrices most of whose entries are zero. There are specifically adapted algorithms for, say, solving linear systems Ax = b for sparse matrices A, such as the conjugate gradient method.^[31]

An algorithm is, roughly speaking, numerical stable, if little deviations (such as rounding errors) do not lead to big deviations in the result. For example, calculating the inverse of a matrix via Laplace's formula (Adj (A) denotes the adjugate matrix of A)

A⁻¹ = Adj(A) / det(A)

may lead to significant rounding errors if the determinant of the matrix is very small. The norm of a matrix can be used to capture the conditioning of linear algebraic problems, such as computing a matrix' inverse.^[32]

Although most computer languages are not designed with commands or libraries for matrices, as early as the 1970s, some engineering desktop computers such as the HP 9830 had ROM cartridges to add BASIC commands for matrices. Some computer languages such as APL were designed to manipulate matrices, and various mathematical programs can be used to aid computing with matrices.^[33]

[edit] Matrix decomposition methods

There are several methods to render matrices into a more easily accessible form. They are generally referred to as matrix transformation or matrix decomposition techniques. The interest of all these decomposition techniques is that they preserve certain properties of the matrices in question, such as determinant, rank or inverse, so that these quantities can be calculated after applying the transformation, or that certain matrix operations are algorithmically easier to carry out for some types of matrices.

The LU decomposition factors matrices as a product of lower (L) and an upper triangular matrices (U).^[34] Once this decomposition is calculated, linear systems can be solved more efficiently, by a simple technique called forward and back substitution. Likewise, inverses of triangular matrices are algorithmically easier to calculate. The Gaussian elimination is a similar algorithm; it transforms any matrix to row echelon form.^[35] Both methods proceed by multiplying the matrix by suitable elementary matrices, which correspond to permuting rows or columns and adding multiples of one row to another row. Singular value decomposition expresses any matrix A as a product UDV^∗, where U and V are unitary matrices and D is a diagonal matrix.

The eigendecomposition or diagonalization expresses A as a product VDV⁻¹, where D is a diagonal matrix and V is a suitable invertible matrix.^[36] If A can be written in this form, it is called diagonalizable. More generally, and applicable to all matrices, the Jordan decomposition transforms a matrix into Jordan normal form, that is to say matrices whose only nonzero entries are the eigenvalues λ₁ to λ_n of A, placed on the main diagonal and possibly entries equal to one directly above the main diagonal, as shown at the right.^[37] Given the eigendecomposition, the n^th power of A (i.e. n-fold iterated matrix multiplication) can be calculated via

Aⁿ = (VDV⁻¹)ⁿ = VDV⁻¹VDV⁻¹...VDV⁻¹ = VDⁿV⁻¹

and the power of a diagonal matrix can be calculated by taking the corresponding powers of the diagonal entries, which is much easier than doing the exponentiation for A instead. This can be used to compute the matrix exponential e^A, a need frequently arising in solving linear differential equations, matrix logarithms and square roots of matrices.^[38] To avoid numerically ill-conditioned situations, further algorithms such as the Schur decomposition can be employed.^[39]

[edit] Abstract algebraic aspects and generalizations

Matrices can be generalized in different ways. Abstract algebra uses matrices with entries in more general fields or even rings, while linear algebra codifies properties of matrices in the notion of linear maps. It is possible to consider matrices with infinitely many columns and rows. Another extension are tensors, which can be seen as higher-dimensional arrays of numbers, as opposed to vectors, which can often be realised as sequences of numbers, while matrices are rectangular or two-dimensional array of numbers.^[40] Matrices, subject to certain requirements tend to form groups known as matrix groups.

[edit] Matrices with more general entries

This article focuses on matrices whose entries are real or complex numbers. However, matrices can be considered with much more general types of entries than real or complex numbers. As a first step of generalization, any field, i.e. a set where addition, subtraction, multiplication and division operations are defined and well-behaved, may be used instead of R or C, for example rational numbers or finite fields. For example, coding theory makes use of matrices over finite fields. Wherever eigenvalues are considered, the choice of the field usually matters insofar as a the characteristic polynomial, despite having real coefficients may have complex solutions. Therefore, the field is often required to be C or any algebraically closed field when such issues arise.

More generally, abstract algebra makes great use of matrices with entries in a ring R.^[41] Rings are a more general notion than fields in that no division operation exists. The very same addition and multiplication operations of matrices extend to this setting, too. The set M(n, R) of all square n-by-n matrices over R is a ring called matrix ring, isomorphic to the endomorphism ring of the left R-module Rⁿ.^[42] If the ring R is commutative, i.e., its multiplication is commutative, then M(n, R) is a unitary noncommutative (unless n = 1) associative algebra over R. The determinant of square matrices can still be defined using the Leibniz formula; a matrix is invertible if and only if its determinant is invertible in R, generalising the situation over a field F, where every nonzero element is invertible.^[43] Matrices over superrings are called supermatrices.^[44]

[edit] Relationship to linear maps

Linear maps Rⁿ → R^m are equivalent to n-by-m matrices, as described above. More generally, any linear map f: V → W between finite-dimensional vector spaces can be described by a matrix A = (a_ij), by choosing bases v₁, ..., v_m, and w₁, ..., w_n, where m and n are the dimensions of V and W, respectively, and requiring

This uniquely determines the entries of the matrix A, but the matrix depends on the choice of the bases: different choices of bases give rise to different, but similar matrices.^[45] Many of the above concrete notions can be reinterpreted in this light, for example, the transpose matrix A^T describes the transpose of the linear map given by A, with respect to the dual bases.^[46]

[edit] Matrix groups

A group is a mathematical structure consisting of a set of objects together with a binary operation, i.e. an operation combining any two objects to a third, subject to certain requirements.^[47] A group in which the objects are matrices and the group operation is matrix multiplication is called a matrix group.^{[nb 2][48]} Since in a group every element has to be invertible, the most general matrix groups are the groups of all invertible matrices of a given size, called the general linear groups.

Any property of matrices that is preserved under matrix products and inverses can be used to define further matrix groups. For example, matrices with a given size and with a determinant of 1 form a subgroup of (i.e. a smaller group contained in) their general linear group, called a special linear group.^[49] Orthogonal matrices, determined by the condition

M^TM = I,

form the orthogonal group.^[50] They are called orthogonal since the associated linear transformations of Rⁿ preserve angles in the sense that the scalar product of two vectors is unchanged after applying M to them:

(Mv) · (Mw) = v · w.^[51]

Every finite group is isomorphic to a matrix group, as one can see by considering the regular representation of the symmetric group.^[52] General groups can be studied using matrix groups, which are comparatively well-understood, by means of representation theory.^[53]

[edit] Infinite matrices

It is also possible to consider matrices with infinitely many rows and/or columns.^[54] The basic operations introduced above are defined the same way in this case. Matrix multiplication, however, and all operations stemming therefrom are only meaningful when restricted to certain matrices, since the sum featuring in the above definition of the matrix product will contain an infinity of summands. An easy way to circumvent this issue is to restrict to matrices all of whose rows (or columns) contain only finitely many nonzero terms. As in the finite case (see above), where matrices describe linear maps, infinite matrices can be used to describe operators on Hilbert spaces, where convergence and continuity questions arise. However, the explicit point of view of matrices tends to obfuscate the matter,^{[nb 3]} and the abstract and more powerful tools of functional analysis are used instead, by relating matrices to linear maps (as in the finite case above), but imposing additional convergence and continuity constraints.

[edit] Empty matrices

An empty matrix is a matrix in which the number of rows or columns (or both) is zero.^[55][56]. An empty matrix has no entries but it still has a well defined number of rows and columns, which are needed for instance in the definition of the matrix product. Thus if A is the 3-by-0 matrix A and B is the 0-by-3 matrix B, then AB is the 3-by-3 zero matrix (corresponding to the null map from a 3-dimensional space V to itself obtained obtained as composition of the unique map f from V to a 0-dimensional space Z, followed by the zero map g from Z back to V), while BA is the 0-by-0 matrix (corresponding to the unique map from Z to itself obtained as composition ). There is no common notation for empty matrices but most computer algebra systems will allow creating them and computing with them. Note that the determinant of the 0-by-0 matrix is 1 (and not 0 as might seem more natural): the Leibniz formula produces this value as a sum over the unique permutation of the empty set, with an empty product as term; also the Laplace expansion for a 1-by-1 matrix makes clear that the value of the 0-by-0 minor should be taken to be 1. This value is also consistent with the fact that the identity map from any finite dimensional space to itself has determinant 1, a fact that is often used as a part of the characterization of determinants.

[edit] Applications

There are numerous applications of matrices, both in mathematics and other sciences. Some of them merely take advantage of the compact representation of a set of numbers in a matrix. For example, in game theory and economics, the payoff matrix encodes the payoff for two players, depending on which out of a given (finite) set of alternatives the players choose.^[57] Text mining and automated thesaurus compilation makes use of document-term matrices such as tf-idf in order to keep track of frequencies of certain words in several documents.^[58]

Complex numbers can be represented by particular real 2-by-2 matrices via

under which addition and multiplication of complex numbers and matrices correspond to each other. For example, 2-by-2 rotation matrices represent the multiplication with some complex number of absolute value 1, as above. A similar interpretation is possible for quaternions.^[59]

Early encryption techniques such as the Hill cipher also used matrices. However, due to the linear nature of matrices, these codes are comparatively easy to break.^[60] Computer graphics uses matrices both to represent objects and to calculate transformations of objects using affine rotation matrices to accomplish tasks such as projecting a three-dimensional object onto a two-dimensional screen, corresponding to a theoretical camera observation.^[61] Matrices over a polynomial ring are important in the study of control theory.

Chemistry makes use of matrices in various ways, particularly since the use of quantum theory to discuss molecular bonding and spectroscopy. Examples are the overlap matrix and the Fock matrix using in solving the Roothaan equations to obtain the molecular orbitals of the Hartree–Fock method.

[edit] Graph theory

The adjacency matrix of a finite graph is a basic notion of graph theory.^[62] It saves which vertices of the graph are connected by an edge. Matrices containing just two different values (0 and 1 meaning for example "yes" and "no") are called logical matrices. The distance (or cost) matrix contains information about distances of the edges.^[63] These concepts can be applied to websites connected hyperlinks or cities connected by roads etc., in which case (unless the road network is extremely dense) the matrices tend to be sparse, i.e. contain few nonzero entries. Therefore, specifically tailored matrix algorithms can be used in network theory.

[edit] Analysis and geometry

The Hessian matrix of a differentiable function f: Rⁿ → R consists of the second derivatives of f with respect to the several coordinate directions, i.e.

^[64]

It encodes information about the local growth behaviour of the function: given a critical point x = (x₁, ..., x_n), i.e., a point where the first partial derivatives of f vanish, the function has a local minimum if the Hessian matrix is positive definite. Quadratic programming can be used to find global minima or maxima of quadratic functions closely related to the ones attached to matrices (see above).^[65]

Another matrix frequently used in geometrical situations is the Jacobi matrix of a differentiable map f: Rⁿ → R^m. If f₁, ..., f_m denote the components of f, then the Jacobi matrix is defined as

^[66]

If n > m, and if the rank of the Jacobi matrix attains its maximal value m, f is locally invertible at that point, by the implicit function theorem.^[67]

Partial differential equations can be classified by considering the matrix of coefficients of the highest-order differential operators of the equation. For elliptic partial differential equations this matrix is positive definite, which has decisive influence on the set of possible solutions of the equation in question.^[68]

The finite element method is an important numerical method to solve partial differential equations, widely applied in simulating complex physical systems. It attempts to approximate the solution to some equation by piecewise linear functions, where the pieces are chosen with respect to a sufficiently fine grid, which in turn can be recast as a matrix equation.^[69]

[edit] Probability theory and statistics

Stochastic matrices are square matrices whose rows are probability vectors, i.e., whose entries sum up to one. Stochastic matrices are used to define Markov chains with finitely many states.^[70] A row of the stochastic matrix gives the probability distribution for the next position of some particle which is currently in the state corresponding to the row. Properties of the Markov chain like absorbing states, i.e. states that any particle attains eventually, can be read off the eigenvectors of the transition matrices.^[71]

Statistics also makes use of matrices in many different forms. Descriptive statistics is concerned with describing data sets, which can often be represented in matrix form, by reducing the amount of data. The covariance matrix encodes the mutual variance of several random variables.^[72] Another technique using matrices are linear least squares, a method that approximates a finite set of pairs (x₁, y₁), (x₂, y₂), ..., (x_N, y_N), by a linear function

y_i ≈ ax_i + b, i = 1, ..., N

which can be formulated in terms of matrices, related to the singular value decomposition of matrices.^[73]

Random matrices are matrices whose entries are random numbers, subject to suitable probability distributions, such as matrix normal distribution. Beyond probability theory, they are applied in domains ranging from number theory to physics.^[74][75]

[edit] Symmetries and transformations in physics

Linear transformations and the associated symmetries play a key role in modern physics. For example, elementary particles in quantum field theory are classified as representations of the Lorentz group of special relativity and, more specifically, by their behavior under the spin group. Concrete representations involving the Pauli matrices and more general gamma matrices are an integral part of the physical description of fermions, which behave as spinors.^[76] For the three lightest quarks, there is a group-theoretical representation involving the special unitary group SU(3); for their calculations, physicists use a convenient matrix representation known as the Gell-Mann matrices, which are also used for the SU(3) gauge group that forms the basis of the modern description of strong nuclear interactions, quantum chromodynamics. The Cabibbo-Kobayashi-Maskawa matrix, in turn, expresses the fact that the basic quark states that are important for weak interactions are not the same as, but linearly related to the basic quark states that define particles with specific and distinct masses.^[77]

[edit] Linear combinations of quantum states

The first model of quantum mechanics (Heisenberg, 1925) represented the theory's operators by infinite-dimensional matrices acting on quantum states.^[78] This is also referred to as matrix mechanics. One particular example is the density matrix that characterizes the "mixed" state of a quantum system as a linear combination of elementary, "pure" eigenstates.^[79]

Another matrix serves as a key tool for describing the scattering experiments which form the cornerstone of experimental particle physics: Collision reactions such as occur in particle accelerators, where non-interacting particles head towards each other and collide in a small interaction zone, with a new set of non-interacting particles as the result, can be described as the scalar product of outgoing particle states and a linear combination of ingoing particle states. The linear combination is given by a matrix known as the S-matrix, which encodes all information about the possible interactions between particles.^[80]

[edit] Normal Modes

A general application of matrices in physics is to the description of linearly coupled harmonic systems. The equations of motion of such systems can be described in matrix form, with a mass matrix multiplying a generalized velocity to give the kinetic term, and a force matrix multiplying a displacement vector to characterize the interactions. The best way to obtain solutions is to determine the system's eigenvectors, its normal modes, by diagonalizing the matrix equation. Techniques like this are crucial when it comes to describing the internal dynamics of molecules: the internal vibrations of systems consisting of mutually bound component atoms.^[81] They are also needed for describing mechanical vibrations, and oscillations in electrical circuits.^[82]

[edit] Geometrical optics

Geometrical optics provides further matrix applications. In this approximative theory, the wave nature of light is neglected. The result is a model in which light rays are indeed geometrical rays. If the deflection of light rays by optical elements is small, the action of a lens or reflective element on a given light ray can be expressed as multiplication of a two-component vector with a two-by-two matrix called ray transfer matrix: the vector's components are the light ray's slope and its distance from the optical axis, while the matrix encodes the properties of the optical element. The matrix characterizing an optical system consisting of a combination of lenses and/or reflective elements is simply the product of the components' matrices.^[83]

[edit] History

Matrices have a long history of application in solving linear equations. The Chinese text The Nine Chapters on the Mathematical Art (Jiu Zhang Suan Shu), from between 300 BC and AD 200, is the first example of the use of matrix methods to solve simultaneous equations,^[84] including the concept of determinants, almost 2000 years before its publication by the Japanese mathematician Seki in 1683 and the German mathematician Leibniz in 1693. Cramer presented Cramer's rule in 1750.

Early matrix theory emphasized determinants more strongly than matrices and an independent matrix concept akin to the modern notion emerged only in 1858, with Cayley's Memoir on the theory of matrices.^[85][86] The term "matrix" was coined by Sylvester, who understood a matrix as an object giving rise to a number of determinants today called minors, that is to say, determinants of smaller matrices which derive from the original one by removing columns and rows. Etymologically, matrix derives from Latin mater (mother).^[87]

The study of determinants sprang from several sources.^[88] Number-theoretical problems led Gauss to relate coefficients of quadratic forms, i.e., expressions such as x² + xy − 2y², and linear maps in three dimensions to matrices. Eisenstein further developed these notions, including the remark that, in modern parlance, matrix products are non-commutative. Cauchy was the first to prove general statements about determinants, using as definition of the determinant of a matrix A = [a_i,j] the following: replace the powers a_j^k by a_jk in the polynomial

( denotes the product of the indicated terms.) He also showed, in 1829, that the eigenvalues of symmetric matrices are real.^[89] Jacobi studied "functional determinants"—later called Jacobi determinants by Sylvester—which can be used to describe geometric transformations at a local (or infinitesimal) level, see above; Kronecker's Vorlesungen über die Theorie der Determinanten^[90] and Weierstrass' Zur Determinantentheorie,^[91] both published in 1903, first treated determinants axiomatically, as opposed to previous more concrete approaches such as the mentioned formula of Cauchy. At that point, determinants were firmly established.

Many theorems were first established for small matrices only, for example the Cayley-Hamilton theorem was proved for 2×2 matrices by Cayley in the aforementioned memoir, and by Hamilton for 4×4 matrices. Frobenius, working on bilinear forms, generalized the theorem to all dimensions (1898). Also at the end of the 19th century the Gauss-Jordan elimination (generalizing a special case now known as Gauss elimination) was established by Jordan. In the early 20th century, matrices attained a central role in linear algebra.^[92]

The inception of matrix mechanics by Heisenberg, Born and Jordan led to studying matrices with infinitely many rows and columns.^[93] Later, von Neumann carried out the mathematical formulation of quantum mechanics, by further developing functional analytic notions such as linear operators on Hilbert spaces, which, very roughly speaking, correspond to Euclidean space, but with an infinity of independent directions.

[edit] Other historical usages of the word "matrix" in mathematics

The word has been used in unusual ways by at least two authors of historical importance.

Bertrand Russell and Alfred North Whitehead in their Principia Mathematica (1910-1913) use the word matrix in the context of their Axiom of reducibility. They proposed this axiom as a means to reduce any function to one of lower type, successively, so that at the "bottom" (0 order) the function will be identical to its extension:

"Let us give the name of matrix to any function, of however many variables, which does not involve any apparent variables. Then any possible function other than a matrix is derived from a matrix by means of generalization, i.e. by considering the proposition which asserts that the function in question is true with all possible values or with some value of one of the arguments, the other argument or arguments remaining undetermined"^[94].

For example a function Φ(x, y) of two variables x and y can be reduced to a collection of functions of a single variable, e.g. y, by "considering" the function for all possible values of "individuals" a_i substituted in place of variable x. And then the resulting collection of functions of the single variable y, i.e. ∀a_i: Φ(a_i, y), can be reduced to a "matrix" of values by "considering" the function for all possible values of "individuals" b_i substituted in place of variable y:

∀b_j∀a_i: Φ(a_i, b_j)

Alfred Tarski in his 1946 Introduction to Logic used the word "matrix" synonymously with the notion of truth table as used in mathematical logic.^[95]