53
Rank (linear algebra) From Wikipedia, the free encyclopedia

Rank (Linear Algebra)

  • Upload
    man

  • View
    22

  • Download
    1

Embed Size (px)

DESCRIPTION

1. From Wikipedia, the free encyclopedia2. Lexicographical order

Citation preview

  • Rank (linear algebra)From Wikipedia, the free encyclopedia

  • Contents

    1 Quotient space (linear algebra) 11.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Quotient of a Banach space by a subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

    1.4.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4.2 Generalization to locally convex spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

    1.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

    2 Rank (linear algebra) 42.1 Main denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Computing the rank of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.3.1 Rank from row echelon forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3.2 Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.4 Proofs that column rank = row rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.4.1 First proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4.2 Second proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    2.5 Alternative denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.6 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.8 Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.9 Matrices as tensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.10 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.11 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.13 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

    3 Rank factorization 113.1 rank(A) = rank(AT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 Rank factorization from row echelon forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

    i

  • ii CONTENTS

    3.4 Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

    4 Rayleigh quotient 134.1 Bounds for HermitianM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.2 Special case of covariance matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

    4.2.1 Formulation using Lagrange multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.3 Use in SturmLiouville theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.4 Generalizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.7 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

    5 2 2 real matrices 175.1 Prole . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.2 Equi-areal mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.3 Functions of 2 2 real matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.4 2 2 real matrices as complex numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    6 Reality structure 216.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

    7 Reduction (mathematics) 237.1 Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237.2 Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237.3 Static (Guyan) Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

    8 Relative dimension 25

    9 Resolvent set 269.1 Denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269.4 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

    10 Restricted isometry property 2810.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2810.2 Restricted Isometric Constant (RIC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2810.3 Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2810.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2910.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

  • CONTENTS iii

    11 Rotas basis conjecture 3011.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3011.2 Partial results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3011.3 Related problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3011.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3111.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3211.6 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

    12 Row and column spaces 3312.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3312.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3312.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3412.4 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3412.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

    13 Row equivalence 3513.1 Elementary row operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3513.2 Row space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3513.3 Equivalence of the denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3613.4 Additional properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3613.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3613.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3613.7 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

    14 Row space 3814.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3914.2 Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3914.3 Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4014.4 Relation to the null space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4014.5 Relation to coimage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4014.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4014.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

    14.7.1 Textbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4114.8 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

    15 Row vector 4215.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4215.2 Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4215.3 Preferred input vectors for matrix transformations . . . . . . . . . . . . . . . . . . . . . . . . . . 4315.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4315.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4315.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

  • iv CONTENTS

    16 Rule of Sarrus 4516.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4616.2 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4616.3 Text and image sources, contributors, and licenses . . . . . . . . . . . . . . . . . . . . . . . . . . 47

    16.3.1 Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4716.3.2 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4816.3.3 Content license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

  • Chapter 1

    Quotient space (linear algebra)

    In linear algebra, the quotient of a vector space V by a subspace N is a vector space obtained by collapsing N tozero. The space obtained is called a quotient space and is denoted V/N (read V mod N or V by N).

    1.1 DenitionFormally, the construction is as follows (Halmos 1974, 21-22). Let V be a vector space over a eld K, and let Nbe a subspace of V. We dene an equivalence relation ~ on V by stating that x ~ y if x y N. That is, x is relatedto y if one can be obtained from the other by adding an element of N. From this denition, one can deduce that anyelement of N is related to the zero vector; more precisely all the vectors in N get mapped into the equivalence classof the zero vector.The equivalence class of x is often denoted

    [x] = x + N

    since it is given by

    [x] = {x + n : n N}.

    The quotient space V/N is then dened as V/~, the set of all equivalence classes over V by ~. Scalar multiplicationand addition are dened on the equivalence classes by

    [x] = [x] for all K, and [x] + [y] = [x+y].

    It is not hard to check that these operations are well-dened (i.e. do not depend on the choice of representative).These operations turn the quotient space V/N into a vector space over K with N being the zero class, [0].The mapping that associates to v V the equivalence class [v] is known as the quotient map.

    1.2 ExamplesLet X = R2 be the standard Cartesian plane, and let Y be a line through the origin in X. Then the quotient space X/Ycan be identied with the space of all lines in X which are parallel to Y. That is to say that, the elements of the setX/Y are lines in X parallel to Y. This gives one way in which to visualize quotient spaces geometrically.Another example is the quotient of Rn by the subspace spanned by the rst m standard basis vectors. The space Rnconsists of all n-tuples of real numbers (x1,,xn). The subspace, identied with Rm, consists of all n-tuples such thatthe last n-m entries are zero: (x1,,xm,0,0,,0). Two vectors of Rn are in the same congruence class modulo the

    1

  • 2 CHAPTER 1. QUOTIENT SPACE (LINEAR ALGEBRA)

    subspace if and only if they are identical in the last nm coordinates. The quotient space Rn/ Rm is isomorphic toRnm in an obvious manner.More generally, if V is an (internal) direct sum of subspaces U andW,

    V = U W

    then the quotient space V/U is naturally isomorphic toW (Halmos 1974, Theorem 22.1).An important example of a functional quotient space is a Lp space.

    1.3 PropertiesThere is a natural epimorphism from V to the quotient space V/U given by sending x to its equivalence class [x]. Thekernel (or nullspace) of this epimorphism is the subspace U. This relationship is neatly summarized by the short exactsequence

    0! U ! V ! V /U ! 0:

    If U is a subspace of V, the dimension of V/U is called the codimension of U in V. Since a basis of V may beconstructed from a basis A of U and a basis B of V/U by adding a representative of each element of B to A, thedimension of V is the sum of the dimensions ofU and V/U. If V is nite-dimensional, it follows that the codimensionof U in V is the dierence between the dimensions of V and U (Halmos 1974, Theorem 22.2):

    codim(U) = dim(V /U) = dim(V ) dim(U):

    Let T : V W be a linear operator. The kernel of T, denoted ker(T), is the set of all x V such that Tx = 0. Thekernel is a subspace of V. The rst isomorphism theorem of linear algebra says that the quotient space V/ker(T)is isomorphic to the image of V in W. An immediate corollary, for nite-dimensional spaces, is the rank-nullitytheorem: the dimension of V is equal to the dimension of the kernel (the nullity of T) plus the dimension of theimage (the rank of T).The cokernel of a linear operator T : V W is dened to be the quotient spaceW/im(T).

    1.4 Quotient of a Banach space by a subspaceIf X is a Banach space andM is a closed subspace of X, then the quotient X/M is again a Banach space. The quotientspace is already endowed with a vector space structure by the construction of the previous section. We dene a normon X/M by

    k[x]kX/M = infm2M

    kxmkX :

    The quotient space X/M is complete with respect to the norm, so it is a Banach space.

    1.4.1 Examples

    Let C[0,1] denote the Banach space of continuous real-valued functions on the interval [0,1] with the sup norm.Denote the subspace of all functions f C[0,1] with f(0) = 0 byM. Then the equivalence class of some function g isdetermined by its value at 0, and the quotient space C[0,1] / M is isomorphic to R.If X is a Hilbert space, then the quotient space X/M is isomorphic to the orthogonal complement of M.

  • 1.5. SEE ALSO 3

    1.4.2 Generalization to locally convex spacesThe quotient of a locally convex space by a closed subspace is again locally convex (Dieudonn 1970, 12.14.8).Indeed, suppose that X is locally convex so that the topology on X is generated by a family of seminorms {p | A} where A is an index set. Let M be a closed subspace, and dene seminorms q by on X/M

    q([x]) = infx2[x]

    p(x):

    Then X/M is a locally convex space, and the topology on it is the quotient topology.If, furthermore, X is metrizable, then so is X/M. If X is a Frchet space, then so is X/M (Dieudonn 1970, 12.11.3).

    1.5 See also quotient set quotient group quotient module quotient space (topology)

    1.6 References Halmos, Paul (1974), Finite dimensional vector spaces, Springer, ISBN 978-0-387-90093-3. Dieudonn, Jean (1970), Treatise on analysis, Volume II, Academic Press.

  • Chapter 2

    Rank (linear algebra)

    In linear algebra, the rank of a matrix A is the dimension of the vector space generated (or spanned) by its columns.[1]This is the same as the dimension of the space spanned by its rows.[2] It is a measure of the "nondegenerateness" ofthe system of linear equations and linear transformation encoded by A. There are multiple equivalent denitions ofrank. A matrixs rank is one of its most fundamental characteristics.The rank is commonly denoted rank(A) or rk(A); sometimes the parentheses are unwritten, as in rank A.

    2.1 Main denitionsIn this section we give some denitions of the rank of a matrix. Many denitions are possible; see Alternativedenitions below for several of these.The column rank of A is the dimension of the column space of A, while the row rank of A is the dimension of therow space of A.A fundamental result in linear algebra is that the column rank and the row rank are always equal. (Two proofs ofthis result are given in Proofs that column rank = row rank below.) This number (i.e., the number of linearlyindependent rows or columns) is simply called the rank of A.A matrix is said to have full rank if its rank equals the largest possible for a matrix of the same dimensions, whichis the lesser of the number of rows and columns. A matrix is said to be rank decient if it does not have full rank.The rank is also the dimension of the image of the linear transformation that is given by multiplication by A. Moregenerally, if a linear operator on a vector space (possibly innite-dimensional) has nite-dimensional image (e.g., anite-rank operator), then the rank of the operator is dened as the dimension of the image.

    2.2 ExamplesThe matrix

    24 1 2 12 3 13 5 0

    35has rank 2: the rst two rows are linearly independent, so the rank is at least 2, but all three rows are linearly dependent(the rst is equal to the sum of the second and third) so the rank must be less than 3.The matrix

    A =

    1 1 0 21 1 0 2

    4

  • 2.3. COMPUTING THE RANK OF A MATRIX 5

    has rank 1: there are nonzero columns, so the rank is positive, but any pair of columns is linearly dependent. Similarly,the transpose

    AT =

    26641 11 10 02 2

    3775of A has rank 1. Indeed, since the column vectors of A are the row vectors of the transpose of A, the statement thatthe column rank of a matrix equals its row rank is equivalent to the statement that the rank of a matrix is equal to therank of its transpose, i.e., rk(A) = rk(AT).

    2.3 Computing the rank of a matrix

    2.3.1 Rank from row echelon formsMain article: Gaussian elimination

    A common approach to nding the rank of a matrix is to reduce it to a simpler form, generally row echelon form, byelementary row operations. Row operations do not change the row space (hence do not change the row rank), and,being invertible, map the column space to an isomorphic space (hence do not change the column rank). Once in rowechelon form, the rank is clearly the same for both row rank and column rank, and equals the number of pivots (orbasic columns) and also the number of non-zero rows.For example, the matrix A given by

    A =

    24 1 2 12 3 13 5 0

    35can be put in reduced row-echelon form by using the following elementary row operations:

    24 1 2 12 3 13 5 0

    35R2 ! 2r1+r2241 2 10 1 33 5 0

    35R3 ! 3r1+r3241 2 10 1 30 1 3

    35R3 ! r2+r3241 2 10 1 30 0 0

    35R1 ! 2r2+r1241 0 50 1 30 0 0

    35The nal matrix (in reduced row echelon form) has two non-zero rows and thus the rank of matrix A is 2.

    2.3.2 ComputationWhen applied to oating point computations on computers, basic Gaussian elimination (LU decomposition) can beunreliable, and a rank-revealing decomposition should be used instead. An eective alternative is the singular valuedecomposition (SVD), but there are other less expensive choices, such as QR decomposition with pivoting (so-calledrank-revealing QR factorization), which are still more numerically robust than Gaussian elimination. Numericaldetermination of rank requires a criterion for deciding when a value, such as a singular value from the SVD, shouldbe treated as zero, a practical choice which depends on both the matrix and the application.

    2.4 Proofs that column rank = row rankThe fact that the column and row ranks of anymatrix are equal forms an important part of the fundamental theorem oflinear algebra. We present two proofs of this result. The rst is short, uses only basic properties of linear combinationsof vectors, and is valid over any eld. The proof is based uponWardlaw (2005).[3] The second is an elegant argumentusing orthogonality and is valid for matrices over the real numbers; it is based upon Mackiw (1995).[2]

  • 6 CHAPTER 2. RANK (LINEAR ALGEBRA)

    2.4.1 First proofLet A be a matrix of size m n (with m rows and n columns). Let the column rank of A be r and let c1,...,cr beany basis for the column space of A. Place these as the columns of an m r matrix C. Every column of A can beexpressed as a linear combination of the r columns in C. This means that there is an r nmatrix R such that A = CR.R is the matrix whose i-th column is formed from the coecients giving the i-th column of A as a linear combinationof the r columns of C. Now, each row of A is given by a linear combination of the r rows of R. Therefore, the rows ofR form a spanning set of the row space of A and, by the Steinitz exchange lemma, the row rank of A cannot exceedr. This proves that the row rank of A is less than or equal to the column rank of A. This result can be applied to anymatrix, so apply the result to the transpose of A. Since the row rank of the transpose of A is the column rank of Aand the column rank of the transpose of A is the row rank of A, this establishes the reverse inequality and we obtainthe equality of the row rank and the column rank of A. (Also see rank factorization.)

    2.4.2 Second proofLet A be an m n matrix with entries in the real numbers whose row rank is r. Therefore, the dimension of the rowspace of A is r. Let x1; x2; : : : ; xr be a basis of the row space of A. We claim that the vectors Ax1; Ax2; : : : ; Axrare linearly independent. To see why, consider a linear homogeneous relation involving these vectors with scalarcoecients c1; c2; : : : ; cr :

    0 = c1Ax1 + c2Ax2 + + crAxr = A(c1x1 + c2x2 + + crxr) = Av;

    where v = c1x1+ c2x2+ + crxr . We make two observations: (a) v is a linear combination of vectors in the rowspace of A, which implies that v belongs to the row space of A, and (b) since A v = 0, the vector v is orthogonal toevery row vector of A and, hence, is orthogonal to every vector in the row space of A. The facts (a) and (b) togetherimply that v is orthogonal to itself, which proves that v = 0 or, by the denition of v,

    c1x1 + c2x2 + + crxr = 0:

    But recall that the xi were chosen as a basis of the row space of A and so are linearly independent. This implies thatc1 = c2 = = cr = 0 . It follows that Ax1; Ax2; : : : ; Axr are linearly independent.Now, each Axi is obviously a vector in the column space of A. So, Ax1; Ax2; : : : ; Axr is a set of r linearly inde-pendent vectors in the column space of A and, hence, the dimension of the column space of A (i.e., the column rankof A) must be at least as big as r. This proves that row rank of A is no larger than the column rank of A. Now applythis result to the transpose of A to get the reverse inequality and conclude as in the previous proof.

    2.5 Alternative denitionsIn all the denitions in this section, the matrix A is taken to be an m n matrix over an arbitrary eld F.

    Dimension of image

    Given the matrix A, there is an associated linear mapping

    f : Fn Fm

    dened by

    f(x) = Ax.

    The rank of A is the dimension of the image of f. This denition has the advantage that it can be applied to any linearmap without need for a specic matrix.

  • 2.5. ALTERNATIVE DEFINITIONS 7

    Rank in terms of nullity

    Given the same linear mapping f as above, the rank is n minus the dimension of the kernel of f. The ranknullitytheorem states that this denition is equivalent to the preceding one.

    Column rank dimension of column space

    The rank of A is the maximal number of linearly independent columns c1; c2; : : : ; ck of A; this is the dimension ofthe column space of A (the column space being the subspace of Fm generated by the columns of A, which is in factjust the image of the linear map f associated to A).

    Row rank dimension of row space

    The rank of A is the maximal number of linearly independent rows of A; this is the dimension of the row space of A.

    Decomposition rank

    The rank of A is the smallest integer k such that A can be factored as A = CR , where C is an m k matrix and R isa k n matrix. In fact, for all integers k, the following are equivalent:

    1. the column rank of A is less than or equal to k,2. there exist k columns c1; : : : ; ck of size m such that every column of A is a linear combination of c1; : : : ; ck ,3. there exist an m k matrix C and a k n matrix R such that A = CR (when k is the rank, this is a rank

    factorization of A),4. there exist k rows r1; : : : ; rk of size n such that every row of A is a linear combination of r1; : : : ; rk ,5. the row rank of A is less than or equal to k.

    Indeed, the following equivalences are obvious: (1) , (2) , (3) , (4) , (5) . For example, to prove (3) from(2), take C to be the matrix whose columns are c1; : : : ; ck from (2). To prove (2) from (3), take c1; : : : ; ck to be thecolumns of C.It follows from the equivalence (1), (5) that the row rank is equal to the column rank.As in the case of the dimension of image characterization, this can be generalized to a denition of the rank ofany linear map: the rank of a linear map f : V W is the minimal dimension k of an intermediate space X suchthat f can be written as the composition of a map V X and a map X W. Unfortunately, this denition does notsuggest an ecient manner to compute the rank (for which it is better to use one of the alternative denitions). Seerank factorization for details.

    Determinantal rank size of largest non-vanishing minor

    The rank of A is the largest order of any non-zero minor in A. (The order of a minor is the side-length of the squaresub-matrix of which it is the determinant.) Like the decomposition rank characterization, this does not give anecient way of computing the rank, but it is useful theoretically: a single non-zero minor witnesses a lower bound(namely its order) for the rank of the matrix, which can be useful (for example) to prove that certain operations donot lower the rank of a matrix.A non-vanishing p-minor (p p submatrix with non-zero determinant) shows that the rows and columns of thatsubmatrix are linearly independent, and thus those rows and columns of the full matrix are linearly independent (inthe full matrix), so the row and column rank are at least as large as the determinantal rank; however, the converse isless straightforward. The equivalence of determinantal rank and column rank is a strengthening of the statement thatif the span of n vectors has dimension p, then p of those vectors span the space (equivalently, that one can choosea spanning set that is a subset of the vectors): the equivalence implies that a subset of the rows and a subset of thecolumns simultaneously dene an invertible submatrix (equivalently, if the span of n vectors has dimension p, then pof these vectors span the space and there is a set of p coordinates on which they are linearly independent).

  • 8 CHAPTER 2. RANK (LINEAR ALGEBRA)

    Tensor rank minimum number of simple tensors

    Main articles: Tensor rank decomposition and Tensor rank

    The rank of A is the smallest number k such that A can be written as a sum of k rank 1 matrices, where a matrixis dened to have rank 1 if and only if it can be written as a nonzero product c r of a column vector c and a rowvector r. This notion of rank is called tensor rank; it can be generalized in the separable models interpretation of thesingular value decomposition.

    2.6 PropertiesWe assume that A is an m n matrix, and we dene the linear map f by f(x) = Ax as above.

    The rank of an m nmatrix is a nonnegative integer and cannot be greater than either m or n. That is, rk(A) min(m, n). A matrix that has rank min(m, n) is said to have full rank; otherwise, the matrix is rank decient.

    Only a zero matrix has rank zero. f is injective if and only if A has rank n (in this case, we say that A has full column rank). f is surjective if and only if A has rank m (in this case, we say that A has full row rank). If A is a square matrix (i.e., m = n), then A is invertible if and only if A has rank n (that is, A has full rank). If B is any n k matrix, then

    rank(AB) min(rank A; rank B):

    If B is an n k matrix of rank n, then

    rank(AB) = rank(A):

    If C is an l m matrix of rank m, then

    rank(CA) = rank(A):

    The rank of A is equal to r if and only if there exists an invertible m m matrix X and an invertible n nmatrix Y such that

    XAY =

    Ir 00 0

    ;

    where Ir denotes the r r identity matrix.

    Sylvesters rank inequality: if A is an m n matrix and B is n k, then

    rank(A) + rank(B) n rank(AB): [lower-roman 1]This is a special case of the next inequality.

  • 2.7. APPLICATIONS 9

    The inequality due to Frobenius: if AB, ABC and BC are dened, then

    rank(AB) + rank(BC) rank(B) + rank(ABC): [lower-roman 2]

    Subadditivity: rank(A + B) rank(A) + rank(B) when A and B are of the same dimension. As a consequence,a rank-k matrix can be written as the sum of k rank-1 matrices, but not fewer.

    The rank of a matrix plus the nullity of the matrix equals the number of columns of the matrix. (This is theranknullity theorem.)

    If A is a matrix over the real numbers then the rank of A and the rank of its corresponding Gram matrix areequal. Thus, for real matrices

    rank(ATA) = rank(AAT) = rank(A) = rank(AT)

    This can be shown by proving equality of their null spaces. Null space of the Gram matrix is given byvectors x for which ATAx = 0 . If this condition is fullled, also holds 0 = xTATAx = jAxj2 .[4]

    If A is a matrix over the complex numbers and A denotes the conjugate transpose of A (i.e., the adjoint of A),then

    rank(A) = rank(A) = rank(AT) = rank(A) = rank(AA):

    2.7 ApplicationsOne useful application of calculating the rank of a matrix is the computation of the number of solutions of a system oflinear equations. According to the RouchCapelli theorem, the system is inconsistent if the rank of the augmentedmatrix is greater than the rank of the coecient matrix. If, on the other hand, the ranks of these two matrices areequal, then the systemmust have at least one solution. The solution is unique if and only if the rank equals the numberof variables. Otherwise the general solution has k free parameters where k is the dierence between the number ofvariables and the rank. In this case (and assuming the system of equations is in the real or complex numbers) thesystem of equations has innitely many solutions.In control theory, the rank of a matrix can be used to determine whether a linear system is controllable, or observable.In the eld of communication complexity, the rank of the communication matrix of a function gives bounds on theamount of communication needed for two parties to compute the function.

    2.8 GeneralizationThere are dierent generalisations of the concept of rank to matrices over arbitrary rings. In those generalisations,column rank, row rank, dimension of column space and dimension of row space of a matrix may be dierent fromthe others or may not exist.Thinking of matrices as tensors, the tensor rank generalizes to arbitrary tensors; note that for tensors of order greaterthan 2 (matrices are order 2 tensors), rank is very hard to compute, unlike for matrices.There is a notion of rank for smooth maps between smooth manifolds. It is equal to the linear rank of the derivative.

    2.9 Matrices as tensorsMatrix rank should not be confused with tensor order, which is called tensor rank. Tensor order is the number ofindices required to write a tensor, and thus matrices all have tensor order 2. More precisely, matrices are tensors of

  • 10 CHAPTER 2. RANK (LINEAR ALGEBRA)

    type (1,1), having one row index and one column index, also called covariant order 1 and contravariant order 1; seeTensor (intrinsic denition) for details.Note that the tensor rank of a matrix can also mean the minimum number of simple tensors necessary to express thematrix as a linear combination, and that this denition does agree with matrix rank as here discussed.

    2.10 See also Matroid rank Nonnegative rank (linear algebra) Rank (dierential topology) Multicollinearity Linear dependence

    2.11 Notes[1] Proof: Apply the ranknullity theorem to the inequality

    dim ker(AB) dim ker(A) + dim ker(B)

    [2] Proof: The map

    C : ker(ABC)/ ker(BC)! ker(AB)/ ker(B)is well-dened and injective. We thus obtain the inequality in terms of dimensions of kernel, which can then be convertedto the inequality in terms of ranks by the ranknullity theorem. Alternatively, if M is a linear subspace then dim(AM) dim(M); apply this inequality to the subspace dened by the (orthogonal) complement of the image of BC in the image ofB, whose dimension is rk(B) rk(BC); its image under A has dimension rk(AB) rk(ABC).

    2.12 References[1] Bourbaki, Algebra, ch. II, 10.12, p. 359

    [2] Mackiw, G. (1995), A Note on the Equality of the Column and Row Rank of a Matrix, Mathematics Magazine 68 (4)

    [3] Wardlaw, William P. (2005), Row Rank Equals Column Rank, Mathematics Magazine 78 (4)

    [4] Mirsky, Leonid (1955). An introduction to linear algebra. Dover Publications. ISBN 978-0-486-66434-7.

    2.13 Further reading Roger A. Horn and Charles R. Johnson (1985). Matrix Analysis. ISBN 978-0-521-38632-6. Kaw, Autar K. Two Chapters from the book Introduction to Matrix Algebra: 1. Vectors and System ofEquations

    Mike Brookes: Matrix Reference Manual.

  • Chapter 3

    Rank factorization

    Given an m n matrix A of rank r , a rank decomposition or rank factorization of A is a product A = CF ,where C is anm r matrix and F is an r n matrix.Every nite-dimensional matrix has a rank decomposition: Let A be anm n matrix whose column rank is r. Therefore, there are r linearly independent columns in A ; equivalently, the dimension of the column space of A isr . Let c1; c2; : : : ; cr be any basis for the column space of A and place them as column vectors to form the m rmatrix C = [c1 : c2 : : : : : cr] . Therefore, every column vector of A is a linear combination of the columns of C .To be precise, if A = [a1 : a2 : : : : : an] is anm n matrix with aj as the j -th column, then

    aj = f1jc1 + f2jc2 + + frjcr;

    where fij 's are the scalar coecients of aj in terms of the basis c1; c2; : : : ; cr . This implies that A = CF , wherefij is the (i; j) -th element of F .

    3.1 rank(A) = rank(AT)An immediate consequence of rank factorization is that the rank of A is equal to the rank of its transpose AT . Sincethe columns of A are the rows of AT , the column rank of A equals its row rank.Proof: To see why this is true, let us rst dene rank to mean column rank. Since A = CF , it follows thatAT = F TCT . From the denition of matrix multiplication, this means that each column ofAT is a linear combinationof the columns of F T . Therefore, the column space of AT is contained within the column space of F T and, hence,rank( AT ) rank( F T ). Now, F T is n r , so there are r columns in F T and, hence, rank( AT ) r = rank( A ).This proves that rank( AT) rank( A ). Now apply the result to AT to obtain the reverse inequality: since (AT)T =A , we can write rank( A ) = rank( (AT)T) rank( AT ). This proves rank( A) rank( AT ). We have, therefore,proved rank( AT) rank( A ) and rank( A ) rank( AT ), so rank( A ) = rank( AT ). (Also see the rst proof ofcolumn rank = row rank under rank).

    3.2 Rank factorization from row echelon formsIn practice, we can construct one specic rank factorization as follows: we can computeB , the reduced row echelonform of A . Then C is obtained by removing from A all non-pivot columns, and F by eliminating all zero rows of B.

    3.3 ExampleConsider the matrix

    11

  • 12 CHAPTER 3. RANK FACTORIZATION

    A =

    26641 3 1 42 7 3 91 5 3 11 2 0 8

    3775 26641 0 2 00 1 1 00 0 0 10 0 0 0

    3775 = B.B is in reduced echelon form. Then C is obtained by removing the third column of A , the only one which is not apivot column, and F by getting rid of the last row of zeroes, so

    C =

    26641 3 42 7 91 5 11 2 8

    3775, F =241 0 2 00 1 1 00 0 0 1

    35.It is straightforward to check that

    A =

    26641 3 1 42 7 3 91 5 3 11 2 0 8

    3775 =26641 3 42 7 91 5 11 2 8

    3775241 0 2 00 1 1 00 0 0 1

    35 = CF .

    3.4 ProofLet P be an n n permutation matrix such that AP = (C;D) in block partitioned form, where the columns of Care the r pivot columns of A . Every column ofD is a linear combination of the columns of C , so there is a matrixG such that D = CG , where the columns of G contain the coecients of each of those linear combinations. SoAP = (C;CG) = C(Ir; G) , Ir being the r r identity matrix. We will show now that (Ir; G) = FP .TransformingAP into its reduced row echelon form amounts to left-multiplying by a matrix E which is a product ofelementary matrices, so EAP = BP = EC(Ir; G) , where EC =

    Ir0

    . We then can write BP =

    Ir G0 0

    , which allows us to identify (Ir; G) = FP , i.e. the nonzero r rows of the reduced echelon form, with the samepermutation on the columns as we did for A . We thus have AP = CFP , and since P is invertible this impliesA = CF , and the proof is complete.

    3.5 References Lay, David C. (2005), Linear Algebra and its Applications (3rd ed.), AddisonWesley, ISBN 978-0-201-70970-4

    Golub, Gene H.; Van Loan, Charles F. (1996), Matrix Computations, Johns Hopkins Studies in MathematicalSciences (3rd ed.), The Johns Hopkins University Press, ISBN 978-0-8018-5414-9

    Stewart, Gilbert W. (1998), Matrix Algorithms. I. Basic Decompositions, SIAM, ISBN 978-0-89871-414-2

  • Chapter 4

    Rayleigh quotient

    In mathematics, for a given complex Hermitian matrixM and nonzero vector x, the Rayleigh quotient[1] R(M;x) ,is dened as:[2][3]

    R(M;x) :=xMxxx

    :

    For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the conjugatetranspose x to the usual transpose x0 . Note that R(M; cx) = R(M;x) for any non-zero real scalar c. Recall thata Hermitian (or real symmetric) matrix has real eigenvalues. It can be shown that, for a given matrix, the Rayleighquotient reaches its minimum valuemin (the smallest eigenvalue ofM) when x is vmin (the corresponding eigenvector).Similarly, R(M;x) max and R(M; vmax) = max .The Rayleigh quotient is used in the min-max theorem to get exact values of all eigenvalues. It is also used ineigenvalue algorithms to obtain an eigenvalue approximation from an eigenvector approximation. Specically, this isthe basis for Rayleigh quotient iteration.The range of the Rayleigh quotient (for matrix that is not necessarily Hermitian) is called a numerical range, (orspectrum in functional analysis). When the matrix is Hermitian, the numerical range is equal to the spectral norm.Still in functional analysis, max is known as the spectral radius. In the context of C*-algebras or algebraic quantummechanics, the function that to M associates the Rayleigh-Ritz quotient R(M,x) for a xed x and M varying throughthe algebra would be referred to as vector state of the algebra.

    4.1 Bounds for HermitianM

    As stated in introduction R(M;x) 2 [min; max] . This is immediate after observing that the Rayleigh quotient is aweighted average of eigenvalues of M:

    R(M;x) =xMxxx

    =

    Pni=1 iy

    2iPn

    i=1 y2i

    where (i; vi) is the i th eigenpair after orthonormalization and yi = vi x is the i th coordinate of x in the eigenbasis.It is then easy to verify that the bounds are attained at the corresponding eigenvectors vmin; vmax .The fact that the quotient is a weighted average of the eigenvalues can be used to identify the second, the third, ...largest eigenvalues. Let max = 1 2 ::: n = min be the eigenvalues in decreasing order. If x isconstrained to be orthogonal to v1 , in which case y1 = v1x = 0 , then R(M;x) has the maximum 2 , which isachieved when x = v2 .

    13

  • 14 CHAPTER 4. RAYLEIGH QUOTIENT

    4.2 Special case of covariance matricesAn empirical covariance matrix M can be represented as the product A' A of the data matrix A pre-multiplied by itstranspose A'. Being a positive semi-denite matrix,M has non-negative eigenvalues, and orthogonal (or othogonalis-able) eigenvectors, which can be demonstrated as follows.Firstly, that the eigenvalues i are non-negative:

    Mvi = A0Avi = ivi

    ) v0iA0Avi = v0iivi) kAvik2 = i kvik2

    ) i = kAvik2

    kvik2 0:

    Secondly, that the eigenvectors vi are orthogonal to one another:

    Mvi = ivi

    ) v0jMvi = iv0jvi) (Mvj)0 vi = jv0jvi) jv0jvi = iv0jvi) (j i) v0jvi = 0) v0jvi = 0

    If the eigenvalues are dierent in the case of multiplicity, the basis can be orthogonalized.To now establish that the Rayleigh quotient is maximised by the eigenvector with the largest eigenvalue, considerdecomposing an arbitrary vector x on the basis of the eigenvectors vi:

    x =

    nXi=1

    ivi;

    where

    i =x0viv0ivi

    =hx; viikvik2

    is the coordinate of x orthogonally projected onto vi. Therefore we have:

    R(M;x) =x0A0Axx0x

    =

    Pnj=1 jvj

    0(A0A) (

    Pni=1 ivi)Pn

    j=1 jvj

    0(Pn

    i=1 ivi)

    which, by orthogonality of the eigenvectors, becomes:

    R(M;x) =

    Pni=1

    2iiPn

    i=1 2i

    =nXi=1

    i(x0vi)2

    (x0x)(v0ivi)

    The last representation establishes that the Rayleigh quotient is the sum of the squared cosines of the angles formedby the vector x and each eigenvector vi, weighted by corresponding eigenvalues.

  • 4.3. USE IN STURMLIOUVILLE THEORY 15

    If a vector x maximizes R(M;x) , then any non-zero scalar multiple kx also maximizes R, so the problem can bereduced to the Lagrange problem of maximizingPni=1 2ii under the constraint thatPni=1 2i = 1 .Dene: i = 2i. This then becomes a linear program, which always attains its maximum at one of the corners of the domain. Amaximum point will have 1 = 1 and i = 0 for all i > 1 (when the eigenvalues are ordered by decreasingmagnitude).Thus, as advertised, the Rayleigh quotient is maximised by the eigenvector with the largest eigenvalue.

    4.2.1 Formulation using Lagrange multipliersAlternatively, this result can be arrived at by the method of Lagrange multipliers. The problem is to nd the criticalpoints of the function

    R(M;x) = xTMx

    subject to the constraint kxk2 = xTx = 1: I.e. to nd the critical points of

    L(x) = xTMx xTx 1 ;where is a Lagrange multiplier. The stationary points of L(x) occur at

    dL(x)dx

    = 0

    ) 2xTMT 2xT = 0)Mx = x

    and

    R(M;x) =xTMx

    xTx=

    xTx

    xTx= :

    Therefore, the eigenvectors x1; ; xn ofM are the critical points of the Rayleigh Quotient and their correspondingeigenvalues 1; ; n are the stationary values of R.This property is the basis for principal components analysis and canonical correlation.

    4.3 Use in SturmLiouville theorySturmLiouville theory concerns the action of the linear operator

    L(y) =1

    w(x)

    ddx

    p(x)

    dy

    dx

    + q(x)y

    on the inner product space dened by

    hy1; y2i =Z ba

    w(x)y1(x)y2(x) dx

    of functions satisfying some specied boundary conditions at a and b. In this case the Rayleigh quotient is

  • 16 CHAPTER 4. RAYLEIGH QUOTIENT

    hy; Lyihy; yi =

    R bay(x)

    ddx

    hp(x) dydx

    i+ q(x)y(x)

    dxR b

    aw(x)y(x)2dx

    :

    This is sometimes presented in an equivalent form, obtained by separating the integral in the numerator and usingintegration by parts:

    hy; Lyihy; yi =

    nR bay(x)

    ddx [p(x)y0(x)] dxo+ nR ba q(x)y(x)2 dxoR baw(x)y(x)2 dx

    =

    ny(x) [p(x)y0(x)]jba

    o+nR b

    ay0(x) [p(x)y0(x)] dx

    o+nR b

    aq(x)y(x)2 dx

    oR baw(x)y(x)2 dx

    =

    np(x)y(x)y0(x)jba

    o+nR b

    a

    p(x)y0(x)2 + q(x)y(x)2

    dxo

    R baw(x)y(x)2 dx

    :

    4.4 Generalizations1. For a given pair (A, B) of matrices, and a given non-zero vector x, the generalized Rayleigh quotient is dened

    as:

    R(A;B;x) := xAxxBx :

    The Generalized Rayleigh Quotient can be reduced to the Rayleigh Quotient R(D;Cx) throughthe transformationD = C1AC1 where CC is the Cholesky decomposition of the Hermitianpositive-denite matrix B.

    2. For a given pair (x, y) of non-zero vectors, and a given Hermitian matrixH, the generalized Rayleigh quotientcan be dened as:

    R(H;x; y) := yHxp

    yyxxwhich coincides with R(H,x) when x=y.

    4.5 See also Field of values Min-max theorem

    4.6 References[1] Also known as the RayleighRitz ratio; named after Walther Ritz and Lord Rayleigh.

    [2] Horn, R. A. and C. A. Johnson. 1985. Matrix Analysis. Cambridge University Press. pp. 176180.

    [3] Parlet B. N. The symmetric eigenvalue problem, SIAM, Classics in Applied Mathematics,1998

    4.7 Further reading Shi Yu, Lon-Charles Tranchevent, BartMoor, YvesMoreau,Kernel-based Data Fusion forMachine Learning:Methods and Applications in Bioinformatics and Text Mining, Ch. 2, Springer, 2011.

  • Chapter 5

    2 2 real matrices

    In mathematics, the set of 22 real matrices is denoted by M(2, R). Two matrices p and q in M(2, R) have a sum p+ q given by matrix addition. The product matrix p q is formed from the dot product of the rows and columns of itsfactors through matrix multiplication. For

    q =

    a bc d

    ;

    let

    q =d bc a

    :

    Then q q* = q* q = (ad bc) I, where I is the 22 identity matrix. The real number ad bc is called the determinantof q. When ad bc 0, q is an invertible matrix, and then

    q1 = q / (ad bc):The collection of all such invertible matrices constitutes the general linear group GL(2, R). In terms of abstractalgebra, M(2, R) with the associated addition and multiplication operations forms a ring, and GL(2, R) is its group ofunits. M(2, R) is also a four-dimensional vector space, so it is considered an associative algebra. It is ring-isomorphicto the coquaternions, but has a dierent prole.The 22 real matrices are in one-one correspondence with the linear mappings of the two-dimensional Cartesiancoordinate system into itself by the rule

    xy

    7!a bc d

    xy

    =

    ax+ bycx+ dy

    :

    5.1 ProleWithin M(2, R), the multiples by real numbers of the identity matrix I may be considered a real line. This real lineis the place where all commutative subrings come together:Let Pm = {xI + ym : x, y R} where m2 { I, 0, I }. Then Pm is a commutative subring and M(2, R) = Pmwhere the union is over all m such that m2 { I, 0, I }.To identify such m, rst square the generic matrix:

    aa+ bc ab+ bdac+ cd bc+ dd

    :

    17

  • 18 CHAPTER 5. 2 2 REAL MATRICES

    When a + d = 0 this square is a diagonal matrix. Thus one assumes d = a when looking for m to form commutativesubrings. Whenmm = I, then bc = 1 aa, an equation describing a hyperbolic paraboloid in the space of parameters(a, b, c). Such an m serves as an imaginary unit. In this case Pm is isomorphic to the eld of (ordinary) complexnumbers.When mm = +I, m is an involutory matrix. Then bc = +1 aa, also giving a hyperbolic paraboloid. If a matrix is anidempotent matrix, it must lie in such a Pm and in this case Pm is isomorphic to the ring of split-complex numbers.The case of a nilpotent matrix, mm = 0, arises when only one of b or c is non-zero, and the commutative subring Pmis then a copy of the dual number plane.When M(2, R) is recongured with a change of basis, this prole changes to the prole of split-quaternions wherethe sets of square roots of I and I take a symmetrical shape as hyperboloids.

    5.2 Equi-areal mappingMain article: Equiareal map

    First transform one dierential vector into another:

    dudv

    =

    p rq s

    dxdy

    =

    p dx+ r dyq dx+ s dy

    :

    Areas are measured with density dx ^ dy , a dierential 2-form which involves the use of exterior algebra. Thetransformed density is

    du ^ dv = 0 + ps dx ^ dy + qr dy ^ dx+ 0= (ps qr) dx ^ dy = (det g) dx ^ dy:

    Thus the equi-areal mappings are identied with SL(2, R) = {g M(2, R) : det(g) = 1}, the special linear group. Giventhe prole above, every such g lies in a commutative subring Pm representing a type of complex plane according tothe square of m. Since g g* = I, one of the following three alternatives occurs:

    mm = I and g is on a circle of Euclidean rotations; or mm = I and g is on an hyperbola of squeeze mappings; or mm = 0 and g is on a line of shear mappings.

    Writing about planar ane mapping, Rafael Artzy made a similar trichotomy of planar, linear mapping in his bookLinear Geometry (1965).

    5.3 Functions of 2 2 real matricesThe commutative subrings of M(2, R) determine the function theory; in particular the three types of subplanes havetheir own algebraic structures which set the value of algebraic expressions. Consideration of the square root functionand the logarithm function serves to illustrate the constraints implied by the special properties of each type of subplanePm described in the above prole. The concept of identity component of the group of units of Pm leads to the polardecomposition of elements of the group of units:

    If mm = I, then z = exp(m). If mm = 0, then z = exp(s m) or z = exp(s m). If mm = I, then z = exp(a m) or z = exp(a m) or z = m exp(a m) or z = m exp(a m).

  • 5.4. 2 2 REAL MATRICES AS COMPLEX NUMBERS 19

    In the rst case exp( m) = cos() + m sin(). In the case of the dual numbers exp(s m) = 1 + s m. Finally, in the caseof split complex numbers there are four components in the group of units. The identity component is parameterizedby and exp(a m) = cosh a + m sinh a.Now

    p exp(am) = p exp(am/2) regardless of the subplane Pm, but the argument of the function must be

    taken from the identity component of its group of units. Half the plane is lost in the case of the dual number structure;three-quarters of the plane must be excluded in the case of the split-complex number structure.Similarly, if exp(a m) is an element of the identity component of the group of units of a plane associated with 22matrix m, then the logarithm function results in a value log + a m. The domain of the logarithm function suers thesame constraints as does the square root function described above: half or three-quarters of Pm must be excluded inthe cases mm = 0 or mm = I.Further function theory can be seen in the article complex functions for the C structure, or in the article motor variablefor the split-complex structure.

    5.4 2 2 real matrices as complex numbersEvery 22 real matrix can be interpreted as one of three types of (generalized[1]) complex numbers: standard complexnumbers, dual numbers, and split-complex numbers. Above, the algebra of 22 matrices is proled as a union ofcomplex planes, all sharing the same real axis. These planes are presented as commutative subrings Pm. We candetermine to which complex plane a given 22 matrix belongs as follows and classify which kind of complex numberthat plane represents.Consider the 22 matrix

    z =

    a bc d

    :

    We seek the complex plane Pm containing z.As noted above, the square of the matrix z is diagonal when a + d = 0. The matrix z must be expressed as the sumof a multiple of the identity matrix I and a matrix in the hyperplane a + d = 0. Projecting z alternately onto thesesubspaces of R4 yields

    z = xI + n; x =a+ d

    2; n = z xI:

    Furthermore,

    n2 = pI where p = (ad)24 + bc .

    Now z is one of three types of complex number:

    If p < 0, then it is an ordinary complex number:

    Let q = 1/pp; m = qn . Thenm2 = I; z = xI +mpp .

    If p = 0, then it is the dual number:

    z = xI + n

    If p > 0, then z is a split-complex number:

    Let q = 1/pp; m = qn . Thenm2 = +I; z = xI +mpp .

    Similarly, a 22 matrix can also be expressed in polar coordinates with the caveat that there are two connectedcomponents of the group of units in the dual number plane, and four components in the split-complex number plane.

  • 20 CHAPTER 5. 2 2 REAL MATRICES

    5.5 References[1] Anthony A. Harkin & Joseph B. Harkin (2004) Geometry of Generalized Complex Numbers, Mathematics Magazine

    77(2):11829

    Rafael Artzy (1965) Linear Geometry, Chapter 2-6 Subgroups of the Plane Ane Group over the Real Field,p. 94, Addison-Wesley.

    Helmut Karzel & Gunter Kist (1985) Kinematic Algebras and their Geometries, found in Rings and Geometry, R. Kaya, P. Plaumann, and K. Strambach editors, pp. 437509, esp 449,50, D.Reidel ISBN 90-277-2112-2 .

    Svetlana Katok (1992) Fuchsian groups, pp. 113, University of Chicago Press ISBN 0-226-42582-7 . Garret Sobczyk (2012). Chapter 2: Complex and Hyperbolic Numbers. New Foundations in Mathematics:The Geometric Concept of Number. Birkhuser. ISBN 978-0-8176-8384-9.

  • Chapter 6

    Reality structure

    In mathematics, a reality structure on a complex vector space V is a decomposition of V into two real subspaces,called the real and imaginary parts of V :

    V = VR iVR:

    Here VR is a real subspace of V, i.e. a subspace of V considered as a vector space over the real numbers. If V hascomplex dimension n (real dimension 2n), then VR must have real dimension n.The standard reality structure on the vector space Cn is the decomposition

    Cn = Rn iRn:

    In the presence of a reality structure, every vector in V has a real part and an imaginary part, each of which is a vectorin VR:

    v = Refvg+ i Imfvg

    In this case, the complex conjugate of a vector v is dened as follows:

    v = Refvg i Imfvg

    This map v 7! v is an antilinear involution, i.e.

    v = v; v + w = v + w; and v = v:

    Conversely, given an antilinear involution v 7! c(v) on a complex vector space V, it is possible to dene a realitystructure on V as follows. Let

    Refvg = 12(v + c(v)) ;

    and dene

    VR = fRefvg j v 2 V g :

    Then

    21

  • 22 CHAPTER 6. REALITY STRUCTURE

    V = VR iVR:

    This is actually the decomposition of V as the eigenspaces of the real linear operator c. The eigenvalues of c are+1 and 1, with eigenspaces VR and i VR, respectively. Typically, the operator c itself, rather than the eigenspacedecomposition it entails, is referred to as the reality structure on V.

    6.1 See also Linear complex structure Complexication

    6.2 References Penrose, Roger; Rindler, Wolfgang (1986), Spinors and space-time. Vol. 2, Cambridge Monographs on Math-ematical Physics, Cambridge University Press, ISBN 978-0-521-25267-6, MR 838301

  • Chapter 7

    Reduction (mathematics)

    For other uses, see Reduction (disambiguation).

    In mathematics, reduction refers to the rewriting of an expression into a simpler form. For example, the process ofrewriting a fraction into one with the smallest whole-number denominator possible (while keeping the numerator aninteger) is called reducing a fraction. Rewriting a radical (or root) expression with the smallest possible wholenumber under the radical symbol is called reducing a radical. Minimizing the number of radicals that appearunderneath other radicals in an expression is called denesting radicals.

    7.1 AlgebraIn linear algebra, reduction refers to applying simple rules to a series of equations or matrices to change them intoa simpler form. In the case of matrices, the process involves manipulating either the rows or the columns of thematrix and so is usually referred to as row-reduction or column-reduction, respectively. Often the aim of reductionis to transform a matrix into its row-reduced echelon form" or row-echelon form"; this is the goal of Gaussianelimination.

    7.2 CalculusIn calculus, reduction refers to using the technique of integration by parts to evaluate a whole class of integrals byreducing them to simpler forms.

    7.3 Static (Guyan) ReductionIn dynamic analysis, static reduction refers to reducing the number of degrees of freedom. Static reduction can also beused in FEA analysis to refer to simplication of a linear algebraic problem. Since a static reduction requires severalinversion steps it is an expensive matrix operation and is prone to some error in the solution. Consider the followingsystem of linear equations in an FEA problem:

    K11 K12K21 K22

    x1x2

    =

    F1F2

    where K and F are known and K, x and F are divided into submatrices as shown above. If F2 contains only zeros,and only x1 is desired, K can be reduced to yield the following system of equations

    K11;reduced

    x1=F1

    23

  • 24 CHAPTER 7. REDUCTION (MATHEMATICS)

    K11,reduced is obtained by writing out the set of equations as follows:

    K11x1 +K12x2 = F1 1) (Eq.

    K21x1 +K22x2 = 0: 2) (Eq.Equation (2) can be solved for x2 (assuming invertibility ofK22 ):

    K122 K21x1 = x2:

    And substituting into (1) gives

    K11x1 K12K122 K21x1 = F1:

    Thus

    K11;reduced = K11 K12K122 K21:

    In a similar fashion, any row/column i of F with a zero value may be eliminated if the corresponding value of xi isnot desired. A reduced K may be reduced again. As a note, since each reduction requires an inversion, and eachinversion is a n3 most large matrices are pre-processed to reduce calculation time.

  • Chapter 8

    Relative dimension

    In mathematics, specically linear algebra and geometry, relative dimension is the dual notion to codimension.In linear algebra, given a quotient map V ! Q , the dierence dim V dim Q is the relative dimension; this equalsthe dimension of the kernel.In ber bundles, the relative dimension of the map is the dimension of the ber.More abstractly, the codimension of a map is the dimension of the cokernel, while the relative dimension of a map isthe dimension of the kernel.These are dual in that the inclusion of a subspace V ! W of codimension k dualizes to yield a quotient mapW ! V of relative dimension k, and conversely.The additivity of codimension under intersection corresponds to the additivity of relative dimension in a ber product.Just as codimension is mostly used for injective maps, relative dimension is mostly used for surjective maps.

    25

  • Chapter 9

    Resolvent set

    In linear algebra and operator theory, the resolvent set of a linear operator is a set of complex numbers for whichthe operator is in some sense "well-behaved". The resolvent set plays an important role in the resolvent formalism.

    9.1 DenitionsLet X be a Banach space and let L : D(L) ! X be a linear operator with domain D(L) X . Let id denote theidentity operator on X. For any 2 C , let

    L = L id: is said to be a regular value if R(;L) , the inverse operator to L

    1. exists;2. is a bounded linear operator;3. is dened on a dense subspace of X.

    The resolvent set of L is the set of all regular values of L:

    (L) = f 2 Cj is a regular value of Lg:The spectrum is the complement of the resolvent set:

    (L) = C n (L):The spectrum can be further decomposed into the point/discrete spectrum (where condition 1 fails), the continu-ous spectrum (where conditions 1 and 3 hold but condition 2 fails) and the residual/compression spectrum (wherecondition 1 holds but condition 3 fails).

    9.2 Properties The resolvent set (L) C of a bounded linear operator L is an open set.

    9.3 References Renardy, Michael; Rogers, Robert C. (2004). An introduction to partial dierential equations. Texts in AppliedMathematics 13 (Second ed.). New York: Springer-Verlag. xiv+434. ISBN 0-387-00444-0. MR 2028503(See section 8.3)

    26

  • 9.4. EXTERNAL LINKS 27

    9.4 External links Voitsekhovskii, M.I. (2001), Resolvent set, in Hazewinkel, Michiel, Encyclopedia of Mathematics, Springer,ISBN 978-1-55608-010-4

    9.5 See also Resolvent formalism Spectrum (functional analysis) Decomposition of spectrum (functional analysis)

  • Chapter 10

    Restricted isometry property

    In linear algebra, the restricted isometry property characterizes matrices which are nearly orthonormal, at leastwhen operating on sparse vectors. The concept was introduced by Emmanuel Cands and Terence Tao[1] and isused to prove many theorems in the eld of compressed sensing.[2] There are no known large matrices with boundedrestricted isometry constants (and computing these constants is strongly NP-hard[3]), but many randommatrices havebeen shown to remain bounded. In particular, it has been shown that with exponentially high probability, randomGaussian, Bernoulli, and partial Fourier matrices satisfy the RIP with number of measurements nearly linear in thesparsity level.[4] The current smallest upper bounds for any large rectangular matrices are for those of Gaussianmatrices.[5] Web forms to evaluate bounds for the Gaussian ensemble are available at the Edinburgh CompressedSensing RIC page.[6]

    10.1 DenitionLet A be an m p matrix and let 1 s p be an integer. Suppose that there exists a constant s 2 (0; 1) such that,for every m s submatrix As of A and for every vector y,

    (1 s)kyk22 kAsyk22 (1 + s)kyk22:

    Then, the matrix A is said to satisfy the s-restricted isometry property with restricted isometry constant s .

    10.2 Restricted Isometric Constant (RIC)The RIP Constant is dened as the inmum of all possible for a given A 2 Rnm .

    K = inf : (1 )kyk22 kAsyk22 (1 + )kyk22

    ; 8jsj K; 8y 2 Rjsj

    It is denoted as K .

    10.3 EigenvaluesFor any matrix that satises the RIP property with a RIC of k , the following condition holds:[7]

    1 K min(AA ) max(AA ) 1 + k

    28

  • 10.4. SEE ALSO 29

    10.4 See also Compressed sensing Mutual coherence (linear algebra) Terence Taos website on compressed sensing lists several related conditions, such as the 'Exact reconstructionprinciple' (ERP) and 'Uniform uncertainty principle' (UUP)

    Nullspace property, another sucient condition for sparse recovery Generalized restricted isometry property,[8] a generalized sucient condition for sparse recovery, where mu-tual coherence and restricted isometry property are both its special forms.

    10.5 References[1] E. J. Candes and T. Tao, Decoding by Linear Programming, IEEE Trans. Inf. Th., 51(12): 42034215 (2005).

    [2] E. J. Candes, J. K. Romberg, and T. Tao, Stable Signal Recovery from Incomplete and Inaccurate Measurements, Com-munications on Pure and Applied Mathematics, Vol. LIX, 12071223 (2006).

    [3] A. M. Tillmann and M. E. Pfetsch, "The Computational Complexity of the Restricted Isometry Property, the NullspaceProperty, and Related Concepts in Compressed Sensing, IEEE Trans. Inf. Th., 60(2): 12481259 (2014)

    [4] F. Yang, S. Wang, and C. Deng, "Compressive sensing of image reconstruction using multi-wavelet transform", IEEE 2010

    [5] B. Bah and J. Tanner Improved Bounds on Restricted Isometry Constants for Gaussian Matrices

    [6] http://ecos.maths.ed.ac.uk/ric_bounds.shtml

    [7] James, Oliver; Lee, Heung-No (2014-02-26). On Eigenvalues of Wishart Matrix for Analysis of Compressive SensingSystems. arXiv:1402.6757 [cs, math].

    [8] Yu Wang, Jinshan Zeng, Zhimin Peng, Xiangyu Chang and Zongben Xu. On Linear Convergence of Adaptively IterativeThresholding Algorithms for Compressed Sensing.

  • Chapter 11

    Rotas basis conjecture

    In linear algebra and matroid theory, Rotas basis conjecture is an unproven conjecture concerning rearrangementsof bases, named after Gian-Carlo Rota. It states that, if X is either a vector space of dimension n or more generallya matroid of rank n, with n disjoint bases Bi, then it is possible to arrange the elements of these bases into an n nmatrix in such a way that the rows of the matrix are exactly the given bases and the columns of the matrix are alsobases. That is, it should be possible to nd a second set of n disjoint bases Ci, each of which consists of one elementfrom each of the bases Bi.

    11.1 ExamplesRotas basis conjecture has a simple formulation for points in the Euclidean plane: it states that, given three triangleswith distinct vertices, with each triangle colored with one of three colors, it must be possible to regroup the ninetriangle vertices into three rainbow triangles having one vertex of each color. The triangles are all required to benon-degenerate, meaning that they do not have all three vertices on a line.To see this as an instance of the basis conjecture, one may use either linear independence of the vectors (xi,yi,1) in athree-dimensional real vector space (where (xi,yi) are the Cartesian coordinates of the triangle vertices) or equivalentlyone may use a matroid of rank three in which a set S of points is independent if either |S| 2 or S forms the threevertices of a non-degenerate triangle. For this linear algebra and this matroid, the bases are exactly the non-degeneratetriangles. Given the three input triangles and the three rainbow triangles, it is possible to arrange the nine vertices intoa 3 3 matrix in which each row contains the vertices of one of the single-color triangles and each column containsthe vertices of one of the rainbow triangles.Analogously, for points in three-dimensional Euclidean space, the conjecture states that the sixteen vertices of fournon-degenerate tetrahedra of four dierent colors may be regrouped into four rainbow tetrahedra.

    11.2 Partial resultsThe statement of Rotas basis conjecture was rst published by Huang & Rota (1994), crediting it (without citation)to Rota in 1989.[1] The basis conjecture has been proven for paving matroids (for all n)[2] and for the case n 3 (forall types of matroid).[3] For arbitrary matroids, it is possible to arrange the basis elements into a matrix the rst (n)columns of which are bases.[4] The basis conjecture for linear algebras over elds of characteristic zero and for evenvalues of n would follow from another conjecture on Latin squares by Alon and Tarsi.[1][5] Based on this implication,the conjecture is known to be true for linear algebras over the real numbers for innitely many values of n.[6]

    11.3 Related problemsIn connection with Tverbergs theorem, Brny & Larman (1992) conjectured that, for every set of r(d + 1) points ind-dimensional Euclidean space, colored with d + 1 colors in such a way that there are r points of each color, there isa way to partition the points into rainbow simplices (sets of d + 1 points with one point of each color) in such a way

    30

  • 11.4. SEE ALSO 31

    The nine vertices of three colored triangles (red, blue, and yellow) regrouped into three rainbow triangles (black edges)

    that the convex hulls of these sets have a nonempty intersection.[7] For instance, the two-dimensional case (proven byBrny and Larman) with r = 3 states that, for every set of nine points in the plane, colored with three colors and threepoints of each color, it is possible to partition the points into three intersecting rainbow triangles, a statement similarto Rotas basis conjecture which states that it is possible to partition the points into three non-degenerate rainbowtriangles. The conjecture of Brny and Larman allows a collinear triple of points to be considered as a rainbowtriangle, whereas Rotas basis conjecture disallows this; on the other hand, Rotas basis conjecture does not requirethe triangles to have a common intersection. Substantial progress on the conjecture of Brny and Larman was madeby Blagojevi, Matschke & Ziegler (2009).[8]

    11.4 See also

    Rotas conjecture, a dierent conjecture by Rota about linear algebra and matroids

  • 32 CHAPTER 11. ROTAS BASIS CONJECTURE

    11.5 References[1] Huang, Rosa; Rota, Gian-Carlo (1994), On the relations of various conjectures on Latin squares and straightening coe-

    cients, Discrete Mathematics 128 (1-3): 225236, doi:10.1016/0012-365X(94)90114-7, MR 1271866. See in particularConjecture 4, p. 226.

    [2] Geelen, Jim; Humphries, Peter J. (2006), Rotas basis conjecture for paving matroids, SIAM Journal on Discrete Mathe-matics 20 (4): 10421045, doi:10.1137/060655596, MR 2272246.

    [3] Chan, Wendy (1995), An exchange property of matroid, Discrete Mathematics 146 (1-3): 299302, doi:10.1016/0012-365X(94)00071-3, MR 1360125.

    [4] Geelen, Jim; Webb, Kerri (2007), On Rotas basis conjecture, SIAM Journal on Discrete Mathematics 21 (3): 802804,doi:10.1137/060666494, MR 2354007.

    [5] Onn, Shmuel (1997), A colorful determinantal identity, a conjecture of Rota, and Latin squares, The American Mathe-matical Monthly 104 (2): 156159, doi:10.2307/2974985, MR 1437419.

    [6] Glynn, David G. (2010), The conjectures of AlonTarsi and Rota in dimension prime minus one, SIAM Journal onDiscrete Mathematics 24 (2): 394399, doi:10.1137/090773751, MR 2646093.

    [7] Brny, I.; Larman, D. G. (1992), A colored version of Tverbergs theorem, Journal of the London Mathematical Society,Second Series 45 (2): 314320, doi:10.1112/jlms/s2-45.2.314, MR 1171558.

    [8] Blagojevi, Pavle V.M.; Matschke, Benjamin; Ziegler, GnterM. (2009),Optimal bounds for the colored Tverberg problem,arXiv:0910.4987.

    11.6 External links Rotas basis conjecture, Open Problem Garden.

  • Chapter 12

    Row and column spaces

    The row space and column space of anm-by-nmatrix are the linear subspaces generated by row vectors and columnvectors, respectively, of the matrix. Its dimension is equal to the rank of the matrix and is at most min(m, n).[1]

    The rest of article will consider matrices of real numbers: row and column spaces are subspace of Rn and Rm realspaces respectively. But row and column spaces can be constructed from matrices with components in any eld andeven a ring.

    12.1 OverviewLet A be an m-by-n matrix. Then

    1. rank(A) = dim(rowsp(A)) = dim(colsp(A)),2. rank(A) = number of pivots in any echelon form of A,3. rank(A) = the maximum number of linearly independent rows or columns of A.

    If one considers the matrix as a linear transformation from Rn to Rm, then the column space of the matrix equals theimage of this linear transformation.The column space of a matrix A is the set of all linear combinations of the columns in A. If A = [a1, ...., a], thencolsp(A) = span {a1, ...., a}.The concept of row space generalises to matrices to C, the eld of complex numbers, or to any eld.Intuitively, given a matrix A, the action of the matrix A on a vector x will return a linear combination of the columnsof A weighted by the coordinates of x as coecients. Another way to look at this is that it will (1) rst project xinto the row space of A, (2) perform an invertible transformation, and (3) place the resulting vector y in the columnspace of A. Thus the result y =A x must reside in the column space of A. See the singular value decomposition formore details on this second interpretation.

    12.2 ExampleGiven a matrix J:

    J =

    26642 4 1 3 21 2 1 0 51 6 2 2 23 6 2 5 1

    3775the rows are r1 = (2,4,1,3,2), r2 = (1,2,1,0,5), r3 = (1,6,2,2,2), r4 = (3,6,2,5,1). Consequently the row space of J isthe subspace of R5 spanned by { r1, r2, r3, r4 }. Since these four row vectors are linearly independent, the row space

    33

  • 34 CHAPTER 12. ROW AND COLUMN SPACES

    is 4-dimensional. Moreover in this case it can be seen that they are all orthogonal to the vector n = (6,1,4,4,0), soit can be deduced that the row space consists of all vectors in R5 that are orthogonal to n.

    12.3 See also Row space Column space null space

    12.4 External links Lecture on column space and nullspace by Gilbert Strang of MIT Row Space and Column Space

    12.5 Notes[1] http://www.cliffsnotes.com/WileyCDA/CliffsReviewTopic/Row-Space-and-Column-Space-of-a-Matrix.topicArticleId-20807,

    articleId-20793.html

  • Chapter 13

    Row equivalence

    In linear algebra, two matrices are row equivalent if one can be changed to the other by a sequence of elementaryrow operations. Alternatively, two m n matrices are row equivalent if and only if they have the same row space.The concept is most commonly applied to matrices that represent systems of linear equations, in which case twomatrices of the same size are row equivalent if and only if the corresponding homogeneous systems have the sameset of solutions, or equivalently the matrices have the same null space.Because elementary row operations are reversible, row equivalence is an equivalence relation. It is commonly denotedby a tilde (~).There is a similar notion of column equivalence, dened by elementary column operations; two matrices are columnequivalent if and only if their transpose matrices are row equivalent. Two rectangular matrices that can be convertedinto one another allowing both elementary row and column operations are called simply equivalent.

    13.1 Elementary row operationsAn elementary row operation is any one of the following moves:

    1. Swap: Swap two rows of a matrix.

    2. Scale: Multiply a row of a matrix by a nonzero constant.

    3. Pivot: Add a multiple of one row of a matrix to another row.

    Two matrices A and B are row equivalent if it is possible to transform A into B by a sequence of elementary rowoperations.

    13.2 Row spaceMain article: Row space

    The row space of a matrix is the set of all possible linear combinations of its row vectors. If the rows of the matrixrepresent a system of linear equations, then the row space consists of all linear equations that can be deduced alge-braically from those in the system. Two m n matrices are row equivalent if and only if they have the same rowspace.For example, the matrices

    1 0 00 1 1

    and

    1 0 01 1 1

    35

  • 36 CHAPTER 13. ROW EQUIVALENCE

    are row equivalent, the row space being all vectors of the forma b b

    . The corresponding systems of homoge-

    neous equations convey the same information:

    x = 0y + z = 0

    and x = 0x+ y + z = 0:

    In particular, both of these systems imply every equation of the form ax+ by + bz = 0:

    13.3 Equivalence of the denitionsThe fact that two matrices are row equivalent if and only if they have the same row space is an important theorem inlinear algebra. The proof is based on the following observations:

    1. Elementary row operations do not aect the row space of a matrix. In particular, any two row equivalentmatrices have the same row space.

    2. Any matrix can be reduced by elementary row operations to a matrix in reduced row echelon form.3. Two matrices in reduced row echelon form have the same row space if and only if they are equal.

    This line of reasoning also proves that every matrix is row equivalent to a unique matrix with reduced row echelonform.

    13.4 Additional properties Because the null space of a matrix is the orthogonal complement of the row space, two matrices are rowequivalent if and only if they have the same null space.

    The rank of a matrix is equal to the dimension of the row space, so row equivalent matrices must have the samerank. This is equal to the number of pivots in the reduced row echelon form.

    A matrix is invertible if and only if it is row equivalent to the identity matrix.

    13.5 See also Elementary row operations Row space Basis (linear algebra) Row reduction (Reduced) row echelon form

    13.6 References Axler, Sheldon Jay (1997), Linear Algebra Done Right (2nd ed.), Springer-Verlag, ISBN 0-387-98259-0 Lay, David C. (August 22, 2005), Linear Algebra and Its Applications (3rd ed.), Addison Wesley, ISBN 978-0-321-28713-7

    Meyer, Carl D. (February 15, 2001), Matrix Analysis and Applied Linear Algebra, Society for Industrial andApplied Mathematics (SIAM), ISBN 978-0-89871-454-8

    Poole, David (2006), Linear Algebra: A Modern Introduction (2nd ed.), Brooks/Cole, ISBN 0-534-99845-3 Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International Leon, Steven J. (2006), Linear Algebra With Applications (7th ed.), Pearson Prentice Hall

  • 13.7. EXTERNAL LINKS 37

    13.7 External links

  • Chapter 14

    Row space

    The row vectors of a matrix. The row space of this matrix is the vector space generated by linear combinations of the row vectors.

    In linear algebra, the row space of a matrix is the set of all possible linear combinations of its row vectors. Let Kbe a eld (such as real or complex numbers). The row space of an m n matrix with components from K is a linearsubspace of the n-space Kn. The dimension of the row space is called the row rank of the matrix.[1]

    A denition for matrices over a ring K (such as integers) is also possible.[2]

    38

  • 14.1. DEFINITION 39

    14.1 DenitionLet K be a eld of scalars. Let A be an m n matrix, with row vectors r1, r2, ... , rm. A linear combination of thesevectors is any vector of the form

    c1r1 + c2r2 + + cmrm;

    where c1, c2, ... , cm are scalars. The set of all possible linear combinations of r1, ... , rm is called the row space ofA. That is, the row space of A is the span of the vectors r1, ... , rm.For example, if

    A =

    1 0 20 1 0

    ;

    then the row vectors are r1 = (1, 0, 2) and r2 = (0, 1, 0). A linear combination of r1 and r2 is any vector of the form

    c1(1; 0; 2) + c2(0; 1; 0) = (c1; c2; 2c1):

    The set of all such vectors is the row space of A. In this case, the row space is precisely the set of vectors (x, y, z) K3satisfying the equation z = 2x (using Cartesian coordinates, this set is a plane through the origin in three-dimensionalspace).For a matrix that represents a homogeneous system of linear equations, the row space consists of all linear equationsthat follow from those in the system.The column space of A is equal to the row space of AT.

    14.2 BasisThe row space is not aected by elementary row operations. This makes it possible to use row reduction to nd abasis for the row space.For example, consider the matrix

    A =

    241 3 22 7 41 5 2

    35:The rows of this matrix span the row space, but they may not be linearly independent, in which case the rows will notbe a basis. To nd a basis, we reduce A to row echelon form:r1, r2, r3 represents the rows.

    241 3 22 7 41 5 2

    35 |{z}r22r1

    241 3 20 1 01 5 2

    35 |{z}r3r1

    241 3 20 1 00 2 0

    35 |{z}r32r2

    241 3 20 1 00 0 0

    35 |{z}r13r2

    241 0 20 1 00 0 0

    35:Once the matrix is in echelon form, the nonzero rows are a basis for the row space. In this case, the basis is { (1, 3,2), (0, 1, 0) }. Another possible basis { (1, 0, 2), (0, 1, 0) } comes from a further reduction.[3]

    This algorithm can be used in general to nd a basis for the span of a set of vectors. If the matrix is further simpliedto reduced row echelon form, then the resulting basis is uniquely determined by the row space.

  • 40 CHAPTER 14. ROW SPACE

    14.3 DimensionMain article: Rank (linear algebra)

    The dimension of the row space is called the rank of the matrix. This is the same as the maximum number of linearlyindependent rows that can be chosen from the matrix, or equivalently the number of pivots. For example, the 3 3matrix in the example above has rank two.[3]

    The rank of a matrix is also equal to the dimension of the column space. The dimension of the null space is calledthe nullity of the matrix, and is related to the rank by the following equation:

    rank(A) + nullity(A) = n;where n is the number of columns of the matrix A. The equation above is known as the rank-nullity theorem.

    14.4 Relation to the null spaceThe null space of matrix A is the set of all vectors x for which Ax = 0. The product of the matrix A and the vector xcan be written in terms of the dot product of vectors:

    Ax =

    26664r1 xr2 x...

    rm x

    37775;where r1, ... , rm are the row vectors of A. Thus Ax = 0 if and only if x is orthogonal (perpendicular) to each of therow vectors of A.It follows that the null space of A is the orthogonal complement to the row space. For example, if the row space isa plane through the origin in three dimensions, then the null space will be the perpendicular line through the origin.This provides a proof of the rank-nullity theorem (see dimension above).The row space and null space are two of the four fundamental subspaces associated with a matrix A (the other twobeing the column space and left null space).

    14.5 Relation to coimageIf V and W are vector spaces, then the kernel of a linear transformation T : V W is the set of vectors v V forwhich T(v) = 0. The kernel of a linear transformation is analogous to the null space of a matrix.If V is an inner product space, then the orthogonal complement to the kernel can be thought of as a generalization ofthe row space. This is sometimes called the coimage of T. The transformation T is one-to-one on its coimage, andthe coimage maps isomorphically onto the image of T.When V is not an inner product space, the coimage of T can be dened as the quotient space V / ker(T).

    14.6 Notes[1] Linear algebra, as discussed in this article, is a very well established mathematical discipline for which there are many

    sources. Almost all of the material in this article can be found in Lay 2005, Meyer 2001, and Strang 2005.[2] A denition and certain properties for rings are the same with replacement of the "vector n-space" Kn with left free

    module" and linear subspace with "submodule". For non-commutative rings this row space is sometimes disambiguatedas left row space.

    [3] The example is valid over real, rational numbers, and other number elds. It is not necessarily correct over elds and ringswith non-zero characteristic.

  • 14.7. REFERENCES 41

    14.7 ReferencesSee also: Linear algebra Further reading

    14.7.1 Textbooks Axler, Sheldon Jay (1997), Linear Algebra Done Right (2nd ed.), Springer-Verlag, ISBN 0-387-98259-0 Lay, David C. (August 22, 2005), Linear Algebra and Its Applications (3rd ed.), Addison Wesley, ISBN 978-0-321-28713-7

    Meyer, Carl D. (February 15, 2001), Matrix Analysis and Applied Linear Algebra, Society for Industrial andApplied Mathematics (SIAM), ISBN 978-0-89871-454-8

    Poole, David (2006), Linear Algebra: A Modern Introduction (2nd ed.), Brooks/Cole, ISBN 0-534-99845-3 Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International Leon, Steven J. (2006), Linear Algebra With Applications (7th ed.), Pearson Prentice Hall

    14.8 External links Weisstein, Eric W., Row Space, MathWorld. Gilbert Strang, MIT Linear Algebra Lecture on the Four Fundamental Subspaces at Google Video, from MITOpenCourseWare

  • Chapter 15

    Row vector

    In linear algebra, a row vector or row matrix is a 1 m matrix, i.e. a matrix consisting of a single row of melements:[1]

    x =x1 x2 : : : xm

    :

    The transpose of a row vector is a column vector:

    x1 x2 : : : xm

    T=

    26664x1x2...xm

    37775:The set of all row vectors forms a vector space (row space) which acts like the dual space to the set of all columnvectors (see row and column spaces), in the sense that any linear functional on the space of column vectors (i.e. anyelement of the dual space) can be represented uniquely as a dot product with a specic row vector.

    15.1 NotationRow vectors are sometimes written using the following non-standard notation:

    x =x1; x2; : : : ; xm

    :

    15.2 Operations

    Matrix multiplication involves the action of multiplying each row vector of one matrix by each column vectorof another matrix.

    The dot product of two vectors a and b is equivalent to multiplying the row vector representation of a by thecolumn vector representation of b:

    a b = a1 a2 a324b1b2b3

    35:42

  • 15.3. PREFERRED INPUT VECTORS FOR MATRIX TRANSFORMATIONS 43

    15.3 Preferred input vectors for matrix transformationsFrequently a row vector presents itself for an operation within n-space expressed by an n by n matrix M:

    v M = p.

    Then p is also a row vector and may present to another n by n matrix Q:

    p Q = t.

    Conveniently, one can write t = p Q = v MQ telling us that the matrix product transformationMQ can take v directlyto t. Continuing with row vectors, matrix transformations further reconguring n-space can be applied to the right ofprevious outputs.In contrast, when a column vector is transformed to become another column under an n by n matrix action, theoperation occurs to the left:

    p = M v and t = Q p ,

    leading to the algebraic expression QM v for the composed output from v input. The matrix transformations mountup to the left in this use of a column vector for input to matrix transformation. The natural bias to read left-to-right,as subsequent transformations are applied in linear algebra, stands against column vector inputs.Nevertheless, using the transpose operation these dierences between inputs of a row or column nature are resolvedby an antihomomorphism between the groups arising on the two sides. The technical construction uses the dual spaceassociated with a vector space to develop the transpose of a linear map.For an instance where this row vector input convention has been used to good eect see Raiz Usmani,[2] where onpage 106 the convention allows the statement The product mapping ST of U intoW [is given] by:

    (ST ) = (S)T = T =

    (The Greek letters represent row vectors).Ludwik Silberstein used row vectors for spacetime events; he applied Lorentz transformation matrices on the rightin his Theory of Relativity in 1914 (see page 143). In 1963 when McGraw-Hill published Dierential Geometry byHeinrich Guggenheimer of the University of Minnesota, he uses the row vector convention in chapter 5, Introductionto transformation groups (eqs. 7a,9b and 12 to 15). When H. S. M. Coxeter reviewed[3] Linear Geometry by RafaelArtzy, he wrote, "[Artzy] is to be congratulated on his choice of the 'left-to-right' convention, which enables him toregard a point as a row matrix instead of the clumsy column that many authors prefer.

    15.4 See also Covariance and contravariance of vectors

    15.5 Notes[1] Meyer (2000), p. 8

    [2] Raiz A. Usmani (1987) Applied Linear Algebra Marcel Dekker ISBN 0824776224. See Chapter 4: Linear Transforma-tions

    [3] Coxeter Review of Linear Geometry from Mathematical Reviews

  • 44 CHAPTER 15. ROW VECTOR

    15.6 ReferencesSee also: Linear algebra Further reading

    Axler, Sheldon Jay (1997), Linear Algebra Done Right (2nd ed.), Springer-Verlag, ISBN 0-387-98259-0 Lay, David C. (August 22, 2005), Linear Algebra and Its Applications (3rd ed.), Addison Wesley, ISBN 978-0-321-28713-7

    Meyer, Carl D. (February 15, 2001), Matrix Analysis and Applied Linear Algebra, Society for Industrial andApplied Mathematics (SIAM), ISBN 978-0-89871-454-8

    Poole, David (2006), Linear Algebra: A Modern Introduction (2nd ed.), Brooks/Cole, ISBN 0-534-99845-3 Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International Leon, Steven J. (2006), Linear Algebra With Applications (7th ed.), Pearson Prentice Hall

  • Chapter 16

    Rule of Sarrus

    Sarrus rule: The determinant of the three columns on the left is the sum of the products along the solid diagonals minus