Upload
jie-fu
View
482
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Matrix calculus reference
Citation preview
Matrix Calculus
Jie FuSaturday, April 8, 2023
https://sites.google.com/site/bigaidream/
Matrix calculus collects the various partial derivatives of a single function with respect to many variables, and/or of a multivariate function with respect to a single variable, into vectors and matrices that can be treated as single entities. Reference: http://en.wikipedia.org/wiki/Matrix_calculus
ContentsMatrix Calculus...............................................................................................................................................................1Scope.............................................................................................................................................................................. 1Notation..........................................................................................................................................................................2Derivatives with Vectors.................................................................................................................................................2
Vector-by-scalar......................................................................................................................................................2Scalar-by-vector......................................................................................................................................................3Vector-by-vector.....................................................................................................................................................3
Derivatives with Matrices...............................................................................................................................................3Matrix-by-scalar......................................................................................................................................................3Scalar-by-matrix......................................................................................................................................................4
Identities.........................................................................................................................................................................5Vector-by-vector identities......................................................................................................................................5Scalar-by-vector identities.......................................................................................................................................6Vector-by-scalar identities......................................................................................................................................8Scalar-by-matrix identities......................................................................................................................................9Matrix-by-scalar identities....................................................................................................................................10Scalar-by-scalar identities.....................................................................................................................................11
With vectors involved...................................................................................................................................11With matrices involved.................................................................................................................................11
Identities in differential form................................................................................................................................11
Scope
For a scalar function of three independent variables, , the gradient is given by the vector equation
,
where represents a unit vector in the direction for . This type of generalized derivative can be seen as the derivative of a scalar, f, with respect to a vector, and its result can be easily collected in vector form.
Notation
All functions are assumed to be of differentiability class C1 unless otherwise noted. Generally letters from first half of the alphabet (a, b, c, …) will be used to denote constants, and from the second half (t, x, y, …) to denote variables.
Derivatives with Vectors
Vector-by-scalar
The derivative of a vector , by a scalar x is written (in numerator layout notation) as
Scalar-by-vector
The derivative of a scalar y by a vector , is written (in numerator layout notation) as
Vector-by-vector
The derivative of a vector function (a vector whose components are functions) , of an independent
vector , is written (in numerator layout notation) as
Derivatives with Matrices
Matrix-by-scalar
The derivative of a matrix function Y by a scalar x is known as the tangent matrix and is given (in numerator layout notation) by
Scalar-by-matrix
The derivative of a scalar y function of a matrix X of independent variables, with respect to the matrix X, is given (in numerator layout notation) by
Identities
Vector-by-vector identities
Scalar-by-vector identities
Vector-by-scalar identities
Scalar-by-matrix identities
Matrix-by-scalar identities
Scalar-by-scalar identities
With vectors involved
With matrices involved
Identities in differential form
It is often easier to work in differential form and then convert back to normal derivatives. This only works well using
the numerator layout.
To convert to normal derivative form, first convert it to one of the following canonical forms, and then use these identities: