Kronecker product

In mathematics, the Kronecker product, denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a special case of a tensor product. The Kronecker product should not be confused with the usual matrix multiplication, which is an entirely different operation.

 Contents

Definition

If A is an m-by-n matrix and B is a p-by-q matrix, then the Kronecker product AB is the mp-by-nq block matrix

[itex] A \otimes B = \begin{bmatrix} a_{11} B & \cdots & a_{1n}B \\ \vdots & \ddots & \vdots \\ a_{m1} B & \cdots & a_{mn} B \end{bmatrix}. [itex]

More explicitly, we have

[itex] A \otimes B = \begin{bmatrix}
  a_{11} b_{11} & a_{11} b_{12} & \cdots & a_{11} b_{1q} &
\cdots & \cdots & a_{1n} b_{11} & a_{1n} b_{12} & \cdots & a_{1n} b_{1q} \\
a_{11} b_{21} & a_{11} b_{22} & \cdots & a_{11} b_{2q} &
\cdots & \cdots & a_{1n} b_{21} & a_{1n} b_{22} & \cdots & a_{1n} b_{2q} \\
\vdots & \vdots & \ddots & \vdots & & & \vdots & \vdots & \ddots & \vdots \\
a_{11} b_{p1} & a_{11} b_{p2} & \cdots & a_{11} b_{pq} &
\cdots & \cdots & a_{1n} b_{p1} & a_{1n} b_{p2} & \cdots & a_{1n} b_{pq} \\
\vdots & \vdots & & \vdots & \ddots & & \vdots & \vdots & & \vdots \\
\vdots & \vdots & & \vdots & & \ddots & \vdots & \vdots & & \vdots \\
a_{m1} b_{11} & a_{m1} b_{12} & \cdots & a_{m1} b_{1q} &
\cdots & \cdots & a_{mn} b_{11} & a_{mn} b_{12} & \cdots & a_{mn} b_{1q} \\
a_{m1} b_{21} & a_{m1} b_{22} & \cdots & a_{m1} b_{2q} &
\cdots & \cdots & a_{mn} b_{21} & a_{mn} b_{22} & \cdots & a_{mn} b_{2q} \\
\vdots & \vdots & \ddots & \vdots & & & \vdots & \vdots & \ddots & \vdots \\
a_{m1} b_{p1} & a_{m1} b_{p2} & \cdots & a_{m1} b_{pq} &
\cdots & \cdots & a_{mn} b_{p1} & a_{mn} b_{p2} & \cdots & a_{mn} b_{pq}


\end{bmatrix}. [itex]

Example

[itex]
 \begin{bmatrix}
1 & 2 \\
3 & 1 \\
\end{bmatrix}


\otimes

 \begin{bmatrix}
0 & 3 \\
2 & 1 \\
\end{bmatrix}


=

 \begin{bmatrix}
1\cdot 0 & 1\cdot 3 & 2\cdot 0 & 2\cdot 3 \\
1\cdot 2 & 1\cdot 1 & 2\cdot 2 & 2\cdot 1 \\
3\cdot 0 & 3\cdot 3 & 1\cdot 0 & 1\cdot 3 \\
3\cdot 2 & 3\cdot 1 & 1\cdot 2 & 1\cdot 1 \\
\end{bmatrix}


=

 \begin{bmatrix}
0 & 3 & 0 & 6 \\
2 & 1 & 4 & 2 \\
0 & 9 & 0 & 3 \\
6 & 3 & 2 & 1
\end{bmatrix}


[itex].

Properties

Bilinearity and associativity

The Kronecker product is a special case of the tensor product, so it is bilinear and associative:

[itex] A \otimes (B+C) = A \otimes B + A \otimes C \qquad \mbox{(if } B \mbox{ and } C \mbox{ have the same size)}, [itex]
[itex] (A+B) \otimes C = A \otimes C + B \otimes C \qquad \mbox{(if } A \mbox{ and } B \mbox{ have the same size)}, [itex]
[itex] (kA) \otimes B = A \otimes (kB) = k(A \otimes B), [itex]
[itex] (A \otimes B) \otimes C = A \otimes (B \otimes C), [itex]

where A, B and C are matrices and k is a scalar.

The Kronecker product is not commutative: in general, AB and BA are different matrices. However, AB and BA are permutation equivalent, meaning that there exist permutation matrices P and Q such that

[itex] A \otimes B = P \, (B \otimes A) \, Q. [itex]

If A and B are square matrices, then AB and BA are even permutation similar, meaning that we can take P = QT.

The mixed-product property

If A, B, C and D are matrices of such size that one can form the matrix products AC and BD, then

[itex] (A \otimes B)(C \otimes D) = AC \otimes BD. [itex]

This is called the mixed-product property, because it mixes the ordinary matrix product and the Kronecker product. It follows that AB is invertible if and only if A and B are invertible, in which case the inverse is given by

[itex] (A \otimes B)^{-1} = A^{-1} \otimes B^{-1}. [itex]

Spectrum

Suppose that A and B are square matrices of size n and q respectively. Let λ1, ..., λn be the eigenvalues of A and μ1, ..., μq be those of B (listed according to multiplicity). Then the eigenvalues of AB are

[itex] \lambda_i \mu_j, \qquad i=1,\ldots,n ,\, j=1,\ldots,q. [itex]

It follows that the trace and determinant of a Kronecker product are given by

[itex] \operatorname{tr}(A \otimes B) = \operatorname{tr} A \, \operatorname{tr} B \quad\mbox{and}\quad \det(A \otimes B) = (\det A)^n (\det B)^q. [itex]

Singular values

If A and B are rectangular matrices, then one can consider their singular values. Suppose that A has rA nonzero singular values, namely

[itex] \sigma_{A,i}, \qquad i = 1, \ldots, r_A. [itex]

Similarly, denote the nonzero singular values of B by

[itex] \sigma_{B,i}, \qquad i = 1, \ldots, r_B. [itex]

Then the Kronecker product AB has rArB nonzero singular values, namely

[itex] \sigma_{A,i} \sigma_{B,j}, \qquad i=1,\ldots,r_A ,\, j=1,\ldots,r_B. [itex]

Since the rank of a matrix equals the number of nonzero singular values, we find that

[itex] \operatorname{rank}(A \otimes B) = \operatorname{rank} A \, \operatorname{rank} B. [itex]

Relation to the abstract tensor product

The Kronecker product of matrices corresponds to the abstract tensor product of linear maps. Specifically, if the matrices A and B represent linear transformations V1W1 and V2W2, respectively, then the matrix AB represents the tensor product of the two maps, V1V2W1W2.

Matrix equations

The Kronecker product can be used to get a convenient representation for some matrix equations. Consider for instance the equation AXB = C, where A, B and C are given matrices and the matrix X is the unknown. We can rewrite this equation as

[itex] (B^\top \otimes A) \, \operatorname{vec} X = \operatorname{vec} C. [itex]

It now follows from the properties of the Kronecker product that the equation AXB = C has a unique solution if and only if A and B are nonsingular.

Here, vec X denotes the vector formed by collecting the entries of he matrix X in one long vector. Specifically, if X is an m-by-n matrix, then

[itex] \operatorname{vec} X = [ x_{11}, x_{21}, \ldots, x_{m1}, x_{12}, x_{22}, \ldots, x_{m2}, \ldots, x_{1n}, x_{2n}, \ldots, x_{mn} ]^\top. [itex]

History

The Kronecker product is named after Leopold Kronecker, even though there is little evidence that he was the first to define and use it. Indeed, in the past the Kronecker product was sometimes called the Zehfuss matrix, after Johann Georg Zehfuss.

References

• Art and Cultures
• Countries of the World (http://www.academickids.com/encyclopedia/index.php/Countries)
• Space and Astronomy