向量（矩阵）求导

Posted on Edited on In Waline:

向量（矩阵）对求导运算总结。

向量（矩阵）求导

1 向量（矩阵）对元素求导

1.1 行向量对元素求导

设 yT=[y1,⋯ ,yn]\boldsymbol{y}^T = [y_1, \cdots, y_n] 为 nn 维行向量，xx 为元素，则

∂yT∂x=[∂y1∂x,⋯ ,∂yn∂x]\frac{\partial \boldsymbol{y}^T}{\partial x} = [\frac{\partial y_1}{\partial x}, \cdots, \frac{\partial y_n}{\partial x}]

1.2 列向量对元素求导

设 y=[y1,⋯ ,ym]T\boldsymbol{y} = [y_1, \cdots, y_m]^T 为 mm 维列向量，xx 为元素，则

∂y∂x=[∂y1∂x⋮∂ym∂x]\frac{\partial \boldsymbol{y}}{\partial x} = \left[ \begin{matrix} \frac{\partial y_1}{\partial x} \\ \vdots \\ \frac{\partial y_m}{\partial x} \end{matrix} \right]

1.3 矩阵对元素求导

设 Y=[yij]Y = [y_{ij}] 为 m×nm\times n 矩阵，xx 为元素，则

∂Y∂x=[∂y11∂x⋯∂y1n∂x⋮⋮⋮∂ym1∂x⋯∂ymn∂x]\frac{\partial Y}{\partial x} = \left[ \begin{matrix} \frac{\partial y_{11}}{\partial x} &\cdots &\frac{\partial y_{1n}}{\partial x} \\ \vdots &\vdots &\vdots \\ \frac{\partial y_{m1}}{\partial x} &\cdots &\frac{\partial y_{mn}}{\partial x} \end{matrix} \right]

2 元素对向量（矩阵）求导

2.1 元素对行向量求导

设 yy 为元素，xT=[x1,⋯ ,xq]\boldsymbol{x}^T = [x_1, \cdots, x_q] 为 qq 维行向量，则

∂y∂xT=[∂y∂x1,⋯ ,∂y∂xq]\frac{\partial y}{\partial \boldsymbol{x}^T} = [\frac{\partial y}{\partial x_1}, \cdots, \frac{\partial y}{\partial x_q}]

2.2 元素对列向量求导

设 yy 为元素，x=[x1,⋯ ,xp]T\boldsymbol{x} = [x_1, \cdots, x_p]^T 为 pp 维行向量，则

∂y∂x=[∂y∂x1⋮∂y∂xp]\frac{\partial y}{\partial \boldsymbol{x}} = \left[ \begin{matrix} \frac{\partial y}{\partial x_1} \\ \vdots \\ \frac{\partial y}{\partial x_p} \end{matrix} \right]

2.3 元素对矩阵求导

设 yy 为元素，X=[xij]X = [x_{ij}] 为 p×qp\times q 矩阵，则

∂y∂X=[∂y∂x11⋯∂y∂x1q⋮⋮⋮∂y∂xp1⋯∂y∂xpq]\frac{\partial y}{\partial X} = \left[ \begin{matrix} \frac{\partial y}{\partial x_{11}} &\cdots &\frac{\partial y}{\partial x_{1q}} \\ \vdots &\vdots &\vdots \\ \frac{\partial y}{\partial x_{p1}} &\cdots &\frac{\partial y}{\partial x_{pq}} \end{matrix} \right]

3 行（列）向量对列（行）向量求导

3.1 行向量对列向量求导

设 yT=[y1,⋯ ,yn]\boldsymbol{y}^T = [y_1, \cdots, y_n] 为 nn 维行向量，x=[x1,⋯ ,xp]T\boldsymbol{x} = [x_1, \cdots, x_p]^T 为 pp 维行向量，则

∂yT∂x=[∂y1∂x1⋯∂yn∂x1⋮⋮⋮∂y1∂xp⋯∂yn∂xp]\frac{\partial \boldsymbol{y}^T}{\partial \boldsymbol{x}} = \left[ \begin{matrix} \frac{\partial y_1}{\partial x_1} &\cdots &\frac{\partial y_n}{\partial x_1} \\ \vdots &\vdots &\vdots \\ \frac{\partial y_1}{\partial x_p} &\cdots &\frac{\partial y_n}{\partial x_p} \end{matrix} \right]

3.2 列向量对行向量求导

设 y=[y1,⋯ ,ym]T\boldsymbol{y} = [y_1, \cdots, y_m]^T 为 mm 维列向量，xT=[x1,⋯ ,xq]\boldsymbol{x}^T = [x_1, \cdots, x_q] 为 qq 维行向量，则

∂y∂xT=[∂y1∂x1⋯∂y1∂xq⋮⋮⋮∂ym∂x1⋯∂ym∂xq]\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{x}^T} = \left[ \begin{matrix} \frac{\partial y_1}{\partial x_1} &\cdots &\frac{\partial y_1}{\partial x_q} \\ \vdots &\vdots &\vdots \\ \frac{\partial y_m}{\partial x_1} &\cdots &\frac{\partial y_m}{\partial x_q} \end{matrix} \right]

4 行（列）向量对行（列）向量求导

4.1 行向量对行向量求导

设 yT=[y1,⋯ ,yn]\boldsymbol{y}^T = [y_1, \cdots, y_n] 为 nn 维行向量，xT=[x1,⋯ ,xq]\boldsymbol{x}^T = [x_1, \cdots, x_q] 为 qq 维行向量，则

∂yT∂xT=[∂yT∂x1,⋯ ,∂yT∂xq]\frac{\partial \boldsymbol{y}^T}{\partial \boldsymbol{x}^T} = [\frac{\partial \boldsymbol{y}^T}{\partial x_1}, \cdots, \frac{\partial \boldsymbol{y}^T}{\partial x_q}]

4.2 列向量对列向量求导

设 y=[y1,⋯ ,ym]T\boldsymbol{y} = [y_1, \cdots, y_m]^T 为 mm 维列向量，x=[x1,⋯ ,xp]T\boldsymbol{x} = [x_1, \cdots, x_p]^T 为 pp 维行向量，则

∂y∂x=[∂y1∂x⋮∂ym∂x]\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{x}} = \left[ \begin{matrix} \frac{\partial y_1}{\partial \boldsymbol{x}} \\ \vdots \\ \frac{\partial y_m}{\partial \boldsymbol{x}} \end{matrix} \right]

5 矩阵对行（列）向量求导

5.1 矩阵对行向量求导

设 Y=[yij]Y = [y_{ij}] 为 m×nm\times n 矩阵，xT=[x1,⋯ ,xq]\boldsymbol{x}^T = [x_1, \cdots, x_q] 为 qq 维行向量，则

∂Y∂xT=[∂Y∂x1,⋯ ,∂Y∂xq]∈Rm×nq\frac{\partial Y}{\partial \boldsymbol{x}^T} = [\frac{\partial Y}{\partial x_1}, \cdots, \frac{\partial Y}{\partial x_q}] \in \mathbb{R}^{m \times nq}

记 Y=[y1T,⋯ ,ymT]TY = [\boldsymbol{y}_1^T, \cdots, \boldsymbol{y}_m^T]^T，则可以视为多个行向量对单个行向量求导

∂Y∂xT=[∂y1T∂xT⋮∂ymT∂xT]=[∂y1T∂x1⋯∂y1T∂xq⋮⋮⋮∂ymT∂x1⋯∂ymT∂xq]∈Rm×nq\frac{\partial Y}{\partial \boldsymbol{x}^T} = \left[ \begin{matrix} \frac{\partial \boldsymbol{y}_1^T}{\partial \boldsymbol{x}^T} \\ \vdots \\ \frac{\partial \boldsymbol{y}_m^T}{\partial \boldsymbol{x}^T} \end{matrix} \right] = \left[ \begin{matrix} \frac{\partial \boldsymbol{y}_1^T}{\partial x_1} &\cdots &\frac{\partial \boldsymbol{y}_1^T}{\partial x_q} \\ \vdots &\vdots &\vdots \\ \frac{\partial \boldsymbol{y}_m^T}{\partial x_1} &\cdots &\frac{\partial \boldsymbol{y}_m^T}{\partial x_q} \end{matrix} \right] \in \mathbb{R}^{m \times nq}

5.2 矩阵对列向量求导

设 Y=[yij]Y = [y_{ij}] 为 m×nm\times n 矩阵，x=[x1,⋯ ,xp]T\boldsymbol{x} = [x_1, \cdots, x_p]^T 为 pp 维列向量，则

∂Y∂x=[∂y11∂x⋯∂y1n∂x⋮⋮⋮∂ym1∂x⋯∂ymn∂x]∈Rmp×n\frac{\partial Y}{\partial \boldsymbol{x}} = \left[ \begin{matrix} \frac{\partial y_{11}}{\partial \boldsymbol{x}} &\cdots &\frac{\partial y_{1n}}{\partial \boldsymbol{x}} \\ \vdots &\vdots &\vdots \\ \frac{\partial y_{m1}}{\partial \boldsymbol{x}} &\cdots &\frac{\partial y_{mn}}{\partial \boldsymbol{x}} \end{matrix} \right] \in \mathbb{R}^{mp \times n}

记 Y=[y1,⋯ ,yn]Y = [\boldsymbol{y}_1, \cdots, \boldsymbol{y}_n]，则可以视为多个列向量对单个列向量求导

∂Y∂xT=[∂y1∂x,⋯ ,∂yn∂x]∈Rmp×n\frac{\partial Y}{\partial \boldsymbol{x}^T} = [\frac{\partial \boldsymbol{y_1}}{\partial \boldsymbol{x}}, \cdots, \frac{\partial \boldsymbol{y_n}}{\partial \boldsymbol{x}}] \in \mathbb{R}^{mp \times n}

6 向量（矩阵）对矩阵求导

6.1 行向量对矩阵求导

设 yT=[y1,⋯ ,yn]\boldsymbol{y}^T = [y_1, \cdots, y_n] 为 nn 维行向量，X=[xij]X = [x_{ij}] 为 p×qp\times q 矩阵，则

∂yT∂X=[∂yT∂x11⋯∂yT∂x1q⋮⋮⋮∂yT∂xp1⋯∂yT∂xpq]∈Rp×nq\frac{\partial \boldsymbol{y}^T}{\partial X} = \left[ \begin{matrix} \frac{\partial \boldsymbol{y}^T}{\partial x_{11}} &\cdots &\frac{\partial \boldsymbol{y}^T}{\partial x_{1q}} \\ \vdots &\vdots &\vdots \\ \frac{\partial \boldsymbol{y}^T}{\partial x_{p1}} &\cdots &\frac{\partial \boldsymbol{y}^T}{\partial x_{pq}} \end{matrix} \right] \in \mathbb{R}^{p \times nq}

记 X=[x1T,⋯ ,xpT]TX = [\boldsymbol{x}_1^T, \cdots, \boldsymbol{x}_p^T]^T，则可以视为单个行向量对多个行向量求导

∂yT∂X=[∂yT∂x1T⋮∂yT∂xpT]∈Rp×nq\frac{\partial \boldsymbol{y}^T}{\partial X} = \left[ \begin{matrix} \frac{\partial \boldsymbol{y}^T}{\partial \boldsymbol{x}_1^T} \\ \vdots \\ \frac{\partial \boldsymbol{y}^T}{\partial \boldsymbol{x}_p^T} \end{matrix} \right] \in \mathbb{R}^{p \times nq}

6.2 列向量对矩阵求导

设 y=[y1,⋯ ,ym]T\boldsymbol{y} = [y_1, \cdots, y_m]^T 为 mm 维列向量，X=[xij]X = [x_{ij}] 为 p×qp\times q 矩阵，则

∂y∂X=[∂y1∂X⋮∂ym∂X]∈Rmp×q\frac{\partial \boldsymbol{y}}{\partial X} = \left[ \begin{matrix} \frac{\partial y_1}{\partial X} \\ \vdots \\ \frac{\partial y_m}{\partial X} \end{matrix} \right] \in \mathbb{R}^{mp \times q}

记 X=[x1,⋯ ,xq]X = [\boldsymbol{x}_1, \cdots, \boldsymbol{x}_q]，则可以视为单个列向量对多个列向量求导

∂y∂X=[∂y∂x1,⋯ ,∂y∂xq]=[∂y1∂x1⋯∂y1∂xq⋮⋮⋮∂ym∂x1⋯∂ym∂xq]∈Rmp×q\frac{\partial \boldsymbol{y}}{\partial X} = [\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{x}_1}, \cdots, \frac{\partial \boldsymbol{y}}{\partial \boldsymbol{x}_q}] = \left[ \begin{matrix} \frac{\partial y_1}{\partial \boldsymbol{x}_1} &\cdots &\frac{\partial y_1}{\partial \boldsymbol{x}_q} \\ \vdots &\vdots &\vdots \\ \frac{\partial y_m}{\partial \boldsymbol{x}_1} &\cdots &\frac{\partial y_m}{\partial \boldsymbol{x}_q} \end{matrix} \right] \in \mathbb{R}^{mp \times q}

6.3 矩阵对矩阵求导

设 Y=[yij]Y = [y_{ij}] 为 m×nm\times n 矩阵，X=[xij]X = [x_{ij}] 为 p×qp\times q 矩阵，则

∂Y∂X=[∂y1T∂x1⋯∂y1T∂xq⋮⋮⋮∂ymT∂x1⋯∂ymT∂xq]∈Rmp×nq\frac{\partial Y}{\partial X} = \left[ \begin{matrix} \frac{\partial \boldsymbol{y}_1^T}{\partial \boldsymbol{x}_1} &\cdots &\frac{\partial \boldsymbol{y}_1^T}{\partial \boldsymbol{x}_q} \\ \vdots &\vdots &\vdots \\ \frac{\partial \boldsymbol{y}_m^T}{\partial \boldsymbol{x}_1} &\cdots &\frac{\partial \boldsymbol{y}_m^T}{\partial \boldsymbol{x}_q} \end{matrix} \right] \in \mathbb{R}^{mp \times nq}

先将 XX 看成列向量，再将 YY 看成行向量。

7 几个推广公式

∂Ax∂xT=A∂(Ax)T∂x=AT\frac{\partial Ax}{\partial x^T} = A \\ \frac{\partial (Ax)^T}{\partial x} = A^T

参考