Formula

Vectorized Equations for Logistic Regression on m Examples

To compute logistic regression and its gradient descent on mm examples, using a for loop to accumulate errors and derivatives takes significant time on large datasets. Vectorization efficiently eliminates the need for explicit for loops. First, stack the mm examples horizontally into matrices XX and YY, so the shape of XX is (nx,m)(n_x, m), where nxn_x is the number of features, and the shape of YY is (1,m)(1, m). Then, compute ZZ and AA:

Z=[z(1)z(m)]=wTX+b=[(wTx(1)+b)(wTx(m)+b)]Z = [z^{(1)} \dots z^{(m)}] = w^T X + b = [(w^T x^{(1)} + b) \dots (w^T x^{(m)} + b)]

A=[a(1)a(m)]=σ(Z)A = [a^{(1)} \dots a^{(m)}] = \sigma(Z)

The shape of ZZ and AA is (1,m)(1, m). The derivatives of the loss function L\mathcal{L} with respect to ZZ, ww, and bb are:

dLdZ=AY\frac{d\mathcal{L}}{dZ} = A - Y

dLdw=1mX(dLdZ)T\frac{d\mathcal{L}}{dw} = \frac{1}{m} X \left(\frac{d\mathcal{L}}{dZ}\right)^T

dLdb=1mdLdZ\frac{d\mathcal{L}}{db} = \frac{1}{m} \sum \frac{d\mathcal{L}}{dZ}

0

1

Updated 2026-06-07

Tags

Data Science