The Modern Mathematics of Deep Learning

Glossary

Symbols have meanings that depend on their mathematical context.

$\sum$ Sum

Add the listed terms.

Explore the example

$\langle x,\theta\rangle$ Dot product

Multiply matching vector entries and add.

Explore the example

$\|x\|_2$ Euclidean norm

Square entries, add, and take the square root.

Explore the example

$A x$ Matrix-vector product

Take the dot product of each matrix row with the vector.

Explore the example

$\nabla r$ Gradient

A vector of rates of change of r with respect to its inputs.

Explore the example

$\ell$ Loss

A number describing prediction error for one example.

Explore the example

$\widehat{\mathcal R}_{s}$ Empirical risk

The mean loss over a finite sample s.

Explore the example

$\varrho_R$ ReLU

The larger of zero and the input.

Explore the example

$\gamma(\theta)$ Margin

For the linked linear binary classifier, the smallest label times prediction.

Explore the example

$\|\phi\|_0$ Nonzero count

The number of nonzero entries in a vector.

Explore the example

$\mu_b,\sigma_b$ Batch mean and standard deviation

The mean and standard deviation of values in a batch.

Explore the example

$K$ Forward map

A rule that maps an unknown candidate to predicted observations.

Explore the example

Glossary

∑\sum∑ Sum

⟨x,θ⟩\langle x,\theta\rangle⟨x,θ⟩ Dot product

∥x∥2\|x\|_2∥x∥2​ Euclidean norm

AxA xAx Matrix-vector product

∇r\nabla r∇r Gradient

ℓ\ellℓ Loss

R^s\widehat{\mathcal R}_{s}Rs​ Empirical risk

ϱR\varrho_RϱR​ ReLU

γ(θ)\gamma(\theta)γ(θ) Margin

∥ϕ∥0\|\phi\|_0∥ϕ∥0​ Nonzero count

μb,σb\mu_b,\sigma_bμb​,σb​ Batch mean and standard deviation

KKK Forward map

$\sum$ Sum

$\langle x,\theta\rangle$ Dot product

$\|x\|_2$ Euclidean norm

$A x$ Matrix-vector product

$\nabla r$ Gradient

$\ell$ Loss

$\widehat{\mathcal R}_{s}$ Empirical risk

$\varrho_R$ ReLU

$\gamma(\theta)$ Margin

$\|\phi\|_0$ Nonzero count

$\mu_b,\sigma_b$ Batch mean and standard deviation

$K$ Forward map