The Modern Mathematics of Deep Learning

Three generalization scales

Penguin learning project

The project uses 342 complete Palmer penguin records: 240 training-pool and 102 held-out rows. The selected training rows determine feature scaling and model parameters. Source CSV

This project trains weights with softmax, unlike the homepage lab, which uses nearest species averages. Same penguin measurements; different classification methods. Predictions can differ.

Generalization and risk bounds · Relate sample behavior to population risk under stated assumptions.

The project supplies finite observations and fitted parameters. A held-out sample does not establish population bounds or prove a theorem.

Consider this concept alongside the project; its source assumptions may differ. These observations do not prove a theorem.

Input: 12 × 2; Held-out evaluation: 102 · #58 · 40.6 mm, 193 mm. Known answer: Adélie; Prediction: Adélie · [≈0.7383, ≈0.2444, ≈0.01737].

Used in these source sections: 1.2 Learning from examples

The symbol ≲ hides a positive constant. These expressions describe scales rather than exact numerical ceilings.

The code calculates the visible factors for a finite function class, a fixed-depth network estimate, and a confidence term.

They show how sample size and complexity enter generalization estimates.

Source formula example

{\varepsilon}^{\mathrm{gen}}\lesssim\sqrt{\frac{\ln(|{\mathcal{F}}|)}{m}}.

illustration · These are displayed factors only. Hidden constants and additional conditions mean the values are not numerical guarantees by themselves.

Symbols

$ε^{gen}$: Supremum of the absolute population-versus-sample risk gap over the function class.
$|ℱ|$: Size of a finite hypothesis class.
$P(N)$: Number of network parameters.
$δ$: Failure probability in (0,1).

Source formula example

This source example uses its own inputs and assumptions. Editing it does not change the project.

Edit a value or the code, then run the example. This is a finite demonstration, not a proof of the paper’s theorem.

Number of functionsNumber of parametersSample sizeFailure probability δ

This example’s data flow

You choose

input.count = 4
input.parameters = 4
input.m = 4
input.delta = 0.5

Calculated here

Result expression: { finiteClassScale: Math.sqrt(Math.log(count) / m), networkScale: Math.sqrt((parameters * Math.log(parameters)) / m), concentrationScale: Math.sqrt(Math.log(1 / delta) / m), }

count = input.count Uses: input.count
parameters = input.parameters Uses: input.parameters
m = input.m Uses: input.m
delta = input.delta Uses: input.delta

Returns

Example result

finiteClassScale: ≈0.5887
networkScale: ≈1.177
concentrationScale: ≈0.4163

JavaScript

Read numbers from input and return the result. JavaScript basics.

Result

Enable JavaScript to run this example.

Technical statement

The finite-class scale follows the countable weighted-class theorem under its bounded-loss and i.i.d. sampling assumptions, with equal prior weights. The network expression is a fixed-depth piecewise-polynomial classification estimate. The concentration expression controls deviation from its expectation with high probability under the source assumptions; constants are hidden in ≲, and the source notes measurability conditions for the expectation.

Source: Source expression