The Modern Mathematics of Deep Learning

Checking size hypotheses

Penguin learning project

The project uses 342 complete Palmer penguin records: 240 training-pool and 102 held-out rows. The selected training rows determine feature scaling and model parameters. Source CSV

This project trains weights with softmax, unlike the homepage lab, which uses nearest species averages. Same penguin measurements; different classification methods. Predictions can differ.

Approximation and depth · Study approximation error and network size under stated conditions.

The project supplies finite observations and fitted parameters. A held-out sample does not establish population bounds or prove a theorem.

Consider this concept alongside the project; its source assumptions may differ. These observations do not prove a theorem.

Input: 12 × 2; Held-out evaluation: 102 · #58 · 40.6 mm, 193 mm. Known answer: Adélie; Prediction: Adélie · [≈0.7383, ≈0.2444, ≈0.01737].

Used in these source sections: 3.1 A rule based on distance

The source depth-separation theorem specifies size bounds for its constructed target network and parameters.

The toy checks the bounds for an identically zero network.

Explicit size conditions state which networks a theorem describes.

Source formula example

\|\theta^{*}\|_{\infty}\leq Cd^{C}

illustration · The zero network satisfies these isolated size conditions but is not the theorem’s constructed hard target. This does not establish depth separation.

Symbols

$θ^*$: Parameters of the target network constructed in the source theorem.
$C$: A positive constant in the theorem.
$d$: Positive input dimension.

Source formula example

This source example uses its own inputs and assumptions. Editing it does not change the project.

Edit a value or the code, then run the example. This is a finite demonstration, not a proof of the paper’s theorem.

Constant CDimension d

This example’s data flow

You choose

input.C = 2
input.d = 2

Initial values in code

parameters = [0, 0]
outputSupremum = 0

Calculated here

Result expression: { parameterNorm, parameterBound: C * d ** C, widthNorm, widthBound: C * d ** 5, outputSupremum, outputBound: 2, }

C = input.C Uses: input.C
d = input.d Uses: input.d
parameterNorm = Math.max(...parameters.map(Math.abs)) Uses: parameters
widths = [d, 1, 1, 1] Uses: d
widthNorm = Math.max(...widths) Uses: widths

Returns

Example result

parameterNorm: 0
parameterBound: 8
widthNorm: 2
widthBound: 64
outputSupremum: 0
outputBound: 2

JavaScript

Read numbers from input and return the result. JavaScript basics.

Result

Enable JavaScript to run this example.

Technical statement

The source asserts existence of a target satisfying these bounds and a separate lower bound on shallow-network approximation. Satisfying the displayed size conditions alone does not imply that lower bound. The supremum of the toy zero network is known exactly over all real inputs.

Source: Theorem hypothesis