The Modern Mathematics of Deep Learning

A universal approximation condition

Penguin learning project

The project uses 342 complete Palmer penguin records: 240 training-pool and 102 held-out rows. The selected training rows determine feature scaling and model parameters. Source CSV

This project trains weights with softmax, unlike the homepage lab, which uses nearest species averages. Same penguin measurements; different classification methods. Predictions can differ.

Approximation and depth · Study approximation error and network size under stated conditions.

The project supplies finite observations and fitted parameters. A held-out sample does not establish population bounds or prove a theorem.

Consider this concept alongside the project; its source assumptions may differ. These observations do not prove a theorem.

Input: 12 × 2; Held-out evaluation: 102 · #58 · 40.6 mm, 193 mm. Known answer: Adélie; Prediction: Adélie · [≈0.7383, ≈0.2444, ≈0.01737].

Used in these source sections: 1.2.2 Can the network express the answer?

The source theorem asks whether every continuous function on a compact set can be approximated arbitrarily well by two-layer networks.

For the toy K=[0,1], target g(x)=x is represented exactly by ReLU(x); its uniform error is zero.

The theorem connects an activation’s properties to which functions networks can approximate.

Source formula example

C(K)\subset\operatorname{cl}(\widetilde{{\mathcal{F}}})

illustration · One exactly represented function is not a proof of density for every continuous function. The sample point is not used to infer the supremum; the zero error follows analytically for all x in [0,1].

Symbols

$C(K)$: Continuous real functions on compact K.
$\operatorname{cl}(\widetilde{\mathcal{F}})$: Closure of the network class in the stated norm.
$L^∞_{loc}$: Functions essentially bounded on every compact subset.

Source formula example

This source example uses its own inputs and assumptions. Editing it does not change the project.

Edit a value or the code, then run the example. This is a finite demonstration, not a proof of the paper’s theorem.

x

This example’s data flow

You choose

input.x = 0.5

Calculated here

Result expression: { activation, target, polynomial, exactUniformError: 0, localBoundOnMinus2To2: 2, }

x = input.x Uses: input.x
activation = Math.max(0, x) Uses: x
target = x Uses: x
polynomial = x ** 2 Uses: x

Returns

Example result

activation: 0.5
target: 0.5
polynomial: 0.25
exactUniformError: 0
localBoundOnMinus2To2: 2

JavaScript

Read numbers from input and return the result. JavaScript basics.

Result

Enable JavaScript to run this example.

Technical statement

The source assumes K compact and an activation that is locally essentially bounded, with closure of discontinuities a Lebesgue null set. It states that the network class is dense in C(K) in the L∞(K) topology if and only if the activation is not equal almost everywhere to a polynomial. ReLU is locally bounded and continuous; the toy shows only one target. The polynomial x² is evaluated to illustrate the notation, not to establish the non-polynomial condition by a single point.

Source: Source expression