The Modern Mathematics of Deep Learning

Three learned inverse components

Penguin learning project

The project uses 342 complete Palmer penguin records: 240 training-pool and 102 held-out rows. The selected training rows determine feature scaling and model parameters. Source CSV

This project trains weights with softmax, unlike the homepage lab, which uses nearest species averages. Same penguin measurements; different classification methods. Predictions can differ.

Inverse problems · Describe reconstruction from observations using a forward operator.

This application has its own inputs and assumptions; the penguin measurements are not its experimental data.

Consider this concept alongside the project; its source assumptions may differ. These observations do not prove a theorem.

Input: 12 × 2; Held-out evaluation: 102 · #58 · 40.6 mm, 193 mm. Known answer: Adélie; Prediction: Adélie · [≈0.7383, ≈0.2444, ≈0.01737].

Used in these source sections: 8.1 Working back from measurements

A model can map observations to candidates, refine candidates, or return a nonnegative regularizer value.

These are distinct ways to include learned functions in an inverse problem.

Source formula example

\Phi(\cdot,\theta)\colon{\mathcal{X}}\to{\mathcal{Y}}

illustration · These invented functions are not trained models and do not establish a general reconstruction guarantee.

Symbols

$X$: Observation space; real numbers in the toy.
$Y$: Candidate space; real numbers in the toy.
$Φ$: A parameterized model; the toy chooses three simple functions.
$θ$: Model parameters, fixed in this toy.

Source formula example

This source example uses its own inputs and assumptions. Editing it does not change the project.

Edit a value or the code, then run the example. This is a finite demonstration, not a proof of the paper’s theorem.

Observation

This example’s data flow

You choose

input.observation = 3

Calculated here

Result expression: { observation, candidate, refined, regularizer }

observation = input.observation Uses: input.observation
candidate = observation / 2 Uses: observation
refined = candidate Uses: candidate
regularizer = candidate ** 2 Uses: candidate

Returns

Example result

observation: 3
candidate: 1.5
refined: 1.5
regularizer: 2.25

JavaScript

Read numbers from input and return the result. JavaScript basics.

Result

Enable JavaScript to run this example.

Technical statement

The three toy functions are x ↦ x/2 from X to Y, y ↦ y from Y to Y, and y ↦ y² from Y to nonnegative numbers. They are examples of the source’s separate model roles.

Source: Source expression