The Modern Mathematics of Deep Learning

A problem in the printed representation

Penguin learning project

The project uses 342 complete Palmer penguin records: 240 training-pool and 102 held-out rows. The selected training rows determine feature scaling and model parameters. Source CSV

This project trains weights with softmax, unlike the homepage lab, which uses nearest species averages. Same penguin measurements; different classification methods. Predictions can differ.

Approximation and depth · Study approximation error and network size under stated conditions.

The project supplies finite observations and fitted parameters. A held-out sample does not establish population bounds or prove a theorem.

Consider this concept alongside the project; its source assumptions may differ. These observations do not prove a theorem.

Input: 12 × 2; Held-out evaluation: 102 · #58 · 40.6 mm, 193 mm. Known answer: Adélie; Prediction: Adélie · [≈0.7383, ≈0.2444, ≈0.01737].

Used in these source sections: 4.2 Approximating with random pieces

The paper prints this equality with Γ equal to a sign times the difference of two indicators. Therefore |Γ| ≤ 1, so the absolute value of the right side cannot exceed Cg.

For g(x) = exp(−πx²), the paper’s Fourier convention gives ĝ(ξ) = exp(−πξ²) and Cg = 1/π. At x = 1/2, the absolute difference 1−exp(−π/4) is larger than 1/π.

This counterexample identifies a mismatch in the printed formula.

Source formula example

g(x)-g(0)=C_{g}\int_{{\mathbb{R}}^{d}}\frac{1}{\|\xi\|_{2}}(e^{2\pi i\langle x,\xi\rangle}-1)\,\mathrm{d}\mu_{g}(\xi)=C_{g}{\mathbbm{E}}\big[\Gamma(\Xi,\widetilde{\Xi})(x)\big],\quad x\in{\mathbb{R}}^{d},

exact · This is a counterexample to the printed equality with the printed definitions. It does not identify a correction or disprove the Barron approximation theorem.

Symbols

$g$: Here g(x)=exp(−πx²), an integrable function.
$C_g$: The source integral ∫|ξ||ĝ(ξ)|dξ, equal to 1/π for this Gaussian.
$Γ$: The printed difference of two indicators, multiplied by a sign in {−1,1}.

Source formula example

This source example uses its own inputs and assumptions. Editing it does not change the project.

Edit a value or the code, then run the example. This is a finite demonstration, not a proof of the paper’s theorem.

x

This example’s data flow

You choose

input.x = 0.5

Calculated here

Result expression: { g, absoluteDifference: difference, maximumPossibleRightSide: Cg, exceedsBound: difference > Cg, }

x = input.x Uses: input.x
g = Math.exp(-Math.PI * x * x) Uses: x
difference = Math.abs(g - 1) Uses: g
Cg = 1 / Math.PI

Returns

Example result

g: ≈0.4559
absoluteDifference: ≈0.5441
maximumPossibleRightSide: ≈0.3183
exceedsBound: true

JavaScript

Read numbers from input and return the result. JavaScript basics.

Result

Enable JavaScript to run this example.

Technical statement

The source defines Γ using two 0-or-1 indicators and a sign ±1. Consequently Γ lies between −1 and 1, and every expectation of Γ also has absolute value at most 1. The Gaussian has a finite Fourier moment Cg = ∫ℝ|ξ| exp(−πξ²)dξ = 1/π. At x=1/2, which lies in the unit ball, the printed equality would require a right-hand side of magnitude greater than Cg. The printed equation and Γ definition were checked against the PDF. This does not determine which factor or definition should change.

Source: Source expression