The Modern Mathematics of Deep Learning

Fourier representation and sampling weights

Penguin learning project

The project uses 342 complete Palmer penguin records: 240 training-pool and 102 held-out rows. The selected training rows determine feature scaling and model parameters. Source CSV

This project trains weights with softmax, unlike the homepage lab, which uses nearest species averages. Same penguin measurements; different classification methods. Predictions can differ.

Approximation and depth · Study approximation error and network size under stated conditions.

The project supplies finite observations and fitted parameters. A held-out sample does not establish population bounds or prove a theorem.

Consider this concept alongside the project; its source assumptions may differ. These observations do not prove a theorem.

Input: 12 × 2; Held-out evaluation: 102 · #58 · 40.6 mm, 193 mm. Known answer: Adélie; Prediction: Adélie · [≈0.7383, ≈0.2444, ≈0.01737].

Used in these source sections: 4.2 Approximating with random pieces

The source rewrites this integral using a frequency-weighted measure, then expresses it as an expectation of random features.

For a one-dimensional Gaussian g(x)=exp(−πx²), the Fourier transform is the same Gaussian and Cg=1/π. We compare a four-point midpoint sum with the exact difference.

The rewriting connects Fourier regularity with random-feature approximation.

Source formula example

\begin{split}g-g(0)&=\int_{{\mathbb{R}}^{d}}\hat{g}(\xi)(e^{2\pi i\langle\cdot,\xi\rangle}-1)\,\mathrm{d}\xi\\ &=C_{g}\int_{{\mathbb{R}}^{d}}\frac{1}{\|\xi\|_{2}}(e^{2\pi i\langle\cdot,\xi\rangle}-1)\frac{1}{C_{g}}\|\xi\|_{2}\hat{g}(\xi)\,\mathrm{d}\xi\\ &=C_{g}\int_{{\mathbb{R}}^{d}}\frac{1}{\|\xi\|_{2}}(e^{2\pi i\langle\cdot,\xi\rangle}-1)\,\mathrm{d}\mu_{g}(\xi),\end{split}

approximation · The four-point sum truncates the integral and uses a coarse step. It is not the full integral, an exact expectation, or a proof of the source representation.

Symbols

$ξ$: A frequency in ℝᵈ; the toy is one-dimensional.
$\hat g$: Fourier transform of g.
$C_g$: The integral of frequency length times Fourier magnitude.
$μ_g$: The source frequency-weighted measure.

Source formula example

This source example uses its own inputs and assumptions. Editing it does not change the project.

Edit a value or the code, then run the example. This is a finite demonstration, not a proof of the paper’s theorem.

Input x

This example’s data flow

You choose

input.x = 0.5

Initial values in code

frequencies = [-1.5, -0.5, 0.5, 1.5]

Calculated here

Result expression: { exactDifference: Math.exp(-Math.PI * x * x) - 1, realSum, imaginarySum, Cg, weightedReal, gIntegral: 1, }

x = input.x Uses: input.x
realTerms = frequencies.map( (xi) => Math.exp(-Math.PI * xi * xi) * (Math.cos(2 * Math.PI * x * xi) - 1), ) Uses: frequencies, x
imaginaryTerms = frequencies.map( (xi) => Math.exp(-Math.PI * xi * xi) * Math.sin(2 * Math.PI * x * xi), ) Uses: frequencies, x
realSum = realTerms.reduce((a, b) => a + b, 0) Uses: realTerms
imaginarySum = imaginaryTerms.reduce((a, b) => a + b, 0) Uses: imaginaryTerms
Cg = 1 / Math.PI
weightedReal = Cg * frequencies.reduce( (sum, xi) => sum + ((Math.cos(2 * Math.PI * x * xi) - 1) / Math.abs(xi)) * ((Math.abs(xi) * Math.exp(-Math.PI * xi * xi)) / Cg), 0, ) Uses: Cg, frequencies, x

Returns

Example result

exactDifference: ≈-0.5441
realSum: ≈-0.9136
imaginarySum: 0
Cg: ≈0.3183
weightedReal: ≈-0.9136
gIntegral: 1

JavaScript

Read numbers from input and return the result. JavaScript basics.

Result

Enable JavaScript to run this example.

Technical statement

The source assumes g∈L¹(ℝᵈ) and finite Cg, with Fourier-transform conditions used in its derivation. It rewrites the integral using the weight ‖ξ‖|ĝ(ξ)|/Cg and then a random threshold feature representation. The toy uses the Gaussian Fourier identity and a symmetric midpoint sum over [−2,2]; it approximates the first integral only.

Source: Source expression