So many inputs!

A photograph might have thousands of pixel values, yet the photos we care about are not every possible arrangement of those pixels. Consider a five-square sign with a single lit square that can move left or right.

Give each coordinate ten possible values. Multiply ten choices for every coordinate.

function gridPoints(choices, dimensions) {
  return { points: choices ** dimensions };
}
const result = gridPoints(10, 3);

The sign has five numbers but only five possible patterns: one for each lit position. Five separate choices do not make a smooth line. Now change the toy: let one brightness dial move all five numbers together, and let it take any value in a range. Those pictures form a continuous line.

One brightness dial sets all five pixels to the same value. The picture needs five numbers, but we can choose only one freely. As t moves continuously, the pictures form a line. Its intrinsic dimension is 1: one free number is enough to locate a picture on that line.

function sharedBrightness(brightness, count) {
  return { pixels: Array(count).fill(brightness), controls: 1 };
}
const result = sharedBrightness(2, 5);

The paper calls this the manifold assumption. If real inputs mostly lie on a smooth lower-dimensional surface, it may be enough to learn the rule on that surface rather than across every possible pixel combination.

This is an assumption about the data, not a property guaranteed for all pictures. The theorem constructions work locally on pieces of the surface and combine those pieces.

Another approach builds an approximation from many small rules. The rules and their weights are chosen for the particular target we want to approximate.

For a particular target, the paper builds small contributions g_j whose weights are already included. At one input, suppose three contributions return 1, 2 and 3. Select the first two and average their outputs.

function sampleAverage(values, count) {
  if (!Number.isInteger(count) || count <= 0 || count > values.length) {
    throw new Error('Count must be a whole number between 1 and array length');
  }
  let total = 0;
  for (let index = 0; index < count; index++) total += values[index];
  return { average: total / count };
}
const result = sampleAverage([1, 2, 3], 2);

In the calculation above, selecting 1 and 2 gives (1 + 2)/2 = 1.5. The flower activity also averages selected values. Its pots show why a sample average can differ from the whole group; adding a pot need not improve the estimate each time. This illustrates averaging, not the paper’s approximation guarantee.

The paper gives a ceiling for expected squared error. A number C_g describes the chosen target; it is not a knob that makes any problem easy. For this calculation choose C_g = 4 and n = 4 small rules. The symbol π is the circle constant, about 3.1416.

function errorBound(targetComplexity, numberOfTerms) {
  const squaredErrorBound = (2 * Math.PI * targetComplexity) ** 2
    / numberOfTerms;
  return { squaredErrorBound };
}
const result = errorBound(4, 4);

The ceiling above is for expected squared error and falls like 1/n. Taking its square root gives a root-mean-square error ceiling: average the squared errors, then take the square root. This ceiling falls like 1/√n. The paper needs a condition on the target’s Fourier representation, a way to write a rule as a mixture of waves. The constant depends on the target.

For C_g = 4 and n = 4, the ceiling (2πC_g)²/n is about 157.91. Using 16 terms divides that ceiling by four, to about 39.48. Under the theorem’s conditions, the expected squared error stays below the ceiling. The error in one particular sample need not decrease every time we add a term.

Sometimes a problem is hard because it contains many changing values, but its rule comes from a scientific equation. A heat equation, for example, tells how temperature spreads.

Imagine a rod with a known initial temperature at each position. To estimate the later temperature at one position, the formula starts random paths there and reads the initial temperature at each path’s endpoint. These are mathematical samples, not claims that heat physically travels backward.

In the paper’s formula, g(x,t) is the later value, φ is the initial temperature, S is a random path endpoint and n is the number of sampled paths. For a worked example, the endpoints give initial temperatures 2 and 4 degrees.

function averageTemperature(first, second) {
  return { sampleAverage: (first + second) / 2 };
}
const result = averageTemperature(2, 4);

Suppose two sampled paths end at positions whose initial temperatures are 2°C and 4°C. Their average is (2 + 4)/2 = 3°C. It estimates the later temperature where the paths started, using initial temperatures where they ended. The positions themselves are not temperatures.

The paper calls a related way to express solutions of certain equations the Feynman–Kac representation. Networks can approximate the paths, initial temperatures and averages. For the particular equations and conditions studied, this can sometimes avoid costs that grow exponentially with the number of inputs. Two sampled paths illustrate the idea; they do not solve a physical problem.