Machine Learning on Eigenform Articles

Machine Learnability as a Measure of Order

Tue, 07 Oct 2025 00:00:00 +0800

TL;DR

Opens from a philosophy-of-maths framing (is maths invented or discovered) to justify treating prime-number patterns as an empirical dataset machine learning can study, rather than a purely logical structure.
Represents primes visually as an Ulam spiral, and shows that an FFT of the spiral reveals fractal-like structure, suggesting multiple overlapping rule sets rather than pure randomness.
Trains the same simple architecture (a 2D U-Net with a ResNet-34 encoder, doing image in-painting) separately on Ulam spiral blocks from seven numeric ranges between 25 million and 500 million, then cross-tests every model on every range’s test set, averaging over repeated runs and resampled confidence intervals to control for the shifting prime density at different scales.
Because prime pixels make up only around 5-6% of any given image, the study measures mean class accuracy rather than raw accuracy to avoid rewarding a trivial “predict all composite” model; the resulting numbers show the model favouring direct prime-pattern recognition at lower magnitudes and elimination-by-composite reasoning at higher ones, confirmed by precision, recall, and F1 decomposition.
Prime and composite F1 curves turn out to be mirror images rather than identical, read as evidence that primality is not simply “not compositeness” but a separately structured phenomenon; a sharp step-change in learnability appears between the 25m and 50m ranges, and learnable order declines noticeably faster than prime density itself does.
The same unmodified architecture also performs strongly on an unrelated task, geological fault-line prediction, which the piece treats as pointing toward a broader technique - using model uncertainty and feature importance to work backward toward the structural “sieves” driving predictions in either domain - alongside speculative cryptographic and Riemann/Chowla-conjecture applications the team says it has not yet pursued.

Key Takeaways

Machine Learnability as a Measure of Order in Aperiodic Sequences

Tue, 09 Sep 2025 00:00:00 +0800

TL;DR

Prime number distribution is deterministic in definition but behaves statistically like a random process, and this paper uses an image-based machine learning model to measure comparative regularity in prime fields at different regions of an Ulam spiral.
Models trained on blocks from the region around 500m outperform models trained on blocks below 25m, implying there is more easily learnable order in the former region than the latter.
Precision and recall scores diverge by region: the model leans toward identifying prime patterns directly at lower numbers and toward eliminating composites at higher numbers, suggesting it favours different classification strategies depending on scale.
These findings align with number theory conjectures that local randomness in prime distribution regularises after scaling by log x, with averages such as density and arithmetic-progression equidistribution coming to dominate at higher magnitudes.
The authors suggest machine learning accuracy itself can serve as an experimental instrument for number theory, offering a way to probe order that isn’t derived analytically.
The method is noted as potentially useful for investigating patterns in strong and weak primes relevant to cryptography.

Key Takeaways

Generalising from Self-Produced Data: Model Training Beyond Human Constraints

Mon, 07 Apr 2025 00:00:00 +0800

TL;DR

Current LLMs are bounded by human-derived training data and by a single level of abstraction that prevents definitive truth judgments about their own outputs.
The paper proposes a framework in which AI agents generate and validate new knowledge directly through interaction with their environment, rather than through further human-curated data.
The central device is an unbounded, ungamable numeric reward - such as annexed disk space or follower count - that requires no human-defined benchmark.
Agents iteratively generate strategies and executable code to maximise this metric, and successful outcomes become the basis for self-retraining.
Empirical validation (does the outcome actually hold up) is used in place of textual similarity to guard against model collapse and the “warm start” problem, with fine-tuning carried out via GRPO.
The system is built from modular agents handling environment analysis, strategy generation, and code synthesis separately, to allow scalable experimentation.

Key Takeaways

Defining T-Schemas via the Parametric Encoding of Second Order Languages in AI Models

Sat, 15 Feb 2025 00:00:00 +0800

TL;DR

The article argues that AI models cannot verify truth from purely within their own formal system (Tarski’s undefinability of truth), so scaling models and data that are entirely human-produced caps their intelligence near - or below - the humans who produced the training and benchmark data.
Two workarounds are proposed: reality-testing against an ungamable external metric the model can act on and observe the results of (echoing the site’s other work on disk space or crypto-wallet balances as reward signals), and building an internal metalanguage - a T-schema - against which future claims can be judged, the way Newton derived formulae from Kepler’s data.
The piece argues that recent evidence about “grokking” - the phenomenon where models suddenly generalise well after being trained far past the point of overfitting - shows models doing exactly this: building an internal higher-order abstraction rather than only memorising.
It cites the Grokfast team’s finding that memorisation and generalisation correspond to distinct fast and slow frequencies in weight updates, and a separate study showing that removing a cluster of training data sharply reduces generalisation while adding a few examples back restores it.
Further cited evidence links grokking to the model discovering a lower-rank encoding solution - achieving more with fewer effective features - which the piece reads as evidence of an internal abstraction layer.
It closes with the company’s own plan: train diffusion models, which reportedly show grokking-like behaviour throughout training and outperform on compositional and coding tasks, on data generated by its own “Generalising Agents,” as a route toward independent knowledge generation.

Key Takeaways