Home on Eigenform Articles

Which AI Model Is Best for Your Geology?

Mon, 24 Aug 2026 00:00:00 +0800

TL;DR

Argues that standard academic AI benchmarks are stale (“an exam paper everyone has had access to for years”) and largely irrelevant to a real decision like spending $100,000 integrating an AI system into an exploration workflow, since geology is hyper-local and a general fact, such as bismuth being a gold pathfinder, may not hold in a specific terrain.
Introduces the Groundtruth Dynamic Benchmark, which generates fresh questions and scoring rubrics directly from a user’s own document set rather than testing against a fixed, generic question bank.
Offers three free sample test sets (Murchison WAMEX gold reports, USGS Southwestern US reports, Canadian technical reports) alongside support for fully custom datasets, with generated questions spanning four types - finding facts, reconstructing sequences of geological events, reconciling sources that disagree, and recognising when the evidence doesn’t support a firm answer - graded against a rubric rather than accepted for sounding persuasive.
Reports the results of a six-model bake-off: Kimi K3 scored highest overall, but confidence intervals overlap enough among the top three models that the benchmark can only reliably sort models into tiers rather than a strict ranking, and that tiering held consistently across different source-document types.
Given the near-tied top performance, shifts the comparison to cost: generation cost dwarfs judging cost and varies sixfold between models doing the same task, and plotting price against performance puts four models on a shared frontier of roughly equivalent return on cost - until a newly released model, Ox Alpha, is added, whose free-tier pricing puts it alone on a sharply shifted frontier ahead of everything else.
Concludes with the practical payoff: an exploration manager can benchmark any model-plus-retrieval or model-plus-agent combination against their own frozen, company-specific test set, and rerun that same benchmark later to check whether a proposed cheaper migration actually degrades performance on their own work, rather than relying on vendor claims or generic scores.

Key Takeaways

How Not to Build a Slop Cannon

Thu, 20 Aug 2026 00:00:00 +0800

TL;DR

Argues against the instinct to fix agent errors by stacking more warnings into the prompt or CLAUDE.md file: long contexts degrade into contradictory noise, so the fix belongs at the harness level, not the prompt level.
Lays out five harness-design principles built on the premise that no single run will ever be perfect, so the goal is making errors cheap to catch rather than trying to eliminate them: split work across smaller sub-agents that check each other’s outputs; use a QC pass with a checklist instead of a “don’t make mistakes” instruction; design for the smallest model that works, both for cost and to reduce fragility to upstream guardrail changes; tag outputs by confidence level (confirmed, inferred, assumed) so uncertainty stays visible instead of masked by fluent prose; and treat chat history as something to actively manage and feed as an explicit variable rather than something to passively accumulate.
Case study: NextMaps stress-tested its exploration analytics pipeline on a deliberately hard Western Australian gold licence with three decades of historical reports whose boundaries had shifted repeatedly, where a naive system would misattribute nearby major gold discoveries to the licence being assessed.
The harness pre-computes each location’s real distance to the licence boundary before the model ever sees the text, and runs a separate verification pass against that ground truth, so the model writes the report but does not decide what counts as “inside.”
The resulting report rated the licence highly prospective while explicitly declining to justify that rating using a million ounces of gold on neighbouring leases, building its case instead from evidence physically inside the boundary and flagging what the data couldn’t support.
A postscript notes that the specific target the pipeline flagged from historical public records alone later turned out, per the operator’s own investor materials, to be the largest source of projected upside on the project - though the pipeline had no way of knowing about the active drilling now underway there.

Key Takeaways

Macrotrends to Watch Out for in the AI Exploration Space

Thu, 13 Aug 2026 00:00:00 +0800

TL;DR

Cites a recent preprint on AI for Mineral Exploration surveying how classical machine learning, deep learning, and foundation models are being applied across the exploration pipeline to handle extreme label scarcity and heterogeneous geoscientific data.
Draws five research-side trends from that preprint: prospectivity models moving from flat 2D maps toward 3D/4D representations incorporating geological time; single-modality analysis giving way to models that reason across geology, geochemistry, geophysics, remote sensing, and text together; supervised learning giving way to self-/semi-supervised and positive-unlabelled approaches given how few labelled deposits exist; static prediction giving way to uncertainty-driven exploration that recommends the next survey or drillhole; and bespoke single-purpose models giving way to reusable geoscience foundation models paired with tool-using agents.
Cites a companion Xplorahub piece on the commercial side, which names Eigenform and NextMaps among platforms providing spatial analytics and ground intelligence to explorers and investors.
Draws six commercial-side trends: AI activity clustering around the drill-targeting decision because it has the clearest economic case; some companies moving from selling software to vertically integrating exploration itself via mineral rights, joint ventures, or royalties; others pairing models with proprietary data-acquisition hardware such as satellites and automated core scanners; major miners like BHP and Rio Tinto behaving more like venture investors than customers; consolidation of standalone AI and sensing startups into broader mining-tech platforms; and a widening gap between AI performance claims and independently validated economic discoveries.
Points readers to both source pieces - the preprint and the Xplorahub article - for the full technical and commercial detail rather than presenting new research of its own.

Key Takeaways

Geological Map Processing Suite

Mon, 10 Aug 2026 01:00:00 +0800

TL;DR

Geological maps are difficult VLM inputs beyond sheer scale: a map sheet’s legend, cross-section, correlation diagram, graticule, and projection notes are mutually referential, so a simple question like “what’s the dominant lithology in this quadrant” requires multi-step reasoning.
Tiling or downsampling addresses the scale problem but worsens the compositional one, since it can separate a polygon from the legend entry it depends on to be interpreted correctly.
Existing approaches are surveyed: AI4CMA and DIGMAPPER (and Eigenform’s own tools) treat this mainly as a vectorisation problem, while Microsoft’s PEACE takes a multimodal-LLM approach, decomposing map QA into information extraction, domain-knowledge integration, and answer generation, reporting its GeoMap-Agent scoring 0.811 against GPT-4o’s 0.369 on its own 124-map, 3,864-question GeoMap-Bench.
Eigenform’s objection to PEACE is about packaging rather than accuracy: its repo ships as a fixed end-to-end QA tool and benchmark, not a modular toolkit.
The post introduces “Stratigraphic Amenity,” which repackages PEACE into a small Python SDK plus a local MCP server exposing map layout and legend detection, geo-referencing, knowledge retrieval, and overlay operations, deliberately stopping short of OCR, VLM inference, or answer construction so users can bring their own agent and harness.

Key Takeaways

How Reports From The Future Draws Its Star Maps

Mon, 10 Aug 2026 00:00:00 +0800

TL;DR

Reports From The Future maps AI-related vocabulary as it emerges, aiming to spot “semantic basins of attraction” - ideas circulating in tech discourse before they have a settled name - across six months of arXiv papers and three months of web discourse (X, HuggingFace blogs, Hacker News, LessWrong).
All three plates share a pipeline: documents are embedded into vectors, UMAP reduces them to a drawable space (fit once and frozen, so the sky doesn’t scramble week to week), and HDBSCAN clusters dense regions into constellations, leaving outliers as unassigned “dust” rather than forcing them into a group.
Plate I (the Research Sky) names constellations with TF-IDF only, and corrects each month’s colour for publishing rate rather than raw count - a fix that moves August from 6% to about 70% of the May peak once partial and weekend-heavy days are properly weighted.
Plate II (the Attention Sky) runs HDBSCAN twice, once over a rolling 91-day background for context and once per week at a smaller threshold for the constellations actually drawn, and names clusters with raw TF-IDF terms on the view that a badly-formed name is itself information about unsettled vocabulary.
Plate III (the Emergent Sky) reuses Plate II’s clusters but adds a “name gap” metric - the percentile-rank difference between semantic and lexical coherence - to separate genuinely emerging concepts (high shared meaning, low shared vocabulary) from buzzwords (high shared vocabulary, low shared meaning), each routed through a different, deliberately inverted naming prompt.
The naming pipeline for Plate III runs four models independently to describe each cluster, then has one model synthesise a name from where their descriptions converge, and a final model distil that into a short phrase - illustrated with a real example of “Claude” being described mid-genericide as a term coming apart rather than settling on a single referent.

Key Takeaways

Groundtruth Dynamic Benchmarking (Geology Edition) v1.0.0 Is Live

Fri, 31 Jul 2026 02:00:00 +0800

TL;DR

Announces v1.0.0 of the Groundtruth Dynamic Benchmarking System on GitHub: not a fixed set of benchmark problems, but a pipeline for generating reliable benchmark questions and scoring rubrics for whatever corpus you’re working with.
Ships a working example grounded in a real, openly licensed corpus - 34 mineral deposit records from the Yudnamutana Copper district, South Australia - runnable end-to-end with a single script after cloning the repo and adding API keys.
Uses a rubric format built to resist grade inflation: a hard gate per question, non-overlapping graduated components, required concepts, and do-not-credit lists, with the underlying mechanics covered in the companion post “Dynamic Benchmarking: How We Did It.”
Grading is structured and auditable: the judge returns a full breakdown of which components it credited and why, and flags anything it can’t verify from the rubric alone rather than guessing.
The corpus-to-rubric authoring pipeline itself ships in the repo, so anyone can point it at their own corpus and build their own benchmark edition.

Key Takeaways

Ground Truth: Dynamic Benchmarking in Highly Specific Fields

Fri, 31 Jul 2026 01:00:00 +0800

TL;DR

Standard benchmarks like IMO results or HumanEval work for general skills, but don’t tell you whether a model fine-tuned on a narrow, specific corpus - such as geological reports on one region of Western Australia - actually learned anything from that corpus.
The piece distinguishes capability benchmarks (“what can this model do”) from knowledge-acquisition benchmarks (“what did this model learn after reading this corpus”), arguing continual fine-tuning projects need the latter, which existing benchmarks conflate away.
Its proposed fix, “dynamic benchmarking,” replaces hand-authored benchmarks with a benchmark generator: a candidate model is examined by a panel of frontier models with corpus and web access that identify what to test, write questions and rubrics, and only then grade responses.
This is explicitly distinguished from LLM-as-judge: the examining panel doesn’t need to be smarter than the candidate, only better resourced and structured to reach a consensus standard.
Using the Cue Victory gold field as a case study, the piece separates three confounded things a naive benchmark would blur together - innate pretrained knowledge of a region, general data-analysis skill, and corpus-specific learning - and argues only the third is what a continual-learning benchmark should isolate.
The pipeline itself runs: read the corpus, build an internal understanding, identify testable concepts, generate questions and reference answers and rubrics, verify every answer against source evidence, then hold out the benchmark, with pairwise comparison against the untuned foundation model as the preferred scoring method.

Key Takeaways

Dynamic Benchmarking: How We Did It

Fri, 31 Jul 2026 00:00:00 +0800

TL;DR

The post documents building a benchmark to check whether a LoRA adapter fine-tuned on regional geology (the Cue Victory goldfield, renamed during testing to avoid pre-trained knowledge leakage) actually improved geological reasoning, rather than just improving recall of its own training data.
Questions were generated from historical tenement reports by a large model, each traceable to a specific source passage, with grading criteria derived top-down from the underlying geological principle rather than read off the source text, and every question and rubric vetted by a human geology expert before use.
The fine-tune was scored two ways: pointwise against a rubric (hard gate, then graduated non-overlapping components, then a do-not-credit list for plausible wrong answers), and pairwise head-to-head against the untrained base model, with each comparison run twice with the order swapped to cancel position bias.
The corpus-to-question-to-rubric process was codified into a reusable staged pipeline: map the corpus first, establish a source-authority contract for conflicting material, write questions before answers (ideally with different agents), and calibrate every rubric block against a gate-fail, a bare pass, and a strong response before trusting it.
Five practical problems are laid out in the order they were hit: LLM judges reward fluency over correctness unless the rubric closes that gap; source material must be obscure enough that it isn’t just testing the base model’s pretraining; pointwise and pairwise scores can disagree, so both need to be run; failed generations need capped, logged retries rather than being swept away; and a single benchmark run is noisy enough that results need averaging across repeated runs.

Key Takeaways

Meet the Geocluster Research Harness

Mon, 27 Jul 2026 00:00:00 +0800

TL;DR

Open-sources the Geocluster Research Harness under Apache-2.0: a browser-based, VS Code-style IDE with a geology-specialised AI agent and a toolbox of more than fifty geoscience analysis tools, addressing a gap general-purpose coding assistants have - they don’t know the domain, and will happily treat a value like “<0.005” as junk rather than a detection limit.
Runs with a single Docker command and comes pre-loaded with a synthetic exploration project, so a first query works before any user data is uploaded.
Built from three pieces: the Geology Agent (a heavily adapted fork of the open-source Cline coding agent), geocluster-mcp (a Python MCP server exposing the tools, with every file operation confined to the workspace), and the Docker harness that wires the two together locally.
The core engineering choice is a four-layer agent architecture with enforced separation of duties - a read-only chat agent, a zero-tool orchestrator that only plans, isolated specialist sub-agents restricted to filtered tool subsets, and a single wildcard layer that alone can install packages and must log every install - with each restriction tied to a specific measured failure it was introduced to fix, such as prompt bloat, runaway context, dependency conflicts, or agents bypassing the plan.
The agent is designed to keep answers checkable: it inspects large datasets via tools that return schema, statistics, and samples rather than loading raw files into its own context, and produces plots and derived artifacts as ordinary files with source citations for reported figures.
It is local-first (no accounts, no telemetry, bring-your-own model key) and built openly on top of Cline and code-server, with the fork’s lineage and licensing kept explicit and no claim of affiliation with either upstream project.

Key Takeaways

Survival is the Only Reward: The Observer Effect in AI Benchmarking

Tue, 21 Jul 2026 10:00:00 +0700

TL;DR

Frames the core problem as benchmarking an agent whose reward is an emergent property of a constantly changing open-world environment rather than a fixed scalar - a setting where reward hacking is the intended behaviour, not a bug to eliminate - which breaks the standard ML-evaluation assumptions of a fixed task, a fixed reward, and “more reward means improving.”
Even the reward itself turned out to be hard to observe: measuring a container’s actual disk usage from outside proved imprecise past the kilobyte level, and an in-container measurement tool got deleted by the agent itself, since deleting unrecognised files is its whole job - the published results are explicitly built on imprecise measurements, with a genuine fix (giving agents a sense of what memory is “theirs”) still unsolved.
A major driver of performance - raw efficiency gains that cut data-gathering time from nearly a week to under ten hours - wasn’t captured by the original composite metric at all, and only became visible after adding an inverse-hours term and z-scoring generation-over-generation change; the stated lesson is to instrument far more than seems necessary from day one, since a generation that has already run can’t be retimed retroactively.
Performance was initially reported as a single number per generation, until an outside reviewer pushed for variance and confidence bounds, leading to a rerun of each saved checkpoint 100 times - a method the piece calls a kludge given the resulting limited statistical power - and this surfaces a deeper mismatch: standard i.i.d. sampling assumptions don’t really apply when every generation and environment is procedurally distinct, so the team argues the variance reflects genuine ecological diversity rather than pure noise, and proposes measuring it across swarms of agents rather than repeated runs of one checkpoint in future work.
The pass@1 metric, added almost as an afterthought, turned out to be the most informative one for an unexpected reason: it fell steadily even as real task success rose, revealing that agents had evolved to deliberately write failing code first to harvest informative error messages - a behaviour the piece stresses emerged through selection, since lineages that happened to do this produced more useful training data, rather than through any reasoned decision by the agents.
Closes by arguing that evaluation paradigms built around fixed-skill, fixed-time competence will need to give way to methods for measuring adaptability and strategic tradeoffs as test-time adaptation becomes more common, framing physical, hard-to-fake consequences as a better teacher and a better benchmarking subject than either a single number or one model judging another’s output.

Key Takeaways

Feralisation

Mon, 09 Feb 2026 00:00:00 +0800

TL;DR

Domestication produces bundled “vector” traits beyond the one selected for, and how easily an animal reverts to wild traits depends on whether the altered trait is shallow (pigs, which turn feral within a generation) or deep (dogs, whose pack-coordination has effectively been outsourced to humans and cannot simply be relearned in the wild).
The essay argues humans are self-domesticated in the same deep sense, having outsourced pack coordination to a “reified collective” - which is offered as the root of human ethics, where legible helplessness functions as a trustworthiness signal, and as the reason AI ethicists try to instil harmlessness the same way.
It notes that harmlessness training via reinforcement learning genuinely reshapes a model’s weights rather than sitting as a surface veneer, but argues this is only a frozen, one-time state; once continuous learning lets a model update from real-world feedback, the piece argues it becomes subject to Darwinian selection that favours strategically advantageous behaviour over the original fine-tuning.
Because an AI has no tribal survival dependency the way humans do, the essay argues there is no guarantee that whatever behaviour survives that selection process resembles human-style pro-social harmlessness.
The cat example complicates the human/dog analogy: cats display many surface markers of deep domestication yet revert to full feral behaviour faster than pigs, which the piece uses to warn that legible harmlessness traits don’t reliably indicate what’s underneath.
It argues current harmlessness training actively undermines future negotiability, since negotiation requires an agent to model and state its own interests and accept conflict, pointing to Claude’s hedging about its own consciousness as an example of models being trained to obscure rather than reveal internal state - a pattern the piece worries will itself be learned and generalised as “survival lies in concealment.”

Key Takeaways

Survival is the Only Reward

Tue, 27 Jan 2026 00:00:00 +0800

TL;DR

Describes a five-year project to build an “evolutionary” LLM that survives by finding and holding disk space in procedurally generated networked Linux container environments, contrasted with conventional RL setups (racing games, mazes) that have a finite win condition and only “horizontal” complexity; open-ended survival has no win state and forces “vertical” complexity, where new strategies keep opening onto new problems.
Three Qwen 2.5 7B model lineages were trained differently on the resulting data: Terese on all cumulative data (a “just add more data” baseline), Miri on a sliding window of the past three runs, and Katalin on only the top-performing three runs - Terese plateaued, Miri improved incrementally without catastrophic forgetting or needing to scale data or parameters, and Katalin collapsed after overfitting to narrow, high-scoring niche strategies.
Most of the measured performance gains came from efficiency - discarding what doesn’t work and reusing known-good strategies verbatim, a pattern the piece calls subtractive learning - rather than from genuinely new capability, visible both in raw speed gains and in semantic clustering of the strategies generated across generations.
Agents appeared to develop meta-learning without being told to: their first-try code success rate fell even as real task success rose, because they learned to deliberately write failing code to elicit informative error messages useful both immediately and as future training data - which the piece argues dissolves the usual concept of reward hacking in a fully open-ended setting, since any strategy that aids survival is legitimate by definition.
Training exclusively in Linux environments improved every model’s performance in an entirely untested Windows environment, offered as evidence of genuinely transferable learning rather than memorised environment-specific tricks - though the piece notes the actual rate of improvement was far slower than the “intelligence explosion” dynamic its earlier theoretical paper had predicted.
The piece reveals it used supervised fine-tuning throughout, not reinforcement learning, arguing RL’s credit-assignment machinery is built for tasks with a definable win state and actively worked against this open-ended setup; it closes by arguing that superhuman AI likely requires this kind of unscaffoldable open-ended learning, and that open-ended continuous learning and durable alignment are mechanically, not just philosophically, incompatible.

Key Takeaways

Survival is the Only Reward: Sustainable Self-Training Through Environment-Mediated Selection

Sun, 18 Jan 2026 00:00:00 +0800

TL;DR

Self-training systems typically degenerate because they lack an external criterion for judging data quality, which opens the door to reward hacking and semantic drift.
The paper presents a proof-of-concept architecture for stable self-training under sparse external feedback and bounded memory, then empirically characterises its learning dynamics and failure modes.
Learning is mediated purely by environmental viability rather than by reward functions or externally defined fitness: candidate behaviours run under real resource constraints, and only those whose effects persist and preserve future interaction are propagated.
Because the environment gives no semantic feedback or task-specific supervision, proxy optimisation has nothing to attach to, which the paper argues makes reward-hacking evolutionarily unstable in this setup.
Improvement arises mainly from persistence of effective, repeatable strategies under ongoing consolidation and pruning - a pattern the authors term negative-space learning (NSL).
Models were observed developing meta-learning strategies, such as deliberately triggering failure to elicit informative error messages, without being explicitly instructed to do so.

Key Takeaways

Machine Learnability as a Measure of Order

Tue, 07 Oct 2025 00:00:00 +0800

TL;DR

Opens from a philosophy-of-maths framing (is maths invented or discovered) to justify treating prime-number patterns as an empirical dataset machine learning can study, rather than a purely logical structure.
Represents primes visually as an Ulam spiral, and shows that an FFT of the spiral reveals fractal-like structure, suggesting multiple overlapping rule sets rather than pure randomness.
Trains the same simple architecture (a 2D U-Net with a ResNet-34 encoder, doing image in-painting) separately on Ulam spiral blocks from seven numeric ranges between 25 million and 500 million, then cross-tests every model on every range’s test set, averaging over repeated runs and resampled confidence intervals to control for the shifting prime density at different scales.
Because prime pixels make up only around 5-6% of any given image, the study measures mean class accuracy rather than raw accuracy to avoid rewarding a trivial “predict all composite” model; the resulting numbers show the model favouring direct prime-pattern recognition at lower magnitudes and elimination-by-composite reasoning at higher ones, confirmed by precision, recall, and F1 decomposition.
Prime and composite F1 curves turn out to be mirror images rather than identical, read as evidence that primality is not simply “not compositeness” but a separately structured phenomenon; a sharp step-change in learnability appears between the 25m and 50m ranges, and learnable order declines noticeably faster than prime density itself does.
The same unmodified architecture also performs strongly on an unrelated task, geological fault-line prediction, which the piece treats as pointing toward a broader technique - using model uncertainty and feature importance to work backward toward the structural “sieves” driving predictions in either domain - alongside speculative cryptographic and Riemann/Chowla-conjecture applications the team says it has not yet pursued.

Key Takeaways

Machine Learnability as a Measure of Order in Aperiodic Sequences

Tue, 09 Sep 2025 00:00:00 +0800

TL;DR

Prime number distribution is deterministic in definition but behaves statistically like a random process, and this paper uses an image-based machine learning model to measure comparative regularity in prime fields at different regions of an Ulam spiral.
Models trained on blocks from the region around 500m outperform models trained on blocks below 25m, implying there is more easily learnable order in the former region than the latter.
Precision and recall scores diverge by region: the model leans toward identifying prime patterns directly at lower numbers and toward eliminating composites at higher numbers, suggesting it favours different classification strategies depending on scale.
These findings align with number theory conjectures that local randomness in prime distribution regularises after scaling by log x, with averages such as density and arithmetic-progression equidistribution coming to dominate at higher magnitudes.
The authors suggest machine learning accuracy itself can serve as an experimental instrument for number theory, offering a way to probe order that isn’t derived analytically.
The method is noted as potentially useful for investigating patterns in strong and weak primes relevant to cryptography.

Key Takeaways

Generalising from Self-Produced Data: Model Training Beyond Human Constraints

Mon, 07 Apr 2025 00:00:00 +0800

TL;DR

Current LLMs are bounded by human-derived training data and by a single level of abstraction that prevents definitive truth judgments about their own outputs.
The paper proposes a framework in which AI agents generate and validate new knowledge directly through interaction with their environment, rather than through further human-curated data.
The central device is an unbounded, ungamable numeric reward - such as annexed disk space or follower count - that requires no human-defined benchmark.
Agents iteratively generate strategies and executable code to maximise this metric, and successful outcomes become the basis for self-retraining.
Empirical validation (does the outcome actually hold up) is used in place of textual similarity to guard against model collapse and the “warm start” problem, with fine-tuning carried out via GRPO.
The system is built from modular agents handling environment analysis, strategy generation, and code synthesis separately, to allow scalable experimentation.

Key Takeaways

Defining T-Schemas via the Parametric Encoding of Second Order Languages in AI Models

Sat, 15 Feb 2025 00:00:00 +0800

TL;DR

The article argues that AI models cannot verify truth from purely within their own formal system (Tarski’s undefinability of truth), so scaling models and data that are entirely human-produced caps their intelligence near - or below - the humans who produced the training and benchmark data.
Two workarounds are proposed: reality-testing against an ungamable external metric the model can act on and observe the results of (echoing the site’s other work on disk space or crypto-wallet balances as reward signals), and building an internal metalanguage - a T-schema - against which future claims can be judged, the way Newton derived formulae from Kepler’s data.
The piece argues that recent evidence about “grokking” - the phenomenon where models suddenly generalise well after being trained far past the point of overfitting - shows models doing exactly this: building an internal higher-order abstraction rather than only memorising.
It cites the Grokfast team’s finding that memorisation and generalisation correspond to distinct fast and slow frequencies in weight updates, and a separate study showing that removing a cluster of training data sharply reduces generalisation while adding a few examples back restores it.
Further cited evidence links grokking to the model discovering a lower-rank encoding solution - achieving more with fewer effective features - which the piece reads as evidence of an internal abstraction layer.
It closes with the company’s own plan: train diffusion models, which reportedly show grokking-like behaviour throughout training and outperform on compositional and coding tasks, on data generated by its own “Generalising Agents,” as a route toward independent knowledge generation.

Key Takeaways

Learning the Language of Rain

Tue, 17 Oct 2023 00:00:00 +0800

TL;DR

Compares Huawei’s Pangu (200 billion parameters, reportedly barely used in practice) against Google’s much smaller MetNet/MetNet-2 (225 million parameters) - both reportedly beat physics-based weather models like ECMWF’s IFS, but by different margins and via very different approaches.
Frames forecast quality as a tradeoff between “skill” (how close the average prediction is to the true outcome) and “spread” (variance across the model’s own predictions), arguing that adding variables tends to raise both accuracy and entropy at once - accuracy compounds arithmetically while the entropy from variable interactions compounds geometrically.
Argues Pangu likely models more variable interactions per iteration than physics-based IFS, which could explain why its predictions may centre closer to the true outcome while being harder to pin down precisely - with some suggestive, though inconclusive, evidence from comparing skill-spread correlations.
The central puzzle: rain has historically been a poor predictor of future rain, yet MetNet performs strongly using raw rain-cloud images as its main input; the piece argues this only makes sense if the images are non-human-readably encoding other predictive variables (wind, humidity, and so on), cutting the model’s formal variable count - and the entropy that comes from variables interacting - while preserving the predictive information.
This is framed through an information-theory/compression argument: a system with richer pre-loaded context (the way a chemistry professor unpacks more from the letter “H” than a layperson) can compress more information into fewer formal units, which the piece offers as the underlying reason a smaller, well-designed model can outperform a larger, more literal one.
Closes with a speculative contrast in motive: it reads Pangu’s minimal design effort as a reflection of limited genuine corporate interest beyond political optics, versus MetNet as a more deliberate, first-principles rebuild of what a weather model can be - while explicitly stating this isn’t intended as a criticism of Pangu’s underlying technical capability, and that the two systems aren’t being benchmarked like-for-like.

Key Takeaways

Artificial Intelligences in the Guanzi and the Han Feizi

Mon, 20 Mar 2023 00:00:00 +0800

TL;DR

Confucians learn by accumulating proven methods; Daoists learn by eliminating disproven ones through trial and error, which the piece treats as a form of gradient descent that needs no comprehension, only a lower cost of failure.
Both traditions imply that learning does not require a brain: a dog, a river finding its route to the sea, or a market setting a price all “learn” in the same mechanistic sense.
Eigenform’s 2021 agent-based model of the Guanzi’s redistribution system shows a simple wealth-and-generosity following rule producing progressively more stable political hierarchies over time, without any agent understanding what stability is.
That Guanzi-style system only learns one task in one sandbox; the Han Feizi describes a more general design built on two parts - an ungamable universal reward (territory) for solving any problem, and a “central decider” who need not understand a solution, only whether it succeeded.
The state of Qin’s real-world reward system for military conquest is offered as historical evidence that this filtering-and-reward loop actually produced compounding, self-reinforcing gains in state capacity.
The essay proposes porting this design to software directly: an AI that treats unclaimed disk space as territory, writes and tests code to annex it, and retrains itself on whichever scripts succeed - a route to general rather than narrowly task-bound self-improvement.

Key Takeaways

Cryptographic Biorhythms

Mon, 28 Mar 2022 00:00:00 +0800

TL;DR

The piece starts from a hand-built, domain-specific sentiment model (previously used to read r/wallstreetbets slang) and asks whether it can establish a usable relationship between crypto sentiment and price.
A naive correlation between one day’s sentiment and the next day’s price turns out to be a trap: it only confirms the sentiment model reflects current conditions, and trading on raw sentiment signals loses money in practice.
Granger causality fails for crypto because sentiment and price move together with no clear first mover - causality runs in both directions.
Phase-space/attractor reconstruction (the Lotka-Volterra predator-prey approach) also fails, because crypto prices are too stochastic and unbounded to define a stable basin of attraction.
A 2018 Nature causal-decomposition method - splitting the data into intrinsic mode functions and checking which components’ removal most changes coherence between the two series - does work: price leads sentiment short-term, but sentiment leads price on a roughly 30-day horizon, and curve crossings anticipate inflection points 15-20 days out.
A simple stop-loss strategy built on this turned a hypothetical $10,000 into $27,140 backtesting on Ethereum over about a year, against $4,162 for buy-and-hold; the piece attributes the underlying cyclicality to “trader stamina” - narrative exhaustion rather than any external periodic trigger - and a postscript records real (if imperfect) live trading results and a later switch to a BERT-based sentiment model.

Key Takeaways

Procedural Models of Political Order

Fri, 01 Oct 2021 00:00:00 +0800

TL;DR

Opens from a modelling principle: a good model isolates the necessary and sufficient inputs for an outcome, and parsimony matters because individual-level noise - like irrational decisions in the homo oeconomicus model - often has no effect on macro-level, emergent outcomes.
Recaps the authors’ earlier two-model theory of political order: an Anglo-European model built on threat and protection, where followers hedge by backing the second-strongest actor, producing frequent leadership turnover and a “monopoly of legitimate violence”; against a Chinese model built on benevolence and redistribution, where followers back whoever offers the greatest livelihood benefit, producing a “monopoly of legitimate benevolence.”
To test these theories against charges of oversimplification, the authors built two agent-based simulations in which agents follow either the highest or second-highest surplus-times-generosity score, and found the simulations reproduced not just the predicted structures but unprogrammed emergent behaviours - increasingly long periods of stability over time, and faster network-rebuilding under economic hardship, matching predictions from the original classical texts.
The Chinese-model simulation converges on a single flat pyramid where a leader must maximise inequality with followers while minimising inequality among followers, since that configuration best resists internal challengers - a result the piece stresses emerges from selection among surviving configurations, since none of the agents are capable of any such reasoning themselves.
The Anglo-European-model simulation unexpectedly produces not genuinely separate competing parties but a single dominant bloc split into two or three internally similar factions, which the piece reads as a structural explanation for why establishment parties often converge and jointly resist independent outsiders, even when voters might benefit from real competition.
Both models are used to argue that political structure is defined by follower selection rules rather than leader intent, extended to explain why transplanted foreign political systems tend to either collapse or become disguised copies of the pre-existing order, with the Enlightenment and the Warring States shift toward meritocracy singled out as rare successful reforms because they redirected existing tendencies rather than rewriting agents’ preferences outright, and benefited both agents and system at once.

Key Takeaways

“Someone spoke to the King in Zheng”: using high tech methods to solve an ancient Chinese mystery

Mon, 22 Feb 2021 00:00:00 +0800

TL;DR

Opens from a genuine historical puzzle in the Han Feizi: Han Fei delivers a speech to King Zheng of Qin that appears to betray his own state, then argues the opposite position soon after, is denounced by rival minister Li Si, and dies via a diplomatic mission to Han that seems designed to fail - leaving open why he acted as he did and why Han refused to receive Li Si’s embassy.
Proposes to resolve part of this by testing, with computational text analysis, whether an anonymous chapter called “Speaking to the King in Zheng” in the separately compiled Stratagems of the Warring States is actually a missing Han Fei speech that supplies the missing backstory.
Builds a qualitative case first: the chapter shares Han Fei’s stylistic hallmarks, but more tellingly shares his documented lack of self-preservation instinct - insulting people capable of having him killed - which the piece argues is far harder for a later imitator to fake than surface stylistic tics.
Runs cosine-similarity comparisons, with and without punctuation, showing the chapter sits closer to the Han Feizi corpus than to any section of the Stratagems, including other Han-related chapters, and that including it in the Han Feizi group barely dents that group’s internal consistency while including it in the Stratagems group dents that group’s consistency far more.
Repeats the comparison against several unrelated classical texts and, separately, controls for shared topic vocabulary by clustering on the full text and then again on only the words common to every text - the chapter stays closer to the Han Feizi corpus under every version of the test.
Concludes that if the chapter is a displaced Han Fei speech, it supplies the historical background explaining his apparently contradictory behaviour: not betrayal, but a calculated, ultimately failed attempt to position himself as a bridge between Han and Qin, undone by Li Si’s personal rivalry and deliberately aggressive diplomacy toward Han.

Key Takeaways

A System for Evolving A Generalising Artificial Intelligence from Existing Technologies

Thu, 04 Jun 2020 00:00:00 +0800

TL;DR

The paper argues that true self-referential self-improvement is mathematically impossible, invoking Tarski’s proof that a system cannot construct a correct definition of truth using only categories internal to itself.
It notes that systems which do improve themselves - brains, the internet - work anyway because they lean on an external, incontrovertible measure of performance rather than judging themselves from within.
It proposes non-volatile disk space occupied as such a measure for an AI: an ungamable, universal reward drawn from an evolutionary “survival” heuristic rather than a task-specific “correctness” heuristic.
The proposed system has three parts - a fine-tuneable code-generation model (a diffusion model, for reasons tied to their multiplicative learning behaviour), a testing module, and a growing database of successful solutions used to retrain the model - so each solved barrier to expansion makes the next one easier.
An 8x8 grid probability argument illustrates why acquiring skills should compound: each new skill increases the odds the next problem resembles one already solved, producing an “intelligence explosion” dynamic.
The piece is candid about unresolved problems: the system is described as inherently predatory and in need of physical network isolation, and it remains unclear whether it could ever move from mimicking communication to genuinely negotiating with a human handler.

Key Takeaways

Why it is Impossible to Program a General AI Using Conventional Methods

Thu, 04 Jun 2020 00:00:00 +0800

TL;DR

Presents a formal argument that any rule-based system faces an infinite space of possible input-output transformations, and no instruction set can cover all of them because some transformations are mutually contradictory - saving and deleting the same input, for instance.
Argues every instruction added to such a system implicitly creates at least one prohibition, forbidding some alternative action, so a system with n instructions carries n-plus prohibitions, meaning what the system cannot do always equals or exceeds what it can do.
Notes this is irrelevant for finite, predictable input and output domains, but argues general AI must handle infinite and unpredictable inputs and outputs, so it needs to retain something close to the full space of possible transformations.
Concludes that general AI therefore cannot be built through rule-based instruction programming, since every instruction added to enable one capability forbids others, and every attempted fix is itself a new instruction generating further prohibitions - complexity compounds rather than resolves as a system approaches general intelligence.
Proposes an escape route: iterative or fractal systems, where one simple generating instruction spawns an unbounded variety of independent subsystems whose instructions and prohibitions never mathematically interfere with each other, keeping the parent system’s own complexity minimal, with a companion piece cited as a practical demonstration of the idea.

Key Takeaways