Geology on Eigenform Articles

Which AI Model Is Best for Your Geology?

Mon, 24 Aug 2026 00:00:00 +0800

TL;DR

Argues that standard academic AI benchmarks are stale (“an exam paper everyone has had access to for years”) and largely irrelevant to a real decision like spending $100,000 integrating an AI system into an exploration workflow, since geology is hyper-local and a general fact, such as bismuth being a gold pathfinder, may not hold in a specific terrain.
Introduces the Groundtruth Dynamic Benchmark, which generates fresh questions and scoring rubrics directly from a user’s own document set rather than testing against a fixed, generic question bank.
Offers three free sample test sets (Murchison WAMEX gold reports, USGS Southwestern US reports, Canadian technical reports) alongside support for fully custom datasets, with generated questions spanning four types - finding facts, reconstructing sequences of geological events, reconciling sources that disagree, and recognising when the evidence doesn’t support a firm answer - graded against a rubric rather than accepted for sounding persuasive.
Reports the results of a six-model bake-off: Kimi K3 scored highest overall, but confidence intervals overlap enough among the top three models that the benchmark can only reliably sort models into tiers rather than a strict ranking, and that tiering held consistently across different source-document types.
Given the near-tied top performance, shifts the comparison to cost: generation cost dwarfs judging cost and varies sixfold between models doing the same task, and plotting price against performance puts four models on a shared frontier of roughly equivalent return on cost - until a newly released model, Ox Alpha, is added, whose free-tier pricing puts it alone on a sharply shifted frontier ahead of everything else.
Concludes with the practical payoff: an exploration manager can benchmark any model-plus-retrieval or model-plus-agent combination against their own frozen, company-specific test set, and rerun that same benchmark later to check whether a proposed cheaper migration actually degrades performance on their own work, rather than relying on vendor claims or generic scores.

Key Takeaways

How Not to Build a Slop Cannon

Thu, 20 Aug 2026 00:00:00 +0800

TL;DR

Argues against the instinct to fix agent errors by stacking more warnings into the prompt or CLAUDE.md file: long contexts degrade into contradictory noise, so the fix belongs at the harness level, not the prompt level.
Lays out five harness-design principles built on the premise that no single run will ever be perfect, so the goal is making errors cheap to catch rather than trying to eliminate them: split work across smaller sub-agents that check each other’s outputs; use a QC pass with a checklist instead of a “don’t make mistakes” instruction; design for the smallest model that works, both for cost and to reduce fragility to upstream guardrail changes; tag outputs by confidence level (confirmed, inferred, assumed) so uncertainty stays visible instead of masked by fluent prose; and treat chat history as something to actively manage and feed as an explicit variable rather than something to passively accumulate.
Case study: NextMaps stress-tested its exploration analytics pipeline on a deliberately hard Western Australian gold licence with three decades of historical reports whose boundaries had shifted repeatedly, where a naive system would misattribute nearby major gold discoveries to the licence being assessed.
The harness pre-computes each location’s real distance to the licence boundary before the model ever sees the text, and runs a separate verification pass against that ground truth, so the model writes the report but does not decide what counts as “inside.”
The resulting report rated the licence highly prospective while explicitly declining to justify that rating using a million ounces of gold on neighbouring leases, building its case instead from evidence physically inside the boundary and flagging what the data couldn’t support.
A postscript notes that the specific target the pipeline flagged from historical public records alone later turned out, per the operator’s own investor materials, to be the largest source of projected upside on the project - though the pipeline had no way of knowing about the active drilling now underway there.

Key Takeaways

Macrotrends to Watch Out for in the AI Exploration Space

Thu, 13 Aug 2026 00:00:00 +0800

TL;DR

Cites a recent preprint on AI for Mineral Exploration surveying how classical machine learning, deep learning, and foundation models are being applied across the exploration pipeline to handle extreme label scarcity and heterogeneous geoscientific data.
Draws five research-side trends from that preprint: prospectivity models moving from flat 2D maps toward 3D/4D representations incorporating geological time; single-modality analysis giving way to models that reason across geology, geochemistry, geophysics, remote sensing, and text together; supervised learning giving way to self-/semi-supervised and positive-unlabelled approaches given how few labelled deposits exist; static prediction giving way to uncertainty-driven exploration that recommends the next survey or drillhole; and bespoke single-purpose models giving way to reusable geoscience foundation models paired with tool-using agents.
Cites a companion Xplorahub piece on the commercial side, which names Eigenform and NextMaps among platforms providing spatial analytics and ground intelligence to explorers and investors.
Draws six commercial-side trends: AI activity clustering around the drill-targeting decision because it has the clearest economic case; some companies moving from selling software to vertically integrating exploration itself via mineral rights, joint ventures, or royalties; others pairing models with proprietary data-acquisition hardware such as satellites and automated core scanners; major miners like BHP and Rio Tinto behaving more like venture investors than customers; consolidation of standalone AI and sensing startups into broader mining-tech platforms; and a widening gap between AI performance claims and independently validated economic discoveries.
Points readers to both source pieces - the preprint and the Xplorahub article - for the full technical and commercial detail rather than presenting new research of its own.

Key Takeaways

Geological Map Processing Suite

Mon, 10 Aug 2026 01:00:00 +0800

TL;DR

Geological maps are difficult VLM inputs beyond sheer scale: a map sheet’s legend, cross-section, correlation diagram, graticule, and projection notes are mutually referential, so a simple question like “what’s the dominant lithology in this quadrant” requires multi-step reasoning.
Tiling or downsampling addresses the scale problem but worsens the compositional one, since it can separate a polygon from the legend entry it depends on to be interpreted correctly.
Existing approaches are surveyed: AI4CMA and DIGMAPPER (and Eigenform’s own tools) treat this mainly as a vectorisation problem, while Microsoft’s PEACE takes a multimodal-LLM approach, decomposing map QA into information extraction, domain-knowledge integration, and answer generation, reporting its GeoMap-Agent scoring 0.811 against GPT-4o’s 0.369 on its own 124-map, 3,864-question GeoMap-Bench.
Eigenform’s objection to PEACE is about packaging rather than accuracy: its repo ships as a fixed end-to-end QA tool and benchmark, not a modular toolkit.
The post introduces “Stratigraphic Amenity,” which repackages PEACE into a small Python SDK plus a local MCP server exposing map layout and legend detection, geo-referencing, knowledge retrieval, and overlay operations, deliberately stopping short of OCR, VLM inference, or answer construction so users can bring their own agent and harness.

Key Takeaways

Groundtruth Dynamic Benchmarking (Geology Edition) v1.0.0 Is Live

Fri, 31 Jul 2026 02:00:00 +0800

TL;DR

Announces v1.0.0 of the Groundtruth Dynamic Benchmarking System on GitHub: not a fixed set of benchmark problems, but a pipeline for generating reliable benchmark questions and scoring rubrics for whatever corpus you’re working with.
Ships a working example grounded in a real, openly licensed corpus - 34 mineral deposit records from the Yudnamutana Copper district, South Australia - runnable end-to-end with a single script after cloning the repo and adding API keys.
Uses a rubric format built to resist grade inflation: a hard gate per question, non-overlapping graduated components, required concepts, and do-not-credit lists, with the underlying mechanics covered in the companion post “Dynamic Benchmarking: How We Did It.”
Grading is structured and auditable: the judge returns a full breakdown of which components it credited and why, and flags anything it can’t verify from the rubric alone rather than guessing.
The corpus-to-rubric authoring pipeline itself ships in the repo, so anyone can point it at their own corpus and build their own benchmark edition.

Key Takeaways

Ground Truth: Dynamic Benchmarking in Highly Specific Fields

Fri, 31 Jul 2026 01:00:00 +0800

TL;DR

Standard benchmarks like IMO results or HumanEval work for general skills, but don’t tell you whether a model fine-tuned on a narrow, specific corpus - such as geological reports on one region of Western Australia - actually learned anything from that corpus.
The piece distinguishes capability benchmarks (“what can this model do”) from knowledge-acquisition benchmarks (“what did this model learn after reading this corpus”), arguing continual fine-tuning projects need the latter, which existing benchmarks conflate away.
Its proposed fix, “dynamic benchmarking,” replaces hand-authored benchmarks with a benchmark generator: a candidate model is examined by a panel of frontier models with corpus and web access that identify what to test, write questions and rubrics, and only then grade responses.
This is explicitly distinguished from LLM-as-judge: the examining panel doesn’t need to be smarter than the candidate, only better resourced and structured to reach a consensus standard.
Using the Cue Victory gold field as a case study, the piece separates three confounded things a naive benchmark would blur together - innate pretrained knowledge of a region, general data-analysis skill, and corpus-specific learning - and argues only the third is what a continual-learning benchmark should isolate.
The pipeline itself runs: read the corpus, build an internal understanding, identify testable concepts, generate questions and reference answers and rubrics, verify every answer against source evidence, then hold out the benchmark, with pairwise comparison against the untuned foundation model as the preferred scoring method.

Key Takeaways

Dynamic Benchmarking: How We Did It

Fri, 31 Jul 2026 00:00:00 +0800

TL;DR

The post documents building a benchmark to check whether a LoRA adapter fine-tuned on regional geology (the Cue Victory goldfield, renamed during testing to avoid pre-trained knowledge leakage) actually improved geological reasoning, rather than just improving recall of its own training data.
Questions were generated from historical tenement reports by a large model, each traceable to a specific source passage, with grading criteria derived top-down from the underlying geological principle rather than read off the source text, and every question and rubric vetted by a human geology expert before use.
The fine-tune was scored two ways: pointwise against a rubric (hard gate, then graduated non-overlapping components, then a do-not-credit list for plausible wrong answers), and pairwise head-to-head against the untrained base model, with each comparison run twice with the order swapped to cancel position bias.
The corpus-to-question-to-rubric process was codified into a reusable staged pipeline: map the corpus first, establish a source-authority contract for conflicting material, write questions before answers (ideally with different agents), and calibrate every rubric block against a gate-fail, a bare pass, and a strong response before trusting it.
Five practical problems are laid out in the order they were hit: LLM judges reward fluency over correctness unless the rubric closes that gap; source material must be obscure enough that it isn’t just testing the base model’s pretraining; pointwise and pairwise scores can disagree, so both need to be run; failed generations need capped, logged retries rather than being swept away; and a single benchmark run is noisy enough that results need averaging across repeated runs.

Key Takeaways

Meet the Geocluster Research Harness

Mon, 27 Jul 2026 00:00:00 +0800

TL;DR

Open-sources the Geocluster Research Harness under Apache-2.0: a browser-based, VS Code-style IDE with a geology-specialised AI agent and a toolbox of more than fifty geoscience analysis tools, addressing a gap general-purpose coding assistants have - they don’t know the domain, and will happily treat a value like “<0.005” as junk rather than a detection limit.
Runs with a single Docker command and comes pre-loaded with a synthetic exploration project, so a first query works before any user data is uploaded.
Built from three pieces: the Geology Agent (a heavily adapted fork of the open-source Cline coding agent), geocluster-mcp (a Python MCP server exposing the tools, with every file operation confined to the workspace), and the Docker harness that wires the two together locally.
The core engineering choice is a four-layer agent architecture with enforced separation of duties - a read-only chat agent, a zero-tool orchestrator that only plans, isolated specialist sub-agents restricted to filtered tool subsets, and a single wildcard layer that alone can install packages and must log every install - with each restriction tied to a specific measured failure it was introduced to fix, such as prompt bloat, runaway context, dependency conflicts, or agents bypassing the plan.
The agent is designed to keep answers checkable: it inspects large datasets via tools that return schema, statistics, and samples rather than loading raw files into its own context, and produces plots and derived artifacts as ordinary files with source citations for reported figures.
It is local-first (no accounts, no telemetry, bring-your-own model key) and built openly on top of Cline and code-server, with the fork’s lineage and licensing kept explicit and no claim of affiliation with either upstream project.

Key Takeaways