Why optioneering needs more than "data"

A human engineer looking at a satellite image of an area and sees things instinctively: bad terrain, a building in the wrong place, a wetland that's going to trigger a permitting fight. An automated system sees none of that unless it's told, explicitly, in structured form.

There's a second, less obvious problem. A core value proposition of data-rich approach early is shifting the discovery of project risk left, surfacing in days what might otherwise not surface until years into a development's lifecycle. That promise only holds if the data available on day one is as good as what an engineer would eventually uncover on day 900. Shifting risk left means front-loading the data problem. It doesn't make the problem smaller, it just moves who has to solve it, and when.

So the requirement was never just "more data." It's data with the operational context an engineer already carries in their head baked into it: what a given terrain can support, what triggers a permitting requirement, which regulatory regime governs a feature depending on where it sits.

The scale of it

Over the last six years we've grown the data underpinning Optioneer by roughly an order of magnitude every year. Our largest customer workspaces are approaching 1 billion tracked features. For scale, that's in the neighbourhood of every building footprint in a large country, tracked several times over.

That curve, not the snapshot, is the point. Someone evaluating what it would take to reproduce this internally isn't pricing a dataset — they're pricing six years of compounding curation, starting from zero.

Underneath that curve sits a library curated from 600+ data providers worldwide, comprising 12,000+ individual datasets, organised into roughly 20 standard planning topics — cultural and historical sites, hydrology and hydrogeology, land rights and planning, energy resources and industrial areas, and so on — mirroring the way EIA-style environmental review is already structured. All of it is included as standard, not gated behind a premium tier.

What "dealing with data" actually looks like

Here's the part that doesn't show up in a demo. "More data" and "clean data" both undersell what's actually happening. The real work is turning a raw government export into a feature that an optimisation engine can reason over — which usually means making four separate judgment calls per source, not one:

1. Does this layer need splitting? Most authoritative sources publish one flat layer where infrastructure planning actually cares about several distinct categories inside it.
2. What constraint severity does each resulting layer get, and does that severity vary by category?
3. Does a buffer apply — and if so, is it because a regulation specifies one, or because the underlying legal concept is unbounded and the platform needs a workable proxy?
4. Is there a permitting or consenting regime attached, and does it vary by sub-category or by jurisdiction?

Below are seven real sources, unedited, showing how those four questions get answered in practice.

Oil and gas wells — North Dakota Department of Mineral Resources

The source publishes a single Wells layer via its ArcGIS REST API. On its own, that layer is nearly useless for planning, an abandoned well and an active one carry completely different risk profiles. We split it by status into four layers (Active, Abandoned, Plugged, Permit Only), filed all four under the Energy Resources and Industrial Areas topic, and assigned differentiated constraint severity: active wells score high, abandoned and plugged wells score medium, permit-only wells score low. One API endpoint becomes four decision-ready layers with four different risk weightings.

Wetlands — Kansas Data Access and Support Center

A single National_Wetland_Inventory_Polygons layer, split by its TYPE field into seven usable layers — emergent wetlands, forested/shrub wetlands, freshwater ponds, lakes, rivers and streams, and more. Filed under Hydrology and Hydrogeology, with some categories (freshwater ponds) elevated all the way to a hard no-go, and the rest scored high. The distinction matters: treating all wetlands as equally restrictive would either over-constrain a route unnecessarily or under-constrain one that should be a non-starter.

Special Areas of Conservation — Natural England (UK)

Here the opposite call is correct: UK conservation law specifies a real, legally binding 500m buffer around a Special Area of Conservation. So the layer is carried twice — once as-is, once pre-buffered at 500m — because the regulation itself defines the geometry, and getting that number wrong isn't a data-quality issue, it's a compliance issue.

Listed buildings — Historic Environment Scotland

Split by category into Listed Building A, B, and C layers. Then a buffer is applied — min. 50m — despite the fact that no such legal buffer exists in UK planning law. Listed building protection works through the concept of "setting": any development that affects a listed building's character or views needs consent, regardless of distance, with no fixed radius. A 50m buffer is our own modelling proxy for an unbounded legal concept, chosen because a routing engine needs something geometrically concrete to act on. Getting this distinction right — legally mandated buffer versus engineering approximation of an unbounded legal concept — is exactly the kind of judgment call that doesn't show up in a raw data export.

Building footprints — U.S. Department of Homeland Security

The single USA Structures layer contains 135,321,228 features. We split it by occupancy class (OCC_CLS) into nine categories — residential, commercial, industrial, agricultural, government, and so on — then duplicate each into buffered versions, typically 100ft, extending to 500ft for residential structures. At this feature count, even a fraction of a percent needing manual attention is a meaningful workload; the split has to happen correctly the first time.

Wetlands — U.S. Fish and Wildlife Service

A Wetlands layer, split by WETLAND_TYPE into seven categories — despite being technically available via REST API, it's large enough that we handle it as manual data rather than a live pull. Several of the resulting layers are then cross-linked to state-specific permitting regimes: Virginia Water Protection Permit (General Permit WP1 & WP2), the Maryland Nontidal Wetlands Permit, and Colorado's SB-40 Wildlife Certification. Same source field, three different governing authorities, depending on where the feature physically sits.

Multiply that per-source judgment across 600+ providers and you start to see why "just scrape the open portals" undersells the problem by roughly an order of magnitude. None of these seven examples required inventing new data — everything came from a public authoritative source. What required expertise was knowing which four questions to ask of each one, and getting the answer right.

Garbage in, gold out

Volume without quality is just noise at scale, and geospatial data is a genuinely messy source to begin with. It's aggregated and published by organisations with wildly varying resources, and real-world features don't fit neatly into a schema — corrupted or missing geometries are common, and every incoming property has to be checked before it's safe to ingest.

Most of that is fixable programmatically. Some of it needs a human GIS specialist making a judgment call on an individual feature — exactly the kind of calls shown above. The problem is scale: each feature is simple in isolation, but at a billion features under management, even a fraction of a percent needing manual review can overwhelm a team. Our pipeline is built to resolve what can be resolved automatically and route only the genuinely hard cases to a specialist, cutting the volume anyone has to personally review by over 99.9%.

Open, but not accessible

Here's the paradox that trips up most people evaluating this space: the underlying data is, in a real sense, free. Governments and agencies mandate that all kinds of organisations publish geospatial data relevant to their remit — wells, wetlands, protected species, listed structures, flood zones. What none of those mandates require is that the data land in a common format, in a common place, on a common schema.

So "open" and "usable" are two entirely different problems, and the cost lives entirely in solving the second one. That's not a weekend integration project, and it's not something a procurement team shortcuts by pointing engineers at open portals for a quarter — as the seven examples above should make clear.

Data that doesn't rot — and where this is going

Data isn't a one-time deliverable. Wells get plugged, wetlands get reclassified, buildings go up and come down. On a project where hundreds of millions of dollars ride on ground conditions, a routing decision built on an 18-month-old dataset isn't a rounding error — it's a liability. Every customer's baseline is refreshed on a fixed, reliable cadence, with two versions retained on a rolling basis, so there's always a known-good prior state to fall back to.

Refreshes are soft-pushed, not forced: when new data lands, existing configs, saved views, and metrics are flagged out of date but keep functioning until a customer chooses to update. That's only possible because of a design decision that runs underneath the whole platform — every asset references the underlying data by a stable layer identity, not by name. Rename a layer, re-split it, or refresh it entirely, and everything built on top of it keeps pointing at the right feature, automatically.

That identity layer is what the rest of the platform is being built on top of. The direction of travel: from any layer, see every config, saved view, and metric that depends on it, and propagate an update selectively rather than all-or-nothing. Let a saved view choose to track "latest data" or lock permanently to the snapshot it was built on. Structure everything else in the platform — metrics, navigation, reporting — around the same layer-based data model, so the data structure isn't a backend detail, it's the organising principle of the product.

‍