Loading image...
Rethinking the 3D scene as a data structure

Rethinking the 3D Scene as a Data Structure

engineering3dperformancesystemsgpusnaptrude
How building a multi-proposal feature for architects turned a UI feature into a graph problem — and what it taught me about GPU memory, state diffing, and systems design.

A simple feature with a systems problem hiding inside

Architects rarely commit to one idea. They want to show a client two or three versions of a building — different room layouts, different floor plans, different structural bets — and let them pick.

That's what Proposals does. Each proposal is its own snapshot of the building: its own objects, its own layout, its own geometry. Switch proposals and the client sees a completely different scene. Objects can also be shared across proposals, so nobody has to rebuild the same staircase three times.

Simple idea. The implementation turned out to be a systems problem wearing a UI feature's clothes.

The naive version fell over fast

The first cut was the obvious one: keep every proposal's objects in the scene at once and toggle visibility. Proposal 1 active? Show its objects, hide the rest. Switch to Proposal 2? Flip the flags.

It worked beautifully on tiny projects and collapsed on real ones. Multi-floor buildings with hundreds of rooms, walls, and structural elements tanked the frame rate, ballooned memory, and occasionally took the whole browser down with them.

The culprit wasn't memory. It was draw calls.

Every object in a 3D scene costs at least one draw call per frame — a "render this geometry, with this material, right here" instruction to the GPU. Three proposals of 200 objects each meant 600 draw calls a frame, even though the user only ever saw 200. The GPU doesn't know which objects matter. It renders whatever you hand it.

At 60fps that's 36,000 wasted draw calls every second — and that was a medium project.

Treating the scene like a data structure, not a picture

The fix started with a reframe: stop thinking of the scene as a visual thing and start thinking of it as a data structure — a mutable graph of nodes with relationships, shared references, and state you manage on purpose.

The first realization was that switching proposals is a diffing problem. Given the current scene and the target proposal, what's the minimal set of changes to get from one to the other?

That's exactly what React's reconciler does for the DOM — diff, apply only what changed, leave the rest alone. We built the same thing for the scene graph. On every switch, the system computes three sets:

  • Remove — in the old proposal, not the new one
  • Add — in the new proposal, not the old one
  • Keep — shared by both, don't touch

Only the delta gets processed. A building with 400 shared objects and 50 unique ones per side becomes a 100-object operation instead of a 400-object one. The switch cost scales with the diff, not the scene.

Buffer pooling, or how to stop paying the allocation tax

Cutting the work down to the delta helped, but a stutter remained. GPU memory allocation is expensive — creating a vertex buffer means a round trip to the graphics driver, a GPU heap allocation, and a data upload across the CPU-GPU bus. Doing that for dozens of objects on every switch was visible to the eye.

So we stopped allocating on switch entirely. Instead of minting new buffers for incoming objects and destroying old ones, we kept a pool of pre-allocated buffers and wrote new geometry into existing slots. The buffers never changed hands — only their contents did.

It's the same trick as connection pooling in a backend, or memory arenas in systems code. Allocation is slow, reuse is fast: pre-allocate what you know you'll need and manage it yourself.

The switch stutter went from noticeable to gone.

Dirty flags, or how to stop redoing work nobody asked for

Profiling turned up a third offender: the scene was re-uploading geometry to the GPU on frames where nothing had changed.

In a collaborative app the render loop never stops, and every frame the system was dutifully re-syncing every object's GPU state — including the wall that hadn't moved in ten minutes. On a few hundred objects, that's a lot of wasted bus bandwidth, every single frame.

The fix was one boolean per object: a dirty flag marking whether its GPU state was out of sync with its CPU state. Objects only re-upload when flagged dirty. That untouched wall now costs zero bandwidth until someone actually touches it.

Tiny change in code. Big change in frame consistency, especially on large scenes sitting idle during collaboration.

Reference counting for shared objects

The diffing approach had one sharp edge: objects shared between proposals.

If an architect shares a structural core between Proposal 1 and Proposal 2, both depend on the same GPU buffer. A naive switch might dispose it on the way out of Proposal 1 — and then Proposal 2 reaches for geometry that's no longer there.

So we reference-counted shared objects. Every proposal that includes an object holds a reference; the buffer is only released when the count hits zero and nobody needs it anymore. Switching proposals never accidentally frees geometry the incoming one depends on. Shared objects just sit still while everything around them gets swapped.

What it taught me

Proposals looked like a product problem — give architects a way to juggle design options. Underneath, it was a resource-management problem: how do you share a pool of GPU resources across multiple states, switch between them cheaply, and avoid redundant work in a loop running 60 times a second?

None of the answers were specific to 3D. Diffing for the minimal delta, pooling to amortize allocation, dirty flags to skip redundant work, reference counting for shared resources — these are general systems patterns. They show up in databases, compilers, operating systems, and game engines for the exact same reasons.

The 3D made them easy to see. The thinking was the same.

Contact

Talk to me about engineering, fast cars, or anything that gets the adrenaline going.

Loading image...
Contact

Theme

Accent color

Gray color

Appearance

Radius

Scaling

Panel background