Project 03Play · desktopPython · pygame · numpy · pyarrow

FlyVision: the garden through a fly's eyes

What does the world look like sampled by 1,709 ommatidia? FlyVision ray-traces an authored garden from one pose and shows each pixel as the nearest of the published eye directions from a microCT-scanned reference fly. Walk around, chase the other flies, press Tab to compare with an ordinary camera at the same pose.

Eye directions

857 left + 852 right, from reiserlab/eyemap_T4

Reproduction of published geometry

neighbour angles to 3.6 × 10⁻¹⁵ rad

Structural holdout

4,548 / 4,733 withheld column pairs recovered (96.1%)

Neural simulation

explicitly off; registration to MaleCNS unresolved
First-person fly view of a garden: a hexagonally sampled scene with a path, bushes, a red fruit and a marker labelled Nearby fly, with a discovery guide in the corner

Fly view. The irregular patches are the measured angular sampling, not a hexagon filter: each screen pixel takes the colour of the nearest published ray from either eye.

The garden

The window opens straight into the world. Click to capture the mouse, then WASD to move, Space/Shift to fly up and down. Three animated flies share the garden with scripted paths and a deterministic proximity response: get close and they move off. A guide leads you through four discoveries (the nearby fly, then a trail through the fruit grove and flowers); G hides it. Tab switches between the fly's sampling and a conventional pinhole camera at exactly the same pose; I opens the full eye panoramas and the provenance of the geometry.

Each session saves its trajectory, raw RGB directional samples, screenshots, discovery events and a parameter/provenance manifest to a new run directory. A verifier replays the first, middle and last frames of a recording and requires the same runtime code hash; the 90-frame acceptance recording reproduces all three exactly.

What it is not

The eye geometry is from a female microCT specimen related to the FAFB reconstruction (Zhao et al., 2025), pinned to a commit of the authors' repository. There is no verified mapping from those rays to the local MaleCNS photoreceptor identities: all 4,107 local photoreceptor-labelled bodies keep null physical bindings and the registered neuron count is zero. The prototype uses a shared camera origin for both eyes, RGB intensities rather than calibrated spectral radiance (no UV channel), and no optical blur, phototransduction or neural dynamics. The 110° display field and the 6° mask are display choices, not physiology. The social flies are scene animation, not circuit models.

The structural atlas

The first milestone built the complete local atlas: 165,122 traced neurons, 25,563,197 raw directed connection rows, 124,025,046 synapses, with neurotransmitter predictions kept as separate fields and never converted to a sign. Visual superclasses contain 103,268 neurons; 23,720 have source hex-coordinate pairs in the eyemap.

The test: hide 4,733 known coordinate pairs from the mapping anchors and ask whether one-hop structural support alone recovers them. It recovers the exact pair for 4,548 neurons (96.1%). Recovery varies by type (Tm4: 41.0%), so the overall score must not be applied to every population. A five-seed coordinate-shuffle control that preserves type, side, connectivity and counts recovers 0.11–0.21%.

Baseline plot: source coordinates, coverage, and per-type holdout recovery
reports/baseline.png: source coordinates, mapping coverage, and the variation in holdout recovery between cell types.

This measures structural consistency within one animal's annotations, which may themselves have been inferred using connectivity. It is not independent evidence of receptive fields or sensory function, and the plan in docs/RESEARCH_PLAN.md says what would be: an eye-to-angle transform verified against published geometry, temporal and synaptic parameters fitted on physiological data, and held-out stimuli before any model is connected to the world.

Run it locally

cd projects/flyvision
python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,optics]'
PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m flyvision.viewer          # the garden
.venv/bin/python -m pytest -q                                           # 24 tests
.venv/bin/python -m flyvision build --out runs/my-baseline --source-dir /path/to/flat-connectome
.venv/bin/python scripts/verify_optics.py runs/social-garden-preview-v1 --out reports/check.json

The garden needs only the pinned eye reference under references/eyemap_T4, which ships with the repository. The atlas build needs the MaleCNS flat files and takes several minutes.