ResultsProjects 08 – 10Experiments 1, 2 and 3
Three experiments put the fly connectome, the full map of the fly's nerve cells and their connections, to practical tests: steering a robot, learning without backpropagation, and finding similar items fast. Each one fixed its rules before it started, including what result would count as a win and when to stop. The answers were mostly no. This page shows what was found, including where the fly's wiring lost.
Data: stage1_headroom.md · stage1b_occlusion.md · full write-up
Data: g1.json · g1b.json · pilot_analysis.json · exp3_analysis.md
Trained until it stopped improving, a 443-neuron circuit from the fly was not clearly better than a direct connection. With the target visible it was slightly worse; with the target hidden the test could not decide. Three randomly rewired copies of the circuit did as well as the real one, so what helped was having loops at all, not the fly's particular wiring. The real circuit also turned out to be quieter than every scrambled version of itself before any training, a difference earlier comparisons had not corrected for.
A pilot of forward-only learning, which never sends an error signal backwards, found results swinging between random starting points about four and a half times more than planned for. That is past the stopping limit fixed in advance, so the main comparison of learning rules was never run.
The fly Bloom filter beat conventional methods only at memory budgets so small that no method worked. The fly's real odour wiring was no better than scrambled wiring when missing data were set to zero, but it was better when they were filled with averages. An idea proposed to rescue it was contradicted. On the standard search benchmark, FlyHash answered about one question for every 200 the best conventional index answered.
All three experiments were preregistered: before running anything that would produce its results, each wrote down what it would compare, how many runs it would make, what result would count as a win, what would count as ruling an effect out, and when it would stop. The analysis code was frozen and fingerprinted, and checked against that fingerprint before every analysis. Each stage ran only once the previous gate's verdict was in. Two of the three ended at a gate: Experiment 1's second test came back unclear, leaving too little to explain for its next stage to be informative, and Experiment 2's pilot tripped its own stopping rule.
That is the methodological point of the work. With the rules fixed in advance, a negative result cannot be argued away afterwards, and a positive one could not have been manufactured by trying analyses until one worked.