The instrument
A general model of the world's birds, cut down to one archipelago
The instrument in use
Point your phone at a bird and Mohua tells you what it is. Leave it running and it keeps a list of everything that called while you were there.
There are two ways to listen. Identify checks every couple of seconds, for a bird you are standing in front of and want named now. Monitor wakes only when something actually calls, so it can run for hours without flattening the battery - this is the one for a walk, or a dawn. Either way the sound is drawn on the screen as it arrives, so you can watch a call as well as hear it.
When wind, rain or traffic is drowning the birds out, Mohua says so rather than guessing at what is underneath.
What it hears becomes a record you keep. Every bird is listed with how many times it called and when, and the five seconds of audio behind each name is saved, so you can play it back and judge for yourself. A ribbon shows when across the day - or a whole season - each species sings, lined up against sunrise. Every bird carries its conservation status, so hearing a kōkako never looks like hearing a blackbird.
It all happens on the phone. No account, no signal, nothing uploaded, and nothing in your record has ever left the device. At the end of a morning you can make a single illustrated plate of what you heard and share that, if you want to.
Google DeepMind's Perch 2.0 knows 14,795 classes of sound. New Zealand does not need them. The model was opened, inspected, and modified - four tensors, kept bit-for-bit at the indices that survive - leaving the birds you can actually hear here, and twelve classes of noise so the app can say what is drowning them out.
The slice is not an approximation. At the kept indices the reduced model's logits are identical to the original's; converting to 16-bit for the phone costs an embedding cosine of 0.999928 and no change at all in which bird comes first.
Nothing is filtered. Wind, rain, traffic, aircraft, dogs and voices are detected, not removed - Perch was trained through a plain log-mel frontend on noisy field recordings, so cleaning the audio moves it off-distribution and the artefacts of denoising look like faint birds.
What Mohua carries
From 491 taxa in DOC's threat classification, after merging subspecies Perch cannot separate and dropping the extinct and the vagrant. 52 families.
Twelve FSD50K classes collapsing to 9 plain words: wind, rain, traffic, voices, engine, aircraft, thunder, running water, a dog.
Included, but marked unhearable. Silence about a bird must mean no class for it, never no bird called.
- Te reo names
- 161159 of them from the OSNZ Checklist, 2022 - the authority.
Yours to set
Whichever you choose, the other name sits directly beneath it - in the list, in the detail view, on the export plate. Te reo leads by default, and reading the pair together every time is a good way to learn them.
Any input iOS can see, chosen in Settings. Mohua then reports the microphone it was actually given rather than the one it asked for, so a recorder still in your bag says so instead of quietly not being there.
The register of standing
What the Department of Conservation says about the birds this thing can hear
Threat classification · 190 birds
- Of concern - declining or worse
- All other categories
59 of 190 are declining or worse. Hearing a kōkako must not look like hearing a blackbird, so the threat status travels with every detection and the interface gives it weight.
The categories are ordinal and the code treats them that way: they can be compared, never averaged. There is deliberately no numeric value on them to tempt an arithmetic mean of a bird's peril.
A further 25 species report no single status. Perch cannot separate the subspecies DOC assessed differently - one class, two verdicts - so the app returns nothing rather than picking the more alarming of the two, and the interface describes the range instead. The first implementation of this got it wrong in an instructive way, and the decision record keeps the wrong version on the page.
The ledger of refusals
The most important page in the book
Perch's own model card says the species logits are uncalibrated and possibly unreliable for rare species, and recommends tuning thresholds on your own data. So 7,489 recordings were scored, and a threshold was fitted per species by cross-validation. 123 species earned one. 90 did not, and the app will not name them.
- Fitted
- 123Median precision 0.80, fitted on a median of 20 clips.
- Declined
- 9060 for want of recordings; 30 because the model is measurably wrong.
A refusal is not a bug being hidden. It is the recorded result of a species whose false positives were counted and found unacceptable - and in most cases the recordings can say precisely which other bird the model is hearing instead.
Where the recordings came from
Mostly Creative Commons; NZ Birds Online's are used for non-commercial research and never redistributed.