NeurIPS 2026 · Evaluations & Datasets Track · Spotlight

StomataBench

Measuring Taxonomic Generalization in Stomatal Detection

Sainath Reddy Gummi1 · Hankui K. Zhang1 · Chulwoo Pack1 · Shyam Solanki1 · Young Chang1,2

1South Dakota State University   2Manitoba Agriculture

Stomata micrograph from OD-Train, framed in its dataset colour
training
Stomata micrograph from OD-Valid, framed in its dataset colour
validation
Stomata micrograph from T-MAWP, framed in its dataset colour
test
Stomata micrograph from T-DCWP, framed in its dataset colour
test
Stomata micrograph from T-MSWP, framed in its dataset colour
test
Stomata micrograph from T-GSH, framed in its dataset colour
test
Increasing biological distance from OD-Train

Detectors are scored on boxes. Biology uses density.

Stomata are the microscopic pores plants use to exchange gas and water. Their density, stomata per mm² of plant surface, links cellular anatomy to photosynthesis, transpiration and water-use efficiency. Object detectors now count them at scale.

What detection benchmarks report
COCO AP
Do the predicted boxes land on the stomata?
What biologists use
Stomatal density
One count per image, from every box above a confidence threshold.

Does a good detector give a good measurement when the species and the microscope change?

Abstract

Stomatal density is widely measured in plant biology because it links cellular anatomy to photosynthesis, transpiration, water-use efficiency, and global change responses across Plantae. Recent work has addressed scalable stomatal phenotyping by training object detectors to localize and count stomata in microscopy images. Unfortunately, most existing stomatal datasets evaluate detectors within a single acquisition pipeline, leaving open whether their predictions remain reliable under out-of-distribution (OOD) deployment. This matters because detection counts are not the final scientific output. The resulting density measurement is the quantity used for biological interpretation.

In contrast, StomataBench evaluates stomatal detection as a scientific measurement problem under structured OOD shift. StomataBench pairs a single training set, OD-Train, with four held-out test sets that separate taxonomic shift, imaging-system shift, annotator shift, and combined multi-source shift. OD-Train contains 21 woody species, 20 eudicots and one magnoliid, and no monocot, gymnosperm, or fern. To make OOD evaluation biologically meaningful, held-out species are arranged on a six-level OOD ladder grounded in taxonomic distance. Beyond stomatal localization, the evaluation tests whether detector outputs preserve calibrated species-level variation in stomatal density. StomataBench provides a benchmark for testing whether stomatal detectors remain useful as measurement tools under deployment-relevant biological and imaging shifts.

One training set, four held-out test sets

Test species sit on a six-level OOD ladder, set by the first taxonomic rank where they differ from the training species.

L0Familiar species, new accession
L1New species, familiar genus
L2New genus, familiar family
L3New family, familiar order
L4New order, same clade
L5New clade: monocots
OD-Traintraining
3,154 images · 134,802 stomata · 21 taxa
Woody plants: 20 eudicots and one magnoliid. No monocots, gymnosperms or ferns.
OD-ValidvalidationL0
138 images · 8,792 stomata · 5 taxa
OD-Train species from new accessions. Used to select thresholds.
T-MAWPTL1L2L3L4
456 images · 17,164 stomata · 17 taxa
New species. Same microscope and annotation protocol as OD-Train.
T-DCWPT+IL0L1L2L3L4
119 images · 2,858 stomata · 39 taxa
Adds a different microscope, the Olympus BX-63.
T-MSWPT+I+AL1L3L4
10,715 images · 514,568 stomata · 18 taxa
Adds another lab’s imaging and its own annotators.
T-GSHT+I+AL4L5
8,229 images · 319,161 stomata · 9 taxa
Multi-source, from soybean and poplar to wheat, barley, rice and maize.

Shift from OD-Train: T taxonomic, I imaging system, A annotator. Images and stomata as released.

Every detector is scored twice

As a detector
COCO AP
AP50:95 over the ranked boxes
As a measurement tool
Count and density
Count MAE, density MAE and within-species density R² at one threshold τ

R² below 0 means the predicted densities are worse than predicting the species mean for every image. All 15 detectors are fine-tuned once on OD-Train and evaluated unchanged. Results use τ = 0.25 for every detector. Thresholds selected on OD-Valid are reported too.

Two-stage CNNFRCNN-R50-1×, FRCNN-R50-3×, FRCNN-R101-1×, FRCNN-X101
One-stage CNNRetina-R50-1×, Retina-R50-3×, Retina-R101-3×
YOLOYOLOv8, YOLOv11, YOLOv12, YOLOv26
TransformerRT-DETRv2, RT-DETRv4
Foundation-model-derivedGDINO-SB, SAM3-OD

Findings

1No detector leads every shift

Test setShifted from OD-TrainAP leaderAP50:95 (%)Second
SpeciesImagingAnnotators
T-MAWPRT-DETRv4
39.08
YOLOv11 37.76
T-DCWPSAM3-OD
46.46
GDINO-SB 43.26
T-MSWPGDINO-SB
45.28
YOLOv11 44.57
T-GSHSAM3-OD
24.93
GDINO-SB 20.96

The best detector depends on the kind of shift, so one held-out test set is not enough to judge transfer.

2Same AP, opposite measurement

The same 45 apple images (Malus domestica, T-MAWP, L1) at τ = 0.25. Brackets are 95% bootstrap intervals.

YOLOv12
AP50:95 (%)
44.59
Density R²
0.83
[0.70, 0.90]
Density MAE (mm−2)
10.2
[7.7, 12.6]
RT-DETRv4
AP50:95 (%)
43.57
Density R²
−0.67
[−1.71, −0.14]
Density MAE (mm−2)
37.6
[32.6, 43.2]

A species’ AP leader is also its density R² leader for only 1 of the 17 T-MAWP species.

3Density recovery is strongest near the training species

R² below 0: worse than the species-mean referenceR² of 0 or above
For each species in OD-Valid, T-MAWP, T-MSWP and T-GSH, a bar splitting the 15 detectors into density R² below 0 and 0 or above. Most L0 and L1 species are mostly teal, and the more distant species and the cereals are mostly or entirely brown.
Each bar is the 15 detectors at τ = 0.25. Grey number: calibrated images. Right: detectors with R² below 0. The 37 T-DCWP taxa, most with three images, are in Figure 3 of the paper.

Wheat (L5): on the 348 calibrated wheat images every detector is below 0. OD-Train has no monocots, so wheat is a stress test.

4The confidence threshold changes the winner

Heatmaps of F1 and count MAE for all 15 detectors on OD-Valid and the four test sets at five threshold settings.
F1 and count MAE of every detector at five threshold settings, from Figure 2 of the paper.
15 of 17

T-MAWP species whose density R² leader changes between τ = 0.25 and 0.75. The count MAE leader changes for 14.

On T-MAWP, RT-DETRv4 has the highest AP but one of the highest count MAEs at τ = 0.25, and the lowest count MAE at 0.50. Wheat R² stays below 0 at every threshold.

Takeaways

1Test across several kinds of shift, not one held-out set.
2Report count and density error next to AP.
3State the confidence threshold with every result.

Benchmark scores are not a deployment guarantee. Calibrate in the target domain, choose thresholds on independent validation data, and keep human review.

BibTeX

@inproceedings{gummi2026stomatabench,
  title     = {StomataBench: Measuring Taxonomic Generalization in Stomatal Detection},
  author    = {Gummi, Sainath Reddy and Zhang, Hankui K. and Pack, Chulwoo and Solanki, Shyam and Chang, Young},
  booktitle = {Advances in Neural Information Processing Systems},
  volume    = {39},
  year      = {2026}
}