Exploring Fully Unsupervised Autoencoders for Early Disease Detection
Hyperspectral imaging of strawberry leaves to detect P. cactorum—no labels, anomaly detection via reconstruction error.
Core Idea
- Use hyperspectral images (≈398–1000 nm) of strawberry leaves; train only on healthy leaves; flag anomalies via reconstruction errors from convolutional autoencoders (2D & 3D).
Data & Preprocessing
- 20 plants imaged daily (~2 weeks), cropped to leaves; 388 samples (321 healthy / 67 infected).
- Preprocessing: mask background; remove noisy bands (<430 nm); SNV normalization; PCA-guided band selection to 17 wavelengths (also compared with full spectrum).
Method
- Train 3D Convolutional Autoencoder on healthy leaves; evaluate infection by image-level reconstruction error with thresholds; metrics: ROC-AUC, PR-AUC, FPR@95%TPR.
- Error-mitigation steps: remove edge artifacts, apply masks post-recon, prefer SNV to reduce illumination scatter.
- Experimented with alternative error metrics (e.g., per-band extreme-pixel rate; vein-focused errors; latent-space similarity).
Alternative Error Metrics
- Latent space similarity based on Benfenati et al. (2023).
- Per-band extreme-error pixel: Own idea, using the rate of all pixels with error values above the 99% quantile of the validation set (only healthy leaves) over all pixels to identify diseases.
- Vein-focused errors: Original idea based on the disease spread in the plant, as it often progresses through the veins. Veins and proximate areas are identified using the highest eigenvalues. The mean absolute error of the found area is taken.
Key Findings
- Fully unsupervised CAEs struggled to cleanly separate healthy vs infected leaves—especially in early stages; best conventional setup reached modest ROC-AUC, and 3D did not clearly outperform 2D.
- “Extreme-pixel” per-band metric showed most promise for day-1 infections (lowest FPR@95%TPR among tested metrics), but overall accuracy remained limited.
- Reduced 17-band sets performed comparably to the full spectrum → careful band selection can cut complexity without large losses.
Takeaways
- Symptoms of P. cactorum resemble benign variation (e.g., drought stress), making purely unsupervised detection intrinsically hard on this dataset.
- Pipeline + preprocessing + error-metric ideas form a reusable baseline for future semi-supervised or hybrid approaches.