Overview
Traditional RGB imaging provides limited spectral information, often insufficient for advanced applications in fields such as agriculture, environmental monitoring, and medical imaging. On the other hand, multispectral cameras can capture data across multiple wavelengths, which makes them expensive and require special integration/software to run. Our deep learning model bridges this gap by enhancing standard RGB images with predicted multispectral data, increasing the data density and by that unlocking new possibilities for image analysis and interpretation. This allows regular users to extract valuable spectral information from ordinary RGB photographs, dramatically expanding the utility of common imaging devices.
Goals
The project aimed to develop a deep learning model capable of enhancing standard RGB images with accurate multispectral data predictions. The system needed to generate high-fidelity spectral information across various wavelengths, achieve high correlation compared to actual multispectral camera outputs, and process images efficiently to support real-time applications.
Solution
We utilize a deep learning model trained on various wavelength responses collect by combining spectral data and RGB images collected with various specialized camera filters. The resulting neural network can predict how an input RGB image would appear across multiple spectral bands. To ensure accuracy, we implemented a verification process comparing our AI-generated multispectral predictions against images captured by actual multispectral cameras.
Results
- Achieved Near-IR correlation of above 0.91 for most samples tested. Samples ranged from fruits and plastic items all the way to soft tissue analogs.
