Robotic Odor Source Localization via Vision and Olfaction Fusion Navigation Algorithm

Odor Source Localization (OSL) technology allows autonomous agents like mobile robots to find an unknown odor source in a given environment. An effective navigation algorithm that guides the robot to approach the odor source is the key to successfully locating the odor source. Downside of traditional olfaction-only OSL methods is that they struggle to localize odor sources in real-world environments with complex airflow. Our proposed solution integrates vision and olfaction sensor modalities to localize odor sources even if olfaction sensing is disrupted by turbulent airflow or vision sensing is impaired by environmental complexities. The model leverages the zero-shot multi-modal reasoning capabilities of large language models (LLMs), negating the requirement of manual knowledge encoding or custom-trained supervised learning models. A key feature of the proposed algorithm is the `High-level Reasoning' module, which encodes the olfaction and vision sensor data into a multi-modal prompt and instructs the LLM to employ a hierarchical reasoning process to select an appropriate high-level navigation behavior. Subsequently, the `Low-level Action' module translates the selected high-level navigation behavior into low-level action commands that can be executed by the mobile robot. To validate our method, we implemented the proposed algorithm on a mobile robot in a complex, real-world search environment that presents challenges to both olfaction and vision-sensing modalities. We compared the performance of our proposed algorithm to single sensory modality-based olfaction-only and vision-only navigation algorithms, and a supervised learning-based vision and olfaction fusion navigation algorithm. Experimental results demonstrate that multi-sensory navigation algorithms are statistically superior to single sensory navigation algorithms. The proposed algorithm outperformed the other algorithms in both laminar and turbulent airflow environments.

We trained a YOLOv7 model to detect visible odor plumes. If the model detects a visible odor plume, the navigation algorithm follows 'Vision-based Navigation'. Otherwise, it follows 'Olfaction-based Navigation' to localize the odor source.

Search area: The focus of the experiment is to test if the proposed navigation algorithm can avoid the obstacle and navigate to the odor source in laminar and turbulent airflow environments.

To determine the effectiveness of the multimodal fusion navigation algorithm, we compared the proposed algorithm to single sensory modality-based 'Olfaction-only' and 'Vision-only' navigation algorithms.

The robotic platform used for this task utilizes a Raspberry Pi Camera for vision sensing, an MQ3 alcohol detector and a WindSonic Anemometer for olfaction sensing, and a LDS-02 Laser Distance Sensor.

The results show that the proposed Fusion navigation algorithm (e1vo and e2vo) outperformed both the olfaction-only (e1o and e2o) and vision-only (e1v and e2v) navigation algorithms. The performance gap was greater in turbulent airflow environment (e2).