Detecting and Mitigating System-Level Anomalies of Vision-Based Controllers

Aryaman Gupta*1

Kaustav Chakraborty*2

Somil Bansal2

1 Indian Institute of Technology (BHU), Varanasi 2 University of Southern California * Equal Contribution

---

Abstract

Autonomous systems, such as self-driving cars and drones, have made significant strides in recent years by leveraging visual inputs and machine learning for decision- making and control. Despite their impressive performance, these vision-based controllers can make erroneous predictions when faced with novel or out-of-distribution inputs. Such errors can cascade to catastrophic system failures and compromise system safety. In this work, we introduce a run-time anomaly monitor to detect and mitigate such closed-loop, system-level failures. Specifically, we leverage a reachability-based frame- work to stress-test the vision-based controller offline and mine its system-level failures. This data is then used to train a classifier that is leveraged online to flag inputs that might cause system breakdowns. The anomaly detector highlights issues that transcend individual modules and pertain to the safety of the overall system. We also design a fallback controller that robustly handles these detected anomalies to preserve system safety. We validate the proposed approach on an autonomous aircraft taxiing system that uses a vision-based controller for taxiing. Our results show the efficacy of the proposed approach in identifying and handling system-level anomalies, outperforming methods such as prediction error- based detection, and ensembling, thereby enhancing the overall safety and robustness of autonomous systems.

---

Paper

Gupta, Chakraborty, Bansal Detecting and Mitigating System-Level Anomalies of Vision-Based Controllers

[pdf]

[Bibtex]

---

Code

 [GitHub]

---

Results

Fig. (a, b) Trajectory followed by the aircraft under the TaxiNet controller (dashed black line) and the safety pipeline (red line). The color shift in the red curve shows velocity variation due to the fallback controller. (c) The grey region represents the system BRT under the TaxiNet controller, and the blue region represents the BRT under the safety pipeline. The BRT obtained using the AD and the fallback controller is appreciably smaller than the one obtained using vanilla TaxiNet. (d) Input image at the start state in (a), causing system failure due to runway boundaries. (e) Input image at the start state in (b), causing system failure due to runway markings.

(Top) TaxiNet controller failure due to starting close to runway boundary(left), Trajectory correction by our safety pipeline (right), (Bottom) TaxiNet controller failure due to white runway markings (left), Trajectory correction by our safety pipeline (right).

---

This webpage template was borrowed from some colorful folks.