On the Adversarial Robustness of Camera-based 3D Object Detection
Authors: Shaoyuan Xie, Zichao Li, Zeyu Wang, Cihang Xie
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this study, we conduct the first comprehensive investigation of the robustness of leading camera-based 3D object detection approaches under various adversarial conditions. We systematically analyze the resilience of these models under two attack settings: white-box and black-box; focusing on two primary objectives: classification and localization. Additionally, we delve into two types of adversarial attack techniques: pixel-based and patch-based. Our experiments yield four interesting findings: (a) bird s-eye-view-based representations exhibit stronger robustness against localization attacks; (b) depth-estimation-free approaches have the potential to show stronger robustness; (c) accurate depth estimation effectively improves robustness for depth-estimation-based methods; (d) incorporating multi-frame benign inputs can effectively mitigate adversarial attacks. |
| Researcher Affiliation | Academia | Shaoyuan Xie EMAIL Department of Computer Science University of California, Irvine; Zichao Li EMAIL Department of Computer Science and Engineering University of California, Santa Cruz; Zeyu Wang EMAIL Department of Computer Science and Engineering University of California, Santa Cruz; Cihang Xie EMAIL Department of Computer Science and Engineering University of California, Santa Cruz |
| Pseudocode | No | The paper describes various attack algorithms (PGD-Adv, FGSM, C&W Attack, Auto PGD Attack) and their modifications in Section 4 'Generating Adversarial Examples', but it does not present them in structured pseudocode or algorithm blocks. The procedural steps are described within paragraph text. |
| Open Source Code | Yes | The code is available at: https://github.com/Daniel-xsy/BEV-Attack. |
| Open Datasets | Yes | Deep neural network-based 3D object detectors (...) have demonstrated promising performance across multiple challenging real-world benchmarks, including the KITTI (Geiger et al., 2012), nu Scenes (Caesar et al., 2020) and Waymo Open Dataset (Sun et al., 2020). |
| Dataset Splits | Yes | To thoroughly assess the model performance, we evaluate both the clean performance and adversarial robustness using the nu Scenes dataset. Given the substantial computational resources required for a full dataset evaluation, we opt for the nu Scenes-mini dataset when probing adversarial robustness. We report two metrics, Mean Average Precision (m AP) and nu Scenes Detection Score (NDS) (Caesar et al., 2020), in our experiments and discussions. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments, such as specific GPU or CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions several deep learning models and optimizers by name (e.g., Adam optimizer, ResNet, VoVNet, Swin Transformer), but it does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions) that would be needed to reproduce the experiments. |
| Experiment Setup | Yes | Pixel-based Attacks. We evaluate pixel-based adversarial attacks using perturbations under the L∞ norm. Our experiment setup fixes the maximum perturbation value at ϵ = 5 and the step size at α = 0.1. The process begins with the introduction of Gaussian noise to randomly perturb input images. Subsequently, we progressively increase the number of attack iterations, ranging from 1 to 50, for both untargeted and localization attacks. (...) For the Auto PGD attack, we use an iteration of 10, the momentum is 0.75 and the initial step size is 0.2ϵ. Patch-based Attacks. The initial patch pattern is generated using a Gaussian Distribution that has a mean and variance identical to the dataset. The attack iteration step size is designated as α = 5 and we maintained the iteration number at 50. The patch scale is incrementally increased from 0.1 to 0.4. |