Visual Reinforcement Learning with Residual Action
Authors: Zhenxian Liu, Peixi Peng, Yonghong Tian
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on Deep Mind Control Suite and CARLA demonstrate that the method could improve different RL baselines significantly, and achieve stateof-the-art performance. ... In this section, we first demonstrate significant improvements in sample efficiency and asymptotic performance brought by Res Act to baseline Deep RAD on two widely used benchmarks in RL, namely DMControl (Tassa et al. 2018)) and CARLA (Dosovitskiy et al. 2017). Subsequently, we conduct multiple ablation experiments to dive deeper into the design choices of Res Act. |
| Researcher Affiliation | Academia | 1National Engineering Research Center of Visual Technology, School of Computer Science, Peking University, China 2School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, China 3Peng Cheng Laboratory, China EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1: Pseudocode for Inference Procedure Black: unmodified Soft Actor-Critic. Orange: Observation Difference Learning. Blue: Residual Action Learning. |
| Open Source Code | Yes | Codes and appendix have also been made public1. 1https://github.com/Liu Zhenxian123/Res Act |
| Open Datasets | Yes | Extensive experiments on Open AI Deep Mind Control Suite (DMControl) (Tassa et al. 2018) and CARLA (Dosovitskiy et al. 2017) have demonstrated that Res Act significantly surpasses previous state-of-the-art. |
| Dataset Splits | No | The paper uses standard environments like Deep Mind Control Suite and CARLA tasks, and refers to evaluations at 100K, 500K, and 1M environment steps, but does not explicitly provide details about training/test/validation splits of a static dataset with percentages, sample counts, or specific predefined splits. It mentions '10 evaluation trajectories after training' but this is for evaluation, not for dataset partitioning. |
| Hardware Specification | No | Computing support was provided by Pengcheng Cloudbrain. No specific GPU, CPU models, or detailed computer specifications are provided for running experiments. |
| Software Dependencies | No | The paper does not explicitly mention any specific software dependencies with version numbers, such as Python, PyTorch, or CUDA versions, in the main text. |
| Experiment Setup | No | Building upon Deep RAD, the introduction of Res Act requires only minor modifications and does not necessitate any additional hyperparameters. ... We have adopted the settings for model structures and hyperparameters used by a range of approaches (Laskin, Srinivas, and Abbeel 2020; Laskin et al. 2020; Yarats, Kostrikov, and Fergus 2020; Zhang et al. 2020). For detailed settings of the experiments, please refer to Appendix G. The main text refers to other works and an appendix for settings, but does not provide specific hyperparameter values or training configurations itself. |