TAU-106K: A New Dataset for Comprehensive Understanding of Traffic Accident
Authors: Yixuan Zhou, Long Bai, Sijia Cai, Bing Deng, Xing Xu, Heng Tao Shen
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate TABot s superior performance in traffic accident understanding, highlighting not only its capabilities in high-level anomaly comprehension but also the robustness of the TAU-106K benchmark. Our code and data will be available at https://github.com/cool-xuan/TABot. |
| Researcher Affiliation | Collaboration | 1 Alibaba Cloud Computing 2 Tongji University (Tongji) |
| Pseudocode | Yes | question_templates = [ "What's happened during {} in the video?".format, "What's the incident in the period of {}?".format, "Maybe something wrong happened during {} in the provided video?".format, "What's the traffic situation in the period of {}?".format, "Is the traffic flow captured by the video normal during {}?".format, "Dose the accident happen during {} in the video?".format, "Does the video record any traffic disruptions or accidents around {}?".format, "Is there any indication of an abnormal traffic event during {}?".format, "Could you identify any mishaps in the time frame of {}?".format, "Are there signs of vehicular distress or accidents within {}?".format ] User: random.choice(question_templates)(annotation["accident_segments"]) <video> GPT: annotation['accident_description'] |
| Open Source Code | Yes | Our code and data will be available at https://github.com/cool-xuan/TABot. |
| Open Datasets | Yes | Our code and data will be available at https://github.com/cool-xuan/TABot. |
| Dataset Splits | Yes | For evaluation purposes, the TAU-106K dataset was split into training and testing sets in a 9:1 ratio, ensuring the same distribution of normal/accident instances and scene continuity across both. |
| Hardware Specification | Yes | We train LLM and both visual adapters of the Grounding GPT model through our TAU-106K dataset for 3 epochs using 8 H800 GPUs. |
| Software Dependencies | No | The paper mentions specific MLLMs used (Grounding GPT-7B, LLaMA70B) but does not provide specific version numbers for underlying software dependencies like programming languages or libraries (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | We train LLM and both visual adapters of the Grounding GPT model through our TAU-106K dataset for 3 epochs using 8 H800 GPUs. The initial learning rate is set to 2e-5 with a batch size of 32, requiring about 20 hours to complete. |