Uni-Sign: Toward Unified Sign Language Understanding at Scale
Authors: Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, Houqiang Li
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments across multiple SLU benchmarks demonstrate that Uni-Sign achieves state-of-the-art performance across multiple downstream SLU tasks. |
| Researcher Affiliation | Academia | 1 Mo E Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China 2 Institute of Artificial Intelligence, Hefei Comprehensive National Science Center 3 University of Texas at Austin EMAIL EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 Pseudocode of the score-aware sampling strategy in a Py Torch-like style. |
| Open Source Code | Yes | Dataset and code are available at github.com/ZechengLi19/Uni-Sign. |
| Open Datasets | Yes | We introduce CSL-News, a large-scale Chinese Sign Language (CSL) dataset containing 1,985 hours of video paired with textual annotations, which enables effective large-scale pre-training. Dataset and code are available at github.com/ZechengLi19/Uni-Sign. |
| Dataset Splits | Yes | For ISLR, we adopt WLASL (Li et al., 2020a) and MSASL (Joze & Koller, 2019) datasets for evaluation. For CSLR, we utilize CSL-Daily (Zhou et al., 2021). SLT task is conducted on the CSL-Daily, How2Sign (Duarte et al., 2021), and Open ASL (Shi et al., 2022) datasets. (Tables 4, 5, and 6 show results for 'Dev' and 'Test' splits for these datasets). |
| Hardware Specification | No | It was also supported by the GPU cluster built by MCC Lab of Information Science and Technology Institution, USTC, and the Supercomputing Center of the USTC. |
| Software Dependencies | No | We implement Uni-Sign using Py Torch (Paszke et al., 2019), employing m T5Base (Xue et al., 2021) as our pre-trained language model. |
| Experiment Setup | Yes | The detailed training recipe is presented in Table 2. Config Stage 1 Stage 2 Stage 3 optimizer Adam W base learning rate 3e-4 weight decay 1e-4 optimizer momentum β1, β2=0.9, 0.999 learning rate schedule cosine decay training epochs 20 5 20 batch size 16 4 8 gradient accumulation 8 8 1 |