PerReactor: Offline Personalised Multiple Appropriate Facial Reaction Generation
Authors: Hengde Zhu, Xiangyu Kong, Weicheng Xie, Xin Huang, Xilin He, Lu Liu, Linlin Shen, Wei Zhang, Hatice Gunes, Siyang Song
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that our Per Reactor not only largely outperformed all existing offline solutions for generating appropriate, diverse and realistic facial reactions, but also is the first offline approach that can effectively generate personalised appropriate facial reactions. ... Table 1 compares our Per Reactor with MAFRG competitors on both offline MAFRG and PMAFRG tasks... Several ablation studies are conducted below to deeply investigate our approach. |
| Researcher Affiliation | Collaboration | 1School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK 2Department of Computer Science, University of Exeter, Exeter, UK 3Affect AI, Anhui, China 4Computer Vision Institute, School of Computer Science & Software Engineering, Shenzhen University, Shenzhen, China 5School of Electronic Science and Engineering (School of Microelectronics), South China Normal University, Foshan, China 6School of Software Technology, Zhejiang University, Hangzhou, China 7Department of Computer Science and Technology, University of Cambridge, Cambridge, UK. |
| Pseudocode | Yes | The pseudocode of this process is provided in the Supplementary Material. |
| Open Source Code | Yes | Code https://github.com/Affect AI/Per Reactor |
| Open Datasets | Yes | Our approach is evaluated on a publicly available MAFRG dataset provided by REACT2024 Challenge 1... These clips are originally recorded by No XI (Cafaro et al. 2017) and RECOLA (Ringeval et al. 2013). |
| Dataset Splits | Yes | It contains 2962 dyadic interaction audio-visual clip pairs (5924 clips in total), including 1594 pairs for training, 562 pairs for validation and 806 for test. |
| Hardware Specification | No | This research used the ALICE High Performance Computing facility at the University of Leicester and the Sulis Tier 2 HPC platform hosted by the Scientific Computing Research Technology Platform at the University of Warwick. Sulis is funded by EPSRC Grant EP/T022108/1 and the HPC Midlands+ consortium. |
| Software Dependencies | No | In our experiments, we utilise the Adam W optimizer (Loshchilov and Hutter 2017) to train our Per Reactor. |
| Experiment Setup | Yes | The default coefficients λd and λm for balancing the regularisation losses are set to 10 and 10 2, respectively. More details are provided in Supplementary Material. |