Scaling up Multimodal Pre-training for Sign Language Understanding

16 August 2024

Wengang Zhou

Papers citing "Scaling up Multimodal Pre-training for Sign Language Understanding"

27 / 27 papers shown

Title
MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognition Weichao Zhao Hezhen Hu Wen-gang Zhou Yunyao Mao Min Wang Houqiang Li SLR 65 10 0 31 May 2024
Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining Benjia Zhou Zhigang Chen Albert Clapés Jun Wan Yanyan Liang Sergio Escalera Zhen Lei Du Zhang SLR 68 55 0 27 Jul 2023
SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding Hezhen Hu Weichao Zhao Wen-gang Zhou Houqiang Li ViT 62 74 0 08 May 2023
CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning Yiting Cheng Fangyun Wei Jianmin Bao Dong Chen Wenqian Zhang SLR 48 30 0 22 Mar 2023
A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation Yutong Chen Fangyun Wei Xiao Sun Zhirong Wu Stephen Lin SLR 61 103 0 08 Mar 2022
Sign Language Video Retrieval with Free-Form Textual Queries A. Duarte Samuel Albanie Xavier Giró-i-Nieto Gül Varol SLR 67 29 0 07 Jan 2022
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 861 29,341 0 26 Feb 2021
TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation Dongxu Li Chenchen Xu Xin Yu Kaihao Zhang Ben Swift H. Suominen Hongdong Li SLR 38 123 0 12 Oct 2020
Watch, read and lookup: learning to spot signs from multiple supervisors Liliane Momeni Gül Varol Samuel Albanie Triantafyllos Afouras Andrew Zisserman 59 44 0 08 Oct 2020
Global-local Enhancement Network for NMFs-aware Sign Language Recognition Hezhen Hu Wen-gang Zhou Junfu Pu Houqiang Li SLR 48 54 0 24 Aug 2020
Quantitative Survey of the State of the Art in Sign Language Recognition Oscar Koller SLR 44 95 0 22 Aug 2020
How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language A. Duarte Shruti Palaskar Lucas Ventura Deepti Ghadiyaram Kenneth DeHaan Florian Metze Jordi Torres Xavier Giró-i-Nieto SLR 30 204 0 18 Aug 2020
Whole-Body Human Pose Estimation in the Wild Sheng Jin Lumin Xu Jin Xu Can Wang Wentao Liu Chao Qian Wanli Ouyang Ping Luo 3DH 174 246 0 23 Jul 2020
Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation Necati Cihan Camgöz Oscar Koller Simon Hadfield Richard Bowden SLR 76 506 0 30 Mar 2020
Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition Hao Zhou Wen-gang Zhou Yun Zhou Houqiang Li NoLa 51 200 0 08 Feb 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 439 42,393 0 03 Dec 2019
Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison Dongxu Li Cristian Rodriguez-Opazo Xin Yu Hongdong Li SLR 38 441 0 24 Oct 2019
Distribution-Aware Coordinate Representation for Human Pose Estimation Feng Zhang Xiatian Zhu Hanbin Dai Mao Ye Ce Zhu 3DH 68 423 0 14 Oct 2019
Deep High-Resolution Representation Learning for Human Pose Estimation Ke Sun Bin Xiao Dong Liu Jingdong Wang 3DV 120 4,049 0 25 Feb 2019
SlowFast Networks for Video Recognition Christoph Feichtenhofer Haoqi Fan Jitendra Malik Kaiming He 164 3,272 0 10 Dec 2018
TSM: Temporal Shift Module for Efficient Video Understanding Ji Lin Chuang Gan Song Han 85 1,688 0 20 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,729 0 11 Oct 2018
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition Sijie Yan Yuanjun Xiong Dahua Lin GNN 232 4,161 0 23 Jan 2018
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset João Carreira Andrew Zisserman 221 8,012 0 22 May 2017
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 374 7,959 0 17 Aug 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.7K 150,006 0 22 Dec 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 533 27,295 0 01 Sep 2014