Knowledge Distillation from BERT Transformer to Speech Transformer for
Intent Classification

Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification

5 August 2021

Maulik C. Madhavi

Haizhou Li

Papers citing "Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification"

14 / 14 papers shown

Title
Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT Stanley Mugisha Rashid Kisitu Florence Tushabe 19 0 0 21 Apr 2025
Large Language Model-assisted Speech and Pointing Benefits Multiple 3D Object Selection in Virtual Reality Junlong Chen Jens Grubert Per Ola Kristensson 28 0 0 28 Oct 2024
Automatic Speech Recognition with BERT and CTC Transformers: A Review Noussaiba Djeffal Hamza Kheddar Djamel Addou A. Mazari Yassine Himeur 37 16 0 12 Oct 2024
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU Yan Li So-Eon Kim Seong-Bae Park S. Han 25 0 0 15 Aug 2024
An approach to optimize inference of the DIART speaker diarization pipeline Roman Aperdannier Sigurd Schacht Alexander Piazza 37 0 0 05 Aug 2024
Domain Adaptation in Intent Classification Systems: A Review Jesse Atuhurra Hidetaka Kamigaito Taro Watanabe Eric Nichols 42 0 0 26 Mar 2024
YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs Cyrus Zhou Zack Hassman Ruize Xu Dhirpal Shah Vaughn Richard Yanjing Li 32 1 0 01 Oct 2023
Improving End-to-End SLU performance with Prosodic Attention and Distillation Shangeth Rajaa 29 2 0 14 May 2023
Skit-S2I: An Indian Accented Speech to Intent dataset Shangeth Rajaa Swaraj Dalmia Kumarmanas Nethil 20 5 0 26 Dec 2022
Bottleneck Low-rank Transformers for Low-resource Spoken Language Understanding Pu Wang Hugo Van hamme VLM 21 4 0 28 Jun 2022
Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Models Ali Hadi Zadeh Mostafa Mahmoud Ameer Abdelhadi Andreas Moshovos MQ 24 31 0 23 Mar 2022
Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification Bidisha Sharma Maulik C. Madhavi Xuehao Zhou Haizhou Li 17 2 0 28 Sep 2021
Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification Bidisha Sharma Maulik C. Madhavi Haizhou Li 21 19 0 15 Feb 2021
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 252 927 0 24 Sep 2019