Exploring the time-domain deep attractor network with two-stream architectures in a reverberant environment

With the success of deep learning in speech signal processing, speaker-independent speech separation under the reverberant environment remains challenging. The deep attractor network (DAN) performs speech separation with speaker attractor, but it is conducted in the time-frequency domain, which is not optimal. The recently proposed convolutional time-domain audio separation network (Conv-TasNet) surpasses ideal masks in anechoic signals, while its architecture renders the problem of separating signals with variable numbers of speakers. Moreover, these models will suffer performance degradation in a reverberant environment. In this study, we propose a time-domain deep attractor network (TD-DAN) with two-stream convolutional networks, which efficiently performs both dereverberation and separation tasks under the condition of variable numbers of speakers. The speaker encoding stream (SES) of TD-DAN models speaker information, and is explored with various waveform encoders. The speech decoding steam (SDS) accepts speaker attractors from SES, and learns to predict early reflections. Experiment results demonstrated that the TD-DAN achieved scale-invariant source-to-distortion ratio (SI-SDR) gains of 10.40/9.78 dB and 9.15/7.92 dB on the reverberant 2- and 3-speaker development/evaluation set, exceeding Conv-TasNet 1.55/1.33 dB and 0.94/1.21 dB, respectively.