HYBRID SPEECH RECOGNITIONWITH DEEP BIDIRECTIONAL LSTM

Abstract

Deep Bidirectional LSTM (DBLSTM) recurrent neural net-works have recently been shown to give state-of-the-art per-formance on the TIMIT speech database. However, the re-sults in that work relied on recurrent-neural-network-specific objective functions, which are difficult to integrate with exist-ing large vocabulary speech recognition systems. This paper investigates the use of DBLSTM as an acoustic model in a standard neural network-HMM hybrid system. We find that a DBLSTM-HMM hybrid gives equally good results on TIMIT as the previous work. It also outperforms both GMM and deep network benchmarks on a subset of the Wall Street Jour-nal corpus. However the improvement in word error rate over the deep network is modest, despite a great increase in frame-level accuracy. We conclude that the hybrid approach with DBLSTM appears to be well suited for tasks where acous-tic modelling predominates. Further investigation needs to be conducted to understand how to better leverage the improve-ments in frame-level accuracy towards better word error rates. Index Terms — DBLSTM, HMM-RNN hybrid 1.