FMLLR

Uses your system voice; voice and rate in settings.

Snapshot

≈ 5 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In signal processing, Feature space Maximum Likelihood Linear Regression (fMLLR) is a global feature transform that are typically applied in a speaker adaptive way, where fMLLR transforms acoustic features to speaker adapted features by a multiplication operation with a transformation matrix. In some literature, fMLLR is also known as the Constrained Maximum Likelihood Linear Regression (cMLLR).

In signal processing, Feature space Maximum Likelihood Linear Regression (fMLLR) is a global feature transform that are typically applied in a speaker adaptive way, where fMLLR transforms acoustic features to speaker adapted features by a multiplication operation with a transformation matrix. In some literature, fMLLR is also known as the Constrained Maximum Likelihood Linear Regression (cMLLR).

Overview

fMLLR transformations are trained in a maximum likelihood sense on adaptation data. These transformations may be estimated in many ways, but only maximum likelihood (ML) estimation is considered in fMLLR. The fMLLR transformation is trained on a particular set of adaptation data, such that it maximizes the likelihood of that adaptation data given a current model-set. This technique is a widely used approach for speaker adaptation in HMM-based speech recognition.

Later research also shows that fMLLR is an excellent acoustic feature for DNN/HMM hybrid speech recognition models.

The advantage of fMLLR includes the following: the adaptation process can be performed within a pre-processing phase, and is independent of the ASR training and decoding process. this type of adapted feature can be applied to deep neural networks (DNN) to replace traditionally used mel-spectrogram in end-to-end speech recognition models. fMLLR's speaker adaptation process leads to a significant performance boost for ASR models, hence outperforming other transform or features like MFCCs (Mel-Frequency Cepstral Coefficients) and FBANKs (Filter bank) coefficients. fMLLR features can be efficiently realized with speech toolkits like Kaldi.

Major problem and disadvantage of fMLLR: when the amount of adaptation data is limited, the transformation matrices tends to easily overfit the given data. In the Kaldi formulation, fMLLR is an affine feature transform of the form x → A x +b , which can be written in the form x →W \hat{x} , where \hat{x} = \begin{bmatrix} x \\ 1 \end{bmatrix} is the acoustic feature x with a 1 appended. Note that this differs from some of the literature where the 1 comes first as \hat{x} = \begin{bmatrix} 1 \\ x \end{bmatrix} .

The sufficient statistics stored are: K=\sum_{t,j,m}\gamma_{j,m}(t)\textstyle \Sigma_{jm}^{-1}\mu_{jm}x(t)^+ \displaystyle where \textstyle \Sigma_{jm}^{-1} \displaystyle is the inverse co-variance matrix.

And for 0 \leq i \leq D where D is the feature dimension: G^{(i)}=\sum_{t,j,m}\gamma_{j,m}(t)\left ( \frac{1}{\sigma^2_{j,m}(i)} \right )x(t)^+x(t)^{+T} \displaystyle For a thorough review that explains fMLLR and the commonly used estimation techniques, see the original paper "Maximum likelihood linear transformations for HMM-based speech recognition ". Note that the Kaldi script that performs the feature transforms of fMLLR differs with by using a column of the inverse in place of the cofactor row.

In other words, the factor of the determinant is ignored, as it does not affect the transform result and can causes potential danger of numerical underflow or overflow. If running on a single machine, change the following lines in $KALDI_ROOT/egs/librispeech/s5/cmd.sh to replace queue.pl to run.pl : export train_cmd="run.pl --mem 2G" export decode_cmd="run.pl --mem 4G" export mkgraph_cmd="run.pl --mem 8G" Change the data path in run.sh to your LibriSpeech data path, the directory LibriSpeech/ should be under that path.

For example: data=/media/user/SSD # example path Install flac with: sudo apt-get install flac Run the Kaldi recipe run.sh for LibriSpeech at least until Stage 13 (included), for simplicity you can use the modified run.sh.

Copy exp/tri4b/trans.* files into exp/tri4b/decode_tgsmall_train_clean_*/ with the following command: mkdir exp/tri4b/decode_tgsmall_train_clean_100 && cp exp/tri4b/trans.* exp/tri4b/decode_tgsmall_train_clean_100/ Compute the fMLLR features by running the following script, the script can also be downloaded here: !/bin/bash . ./cmd.sh ## You'll want to change cmd.sh to something that will work on your system. . ./path.sh ## Source the tools/utils (import the queue.pl) gmmdir=exp/tri4b for chunk in dev_clean test_clean train_clean_100 train_clean_360 ; do dir=fmllr/$chunk steps/nnet/make_fmllr_feats.sh --nj 10 --cmd "$train_cmd" \ --transform-dir $gmmdir/decode_tgsmall_$chunk \ $dir data/$chunk $gmmdir $dir/log $dir/data || exit 1 compute-cmvn-stats --spk2utt=ark:data/$chunk/spk2utt scp:fmllr/$chunk/feats.scp ark:$dir/data/cmvn_speaker.ark done Compute alignments using: alignments on dev_clean and test_clean steps/align_fmllr.sh --nj 10 data/dev_clean data/lang exp/tri4b exp/tri4b_ali_dev_clean steps/align_fmllr.sh --nj 10 data/test_clean data/lang exp/tri4b exp/tri4b_ali_test_clean steps/align_fmllr.sh --nj 30 data/train_clean_100 data/lang exp/tri4b exp/tri4b_ali_clean_100 steps/align_fmllr.sh --nj 30 data/train_clean_360 data/lang exp/tri4b exp/tri4b_ali_clean_360 Apply CMVN and dump the fMLLR features to new .ark files, the script can also be downloaded here: !/bin/bash data=/user/kaldi/egs/librispeech/s5 ## You'll want to change this path to something that will work on your system. rm -rf $data/fmllr_cmvn/ mkdir $data/fmllr_cmvn/ for part in dev_clean test_clean train_clean_100 train_clean_360; do mkdir $data/fmllr_cmvn/$part/ apply-cmvn --utt2spk=ark:$data/fmllr/$part/utt2spk ark:$data/fmllr/$part/data/cmvn_speaker.ark scp:$data/fmllr/$part/feats.scp ark:- | add-deltas --delta-order=0 ark:- ark:$data/fmllr_cmvn/$part/fmllr_cmvn.ark done du -sh $data/fmllr_cmvn/* echo "Done!" Use the Python script to convert Kaldi generated .ark features to .npy for your own dataloader, an example Python script is provided: python ark2libri.py

See also

Feature scaling MFCC FBANK Kaldi

References

Text derived from https://en.wikipedia.org/w/index.php?oldid=1374345994 under CC BY-SA 4.0; modifications have been made.

Provenance

snapshot enwiki-001374345994 · captured 1789123889.0 · version history

Cite this page as: encyclopegia.org, “FMLLR”, snapshot enwiki-001374345994. https://encyclopegia.org/a/fmllr

Sources cited in this revision

1 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.