Commit 7e47d204 authored by hubert's avatar hubert
Browse files

fix first few format errors

parent 37deb363
Loading
Loading
Loading
Loading
+10 −8
Original line number Diff line number Diff line
# knn_ast_KD_nmt

Implementation of Imitation-based Knowledge Distillation [1] from the paper "Autoregressive Knowledge Distillation through Imitation Learning" [2] for Automatic Speech Translation (AST).
## general information

Implementation of [Imitation-based Knowledge Distillation][https://github.com/asappresearch/imitkd] from the paper ["Autoregressive Knowledge Distillation through Imitation Learning"][https://arxiv.org/abs/2009.07253] for Automatic Speech Translation (AST).
Instead of an AST expert, The expert model is a trained Neural Machine Translation model.

The implementation is entirely based the fairseq framework [3], specifically on the speech-to-text module [4].
For usage of the fairseq framework please see the fairseq documentation [5].
The implementation is entirely based the [fairseq framework][https://github.com/facebookresearch/fairseq], specifically on the [speech-to-text module][https://github.com/facebookresearch/fairseq/tree/main/examples/speech_to_text].
For usage of the fairseq framework please see the [fairseq documentation][https://fairseq.readthedocs.io/en/latest/].


In order for ImitKD to work, several changes were made to the fairseq framework:
@@ -20,11 +22,11 @@ In order for ImitKD to work, several changes were made to the fairseq framework:


The best way to run experiments with generated transcripts is to:
    - use the ASR model to transcribe the speech data
    - use the NMT expert model to translate those transcripts if you want to use generated target translations
    - run `create_wmt19_generated_dataset.py` to create a new dataset of generated trancripts:
        - `python create_wmt19_generated_dataset.py -o ${fairseq-generate log file of NMT expert's translations} -a ${fairseq-generate log file of ASR model's transcripts} -d ${AST dataset file}`
    - use the new dataset just as the original datasets 
    1. use the ASR model to transcribe the speech data
    2. use the NMT expert model to translate those transcripts if you want to use generated target translations
    3. run `create_wmt19_generated_dataset.py` to create a new dataset of generated trancripts:
        ``python create_wmt19_generated_dataset.py -o ${fairseq-generate log file of NMT expert's translations} -a ${fairseq-generate log file of ASR model's transcripts} -d ${AST dataset file}``
    4. use the new dataset just as the original datasets