baseline model checkpoint. training batch (4e6405d4) · Commits · kotzaneck / IRL Machine Translation Project

batch_output/IWSLT14-Baseline-Training-output.txt

0 → 100644

+152 −0

Original line number	Diff line number	Diff line
		2021-01-21 09:23:56,256 - INFO - root - Hello! This is Joey-NMT (version 1.0).
		/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/field.py:150: UserWarning: Field class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
		warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
		2021-01-21 09:23:56,299 - INFO - joeynmt.data - loading training data...
		/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/example.py:78: UserWarning: Example class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
		warnings.warn('Example class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.', UserWarning)
		2021-01-21 09:24:03,837 - INFO - joeynmt.data - building vocabulary...
		2021-01-21 09:24:24,807 - INFO - joeynmt.data - loading dev data...
		2021-01-21 09:24:25,121 - INFO - joeynmt.data - loading test data...
		2021-01-21 09:24:25,401 - INFO - joeynmt.data - data loaded.
		2021-01-21 09:24:27,027 - INFO - joeynmt.training - Total params: 60662860
		2021-01-21 09:24:33,542 - INFO - joeynmt.helpers - cfg.name : iwslt14-deen-bpe
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.src : de
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.trg : en
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.train : test/data/iwslt14/train.bpe.32000
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.dev : test/data/iwslt14/valid.bpe.32000
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.test : test/data/iwslt14/test.bpe.32000
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.level : bpe
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.lowercase : True
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.max_sent_length : 62
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.src_voc_min_freq : 1
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.src_voc_limit : 32000
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.trg_voc_min_freq : 1
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.trg_voc_limit : 32000
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.testing.beam_size : 5
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.testing.alpha : 1.0
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.optimizer : adam
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.normalization : tokens
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.learning_rate : 0.0002
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.batch_size : 40
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.scheduling : plateau
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.patience : 8
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.weight_decay : 0.0
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.decrease_factor : 0.7
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.early_stopping_metric : loss
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.epochs : 100
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.validation_freq : 2000
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.logging_freq : 500
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.eval_metric : bleu
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.model_dir : models/iwslt14_deen_bpe
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.overwrite : True
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.shuffle : True
		2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.use_cuda : True
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.training.max_output_length : 80
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.training.print_valid_sents : [0, 1, 2, 3, 4]
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.rnn_type : gru
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.embeddings.embedding_dim : 620
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.embeddings.scale : False
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.hidden_size : 500
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.bidirectional : True
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.dropout : 0.0
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.num_layers : 1
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.rnn_type : gru
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.embeddings.embedding_dim : 620
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.embeddings.scale : False
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.emb_scale : False
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.hidden_size : 1000
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.dropout : 0.0
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.hidden_dropout : 0.2
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.num_layers : 1
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.input_feeding : True
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.init_hidden : bridge
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.attention : bahdanau
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - Data set sizes:
		train 156191,
		valid 7245,
		test 6750
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - First training example:
		[SRC] und was menschliche gesundheit ist , kann auch ziemlich kompliziert sein .
		[TRG] and it can be a very complicated thing , what human health is .
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - First 10 words (src): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) , (5) . (6) und (7) die (8) sie (9) das
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - First 10 words (trg): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) , (5) . (6) the (7) and (8) to (9) of
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - Number of Src words (types): 24994
		2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - Number of Trg words (types): 17859
		2021-01-21 09:24:33,545 - INFO - joeynmt.training - Model(
		encoder=RecurrentEncoder(GRU(620, 500, batch_first=True, bidirectional=True)),
		decoder=RecurrentDecoder(rnn=GRU(1620, 1000, batch_first=True), attention=BahdanauAttention),
		src_embed=Embeddings(embedding_dim=620, vocab_size=24994),
		trg_embed=Embeddings(embedding_dim=620, vocab_size=17859))
		/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/iterator.py:48: UserWarning: BucketIterator class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
		warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
		2021-01-21 09:24:33,572 - INFO - joeynmt.training - Train stats:
		device: cuda
		n_gpu: 1
		16-bits training: False
		gradient accumulation: 1
		batch size per device: 40
		total batch size (w. parallel & accumulation): 40
		2021-01-21 09:24:33,572 - INFO - joeynmt.training - EPOCH 1
		/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/batch.py:23: UserWarning: Batch class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
		warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
		2021-01-21 09:35:01,799 - INFO - joeynmt.training - Epoch 1, Step: 500, Batch Loss: 5.557352, Tokens per Sec: 674, Lr: 0.000200
		2021-01-21 09:45:24,551 - INFO - joeynmt.training - Epoch 1, Step: 1000, Batch Loss: 5.937672, Tokens per Sec: 671, Lr: 0.000200
		2021-01-21 09:55:40,682 - INFO - joeynmt.training - Epoch 1, Step: 1500, Batch Loss: 5.576888, Tokens per Sec: 685, Lr: 0.000200
		2021-01-21 10:06:13,392 - INFO - joeynmt.training - Epoch 1, Step: 2000, Batch Loss: 5.395145, Tokens per Sec: 663, Lr: 0.000200
		2021-01-21 10:16:20,640 - WARNING - sacrebleu - That's 100 lines that end in a tokenized period ('.')
		2021-01-21 10:16:20,641 - WARNING - sacrebleu - It looks like you forgot to detokenize your test data, which may hurt your score.
		2021-01-21 10:16:20,641 - WARNING - sacrebleu - If you insist your data is detokenized, or don't care, you can suppress this message with '--force'.
		2021-01-21 10:16:24,678 - INFO - joeynmt.training - Hooray! New best validation result [loss]!
		2021-01-21 10:16:24,679 - INFO - joeynmt.training - Saving new checkpoint.
		2021-01-21 10:16:34,410 - INFO - joeynmt.training - Example #0
		2021-01-21 10:16:34,411 - INFO - joeynmt.training - Source: es ist diese pyramide .
		2021-01-21 10:16:34,411 - INFO - joeynmt.training - Reference: it 's that pyramid .
		2021-01-21 10:16:34,411 - INFO - joeynmt.training - Hypothesis: it 's a lot of the .
		2021-01-21 10:16:34,411 - INFO - joeynmt.training - Example #1
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Source: durch die muttermilch .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Reference: in mother 's milk .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Hypothesis: and the first is .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Example #2
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Source: es enthielt das zwei- , drei- , bis 400-fache des grenzwerts an schadstoffen der laut epa erlaubt war .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Reference: it had two-to-three-to-400 times the toxic loads ever allowed by the epa .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Hypothesis: and we 're the first , and we 're the lot of the world , and we 're the lot of the world .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Example #3
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Source: oft ist es abwasser , was uns verstopft .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Reference: often what jams us up is sewage .
		2021-01-21 10:16:34,412 - INFO - joeynmt.training - Hypothesis: and we 're a 't have a lot of the world .
		2021-01-21 10:16:34,413 - INFO - joeynmt.training - Example #4
		2021-01-21 10:16:34,413 - INFO - joeynmt.training - Source: was macht man , wenn man solch eine unterbrechung im fluss hat ?
		2021-01-21 10:16:34,413 - INFO - joeynmt.training - Reference: what do you do when you have this sort of disrupted flow ?
		2021-01-21 10:16:34,413 - INFO - joeynmt.training - Hypothesis: and i 'm going to be the lot of the way , " what 's you do ?
		2021-01-21 10:16:34,413 - INFO - joeynmt.training - Validation result (greedy) at epoch 1, step 2000: bleu: 3.68, loss: 850449.3750, ppl: 201.7941, duration: 621.0200s
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_yticklabels(row_labels, minor=False)
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_yticklabels(row_labels, minor=False)
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_yticklabels(row_labels, minor=False)
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_yticklabels(row_labels, minor=False)
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_yticklabels(row_labels, minor=False)
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
		/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
		ax.set_yticklabels(row_labels, minor=False)
		/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/batch.py:23: UserWarning: Batch class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
		warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
		2021-01-21 10:27:11,738 - INFO - joeynmt.training - Epoch 1, Step: 2500, Batch Loss: 4.557628, Tokens per Sec: 660, Lr: 0.000200
		2021-01-21 10:37:37,498 - INFO - joeynmt.training - Epoch 1, Step: 3000, Batch Loss: 4.029715, Tokens per Sec: 672, Lr: 0.000200
		2021-01-21 10:47:54,337 - INFO - joeynmt.training - Epoch 1, Step: 3500, Batch Loss: 4.919943, Tokens per Sec: 682, Lr: 0.000200
		2021-01-21 10:56:31,294 - INFO - joeynmt.training - Epoch 1: total training loss 20363.30
		2021-01-21 10:56:31,295 - INFO - joeynmt.training - EPOCH 2

batch_output/nvcc-output.txt

0 → 100644

+1 −0

Original line number	Diff line number	Diff line
		test

configs/iwslt14_deen_bpe.yaml

+1 −1

Original line number	Diff line number	Diff line
		@@ -22,7 +22,7 @@ training:
		optimizer: "adam"
		normalization: "tokens"
		learning_rate: 0.0002
		batch_size: 80
		batch_size: 40
		scheduling: "plateau"
		patience: 8
		#clip_grad_norm: 1.0

models/iwslt14_deen_bpe/2000.ckpt

0 → 100644

+694 MiB

File added.

No diff preview for this file type.

View file

models/iwslt14_deen_bpe/2000.hyps

0 → 100644

+7245 −0

File added.

File size exceeds preview limit.

View file