Commit 4e6405d4 authored by kotzaneck's avatar kotzaneck
Browse files

baseline model checkpoint. training batch

parent c084ca4e
Loading
Loading
Loading
Loading
+152 −0
Original line number Diff line number Diff line
2021-01-21 09:23:56,256 - INFO - root - Hello! This is Joey-NMT (version 1.0).
/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/field.py:150: UserWarning: Field class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
  warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
2021-01-21 09:23:56,299 - INFO - joeynmt.data - loading training data...
/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/example.py:78: UserWarning: Example class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
  warnings.warn('Example class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.', UserWarning)
2021-01-21 09:24:03,837 - INFO - joeynmt.data - building vocabulary...
2021-01-21 09:24:24,807 - INFO - joeynmt.data - loading dev data...
2021-01-21 09:24:25,121 - INFO - joeynmt.data - loading test data...
2021-01-21 09:24:25,401 - INFO - joeynmt.data - data loaded.
2021-01-21 09:24:27,027 - INFO - joeynmt.training - Total params: 60662860
2021-01-21 09:24:33,542 - INFO - joeynmt.helpers - cfg.name                           : iwslt14-deen-bpe
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.src                       : de
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.trg                       : en
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.train                     : test/data/iwslt14/train.bpe.32000
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.dev                       : test/data/iwslt14/valid.bpe.32000
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.test                      : test/data/iwslt14/test.bpe.32000
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.level                     : bpe
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.lowercase                 : True
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.max_sent_length           : 62
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.src_voc_min_freq          : 1
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.src_voc_limit             : 32000
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.trg_voc_min_freq          : 1
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.data.trg_voc_limit             : 32000
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.testing.beam_size              : 5
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.testing.alpha                  : 1.0
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.optimizer             : adam
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.normalization         : tokens
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.learning_rate         : 0.0002
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.batch_size            : 40
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.scheduling            : plateau
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.patience              : 8
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.weight_decay          : 0.0
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.decrease_factor       : 0.7
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.early_stopping_metric : loss
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.epochs                : 100
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.validation_freq       : 2000
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.logging_freq          : 500
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.eval_metric           : bleu
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.model_dir             : models/iwslt14_deen_bpe
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.overwrite             : True
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.shuffle               : True
2021-01-21 09:24:33,543 - INFO - joeynmt.helpers - cfg.training.use_cuda              : True
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.training.max_output_length     : 80
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.training.print_valid_sents     : [0, 1, 2, 3, 4]
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.rnn_type         : gru
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.embeddings.embedding_dim : 620
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.embeddings.scale : False
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.hidden_size      : 500
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.bidirectional    : True
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.dropout          : 0.0
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.encoder.num_layers       : 1
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.rnn_type         : gru
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.embeddings.embedding_dim : 620
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.embeddings.scale : False
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.emb_scale        : False
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.hidden_size      : 1000
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.dropout          : 0.0
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.hidden_dropout   : 0.2
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.num_layers       : 1
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.input_feeding    : True
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.init_hidden      : bridge
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - cfg.model.decoder.attention        : bahdanau
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - Data set sizes: 
	train 156191,
	valid 7245,
	test 6750
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - First training example:
	[SRC] und was menschliche gesundheit ist , kann auch ziemlich kompliziert sein .
	[TRG] and it can be a very complicated thing , what human health is .
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - First 10 words (src): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) , (5) . (6) und (7) die (8) sie (9) das
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - First 10 words (trg): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) , (5) . (6) the (7) and (8) to (9) of
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - Number of Src words (types): 24994
2021-01-21 09:24:33,544 - INFO - joeynmt.helpers - Number of Trg words (types): 17859
2021-01-21 09:24:33,545 - INFO - joeynmt.training - Model(
	encoder=RecurrentEncoder(GRU(620, 500, batch_first=True, bidirectional=True)),
	decoder=RecurrentDecoder(rnn=GRU(1620, 1000, batch_first=True), attention=BahdanauAttention),
	src_embed=Embeddings(embedding_dim=620, vocab_size=24994),
	trg_embed=Embeddings(embedding_dim=620, vocab_size=17859))
/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/iterator.py:48: UserWarning: BucketIterator class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
  warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
2021-01-21 09:24:33,572 - INFO - joeynmt.training - Train stats:
	device: cuda
	n_gpu: 1
	16-bits training: False
	gradient accumulation: 1
	batch size per device: 40
	total batch size (w. parallel & accumulation): 40
2021-01-21 09:24:33,572 - INFO - joeynmt.training - EPOCH 1
/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/batch.py:23: UserWarning: Batch class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
  warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
2021-01-21 09:35:01,799 - INFO - joeynmt.training - Epoch   1, Step:      500, Batch Loss:     5.557352, Tokens per Sec:      674, Lr: 0.000200
2021-01-21 09:45:24,551 - INFO - joeynmt.training - Epoch   1, Step:     1000, Batch Loss:     5.937672, Tokens per Sec:      671, Lr: 0.000200
2021-01-21 09:55:40,682 - INFO - joeynmt.training - Epoch   1, Step:     1500, Batch Loss:     5.576888, Tokens per Sec:      685, Lr: 0.000200
2021-01-21 10:06:13,392 - INFO - joeynmt.training - Epoch   1, Step:     2000, Batch Loss:     5.395145, Tokens per Sec:      663, Lr: 0.000200
2021-01-21 10:16:20,640 - WARNING - sacrebleu - That's 100 lines that end in a tokenized period ('.')
2021-01-21 10:16:20,641 - WARNING - sacrebleu - It looks like you forgot to detokenize your test data, which may hurt your score.
2021-01-21 10:16:20,641 - WARNING - sacrebleu - If you insist your data is detokenized, or don't care, you can suppress this message with '--force'.
2021-01-21 10:16:24,678 - INFO - joeynmt.training - Hooray! New best validation result [loss]!
2021-01-21 10:16:24,679 - INFO - joeynmt.training - Saving new checkpoint.
2021-01-21 10:16:34,410 - INFO - joeynmt.training - Example #0
2021-01-21 10:16:34,411 - INFO - joeynmt.training - 	Source:     es ist diese pyramide .
2021-01-21 10:16:34,411 - INFO - joeynmt.training - 	Reference:  it &apos;s that pyramid .
2021-01-21 10:16:34,411 - INFO - joeynmt.training - 	Hypothesis: it &apos;s a lot of the .
2021-01-21 10:16:34,411 - INFO - joeynmt.training - Example #1
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Source:     durch die muttermilch .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Reference:  in mother &apos;s milk .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Hypothesis: and the first is .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - Example #2
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Source:     es enthielt das zwei- , drei- , bis 400-fache des grenzwerts an schadstoffen der laut epa erlaubt war .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Reference:  it had two-to-three-to-400 times the toxic loads ever allowed by the epa .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Hypothesis: and we &apos;re the first , and we &apos;re the lot of the world , and we &apos;re the lot of the world .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - Example #3
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Source:     oft ist es abwasser , was uns verstopft .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Reference:  often what jams us up is sewage .
2021-01-21 10:16:34,412 - INFO - joeynmt.training - 	Hypothesis: and we &apos;re a &apos;t have a lot of the world .
2021-01-21 10:16:34,413 - INFO - joeynmt.training - Example #4
2021-01-21 10:16:34,413 - INFO - joeynmt.training - 	Source:     was macht man , wenn man solch eine unterbrechung im fluss hat ?
2021-01-21 10:16:34,413 - INFO - joeynmt.training - 	Reference:  what do you do when you have this sort of disrupted flow ?
2021-01-21 10:16:34,413 - INFO - joeynmt.training - 	Hypothesis: and i &apos;m going to be the lot of the way , &quot; what &apos;s you do ?
2021-01-21 10:16:34,413 - INFO - joeynmt.training - Validation result (greedy) at epoch   1, step     2000: bleu:   3.68, loss: 850449.3750, ppl: 201.7941, duration: 621.0200s
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_yticklabels(row_labels, minor=False)
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_yticklabels(row_labels, minor=False)
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_yticklabels(row_labels, minor=False)
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_yticklabels(row_labels, minor=False)
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_yticklabels(row_labels, minor=False)
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:57: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(column_labels, minor=False, rotation="vertical")
/home/students/kotzaneck/irl-project/joeynmt/plotting.py:58: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_yticklabels(row_labels, minor=False)
/home/students/kotzaneck/.local/lib/python3.8/site-packages/torchtext/data/batch.py:23: UserWarning: Batch class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.
  warnings.warn('{} class will be retired in the 0.8.0 release and moved to torchtext.legacy. Please see 0.7.0 release notes for further information.'.format(self.__class__.__name__), UserWarning)
2021-01-21 10:27:11,738 - INFO - joeynmt.training - Epoch   1, Step:     2500, Batch Loss:     4.557628, Tokens per Sec:      660, Lr: 0.000200
2021-01-21 10:37:37,498 - INFO - joeynmt.training - Epoch   1, Step:     3000, Batch Loss:     4.029715, Tokens per Sec:      672, Lr: 0.000200
2021-01-21 10:47:54,337 - INFO - joeynmt.training - Epoch   1, Step:     3500, Batch Loss:     4.919943, Tokens per Sec:      682, Lr: 0.000200
2021-01-21 10:56:31,294 - INFO - joeynmt.training - Epoch   1: total training loss 20363.30
2021-01-21 10:56:31,295 - INFO - joeynmt.training - EPOCH 2
+1 −0
Original line number Diff line number Diff line
test
+1 −1
Original line number Diff line number Diff line
@@ -22,7 +22,7 @@ training:
    optimizer: "adam"
    normalization: "tokens"
    learning_rate: 0.0002
    batch_size: 80
    batch_size: 40
    scheduling: "plateau"
    patience: 8
    #clip_grad_norm: 1.0
+694 MiB

File added.

No diff preview for this file type.

+7245 −0

File added.

File size exceeds preview limit.

Loading