WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************

===================================BUG REPORT===================================

===================================BUG REPORT===================================Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes


 Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues 
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues================================================================================

================================================================================
bin /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
bin /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.soCUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so

CUDA SETUP: Highest compute capability among GPUs detected: 8.6CUDA SETUP: Highest compute capability among GPUs detected: 8.6

CUDA SETUP: Detected CUDA version 121CUDA SETUP: Detected CUDA version 121

CUDA SETUP: Loading binary /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...CUDA SETUP: Loading binary /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...

No neptune configured, set NEPTUNE_API_TOKEN env var.

    Example runs on 4 GPUs:
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='decapoda-research/llama-7b-hf' --data_path=data/config.json --run_id=0 &> 0.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='decapoda-research/llama-30b-hf' --data_path=data/config.json --batch_size=16 --micro_batch_size=1 --run_id=1 --save_code=True &> 1.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='EleutherAI/gpt-j-6B' --data_path=data/config.json --run_id=2 &> 2.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='EleutherAI/gpt-neox-20b' --data_path=data/config.json --run_id=8 --batch_size=16 --micro_batch_size=4 &> 8.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='togethercomputer/GPT-NeoXT-Chat-Base-20B' --data_path=data/config.json --prompt_type='dai_faq' --run_id=13 --batch_size=16 --micro_batch_size=4 --num_epochs=100 --val_set_size=0 data_mix_in_path='' &> 13.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='togethercomputer/GPT-NeoXT-Chat-Base-20B' --data_path=data/config.json --run_id=28 --batch_size=16 --micro_batch_size=4 --num_epochs=8 --val_set_size=0 --data_mix_in_factor=0.1 --data_mix_in_prompt_type='human_bot' --save_code=True --cutoff_len=512  &> 28.log

    All metrics:
    CUDA_VISIBLE_DEVICES= finetune.py --data_mix_in_factor=0 --eval_steps=100 --warmup_steps=2 --val_set_size=100 --val_metrics="['bleu', 'rouge', 'sacrebleu', 'meteor']"

    # Fine-tune 20B on 24GB GPUs across 3 nodes with 3+2+2 GPUs
    rippa>
NCCL_P2P_LEVEL=LOC WORLD_SIZE=7 CUDA_VISIBLE_DEVICES="0,1,2" torchrun --node_rank 0 --nproc_per_node=3 --master_port=1234 --nnodes=3 --master_addr=10.10.10.2 finetune.py --data_path=merged_shuffled_OIG_87f6a1e788.json --micro_batch_size=1 --batch_size=7 --cutoff_len=512 --run_id=17 &>log.17.rank0
    ova>
NCCL_P2P_LEVEL=LOC WORLD_SIZE=7 CUDA_VISIBLE_DEVICES="0,1" torchrun --node_rank 1 --nproc_per_node=2 --master_port=1234 --nnodes=3 --master_addr=10.10.10.2 finetune.py --data_path=merged_shuffled_OIG_87f6a1e788.json --micro_batch_size=1 --batch_size=7 --cutoff_len=512 --run_id=17 &>log.17.rank1
    timemachine>
NCCL_P2P_LEVEL=LOC WORLD_SIZE=7 CUDA_VISIBLE_DEVICES="0,1" torchrun --node_rank 2 --nproc_per_node=2 --master_port=1234 --nnodes=3 --master_addr=10.10.10.2 finetune.py --data_path=merged_shuffled_OIG_87f6a1e788.json --micro_batch_size=1 --batch_size=7 --cutoff_len=512 --run_id=17 &>log.17.rank2

    
local_rank: 0
global rank: 2
local_rank: 1
global rank: 3
Training model with params:
save_code: False
run_id: 12
base_model: EleutherAI/pythia-6.9b
tokenizer_base_model: EleutherAI/pythia-6.9b
data_path: openassistant_oasst1.json
data_col_dict: None
valid_path: None
data_mix_in_path: 0-hero/OIG-small-chip2
data_mix_in_factor: 0.0
data_mix_in_col_dict: {'user': 'instruction', 'chip2': 'output'}
data_mix_in_prompt_type: instruct
output_dir: pythia-6.9b.openassistant_oasst1.json.1_epochs.ccc4630fee98c5b838b4357143f7466ec02674c3.12
lora_weights: 
batch_size: 128
micro_batch_size: 4
gradient_checkpointing: False
fp16: True
num_epochs: 1
learning_rate: 0.0003
val_set_size: None
val_metrics: []
eval_steps: None
eval_epochs: None
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules: None
llama_type: False
group_by_length: False
resume_from_checkpoint: None
ddp: True
local_files_only: False
resume_download: True
warmup_steps: 100
logging_steps: 1
save_steps: None
add_eos_token: False
world_size: 4
local_rank: 0
rank: 2
gpus: 4
device_map: auto
gradient_accumulation_steps: 32
cutoff_len: 256
prompt_type: plain
train_on_inputs: True
Command: finetune.py --base_model=EleutherAI/pythia-6.9b --data_path=openassistant_oasst1.json --prompt_type=plain --run_id=12 --cutoff_len=256
Hash: ccc4630fee98c5b838b4357143f7466ec02674c3
Distributed: data parallel

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:09<00:09,  9.43s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:09<00:09,  9.49s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:12<00:00,  5.87s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:12<00:00,  6.41s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:12<00:00,  5.87s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:12<00:00,  6.40s/it]
PeftModelForCausalLM(
  (base_model): LoraModel(
    (model): GPTNeoXForCausalLM(
      (gpt_neox): GPTNeoXModel(
        (embed_in): Embedding(50432, 4096)
        (layers): ModuleList(
          (0-31): 32 x GPTNeoXLayer(
            (input_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (post_attention_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (attention): GPTNeoXAttention(
              (rotary_emb): RotaryEmbedding()
              (query_key_value): MergedLinear8bitLt(
                in_features=4096, out_features=12288, bias=True
                (lora_dropout): Dropout(p=0.05, inplace=False)
                (lora_A): Linear(in_features=4096, out_features=16, bias=False)
                (lora_B): Conv1d(16, 8192, kernel_size=(1,), stride=(1,), groups=2, bias=False)
              )
              (dense): Linear8bitLt(in_features=4096, out_features=4096, bias=True)
            )
            (mlp): GPTNeoXMLP(
              (dense_h_to_4h): Linear8bitLt(in_features=4096, out_features=16384, bias=True)
              (dense_4h_to_h): Linear8bitLt(in_features=16384, out_features=4096, bias=True)
              (act): GELUActivation()
            )
          )
        )
        (final_layer_norm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
      )
      (embed_out): Linear(in_features=4096, out_features=50432, bias=False)
    )
  )
)
trainable params: 4194304 || all params: 6861496320 || trainable%: 0.06112812430977155
PeftModelForCausalLM(
  (base_model): LoraModel(
    (model): GPTNeoXForCausalLM(
      (gpt_neox): GPTNeoXModel(
        (embed_in): Embedding(50432, 4096)
        (layers): ModuleList(
          (0-31): 32 x GPTNeoXLayer(
            (input_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (post_attention_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (attention): GPTNeoXAttention(
              (rotary_emb): RotaryEmbedding()
              (query_key_value): MergedLinear8bitLt(
                in_features=4096, out_features=12288, bias=True
                (lora_dropout): Dropout(p=0.05, inplace=False)
                (lora_A): Linear(in_features=4096, out_features=16, bias=False)
                (lora_B): Conv1d(16, 8192, kernel_size=(1,), stride=(1,), groups=2, bias=False)
              )
              (dense): Linear8bitLt(in_features=4096, out_features=4096, bias=True)
            )
            (mlp): GPTNeoXMLP(
              (dense_h_to_4h): Linear8bitLt(in_features=4096, out_features=16384, bias=True)
              (dense_4h_to_h): Linear8bitLt(in_features=16384, out_features=4096, bias=True)
              (act): GELUActivation()
            )
          )
        )
        (final_layer_norm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
      )
      (embed_out): Linear(in_features=4096, out_features=50432, bias=False)
    )
  )
)
trainable params: 4194304 || all params: 6861496320 || trainable%: 0.06112812430977155
Using Validation Metrics: []
Supported Metrics: ['bleu', 'rouge', 'sacrebleu', 'meteor']
Auto set val_set_size 1000
Found cached dataset json (/home/arno/.cache/huggingface/datasets/json/default-cdbbb11a140d3908/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 72.29it/s]
Found cached dataset json (/home/arno/.cache/huggingface/datasets/json/default-cdbbb11a140d3908/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 275.25it/s]

Map (num_proc=22):   0%|          | 0/46283 [00:00<?, ? examples/s]
Map (num_proc=22):   0%|          | 0/46283 [00:00<?, ? examples/s]
Map (num_proc=22):   0%|          | 21/46283 [00:00<09:39, 79.85 examples/s]
Map (num_proc=22):   0%|          | 32/46283 [00:00<05:42, 135.15 examples/s]
Map (num_proc=22):   0%|          | 175/46283 [00:00<01:19, 583.47 examples/s]
Map (num_proc=22):   0%|          | 129/46283 [00:00<01:47, 430.90 examples/s]
Map (num_proc=22):   1%|          | 474/46283 [00:00<00:33, 1357.27 examples/s]
Map (num_proc=22):   1%|          | 325/46283 [00:00<00:48, 946.56 examples/s]
Map (num_proc=22):   2%|▏         | 799/46283 [00:00<00:21, 2096.48 examples/s]
Map (num_proc=22):   2%|▏         | 1042/46283 [00:00<00:18, 2408.67 examples/s]
Map (num_proc=22):   3%|▎         | 1286/46283 [00:00<00:15, 2918.22 examples/s]
Map (num_proc=22):   3%|▎         | 1615/46283 [00:00<00:13, 3354.22 examples/s]
Map (num_proc=22):   4%|▍         | 1836/46283 [00:00<00:12, 3632.72 examples/s]
Map (num_proc=22):   5%|▌         | 2335/46283 [00:00<00:10, 4365.12 examples/s]
Map (num_proc=22):   7%|▋         | 3190/46283 [00:00<00:07, 5540.88 examples/s]
Map (num_proc=22):   5%|▌         | 2389/46283 [00:00<00:10, 4101.34 examples/s]
Map (num_proc=22):   8%|▊         | 3924/46283 [00:01<00:07, 6034.17 examples/s]
Map (num_proc=22):   7%|▋         | 3054/46283 [00:00<00:09, 4701.28 examples/s]
Map (num_proc=22):  10%|█         | 4728/46283 [00:01<00:06, 6585.00 examples/s]
Map (num_proc=22):   8%|▊         | 3773/46283 [00:01<00:08, 5188.60 examples/s]
Map (num_proc=22):  12%|█▏        | 5572/46283 [00:01<00:05, 7111.62 examples/s]
Map (num_proc=22):  10%|█         | 4818/46283 [00:01<00:06, 6673.29 examples/s]
Map (num_proc=22):  14%|█▍        | 6401/46283 [00:01<00:05, 7453.30 examples/s]
Map (num_proc=22):  12%|█▏        | 5592/46283 [00:01<00:05, 6979.63 examples/s]
Map (num_proc=22):  16%|█▌        | 7315/46283 [00:01<00:04, 7830.78 examples/s]
Map (num_proc=22):  14%|█▍        | 6439/46283 [00:01<00:05, 7387.10 examples/s]
Map (num_proc=22):  18%|█▊        | 8178/46283 [00:01<00:04, 7999.85 examples/s]
Map (num_proc=22):  16%|█▌        | 7347/46283 [00:01<00:05, 7782.88 examples/s]
Map (num_proc=22):  20%|█▉        | 9074/46283 [00:01<00:04, 8205.50 examples/s]
Map (num_proc=22):  18%|█▊        | 8274/46283 [00:01<00:04, 8137.56 examples/s]
Map (num_proc=22):  21%|██▏       | 9929/46283 [00:01<00:04, 8208.76 examples/s]
Map (num_proc=22):  20%|█▉        | 9211/46283 [00:01<00:04, 8390.60 examples/s]
Map (num_proc=22):  23%|██▎       | 10856/46283 [00:01<00:04, 8476.30 examples/s]
Map (num_proc=22):  22%|██▏       | 10110/46283 [00:01<00:04, 8510.61 examples/s]
Map (num_proc=22):  25%|██▌       | 11745/46283 [00:01<00:04, 8458.18 examples/s]
Map (num_proc=22):  24%|██▎       | 10983/46283 [00:01<00:04, 8541.17 examples/s]
Map (num_proc=22):  27%|██▋       | 12649/46283 [00:02<00:03, 8608.21 examples/s]
Map (num_proc=22):  26%|██▌       | 11860/46283 [00:02<00:04, 8485.58 examples/s]
Map (num_proc=22):  29%|██▉       | 13549/46283 [00:02<00:03, 8652.48 examples/s]
Map (num_proc=22):  27%|██▋       | 12717/46283 [00:02<00:03, 8430.45 examples/s]
Map (num_proc=22):  31%|███       | 14432/46283 [00:02<00:03, 8544.97 examples/s]
Map (num_proc=22):  29%|██▉       | 13565/46283 [00:02<00:03, 8350.06 examples/s]
Map (num_proc=22):  33%|███▎      | 15378/46283 [00:02<00:03, 8765.35 examples/s]
Map (num_proc=22):  31%|███▏      | 14476/46283 [00:02<00:03, 8539.38 examples/s]
Map (num_proc=22):  35%|███▌      | 16282/46283 [00:02<00:03, 8618.29 examples/s]
Map (num_proc=22):  33%|███▎      | 15399/46283 [00:02<00:03, 8703.59 examples/s]
Map (num_proc=22):  37%|███▋      | 17165/46283 [00:02<00:03, 8411.85 examples/s]
Map (num_proc=22):  35%|███▌      | 16297/46283 [00:02<00:03, 8690.96 examples/s]
Map (num_proc=22):  39%|███▉      | 18021/46283 [00:02<00:03, 8197.24 examples/s]
Map (num_proc=22):  37%|███▋      | 17177/46283 [00:02<00:03, 8418.59 examples/s]
Map (num_proc=22):  41%|████      | 18844/46283 [00:02<00:03, 8086.40 examples/s]
Map (num_proc=22):  39%|███▉      | 18032/46283 [00:02<00:03, 7835.97 examples/s]
Map (num_proc=22):  42%|████▏     | 19663/46283 [00:02<00:03, 8030.52 examples/s]
Map (num_proc=22):  41%|████      | 18840/46283 [00:02<00:03, 7816.74 examples/s]
Map (num_proc=22):  44%|████▍     | 20471/46283 [00:03<00:03, 7945.05 examples/s]
Map (num_proc=22):  42%|████▏     | 19632/46283 [00:02<00:03, 7725.68 examples/s]
Map (num_proc=22):  46%|████▌     | 21275/46283 [00:03<00:03, 7401.02 examples/s]
Map (num_proc=22):  44%|████▍     | 20415/46283 [00:03<00:03, 7642.86 examples/s]
Map (num_proc=22):  48%|████▊     | 22099/46283 [00:03<00:03, 7491.07 examples/s]
Map (num_proc=22):  46%|████▌     | 21189/46283 [00:03<00:03, 7607.52 examples/s]
Map (num_proc=22):  49%|████▉     | 22888/46283 [00:03<00:03, 7557.49 examples/s]
Map (num_proc=22):  47%|████▋     | 21965/46283 [00:03<00:03, 7533.10 examples/s]
Map (num_proc=22):  49%|████▉     | 22743/46283 [00:03<00:03, 7575.34 examples/s]
Map (num_proc=22):  51%|█████     | 23652/46283 [00:03<00:03, 7282.55 examples/s]
Map (num_proc=22):  51%|█████     | 23533/46283 [00:03<00:02, 7666.57 examples/s]
Map (num_proc=22):  53%|█████▎    | 24401/46283 [00:03<00:02, 7302.37 examples/s]
Map (num_proc=22):  53%|█████▎    | 24319/46283 [00:03<00:02, 7627.36 examples/s]
Map (num_proc=22):  54%|█████▍    | 25142/46283 [00:03<00:02, 7277.39 examples/s]
Map (num_proc=22):  56%|█████▌    | 25924/46283 [00:03<00:02, 7413.77 examples/s]
Map (num_proc=22):  54%|█████▍    | 25105/46283 [00:03<00:02, 7328.51 examples/s]
Map (num_proc=22):  58%|█████▊    | 26758/46283 [00:03<00:02, 7681.07 examples/s]
Map (num_proc=22):  56%|█████▌    | 25853/46283 [00:03<00:02, 7266.45 examples/s]
Map (num_proc=22):  59%|█████▉    | 27538/46283 [00:03<00:02, 7637.72 examples/s]
Map (num_proc=22):  58%|█████▊    | 26654/46283 [00:03<00:02, 7393.51 examples/s]
Map (num_proc=22):  61%|██████▏   | 28408/46283 [00:04<00:02, 7924.48 examples/s]
Map (num_proc=22):  59%|█████▉    | 27442/46283 [00:04<00:02, 7465.86 examples/s]
Map (num_proc=22):  63%|██████▎   | 29269/46283 [00:04<00:02, 8066.55 examples/s]
Map (num_proc=22):  61%|██████    | 28330/46283 [00:04<00:02, 7790.17 examples/s]
Map (num_proc=22):  65%|██████▌   | 30120/46283 [00:04<00:01, 8185.88 examples/s]
Map (num_proc=22):  63%|██████▎   | 29170/46283 [00:04<00:02, 7872.46 examples/s]
Map (num_proc=22):  67%|██████▋   | 31008/46283 [00:04<00:01, 8347.96 examples/s]
Map (num_proc=22):  65%|██████▍   | 30025/46283 [00:04<00:02, 8056.81 examples/s]
Map (num_proc=22):  69%|██████▉   | 31893/46283 [00:04<00:01, 8332.38 examples/s]
Map (num_proc=22):  67%|██████▋   | 30903/46283 [00:04<00:01, 8248.26 examples/s]
Map (num_proc=22):  71%|███████   | 32782/46283 [00:04<00:01, 8405.32 examples/s]
Map (num_proc=22):  69%|██████▊   | 31774/46283 [00:04<00:01, 8355.91 examples/s]
Map (num_proc=22):  73%|███████▎  | 33697/46283 [00:04<00:01, 8480.98 examples/s]
Map (num_proc=22):  71%|███████   | 32654/46283 [00:04<00:01, 8485.05 examples/s]
Map (num_proc=22):  75%|███████▍  | 34582/46283 [00:04<00:01, 8544.75 examples/s]
Map (num_proc=22):  72%|███████▏  | 33514/46283 [00:04<00:01, 8517.48 examples/s]
Map (num_proc=22):  77%|███████▋  | 35456/46283 [00:04<00:01, 8562.20 examples/s]
Map (num_proc=22):  74%|███████▍  | 34375/46283 [00:04<00:01, 8495.48 examples/s]
Map (num_proc=22):  79%|███████▊  | 36349/46283 [00:04<00:01, 8550.76 examples/s]
Map (num_proc=22):  76%|███████▌  | 35229/46283 [00:04<00:01, 8428.91 examples/s]
Map (num_proc=22):  80%|████████  | 37215/46283 [00:05<00:01, 8258.27 examples/s]
Map (num_proc=22):  78%|███████▊  | 36081/46283 [00:05<00:01, 7796.00 examples/s]
Map (num_proc=22):  82%|████████▏ | 38059/46283 [00:05<00:00, 8241.06 examples/s]
Map (num_proc=22):  80%|███████▉  | 36911/46283 [00:05<00:01, 7880.29 examples/s]
Map (num_proc=22):  84%|████████▍ | 38888/46283 [00:05<00:00, 8181.20 examples/s]
Map (num_proc=22):  82%|████████▏ | 37766/46283 [00:05<00:01, 8004.66 examples/s]
Map (num_proc=22):  86%|████████▌ | 39715/46283 [00:05<00:00, 7910.53 examples/s]
Map (num_proc=22):  83%|████████▎ | 38576/46283 [00:05<00:00, 7850.86 examples/s]
Map (num_proc=22):  85%|████████▌ | 39388/46283 [00:05<00:00, 7922.65 examples/s]
Map (num_proc=22):  88%|████████▊ | 40520/46283 [00:05<00:00, 7623.03 examples/s]
Map (num_proc=22):  89%|████████▉ | 41379/46283 [00:05<00:00, 7852.89 examples/s]
Map (num_proc=22):  87%|████████▋ | 40196/46283 [00:05<00:00, 7886.40 examples/s]
Map (num_proc=22):  89%|████████▊ | 41018/46283 [00:05<00:00, 7956.39 examples/s]
Map (num_proc=22):  91%|█████████ | 42170/46283 [00:05<00:00, 7768.58 examples/s]
Map (num_proc=22):  90%|█████████ | 41835/46283 [00:05<00:00, 7611.65 examples/s]
Map (num_proc=22):  93%|█████████▎| 42981/46283 [00:05<00:00, 7440.61 examples/s]
Map (num_proc=22):  92%|█████████▏| 42603/46283 [00:05<00:00, 7379.06 examples/s]
Map (num_proc=22):  95%|█████████▍| 43750/46283 [00:05<00:00, 7134.92 examples/s]
Map (num_proc=22):  94%|█████████▎| 43367/46283 [00:06<00:00, 7351.35 examples/s]
Map (num_proc=22):  96%|█████████▌| 44473/46283 [00:06<00:00, 6296.07 examples/s]
Map (num_proc=22):  95%|█████████▌| 44113/46283 [00:06<00:00, 7031.08 examples/s]
Map (num_proc=22):  98%|█████████▊| 45130/46283 [00:06<00:00, 5357.81 examples/s]
Map (num_proc=22):  97%|█████████▋| 44833/46283 [00:06<00:00, 6537.14 examples/s]
Map (num_proc=22):  99%|█████████▉| 45710/46283 [00:06<00:00, 5301.46 examples/s]
Map (num_proc=22):  98%|█████████▊| 45504/46283 [00:06<00:00, 5471.73 examples/s]
Map (num_proc=22): 100%|█████████▉| 46118/46283 [00:06<00:00, 4684.98 examples/s]
Map (num_proc=22): 100%|█████████▉| 46276/46283 [00:06<00:00, 3393.27 examples/s]
                                                                                 

                                                                                 
Final fine-tuning data:
Train Dataset({
    features: ['input', 'prompt_type', 'source', 'input_ids', 'attention_mask', 'labels'],
    num_rows: 46283
})
Valid None
Sample input: {'input': ['<human>: Hazme un análisis de cuántas personas hablan español en Europa y su posible influencia en un futuro a medio plazo.\n<bot>: El español es uno de los idiomas más hablados del mundo, con alrededor de 580 millones de hablantes en todo el mundo. En Europa, el español es el segundo idioma más hablado después del inglés, y es hablado como lengua materna por alrededor de 40 millones de personas en países como España, Portugal y Andorra.\n\nAdemás, el español es un idioma muy estudiado en Europa, con una gran cantidad de personas que lo aprenden como segunda lengua. Se estima que alrededor de 15 millones de personas estudian español como lengua extranjera en Europa.\n\nEn cuanto a la influencia del español en un futuro a medio plazo, se espera que siga siendo un idioma importante debido al crecimiento demográfico de los países hispanohablantes en América Latina y a la importancia creciente de España en la Unión Europea y en el mundo en general. Además, la influencia cultural y lingüística de los países hispanohablantes en Europa también puede aumentar en los próximos años.\n\nEn resumen, el español es un idioma importante en Europa y su influencia puede seguir creciendo en el futuro debido a su número de hablantes, su popularidad como lengua extranjera y la importancia de los países hispanohablantes en América Latina y en Europa.\n'], 'prompt_type': ['plain'], 'source': ['OpenAssistant/oasst1']}
Auto set eval_steps to 18 out of 361 total training steps
Auto step save_steps to 18
You are adding a <class 'transformers.integrations.TensorBoardCallback'> to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is
:DefaultFlowCallback
TensorBoardCallback
You are adding a <class 'transformers.integrations.TensorBoardCallback'> to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is
:DefaultFlowCallback
TensorBoardCallback

  0%|          | 0/361 [00:00<?, ?it/s]You're using a GPTNeoXTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
You're using a GPTNeoXTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:219: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at ../aten/src/ATen/native/TensorCompare.cpp:493.)
  attn_scores = torch.where(causal_mask, attn_scores, mask_value)
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:219: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at ../aten/src/ATen/native/TensorCompare.cpp:493.)
  attn_scores = torch.where(causal_mask, attn_scores, mask_value)

  0%|          | 1/361 [00:17<1:43:45, 17.29s/it]
                                                 
{'loss': 2.0578, 'learning_rate': 2.9999999999999997e-06, 'epoch': 0.0}

  0%|          | 1/361 [00:17<1:43:45, 17.29s/it]
  1%|          | 2/361 [00:31<1:31:01, 15.21s/it]
                                                 
{'loss': 2.1274, 'learning_rate': 5.999999999999999e-06, 'epoch': 0.01}

  1%|          | 2/361 [00:31<1:31:01, 15.21s/it]
  1%|          | 3/361 [00:44<1:26:59, 14.58s/it]
                                                 
{'loss': 2.115, 'learning_rate': 8.999999999999999e-06, 'epoch': 0.01}

  1%|          | 3/361 [00:44<1:26:59, 14.58s/it]
  1%|          | 4/361 [00:58<1:25:02, 14.29s/it]
                                                 
{'loss': 2.0605, 'learning_rate': 1.1999999999999999e-05, 'epoch': 0.01}

  1%|          | 4/361 [00:58<1:25:02, 14.29s/it]
  1%|▏         | 5/361 [01:12<1:23:57, 14.15s/it]
                                                 
{'loss': 1.9713, 'learning_rate': 1.4999999999999999e-05, 'epoch': 0.01}

  1%|▏         | 5/361 [01:12<1:23:57, 14.15s/it]
  2%|▏         | 6/361 [01:26<1:23:07, 14.05s/it]
                                                 
{'loss': 2.0504, 'learning_rate': 1.7999999999999997e-05, 'epoch': 0.02}

  2%|▏         | 6/361 [01:26<1:23:07, 14.05s/it]
  2%|▏         | 7/361 [01:40<1:22:29, 13.98s/it]
                                                 
{'loss': 2.0258, 'learning_rate': 2.1e-05, 'epoch': 0.02}

  2%|▏         | 7/361 [01:40<1:22:29, 13.98s/it]
  2%|▏         | 8/361 [01:54<1:22:00, 13.94s/it]
                                                 
{'loss': 2.0462, 'learning_rate': 2.3999999999999997e-05, 'epoch': 0.02}

  2%|▏         | 8/361 [01:54<1:22:00, 13.94s/it]
  2%|▏         | 9/361 [02:08<1:21:44, 13.93s/it]
                                                 
{'loss': 2.0193, 'learning_rate': 2.6999999999999996e-05, 'epoch': 0.02}

  2%|▏         | 9/361 [02:08<1:21:44, 13.93s/it]
  3%|▎         | 10/361 [02:22<1:21:37, 13.95s/it]
                                                  
{'loss': 1.9965, 'learning_rate': 2.9999999999999997e-05, 'epoch': 0.03}

  3%|▎         | 10/361 [02:22<1:21:37, 13.95s/it]
  3%|▎         | 11/361 [02:35<1:20:55, 13.87s/it]
                                                  
{'loss': 2.1366, 'learning_rate': 3.2999999999999996e-05, 'epoch': 0.03}

  3%|▎         | 11/361 [02:35<1:20:55, 13.87s/it]
  3%|▎         | 12/361 [02:49<1:20:52, 13.90s/it]
                                                  
{'loss': 2.0466, 'learning_rate': 3.5999999999999994e-05, 'epoch': 0.03}

  3%|▎         | 12/361 [02:49<1:20:52, 13.90s/it]
  4%|▎         | 13/361 [03:03<1:20:57, 13.96s/it]
                                                  
{'loss': 2.0387, 'learning_rate': 3.9e-05, 'epoch': 0.04}

  4%|▎         | 13/361 [03:03<1:20:57, 13.96s/it]
  4%|▍         | 14/361 [03:17<1:20:14, 13.87s/it]
                                                  
{'loss': 2.0183, 'learning_rate': 4.2e-05, 'epoch': 0.04}

  4%|▍         | 14/361 [03:17<1:20:14, 13.87s/it]
  4%|▍         | 15/361 [03:31<1:20:10, 13.90s/it]
                                                  
{'loss': 1.9501, 'learning_rate': 4.4999999999999996e-05, 'epoch': 0.04}

  4%|▍         | 15/361 [03:31<1:20:10, 13.90s/it]
  4%|▍         | 16/361 [03:45<1:20:05, 13.93s/it]
                                                  
{'loss': 1.9512, 'learning_rate': 4.7999999999999994e-05, 'epoch': 0.04}

  4%|▍         | 16/361 [03:45<1:20:05, 13.93s/it]
  5%|▍         | 17/361 [03:59<1:20:10, 13.99s/it]
                                                  
{'loss': 1.8547, 'learning_rate': 5.1e-05, 'epoch': 0.05}

  5%|▍         | 17/361 [03:59<1:20:10, 13.99s/it]
  5%|▍         | 18/361 [04:13<1:20:23, 14.06s/it]
                                                  
{'loss': 1.9935, 'learning_rate': 5.399999999999999e-05, 'epoch': 0.05}

  5%|▍         | 18/361 [04:13<1:20:23, 14.06s/it]
  5%|▌         | 19/361 [04:28<1:21:05, 14.23s/it]
                                                  
{'loss': 1.9382, 'learning_rate': 5.6999999999999996e-05, 'epoch': 0.05}

  5%|▌         | 19/361 [04:28<1:21:05, 14.23s/it]
  6%|▌         | 20/361 [04:42<1:20:45, 14.21s/it]
                                                  
{'loss': 1.9917, 'learning_rate': 5.9999999999999995e-05, 'epoch': 0.06}

  6%|▌         | 20/361 [04:42<1:20:45, 14.21s/it]
  6%|▌         | 21/361 [04:56<1:20:37, 14.23s/it]
                                                  
{'loss': 1.9226, 'learning_rate': 6.299999999999999e-05, 'epoch': 0.06}

  6%|▌         | 21/361 [04:56<1:20:37, 14.23s/it]
  6%|▌         | 22/361 [05:11<1:20:30, 14.25s/it]
                                                  
{'loss': 1.8353, 'learning_rate': 6.599999999999999e-05, 'epoch': 0.06}

  6%|▌         | 22/361 [05:11<1:20:30, 14.25s/it]
  6%|▋         | 23/361 [05:25<1:20:19, 14.26s/it]
                                                  
{'loss': 1.9255, 'learning_rate': 6.9e-05, 'epoch': 0.06}

  6%|▋         | 23/361 [05:25<1:20:19, 14.26s/it]
  7%|▋         | 24/361 [05:39<1:20:09, 14.27s/it]
                                                  
{'loss': 1.7802, 'learning_rate': 7.199999999999999e-05, 'epoch': 0.07}

  7%|▋         | 24/361 [05:39<1:20:09, 14.27s/it]
  7%|▋         | 25/361 [05:54<1:19:57, 14.28s/it]
                                                  
{'loss': 1.8964, 'learning_rate': 7.5e-05, 'epoch': 0.07}

  7%|▋         | 25/361 [05:54<1:19:57, 14.28s/it]
  7%|▋         | 26/361 [06:08<1:19:54, 14.31s/it]
                                                  
{'loss': 1.8405, 'learning_rate': 7.8e-05, 'epoch': 0.07}

  7%|▋         | 26/361 [06:08<1:19:54, 14.31s/it]
  7%|▋         | 27/361 [06:22<1:19:42, 14.32s/it]
                                                  
{'loss': 1.7834, 'learning_rate': 8.1e-05, 'epoch': 0.07}

  7%|▋         | 27/361 [06:22<1:19:42, 14.32s/it]
  8%|▊         | 28/361 [06:37<1:19:29, 14.32s/it]
                                                  
{'loss': 1.8501, 'learning_rate': 8.4e-05, 'epoch': 0.08}

  8%|▊         | 28/361 [06:37<1:19:29, 14.32s/it]
  8%|▊         | 29/361 [06:50<1:18:22, 14.16s/it]
                                                  
{'loss': 1.7776, 'learning_rate': 8.699999999999999e-05, 'epoch': 0.08}

  8%|▊         | 29/361 [06:50<1:18:22, 14.16s/it]
  8%|▊         | 30/361 [07:05<1:18:08, 14.16s/it]
                                                  
{'loss': 1.8992, 'learning_rate': 8.999999999999999e-05, 'epoch': 0.08}

  8%|▊         | 30/361 [07:05<1:18:08, 14.16s/it]
  9%|▊         | 31/361 [07:19<1:18:13, 14.22s/it]
                                                  
{'loss': 1.8197, 'learning_rate': 9.3e-05, 'epoch': 0.09}

  9%|▊         | 31/361 [07:19<1:18:13, 14.22s/it]
  9%|▉         | 32/361 [07:34<1:19:39, 14.53s/it]
                                                  
{'loss': 1.7598, 'learning_rate': 9.599999999999999e-05, 'epoch': 0.09}

  9%|▉         | 32/361 [07:34<1:19:39, 14.53s/it]
  9%|▉         | 33/361 [07:49<1:19:07, 14.48s/it]
                                                  
{'loss': 1.765, 'learning_rate': 9.9e-05, 'epoch': 0.09}

  9%|▉         | 33/361 [07:49<1:19:07, 14.48s/it]
  9%|▉         | 34/361 [08:03<1:18:21, 14.38s/it]
                                                  
{'loss': 1.7491, 'learning_rate': 0.000102, 'epoch': 0.09}

  9%|▉         | 34/361 [08:03<1:18:21, 14.38s/it]
 10%|▉         | 35/361 [08:17<1:18:13, 14.40s/it]
                                                  
{'loss': 1.8351, 'learning_rate': 0.00010499999999999999, 'epoch': 0.1}

 10%|▉         | 35/361 [08:17<1:18:13, 14.40s/it]
 10%|▉         | 36/361 [08:31<1:17:24, 14.29s/it]
                                                  
{'loss': 1.7606, 'learning_rate': 0.00010799999999999998, 'epoch': 0.1}

 10%|▉         | 36/361 [08:31<1:17:24, 14.29s/it]
 10%|█         | 37/361 [08:46<1:17:49, 14.41s/it]
                                                  
{'loss': 1.8027, 'learning_rate': 0.00011099999999999999, 'epoch': 0.1}

 10%|█         | 37/361 [08:46<1:17:49, 14.41s/it]
 11%|█         | 38/361 [09:00<1:17:31, 14.40s/it]
                                                  
{'loss': 1.7885, 'learning_rate': 0.00011399999999999999, 'epoch': 0.11}

 11%|█         | 38/361 [09:00<1:17:31, 14.40s/it]
 11%|█         | 39/361 [09:15<1:17:14, 14.39s/it]
                                                  
{'loss': 1.7619, 'learning_rate': 0.000117, 'epoch': 0.11}

 11%|█         | 39/361 [09:15<1:17:14, 14.39s/it]
 11%|█         | 40/361 [09:29<1:16:55, 14.38s/it]
                                                  
{'loss': 1.7241, 'learning_rate': 0.00011999999999999999, 'epoch': 0.11}

 11%|█         | 40/361 [09:29<1:16:55, 14.38s/it]
 11%|█▏        | 41/361 [09:43<1:16:41, 14.38s/it]
                                                  
{'loss': 1.7212, 'learning_rate': 0.00012299999999999998, 'epoch': 0.11}

 11%|█▏        | 41/361 [09:43<1:16:41, 14.38s/it]
 12%|█▏        | 42/361 [09:58<1:16:26, 14.38s/it]
                                                  
{'loss': 1.6807, 'learning_rate': 0.00012599999999999997, 'epoch': 0.12}

 12%|█▏        | 42/361 [09:58<1:16:26, 14.38s/it]
 12%|█▏        | 43/361 [10:12<1:16:12, 14.38s/it]
                                                  
{'loss': 1.7306, 'learning_rate': 0.000129, 'epoch': 0.12}

 12%|█▏        | 43/361 [10:12<1:16:12, 14.38s/it]
 12%|█▏        | 44/361 [10:26<1:15:54, 14.37s/it]
                                                  
{'loss': 1.6999, 'learning_rate': 0.00013199999999999998, 'epoch': 0.12}

 12%|█▏        | 44/361 [10:26<1:15:54, 14.37s/it]
 12%|█▏        | 45/361 [10:41<1:15:24, 14.32s/it]
                                                  
{'loss': 1.7661, 'learning_rate': 0.000135, 'epoch': 0.12}

 12%|█▏        | 45/361 [10:41<1:15:24, 14.32s/it]
 13%|█▎        | 46/361 [10:55<1:15:14, 14.33s/it]
                                                  
{'loss': 1.6854, 'learning_rate': 0.000138, 'epoch': 0.13}

 13%|█▎        | 46/361 [10:55<1:15:14, 14.33s/it]
 13%|█▎        | 47/361 [11:09<1:15:02, 14.34s/it]
                                                  
{'loss': 1.764, 'learning_rate': 0.00014099999999999998, 'epoch': 0.13}

 13%|█▎        | 47/361 [11:09<1:15:02, 14.34s/it]
 13%|█▎        | 48/361 [11:24<1:14:51, 14.35s/it]
                                                  
{'loss': 1.7706, 'learning_rate': 0.00014399999999999998, 'epoch': 0.13}

 13%|█▎        | 48/361 [11:24<1:14:51, 14.35s/it]
 14%|█▎        | 49/361 [11:38<1:14:39, 14.36s/it]
                                                  
{'loss': 1.6817, 'learning_rate': 0.000147, 'epoch': 0.14}

 14%|█▎        | 49/361 [11:38<1:14:39, 14.36s/it]
 14%|█▍        | 50/361 [11:53<1:14:29, 14.37s/it]
                                                  
{'loss': 1.6641, 'learning_rate': 0.00015, 'epoch': 0.14}

 14%|█▍        | 50/361 [11:53<1:14:29, 14.37s/it]
 14%|█▍        | 51/361 [12:07<1:14:14, 14.37s/it]
                                                  
{'loss': 1.7635, 'learning_rate': 0.00015299999999999998, 'epoch': 0.14}

 14%|█▍        | 51/361 [12:07<1:14:14, 14.37s/it]
 14%|█▍        | 52/361 [12:21<1:14:00, 14.37s/it]
                                                  
{'loss': 1.8139, 'learning_rate': 0.000156, 'epoch': 0.14}

 14%|█▍        | 52/361 [12:21<1:14:00, 14.37s/it]
 15%|█▍        | 53/361 [12:36<1:13:39, 14.35s/it]
                                                  
{'loss': 1.8296, 'learning_rate': 0.000159, 'epoch': 0.15}

 15%|█▍        | 53/361 [12:36<1:13:39, 14.35s/it]
 15%|█▍        | 54/361 [12:50<1:13:23, 14.34s/it]
                                                  
{'loss': 1.7027, 'learning_rate': 0.000162, 'epoch': 0.15}

 15%|█▍        | 54/361 [12:50<1:13:23, 14.34s/it]
 15%|█▌        | 55/361 [13:05<1:13:42, 14.45s/it]
                                                  
{'loss': 1.7054, 'learning_rate': 0.000165, 'epoch': 0.15}

 15%|█▌        | 55/361 [13:05<1:13:42, 14.45s/it]
 16%|█▌        | 56/361 [13:19<1:13:17, 14.42s/it]
                                                  
{'loss': 1.7047, 'learning_rate': 0.000168, 'epoch': 0.15}

 16%|█▌        | 56/361 [13:19<1:13:17, 14.42s/it]
 16%|█▌        | 57/361 [13:33<1:12:58, 14.40s/it]
                                                  
{'loss': 1.6573, 'learning_rate': 0.00017099999999999998, 'epoch': 0.16}

 16%|█▌        | 57/361 [13:33<1:12:58, 14.40s/it]
 16%|█▌        | 58/361 [13:48<1:12:32, 14.37s/it]
                                                  
{'loss': 1.6876, 'learning_rate': 0.00017399999999999997, 'epoch': 0.16}

 16%|█▌        | 58/361 [13:48<1:12:32, 14.37s/it]
 16%|█▋        | 59/361 [14:02<1:12:10, 14.34s/it]
                                                  
{'loss': 1.6459, 'learning_rate': 0.00017699999999999997, 'epoch': 0.16}

 16%|█▋        | 59/361 [14:02<1:12:10, 14.34s/it]
 17%|█▋        | 60/361 [14:16<1:11:55, 14.34s/it]
                                                  
{'loss': 1.5779, 'learning_rate': 0.00017999999999999998, 'epoch': 0.17}

 17%|█▋        | 60/361 [14:16<1:11:55, 14.34s/it]
 17%|█▋        | 61/361 [14:30<1:11:38, 14.33s/it]
                                                  
{'loss': 1.7339, 'learning_rate': 0.00018299999999999998, 'epoch': 0.17}

 17%|█▋        | 61/361 [14:30<1:11:38, 14.33s/it]
 17%|█▋        | 62/361 [14:45<1:11:26, 14.34s/it]
                                                  
{'loss': 1.677, 'learning_rate': 0.000186, 'epoch': 0.17}

 17%|█▋        | 62/361 [14:45<1:11:26, 14.34s/it]
 17%|█▋        | 63/361 [14:59<1:10:29, 14.19s/it]
                                                  
{'loss': 1.7351, 'learning_rate': 0.00018899999999999999, 'epoch': 0.17}

 17%|█▋        | 63/361 [14:59<1:10:29, 14.19s/it]
 18%|█▊        | 64/361 [15:13<1:10:25, 14.23s/it]
                                                  
{'loss': 1.8191, 'learning_rate': 0.00019199999999999998, 'epoch': 0.18}

 18%|█▊        | 64/361 [15:13<1:10:25, 14.23s/it]
 18%|█▊        | 65/361 [15:27<1:10:20, 14.26s/it]
                                                  
{'loss': 1.7331, 'learning_rate': 0.000195, 'epoch': 0.18}

 18%|█▊        | 65/361 [15:27<1:10:20, 14.26s/it]
 18%|█▊        | 66/361 [15:41<1:09:42, 14.18s/it]
                                                  
{'loss': 1.677, 'learning_rate': 0.000198, 'epoch': 0.18}

 18%|█▊        | 66/361 [15:41<1:09:42, 14.18s/it]
 19%|█▊        | 67/361 [15:55<1:09:07, 14.11s/it]
                                                  
{'loss': 1.6495, 'learning_rate': 0.000201, 'epoch': 0.19}

 19%|█▊        | 67/361 [15:55<1:09:07, 14.11s/it]
 19%|█▉        | 68/361 [16:09<1:08:56, 14.12s/it]
                                                  
{'loss': 1.6968, 'learning_rate': 0.000204, 'epoch': 0.19}

 19%|█▉        | 68/361 [16:09<1:08:56, 14.12s/it]
 19%|█▉        | 69/361 [16:24<1:09:04, 14.19s/it]
                                                  
{'loss': 1.6121, 'learning_rate': 0.00020699999999999996, 'epoch': 0.19}

 19%|█▉        | 69/361 [16:24<1:09:04, 14.19s/it]
 19%|█▉        | 70/361 [16:38<1:08:59, 14.22s/it]
                                                  
{'loss': 1.6959, 'learning_rate': 0.00020999999999999998, 'epoch': 0.19}

 19%|█▉        | 70/361 [16:38<1:08:59, 14.22s/it]
 20%|█▉        | 71/361 [16:52<1:08:52, 14.25s/it]
                                                  
{'loss': 1.6846, 'learning_rate': 0.00021299999999999997, 'epoch': 0.2}

 20%|█▉        | 71/361 [16:52<1:08:52, 14.25s/it]
 20%|█▉        | 72/361 [17:07<1:08:42, 14.27s/it]
                                                  
{'loss': 1.704, 'learning_rate': 0.00021599999999999996, 'epoch': 0.2}

 20%|█▉        | 72/361 [17:07<1:08:42, 14.27s/it]
 20%|██        | 73/361 [17:21<1:09:07, 14.40s/it]
                                                  
{'loss': 1.7242, 'learning_rate': 0.00021899999999999998, 'epoch': 0.2}

 20%|██        | 73/361 [17:21<1:09:07, 14.40s/it]
 20%|██        | 74/361 [17:36<1:08:48, 14.39s/it]
                                                  
{'loss': 1.6509, 'learning_rate': 0.00022199999999999998, 'epoch': 0.2}

 20%|██        | 74/361 [17:36<1:08:48, 14.39s/it]
 21%|██        | 75/361 [17:50<1:08:27, 14.36s/it]
                                                  
{'loss': 1.6407, 'learning_rate': 0.000225, 'epoch': 0.21}

 21%|██        | 75/361 [17:50<1:08:27, 14.36s/it]
 21%|██        | 76/361 [18:04<1:08:15, 14.37s/it]
                                                  
{'loss': 1.6558, 'learning_rate': 0.00022799999999999999, 'epoch': 0.21}

 21%|██        | 76/361 [18:04<1:08:15, 14.37s/it]
 21%|██▏       | 77/361 [18:18<1:07:14, 14.21s/it]
                                                  
{'loss': 1.8435, 'learning_rate': 0.00023099999999999998, 'epoch': 0.21}

 21%|██▏       | 77/361 [18:18<1:07:14, 14.21s/it]
 22%|██▏       | 78/361 [18:33<1:07:17, 14.27s/it]
                                                  
{'loss': 1.6606, 'learning_rate': 0.000234, 'epoch': 0.22}

 22%|██▏       | 78/361 [18:33<1:07:17, 14.27s/it]
 22%|██▏       | 79/361 [18:47<1:07:09, 14.29s/it]
                                                  
{'loss': 1.6938, 'learning_rate': 0.000237, 'epoch': 0.22}

 22%|██▏       | 79/361 [18:47<1:07:09, 14.29s/it]
 22%|██▏       | 80/361 [19:01<1:06:58, 14.30s/it]
                                                  
{'loss': 1.7046, 'learning_rate': 0.00023999999999999998, 'epoch': 0.22}

 22%|██▏       | 80/361 [19:01<1:06:58, 14.30s/it]
 22%|██▏       | 81/361 [19:16<1:06:47, 14.31s/it]
                                                  
{'loss': 1.6755, 'learning_rate': 0.000243, 'epoch': 0.22}

 22%|██▏       | 81/361 [19:16<1:06:47, 14.31s/it]
 23%|██▎       | 82/361 [19:30<1:06:38, 14.33s/it]
                                                  
{'loss': 1.7015, 'learning_rate': 0.00024599999999999996, 'epoch': 0.23}

 23%|██▎       | 82/361 [19:30<1:06:38, 14.33s/it]
 23%|██▎       | 83/361 [19:44<1:06:31, 14.36s/it]
                                                  
{'loss': 1.6543, 'learning_rate': 0.000249, 'epoch': 0.23}

 23%|██▎       | 83/361 [19:44<1:06:31, 14.36s/it]
 23%|██▎       | 84/361 [19:59<1:06:20, 14.37s/it]
                                                  
{'loss': 1.6761, 'learning_rate': 0.00025199999999999995, 'epoch': 0.23}

 23%|██▎       | 84/361 [19:59<1:06:20, 14.37s/it]
 24%|██▎       | 85/361 [20:13<1:06:10, 14.39s/it]
                                                  
{'loss': 1.6832, 'learning_rate': 0.00025499999999999996, 'epoch': 0.24}

 24%|██▎       | 85/361 [20:13<1:06:10, 14.39s/it]
 24%|██▍       | 86/361 [20:28<1:05:57, 14.39s/it]
                                                  
{'loss': 1.7009, 'learning_rate': 0.000258, 'epoch': 0.24}

 24%|██▍       | 86/361 [20:28<1:05:57, 14.39s/it]
 24%|██▍       | 87/361 [20:42<1:05:47, 14.41s/it]
                                                  
{'loss': 1.6612, 'learning_rate': 0.000261, 'epoch': 0.24}

 24%|██▍       | 87/361 [20:42<1:05:47, 14.41s/it]
 24%|██▍       | 88/361 [20:57<1:05:33, 14.41s/it]
                                                  
{'loss': 1.6715, 'learning_rate': 0.00026399999999999997, 'epoch': 0.24}

 24%|██▍       | 88/361 [20:57<1:05:33, 14.41s/it]
 25%|██▍       | 89/361 [21:11<1:05:19, 14.41s/it]
                                                  
{'loss': 1.6567, 'learning_rate': 0.000267, 'epoch': 0.25}

 25%|██▍       | 89/361 [21:11<1:05:19, 14.41s/it]
 25%|██▍       | 90/361 [21:25<1:04:52, 14.36s/it]
                                                  
{'loss': 1.6755, 'learning_rate': 0.00027, 'epoch': 0.25}

 25%|██▍       | 90/361 [21:25<1:04:52, 14.36s/it]
 25%|██▌       | 91/361 [21:40<1:05:05, 14.47s/it]
                                                  
{'loss': 1.6006, 'learning_rate': 0.00027299999999999997, 'epoch': 0.25}

 25%|██▌       | 91/361 [21:40<1:05:05, 14.47s/it]
 25%|██▌       | 92/361 [21:54<1:04:18, 14.34s/it]
                                                  
{'loss': 1.6207, 'learning_rate': 0.000276, 'epoch': 0.25}

 25%|██▌       | 92/361 [21:54<1:04:18, 14.34s/it]
 26%|██▌       | 93/361 [22:08<1:04:04, 14.34s/it]
                                                  
{'loss': 1.7304, 'learning_rate': 0.000279, 'epoch': 0.26}

 26%|██▌       | 93/361 [22:08<1:04:04, 14.34s/it]
 26%|██▌       | 94/361 [22:23<1:03:53, 14.36s/it]
                                                  
{'loss': 1.6711, 'learning_rate': 0.00028199999999999997, 'epoch': 0.26}

 26%|██▌       | 94/361 [22:23<1:03:53, 14.36s/it]
 26%|██▋       | 95/361 [22:37<1:03:41, 14.37s/it]
                                                  
{'loss': 1.6366, 'learning_rate': 0.000285, 'epoch': 0.26}

 26%|██▋       | 95/361 [22:37<1:03:41, 14.37s/it]
 27%|██▋       | 96/361 [22:52<1:03:30, 14.38s/it]
                                                  
{'loss': 1.6566, 'learning_rate': 0.00028799999999999995, 'epoch': 0.27}

 27%|██▋       | 96/361 [22:52<1:03:30, 14.38s/it]
 27%|██▋       | 97/361 [23:06<1:03:16, 14.38s/it]
                                                  
{'loss': 1.6478, 'learning_rate': 0.00029099999999999997, 'epoch': 0.27}

 27%|██▋       | 97/361 [23:06<1:03:16, 14.38s/it]
 27%|██▋       | 98/361 [23:20<1:03:04, 14.39s/it]
                                                  
{'loss': 1.6882, 'learning_rate': 0.000294, 'epoch': 0.27}

 27%|██▋       | 98/361 [23:20<1:03:04, 14.39s/it]
 27%|██▋       | 99/361 [23:35<1:02:48, 14.39s/it]
                                                  
{'loss': 1.6857, 'learning_rate': 0.00029699999999999996, 'epoch': 0.27}

 27%|██▋       | 99/361 [23:35<1:02:48, 14.39s/it]
 28%|██▊       | 100/361 [23:49<1:02:20, 14.33s/it]
                                                   
{'loss': 1.6723, 'learning_rate': 0.0003, 'epoch': 0.28}

 28%|██▊       | 100/361 [23:49<1:02:20, 14.33s/it]
 28%|██▊       | 101/361 [24:03<1:02:04, 14.33s/it]
                                                   
{'loss': 1.6893, 'learning_rate': 0.00029885057471264366, 'epoch': 0.28}

 28%|██▊       | 101/361 [24:03<1:02:04, 14.33s/it]
 28%|██▊       | 102/361 [24:18<1:01:53, 14.34s/it]
                                                   
{'loss': 1.6546, 'learning_rate': 0.00029770114942528734, 'epoch': 0.28}

 28%|██▊       | 102/361 [24:18<1:01:53, 14.34s/it]
 29%|██▊       | 103/361 [24:32<1:01:37, 14.33s/it]
                                                   
{'loss': 1.6462, 'learning_rate': 0.00029655172413793103, 'epoch': 0.28}

 29%|██▊       | 103/361 [24:32<1:01:37, 14.33s/it]
 29%|██▉       | 104/361 [24:46<1:01:04, 14.26s/it]
                                                   
{'loss': 1.7179, 'learning_rate': 0.00029540229885057466, 'epoch': 0.29}

 29%|██▉       | 104/361 [24:46<1:01:04, 14.26s/it]
 29%|██▉       | 105/361 [25:00<1:00:24, 14.16s/it]
                                                   
{'loss': 1.7034, 'learning_rate': 0.00029425287356321835, 'epoch': 0.29}

 29%|██▉       | 105/361 [25:00<1:00:24, 14.16s/it]
 29%|██▉       | 106/361 [25:14<1:00:15, 14.18s/it]
                                                   
{'loss': 1.6211, 'learning_rate': 0.00029310344827586203, 'epoch': 0.29}

 29%|██▉       | 106/361 [25:14<1:00:15, 14.18s/it]
 30%|██▉       | 107/361 [25:29<1:00:21, 14.26s/it]
                                                   
{'loss': 1.6253, 'learning_rate': 0.0002919540229885057, 'epoch': 0.3}

 30%|██▉       | 107/361 [25:29<1:00:21, 14.26s/it]
 30%|██▉       | 108/361 [25:43<1:00:17, 14.30s/it]
                                                   
{'loss': 1.6153, 'learning_rate': 0.0002908045977011494, 'epoch': 0.3}

 30%|██▉       | 108/361 [25:43<1:00:17, 14.30s/it]
 30%|███       | 109/361 [25:58<1:00:38, 14.44s/it]
                                                   
{'loss': 1.6277, 'learning_rate': 0.0002896551724137931, 'epoch': 0.3}

 30%|███       | 109/361 [25:58<1:00:38, 14.44s/it]
 30%|███       | 110/361 [26:12<1:00:17, 14.41s/it]
                                                   
{'loss': 1.6665, 'learning_rate': 0.00028850574712643677, 'epoch': 0.3}

 30%|███       | 110/361 [26:12<1:00:17, 14.41s/it]
 31%|███       | 111/361 [26:26<59:54, 14.38s/it]  
                                                 
{'loss': 1.6547, 'learning_rate': 0.0002873563218390804, 'epoch': 0.31}

 31%|███       | 111/361 [26:26<59:54, 14.38s/it]
 31%|███       | 112/361 [26:41<59:41, 14.38s/it]
                                                 
{'loss': 1.6509, 'learning_rate': 0.0002862068965517241, 'epoch': 0.31}

 31%|███       | 112/361 [26:41<59:41, 14.38s/it]
 31%|███▏      | 113/361 [26:55<59:26, 14.38s/it]
                                                 
{'loss': 1.603, 'learning_rate': 0.00028505747126436777, 'epoch': 0.31}

 31%|███▏      | 113/361 [26:55<59:26, 14.38s/it]
 32%|███▏      | 114/361 [27:09<58:57, 14.32s/it]
                                                 
{'loss': 1.6052, 'learning_rate': 0.00028390804597701146, 'epoch': 0.32}

 32%|███▏      | 114/361 [27:09<58:57, 14.32s/it]
 32%|███▏      | 115/361 [27:24<58:34, 14.28s/it]
                                                 
{'loss': 1.7615, 'learning_rate': 0.00028275862068965514, 'epoch': 0.32}

 32%|███▏      | 115/361 [27:24<58:34, 14.28s/it]
 32%|███▏      | 116/361 [27:38<58:32, 14.34s/it]
                                                 
{'loss': 1.6572, 'learning_rate': 0.00028160919540229883, 'epoch': 0.32}

 32%|███▏      | 116/361 [27:38<58:32, 14.34s/it]
 32%|███▏      | 117/361 [27:52<58:20, 14.35s/it]
                                                 
{'loss': 1.6871, 'learning_rate': 0.0002804597701149425, 'epoch': 0.32}

 32%|███▏      | 117/361 [27:52<58:20, 14.35s/it]
 33%|███▎      | 118/361 [28:07<58:07, 14.35s/it]
                                                 
{'loss': 1.6544, 'learning_rate': 0.0002793103448275862, 'epoch': 0.33}

 33%|███▎      | 118/361 [28:07<58:07, 14.35s/it]
 33%|███▎      | 119/361 [28:21<57:47, 14.33s/it]
                                                 
{'loss': 1.6125, 'learning_rate': 0.0002781609195402299, 'epoch': 0.33}

 33%|███▎      | 119/361 [28:21<57:47, 14.33s/it]
 33%|███▎      | 120/361 [28:35<57:35, 14.34s/it]
                                                 
{'loss': 1.6059, 'learning_rate': 0.00027701149425287357, 'epoch': 0.33}

 33%|███▎      | 120/361 [28:35<57:35, 14.34s/it]
 34%|███▎      | 121/361 [28:50<57:24, 14.35s/it]
                                                 
{'loss': 1.6807, 'learning_rate': 0.0002758620689655172, 'epoch': 0.33}

 34%|███▎      | 121/361 [28:50<57:24, 14.35s/it]
 34%|███▍      | 122/361 [29:04<57:11, 14.36s/it]
                                                 
{'loss': 1.6111, 'learning_rate': 0.0002747126436781609, 'epoch': 0.34}

 34%|███▍      | 122/361 [29:04<57:11, 14.36s/it]
 34%|███▍      | 123/361 [29:18<56:58, 14.36s/it]
                                                 
{'loss': 1.6702, 'learning_rate': 0.00027356321839080457, 'epoch': 0.34}

 34%|███▍      | 123/361 [29:19<56:58, 14.36s/it]
 34%|███▍      | 124/361 [29:33<56:48, 14.38s/it]
                                                 
{'loss': 1.6357, 'learning_rate': 0.00027241379310344825, 'epoch': 0.34}

 34%|███▍      | 124/361 [29:33<56:48, 14.38s/it]
 35%|███▍      | 125/361 [29:47<56:31, 14.37s/it]
                                                 
{'loss': 1.6253, 'learning_rate': 0.00027126436781609194, 'epoch': 0.35}

 35%|███▍      | 125/361 [29:47<56:31, 14.37s/it]
 35%|███▍      | 126/361 [30:01<56:04, 14.32s/it]
                                                 
{'loss': 1.5386, 'learning_rate': 0.0002701149425287356, 'epoch': 0.35}

 35%|███▍      | 126/361 [30:01<56:04, 14.32s/it]
 35%|███▌      | 127/361 [30:16<56:21, 14.45s/it]
                                                 
{'loss': 1.6946, 'learning_rate': 0.0002689655172413793, 'epoch': 0.35}

 35%|███▌      | 127/361 [30:16<56:21, 14.45s/it]
 35%|███▌      | 128/361 [30:31<56:00, 14.42s/it]
                                                 
{'loss': 1.6903, 'learning_rate': 0.00026781609195402294, 'epoch': 0.35}

 35%|███▌      | 128/361 [30:31<56:00, 14.42s/it]
 36%|███▌      | 129/361 [30:45<55:43, 14.41s/it]
                                                 
{'loss': 1.6767, 'learning_rate': 0.0002666666666666666, 'epoch': 0.36}

 36%|███▌      | 129/361 [30:45<55:43, 14.41s/it]
 36%|███▌      | 130/361 [30:59<55:15, 14.35s/it]
                                                 
{'loss': 1.6023, 'learning_rate': 0.0002655172413793103, 'epoch': 0.36}

 36%|███▌      | 130/361 [30:59<55:15, 14.35s/it]
 36%|███▋      | 131/361 [31:14<55:02, 14.36s/it]
                                                 
{'loss': 1.6751, 'learning_rate': 0.000264367816091954, 'epoch': 0.36}

 36%|███▋      | 131/361 [31:14<55:02, 14.36s/it]
 37%|███▋      | 132/361 [31:28<54:52, 14.38s/it]
                                                 
{'loss': 1.6141, 'learning_rate': 0.0002632183908045977, 'epoch': 0.37}

 37%|███▋      | 132/361 [31:28<54:52, 14.38s/it]
 37%|███▋      | 133/361 [31:42<54:34, 14.36s/it]
                                                 
{'loss': 1.6229, 'learning_rate': 0.00026206896551724137, 'epoch': 0.37}

 37%|███▋      | 133/361 [31:42<54:34, 14.36s/it]
 37%|███▋      | 134/361 [31:57<54:21, 14.37s/it]
                                                 
{'loss': 1.6925, 'learning_rate': 0.00026091954022988505, 'epoch': 0.37}

 37%|███▋      | 134/361 [31:57<54:21, 14.37s/it]
 37%|███▋      | 135/361 [32:11<54:12, 14.39s/it]
                                                 
{'loss': 1.6636, 'learning_rate': 0.0002597701149425287, 'epoch': 0.37}

 37%|███▋      | 135/361 [32:11<54:12, 14.39s/it]
 38%|███▊      | 136/361 [32:25<53:56, 14.38s/it]
                                                 
{'loss': 1.6155, 'learning_rate': 0.00025862068965517237, 'epoch': 0.38}

 38%|███▊      | 136/361 [32:25<53:56, 14.38s/it]
 38%|███▊      | 137/361 [32:40<53:43, 14.39s/it]
                                                 
{'loss': 1.6244, 'learning_rate': 0.00025747126436781605, 'epoch': 0.38}

 38%|███▊      | 137/361 [32:40<53:43, 14.39s/it]
 38%|███▊      | 138/361 [32:54<53:28, 14.39s/it]
                                                 
{'loss': 1.6196, 'learning_rate': 0.00025632183908045974, 'epoch': 0.38}

 38%|███▊      | 138/361 [32:54<53:28, 14.39s/it]
 39%|███▊      | 139/361 [33:09<53:11, 14.38s/it]
                                                 
{'loss': 1.6389, 'learning_rate': 0.0002551724137931034, 'epoch': 0.38}

 39%|███▊      | 139/361 [33:09<53:11, 14.38s/it]
 39%|███▉      | 140/361 [33:23<52:54, 14.36s/it]
                                                 
{'loss': 1.6491, 'learning_rate': 0.0002540229885057471, 'epoch': 0.39}

 39%|███▉      | 140/361 [33:23<52:54, 14.36s/it]
 39%|███▉      | 141/361 [33:37<52:28, 14.31s/it]
                                                 
{'loss': 1.6818, 'learning_rate': 0.0002528735632183908, 'epoch': 0.39}

 39%|███▉      | 141/361 [33:37<52:28, 14.31s/it]
 39%|███▉      | 142/361 [33:52<52:17, 14.33s/it]
                                                 
{'loss': 1.634, 'learning_rate': 0.0002517241379310345, 'epoch': 0.39}

 39%|███▉      | 142/361 [33:52<52:17, 14.33s/it]
 40%|███▉      | 143/361 [34:06<52:07, 14.35s/it]
                                                 
{'loss': 1.6917, 'learning_rate': 0.00025057471264367816, 'epoch': 0.4}

 40%|███▉      | 143/361 [34:06<52:07, 14.35s/it]
 40%|███▉      | 144/361 [34:20<51:56, 14.36s/it]
                                                 
{'loss': 1.5505, 'learning_rate': 0.00024942528735632185, 'epoch': 0.4}

 40%|███▉      | 144/361 [34:20<51:56, 14.36s/it]
 40%|████      | 145/361 [34:35<52:02, 14.46s/it]
                                                 
{'loss': 1.5834, 'learning_rate': 0.0002482758620689655, 'epoch': 0.4}

 40%|████      | 145/361 [34:35<52:02, 14.46s/it]
 40%|████      | 146/361 [34:49<51:39, 14.42s/it]
                                                 
{'loss': 1.6417, 'learning_rate': 0.00024712643678160916, 'epoch': 0.4}

 40%|████      | 146/361 [34:49<51:39, 14.42s/it]
 41%|████      | 147/361 [35:04<51:17, 14.38s/it]
                                                 
{'loss': 1.6793, 'learning_rate': 0.00024597701149425285, 'epoch': 0.41}

 41%|████      | 147/361 [35:04<51:17, 14.38s/it]
 41%|████      | 148/361 [35:18<51:02, 14.38s/it]
                                                 
{'loss': 1.6625, 'learning_rate': 0.00024482758620689653, 'epoch': 0.41}

 41%|████      | 148/361 [35:18<51:02, 14.38s/it]
 41%|████▏     | 149/361 [35:32<50:48, 14.38s/it]
                                                 
{'loss': 1.6361, 'learning_rate': 0.0002436781609195402, 'epoch': 0.41}

 41%|████▏     | 149/361 [35:32<50:48, 14.38s/it]
 42%|████▏     | 150/361 [35:47<50:34, 14.38s/it]
                                                 
{'loss': 1.6773, 'learning_rate': 0.0002425287356321839, 'epoch': 0.41}

 42%|████▏     | 150/361 [35:47<50:34, 14.38s/it]
 42%|████▏     | 151/361 [36:01<50:14, 14.35s/it]
                                                 
{'loss': 1.6375, 'learning_rate': 0.00024137931034482756, 'epoch': 0.42}

 42%|████▏     | 151/361 [36:01<50:14, 14.35s/it]
 42%|████▏     | 152/361 [36:15<50:01, 14.36s/it]
                                                 
{'loss': 1.6254, 'learning_rate': 0.00024022988505747125, 'epoch': 0.42}

 42%|████▏     | 152/361 [36:15<50:01, 14.36s/it]
 42%|████▏     | 153/361 [36:30<49:48, 14.37s/it]
                                                 
{'loss': 1.6667, 'learning_rate': 0.0002390804597701149, 'epoch': 0.42}

 42%|████▏     | 153/361 [36:30<49:48, 14.37s/it]
 43%|████▎     | 154/361 [36:44<49:35, 14.37s/it]
                                                 
{'loss': 1.5985, 'learning_rate': 0.00023793103448275862, 'epoch': 0.43}

 43%|████▎     | 154/361 [36:44<49:35, 14.37s/it]
 43%|████▎     | 155/361 [36:58<49:11, 14.33s/it]
                                                 
{'loss': 1.6405, 'learning_rate': 0.00023678160919540228, 'epoch': 0.43}

 43%|████▎     | 155/361 [36:58<49:11, 14.33s/it]
 43%|████▎     | 156/361 [37:12<48:32, 14.21s/it]
                                                 
{'loss': 1.6028, 'learning_rate': 0.00023563218390804593, 'epoch': 0.43}

 43%|████▎     | 156/361 [37:12<48:32, 14.21s/it]
 43%|████▎     | 157/361 [37:27<48:34, 14.29s/it]
                                                 
{'loss': 1.7533, 'learning_rate': 0.00023448275862068965, 'epoch': 0.43}

 43%|████▎     | 157/361 [37:27<48:34, 14.29s/it]
 44%|████▍     | 158/361 [37:41<48:14, 14.26s/it]
                                                 
{'loss': 1.5809, 'learning_rate': 0.0002333333333333333, 'epoch': 0.44}

 44%|████▍     | 158/361 [37:41<48:14, 14.26s/it]
 44%|████▍     | 159/361 [37:55<47:56, 14.24s/it]
                                                 
{'loss': 1.6312, 'learning_rate': 0.000232183908045977, 'epoch': 0.44}

 44%|████▍     | 159/361 [37:55<47:56, 14.24s/it]
 44%|████▍     | 160/361 [38:10<47:50, 14.28s/it]
                                                 
{'loss': 1.6728, 'learning_rate': 0.00023103448275862065, 'epoch': 0.44}

 44%|████▍     | 160/361 [38:10<47:50, 14.28s/it]
 45%|████▍     | 161/361 [38:24<47:43, 14.32s/it]
                                                 
{'loss': 1.6719, 'learning_rate': 0.00022988505747126436, 'epoch': 0.45}

 45%|████▍     | 161/361 [38:24<47:43, 14.32s/it]
 45%|████▍     | 162/361 [38:38<47:33, 14.34s/it]
                                                 
{'loss': 1.6392, 'learning_rate': 0.00022873563218390802, 'epoch': 0.45}

 45%|████▍     | 162/361 [38:38<47:33, 14.34s/it]
 45%|████▌     | 163/361 [38:53<47:43, 14.46s/it]
                                                 
{'loss': 1.5975, 'learning_rate': 0.0002275862068965517, 'epoch': 0.45}

 45%|████▌     | 163/361 [38:53<47:43, 14.46s/it]
 45%|████▌     | 164/361 [39:07<47:23, 14.43s/it]
                                                 
{'loss': 1.5896, 'learning_rate': 0.0002264367816091954, 'epoch': 0.45}

 45%|████▌     | 164/361 [39:07<47:23, 14.43s/it]
 46%|████▌     | 165/361 [39:22<47:06, 14.42s/it]
                                                 
{'loss': 1.591, 'learning_rate': 0.00022528735632183907, 'epoch': 0.46}

 46%|████▌     | 165/361 [39:22<47:06, 14.42s/it]
 46%|████▌     | 166/361 [39:36<46:54, 14.43s/it]
                                                 
{'loss': 1.6889, 'learning_rate': 0.00022413793103448273, 'epoch': 0.46}

 46%|████▌     | 166/361 [39:36<46:54, 14.43s/it]
 46%|████▋     | 167/361 [39:51<46:41, 14.44s/it]
                                                 
{'loss': 1.6009, 'learning_rate': 0.00022298850574712644, 'epoch': 0.46}

 46%|████▋     | 167/361 [39:51<46:41, 14.44s/it]
 47%|████▋     | 168/361 [40:05<46:20, 14.41s/it]
                                                 
{'loss': 1.6512, 'learning_rate': 0.0002218390804597701, 'epoch': 0.46}

 47%|████▋     | 168/361 [40:05<46:20, 14.41s/it]
 47%|████▋     | 169/361 [40:20<46:06, 14.41s/it]
                                                 
{'loss': 1.57, 'learning_rate': 0.00022068965517241376, 'epoch': 0.47}

 47%|████▋     | 169/361 [40:20<46:06, 14.41s/it]
 47%|████▋     | 170/361 [40:34<45:35, 14.32s/it]
                                                 
{'loss': 1.6568, 'learning_rate': 0.00021954022988505744, 'epoch': 0.47}

 47%|████▋     | 170/361 [40:34<45:35, 14.32s/it]
 47%|████▋     | 171/361 [40:48<45:13, 14.28s/it]
                                                 
{'loss': 1.6009, 'learning_rate': 0.00021839080459770113, 'epoch': 0.47}

 47%|████▋     | 171/361 [40:48<45:13, 14.28s/it]
 48%|████▊     | 172/361 [41:02<45:07, 14.33s/it]
                                                 
{'loss': 1.668, 'learning_rate': 0.00021724137931034481, 'epoch': 0.48}

 48%|████▊     | 172/361 [41:02<45:07, 14.33s/it]
 48%|████▊     | 173/361 [41:17<44:57, 14.35s/it]
                                                 
{'loss': 1.5728, 'learning_rate': 0.00021609195402298847, 'epoch': 0.48}

 48%|████▊     | 173/361 [41:17<44:57, 14.35s/it]
 48%|████▊     | 174/361 [41:31<44:44, 14.36s/it]
                                                 
{'loss': 1.6792, 'learning_rate': 0.00021494252873563218, 'epoch': 0.48}

 48%|████▊     | 174/361 [41:31<44:44, 14.36s/it]
 48%|████▊     | 175/361 [41:45<44:33, 14.37s/it]
                                                 
{'loss': 1.6456, 'learning_rate': 0.00021379310344827584, 'epoch': 0.48}

 48%|████▊     | 175/361 [41:45<44:33, 14.37s/it]
 49%|████▉     | 176/361 [42:00<44:21, 14.38s/it]
                                                 
{'loss': 1.6221, 'learning_rate': 0.00021264367816091953, 'epoch': 0.49}

 49%|████▉     | 176/361 [42:00<44:21, 14.38s/it]
 49%|████▉     | 177/361 [42:14<44:09, 14.40s/it]
                                                 
{'loss': 1.5552, 'learning_rate': 0.00021149425287356319, 'epoch': 0.49}

 49%|████▉     | 177/361 [42:14<44:09, 14.40s/it]
 49%|████▉     | 178/361 [42:29<43:55, 14.40s/it]
                                                 
{'loss': 1.5363, 'learning_rate': 0.0002103448275862069, 'epoch': 0.49}

 49%|████▉     | 178/361 [42:29<43:55, 14.40s/it]
 50%|████▉     | 179/361 [42:43<43:41, 14.40s/it]
                                                 
{'loss': 1.6783, 'learning_rate': 0.00020919540229885056, 'epoch': 0.49}

 50%|████▉     | 179/361 [42:43<43:41, 14.40s/it]
 50%|████▉     | 180/361 [42:57<43:25, 14.39s/it]
                                                 
{'loss': 1.7241, 'learning_rate': 0.00020804597701149421, 'epoch': 0.5}

 50%|████▉     | 180/361 [42:57<43:25, 14.39s/it]
 50%|█████     | 181/361 [43:12<43:29, 14.50s/it]
                                                 
{'loss': 1.6665, 'learning_rate': 0.00020689655172413793, 'epoch': 0.5}

 50%|█████     | 181/361 [43:12<43:29, 14.50s/it]
 50%|█████     | 182/361 [43:27<43:07, 14.45s/it]
                                                 
{'loss': 1.6362, 'learning_rate': 0.00020574712643678158, 'epoch': 0.5}

 50%|█████     | 182/361 [43:27<43:07, 14.45s/it]
 51%|█████     | 183/361 [43:41<42:49, 14.44s/it]
                                                 
{'loss': 1.6315, 'learning_rate': 0.00020459770114942527, 'epoch': 0.51}

 51%|█████     | 183/361 [43:41<42:49, 14.44s/it]
 51%|█████     | 184/361 [43:55<42:29, 14.41s/it]
                                                 
{'loss': 1.6538, 'learning_rate': 0.00020344827586206893, 'epoch': 0.51}

 51%|█████     | 184/361 [43:55<42:29, 14.41s/it]
 51%|█████     | 185/361 [44:10<42:18, 14.42s/it]
                                                 
{'loss': 1.5876, 'learning_rate': 0.00020229885057471264, 'epoch': 0.51}

 51%|█████     | 185/361 [44:10<42:18, 14.42s/it]
 52%|█████▏    | 186/361 [44:24<41:57, 14.38s/it]
                                                 
{'loss': 1.5683, 'learning_rate': 0.0002011494252873563, 'epoch': 0.51}

 52%|█████▏    | 186/361 [44:24<41:57, 14.38s/it]
 52%|█████▏    | 187/361 [44:38<41:45, 14.40s/it]
                                                 
{'loss': 1.7053, 'learning_rate': 0.00019999999999999998, 'epoch': 0.52}

 52%|█████▏    | 187/361 [44:39<41:45, 14.40s/it]
 52%|█████▏    | 188/361 [44:53<41:29, 14.39s/it]
                                                 
{'loss': 1.671, 'learning_rate': 0.00019885057471264367, 'epoch': 0.52}

 52%|█████▏    | 188/361 [44:53<41:29, 14.39s/it]
 52%|█████▏    | 189/361 [45:07<41:15, 14.39s/it]
                                                 
{'loss': 1.6793, 'learning_rate': 0.00019770114942528735, 'epoch': 0.52}

 52%|█████▏    | 189/361 [45:07<41:15, 14.39s/it]
 53%|█████▎    | 190/361 [45:22<41:00, 14.39s/it]
                                                 
{'loss': 1.5236, 'learning_rate': 0.000196551724137931, 'epoch': 0.53}

 53%|█████▎    | 190/361 [45:22<41:00, 14.39s/it]
 53%|█████▎    | 191/361 [45:36<40:48, 14.40s/it]
                                                 
{'loss': 1.6141, 'learning_rate': 0.00019540229885057467, 'epoch': 0.53}

 53%|█████▎    | 191/361 [45:36<40:48, 14.40s/it]
 53%|█████▎    | 192/361 [45:50<40:34, 14.40s/it]
                                                 
{'loss': 1.5561, 'learning_rate': 0.00019425287356321838, 'epoch': 0.53}

 53%|█████▎    | 192/361 [45:50<40:34, 14.40s/it]
 53%|█████▎    | 193/361 [46:05<40:19, 14.40s/it]
                                                 
{'loss': 1.5873, 'learning_rate': 0.00019310344827586204, 'epoch': 0.53}

 53%|█████▎    | 193/361 [46:05<40:19, 14.40s/it]
 54%|█████▎    | 194/361 [46:19<40:02, 14.39s/it]
                                                 
{'loss': 1.5669, 'learning_rate': 0.00019195402298850572, 'epoch': 0.54}

 54%|█████▎    | 194/361 [46:19<40:02, 14.39s/it]
 54%|█████▍    | 195/361 [46:34<39:47, 14.38s/it]
                                                 
{'loss': 1.6227, 'learning_rate': 0.0001908045977011494, 'epoch': 0.54}

 54%|█████▍    | 195/361 [46:34<39:47, 14.38s/it]
 54%|█████▍    | 196/361 [46:48<39:30, 14.36s/it]
                                                 
{'loss': 1.5957, 'learning_rate': 0.0001896551724137931, 'epoch': 0.54}

 54%|█████▍    | 196/361 [46:48<39:30, 14.36s/it]
 55%|█████▍    | 197/361 [47:02<39:15, 14.36s/it]
                                                 
{'loss': 1.6133, 'learning_rate': 0.00018850574712643675, 'epoch': 0.54}

 55%|█████▍    | 197/361 [47:02<39:15, 14.36s/it]
 55%|█████▍    | 198/361 [47:17<39:04, 14.39s/it]
                                                 
{'loss': 1.6168, 'learning_rate': 0.00018735632183908046, 'epoch': 0.55}

 55%|█████▍    | 198/361 [47:17<39:04, 14.39s/it]
 55%|█████▌    | 199/361 [47:32<39:10, 14.51s/it]
                                                 
{'loss': 1.573, 'learning_rate': 0.00018620689655172412, 'epoch': 0.55}

 55%|█████▌    | 199/361 [47:32<39:10, 14.51s/it]
 55%|█████▌    | 200/361 [47:46<38:49, 14.47s/it]
                                                 
{'loss': 1.6656, 'learning_rate': 0.0001850574712643678, 'epoch': 0.55}

 55%|█████▌    | 200/361 [47:46<38:49, 14.47s/it]
 56%|█████▌    | 201/361 [48:00<38:31, 14.45s/it]
                                                 
{'loss': 1.6323, 'learning_rate': 0.00018390804597701147, 'epoch': 0.56}

 56%|█████▌    | 201/361 [48:00<38:31, 14.45s/it]
 56%|█████▌    | 202/361 [48:15<38:12, 14.42s/it]
                                                 
{'loss': 1.6756, 'learning_rate': 0.00018275862068965518, 'epoch': 0.56}

 56%|█████▌    | 202/361 [48:15<38:12, 14.42s/it]
 56%|█████▌    | 203/361 [48:29<37:55, 14.40s/it]
                                                 
{'loss': 1.6048, 'learning_rate': 0.00018160919540229884, 'epoch': 0.56}

 56%|█████▌    | 203/361 [48:29<37:55, 14.40s/it]
 57%|█████▋    | 204/361 [48:43<37:40, 14.40s/it]
                                                 
{'loss': 1.5862, 'learning_rate': 0.0001804597701149425, 'epoch': 0.56}

 57%|█████▋    | 204/361 [48:43<37:40, 14.40s/it]
 57%|█████▋    | 205/361 [48:58<37:23, 14.38s/it]
                                                 
{'loss': 1.6327, 'learning_rate': 0.0001793103448275862, 'epoch': 0.57}

 57%|█████▋    | 205/361 [48:58<37:23, 14.38s/it]
 57%|█████▋    | 206/361 [49:12<37:13, 14.41s/it]
                                                 
{'loss': 1.6932, 'learning_rate': 0.00017816091954022986, 'epoch': 0.57}

 57%|█████▋    | 206/361 [49:12<37:13, 14.41s/it]
 57%|█████▋    | 207/361 [49:27<36:59, 14.41s/it]
                                                 
{'loss': 1.643, 'learning_rate': 0.00017701149425287355, 'epoch': 0.57}

 57%|█████▋    | 207/361 [49:27<36:59, 14.41s/it]
 58%|█████▊    | 208/361 [49:41<36:44, 14.41s/it]
                                                 
{'loss': 1.5924, 'learning_rate': 0.0001758620689655172, 'epoch': 0.58}

 58%|█████▊    | 208/361 [49:41<36:44, 14.41s/it]
 58%|█████▊    | 209/361 [49:55<36:28, 14.40s/it]
                                                 
{'loss': 1.5611, 'learning_rate': 0.00017471264367816092, 'epoch': 0.58}

 58%|█████▊    | 209/361 [49:55<36:28, 14.40s/it]
 58%|█████▊    | 210/361 [50:10<36:11, 14.38s/it]
                                                 
{'loss': 1.6712, 'learning_rate': 0.00017356321839080458, 'epoch': 0.58}

 58%|█████▊    | 210/361 [50:10<36:11, 14.38s/it]
 58%|█████▊    | 211/361 [50:24<35:55, 14.37s/it]
                                                 
{'loss': 1.5662, 'learning_rate': 0.00017241379310344826, 'epoch': 0.58}

 58%|█████▊    | 211/361 [50:24<35:55, 14.37s/it]
 59%|█████▊    | 212/361 [50:38<35:39, 14.36s/it]
                                                 
{'loss': 1.6277, 'learning_rate': 0.00017126436781609195, 'epoch': 0.59}

 59%|█████▊    | 212/361 [50:38<35:39, 14.36s/it]
 59%|█████▉    | 213/361 [50:53<35:26, 14.36s/it]
                                                 
{'loss': 1.6404, 'learning_rate': 0.00017011494252873563, 'epoch': 0.59}

 59%|█████▉    | 213/361 [50:53<35:26, 14.36s/it]
 59%|█████▉    | 214/361 [51:07<35:05, 14.32s/it]
                                                 
{'loss': 1.7069, 'learning_rate': 0.0001689655172413793, 'epoch': 0.59}

 59%|█████▉    | 214/361 [51:07<35:05, 14.32s/it]
 60%|█████▉    | 215/361 [51:21<34:56, 14.36s/it]
                                                 
{'loss': 1.5288, 'learning_rate': 0.00016781609195402295, 'epoch': 0.59}

 60%|█████▉    | 215/361 [51:21<34:56, 14.36s/it]
 60%|█████▉    | 216/361 [51:35<34:24, 14.24s/it]
                                                 
{'loss': 1.6344, 'learning_rate': 0.00016666666666666666, 'epoch': 0.6}

 60%|█████▉    | 216/361 [51:35<34:24, 14.24s/it]
 60%|██████    | 217/361 [51:50<34:32, 14.39s/it]
                                                 
{'loss': 1.5356, 'learning_rate': 0.00016551724137931032, 'epoch': 0.6}

 60%|██████    | 217/361 [51:50<34:32, 14.39s/it]
 60%|██████    | 218/361 [52:05<34:16, 14.38s/it]
                                                 
{'loss': 1.6366, 'learning_rate': 0.000164367816091954, 'epoch': 0.6}

 60%|██████    | 218/361 [52:05<34:16, 14.38s/it]
 61%|██████    | 219/361 [52:19<34:01, 14.38s/it]
                                                 
{'loss': 1.6323, 'learning_rate': 0.0001632183908045977, 'epoch': 0.61}

 61%|██████    | 219/361 [52:19<34:01, 14.38s/it]
 61%|██████    | 220/361 [52:33<33:49, 14.39s/it]
                                                 
{'loss': 1.6561, 'learning_rate': 0.00016206896551724137, 'epoch': 0.61}

 61%|██████    | 220/361 [52:33<33:49, 14.39s/it]
 61%|██████    | 221/361 [52:48<33:35, 14.40s/it]
                                                 
{'loss': 1.6485, 'learning_rate': 0.00016091954022988503, 'epoch': 0.61}

 61%|██████    | 221/361 [52:48<33:35, 14.40s/it]
 61%|██████▏   | 222/361 [53:02<33:19, 14.38s/it]
                                                 
{'loss': 1.6628, 'learning_rate': 0.00015977011494252874, 'epoch': 0.61}

 61%|██████▏   | 222/361 [53:02<33:19, 14.38s/it]
 62%|██████▏   | 223/361 [53:16<32:53, 14.30s/it]
                                                 
{'loss': 1.6682, 'learning_rate': 0.0001586206896551724, 'epoch': 0.62}

 62%|██████▏   | 223/361 [53:16<32:53, 14.30s/it]
 62%|██████▏   | 224/361 [53:31<32:41, 14.32s/it]
                                                 
{'loss': 1.5617, 'learning_rate': 0.0001574712643678161, 'epoch': 0.62}

 62%|██████▏   | 224/361 [53:31<32:41, 14.32s/it]
 62%|██████▏   | 225/361 [53:45<32:28, 14.33s/it]
                                                 
{'loss': 1.6143, 'learning_rate': 0.00015632183908045975, 'epoch': 0.62}

 62%|██████▏   | 225/361 [53:45<32:28, 14.33s/it]
 63%|██████▎   | 226/361 [53:59<32:18, 14.36s/it]
                                                 
{'loss': 1.6176, 'learning_rate': 0.00015517241379310346, 'epoch': 0.62}

 63%|██████▎   | 226/361 [53:59<32:18, 14.36s/it]
 63%|██████▎   | 227/361 [54:14<32:07, 14.38s/it]
                                                 
{'loss': 1.6362, 'learning_rate': 0.00015402298850574712, 'epoch': 0.63}

 63%|██████▎   | 227/361 [54:14<32:07, 14.38s/it]
 63%|██████▎   | 228/361 [54:28<31:54, 14.40s/it]
                                                 
{'loss': 1.6523, 'learning_rate': 0.00015287356321839077, 'epoch': 0.63}

 63%|██████▎   | 228/361 [54:28<31:54, 14.40s/it]
 63%|██████▎   | 229/361 [54:43<31:41, 14.41s/it]
                                                 
{'loss': 1.6159, 'learning_rate': 0.00015172413793103449, 'epoch': 0.63}

 63%|██████▎   | 229/361 [54:43<31:41, 14.41s/it]
 64%|██████▎   | 230/361 [54:57<31:26, 14.40s/it]
                                                 
{'loss': 1.6564, 'learning_rate': 0.00015057471264367814, 'epoch': 0.64}

 64%|██████▎   | 230/361 [54:57<31:26, 14.40s/it]
 64%|██████▍   | 231/361 [55:11<31:08, 14.37s/it]
                                                 
{'loss': 1.6519, 'learning_rate': 0.00014942528735632183, 'epoch': 0.64}

 64%|██████▍   | 231/361 [55:11<31:08, 14.37s/it]
 64%|██████▍   | 232/361 [55:26<30:54, 14.37s/it]
                                                 
{'loss': 1.6385, 'learning_rate': 0.00014827586206896551, 'epoch': 0.64}

 64%|██████▍   | 232/361 [55:26<30:54, 14.37s/it]
 65%|██████▍   | 233/361 [55:40<30:26, 14.27s/it]
                                                 
{'loss': 1.5886, 'learning_rate': 0.00014712643678160917, 'epoch': 0.64}

 65%|██████▍   | 233/361 [55:40<30:26, 14.27s/it]
 65%|██████▍   | 234/361 [55:54<30:16, 14.30s/it]
                                                 
{'loss': 1.6783, 'learning_rate': 0.00014597701149425286, 'epoch': 0.65}

 65%|██████▍   | 234/361 [55:54<30:16, 14.30s/it]
 65%|██████▌   | 235/361 [56:09<30:20, 14.45s/it]
                                                 
{'loss': 1.629, 'learning_rate': 0.00014482758620689654, 'epoch': 0.65}

 65%|██████▌   | 235/361 [56:09<30:20, 14.45s/it]
 65%|██████▌   | 236/361 [56:23<30:02, 14.42s/it]
                                                 
{'loss': 1.6013, 'learning_rate': 0.0001436781609195402, 'epoch': 0.65}

 65%|██████▌   | 236/361 [56:23<30:02, 14.42s/it]
 66%|██████▌   | 237/361 [56:38<29:46, 14.41s/it]
                                                 
{'loss': 1.5773, 'learning_rate': 0.00014252873563218389, 'epoch': 0.66}

 66%|██████▌   | 237/361 [56:38<29:46, 14.41s/it]
 66%|██████▌   | 238/361 [56:52<29:30, 14.39s/it]
                                                 
{'loss': 1.6214, 'learning_rate': 0.00014137931034482757, 'epoch': 0.66}

 66%|██████▌   | 238/361 [56:52<29:30, 14.39s/it]
 66%|██████▌   | 239/361 [57:06<29:13, 14.37s/it]
                                                 
{'loss': 1.6043, 'learning_rate': 0.00014022988505747126, 'epoch': 0.66}

 66%|██████▌   | 239/361 [57:06<29:13, 14.37s/it]
 66%|██████▋   | 240/361 [57:21<29:00, 14.38s/it]
                                                 
{'loss': 1.6057, 'learning_rate': 0.00013908045977011494, 'epoch': 0.66}

 66%|██████▋   | 240/361 [57:21<29:00, 14.38s/it]
 67%|██████▋   | 241/361 [57:35<28:42, 14.36s/it]
                                                 
{'loss': 1.5984, 'learning_rate': 0.0001379310344827586, 'epoch': 0.67}

 67%|██████▋   | 241/361 [57:35<28:42, 14.36s/it]
 67%|██████▋   | 242/361 [57:49<28:30, 14.38s/it]
                                                 
{'loss': 1.6329, 'learning_rate': 0.00013678160919540228, 'epoch': 0.67}

 67%|██████▋   | 242/361 [57:49<28:30, 14.38s/it]
 67%|██████▋   | 243/361 [58:04<28:15, 14.37s/it]
                                                 
{'loss': 1.6342, 'learning_rate': 0.00013563218390804597, 'epoch': 0.67}

 67%|██████▋   | 243/361 [58:04<28:15, 14.37s/it]
 68%|██████▊   | 244/361 [58:18<28:03, 14.39s/it]
                                                 
{'loss': 1.5317, 'learning_rate': 0.00013448275862068965, 'epoch': 0.67}

 68%|██████▊   | 244/361 [58:18<28:03, 14.39s/it]
 68%|██████▊   | 245/361 [58:33<27:50, 14.40s/it]
                                                 
{'loss': 1.6759, 'learning_rate': 0.0001333333333333333, 'epoch': 0.68}

 68%|██████▊   | 245/361 [58:33<27:50, 14.40s/it]
 68%|██████▊   | 246/361 [58:47<27:37, 14.42s/it]
                                                 
{'loss': 1.6846, 'learning_rate': 0.000132183908045977, 'epoch': 0.68}

 68%|██████▊   | 246/361 [58:47<27:37, 14.42s/it]
 68%|██████▊   | 247/361 [59:02<27:24, 14.43s/it]
                                                 
{'loss': 1.5821, 'learning_rate': 0.00013103448275862068, 'epoch': 0.68}

 68%|██████▊   | 247/361 [59:02<27:24, 14.43s/it]
 69%|██████▊   | 248/361 [59:16<27:11, 14.44s/it]
                                                 
{'loss': 1.5665, 'learning_rate': 0.00012988505747126434, 'epoch': 0.69}

 69%|██████▊   | 248/361 [59:16<27:11, 14.44s/it]
 69%|██████▉   | 249/361 [59:30<26:57, 14.44s/it]
                                                 
{'loss': 1.5722, 'learning_rate': 0.00012873563218390803, 'epoch': 0.69}

 69%|██████▉   | 249/361 [59:31<26:57, 14.44s/it]
 69%|██████▉   | 250/361 [59:45<26:45, 14.46s/it]
                                                 
{'loss': 1.6493, 'learning_rate': 0.0001275862068965517, 'epoch': 0.69}

 69%|██████▉   | 250/361 [59:45<26:45, 14.46s/it]
 70%|██████▉   | 251/361 [59:59<26:29, 14.45s/it]
                                                 
{'loss': 1.6815, 'learning_rate': 0.0001264367816091954, 'epoch': 0.69}

 70%|██████▉   | 251/361 [59:59<26:29, 14.45s/it]
 70%|██████▉   | 252/361 [1:00:14<26:12, 14.43s/it]
                                                   
{'loss': 1.6594, 'learning_rate': 0.00012528735632183908, 'epoch': 0.7}

 70%|██████▉   | 252/361 [1:00:14<26:12, 14.43s/it]
 70%|███████   | 253/361 [1:00:28<25:53, 14.38s/it]
                                                   
{'loss': 1.6468, 'learning_rate': 0.00012413793103448274, 'epoch': 0.7}

 70%|███████   | 253/361 [1:00:28<25:53, 14.38s/it]
 70%|███████   | 254/361 [1:00:42<25:37, 14.37s/it]
                                                   
{'loss': 1.6097, 'learning_rate': 0.00012298850574712642, 'epoch': 0.7}

 70%|███████   | 254/361 [1:00:42<25:37, 14.37s/it]
 71%|███████   | 255/361 [1:00:57<25:23, 14.37s/it]
                                                   
{'loss': 1.5916, 'learning_rate': 0.0001218390804597701, 'epoch': 0.71}

 71%|███████   | 255/361 [1:00:57<25:23, 14.37s/it]
 71%|███████   | 256/361 [1:01:11<25:10, 14.38s/it]
                                                   
{'loss': 1.5732, 'learning_rate': 0.00012068965517241378, 'epoch': 0.71}

 71%|███████   | 256/361 [1:01:11<25:10, 14.38s/it]
 71%|███████   | 257/361 [1:01:26<24:53, 14.36s/it]
                                                   
{'loss': 1.6186, 'learning_rate': 0.00011954022988505745, 'epoch': 0.71}

 71%|███████   | 257/361 [1:01:26<24:53, 14.36s/it]
 71%|███████▏  | 258/361 [1:01:40<24:39, 14.36s/it]
                                                   
{'loss': 1.5508, 'learning_rate': 0.00011839080459770114, 'epoch': 0.71}

 71%|███████▏  | 258/361 [1:01:40<24:39, 14.36s/it]
 72%|███████▏  | 259/361 [1:01:54<24:12, 14.24s/it]
                                                   
{'loss': 1.6374, 'learning_rate': 0.00011724137931034482, 'epoch': 0.72}

 72%|███████▏  | 259/361 [1:01:54<24:12, 14.24s/it]
 72%|███████▏  | 260/361 [1:02:08<23:53, 14.19s/it]
                                                   
{'loss': 1.6315, 'learning_rate': 0.0001160919540229885, 'epoch': 0.72}

 72%|███████▏  | 260/361 [1:02:08<23:53, 14.19s/it]
 72%|███████▏  | 261/361 [1:02:22<23:43, 14.24s/it]
                                                   
{'loss': 1.6099, 'learning_rate': 0.00011494252873563218, 'epoch': 0.72}

 72%|███████▏  | 261/361 [1:02:22<23:43, 14.24s/it]
 73%|███████▎  | 262/361 [1:02:37<23:35, 14.29s/it]
                                                   
{'loss': 1.5565, 'learning_rate': 0.00011379310344827585, 'epoch': 0.72}

 73%|███████▎  | 262/361 [1:02:37<23:35, 14.29s/it]
 73%|███████▎  | 263/361 [1:02:50<23:06, 14.14s/it]
                                                   
{'loss': 1.5707, 'learning_rate': 0.00011264367816091954, 'epoch': 0.73}

 73%|███████▎  | 263/361 [1:02:50<23:06, 14.14s/it]
 73%|███████▎  | 264/361 [1:03:05<22:57, 14.20s/it]
                                                   
{'loss': 1.6592, 'learning_rate': 0.00011149425287356322, 'epoch': 0.73}

 73%|███████▎  | 264/361 [1:03:05<22:57, 14.20s/it]
 73%|███████▎  | 265/361 [1:03:19<22:48, 14.26s/it]
                                                   
{'loss': 1.5664, 'learning_rate': 0.00011034482758620688, 'epoch': 0.73}

 73%|███████▎  | 265/361 [1:03:19<22:48, 14.26s/it]
 74%|███████▎  | 266/361 [1:03:34<22:38, 14.30s/it]
                                                   
{'loss': 1.5931, 'learning_rate': 0.00010919540229885056, 'epoch': 0.74}

 74%|███████▎  | 266/361 [1:03:34<22:38, 14.30s/it]
 74%|███████▍  | 267/361 [1:03:48<22:17, 14.23s/it]
                                                   
{'loss': 1.5848, 'learning_rate': 0.00010804597701149424, 'epoch': 0.74}

 74%|███████▍  | 267/361 [1:03:48<22:17, 14.23s/it]
 74%|███████▍  | 268/361 [1:04:02<22:08, 14.28s/it]
                                                   
{'loss': 1.6641, 'learning_rate': 0.00010689655172413792, 'epoch': 0.74}

 74%|███████▍  | 268/361 [1:04:02<22:08, 14.28s/it]
 75%|███████▍  | 269/361 [1:04:16<21:56, 14.31s/it]
                                                   
{'loss': 1.7284, 'learning_rate': 0.00010574712643678159, 'epoch': 0.74}

 75%|███████▍  | 269/361 [1:04:16<21:56, 14.31s/it]
 75%|███████▍  | 270/361 [1:04:31<21:42, 14.32s/it]
                                                   
{'loss': 1.7095, 'learning_rate': 0.00010459770114942528, 'epoch': 0.75}

 75%|███████▍  | 270/361 [1:04:31<21:42, 14.32s/it]
 75%|███████▌  | 271/361 [1:04:46<21:40, 14.45s/it]
                                                   
{'loss': 1.5938, 'learning_rate': 0.00010344827586206896, 'epoch': 0.75}

 75%|███████▌  | 271/361 [1:04:46<21:40, 14.45s/it]
 75%|███████▌  | 272/361 [1:05:00<21:15, 14.33s/it]
                                                   
{'loss': 1.6439, 'learning_rate': 0.00010229885057471263, 'epoch': 0.75}

 75%|███████▌  | 272/361 [1:05:00<21:15, 14.33s/it]
 76%|███████▌  | 273/361 [1:05:14<21:02, 14.35s/it]
                                                   
{'loss': 1.6179, 'learning_rate': 0.00010114942528735632, 'epoch': 0.75}

 76%|███████▌  | 273/361 [1:05:14<21:02, 14.35s/it]
 76%|███████▌  | 274/361 [1:05:28<20:49, 14.37s/it]
                                                   
{'loss': 1.5525, 'learning_rate': 9.999999999999999e-05, 'epoch': 0.76}

 76%|███████▌  | 274/361 [1:05:28<20:49, 14.37s/it]
 76%|███████▌  | 275/361 [1:05:43<20:36, 14.38s/it]
                                                   
{'loss': 1.5942, 'learning_rate': 9.885057471264368e-05, 'epoch': 0.76}

 76%|███████▌  | 275/361 [1:05:43<20:36, 14.38s/it]
 76%|███████▋  | 276/361 [1:05:57<20:22, 14.39s/it]
                                                   
{'loss': 1.6181, 'learning_rate': 9.770114942528733e-05, 'epoch': 0.76}

 76%|███████▋  | 276/361 [1:05:57<20:22, 14.39s/it]
 77%|███████▋  | 277/361 [1:06:12<20:08, 14.38s/it]
                                                   
{'loss': 1.6865, 'learning_rate': 9.655172413793102e-05, 'epoch': 0.77}

 77%|███████▋  | 277/361 [1:06:12<20:08, 14.38s/it]
 77%|███████▋  | 278/361 [1:06:26<19:45, 14.28s/it]
                                                   
{'loss': 1.6395, 'learning_rate': 9.54022988505747e-05, 'epoch': 0.77}

 77%|███████▋  | 278/361 [1:06:26<19:45, 14.28s/it]
 77%|███████▋  | 279/361 [1:06:40<19:35, 14.34s/it]
                                                   
{'loss': 1.5693, 'learning_rate': 9.425287356321838e-05, 'epoch': 0.77}

 77%|███████▋  | 279/361 [1:06:40<19:35, 14.34s/it]
 78%|███████▊  | 280/361 [1:06:54<19:23, 14.36s/it]
                                                   
{'loss': 1.6367, 'learning_rate': 9.310344827586206e-05, 'epoch': 0.77}

 78%|███████▊  | 280/361 [1:06:55<19:23, 14.36s/it]
 78%|███████▊  | 281/361 [1:07:09<19:06, 14.33s/it]
                                                   
{'loss': 1.6328, 'learning_rate': 9.195402298850573e-05, 'epoch': 0.78}

 78%|███████▊  | 281/361 [1:07:09<19:06, 14.33s/it]
 78%|███████▊  | 282/361 [1:07:23<18:54, 14.35s/it]
                                                   
{'loss': 1.6438, 'learning_rate': 9.080459770114942e-05, 'epoch': 0.78}

 78%|███████▊  | 282/361 [1:07:23<18:54, 14.35s/it]
 78%|███████▊  | 283/361 [1:07:38<18:40, 14.37s/it]
                                                   
{'loss': 1.6292, 'learning_rate': 8.96551724137931e-05, 'epoch': 0.78}

 78%|███████▊  | 283/361 [1:07:38<18:40, 14.37s/it]
 79%|███████▊  | 284/361 [1:07:52<18:28, 14.40s/it]
                                                   
{'loss': 1.5556, 'learning_rate': 8.850574712643677e-05, 'epoch': 0.79}

 79%|███████▊  | 284/361 [1:07:52<18:28, 14.40s/it]
 79%|███████▉  | 285/361 [1:08:06<18:15, 14.41s/it]
                                                   
{'loss': 1.6025, 'learning_rate': 8.735632183908046e-05, 'epoch': 0.79}

 79%|███████▉  | 285/361 [1:08:06<18:15, 14.41s/it]
 79%|███████▉  | 286/361 [1:08:21<18:02, 14.43s/it]
                                                   
{'loss': 1.6471, 'learning_rate': 8.620689655172413e-05, 'epoch': 0.79}

 79%|███████▉  | 286/361 [1:08:21<18:02, 14.43s/it]
 80%|███████▉  | 287/361 [1:08:35<17:47, 14.43s/it]
                                                   
{'loss': 1.5813, 'learning_rate': 8.505747126436782e-05, 'epoch': 0.79}

 80%|███████▉  | 287/361 [1:08:35<17:47, 14.43s/it]
 80%|███████▉  | 288/361 [1:08:50<17:32, 14.41s/it]
                                                   
{'loss': 1.6622, 'learning_rate': 8.390804597701147e-05, 'epoch': 0.8}

 80%|███████▉  | 288/361 [1:08:50<17:32, 14.41s/it]
 80%|████████  | 289/361 [1:09:05<17:25, 14.52s/it]
                                                   
{'loss': 1.579, 'learning_rate': 8.275862068965516e-05, 'epoch': 0.8}

 80%|████████  | 289/361 [1:09:05<17:25, 14.52s/it]
 80%|████████  | 290/361 [1:09:19<17:05, 14.45s/it]
                                                   
{'loss': 1.6367, 'learning_rate': 8.160919540229884e-05, 'epoch': 0.8}

 80%|████████  | 290/361 [1:09:19<17:05, 14.45s/it]
 81%|████████  | 291/361 [1:09:33<16:49, 14.42s/it]
                                                   
{'loss': 1.5652, 'learning_rate': 8.045977011494252e-05, 'epoch': 0.8}

 81%|████████  | 291/361 [1:09:33<16:49, 14.42s/it]
 81%|████████  | 292/361 [1:09:47<16:24, 14.26s/it]
                                                   
{'loss': 1.5645, 'learning_rate': 7.93103448275862e-05, 'epoch': 0.81}

 81%|████████  | 292/361 [1:09:47<16:24, 14.26s/it]
 81%|████████  | 293/361 [1:10:01<16:08, 14.24s/it]
                                                   
{'loss': 1.656, 'learning_rate': 7.816091954022987e-05, 'epoch': 0.81}

 81%|████████  | 293/361 [1:10:01<16:08, 14.24s/it]
 81%|████████▏ | 294/361 [1:10:15<15:51, 14.20s/it]
                                                   
{'loss': 1.707, 'learning_rate': 7.701149425287356e-05, 'epoch': 0.81}

 81%|████████▏ | 294/361 [1:10:15<15:51, 14.20s/it]
 82%|████████▏ | 295/361 [1:10:29<15:34, 14.15s/it]
                                                   
{'loss': 1.5999, 'learning_rate': 7.586206896551724e-05, 'epoch': 0.82}

 82%|████████▏ | 295/361 [1:10:29<15:34, 14.15s/it]
 82%|████████▏ | 296/361 [1:10:44<15:24, 14.23s/it]
                                                   
{'loss': 1.709, 'learning_rate': 7.471264367816091e-05, 'epoch': 0.82}

 82%|████████▏ | 296/361 [1:10:44<15:24, 14.23s/it]
 82%|████████▏ | 297/361 [1:10:58<15:15, 14.30s/it]
                                                   
{'loss': 1.5729, 'learning_rate': 7.356321839080459e-05, 'epoch': 0.82}

 82%|████████▏ | 297/361 [1:10:58<15:15, 14.30s/it]
 83%|████████▎ | 298/361 [1:11:13<15:03, 14.34s/it]
                                                   
{'loss': 1.6196, 'learning_rate': 7.241379310344827e-05, 'epoch': 0.82}

 83%|████████▎ | 298/361 [1:11:13<15:03, 14.34s/it]
 83%|████████▎ | 299/361 [1:11:27<14:50, 14.36s/it]
                                                   
{'loss': 1.609, 'learning_rate': 7.126436781609194e-05, 'epoch': 0.83}

 83%|████████▎ | 299/361 [1:11:27<14:50, 14.36s/it]
 83%|████████▎ | 300/361 [1:11:41<14:36, 14.36s/it]
                                                   
{'loss': 1.6102, 'learning_rate': 7.011494252873563e-05, 'epoch': 0.83}

 83%|████████▎ | 300/361 [1:11:41<14:36, 14.36s/it]
 83%|████████▎ | 301/361 [1:11:56<14:21, 14.36s/it]
                                                   
{'loss': 1.6356, 'learning_rate': 6.89655172413793e-05, 'epoch': 0.83}

 83%|████████▎ | 301/361 [1:11:56<14:21, 14.36s/it]
 84%|████████▎ | 302/361 [1:12:10<14:08, 14.37s/it]
                                                   
{'loss': 1.5914, 'learning_rate': 6.781609195402298e-05, 'epoch': 0.84}

 84%|████████▎ | 302/361 [1:12:10<14:08, 14.37s/it]
 84%|████████▍ | 303/361 [1:12:24<13:45, 14.24s/it]
                                                   
{'loss': 1.608, 'learning_rate': 6.666666666666666e-05, 'epoch': 0.84}

 84%|████████▍ | 303/361 [1:12:24<13:45, 14.24s/it]
 84%|████████▍ | 304/361 [1:12:39<13:34, 14.29s/it]
                                                   
{'loss': 1.6081, 'learning_rate': 6.551724137931034e-05, 'epoch': 0.84}

 84%|████████▍ | 304/361 [1:12:39<13:34, 14.29s/it]
 84%|████████▍ | 305/361 [1:12:52<13:13, 14.17s/it]
                                                   
{'loss': 1.5825, 'learning_rate': 6.436781609195401e-05, 'epoch': 0.84}

 84%|████████▍ | 305/361 [1:12:52<13:13, 14.17s/it]
 85%|████████▍ | 306/361 [1:13:07<13:03, 14.25s/it]
                                                   
{'loss': 1.5832, 'learning_rate': 6.32183908045977e-05, 'epoch': 0.85}

 85%|████████▍ | 306/361 [1:13:07<13:03, 14.25s/it]
 85%|████████▌ | 307/361 [1:13:22<12:57, 14.40s/it]
                                                   
{'loss': 1.5786, 'learning_rate': 6.206896551724137e-05, 'epoch': 0.85}

 85%|████████▌ | 307/361 [1:13:22<12:57, 14.40s/it]
 85%|████████▌ | 308/361 [1:13:36<12:43, 14.40s/it]
                                                   
{'loss': 1.5374, 'learning_rate': 6.091954022988505e-05, 'epoch': 0.85}

 85%|████████▌ | 308/361 [1:13:36<12:43, 14.40s/it]
 86%|████████▌ | 309/361 [1:13:50<12:20, 14.24s/it]
                                                   
{'loss': 1.5281, 'learning_rate': 5.9770114942528726e-05, 'epoch': 0.85}

 86%|████████▌ | 309/361 [1:13:50<12:20, 14.24s/it]
 86%|████████▌ | 310/361 [1:14:04<12:08, 14.28s/it]
                                                   
{'loss': 1.6609, 'learning_rate': 5.862068965517241e-05, 'epoch': 0.86}

 86%|████████▌ | 310/361 [1:14:04<12:08, 14.28s/it]
 86%|████████▌ | 311/361 [1:14:19<11:56, 14.33s/it]
                                                   
{'loss': 1.6847, 'learning_rate': 5.747126436781609e-05, 'epoch': 0.86}

 86%|████████▌ | 311/361 [1:14:19<11:56, 14.33s/it]
 86%|████████▋ | 312/361 [1:14:33<11:43, 14.35s/it]
                                                   
{'loss': 1.6644, 'learning_rate': 5.632183908045977e-05, 'epoch': 0.86}

 86%|████████▋ | 312/361 [1:14:33<11:43, 14.35s/it]
 87%|████████▋ | 313/361 [1:14:48<11:29, 14.37s/it]
                                                   
{'loss': 1.6139, 'learning_rate': 5.517241379310344e-05, 'epoch': 0.87}

 87%|████████▋ | 313/361 [1:14:48<11:29, 14.37s/it]
 87%|████████▋ | 314/361 [1:15:02<11:14, 14.34s/it]
                                                   
{'loss': 1.5847, 'learning_rate': 5.402298850574712e-05, 'epoch': 0.87}

 87%|████████▋ | 314/361 [1:15:02<11:14, 14.34s/it]
 87%|████████▋ | 315/361 [1:15:16<10:59, 14.33s/it]
                                                   
{'loss': 1.6658, 'learning_rate': 5.2873563218390796e-05, 'epoch': 0.87}

 87%|████████▋ | 315/361 [1:15:16<10:59, 14.33s/it]
 88%|████████▊ | 316/361 [1:15:31<10:45, 14.34s/it]
                                                   
{'loss': 1.6154, 'learning_rate': 5.172413793103448e-05, 'epoch': 0.87}

 88%|████████▊ | 316/361 [1:15:31<10:45, 14.34s/it]
 88%|████████▊ | 317/361 [1:15:45<10:30, 14.33s/it]
                                                   
{'loss': 1.5801, 'learning_rate': 5.057471264367816e-05, 'epoch': 0.88}

 88%|████████▊ | 317/361 [1:15:45<10:30, 14.33s/it]
 88%|████████▊ | 318/361 [1:15:59<10:17, 14.35s/it]
                                                   
{'loss': 1.5948, 'learning_rate': 4.942528735632184e-05, 'epoch': 0.88}

 88%|████████▊ | 318/361 [1:15:59<10:17, 14.35s/it]
 88%|████████▊ | 319/361 [1:16:14<10:02, 14.36s/it]
                                                   
{'loss': 1.6545, 'learning_rate': 4.827586206896551e-05, 'epoch': 0.88}

 88%|████████▊ | 319/361 [1:16:14<10:02, 14.36s/it]
 89%|████████▊ | 320/361 [1:16:28<09:49, 14.37s/it]
                                                   
{'loss': 1.6334, 'learning_rate': 4.712643678160919e-05, 'epoch': 0.88}

 89%|████████▊ | 320/361 [1:16:28<09:49, 14.37s/it]
 89%|████████▉ | 321/361 [1:16:42<09:34, 14.35s/it]
                                                   
{'loss': 1.7223, 'learning_rate': 4.5977011494252866e-05, 'epoch': 0.89}

 89%|████████▉ | 321/361 [1:16:42<09:34, 14.35s/it]
 89%|████████▉ | 322/361 [1:16:57<09:20, 14.37s/it]
                                                   
{'loss': 1.5846, 'learning_rate': 4.482758620689655e-05, 'epoch': 0.89}

 89%|████████▉ | 322/361 [1:16:57<09:20, 14.37s/it]
 89%|████████▉ | 323/361 [1:17:11<09:05, 14.36s/it]
                                                   
{'loss': 1.5304, 'learning_rate': 4.367816091954023e-05, 'epoch': 0.89}

 89%|████████▉ | 323/361 [1:17:11<09:05, 14.36s/it]
 90%|████████▉ | 324/361 [1:17:25<08:51, 14.37s/it]
                                                   
{'loss': 1.5876, 'learning_rate': 4.252873563218391e-05, 'epoch': 0.9}

 90%|████████▉ | 324/361 [1:17:25<08:51, 14.37s/it]
 90%|█████████ | 325/361 [1:17:40<08:36, 14.33s/it]
                                                   
{'loss': 1.6072, 'learning_rate': 4.137931034482758e-05, 'epoch': 0.9}

 90%|█████████ | 325/361 [1:17:40<08:36, 14.33s/it]
 90%|█████████ | 326/361 [1:17:54<08:21, 14.33s/it]
                                                   
{'loss': 1.5628, 'learning_rate': 4.022988505747126e-05, 'epoch': 0.9}

 90%|█████████ | 326/361 [1:17:54<08:21, 14.33s/it]
 91%|█████████ | 327/361 [1:18:08<08:07, 14.34s/it]
                                                   
{'loss': 1.7227, 'learning_rate': 3.9080459770114936e-05, 'epoch': 0.9}

 91%|█████████ | 327/361 [1:18:08<08:07, 14.34s/it]
 91%|█████████ | 328/361 [1:18:23<07:51, 14.28s/it]
                                                   
{'loss': 1.5332, 'learning_rate': 3.793103448275862e-05, 'epoch': 0.91}

 91%|█████████ | 328/361 [1:18:23<07:51, 14.28s/it]
 91%|█████████ | 329/361 [1:18:37<07:37, 14.28s/it]
                                                   
{'loss': 1.6123, 'learning_rate': 3.678160919540229e-05, 'epoch': 0.91}

 91%|█████████ | 329/361 [1:18:37<07:37, 14.28s/it]
 91%|█████████▏| 330/361 [1:18:51<07:20, 14.21s/it]
                                                   
{'loss': 1.6557, 'learning_rate': 3.563218390804597e-05, 'epoch': 0.91}

 91%|█████████▏| 330/361 [1:18:51<07:20, 14.21s/it]
 92%|█████████▏| 331/361 [1:19:05<07:02, 14.09s/it]
                                                   
{'loss': 1.6062, 'learning_rate': 3.448275862068965e-05, 'epoch': 0.92}

 92%|█████████▏| 331/361 [1:19:05<07:02, 14.09s/it]
 92%|█████████▏| 332/361 [1:19:19<06:50, 14.15s/it]
                                                   
{'loss': 1.6083, 'learning_rate': 3.333333333333333e-05, 'epoch': 0.92}

 92%|█████████▏| 332/361 [1:19:19<06:50, 14.15s/it]
 92%|█████████▏| 333/361 [1:19:33<06:37, 14.19s/it]
                                                   
{'loss': 1.599, 'learning_rate': 3.2183908045977006e-05, 'epoch': 0.92}

 92%|█████████▏| 333/361 [1:19:33<06:37, 14.19s/it]
 93%|█████████▎| 334/361 [1:19:48<06:25, 14.27s/it]
                                                   
{'loss': 1.5667, 'learning_rate': 3.1034482758620685e-05, 'epoch': 0.92}

 93%|█████████▎| 334/361 [1:19:48<06:25, 14.27s/it]
 93%|█████████▎| 335/361 [1:20:02<06:11, 14.29s/it]
                                                   
{'loss': 1.5218, 'learning_rate': 2.9885057471264363e-05, 'epoch': 0.93}

 93%|█████████▎| 335/361 [1:20:02<06:11, 14.29s/it]
 93%|█████████▎| 336/361 [1:20:16<05:56, 14.27s/it]
                                                   
{'loss': 1.5904, 'learning_rate': 2.8735632183908045e-05, 'epoch': 0.93}

 93%|█████████▎| 336/361 [1:20:16<05:56, 14.27s/it]
 93%|█████████▎| 337/361 [1:20:31<05:43, 14.32s/it]
                                                   
{'loss': 1.6499, 'learning_rate': 2.758620689655172e-05, 'epoch': 0.93}

 93%|█████████▎| 337/361 [1:20:31<05:43, 14.32s/it]
 94%|█████████▎| 338/361 [1:20:45<05:28, 14.29s/it]
                                                   
{'loss': 1.5828, 'learning_rate': 2.6436781609195398e-05, 'epoch': 0.93}

 94%|█████████▎| 338/361 [1:20:45<05:28, 14.29s/it]
 94%|█████████▍| 339/361 [1:20:59<05:14, 14.31s/it]
                                                   
{'loss': 1.6192, 'learning_rate': 2.528735632183908e-05, 'epoch': 0.94}

 94%|█████████▍| 339/361 [1:20:59<05:14, 14.31s/it]
 94%|█████████▍| 340/361 [1:21:14<05:01, 14.33s/it]
                                                   
{'loss': 1.6615, 'learning_rate': 2.4137931034482755e-05, 'epoch': 0.94}

 94%|█████████▍| 340/361 [1:21:14<05:01, 14.33s/it]
 94%|█████████▍| 341/361 [1:21:28<04:46, 14.34s/it]
                                                   
{'loss': 1.5419, 'learning_rate': 2.2988505747126433e-05, 'epoch': 0.94}

 94%|█████████▍| 341/361 [1:21:28<04:46, 14.34s/it]
 95%|█████████▍| 342/361 [1:21:42<04:32, 14.34s/it]
                                                   
{'loss': 1.5987, 'learning_rate': 2.1839080459770115e-05, 'epoch': 0.95}

 95%|█████████▍| 342/361 [1:21:42<04:32, 14.34s/it]
 95%|█████████▌| 343/361 [1:21:57<04:18, 14.35s/it]
                                                   
{'loss': 1.6338, 'learning_rate': 2.068965517241379e-05, 'epoch': 0.95}

 95%|█████████▌| 343/361 [1:21:57<04:18, 14.35s/it]
 95%|█████████▌| 344/361 [1:22:11<04:04, 14.36s/it]
                                                   
{'loss': 1.5162, 'learning_rate': 1.9540229885057468e-05, 'epoch': 0.95}

 95%|█████████▌| 344/361 [1:22:11<04:04, 14.36s/it]
 96%|█████████▌| 345/361 [1:22:25<03:49, 14.36s/it]
                                                   
{'loss': 1.6885, 'learning_rate': 1.8390804597701147e-05, 'epoch': 0.95}

 96%|█████████▌| 345/361 [1:22:25<03:49, 14.36s/it]
 96%|█████████▌| 346/361 [1:22:40<03:35, 14.38s/it]
                                                   
{'loss': 1.65, 'learning_rate': 1.7241379310344825e-05, 'epoch': 0.96}

 96%|█████████▌| 346/361 [1:22:40<03:35, 14.38s/it]
 96%|█████████▌| 347/361 [1:22:54<03:21, 14.38s/it]
                                                   
{'loss': 1.5733, 'learning_rate': 1.6091954022988503e-05, 'epoch': 0.96}

 96%|█████████▌| 347/361 [1:22:54<03:21, 14.38s/it]
 96%|█████████▋| 348/361 [1:23:08<03:06, 14.33s/it]
                                                   
{'loss': 1.6181, 'learning_rate': 1.4942528735632182e-05, 'epoch': 0.96}

 96%|█████████▋| 348/361 [1:23:08<03:06, 14.33s/it]
 97%|█████████▋| 349/361 [1:23:23<02:52, 14.34s/it]
                                                   
{'loss': 1.5732, 'learning_rate': 1.379310344827586e-05, 'epoch': 0.97}

 97%|█████████▋| 349/361 [1:23:23<02:52, 14.34s/it]
 97%|█████████▋| 350/361 [1:23:37<02:37, 14.33s/it]
                                                   
{'loss': 1.6196, 'learning_rate': 1.264367816091954e-05, 'epoch': 0.97}

 97%|█████████▋| 350/361 [1:23:37<02:37, 14.33s/it]
 97%|█████████▋| 351/361 [1:23:51<02:23, 14.33s/it]
                                                   
{'loss': 1.5964, 'learning_rate': 1.1494252873563217e-05, 'epoch': 0.97}

 97%|█████████▋| 351/361 [1:23:51<02:23, 14.33s/it]
 98%|█████████▊| 352/361 [1:24:06<02:09, 14.34s/it]
                                                   
{'loss': 1.5926, 'learning_rate': 1.0344827586206895e-05, 'epoch': 0.97}

 98%|█████████▊| 352/361 [1:24:06<02:09, 14.34s/it]
 98%|█████████▊| 353/361 [1:24:20<01:54, 14.29s/it]
                                                   
{'loss': 1.6532, 'learning_rate': 9.195402298850573e-06, 'epoch': 0.98}

 98%|█████████▊| 353/361 [1:24:20<01:54, 14.29s/it]
 98%|█████████▊| 354/361 [1:24:34<01:40, 14.31s/it]
                                                   
{'loss': 1.5908, 'learning_rate': 8.045977011494252e-06, 'epoch': 0.98}

 98%|█████████▊| 354/361 [1:24:34<01:40, 14.31s/it]
 98%|█████████▊| 355/361 [1:24:49<01:25, 14.32s/it]
                                                   
{'loss': 1.6316, 'learning_rate': 6.89655172413793e-06, 'epoch': 0.98}

 98%|█████████▊| 355/361 [1:24:49<01:25, 14.32s/it]
 99%|█████████▊| 356/361 [1:25:03<01:11, 14.33s/it]
                                                   
{'loss': 1.6225, 'learning_rate': 5.747126436781608e-06, 'epoch': 0.98}

 99%|█████████▊| 356/361 [1:25:03<01:11, 14.33s/it]
 99%|█████████▉| 357/361 [1:25:17<00:57, 14.35s/it]
                                                   
{'loss': 1.6497, 'learning_rate': 4.597701149425287e-06, 'epoch': 0.99}

 99%|█████████▉| 357/361 [1:25:17<00:57, 14.35s/it]
 99%|█████████▉| 358/361 [1:25:32<00:43, 14.36s/it]
                                                   
{'loss': 1.53, 'learning_rate': 3.448275862068965e-06, 'epoch': 0.99}

 99%|█████████▉| 358/361 [1:25:32<00:43, 14.36s/it]
 99%|█████████▉| 359/361 [1:25:46<00:28, 14.37s/it]
                                                   
{'loss': 1.5791, 'learning_rate': 2.2988505747126433e-06, 'epoch': 0.99}

 99%|█████████▉| 359/361 [1:25:46<00:28, 14.37s/it]
100%|█████████▉| 360/361 [1:26:01<00:14, 14.37s/it]
                                                   
{'loss': 1.6448, 'learning_rate': 1.1494252873563217e-06, 'epoch': 1.0}

100%|█████████▉| 360/361 [1:26:01<00:14, 14.37s/it]
100%|██████████| 361/361 [1:26:15<00:00, 14.32s/it]
                                                   
{'loss': 1.6354, 'learning_rate': 0.0, 'epoch': 1.0}

100%|██████████| 361/361 [1:26:15<00:00, 14.32s/it]
                                                   
{'train_runtime': 5193.7956, 'train_samples_per_second': 8.911, 'train_steps_per_second': 0.07, 'train_loss': 1.6677727507752396, 'epoch': 1.0}

100%|██████████| 361/361 [1:26:15<00:00, 14.32s/it]
100%|██████████| 361/361 [1:26:15<00:00, 14.34s/it]

 If there's a warning about missing keys above, please disregard :)
