WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
bin /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.soCUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6

CUDA SETUP: Detected CUDA version 121CUDA SETUP: Highest compute capability among GPUs detected: 8.6

CUDA SETUP: Loading binary /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...CUDA SETUP: Detected CUDA version 121

CUDA SETUP: Loading binary /nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...
No neptune configured, set NEPTUNE_API_TOKEN env var.

    Example runs on 4 GPUs:
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='decapoda-research/llama-7b-hf' --data_path=data/config.json --run_id=0 &> 0.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='decapoda-research/llama-30b-hf' --data_path=data/config.json --batch_size=16 --micro_batch_size=1 --run_id=1 --save_code=True &> 1.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='EleutherAI/gpt-j-6B' --data_path=data/config.json --run_id=2 &> 2.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='EleutherAI/gpt-neox-20b' --data_path=data/config.json --run_id=8 --batch_size=16 --micro_batch_size=4 &> 8.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='togethercomputer/GPT-NeoXT-Chat-Base-20B' --data_path=data/config.json --prompt_type='dai_faq' --run_id=13 --batch_size=16 --micro_batch_size=4 --num_epochs=100 --val_set_size=0 data_mix_in_path='' &> 13.log
    WORLD_SIZE=4 CUDA_VISIBLE_DEVICES="0,1,2,3" torchrun --nproc_per_node=4 finetune.py --base_model='togethercomputer/GPT-NeoXT-Chat-Base-20B' --data_path=data/config.json --run_id=28 --batch_size=16 --micro_batch_size=4 --num_epochs=8 --val_set_size=0 --data_mix_in_factor=0.1 --data_mix_in_prompt_type='human_bot' --save_code=True --cutoff_len=512  &> 28.log

    All metrics:
    CUDA_VISIBLE_DEVICES= finetune.py --data_mix_in_factor=0 --eval_steps=100 --warmup_steps=2 --val_set_size=100 --val_metrics="['bleu', 'rouge', 'sacrebleu', 'meteor']"

    # Fine-tune 20B on 24GB GPUs across 3 nodes with 3+2+2 GPUs
    rippa>
NCCL_P2P_LEVEL=LOC WORLD_SIZE=7 CUDA_VISIBLE_DEVICES="0,1,2" torchrun --node_rank 0 --nproc_per_node=3 --master_port=1234 --nnodes=3 --master_addr=10.10.10.2 finetune.py --data_path=merged_shuffled_OIG_87f6a1e788.json --micro_batch_size=1 --batch_size=7 --cutoff_len=512 --run_id=17 &>log.17.rank0
    ova>
NCCL_P2P_LEVEL=LOC WORLD_SIZE=7 CUDA_VISIBLE_DEVICES="0,1" torchrun --node_rank 1 --nproc_per_node=2 --master_port=1234 --nnodes=3 --master_addr=10.10.10.2 finetune.py --data_path=merged_shuffled_OIG_87f6a1e788.json --micro_batch_size=1 --batch_size=7 --cutoff_len=512 --run_id=17 &>log.17.rank1
    timemachine>
NCCL_P2P_LEVEL=LOC WORLD_SIZE=7 CUDA_VISIBLE_DEVICES="0,1" torchrun --node_rank 2 --nproc_per_node=2 --master_port=1234 --nnodes=3 --master_addr=10.10.10.2 finetune.py --data_path=merged_shuffled_OIG_87f6a1e788.json --micro_batch_size=1 --batch_size=7 --cutoff_len=512 --run_id=17 &>log.17.rank2

    
local_rank: 1
global rank: 1
local_rank: 0
global rank: 0
Training model with params:
save_code: False
run_id: 9
base_model: EleutherAI/pythia-6.9b
tokenizer_base_model: EleutherAI/pythia-6.9b
data_path: h2ogpt-oig-oasst1-instruct-cleaned-v1.json
data_col_dict: None
valid_path: None
data_mix_in_path: 0-hero/OIG-small-chip2
data_mix_in_factor: 0.0
data_mix_in_col_dict: {'user': 'instruction', 'chip2': 'output'}
data_mix_in_prompt_type: instruct
output_dir: pythia-6.9b.h2ogpt-oig-oasst1-instruct-cleaned-v1.json.1_epochs.5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a.9
lora_weights: 
batch_size: 128
micro_batch_size: 4
gradient_checkpointing: False
fp16: True
num_epochs: 1
learning_rate: 0.0003
val_set_size: None
val_metrics: []
eval_steps: None
eval_epochs: None
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules: None
llama_type: False
group_by_length: False
resume_from_checkpoint: None
ddp: True
local_files_only: False
resume_download: True
warmup_steps: 100
logging_steps: 1
save_steps: None
add_eos_token: False
world_size: 4
local_rank: 0
rank: 0
gpus: 4
device_map: auto
gradient_accumulation_steps: 32
cutoff_len: 256
prompt_type: plain
train_on_inputs: True
Command: finetune.py --base_model=EleutherAI/pythia-6.9b --data_path=h2ogpt-oig-oasst1-instruct-cleaned-v1.json --prompt_type=plain --run_id=9 --cutoff_len=256
Hash: 5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a
Distributed: data parallel

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:05<00:05,  5.19s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:07<00:00,  3.31s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:07<00:00,  3.59s/it]

Loading checkpoint shards:  50%|█████     | 1/2 [00:07<00:07,  7.23s/it]PeftModelForCausalLM(
  (base_model): LoraModel(
    (model): GPTNeoXForCausalLM(
      (gpt_neox): GPTNeoXModel(
        (embed_in): Embedding(50432, 4096)
        (layers): ModuleList(
          (0-31): 32 x GPTNeoXLayer(
            (input_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (post_attention_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (attention): GPTNeoXAttention(
              (rotary_emb): RotaryEmbedding()
              (query_key_value): MergedLinear8bitLt(
                in_features=4096, out_features=12288, bias=True
                (lora_dropout): Dropout(p=0.05, inplace=False)
                (lora_A): Linear(in_features=4096, out_features=16, bias=False)
                (lora_B): Conv1d(16, 8192, kernel_size=(1,), stride=(1,), groups=2, bias=False)
              )
              (dense): Linear8bitLt(in_features=4096, out_features=4096, bias=True)
            )
            (mlp): GPTNeoXMLP(
              (dense_h_to_4h): Linear8bitLt(in_features=4096, out_features=16384, bias=True)
              (dense_4h_to_h): Linear8bitLt(in_features=16384, out_features=4096, bias=True)
              (act): GELUActivation()
            )
          )
        )
        (final_layer_norm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
      )
      (embed_out): Linear(in_features=4096, out_features=50432, bias=False)
    )
  )
)
trainable params: 4194304 || all params: 6861496320 || trainable%: 0.06112812430977155
Using Validation Metrics: []
Supported Metrics: ['bleu', 'rouge', 'sacrebleu', 'meteor']
Auto set val_set_size 1000
Found cached dataset json (/home/arno/.cache/huggingface/datasets/json/default-3ad0987ceeef17aa/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 274.64it/s]

Map (num_proc=16):   0%|          | 0/349837 [00:00<?, ? examples/s]
Map (num_proc=16):   0%|          | 1/349837 [00:00<17:52:57,  5.43 examples/s]
Map (num_proc=16):   0%|          | 330/349837 [00:00<04:53, 1189.38 examples/s]
Map (num_proc=16):   0%|          | 755/349837 [00:00<02:36, 2227.35 examples/s]
Map (num_proc=16):   0%|          | 1327/349837 [00:00<01:43, 3363.89 examples/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:09<00:00,  4.41s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:09<00:00,  4.84s/it]

Map (num_proc=16):   1%|          | 2257/349837 [00:00<01:06, 5244.08 examples/s]
Map (num_proc=16):   1%|          | 3535/349837 [00:00<00:45, 7552.26 examples/s]
Map (num_proc=16):   2%|▏         | 5359/349837 [00:00<00:31, 10826.19 examples/s]
Map (num_proc=16):   2%|▏         | 7810/349837 [00:00<00:22, 14930.91 examples/s]
Map (num_proc=16):   3%|▎         | 10245/349837 [00:01<00:19, 17189.16 examples/s]PeftModelForCausalLM(
  (base_model): LoraModel(
    (model): GPTNeoXForCausalLM(
      (gpt_neox): GPTNeoXModel(
        (embed_in): Embedding(50432, 4096)
        (layers): ModuleList(
          (0-31): 32 x GPTNeoXLayer(
            (input_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (post_attention_layernorm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
            (attention): GPTNeoXAttention(
              (rotary_emb): RotaryEmbedding()
              (query_key_value): MergedLinear8bitLt(
                in_features=4096, out_features=12288, bias=True
                (lora_dropout): Dropout(p=0.05, inplace=False)
                (lora_A): Linear(in_features=4096, out_features=16, bias=False)
                (lora_B): Conv1d(16, 8192, kernel_size=(1,), stride=(1,), groups=2, bias=False)
              )
              (dense): Linear8bitLt(in_features=4096, out_features=4096, bias=True)
            )
            (mlp): GPTNeoXMLP(
              (dense_h_to_4h): Linear8bitLt(in_features=4096, out_features=16384, bias=True)
              (dense_4h_to_h): Linear8bitLt(in_features=16384, out_features=4096, bias=True)
              (act): GELUActivation()
            )
          )
        )
        (final_layer_norm): LayerNorm((4096,), eps=1e-05, elementwise_affine=True)
      )
      (embed_out): Linear(in_features=4096, out_features=50432, bias=False)
    )
  )
)
trainable params: 4194304 || all params: 6861496320 || trainable%: 0.06112812430977155

Map (num_proc=16):   4%|▎         | 12314/349837 [00:01<00:18, 18108.76 examples/s]
Map (num_proc=16):   4%|▍         | 14751/349837 [00:01<00:16, 19916.05 examples/s]
Map (num_proc=16):   5%|▍         | 17064/349837 [00:01<00:16, 20692.45 examples/s]
Map (num_proc=16):   6%|▌         | 19331/349837 [00:01<00:15, 21180.98 examples/s]
Map (num_proc=16):   6%|▌         | 21486/349837 [00:01<00:15, 21282.36 examples/s]Found cached dataset json (/home/arno/.cache/huggingface/datasets/json/default-3ad0987ceeef17aa/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 127.01it/s]

Map (num_proc=16):   7%|▋         | 23632/349837 [00:01<00:15, 21037.84 examples/s]
Map (num_proc=16):   7%|▋         | 25771/349837 [00:01<00:16, 19438.79 examples/s]
Map (num_proc=16):   8%|▊         | 27798/349837 [00:01<00:17, 18792.54 examples/s]
Map (num_proc=16):   9%|▊         | 29741/349837 [00:02<00:16, 18921.36 examples/s]
Map (num_proc=16):   9%|▉         | 31659/349837 [00:02<00:16, 18884.26 examples/s]
Map (num_proc=16):  10%|▉         | 33757/349837 [00:02<00:16, 19352.39 examples/s]
Map (num_proc=16):  10%|█         | 35873/349837 [00:02<00:16, 19608.12 examples/s]
Map (num_proc=16):   0%|          | 0/349837 [00:00<?, ? examples/s]
Map (num_proc=16):  11%|█         | 37905/349837 [00:02<00:15, 19691.42 examples/s]
Map (num_proc=16):  11%|█▏        | 39938/349837 [00:02<00:15, 19846.90 examples/s]
Map (num_proc=16):   0%|          | 25/349837 [00:00<53:42, 108.54 examples/s]
Map (num_proc=16):  12%|█▏        | 41929/349837 [00:02<00:16, 18823.51 examples/s]
Map (num_proc=16):   0%|          | 346/349837 [00:00<05:08, 1132.30 examples/s]
Map (num_proc=16):  13%|█▎        | 43864/349837 [00:02<00:17, 17273.48 examples/s]
Map (num_proc=16):   0%|          | 794/349837 [00:00<02:45, 2102.70 examples/s]
Map (num_proc=16):  13%|█▎        | 45655/349837 [00:02<00:19, 15473.23 examples/s]
Map (num_proc=16):   0%|          | 1356/349837 [00:00<02:01, 2876.01 examples/s]
Map (num_proc=16):  14%|█▎        | 47257/349837 [00:03<00:21, 14161.17 examples/s]
Map (num_proc=16):   1%|          | 2285/349837 [00:00<01:24, 4132.29 examples/s]
Map (num_proc=16):  14%|█▍        | 48736/349837 [00:03<00:23, 12881.52 examples/s]
Map (num_proc=16):   1%|          | 3586/349837 [00:00<00:54, 6402.40 examples/s]
Map (num_proc=16):  14%|█▍        | 50082/349837 [00:03<00:24, 12276.46 examples/s]
Map (num_proc=16):   1%|▏         | 4420/349837 [00:00<00:50, 6853.43 examples/s]
Map (num_proc=16):  15%|█▍        | 51350/349837 [00:03<00:25, 11690.96 examples/s]
Map (num_proc=16):   2%|▏         | 5398/349837 [00:01<00:45, 7653.42 examples/s]
Map (num_proc=16):   2%|▏         | 6432/349837 [00:01<00:40, 8382.35 examples/s]
Map (num_proc=16):  15%|█▌        | 52544/349837 [00:03<00:26, 11432.12 examples/s]
Map (num_proc=16):   2%|▏         | 7566/349837 [00:01<00:37, 9040.47 examples/s]
Map (num_proc=16):  15%|█▌        | 53708/349837 [00:03<00:25, 11390.30 examples/s]
Map (num_proc=16):   2%|▏         | 8661/349837 [00:01<00:35, 9583.14 examples/s]
Map (num_proc=16):  16%|█▌        | 54863/349837 [00:03<00:26, 11228.27 examples/s]
Map (num_proc=16):   3%|▎         | 9711/349837 [00:01<00:35, 9638.63 examples/s]
Map (num_proc=16):  16%|█▌        | 56011/349837 [00:03<00:26, 11193.52 examples/s]
Map (num_proc=16):   3%|▎         | 10779/349837 [00:01<00:34, 9897.62 examples/s]
Map (num_proc=16):  16%|█▋        | 57164/349837 [00:03<00:26, 11159.30 examples/s]
Map (num_proc=16):   3%|▎         | 11807/349837 [00:01<00:34, 9673.07 examples/s]
Map (num_proc=16):  17%|█▋        | 58281/349837 [00:04<00:26, 10846.62 examples/s]
Map (num_proc=16):  17%|█▋        | 59369/349837 [00:04<00:26, 10817.43 examples/s]
Map (num_proc=16):   4%|▎         | 12797/349837 [00:01<00:35, 9446.83 examples/s]
Map (num_proc=16):   4%|▍         | 13766/349837 [00:01<00:36, 9254.54 examples/s]
Map (num_proc=16):  17%|█▋        | 60461/349837 [00:04<00:28, 10268.20 examples/s]
Map (num_proc=16):   4%|▍         | 14724/349837 [00:02<00:37, 8928.89 examples/s]
Map (num_proc=16):  18%|█▊        | 61524/349837 [00:04<00:30, 9397.56 examples/s] 
Map (num_proc=16):   4%|▍         | 15709/349837 [00:02<00:36, 9077.52 examples/s]
Map (num_proc=16):  18%|█▊        | 62510/349837 [00:04<00:31, 9201.42 examples/s]
Map (num_proc=16):   5%|▍         | 16651/349837 [00:02<00:40, 8325.86 examples/s]
Map (num_proc=16):  18%|█▊        | 63457/349837 [00:04<00:31, 9217.90 examples/s]
Map (num_proc=16):   5%|▌         | 17509/349837 [00:02<00:40, 8306.87 examples/s]
Map (num_proc=16):  18%|█▊        | 64431/349837 [00:04<00:30, 9276.39 examples/s]
Map (num_proc=16):   5%|▌         | 18421/349837 [00:02<00:38, 8507.57 examples/s]
Map (num_proc=16):  19%|█▊        | 65369/349837 [00:04<00:32, 8677.90 examples/s]
Map (num_proc=16):   6%|▌         | 19297/349837 [00:02<00:38, 8508.87 examples/s]
Map (num_proc=16):  19%|█▉        | 66289/349837 [00:05<00:32, 8637.98 examples/s]
Map (num_proc=16):   6%|▌         | 20259/349837 [00:02<00:37, 8715.46 examples/s]
Map (num_proc=16):  19%|█▉        | 67187/349837 [00:05<00:32, 8711.98 examples/s]
Map (num_proc=16):   6%|▌         | 21374/349837 [00:02<00:35, 9314.72 examples/s]
Map (num_proc=16):  19%|█▉        | 68113/349837 [00:05<00:31, 8866.26 examples/s]
Map (num_proc=16):   6%|▋         | 22338/349837 [00:02<00:34, 9396.91 examples/s]
Map (num_proc=16):  20%|█▉        | 69024/349837 [00:05<00:32, 8746.86 examples/s]
Map (num_proc=16):   7%|▋         | 23311/349837 [00:03<00:34, 9381.91 examples/s]
Map (num_proc=16):  20%|██        | 70018/349837 [00:05<00:30, 9069.17 examples/s]
Map (num_proc=16):   7%|▋         | 24473/349837 [00:03<00:32, 10030.98 examples/s]
Map (num_proc=16):  20%|██        | 71116/349837 [00:05<00:28, 9611.97 examples/s]
Map (num_proc=16):   7%|▋         | 25480/349837 [00:03<00:32, 10027.24 examples/s]
Map (num_proc=16):  21%|██        | 72269/349837 [00:05<00:27, 10165.27 examples/s]
Map (num_proc=16):   8%|▊         | 26526/349837 [00:03<00:32, 9886.68 examples/s] 
Map (num_proc=16):  21%|██        | 73303/349837 [00:05<00:27, 9955.95 examples/s] 
Map (num_proc=16):   8%|▊         | 27549/349837 [00:03<00:32, 9942.42 examples/s]
Map (num_proc=16):  21%|██        | 74311/349837 [00:05<00:27, 9903.79 examples/s]
Map (num_proc=16):   8%|▊         | 28584/349837 [00:03<00:31, 10057.69 examples/s]
Map (num_proc=16):  22%|██▏       | 75306/349837 [00:05<00:29, 9394.08 examples/s]
Map (num_proc=16):   8%|▊         | 29611/349837 [00:03<00:32, 9760.79 examples/s] 
Map (num_proc=16):  22%|██▏       | 76268/349837 [00:06<00:30, 8979.62 examples/s]
Map (num_proc=16):   9%|▉         | 30627/349837 [00:03<00:35, 8977.18 examples/s]
Map (num_proc=16):  22%|██▏       | 77177/349837 [00:06<00:31, 8770.53 examples/s]
Map (num_proc=16):   9%|▉         | 31566/349837 [00:03<00:35, 9086.76 examples/s]
Map (num_proc=16):  22%|██▏       | 78100/349837 [00:06<00:30, 8805.04 examples/s]
Map (num_proc=16):   9%|▉         | 32512/349837 [00:03<00:37, 8539.34 examples/s]
Map (num_proc=16):  23%|██▎       | 78997/349837 [00:06<00:31, 8687.86 examples/s]
Map (num_proc=16):  10%|▉         | 33381/349837 [00:04<00:37, 8422.99 examples/s]
Map (num_proc=16):  23%|██▎       | 79893/349837 [00:06<00:31, 8554.35 examples/s]
Map (num_proc=16):  10%|▉         | 34316/349837 [00:04<00:36, 8649.93 examples/s]
Map (num_proc=16):  23%|██▎       | 80783/349837 [00:06<00:31, 8580.35 examples/s]
Map (num_proc=16):  10%|█         | 35205/349837 [00:04<00:36, 8685.11 examples/s]
Map (num_proc=16):  23%|██▎       | 81915/349837 [00:06<00:28, 9302.50 examples/s]
Map (num_proc=16):  10%|█         | 36268/349837 [00:04<00:34, 9201.41 examples/s]
Map (num_proc=16):  24%|██▎       | 83056/349837 [00:06<00:27, 9827.99 examples/s]
Map (num_proc=16):  11%|█         | 37197/349837 [00:04<00:34, 9131.77 examples/s]
Map (num_proc=16):  24%|██▍       | 84059/349837 [00:06<00:27, 9688.61 examples/s]
Map (num_proc=16):  11%|█         | 38132/349837 [00:04<00:34, 9073.17 examples/s]
Map (num_proc=16):  24%|██▍       | 85073/349837 [00:07<00:26, 9810.89 examples/s]
Map (num_proc=16):  11%|█         | 39154/349837 [00:04<00:33, 9404.04 examples/s]
Map (num_proc=16):  25%|██▍       | 86110/349837 [00:07<00:26, 9969.87 examples/s]
Map (num_proc=16):  11%|█▏        | 40216/349837 [00:04<00:32, 9665.15 examples/s]
Map (num_proc=16):  25%|██▍       | 87111/349837 [00:07<00:26, 9969.18 examples/s]
Map (num_proc=16):  12%|█▏        | 41255/349837 [00:04<00:31, 9842.97 examples/s]
Map (num_proc=16):  25%|██▌       | 88153/349837 [00:07<00:25, 10090.03 examples/s]
Map (num_proc=16):  12%|█▏        | 42281/349837 [00:05<00:31, 9806.28 examples/s]
Map (num_proc=16):  25%|██▌       | 89180/349837 [00:07<00:25, 10134.89 examples/s]
Map (num_proc=16):  12%|█▏        | 43300/349837 [00:05<00:31, 9859.17 examples/s]
Map (num_proc=16):  26%|██▌       | 90217/349837 [00:07<00:26, 9864.04 examples/s] 
Map (num_proc=16):  13%|█▎        | 44289/349837 [00:05<00:32, 9283.99 examples/s]
Map (num_proc=16):  26%|██▌       | 91212/349837 [00:07<00:29, 8902.57 examples/s]
Map (num_proc=16):  13%|█▎        | 45232/349837 [00:05<00:35, 8611.85 examples/s]
Map (num_proc=16):  26%|██▋       | 92129/349837 [00:07<00:30, 8355.71 examples/s]
Map (num_proc=16):  13%|█▎        | 46170/349837 [00:05<00:34, 8722.22 examples/s]
Map (num_proc=16):  27%|██▋       | 93062/349837 [00:07<00:29, 8611.71 examples/s]
Map (num_proc=16):  13%|█▎        | 47143/349837 [00:05<00:34, 8900.99 examples/s]
Map (num_proc=16):  27%|██▋       | 94092/349837 [00:07<00:28, 9076.21 examples/s]
Map (num_proc=16):  14%|█▎        | 48055/349837 [00:05<00:33, 8949.72 examples/s]
Map (num_proc=16):  27%|██▋       | 95103/349837 [00:08<00:27, 9200.09 examples/s]
Map (num_proc=16):  14%|█▍        | 48984/349837 [00:05<00:34, 8812.68 examples/s]
Map (num_proc=16):  27%|██▋       | 96051/349837 [00:08<00:27, 9195.74 examples/s]
Map (num_proc=16):  14%|█▍        | 49962/349837 [00:05<00:33, 8997.19 examples/s]
Map (num_proc=16):  28%|██▊       | 97025/349837 [00:08<00:27, 9332.95 examples/s]
Map (num_proc=16):  15%|█▍        | 50881/349837 [00:06<00:34, 8760.57 examples/s]
Map (num_proc=16):  28%|██▊       | 98050/349837 [00:08<00:26, 9523.08 examples/s]
Map (num_proc=16):  15%|█▍        | 51842/349837 [00:06<00:33, 8975.91 examples/s]
Map (num_proc=16):  28%|██▊       | 99041/349837 [00:08<00:26, 9566.38 examples/s]
Map (num_proc=16):  15%|█▌        | 52836/349837 [00:06<00:32, 9053.08 examples/s]
Map (num_proc=16):  29%|██▊       | 100085/349837 [00:08<00:25, 9807.18 examples/s]
Map (num_proc=16):  15%|█▌        | 53757/349837 [00:06<00:33, 8816.87 examples/s]
Map (num_proc=16):  29%|██▉       | 101088/349837 [00:08<00:25, 9803.25 examples/s]
Map (num_proc=16):  16%|█▌        | 54720/349837 [00:06<00:32, 8976.43 examples/s]
Map (num_proc=16):  29%|██▉       | 102093/349837 [00:08<00:26, 9527.69 examples/s]
Map (num_proc=16):  16%|█▌        | 55733/349837 [00:06<00:31, 9286.20 examples/s]
Map (num_proc=16):  29%|██▉       | 103200/349837 [00:08<00:24, 9937.11 examples/s]
Map (num_proc=16):  16%|█▌        | 56766/349837 [00:06<00:30, 9475.60 examples/s]
Map (num_proc=16):  30%|██▉       | 104369/349837 [00:09<00:23, 10359.01 examples/s]
Map (num_proc=16):  17%|█▋        | 57839/349837 [00:06<00:29, 9766.75 examples/s]
Map (num_proc=16):  30%|███       | 105451/349837 [00:09<00:23, 10415.53 examples/s]
Map (num_proc=16):  17%|█▋        | 59005/349837 [00:06<00:28, 10207.29 examples/s]
Map (num_proc=16):  30%|███       | 106501/349837 [00:09<00:24, 10036.32 examples/s]
Map (num_proc=16):  17%|█▋        | 60032/349837 [00:06<00:29, 9752.92 examples/s] 
Map (num_proc=16):  31%|███       | 107516/349837 [00:09<00:27, 8899.90 examples/s] 
Map (num_proc=16):  17%|█▋        | 61054/349837 [00:07<00:30, 9383.40 examples/s]
Map (num_proc=16):  31%|███       | 108453/349837 [00:09<00:28, 8588.27 examples/s]
Map (num_proc=16):  18%|█▊        | 62022/349837 [00:07<00:32, 8817.03 examples/s]
Map (num_proc=16):  31%|███▏      | 109329/349837 [00:09<00:27, 8615.79 examples/s]
Map (num_proc=16):  18%|█▊        | 62965/349837 [00:07<00:32, 8933.27 examples/s]
Map (num_proc=16):  32%|███▏      | 110304/349837 [00:09<00:26, 8901.05 examples/s]
Map (num_proc=16):  18%|█▊        | 63870/349837 [00:07<00:32, 8725.88 examples/s]
Map (num_proc=16):  32%|███▏      | 111350/349837 [00:09<00:25, 9332.02 examples/s]
Map (num_proc=16):  19%|█▊        | 64756/349837 [00:07<00:33, 8601.28 examples/s]
Map (num_proc=16):  32%|███▏      | 112308/349837 [00:09<00:25, 9387.88 examples/s]
Map (num_proc=16):  19%|█▉        | 65699/349837 [00:07<00:32, 8740.92 examples/s]
Map (num_proc=16):  32%|███▏      | 113387/349837 [00:10<00:24, 9549.45 examples/s]
Map (num_proc=16):  19%|█▉        | 66595/349837 [00:07<00:32, 8608.78 examples/s]
Map (num_proc=16):  33%|███▎      | 114371/349837 [00:10<00:24, 9630.16 examples/s]
Map (num_proc=16):  19%|█▉        | 67495/349837 [00:07<00:32, 8651.76 examples/s]
Map (num_proc=16):  33%|███▎      | 115353/349837 [00:10<00:25, 9334.33 examples/s]
Map (num_proc=16):  20%|█▉        | 68380/349837 [00:07<00:32, 8612.27 examples/s]
Map (num_proc=16):  33%|███▎      | 116367/349837 [00:10<00:25, 9329.83 examples/s]
Map (num_proc=16):  20%|█▉        | 69312/349837 [00:08<00:32, 8698.94 examples/s]
Map (num_proc=16):  34%|███▎      | 117352/349837 [00:10<00:25, 9287.19 examples/s]
Map (num_proc=16):  20%|██        | 70291/349837 [00:08<00:31, 9010.28 examples/s]
Map (num_proc=16):  34%|███▍      | 118368/349837 [00:10<00:24, 9473.86 examples/s]
Map (num_proc=16):  20%|██        | 71355/349837 [00:08<00:29, 9299.71 examples/s]
Map (num_proc=16):  34%|███▍      | 119411/349837 [00:10<00:23, 9627.50 examples/s]
Map (num_proc=16):  21%|██        | 72519/349837 [00:08<00:27, 9905.57 examples/s]
Map (num_proc=16):  34%|███▍      | 120562/349837 [00:10<00:22, 10087.56 examples/s]
Map (num_proc=16):  21%|██        | 73630/349837 [00:08<00:27, 10002.86 examples/s]
Map (num_proc=16):  35%|███▍      | 121604/349837 [00:10<00:23, 9786.54 examples/s] 
Map (num_proc=16):  21%|██▏       | 74673/349837 [00:08<00:27, 9974.35 examples/s] 
Map (num_proc=16):  35%|███▌      | 122594/349837 [00:10<00:24, 9256.31 examples/s]
Map (num_proc=16):  22%|██▏       | 75672/349837 [00:08<00:28, 9790.88 examples/s]
Map (num_proc=16):  35%|███▌      | 123548/349837 [00:11<00:25, 9007.53 examples/s]
Map (num_proc=16):  22%|██▏       | 76688/349837 [00:08<00:29, 9264.08 examples/s]
Map (num_proc=16):  36%|███▌      | 124492/349837 [00:11<00:25, 8974.45 examples/s]
Map (num_proc=16):  22%|██▏       | 77631/349837 [00:08<00:29, 9190.85 examples/s]
Map (num_proc=16):  36%|███▌      | 125402/349837 [00:11<00:25, 8961.15 examples/s]
Map (num_proc=16):  22%|██▏       | 78581/349837 [00:09<00:31, 8721.20 examples/s]
Map (num_proc=16):  36%|███▌      | 126301/349837 [00:11<00:25, 8920.31 examples/s]
Map (num_proc=16):  23%|██▎       | 79593/349837 [00:09<00:29, 9066.48 examples/s]
Map (num_proc=16):  36%|███▋      | 127244/349837 [00:11<00:24, 9001.58 examples/s]
Map (num_proc=16):  23%|██▎       | 80643/349837 [00:09<00:28, 9435.64 examples/s]
Map (num_proc=16):  37%|███▋      | 128227/349837 [00:11<00:24, 9199.51 examples/s]
Map (num_proc=16):  23%|██▎       | 81596/349837 [00:09<00:28, 9294.42 examples/s]
Map (num_proc=16):  37%|███▋      | 129373/349837 [00:11<00:22, 9803.22 examples/s]
Map (num_proc=16):  24%|██▎       | 82540/349837 [00:09<00:28, 9287.71 examples/s]
Map (num_proc=16):  37%|███▋      | 130376/349837 [00:11<00:22, 9833.77 examples/s]
Map (num_proc=16):  24%|██▍       | 83482/349837 [00:09<00:29, 9040.64 examples/s]
Map (num_proc=16):  38%|███▊      | 131412/349837 [00:11<00:23, 9354.28 examples/s]
Map (num_proc=16):  24%|██▍       | 84424/349837 [00:09<00:29, 9117.69 examples/s]
Map (num_proc=16):  38%|███▊      | 132433/349837 [00:12<00:22, 9518.36 examples/s]
Map (num_proc=16):  24%|██▍       | 85358/349837 [00:09<00:30, 8676.58 examples/s]
Map (num_proc=16):  38%|███▊      | 133422/349837 [00:12<00:22, 9496.75 examples/s]
Map (num_proc=16):  38%|███▊      | 134448/349837 [00:12<00:22, 9706.63 examples/s]
Map (num_proc=16):  25%|██▍       | 86260/349837 [00:09<00:30, 8618.94 examples/s]
Map (num_proc=16):  39%|███▊      | 135431/349837 [00:12<00:22, 9707.84 examples/s]
Map (num_proc=16):  25%|██▍       | 87386/349837 [00:09<00:28, 9289.39 examples/s]
Map (num_proc=16):  39%|███▉      | 136423/349837 [00:12<00:22, 9656.82 examples/s]
Map (num_proc=16):  25%|██▌       | 88442/349837 [00:10<00:27, 9643.93 examples/s]
Map (num_proc=16):  26%|██▌       | 89439/349837 [00:10<00:26, 9724.99 examples/s]
Map (num_proc=16):  39%|███▉      | 137449/349837 [00:12<00:22, 9647.97 examples/s]
Map (num_proc=16):  26%|██▌       | 90453/349837 [00:10<00:26, 9736.28 examples/s]
Map (num_proc=16):  40%|███▉      | 138453/349837 [00:12<00:21, 9690.32 examples/s]
Map (num_proc=16):  26%|██▌       | 91441/349837 [00:10<00:26, 9596.37 examples/s]
Map (num_proc=16):  40%|███▉      | 139447/349837 [00:12<00:23, 9093.28 examples/s]
Map (num_proc=16):  26%|██▋       | 92419/349837 [00:10<00:28, 8999.66 examples/s]
Map (num_proc=16):  40%|████      | 140390/349837 [00:12<00:23, 8840.49 examples/s]
Map (num_proc=16):  27%|██▋       | 93369/349837 [00:10<00:28, 9113.65 examples/s]
Map (num_proc=16):  40%|████      | 141319/349837 [00:13<00:24, 8358.19 examples/s]
Map (num_proc=16):  27%|██▋       | 94294/349837 [00:10<00:28, 8940.84 examples/s]
Map (num_proc=16):  41%|████      | 142255/349837 [00:13<00:24, 8581.40 examples/s]
Map (num_proc=16):  27%|██▋       | 95244/349837 [00:10<00:28, 9068.52 examples/s]
Map (num_proc=16):  41%|████      | 143239/349837 [00:13<00:23, 8914.92 examples/s]
Map (num_proc=16):  27%|██▋       | 96191/349837 [00:10<00:28, 9011.87 examples/s]
Map (num_proc=16):  41%|████      | 144171/349837 [00:13<00:22, 9024.87 examples/s]
Map (num_proc=16):  28%|██▊       | 97131/349837 [00:11<00:28, 8812.21 examples/s]
Map (num_proc=16):  41%|████▏     | 145137/349837 [00:13<00:22, 9141.34 examples/s]
Map (num_proc=16):  28%|██▊       | 98031/349837 [00:11<00:28, 8757.81 examples/s]
Map (num_proc=16):  42%|████▏     | 146078/349837 [00:13<00:23, 8701.74 examples/s]
Map (num_proc=16):  28%|██▊       | 98917/349837 [00:11<00:28, 8679.73 examples/s]
Map (num_proc=16):  42%|████▏     | 147033/349837 [00:13<00:22, 8828.20 examples/s]
Map (num_proc=16):  29%|██▊       | 99800/349837 [00:11<00:28, 8662.97 examples/s]
Map (num_proc=16):  42%|████▏     | 148166/349837 [00:13<00:21, 9469.22 examples/s]
Map (num_proc=16):  29%|██▉       | 100689/349837 [00:11<00:29, 8493.83 examples/s]
Map (num_proc=16):  43%|████▎     | 149252/349837 [00:13<00:20, 9754.16 examples/s]
Map (num_proc=16):  29%|██▉       | 101605/349837 [00:11<00:28, 8638.29 examples/s]
Map (num_proc=16):  43%|████▎     | 150258/349837 [00:13<00:20, 9735.39 examples/s]
Map (num_proc=16):  29%|██▉       | 102486/349837 [00:11<00:28, 8613.96 examples/s]
Map (num_proc=16):  43%|████▎     | 151275/349837 [00:14<00:20, 9824.73 examples/s]
Map (num_proc=16):  30%|██▉       | 103527/349837 [00:11<00:26, 9135.89 examples/s]
Map (num_proc=16):  44%|████▎     | 152333/349837 [00:14<00:19, 9998.20 examples/s]
Map (num_proc=16):  30%|██▉       | 104576/349837 [00:11<00:25, 9481.60 examples/s]
Map (num_proc=16):  44%|████▍     | 153361/349837 [00:14<00:19, 10057.92 examples/s]
Map (num_proc=16):  30%|███       | 105588/349837 [00:11<00:25, 9662.80 examples/s]
Map (num_proc=16):  44%|████▍     | 154469/349837 [00:14<00:19, 10268.06 examples/s]
Map (num_proc=16):  30%|███       | 106589/349837 [00:12<00:26, 9330.67 examples/s]
Map (num_proc=16):  44%|████▍     | 155534/349837 [00:14<00:20, 9509.67 examples/s] 
Map (num_proc=16):  31%|███       | 107532/349837 [00:12<00:26, 9067.32 examples/s]
Map (num_proc=16):  45%|████▍     | 156507/349837 [00:14<00:20, 9221.18 examples/s]
Map (num_proc=16):  31%|███       | 108455/349837 [00:12<00:27, 8810.59 examples/s]
Map (num_proc=16):  45%|████▌     | 157443/349837 [00:14<00:21, 8852.54 examples/s]
Map (num_proc=16):  31%|███▏      | 109424/349837 [00:12<00:26, 9040.08 examples/s]
Map (num_proc=16):  45%|████▌     | 158438/349837 [00:14<00:21, 9013.76 examples/s]
Map (num_proc=16):  32%|███▏      | 110397/349837 [00:12<00:26, 9181.24 examples/s]
Map (num_proc=16):  46%|████▌     | 159495/349837 [00:14<00:20, 9424.23 examples/s]
Map (num_proc=16):  32%|███▏      | 111332/349837 [00:12<00:25, 9216.54 examples/s]
Map (num_proc=16):  46%|████▌     | 160571/349837 [00:15<00:19, 9718.02 examples/s]
Map (num_proc=16):  32%|███▏      | 112269/349837 [00:12<00:26, 8926.84 examples/s]
Map (num_proc=16):  46%|████▌     | 161582/349837 [00:15<00:19, 9784.90 examples/s]
Map (num_proc=16):  32%|███▏      | 113167/349837 [00:12<00:26, 8832.41 examples/s]
Map (num_proc=16):  46%|████▋     | 162579/349837 [00:15<00:19, 9639.32 examples/s]
Map (num_proc=16):  33%|███▎      | 114087/349837 [00:12<00:27, 8485.81 examples/s]
Map (num_proc=16):  47%|████▋     | 163558/349837 [00:15<00:19, 9625.22 examples/s]
Map (num_proc=16):  33%|███▎      | 114964/349837 [00:13<00:27, 8531.97 examples/s]
Map (num_proc=16):  47%|████▋     | 164533/349837 [00:15<00:19, 9656.21 examples/s]
Map (num_proc=16):  33%|███▎      | 115956/349837 [00:13<00:26, 8886.55 examples/s]
Map (num_proc=16):  47%|████▋     | 165615/349837 [00:15<00:18, 9927.36 examples/s]
Map (num_proc=16):  33%|███▎      | 116987/349837 [00:13<00:25, 9271.09 examples/s]
Map (num_proc=16):  48%|████▊     | 166631/349837 [00:15<00:18, 9856.51 examples/s]
Map (num_proc=16):  34%|███▎      | 117934/349837 [00:13<00:25, 9225.72 examples/s]
Map (num_proc=16):  48%|████▊     | 167644/349837 [00:15<00:18, 9887.50 examples/s]
Map (num_proc=16):  34%|███▍      | 118894/349837 [00:13<00:25, 9091.41 examples/s]
Map (num_proc=16):  48%|████▊     | 168725/349837 [00:15<00:17, 10144.58 examples/s]
Map (num_proc=16):  34%|███▍      | 119893/349837 [00:13<00:24, 9328.22 examples/s]
Map (num_proc=16):  49%|████▊     | 169745/349837 [00:15<00:17, 10157.65 examples/s]
Map (num_proc=16):  35%|███▍      | 120958/349837 [00:13<00:23, 9675.78 examples/s]
Map (num_proc=16):  49%|████▉     | 170782/349837 [00:16<00:18, 9511.74 examples/s] 
Map (num_proc=16):  35%|███▍      | 121939/349837 [00:13<00:24, 9475.98 examples/s]
Map (num_proc=16):  49%|████▉     | 171751/349837 [00:16<00:19, 8946.21 examples/s]
Map (num_proc=16):  35%|███▌      | 122894/349837 [00:13<00:25, 8944.20 examples/s]
Map (num_proc=16):  49%|████▉     | 172672/349837 [00:16<00:20, 8718.02 examples/s]
Map (num_proc=16):  35%|███▌      | 123796/349837 [00:14<00:26, 8686.46 examples/s]
Map (num_proc=16):  50%|████▉     | 173617/349837 [00:16<00:19, 8857.41 examples/s]
Map (num_proc=16):  36%|███▌      | 124685/349837 [00:14<00:26, 8614.15 examples/s]
Map (num_proc=16):  50%|████▉     | 174605/349837 [00:16<00:19, 8979.80 examples/s]
Map (num_proc=16):  36%|███▌      | 125645/349837 [00:14<00:25, 8867.91 examples/s]
Map (num_proc=16):  50%|█████     | 175548/349837 [00:16<00:19, 9078.48 examples/s]
Map (num_proc=16):  36%|███▌      | 126715/349837 [00:14<00:24, 9280.46 examples/s]
Map (num_proc=16):  50%|█████     | 176462/349837 [00:16<00:19, 8902.74 examples/s]
Map (num_proc=16):  37%|███▋      | 127748/349837 [00:14<00:23, 9488.60 examples/s]
Map (num_proc=16):  51%|█████     | 177533/349837 [00:16<00:18, 9214.26 examples/s]
Map (num_proc=16):  37%|███▋      | 128779/349837 [00:14<00:23, 9517.47 examples/s]
Map (num_proc=16):  51%|█████     | 178460/349837 [00:16<00:18, 9091.60 examples/s]
Map (num_proc=16):  37%|███▋      | 129742/349837 [00:14<00:25, 8782.29 examples/s]
Map (num_proc=16):  51%|█████▏    | 179394/349837 [00:17<00:19, 8708.68 examples/s]
Map (num_proc=16):  37%|███▋      | 130635/349837 [00:14<00:26, 8243.76 examples/s]
Map (num_proc=16):  52%|█████▏    | 180291/349837 [00:17<00:20, 8249.08 examples/s]
Map (num_proc=16):  38%|███▊      | 131478/349837 [00:14<00:27, 7913.76 examples/s]
Map (num_proc=16):  52%|█████▏    | 181132/349837 [00:17<00:20, 8227.05 examples/s]
Map (num_proc=16):  38%|███▊      | 132293/349837 [00:15<00:27, 7818.92 examples/s]
Map (num_proc=16):  52%|█████▏    | 181974/349837 [00:17<00:20, 8229.22 examples/s]
Map (num_proc=16):  38%|███▊      | 133081/349837 [00:15<00:28, 7590.84 examples/s]
Map (num_proc=16):  52%|█████▏    | 182820/349837 [00:17<00:21, 7752.49 examples/s]
Map (num_proc=16):  38%|███▊      | 133846/349837 [00:15<00:28, 7501.35 examples/s]
Map (num_proc=16):  52%|█████▏    | 183628/349837 [00:17<00:21, 7760.87 examples/s]
Map (num_proc=16):  38%|███▊      | 134675/349837 [00:15<00:28, 7638.40 examples/s]
Map (num_proc=16):  53%|█████▎    | 184426/349837 [00:17<00:21, 7724.71 examples/s]
Map (num_proc=16):  39%|███▊      | 135533/349837 [00:15<00:27, 7888.81 examples/s]
Map (num_proc=16):  53%|█████▎    | 185219/349837 [00:17<00:21, 7649.68 examples/s]
Map (num_proc=16):  39%|███▉      | 136530/349837 [00:15<00:25, 8412.72 examples/s]
Map (num_proc=16):  53%|█████▎    | 186315/349837 [00:17<00:19, 8544.32 examples/s]
Map (num_proc=16):  39%|███▉      | 137496/349837 [00:15<00:24, 8733.23 examples/s]
Map (num_proc=16):  54%|█████▎    | 187267/349837 [00:18<00:18, 8775.27 examples/s]
Map (num_proc=16):  40%|███▉      | 138386/349837 [00:15<00:24, 8732.84 examples/s]
Map (num_proc=16):  54%|█████▍    | 188156/349837 [00:18<00:19, 8406.93 examples/s]
Map (num_proc=16):  40%|███▉      | 139272/349837 [00:15<00:25, 8317.79 examples/s]
Map (num_proc=16):  54%|█████▍    | 189019/349837 [00:18<00:20, 7992.05 examples/s]
Map (num_proc=16):  40%|████      | 140285/349837 [00:15<00:24, 8628.09 examples/s]
Map (num_proc=16):  54%|█████▍    | 189974/349837 [00:18<00:19, 8355.36 examples/s]
Map (num_proc=16):  40%|████      | 141176/349837 [00:16<00:24, 8679.83 examples/s]
Map (num_proc=16):  55%|█████▍    | 191029/349837 [00:18<00:17, 8884.58 examples/s]
Map (num_proc=16):  41%|████      | 142249/349837 [00:16<00:22, 9212.81 examples/s]
Map (num_proc=16):  55%|█████▍    | 192039/349837 [00:18<00:17, 9161.18 examples/s]
Map (num_proc=16):  41%|████      | 143228/349837 [00:16<00:22, 9243.42 examples/s]
Map (num_proc=16):  55%|█████▌    | 193105/349837 [00:18<00:16, 9566.65 examples/s]
Map (num_proc=16):  41%|████      | 144254/349837 [00:16<00:21, 9528.67 examples/s]
Map (num_proc=16):  55%|█████▌    | 194138/349837 [00:18<00:16, 9548.68 examples/s]
Map (num_proc=16):  42%|████▏     | 145217/349837 [00:16<00:22, 9254.89 examples/s]
Map (num_proc=16):  56%|█████▌    | 195132/349837 [00:18<00:16, 9409.79 examples/s]
Map (num_proc=16):  42%|████▏     | 146165/349837 [00:16<00:22, 9190.26 examples/s]
Map (num_proc=16):  56%|█████▌    | 196206/349837 [00:19<00:15, 9682.05 examples/s]
Map (num_proc=16):  42%|████▏     | 147115/349837 [00:16<00:21, 9271.76 examples/s]
Map (num_proc=16):  56%|█████▋    | 197190/349837 [00:19<00:15, 9577.99 examples/s]
Map (num_proc=16):  42%|████▏     | 148051/349837 [00:16<00:22, 9011.05 examples/s]
Map (num_proc=16):  57%|█████▋    | 198155/349837 [00:19<00:16, 9440.36 examples/s]
Map (num_proc=16):  43%|████▎     | 148999/349837 [00:16<00:21, 9132.46 examples/s]
Map (num_proc=16):  57%|█████▋    | 199115/349837 [00:19<00:16, 9376.27 examples/s]
Map (num_proc=16):  43%|████▎     | 149942/349837 [00:17<00:21, 9146.58 examples/s]
Map (num_proc=16):  57%|█████▋    | 200146/349837 [00:19<00:15, 9631.65 examples/s]
Map (num_proc=16):  43%|████▎     | 150907/349837 [00:17<00:21, 9281.46 examples/s]
Map (num_proc=16):  58%|█████▊    | 201173/349837 [00:19<00:15, 9701.62 examples/s]
Map (num_proc=16):  43%|████▎     | 151901/349837 [00:17<00:20, 9474.36 examples/s]
Map (num_proc=16):  58%|█████▊    | 202317/349837 [00:19<00:14, 10182.53 examples/s]
Map (num_proc=16):  44%|████▎     | 152867/349837 [00:17<00:21, 9348.07 examples/s]
Map (num_proc=16):  58%|█████▊    | 203343/349837 [00:19<00:15, 9548.94 examples/s] 
Map (num_proc=16):  44%|████▍     | 153806/349837 [00:17<00:21, 8996.36 examples/s]
Map (num_proc=16):  58%|█████▊    | 204327/349837 [00:19<00:16, 9080.79 examples/s]
Map (num_proc=16):  44%|████▍     | 154757/349837 [00:17<00:21, 9105.61 examples/s]
Map (num_proc=16):  59%|█████▊    | 205302/349837 [00:19<00:15, 9192.54 examples/s]
Map (num_proc=16):  45%|████▍     | 155761/349837 [00:17<00:20, 9342.27 examples/s]
Map (num_proc=16):  59%|█████▉    | 206247/349837 [00:20<00:15, 9131.50 examples/s]
Map (num_proc=16):  45%|████▍     | 156730/349837 [00:17<00:20, 9234.40 examples/s]
Map (num_proc=16):  59%|█████▉    | 207253/349837 [00:20<00:15, 9346.71 examples/s]
Map (num_proc=16):  45%|████▌     | 157674/349837 [00:17<00:20, 9258.76 examples/s]
Map (num_proc=16):  60%|█████▉    | 208228/349837 [00:20<00:14, 9449.33 examples/s]
Map (num_proc=16):  45%|████▌     | 158674/349837 [00:17<00:20, 9404.03 examples/s]
Map (num_proc=16):  60%|█████▉    | 209304/349837 [00:20<00:14, 9678.82 examples/s]
Map (num_proc=16):  46%|████▌     | 159641/349837 [00:18<00:20, 9390.74 examples/s]
Map (num_proc=16):  60%|██████    | 210314/349837 [00:20<00:14, 9615.71 examples/s]
Map (num_proc=16):  46%|████▌     | 160612/349837 [00:18<00:20, 9353.68 examples/s]
Map (num_proc=16):  60%|██████    | 211281/349837 [00:20<00:14, 9564.22 examples/s]
Map (num_proc=16):  46%|████▌     | 161554/349837 [00:18<00:21, 8636.75 examples/s]
Map (num_proc=16):  61%|██████    | 212240/349837 [00:20<00:15, 9124.49 examples/s]
Map (num_proc=16):  46%|████▋     | 162532/349837 [00:18<00:21, 8811.47 examples/s]
Map (num_proc=16):  61%|██████    | 213182/349837 [00:20<00:14, 9141.39 examples/s]
Map (num_proc=16):  47%|████▋     | 163454/349837 [00:18<00:21, 8745.53 examples/s]
Map (num_proc=16):  61%|██████    | 214208/349837 [00:20<00:14, 9334.78 examples/s]
Map (num_proc=16):  47%|████▋     | 164553/349837 [00:18<00:19, 9334.31 examples/s]
Map (num_proc=16):  62%|██████▏   | 215253/349837 [00:21<00:14, 9452.96 examples/s]
Map (num_proc=16):  47%|████▋     | 165536/349837 [00:18<00:19, 9408.09 examples/s]
Map (num_proc=16):  62%|██████▏   | 216288/349837 [00:21<00:13, 9691.93 examples/s]
Map (num_proc=16):  48%|████▊     | 166523/349837 [00:18<00:19, 9532.70 examples/s]
Map (num_proc=16):  62%|██████▏   | 217412/349837 [00:21<00:13, 9893.53 examples/s]
Map (num_proc=16):  48%|████▊     | 167506/349837 [00:18<00:19, 9382.97 examples/s]
Map (num_proc=16):  62%|██████▏   | 218471/349837 [00:21<00:13, 9958.56 examples/s]
Map (num_proc=16):  48%|████▊     | 168464/349837 [00:19<00:19, 9368.50 examples/s]
Map (num_proc=16):  63%|██████▎   | 219506/349837 [00:21<00:13, 9874.26 examples/s]
Map (num_proc=16):  48%|████▊     | 169429/349837 [00:19<00:19, 9139.58 examples/s]
Map (num_proc=16):  63%|██████▎   | 220522/349837 [00:21<00:13, 9328.51 examples/s]
Map (num_proc=16):  49%|████▊     | 170363/349837 [00:19<00:19, 9040.34 examples/s]
Map (num_proc=16):  63%|██████▎   | 221462/349837 [00:21<00:13, 9244.36 examples/s]
Map (num_proc=16):  49%|████▉     | 171433/349837 [00:19<00:19, 9379.07 examples/s]
Map (num_proc=16):  64%|██████▎   | 222399/349837 [00:21<00:13, 9256.98 examples/s]
Map (num_proc=16):  49%|████▉     | 172426/349837 [00:19<00:18, 9392.28 examples/s]
Map (num_proc=16):  64%|██████▍   | 223328/349837 [00:21<00:13, 9122.78 examples/s]
Map (num_proc=16):  50%|████▉     | 173401/349837 [00:19<00:18, 9334.63 examples/s]
Map (num_proc=16):  64%|██████▍   | 224261/349837 [00:22<00:13, 9081.00 examples/s]
Map (num_proc=16):  50%|████▉     | 174418/349837 [00:19<00:18, 9539.96 examples/s]
Map (num_proc=16):  64%|██████▍   | 225201/349837 [00:22<00:13, 9152.74 examples/s]
Map (num_proc=16):  50%|█████     | 175484/349837 [00:19<00:17, 9834.92 examples/s]
Map (num_proc=16):  65%|██████▍   | 226279/349837 [00:22<00:12, 9520.84 examples/s]
Map (num_proc=16):  50%|█████     | 176492/349837 [00:19<00:18, 9577.64 examples/s]
Map (num_proc=16):  65%|██████▍   | 227309/349837 [00:22<00:12, 9704.18 examples/s]
Map (num_proc=16):  51%|█████     | 177477/349837 [00:19<00:18, 9401.34 examples/s]
Map (num_proc=16):  65%|██████▌   | 228290/349837 [00:22<00:12, 9721.46 examples/s]
Map (num_proc=16):  51%|█████     | 178453/349837 [00:20<00:19, 8718.93 examples/s]
Map (num_proc=16):  66%|██████▌   | 229348/349837 [00:22<00:12, 9842.82 examples/s]
Map (num_proc=16):  51%|█████▏    | 179432/349837 [00:20<00:18, 8990.87 examples/s]
Map (num_proc=16):  66%|██████▌   | 230337/349837 [00:22<00:12, 9854.19 examples/s]
Map (num_proc=16):  52%|█████▏    | 180406/349837 [00:20<00:18, 9200.43 examples/s]
Map (num_proc=16):  66%|██████▌   | 231347/349837 [00:22<00:11, 9909.11 examples/s]
Map (num_proc=16):  52%|█████▏    | 181363/349837 [00:20<00:18, 9206.11 examples/s]
Map (num_proc=16):  66%|██████▋   | 232377/349837 [00:22<00:11, 9819.77 examples/s]
Map (num_proc=16):  52%|█████▏    | 182377/349837 [00:20<00:17, 9451.52 examples/s]
Map (num_proc=16):  67%|██████▋   | 233390/349837 [00:22<00:12, 9444.88 examples/s]
Map (num_proc=16):  52%|█████▏    | 183413/349837 [00:20<00:17, 9686.25 examples/s]
Map (num_proc=16):  67%|██████▋   | 234369/349837 [00:23<00:12, 9352.42 examples/s]
Map (num_proc=16):  53%|█████▎    | 184397/349837 [00:20<00:17, 9548.00 examples/s]
Map (num_proc=16):  67%|██████▋   | 235365/349837 [00:23<00:12, 9494.54 examples/s]
Map (num_proc=16):  53%|█████▎    | 185369/349837 [00:20<00:18, 9111.34 examples/s]
Map (num_proc=16):  68%|██████▊   | 236347/349837 [00:23<00:12, 9108.07 examples/s]
Map (num_proc=16):  53%|█████▎    | 186294/349837 [00:20<00:18, 8988.52 examples/s]
Map (num_proc=16):  68%|██████▊   | 237301/349837 [00:23<00:12, 9133.47 examples/s]
Map (num_proc=16):  54%|█████▎    | 187258/349837 [00:21<00:17, 9036.27 examples/s]
Map (num_proc=16):  68%|██████▊   | 238233/349837 [00:23<00:12, 8950.97 examples/s]
Map (num_proc=16):  54%|█████▍    | 188238/349837 [00:21<00:17, 9086.86 examples/s]
Map (num_proc=16):  68%|██████▊   | 239139/349837 [00:23<00:12, 8861.78 examples/s]
Map (num_proc=16):  54%|█████▍    | 189190/349837 [00:21<00:17, 9154.35 examples/s]
Map (num_proc=16):  69%|██████▊   | 240089/349837 [00:23<00:12, 9029.22 examples/s]
Map (num_proc=16):  54%|█████▍    | 190122/349837 [00:21<00:18, 8776.19 examples/s]
Map (num_proc=16):  69%|██████▉   | 241020/349837 [00:23<00:12, 9063.64 examples/s]
Map (num_proc=16):  55%|█████▍    | 191028/349837 [00:21<00:18, 8803.83 examples/s]
Map (num_proc=16):  69%|██████▉   | 241946/349837 [00:23<00:11, 9058.60 examples/s]
Map (num_proc=16):  55%|█████▍    | 191981/349837 [00:21<00:17, 8989.32 examples/s]
Map (num_proc=16):  69%|██████▉   | 242956/349837 [00:23<00:11, 9302.39 examples/s]
Map (num_proc=16):  55%|█████▌    | 192899/349837 [00:21<00:17, 8756.63 examples/s]
Map (num_proc=16):  70%|██████▉   | 243933/349837 [00:24<00:11, 9428.44 examples/s]
Map (num_proc=16):  55%|█████▌    | 193851/349837 [00:21<00:17, 8949.57 examples/s]
Map (num_proc=16):  70%|██████▉   | 244880/349837 [00:24<00:11, 9173.77 examples/s]
Map (num_proc=16):  56%|█████▌    | 194789/349837 [00:21<00:17, 8703.26 examples/s]
Map (num_proc=16):  70%|███████   | 245964/349837 [00:24<00:10, 9529.50 examples/s]
Map (num_proc=16):  56%|█████▌    | 195671/349837 [00:22<00:18, 8563.47 examples/s]
Map (num_proc=16):  71%|███████   | 246987/349837 [00:24<00:10, 9717.08 examples/s]
Map (num_proc=16):  56%|█████▌    | 196634/349837 [00:22<00:17, 8826.55 examples/s]
Map (num_proc=16):  71%|███████   | 248043/349837 [00:24<00:10, 9540.89 examples/s]
Map (num_proc=16):  56%|█████▋    | 197637/349837 [00:22<00:16, 9125.08 examples/s]
Map (num_proc=16):  71%|███████   | 249018/349837 [00:24<00:10, 9446.38 examples/s]
Map (num_proc=16):  57%|█████▋    | 198608/349837 [00:22<00:16, 9222.31 examples/s]
Map (num_proc=16):  71%|███████▏  | 250023/349837 [00:24<00:10, 9562.84 examples/s]
Map (num_proc=16):  57%|█████▋    | 199602/349837 [00:22<00:16, 9331.58 examples/s]
Map (num_proc=16):  72%|███████▏  | 251007/349837 [00:24<00:10, 9482.86 examples/s]
Map (num_proc=16):  57%|█████▋    | 200537/349837 [00:22<00:16, 8923.46 examples/s]
Map (num_proc=16):  72%|███████▏  | 251984/349837 [00:24<00:10, 9071.22 examples/s]
Map (num_proc=16):  58%|█████▊    | 201466/349837 [00:22<00:16, 8834.37 examples/s]
Map (num_proc=16):  72%|███████▏  | 252902/349837 [00:25<00:10, 8869.98 examples/s]
Map (num_proc=16):  58%|█████▊    | 202482/349837 [00:22<00:16, 9169.38 examples/s]
Map (num_proc=16):  73%|███████▎  | 253890/349837 [00:25<00:10, 9112.24 examples/s]
Map (num_proc=16):  58%|█████▊    | 203453/349837 [00:22<00:15, 9162.08 examples/s]
Map (num_proc=16):  73%|███████▎  | 254886/349837 [00:25<00:10, 9303.21 examples/s]
Map (num_proc=16):  58%|█████▊    | 204489/349837 [00:22<00:15, 9307.76 examples/s]
Map (num_proc=16):  73%|███████▎  | 255904/349837 [00:25<00:09, 9550.04 examples/s]
Map (num_proc=16):  59%|█████▊    | 205479/349837 [00:23<00:15, 9452.24 examples/s]
Map (num_proc=16):  73%|███████▎  | 256869/349837 [00:25<00:09, 9560.97 examples/s]
Map (num_proc=16):  59%|█████▉    | 206549/349837 [00:23<00:14, 9741.35 examples/s]
Map (num_proc=16):  74%|███████▎  | 257856/349837 [00:25<00:09, 9525.99 examples/s]
Map (num_proc=16):  59%|█████▉    | 207547/349837 [00:23<00:14, 9560.85 examples/s]
Map (num_proc=16):  74%|███████▍  | 258818/349837 [00:25<00:09, 9350.48 examples/s]
Map (num_proc=16):  60%|█████▉    | 208536/349837 [00:23<00:14, 9527.91 examples/s]
Map (num_proc=16):  74%|███████▍  | 259923/349837 [00:25<00:09, 9665.34 examples/s]
Map (num_proc=16):  60%|█████▉    | 209522/349837 [00:23<00:15, 9071.96 examples/s]
Map (num_proc=16):  75%|███████▍  | 261070/349837 [00:25<00:08, 10128.47 examples/s]
Map (num_proc=16):  60%|██████    | 210440/349837 [00:23<00:15, 8987.34 examples/s]
Map (num_proc=16):  75%|███████▍  | 262103/349837 [00:25<00:08, 9936.99 examples/s] 
Map (num_proc=16):  60%|██████    | 211353/349837 [00:23<00:15, 8927.91 examples/s]
Map (num_proc=16):  75%|███████▌  | 263115/349837 [00:26<00:09, 9568.53 examples/s]
Map (num_proc=16):  61%|██████    | 212250/349837 [00:23<00:15, 8874.74 examples/s]
Map (num_proc=16):  75%|███████▌  | 264084/349837 [00:26<00:09, 9000.43 examples/s]
Map (num_proc=16):  61%|██████    | 213145/349837 [00:23<00:15, 8845.33 examples/s]
Map (num_proc=16):  76%|███████▌  | 265012/349837 [00:26<00:10, 8234.67 examples/s]
Map (num_proc=16):  61%|██████    | 214039/349837 [00:24<00:15, 8609.42 examples/s]
Map (num_proc=16):  76%|███████▌  | 265858/349837 [00:26<00:10, 8089.66 examples/s]
Map (num_proc=16):  61%|██████▏   | 214941/349837 [00:24<00:15, 8684.52 examples/s]
Map (num_proc=16):  62%|██████▏   | 215869/349837 [00:24<00:15, 8811.29 examples/s]
Map (num_proc=16):  76%|███████▌  | 266680/349837 [00:26<00:11, 7447.93 examples/s]
Map (num_proc=16):  62%|██████▏   | 216757/349837 [00:24<00:15, 8541.26 examples/s]
Map (num_proc=16):  76%|███████▋  | 267564/349837 [00:26<00:10, 7717.75 examples/s]
Map (num_proc=16):  62%|██████▏   | 217617/349837 [00:24<00:15, 8556.50 examples/s]
Map (num_proc=16):  77%|███████▋  | 268579/349837 [00:26<00:09, 8291.03 examples/s]
Map (num_proc=16):  62%|██████▏   | 218611/349837 [00:24<00:14, 8812.24 examples/s]
Map (num_proc=16):  77%|███████▋  | 269553/349837 [00:26<00:09, 8651.22 examples/s]
Map (num_proc=16):  63%|██████▎   | 219660/349837 [00:24<00:14, 9220.80 examples/s]
Map (num_proc=16):  77%|███████▋  | 270577/349837 [00:27<00:08, 9058.52 examples/s]
Map (num_proc=16):  63%|██████▎   | 220684/349837 [00:24<00:13, 9499.62 examples/s]
Map (num_proc=16):  78%|███████▊  | 271535/349837 [00:27<00:08, 9140.52 examples/s]
Map (num_proc=16):  63%|██████▎   | 221740/349837 [00:24<00:13, 9767.31 examples/s]
Map (num_proc=16):  78%|███████▊  | 272478/349837 [00:27<00:08, 9194.04 examples/s]
Map (num_proc=16):  64%|██████▎   | 222719/349837 [00:24<00:13, 9597.24 examples/s]
Map (num_proc=16):  78%|███████▊  | 273413/349837 [00:27<00:08, 9132.96 examples/s]
Map (num_proc=16):  78%|███████▊  | 274337/349837 [00:27<00:08, 9084.35 examples/s]
Map (num_proc=16):  64%|██████▍   | 223706/349837 [00:25<00:13, 9179.80 examples/s]
Map (num_proc=16):  64%|██████▍   | 224680/349837 [00:25<00:13, 9185.61 examples/s]
Map (num_proc=16):  79%|███████▊  | 275269/349837 [00:27<00:09, 8182.61 examples/s]
Map (num_proc=16):  64%|██████▍   | 225624/349837 [00:25<00:13, 9158.16 examples/s]
Map (num_proc=16):  79%|███████▉  | 276127/349837 [00:27<00:08, 8200.20 examples/s]
Map (num_proc=16):  65%|██████▍   | 226549/349837 [00:25<00:13, 9097.01 examples/s]
Map (num_proc=16):  79%|███████▉  | 276996/349837 [00:27<00:09, 8034.70 examples/s]
Map (num_proc=16):  65%|██████▌   | 227567/349837 [00:25<00:13, 9392.59 examples/s]
Map (num_proc=16):  79%|███████▉  | 277820/349837 [00:27<00:08, 8056.92 examples/s]
Map (num_proc=16):  65%|██████▌   | 228555/349837 [00:25<00:12, 9508.31 examples/s]
Map (num_proc=16):  80%|███████▉  | 278659/349837 [00:27<00:08, 8073.28 examples/s]
Map (num_proc=16):  66%|██████▌   | 229553/349837 [00:25<00:12, 9469.74 examples/s]
Map (num_proc=16):  80%|███████▉  | 279530/349837 [00:28<00:08, 8228.86 examples/s]
Map (num_proc=16):  66%|██████▌   | 230516/349837 [00:25<00:12, 9504.84 examples/s]
Map (num_proc=16):  80%|████████  | 280415/349837 [00:28<00:08, 8362.31 examples/s]
Map (num_proc=16):  66%|██████▌   | 231572/349837 [00:25<00:12, 9804.48 examples/s]
Map (num_proc=16):  80%|████████  | 281255/349837 [00:28<00:08, 8011.76 examples/s]
Map (num_proc=16):  66%|██████▋   | 232571/349837 [00:26<00:13, 8604.31 examples/s]
Map (num_proc=16):  81%|████████  | 282074/349837 [00:28<00:08, 7557.82 examples/s]
Map (num_proc=16):  67%|██████▋   | 233474/349837 [00:26<00:13, 8659.38 examples/s]
Map (num_proc=16):  81%|████████  | 282862/349837 [00:28<00:09, 7396.75 examples/s]
Map (num_proc=16):  67%|██████▋   | 234388/349837 [00:26<00:13, 8640.67 examples/s]
Map (num_proc=16):  81%|████████  | 283659/349837 [00:28<00:09, 7306.06 examples/s]
Map (num_proc=16):  67%|██████▋   | 235278/349837 [00:26<00:13, 8634.30 examples/s]
Map (num_proc=16):  81%|████████▏ | 284485/349837 [00:28<00:08, 7448.43 examples/s]
Map (num_proc=16):  68%|██████▊   | 236204/349837 [00:26<00:12, 8795.82 examples/s]
Map (num_proc=16):  82%|████████▏ | 285245/349837 [00:28<00:08, 7333.37 examples/s]
Map (num_proc=16):  68%|██████▊   | 237240/349837 [00:26<00:12, 9209.40 examples/s]
Map (num_proc=16):  82%|████████▏ | 286128/349837 [00:28<00:08, 7710.46 examples/s]
Map (num_proc=16):  68%|██████▊   | 238334/349837 [00:26<00:11, 9697.00 examples/s]
Map (num_proc=16):  82%|████████▏ | 287017/349837 [00:29<00:07, 8006.12 examples/s]
Map (num_proc=16):  68%|██████▊   | 239311/349837 [00:26<00:12, 9147.10 examples/s]
Map (num_proc=16):  82%|████████▏ | 287940/349837 [00:29<00:07, 8352.63 examples/s]
Map (num_proc=16):  69%|██████▊   | 240240/349837 [00:26<00:12, 9128.09 examples/s]
Map (num_proc=16):  83%|████████▎ | 288915/349837 [00:29<00:07, 8696.49 examples/s]
Map (num_proc=16):  69%|██████▉   | 241234/349837 [00:26<00:11, 9309.09 examples/s]
Map (num_proc=16):  83%|████████▎ | 289805/349837 [00:29<00:07, 8529.23 examples/s]
Map (num_proc=16):  69%|██████▉   | 242196/349837 [00:27<00:12, 8894.13 examples/s]
Map (num_proc=16):  83%|████████▎ | 290843/349837 [00:29<00:06, 9036.68 examples/s]
Map (num_proc=16):  69%|██████▉   | 243115/349837 [00:27<00:12, 8892.65 examples/s]
Map (num_proc=16):  83%|████████▎ | 291766/349837 [00:29<00:06, 8859.16 examples/s]
Map (num_proc=16):  70%|██████▉   | 244016/349837 [00:27<00:12, 8774.80 examples/s]
Map (num_proc=16):  84%|████████▎ | 292668/349837 [00:29<00:06, 8727.99 examples/s]
Map (num_proc=16):  70%|███████   | 244915/349837 [00:27<00:11, 8796.62 examples/s]
Map (num_proc=16):  84%|████████▍ | 293564/349837 [00:29<00:06, 8575.31 examples/s]
Map (num_proc=16):  70%|███████   | 245931/349837 [00:27<00:11, 9176.43 examples/s]
Map (num_proc=16):  84%|████████▍ | 294501/349837 [00:29<00:06, 8694.88 examples/s]
Map (num_proc=16):  71%|███████   | 246898/349837 [00:27<00:11, 9318.02 examples/s]
Map (num_proc=16):  84%|████████▍ | 295424/349837 [00:30<00:06, 8828.49 examples/s]
Map (num_proc=16):  71%|███████   | 247843/349837 [00:27<00:11, 9104.26 examples/s]
Map (num_proc=16):  85%|████████▍ | 296325/349837 [00:30<00:06, 8787.89 examples/s]
Map (num_proc=16):  71%|███████   | 248762/349837 [00:27<00:11, 8829.72 examples/s]
Map (num_proc=16):  85%|████████▍ | 297240/349837 [00:30<00:06, 8393.81 examples/s]
Map (num_proc=16):  71%|███████▏  | 249700/349837 [00:27<00:11, 8927.71 examples/s]
Map (num_proc=16):  85%|████████▌ | 298116/349837 [00:30<00:06, 8435.55 examples/s]
Map (num_proc=16):  72%|███████▏  | 250656/349837 [00:28<00:10, 9053.79 examples/s]
Map (num_proc=16):  85%|████████▌ | 298976/349837 [00:30<00:06, 7879.93 examples/s]
Map (num_proc=16):  72%|███████▏  | 251704/349837 [00:28<00:10, 9390.29 examples/s]
Map (num_proc=16):  86%|████████▌ | 299779/349837 [00:30<00:06, 7840.66 examples/s]
Map (num_proc=16):  72%|███████▏  | 252803/349837 [00:28<00:09, 9815.13 examples/s]
Map (num_proc=16):  86%|████████▌ | 300657/349837 [00:30<00:06, 7932.21 examples/s]
Map (num_proc=16):  73%|███████▎  | 253801/349837 [00:28<00:09, 9744.61 examples/s]
Map (num_proc=16):  86%|████████▌ | 301484/349837 [00:30<00:06, 7760.54 examples/s]
Map (num_proc=16):  73%|███████▎  | 254882/349837 [00:28<00:09, 10018.07 examples/s]
Map (num_proc=16):  86%|████████▋ | 302307/349837 [00:30<00:06, 7874.79 examples/s]
Map (num_proc=16):  73%|███████▎  | 255993/349837 [00:28<00:09, 10248.50 examples/s]
Map (num_proc=16):  87%|████████▋ | 303098/349837 [00:31<00:05, 7866.16 examples/s]
Map (num_proc=16):  73%|███████▎  | 257043/349837 [00:28<00:09, 10254.06 examples/s]
Map (num_proc=16):  87%|████████▋ | 303906/349837 [00:31<00:05, 7867.84 examples/s]
Map (num_proc=16):  74%|███████▍  | 258101/349837 [00:28<00:09, 9412.84 examples/s] 
Map (num_proc=16):  87%|████████▋ | 304707/349837 [00:31<00:06, 7518.75 examples/s]
Map (num_proc=16):  74%|███████▍  | 259174/349837 [00:28<00:09, 9743.22 examples/s]
Map (num_proc=16):  87%|████████▋ | 305504/349837 [00:31<00:06, 7314.16 examples/s]
Map (num_proc=16):  74%|███████▍  | 260293/349837 [00:28<00:08, 10143.72 examples/s]
Map (num_proc=16):  88%|████████▊ | 306246/349837 [00:31<00:06, 6980.07 examples/s]
Map (num_proc=16):  75%|███████▍  | 261332/349837 [00:29<00:08, 9935.65 examples/s] 
Map (num_proc=16):  88%|████████▊ | 306959/349837 [00:31<00:06, 6871.87 examples/s]
Map (num_proc=16):  75%|███████▍  | 262349/349837 [00:29<00:09, 9337.10 examples/s]
Map (num_proc=16):  88%|████████▊ | 307879/349837 [00:31<00:05, 7502.05 examples/s]
Map (num_proc=16):  75%|███████▌  | 263331/349837 [00:29<00:09, 9297.56 examples/s]
Map (num_proc=16):  88%|████████▊ | 308855/349837 [00:31<00:05, 8135.80 examples/s]
Map (num_proc=16):  76%|███████▌  | 264270/349837 [00:29<00:09, 9123.14 examples/s]
Map (num_proc=16):  89%|████████▊ | 309683/349837 [00:31<00:05, 7992.81 examples/s]
Map (num_proc=16):  76%|███████▌  | 265350/349837 [00:29<00:08, 9580.11 examples/s]
Map (num_proc=16):  89%|████████▉ | 310497/349837 [00:31<00:05, 7820.66 examples/s]
Map (num_proc=16):  76%|███████▌  | 266445/349837 [00:29<00:08, 9941.29 examples/s]
Map (num_proc=16):  89%|████████▉ | 311285/349837 [00:32<00:05, 7639.43 examples/s]
Map (num_proc=16):  76%|███████▋  | 267528/349837 [00:29<00:08, 10138.83 examples/s]
Map (num_proc=16):  89%|████████▉ | 312097/349837 [00:32<00:04, 7693.48 examples/s]
Map (num_proc=16):  77%|███████▋  | 268608/349837 [00:29<00:07, 10320.69 examples/s]
Map (num_proc=16):  89%|████████▉ | 312965/349837 [00:32<00:04, 7946.48 examples/s]
Map (num_proc=16):  77%|███████▋  | 269650/349837 [00:29<00:07, 10349.16 examples/s]
Map (num_proc=16):  90%|████████▉ | 313779/349837 [00:32<00:04, 7752.33 examples/s]
Map (num_proc=16):  77%|███████▋  | 270697/349837 [00:30<00:07, 10008.62 examples/s]
Map (num_proc=16):  90%|████████▉ | 314602/349837 [00:32<00:04, 7865.42 examples/s]
Map (num_proc=16):  78%|███████▊  | 271728/349837 [00:30<00:08, 9544.55 examples/s] 
Map (num_proc=16):  90%|█████████ | 315397/349837 [00:32<00:04, 7846.53 examples/s]
Map (num_proc=16):  78%|███████▊  | 272696/349837 [00:30<00:08, 9081.06 examples/s]
Map (num_proc=16):  90%|█████████ | 316326/349837 [00:32<00:04, 8237.98 examples/s]
Map (num_proc=16):  78%|███████▊  | 273620/349837 [00:30<00:08, 8810.95 examples/s]
Map (num_proc=16):  91%|█████████ | 317312/349837 [00:32<00:03, 8687.66 examples/s]
Map (num_proc=16):  78%|███████▊  | 274525/349837 [00:30<00:09, 8255.35 examples/s]
Map (num_proc=16):  91%|█████████ | 318202/349837 [00:32<00:03, 8427.64 examples/s]
Map (num_proc=16):  91%|█████████ | 319059/349837 [00:33<00:03, 8097.50 examples/s]
Map (num_proc=16):  79%|███████▊  | 275370/349837 [00:30<00:10, 7256.15 examples/s]
Map (num_proc=16):  91%|█████████▏| 319884/349837 [00:33<00:03, 7512.25 examples/s]
Map (num_proc=16):  79%|███████▉  | 276135/349837 [00:30<00:10, 6983.65 examples/s]
Map (num_proc=16):  92%|█████████▏| 320679/349837 [00:33<00:03, 7560.35 examples/s]
Map (num_proc=16):  79%|███████▉  | 276872/349837 [00:30<00:10, 6742.28 examples/s]
Map (num_proc=16):  92%|█████████▏| 321581/349837 [00:33<00:03, 7938.58 examples/s]
Map (num_proc=16):  79%|███████▉  | 277627/349837 [00:31<00:10, 6799.73 examples/s]
Map (num_proc=16):  92%|█████████▏| 322447/349837 [00:33<00:03, 7997.83 examples/s]
Map (num_proc=16):  80%|███████▉  | 278451/349837 [00:31<00:10, 7063.35 examples/s]
Map (num_proc=16):  92%|█████████▏| 323286/349837 [00:33<00:03, 7729.17 examples/s]
Map (num_proc=16):  80%|███████▉  | 279180/349837 [00:31<00:10, 7030.41 examples/s]
Map (num_proc=16):  93%|█████████▎| 324074/349837 [00:33<00:03, 7359.41 examples/s]
Map (num_proc=16):  80%|████████  | 279911/349837 [00:31<00:09, 7029.18 examples/s]
Map (num_proc=16):  80%|████████  | 280673/349837 [00:31<00:09, 7108.66 examples/s]
Map (num_proc=16):  93%|█████████▎| 324833/349837 [00:33<00:03, 7279.81 examples/s]
Map (num_proc=16):  93%|█████████▎| 325719/349837 [00:33<00:03, 7692.97 examples/s]
Map (num_proc=16):  80%|████████  | 281400/349837 [00:31<00:09, 7061.91 examples/s]
Map (num_proc=16):  93%|█████████▎| 326522/349837 [00:34<00:03, 7746.62 examples/s]
Map (num_proc=16):  81%|████████  | 282129/349837 [00:31<00:09, 7069.37 examples/s]
Map (num_proc=16):  94%|█████████▎| 327378/349837 [00:34<00:02, 7979.33 examples/s]
Map (num_proc=16):  81%|████████  | 282886/349837 [00:31<00:09, 6826.20 examples/s]
Map (num_proc=16):  94%|█████████▍| 328215/349837 [00:34<00:02, 7750.49 examples/s]
Map (num_proc=16):  81%|████████  | 283576/349837 [00:31<00:10, 6445.57 examples/s]
Map (num_proc=16):  94%|█████████▍| 329032/349837 [00:34<00:02, 7799.70 examples/s]
Map (num_proc=16):  81%|████████▏ | 284341/349837 [00:32<00:09, 6654.01 examples/s]
Map (num_proc=16):  94%|█████████▍| 329992/349837 [00:34<00:02, 8169.63 examples/s]
Map (num_proc=16):  81%|████████▏ | 285045/349837 [00:32<00:09, 6724.77 examples/s]
Map (num_proc=16):  95%|█████████▍| 330833/349837 [00:34<00:02, 7960.42 examples/s]
Map (num_proc=16):  82%|████████▏ | 285721/349837 [00:32<00:09, 6550.04 examples/s]
Map (num_proc=16):  95%|█████████▍| 331755/349837 [00:34<00:02, 8276.59 examples/s]
Map (num_proc=16):  82%|████████▏ | 286581/349837 [00:32<00:08, 7069.51 examples/s]
Map (num_proc=16):  95%|█████████▌| 332673/349837 [00:34<00:02, 8536.24 examples/s]
Map (num_proc=16):  82%|████████▏ | 287547/349837 [00:32<00:08, 7779.22 examples/s]
Map (num_proc=16):  95%|█████████▌| 333543/349837 [00:34<00:01, 8463.21 examples/s]
Map (num_proc=16):  82%|████████▏ | 288560/349837 [00:32<00:07, 8448.37 examples/s]
Map (num_proc=16):  96%|█████████▌| 334408/349837 [00:35<00:01, 7832.32 examples/s]
Map (num_proc=16):  83%|████████▎ | 289450/349837 [00:32<00:07, 8119.31 examples/s]
Map (num_proc=16):  96%|█████████▌| 335242/349837 [00:35<00:01, 7848.76 examples/s]
Map (num_proc=16):  83%|████████▎ | 290294/349837 [00:32<00:07, 8017.39 examples/s]
Map (num_proc=16):  96%|█████████▌| 336090/349837 [00:35<00:01, 7776.91 examples/s]
Map (num_proc=16):  83%|████████▎ | 291168/349837 [00:32<00:07, 8189.07 examples/s]
Map (num_proc=16):  96%|█████████▋| 336906/349837 [00:35<00:01, 7824.30 examples/s]
Map (num_proc=16):  83%|████████▎ | 292077/349837 [00:32<00:06, 8404.85 examples/s]
Map (num_proc=16):  97%|█████████▋| 337884/349837 [00:35<00:01, 8368.33 examples/s]
Map (num_proc=16):  84%|████████▍ | 293233/349837 [00:33<00:06, 9318.39 examples/s]
Map (num_proc=16):  84%|████████▍ | 294331/349837 [00:33<00:05, 9600.50 examples/s]
Map (num_proc=16):  97%|█████████▋| 338753/349837 [00:35<00:01, 7661.19 examples/s]
Map (num_proc=16):  84%|████████▍ | 295518/349837 [00:33<00:05, 10160.54 examples/s]
Map (num_proc=16):  97%|█████████▋| 339577/349837 [00:35<00:01, 7369.61 examples/s]
Map (num_proc=16):  85%|████████▍ | 296567/349837 [00:33<00:05, 10043.69 examples/s]
Map (num_proc=16):  97%|█████████▋| 340331/349837 [00:35<00:01, 7406.34 examples/s]
Map (num_proc=16):  85%|████████▌ | 297587/349837 [00:33<00:05, 10028.34 examples/s]
Map (num_proc=16):  97%|█████████▋| 341085/349837 [00:35<00:01, 7333.94 examples/s]
Map (num_proc=16):  85%|████████▌ | 298932/349837 [00:33<00:04, 10973.23 examples/s]
Map (num_proc=16):  98%|█████████▊| 341861/349837 [00:35<00:01, 7432.03 examples/s]
Map (num_proc=16):  86%|████████▌ | 300054/349837 [00:33<00:04, 10900.66 examples/s]
Map (num_proc=16):  98%|█████████▊| 342641/349837 [00:36<00:00, 7434.30 examples/s]
Map (num_proc=16):  86%|████████▌ | 301162/349837 [00:33<00:04, 10896.55 examples/s]
Map (num_proc=16):  98%|█████████▊| 343436/349837 [00:36<00:00, 7494.35 examples/s]
Map (num_proc=16):  86%|████████▋ | 302365/349837 [00:33<00:04, 11064.34 examples/s]
Map (num_proc=16):  98%|█████████▊| 344250/349837 [00:36<00:00, 7375.54 examples/s]
Map (num_proc=16):  87%|████████▋ | 303516/349837 [00:34<00:04, 10780.79 examples/s]
Map (num_proc=16):  99%|█████████▊| 345060/349837 [00:36<00:00, 7566.08 examples/s]
Map (num_proc=16):  87%|████████▋ | 304626/349837 [00:34<00:04, 10848.76 examples/s]
Map (num_proc=16):  99%|█████████▉| 345845/349837 [00:36<00:00, 7519.93 examples/s]
Map (num_proc=16):  87%|████████▋ | 305769/349837 [00:34<00:04, 10713.59 examples/s]
Map (num_proc=16):  99%|█████████▉| 346614/349837 [00:36<00:00, 7044.58 examples/s]
Map (num_proc=16):  88%|████████▊ | 306865/349837 [00:34<00:04, 10462.44 examples/s]
Map (num_proc=16):  99%|█████████▉| 347363/349837 [00:36<00:00, 6608.64 examples/s]
Map (num_proc=16):  88%|████████▊ | 307915/349837 [00:34<00:04, 10438.96 examples/s]
Map (num_proc=16):  88%|████████▊ | 309005/349837 [00:34<00:03, 10494.36 examples/s]
Map (num_proc=16):  99%|█████████▉| 348073/349837 [00:36<00:00, 6309.96 examples/s]
Map (num_proc=16):  89%|████████▊ | 310120/349837 [00:34<00:03, 10655.72 examples/s]
Map (num_proc=16): 100%|█████████▉| 348784/349837 [00:37<00:00, 5855.91 examples/s]
Map (num_proc=16):  89%|████████▉ | 311435/349837 [00:34<00:03, 11374.45 examples/s]
Map (num_proc=16):  89%|████████▉ | 312849/349837 [00:34<00:03, 12143.75 examples/s]
Map (num_proc=16): 100%|█████████▉| 349394/349837 [00:37<00:00, 4732.12 examples/s]
Map (num_proc=16):  90%|████████▉ | 314204/349837 [00:34<00:02, 12367.94 examples/s]
Map (num_proc=16):  90%|█████████ | 315666/349837 [00:35<00:02, 12909.51 examples/s]
Map (num_proc=16):  91%|█████████ | 317431/349837 [00:35<00:02, 14162.52 examples/s]
Map (num_proc=16):  91%|█████████ | 318895/349837 [00:35<00:02, 14286.25 examples/s]
Map (num_proc=16):  92%|█████████▏| 320551/349837 [00:35<00:01, 14832.60 examples/s]
Map (num_proc=16):  92%|█████████▏| 322299/349837 [00:35<00:01, 15582.43 examples/s]
Map (num_proc=16):  93%|█████████▎| 323871/349837 [00:35<00:01, 15580.27 examples/s]
Map (num_proc=16):  93%|█████████▎| 325452/349837 [00:35<00:01, 14794.12 examples/s]
Map (num_proc=16):  94%|█████████▎| 327254/349837 [00:35<00:01, 15662.61 examples/s]
Map (num_proc=16):  94%|█████████▍| 328934/349837 [00:35<00:01, 15965.43 examples/s]
Map (num_proc=16):  95%|█████████▍| 330703/349837 [00:35<00:01, 16427.34 examples/s]
Map (num_proc=16):  95%|█████████▌| 332388/349837 [00:36<00:01, 16412.73 examples/s]
Map (num_proc=16):  95%|█████████▌| 334077/349837 [00:36<00:00, 15960.11 examples/s]
Map (num_proc=16):  96%|█████████▌| 335758/349837 [00:36<00:00, 15653.26 examples/s]
Map (num_proc=16):  96%|█████████▋| 337346/349837 [00:36<00:00, 15710.56 examples/s]
Map (num_proc=16):  97%|█████████▋| 338989/349837 [00:36<00:00, 15910.77 examples/s]
Map (num_proc=16):  97%|█████████▋| 340648/349837 [00:36<00:00, 14776.66 examples/s]
Map (num_proc=16):  98%|█████████▊| 342215/349837 [00:36<00:00, 13217.43 examples/s]
Map (num_proc=16):  98%|█████████▊| 343612/349837 [00:36<00:00, 12582.86 examples/s]
Map (num_proc=16):  99%|█████████▊| 344992/349837 [00:37<00:00, 11378.68 examples/s]
Map (num_proc=16):  99%|█████████▉| 346195/349837 [00:37<00:00, 9716.02 examples/s] 
Map (num_proc=16):  99%|█████████▉| 347311/349837 [00:37<00:00, 8500.10 examples/s]
                                                                                   
Final fine-tuning data:
Train Dataset({
    features: ['input', 'source', 'prompt_type', 'input_ids', 'attention_mask', 'labels'],
    num_rows: 349837
})
Valid None
Sample input: {'input': ["<human>: Why some mold of cheese is safe?\n<bot>: I apologize, but I'm not sure I fully understand your question. Can you rephrase it or provide more context about what specifically you would like to know regarding mold and cheese safety? Some possible follow-up questions could be:\n- Are you asking why some types of cheese use mold as an ingredient? If so, certain molds are intentionally added to cheeses like Camembert or Brie to develop flavor and texture.\n- Are you concerned about health risks from eating moldy cheese? If so, some molds on cheese can be harmful, while others are safe to eat in small amounts or if the cheese has been properly aged.\n- Is there another aspect of cheese and mold you're curious about? If you can provide more details I may be able to provide helpful information.\n\n<human>: Why is the mould in Camembert and Brie safe but not all other types? How do they make it safe?\n\n<bot>: The molds used in the production of Camembert and Brie cheese are specifically chosen and controlled during the cheese-making process to ensure they are safe for human consumption. These molds are a type of Penicillium that have been selected for their ability to break down and transform the cheese's texture and flavor while also being safe to eat.\n\nOther types of molds that may grow on cheese, such as Aspergillus or Fusarium, can produce harmful toxins that can make people sick. To prevent the growth of these harmful molds, cheese manufacturers may use methods such as pasteurization, sterilization, or other types of processing to kill off any harmful bacteria or fungi.\n\nIn the case of Camembert and Brie, the molds are carefully monitored and controlled during the cheese-making process to ensure they grow under specific conditions, such as controlled temperature and humidity, which help prevent the growth of harmful bacteria or other molds. Additionally, the cheeses are typically aged for several weeks, which further reduces the risk of harmful bacteria or fungi surviving in the cheese.\n"], 'source': ['OpenAssistant/oasst1'], 'prompt_type': ['plain']}

Map (num_proc=16): 100%|█████████▉| 348244/349837 [00:37<00:00, 7115.58 examples/s]Auto set eval_steps to 136 out of 2733 total training steps
Auto step save_steps to 136

Map (num_proc=16): 100%|█████████▉| 349076/349837 [00:37<00:00, 6069.39 examples/s]
Map (num_proc=16): 100%|█████████▉| 349797/349837 [00:38<00:00, 5174.02 examples/s]
                                                                                   
You are adding a <class 'transformers.integrations.TensorBoardCallback'> to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is
:DefaultFlowCallback
TensorBoardCallback
You are adding a <class 'transformers.integrations.TensorBoardCallback'> to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is
:DefaultFlowCallback
TensorBoardCallback
You're using a GPTNeoXTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.

  0%|          | 0/2733 [00:00<?, ?it/s]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")
You're using a GPTNeoXTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:219: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at ../aten/src/ATen/native/TensorCompare.cpp:493.)
  attn_scores = torch.where(causal_mask, attn_scores, mask_value)
/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:219: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at ../aten/src/ATen/native/TensorCompare.cpp:493.)
  attn_scores = torch.where(causal_mask, attn_scores, mask_value)

  0%|          | 1/2733 [00:17<13:04:52, 17.24s/it]
                                                   
{'loss': 2.4669, 'learning_rate': 2.9999999999999997e-06, 'epoch': 0.0}

  0%|          | 1/2733 [00:17<13:04:52, 17.24s/it]
  0%|          | 2/2733 [00:30<11:22:33, 15.00s/it]
                                                   
{'loss': 2.4021, 'learning_rate': 5.999999999999999e-06, 'epoch': 0.0}

  0%|          | 2/2733 [00:30<11:22:33, 15.00s/it]
  0%|          | 3/2733 [00:43<10:40:29, 14.08s/it]
                                                   
{'loss': 2.4267, 'learning_rate': 8.999999999999999e-06, 'epoch': 0.0}

  0%|          | 3/2733 [00:43<10:40:29, 14.08s/it]
  0%|          | 4/2733 [00:56<10:26:22, 13.77s/it]
                                                   
{'loss': 2.536, 'learning_rate': 1.1999999999999999e-05, 'epoch': 0.0}

  0%|          | 4/2733 [00:56<10:26:22, 13.77s/it]
  0%|          | 5/2733 [01:10<10:22:08, 13.68s/it]
                                                   
{'loss': 2.4781, 'learning_rate': 1.4999999999999999e-05, 'epoch': 0.0}

  0%|          | 5/2733 [01:10<10:22:08, 13.68s/it]
  0%|          | 6/2733 [01:23<10:10:27, 13.43s/it]
                                                   
{'loss': 2.3465, 'learning_rate': 1.7999999999999997e-05, 'epoch': 0.0}

  0%|          | 6/2733 [01:23<10:10:27, 13.43s/it]
  0%|          | 7/2733 [01:36<10:06:28, 13.35s/it]
                                                   
{'loss': 2.5, 'learning_rate': 2.1e-05, 'epoch': 0.0}

  0%|          | 7/2733 [01:36<10:06:28, 13.35s/it]
  0%|          | 8/2733 [01:50<10:08:54, 13.41s/it]
                                                   
{'loss': 2.3567, 'learning_rate': 2.3999999999999997e-05, 'epoch': 0.0}

  0%|          | 8/2733 [01:50<10:08:54, 13.41s/it]
  0%|          | 9/2733 [02:03<10:06:43, 13.36s/it]
                                                   
{'loss': 2.3946, 'learning_rate': 2.6999999999999996e-05, 'epoch': 0.0}

  0%|          | 9/2733 [02:03<10:06:43, 13.36s/it]
  0%|          | 10/2733 [02:16<10:09:45, 13.44s/it]
                                                    
{'loss': 2.4574, 'learning_rate': 2.9999999999999997e-05, 'epoch': 0.0}

  0%|          | 10/2733 [02:17<10:09:45, 13.44s/it]
  0%|          | 11/2733 [02:30<10:05:21, 13.34s/it]
                                                    
{'loss': 2.5302, 'learning_rate': 3.2999999999999996e-05, 'epoch': 0.0}

  0%|          | 11/2733 [02:30<10:05:21, 13.34s/it]
  0%|          | 12/2733 [02:43<10:00:20, 13.24s/it]
                                                    
{'loss': 2.4725, 'learning_rate': 3.5999999999999994e-05, 'epoch': 0.0}

  0%|          | 12/2733 [02:43<10:00:20, 13.24s/it]
  0%|          | 13/2733 [02:56<10:03:55, 13.32s/it]
                                                    
{'loss': 2.3853, 'learning_rate': 3.9e-05, 'epoch': 0.0}

  0%|          | 13/2733 [02:56<10:03:55, 13.32s/it]
  1%|          | 14/2733 [03:08<9:48:27, 12.99s/it] 
                                                   
{'loss': 2.4519, 'learning_rate': 4.2e-05, 'epoch': 0.01}

  1%|          | 14/2733 [03:08<9:48:27, 12.99s/it]
  1%|          | 15/2733 [03:22<9:55:26, 13.14s/it]
                                                   
{'loss': 2.41, 'learning_rate': 4.4999999999999996e-05, 'epoch': 0.01}

  1%|          | 15/2733 [03:22<9:55:26, 13.14s/it]
  1%|          | 16/2733 [03:35<9:59:11, 13.23s/it]
                                                   
{'loss': 2.4896, 'learning_rate': 4.7999999999999994e-05, 'epoch': 0.01}

  1%|          | 16/2733 [03:35<9:59:11, 13.23s/it]
  1%|          | 17/2733 [03:48<9:56:14, 13.17s/it]
                                                   
{'loss': 2.3559, 'learning_rate': 5.1e-05, 'epoch': 0.01}

  1%|          | 17/2733 [03:48<9:56:14, 13.17s/it]
  1%|          | 18/2733 [04:02<9:56:02, 13.17s/it]
                                                   
{'loss': 2.2955, 'learning_rate': 5.399999999999999e-05, 'epoch': 0.01}

  1%|          | 18/2733 [04:02<9:56:02, 13.17s/it]
  1%|          | 19/2733 [04:15<9:55:33, 13.17s/it]
                                                   
{'loss': 2.415, 'learning_rate': 5.6999999999999996e-05, 'epoch': 0.01}

  1%|          | 19/2733 [04:15<9:55:33, 13.17s/it]
  1%|          | 20/2733 [04:28<9:56:32, 13.19s/it]
                                                   
{'loss': 2.2907, 'learning_rate': 5.9999999999999995e-05, 'epoch': 0.01}

  1%|          | 20/2733 [04:28<9:56:32, 13.19s/it]
  1%|          | 21/2733 [04:40<9:43:33, 12.91s/it]
                                                   
{'loss': 2.2822, 'learning_rate': 6.299999999999999e-05, 'epoch': 0.01}

  1%|          | 21/2733 [04:40<9:43:33, 12.91s/it]
  1%|          | 22/2733 [04:54<9:52:32, 13.11s/it]
                                                   
{'loss': 2.2411, 'learning_rate': 6.599999999999999e-05, 'epoch': 0.01}

  1%|          | 22/2733 [04:54<9:52:32, 13.11s/it]
  1%|          | 23/2733 [05:07<9:58:14, 13.25s/it]
                                                   
{'loss': 2.3236, 'learning_rate': 6.9e-05, 'epoch': 0.01}

  1%|          | 23/2733 [05:07<9:58:14, 13.25s/it]
  1%|          | 24/2733 [05:20<9:56:11, 13.20s/it]
                                                   
{'loss': 2.2371, 'learning_rate': 7.199999999999999e-05, 'epoch': 0.01}

  1%|          | 24/2733 [05:20<9:56:11, 13.20s/it]
  1%|          | 25/2733 [05:34<10:05:07, 13.41s/it]
                                                    
{'loss': 2.2052, 'learning_rate': 7.5e-05, 'epoch': 0.01}

  1%|          | 25/2733 [05:34<10:05:07, 13.41s/it]
  1%|          | 26/2733 [05:48<10:04:02, 13.39s/it]
                                                    
{'loss': 2.2836, 'learning_rate': 7.8e-05, 'epoch': 0.01}

  1%|          | 26/2733 [05:48<10:04:02, 13.39s/it]
  1%|          | 27/2733 [06:01<10:07:34, 13.47s/it]
                                                    
{'loss': 2.3205, 'learning_rate': 8.1e-05, 'epoch': 0.01}

  1%|          | 27/2733 [06:01<10:07:34, 13.47s/it]
  1%|          | 28/2733 [06:15<10:10:50, 13.55s/it]
                                                    
{'loss': 2.1997, 'learning_rate': 8.4e-05, 'epoch': 0.01}

  1%|          | 28/2733 [06:15<10:10:50, 13.55s/it]
  1%|          | 29/2733 [06:28<10:06:26, 13.46s/it]
                                                    
{'loss': 2.1437, 'learning_rate': 8.699999999999999e-05, 'epoch': 0.01}

  1%|          | 29/2733 [06:28<10:06:26, 13.46s/it]
  1%|          | 30/2733 [06:42<10:03:53, 13.40s/it]
                                                    
{'loss': 2.1755, 'learning_rate': 8.999999999999999e-05, 'epoch': 0.01}

  1%|          | 30/2733 [06:42<10:03:53, 13.40s/it]
  1%|          | 31/2733 [06:55<10:02:16, 13.37s/it]
                                                    
{'loss': 2.2736, 'learning_rate': 9.3e-05, 'epoch': 0.01}

  1%|          | 31/2733 [06:55<10:02:16, 13.37s/it]
  1%|          | 32/2733 [07:08<9:54:09, 13.20s/it] 
                                                   
{'loss': 2.1713, 'learning_rate': 9.599999999999999e-05, 'epoch': 0.01}

  1%|          | 32/2733 [07:08<9:54:09, 13.20s/it]
  1%|          | 33/2733 [07:21<10:02:30, 13.39s/it]
                                                    
{'loss': 2.1886, 'learning_rate': 9.9e-05, 'epoch': 0.01}

  1%|          | 33/2733 [07:22<10:02:30, 13.39s/it]
  1%|          | 34/2733 [07:34<9:54:23, 13.21s/it] 
                                                   
{'loss': 2.1319, 'learning_rate': 0.000102, 'epoch': 0.01}

  1%|          | 34/2733 [07:34<9:54:23, 13.21s/it]
  1%|▏         | 35/2733 [07:47<9:48:16, 13.08s/it]
                                                   
{'loss': 2.1762, 'learning_rate': 0.00010499999999999999, 'epoch': 0.01}

  1%|▏         | 35/2733 [07:47<9:48:16, 13.08s/it]
  1%|▏         | 36/2733 [08:00<9:49:42, 13.12s/it]
                                                   
{'loss': 2.054, 'learning_rate': 0.00010799999999999998, 'epoch': 0.01}

  1%|▏         | 36/2733 [08:00<9:49:42, 13.12s/it]
  1%|▏         | 37/2733 [08:13<9:41:55, 12.95s/it]
                                                   
{'loss': 2.1185, 'learning_rate': 0.00011099999999999999, 'epoch': 0.01}

  1%|▏         | 37/2733 [08:13<9:41:55, 12.95s/it]
  1%|▏         | 38/2733 [08:26<9:51:16, 13.16s/it]
                                                   
{'loss': 1.9972, 'learning_rate': 0.00011399999999999999, 'epoch': 0.01}

  1%|▏         | 38/2733 [08:27<9:51:16, 13.16s/it]
  1%|▏         | 39/2733 [08:40<9:53:48, 13.22s/it]
                                                   
{'loss': 2.0246, 'learning_rate': 0.000117, 'epoch': 0.01}

  1%|▏         | 39/2733 [08:40<9:53:48, 13.22s/it]
  1%|▏         | 40/2733 [08:53<9:58:39, 13.34s/it]
                                                   
{'loss': 1.9886, 'learning_rate': 0.00011999999999999999, 'epoch': 0.01}

  1%|▏         | 40/2733 [08:53<9:58:39, 13.34s/it]
  2%|▏         | 41/2733 [09:08<10:08:36, 13.56s/it]
                                                    
{'loss': 2.0308, 'learning_rate': 0.00012299999999999998, 'epoch': 0.02}

  2%|▏         | 41/2733 [09:08<10:08:36, 13.56s/it]
  2%|▏         | 42/2733 [09:21<10:10:38, 13.62s/it]
                                                    
{'loss': 2.1187, 'learning_rate': 0.00012599999999999997, 'epoch': 0.02}

  2%|▏         | 42/2733 [09:21<10:10:38, 13.62s/it]
  2%|▏         | 43/2733 [09:35<10:06:11, 13.52s/it]
                                                    
{'loss': 2.0477, 'learning_rate': 0.000129, 'epoch': 0.02}

  2%|▏         | 43/2733 [09:35<10:06:11, 13.52s/it]
  2%|▏         | 44/2733 [09:48<10:10:18, 13.62s/it]
                                                    
{'loss': 2.1308, 'learning_rate': 0.00013199999999999998, 'epoch': 0.02}

  2%|▏         | 44/2733 [09:48<10:10:18, 13.62s/it]
  2%|▏         | 45/2733 [10:01<9:58:51, 13.37s/it] 
                                                   
{'loss': 2.0661, 'learning_rate': 0.000135, 'epoch': 0.02}

  2%|▏         | 45/2733 [10:01<9:58:51, 13.37s/it]
  2%|▏         | 46/2733 [10:15<9:59:04, 13.38s/it]
                                                   
{'loss': 2.0863, 'learning_rate': 0.000138, 'epoch': 0.02}

  2%|▏         | 46/2733 [10:15<9:59:04, 13.38s/it]
  2%|▏         | 47/2733 [10:28<10:03:30, 13.48s/it]
                                                    
{'loss': 2.04, 'learning_rate': 0.00014099999999999998, 'epoch': 0.02}

  2%|▏         | 47/2733 [10:28<10:03:30, 13.48s/it]
  2%|▏         | 48/2733 [10:42<10:03:21, 13.48s/it]
                                                    
{'loss': 2.0057, 'learning_rate': 0.00014399999999999998, 'epoch': 0.02}

  2%|▏         | 48/2733 [10:42<10:03:21, 13.48s/it]
  2%|▏         | 49/2733 [10:55<10:04:52, 13.52s/it]
                                                    
{'loss': 2.0474, 'learning_rate': 0.000147, 'epoch': 0.02}

  2%|▏         | 49/2733 [10:55<10:04:52, 13.52s/it]
  2%|▏         | 50/2733 [11:09<10:05:54, 13.55s/it]
                                                    
{'loss': 2.0223, 'learning_rate': 0.00015, 'epoch': 0.02}

  2%|▏         | 50/2733 [11:09<10:05:54, 13.55s/it]
  2%|▏         | 51/2733 [11:23<10:06:53, 13.58s/it]
                                                    
{'loss': 2.0164, 'learning_rate': 0.00015299999999999998, 'epoch': 0.02}

  2%|▏         | 51/2733 [11:23<10:06:53, 13.58s/it]
  2%|▏         | 52/2733 [11:37<10:09:50, 13.65s/it]
                                                    
{'loss': 1.9681, 'learning_rate': 0.000156, 'epoch': 0.02}

  2%|▏         | 52/2733 [11:37<10:09:50, 13.65s/it]
  2%|▏         | 53/2733 [11:50<10:01:39, 13.47s/it]
                                                    
{'loss': 1.9362, 'learning_rate': 0.000159, 'epoch': 0.02}

  2%|▏         | 53/2733 [11:50<10:01:39, 13.47s/it]
  2%|▏         | 54/2733 [12:03<9:57:15, 13.38s/it] 
                                                   
{'loss': 2.0519, 'learning_rate': 0.000162, 'epoch': 0.02}

  2%|▏         | 54/2733 [12:03<9:57:15, 13.38s/it]
  2%|▏         | 55/2733 [12:17<10:03:21, 13.52s/it]
                                                    
{'loss': 2.1179, 'learning_rate': 0.000165, 'epoch': 0.02}

  2%|▏         | 55/2733 [12:17<10:03:21, 13.52s/it]
  2%|▏         | 56/2733 [12:30<10:06:20, 13.59s/it]
                                                    
{'loss': 2.0879, 'learning_rate': 0.000168, 'epoch': 0.02}

  2%|▏         | 56/2733 [12:30<10:06:20, 13.59s/it]
  2%|▏         | 57/2733 [12:44<10:05:04, 13.57s/it]
                                                    
{'loss': 2.0379, 'learning_rate': 0.00017099999999999998, 'epoch': 0.02}

  2%|▏         | 57/2733 [12:44<10:05:04, 13.57s/it]
  2%|▏         | 58/2733 [12:57<9:55:16, 13.35s/it] 
                                                   
{'loss': 2.037, 'learning_rate': 0.00017399999999999997, 'epoch': 0.02}

  2%|▏         | 58/2733 [12:57<9:55:16, 13.35s/it]
  2%|▏         | 59/2733 [13:10<9:59:29, 13.45s/it]
                                                   
{'loss': 1.9836, 'learning_rate': 0.00017699999999999997, 'epoch': 0.02}

  2%|▏         | 59/2733 [13:10<9:59:29, 13.45s/it]
  2%|▏         | 60/2733 [13:24<9:59:55, 13.47s/it]
                                                   
{'loss': 2.046, 'learning_rate': 0.00017999999999999998, 'epoch': 0.02}

  2%|▏         | 60/2733 [13:24<9:59:55, 13.47s/it]
  2%|▏         | 61/2733 [13:37<9:49:36, 13.24s/it]
                                                   
{'loss': 1.8498, 'learning_rate': 0.00018299999999999998, 'epoch': 0.02}

  2%|▏         | 61/2733 [13:37<9:49:36, 13.24s/it]
  2%|▏         | 62/2733 [13:50<9:49:08, 13.23s/it]
                                                   
{'loss': 1.9899, 'learning_rate': 0.000186, 'epoch': 0.02}

  2%|▏         | 62/2733 [13:50<9:49:08, 13.23s/it]
  2%|▏         | 63/2733 [14:02<9:40:27, 13.04s/it]
                                                   
{'loss': 1.9883, 'learning_rate': 0.00018899999999999999, 'epoch': 0.02}

  2%|▏         | 63/2733 [14:02<9:40:27, 13.04s/it]
  2%|▏         | 64/2733 [14:16<9:44:54, 13.15s/it]
                                                   
{'loss': 2.0203, 'learning_rate': 0.00019199999999999998, 'epoch': 0.02}

  2%|▏         | 64/2733 [14:16<9:44:54, 13.15s/it]
  2%|▏         | 65/2733 [14:29<9:44:22, 13.14s/it]
                                                   
{'loss': 1.9885, 'learning_rate': 0.000195, 'epoch': 0.02}

  2%|▏         | 65/2733 [14:29<9:44:22, 13.14s/it]
  2%|▏         | 66/2733 [14:42<9:45:10, 13.16s/it]
                                                   
{'loss': 2.0359, 'learning_rate': 0.000198, 'epoch': 0.02}

  2%|▏         | 66/2733 [14:42<9:45:10, 13.16s/it]
  2%|▏         | 67/2733 [14:56<10:00:21, 13.51s/it]
                                                    
{'loss': 2.0371, 'learning_rate': 0.000201, 'epoch': 0.02}

  2%|▏         | 67/2733 [14:56<10:00:21, 13.51s/it]
  2%|▏         | 68/2733 [15:10<10:02:44, 13.57s/it]
                                                    
{'loss': 1.9686, 'learning_rate': 0.000204, 'epoch': 0.02}

  2%|▏         | 68/2733 [15:10<10:02:44, 13.57s/it]
  3%|▎         | 69/2733 [15:24<10:03:08, 13.58s/it]
                                                    
{'loss': 1.9324, 'learning_rate': 0.00020699999999999996, 'epoch': 0.03}

  3%|▎         | 69/2733 [15:24<10:03:08, 13.58s/it]
  3%|▎         | 70/2733 [15:38<10:08:35, 13.71s/it]
                                                    
{'loss': 1.8703, 'learning_rate': 0.00020999999999999998, 'epoch': 0.03}

  3%|▎         | 70/2733 [15:38<10:08:35, 13.71s/it]
  3%|▎         | 71/2733 [15:51<10:01:57, 13.57s/it]
                                                    
{'loss': 2.0131, 'learning_rate': 0.00021299999999999997, 'epoch': 0.03}

  3%|▎         | 71/2733 [15:51<10:01:57, 13.57s/it]
  3%|▎         | 72/2733 [16:04<9:52:55, 13.37s/it] 
                                                   
{'loss': 1.9011, 'learning_rate': 0.00021599999999999996, 'epoch': 0.03}

  3%|▎         | 72/2733 [16:04<9:52:55, 13.37s/it]
  3%|▎         | 73/2733 [16:17<9:49:42, 13.30s/it]
                                                   
{'loss': 1.9288, 'learning_rate': 0.00021899999999999998, 'epoch': 0.03}

  3%|▎         | 73/2733 [16:17<9:49:42, 13.30s/it]
  3%|▎         | 74/2733 [16:31<9:56:55, 13.47s/it]
                                                   
{'loss': 1.9511, 'learning_rate': 0.00022199999999999998, 'epoch': 0.03}

  3%|▎         | 74/2733 [16:31<9:56:55, 13.47s/it]
  3%|▎         | 75/2733 [16:45<10:05:40, 13.67s/it]
                                                    
{'loss': 1.9766, 'learning_rate': 0.000225, 'epoch': 0.03}

  3%|▎         | 75/2733 [16:45<10:05:40, 13.67s/it]
  3%|▎         | 76/2733 [16:58<9:57:12, 13.49s/it] 
                                                   
{'loss': 1.9434, 'learning_rate': 0.00022799999999999999, 'epoch': 0.03}

  3%|▎         | 76/2733 [16:58<9:57:12, 13.49s/it]
  3%|▎         | 77/2733 [17:12<9:59:49, 13.55s/it]
                                                   
{'loss': 1.9732, 'learning_rate': 0.00023099999999999998, 'epoch': 0.03}

  3%|▎         | 77/2733 [17:12<9:59:49, 13.55s/it]
  3%|▎         | 78/2733 [17:26<10:09:45, 13.78s/it]
                                                    
{'loss': 1.9249, 'learning_rate': 0.000234, 'epoch': 0.03}

  3%|▎         | 78/2733 [17:26<10:09:45, 13.78s/it]
  3%|▎         | 79/2733 [17:40<10:04:15, 13.66s/it]
                                                    
{'loss': 1.8715, 'learning_rate': 0.000237, 'epoch': 0.03}

  3%|▎         | 79/2733 [17:40<10:04:15, 13.66s/it]
  3%|▎         | 80/2733 [17:54<10:10:12, 13.80s/it]
                                                    
{'loss': 1.8922, 'learning_rate': 0.00023999999999999998, 'epoch': 0.03}

  3%|▎         | 80/2733 [17:54<10:10:12, 13.80s/it]
  3%|▎         | 81/2733 [18:07<10:01:35, 13.61s/it]
                                                    
{'loss': 1.9206, 'learning_rate': 0.000243, 'epoch': 0.03}

  3%|▎         | 81/2733 [18:07<10:01:35, 13.61s/it]
  3%|▎         | 82/2733 [18:20<9:54:35, 13.46s/it] 
                                                   
{'loss': 1.9554, 'learning_rate': 0.00024599999999999996, 'epoch': 0.03}

  3%|▎         | 82/2733 [18:20<9:54:35, 13.46s/it]
  3%|▎         | 83/2733 [18:33<9:48:40, 13.33s/it]
                                                   
{'loss': 1.9314, 'learning_rate': 0.000249, 'epoch': 0.03}

  3%|▎         | 83/2733 [18:33<9:48:40, 13.33s/it]
  3%|▎         | 84/2733 [18:47<9:53:09, 13.43s/it]
                                                   
{'loss': 1.9389, 'learning_rate': 0.00025199999999999995, 'epoch': 0.03}

  3%|▎         | 84/2733 [18:47<9:53:09, 13.43s/it]
  3%|▎         | 85/2733 [19:00<9:48:11, 13.33s/it]
                                                   
{'loss': 1.9386, 'learning_rate': 0.00025499999999999996, 'epoch': 0.03}

  3%|▎         | 85/2733 [19:00<9:48:11, 13.33s/it]
  3%|▎         | 86/2733 [19:14<10:00:47, 13.62s/it]
                                                    
{'loss': 1.9764, 'learning_rate': 0.000258, 'epoch': 0.03}

  3%|▎         | 86/2733 [19:14<10:00:47, 13.62s/it]
  3%|▎         | 87/2733 [19:27<9:56:40, 13.53s/it] 
                                                   
{'loss': 1.9037, 'learning_rate': 0.000261, 'epoch': 0.03}

  3%|▎         | 87/2733 [19:27<9:56:40, 13.53s/it]
  3%|▎         | 88/2733 [19:40<9:42:55, 13.22s/it]
                                                   
{'loss': 1.8477, 'learning_rate': 0.00026399999999999997, 'epoch': 0.03}

  3%|▎         | 88/2733 [19:40<9:42:55, 13.22s/it]
  3%|▎         | 89/2733 [19:53<9:44:40, 13.27s/it]
                                                   
{'loss': 1.8915, 'learning_rate': 0.000267, 'epoch': 0.03}

  3%|▎         | 89/2733 [19:53<9:44:40, 13.27s/it]
  3%|▎         | 90/2733 [20:07<9:45:00, 13.28s/it]
                                                   
{'loss': 1.9347, 'learning_rate': 0.00027, 'epoch': 0.03}

  3%|▎         | 90/2733 [20:07<9:45:00, 13.28s/it]
  3%|▎         | 91/2733 [20:20<9:43:14, 13.25s/it]
                                                   
{'loss': 1.8178, 'learning_rate': 0.00027299999999999997, 'epoch': 0.03}

  3%|▎         | 91/2733 [20:20<9:43:14, 13.25s/it]
  3%|▎         | 92/2733 [20:34<9:56:54, 13.56s/it]
                                                   
{'loss': 1.9288, 'learning_rate': 0.000276, 'epoch': 0.03}

  3%|▎         | 92/2733 [20:34<9:56:54, 13.56s/it]
  3%|▎         | 93/2733 [20:48<10:04:14, 13.73s/it]
                                                    
{'loss': 1.9483, 'learning_rate': 0.000279, 'epoch': 0.03}

  3%|▎         | 93/2733 [20:48<10:04:14, 13.73s/it]
  3%|▎         | 94/2733 [21:02<10:07:29, 13.81s/it]
                                                    
{'loss': 1.9095, 'learning_rate': 0.00028199999999999997, 'epoch': 0.03}

  3%|▎         | 94/2733 [21:02<10:07:29, 13.81s/it]
  3%|▎         | 95/2733 [21:16<10:08:47, 13.85s/it]
                                                    
{'loss': 1.8516, 'learning_rate': 0.000285, 'epoch': 0.03}

  3%|▎         | 95/2733 [21:16<10:08:47, 13.85s/it]
  4%|▎         | 96/2733 [21:30<10:04:06, 13.75s/it]
                                                    
{'loss': 1.9165, 'learning_rate': 0.00028799999999999995, 'epoch': 0.04}

  4%|▎         | 96/2733 [21:30<10:04:06, 13.75s/it]
  4%|▎         | 97/2733 [21:43<9:56:40, 13.58s/it] 
                                                   
{'loss': 1.9843, 'learning_rate': 0.00029099999999999997, 'epoch': 0.04}

  4%|▎         | 97/2733 [21:43<9:56:40, 13.58s/it]
  4%|▎         | 98/2733 [21:56<9:51:54, 13.48s/it]
                                                   
{'loss': 1.9312, 'learning_rate': 0.000294, 'epoch': 0.04}

  4%|▎         | 98/2733 [21:56<9:51:54, 13.48s/it]
  4%|▎         | 99/2733 [22:10<9:56:37, 13.59s/it]
                                                   
{'loss': 1.8532, 'learning_rate': 0.00029699999999999996, 'epoch': 0.04}

  4%|▎         | 99/2733 [22:10<9:56:37, 13.59s/it]
  4%|▎         | 100/2733 [22:24<9:57:35, 13.62s/it]
                                                    
{'loss': 1.759, 'learning_rate': 0.0003, 'epoch': 0.04}

  4%|▎         | 100/2733 [22:24<9:57:35, 13.62s/it]
  4%|▎         | 101/2733 [22:37<9:54:14, 13.55s/it]
                                                    
{'loss': 1.9521, 'learning_rate': 0.0002998860615267755, 'epoch': 0.04}

  4%|▎         | 101/2733 [22:37<9:54:14, 13.55s/it]
  4%|▎         | 102/2733 [22:51<9:55:49, 13.59s/it]
                                                    
{'loss': 1.9382, 'learning_rate': 0.00029977212305355103, 'epoch': 0.04}

  4%|▎         | 102/2733 [22:51<9:55:49, 13.59s/it]
  4%|▍         | 103/2733 [23:04<9:57:59, 13.64s/it]
                                                    
{'loss': 1.8427, 'learning_rate': 0.00029965818458032656, 'epoch': 0.04}

  4%|▍         | 103/2733 [23:04<9:57:59, 13.64s/it]
  4%|▍         | 104/2733 [23:18<9:52:02, 13.51s/it]
                                                    
{'loss': 1.8346, 'learning_rate': 0.00029954424610710215, 'epoch': 0.04}

  4%|▍         | 104/2733 [23:18<9:52:02, 13.51s/it]
  4%|▍         | 105/2733 [23:32<10:02:17, 13.75s/it]
                                                     
{'loss': 1.8116, 'learning_rate': 0.0002994303076338777, 'epoch': 0.04}

  4%|▍         | 105/2733 [23:32<10:02:17, 13.75s/it]
  4%|▍         | 106/2733 [23:46<10:01:01, 13.73s/it]
                                                     
{'loss': 1.885, 'learning_rate': 0.0002993163691606532, 'epoch': 0.04}

  4%|▍         | 106/2733 [23:46<10:01:01, 13.73s/it]
  4%|▍         | 107/2733 [23:59<9:55:34, 13.61s/it] 
                                                    
{'loss': 1.8061, 'learning_rate': 0.00029920243068742874, 'epoch': 0.04}

  4%|▍         | 107/2733 [23:59<9:55:34, 13.61s/it]
  4%|▍         | 108/2733 [24:12<9:51:25, 13.52s/it]
                                                    
{'loss': 1.9529, 'learning_rate': 0.00029908849221420427, 'epoch': 0.04}

  4%|▍         | 108/2733 [24:12<9:51:25, 13.52s/it]
  4%|▍         | 109/2733 [24:26<9:53:04, 13.56s/it]
                                                    
{'loss': 1.871, 'learning_rate': 0.00029897455374097986, 'epoch': 0.04}

  4%|▍         | 109/2733 [24:26<9:53:04, 13.56s/it]
  4%|▍         | 110/2733 [24:40<9:54:29, 13.60s/it]
                                                    
{'loss': 1.8779, 'learning_rate': 0.0002988606152677554, 'epoch': 0.04}

  4%|▍         | 110/2733 [24:40<9:54:29, 13.60s/it]
  4%|▍         | 111/2733 [24:52<9:45:52, 13.41s/it]
                                                    
{'loss': 1.7539, 'learning_rate': 0.0002987466767945309, 'epoch': 0.04}

  4%|▍         | 111/2733 [24:53<9:45:52, 13.41s/it]
  4%|▍         | 112/2733 [25:06<9:50:05, 13.51s/it]
                                                    
{'loss': 1.8744, 'learning_rate': 0.00029863273832130645, 'epoch': 0.04}

  4%|▍         | 112/2733 [25:06<9:50:05, 13.51s/it]
  4%|▍         | 113/2733 [25:19<9:41:46, 13.32s/it]
                                                    
{'loss': 1.7671, 'learning_rate': 0.00029851879984808203, 'epoch': 0.04}

  4%|▍         | 113/2733 [25:19<9:41:46, 13.32s/it]
  4%|▍         | 114/2733 [25:32<9:32:23, 13.11s/it]
                                                    
{'loss': 1.8417, 'learning_rate': 0.00029840486137485756, 'epoch': 0.04}

  4%|▍         | 114/2733 [25:32<9:32:23, 13.11s/it]
  4%|▍         | 115/2733 [25:45<9:35:42, 13.19s/it]
                                                    
{'loss': 1.8433, 'learning_rate': 0.0002982909229016331, 'epoch': 0.04}

  4%|▍         | 115/2733 [25:45<9:35:42, 13.19s/it]
  4%|▍         | 116/2733 [25:58<9:32:52, 13.13s/it]
                                                    
{'loss': 1.8158, 'learning_rate': 0.0002981769844284086, 'epoch': 0.04}

  4%|▍         | 116/2733 [25:58<9:32:52, 13.13s/it]
  4%|▍         | 117/2733 [26:12<9:41:18, 13.33s/it]
                                                    
{'loss': 1.8005, 'learning_rate': 0.00029806304595518415, 'epoch': 0.04}

  4%|▍         | 117/2733 [26:12<9:41:18, 13.33s/it]
  4%|▍         | 118/2733 [26:25<9:40:53, 13.33s/it]
                                                    
{'loss': 1.9151, 'learning_rate': 0.00029794910748195974, 'epoch': 0.04}

  4%|▍         | 118/2733 [26:25<9:40:53, 13.33s/it]
  4%|▍         | 119/2733 [26:39<9:46:35, 13.46s/it]
                                                    
{'loss': 1.8715, 'learning_rate': 0.00029783516900873527, 'epoch': 0.04}

  4%|▍         | 119/2733 [26:39<9:46:35, 13.46s/it]
  4%|▍         | 120/2733 [26:53<9:51:52, 13.59s/it]
                                                    
{'loss': 1.8389, 'learning_rate': 0.0002977212305355108, 'epoch': 0.04}

  4%|▍         | 120/2733 [26:53<9:51:52, 13.59s/it]
  4%|▍         | 121/2733 [27:07<9:58:48, 13.76s/it]
                                                    
{'loss': 1.8622, 'learning_rate': 0.00029760729206228633, 'epoch': 0.04}

  4%|▍         | 121/2733 [27:07<9:58:48, 13.76s/it]
  4%|▍         | 122/2733 [27:21<9:57:32, 13.73s/it]
                                                    
{'loss': 1.8651, 'learning_rate': 0.00029749335358906186, 'epoch': 0.04}

  4%|▍         | 122/2733 [27:21<9:57:32, 13.73s/it]
  5%|▍         | 123/2733 [27:34<9:50:13, 13.57s/it]
                                                    
{'loss': 1.8625, 'learning_rate': 0.00029737941511583744, 'epoch': 0.05}

  5%|▍         | 123/2733 [27:34<9:50:13, 13.57s/it]
  5%|▍         | 124/2733 [27:47<9:38:55, 13.31s/it]
                                                    
{'loss': 1.8188, 'learning_rate': 0.000297265476642613, 'epoch': 0.05}

  5%|▍         | 124/2733 [27:47<9:38:55, 13.31s/it]
  5%|▍         | 125/2733 [28:00<9:37:40, 13.29s/it]
                                                    
{'loss': 1.817, 'learning_rate': 0.0002971515381693885, 'epoch': 0.05}

  5%|▍         | 125/2733 [28:00<9:37:40, 13.29s/it]
  5%|▍         | 126/2733 [28:14<9:48:13, 13.54s/it]
                                                    
{'loss': 1.8693, 'learning_rate': 0.00029703759969616403, 'epoch': 0.05}

  5%|▍         | 126/2733 [28:14<9:48:13, 13.54s/it]
  5%|▍         | 127/2733 [28:27<9:43:18, 13.43s/it]
                                                    
{'loss': 1.8483, 'learning_rate': 0.0002969236612229396, 'epoch': 0.05}

  5%|▍         | 127/2733 [28:27<9:43:18, 13.43s/it]
  5%|▍         | 128/2733 [28:40<9:35:25, 13.25s/it]
                                                    
{'loss': 1.8245, 'learning_rate': 0.00029680972274971515, 'epoch': 0.05}

  5%|▍         | 128/2733 [28:40<9:35:25, 13.25s/it]
  5%|▍         | 129/2733 [28:52<9:17:01, 12.83s/it]
                                                    
{'loss': 1.7041, 'learning_rate': 0.0002966957842764907, 'epoch': 0.05}

  5%|▍         | 129/2733 [28:52<9:17:01, 12.83s/it]
  5%|▍         | 130/2733 [29:04<9:11:46, 12.72s/it]
                                                    
{'loss': 1.776, 'learning_rate': 0.0002965818458032662, 'epoch': 0.05}

  5%|▍         | 130/2733 [29:04<9:11:46, 12.72s/it]
  5%|▍         | 131/2733 [29:17<9:16:57, 12.84s/it]
                                                    
{'loss': 1.8091, 'learning_rate': 0.00029646790733004174, 'epoch': 0.05}

  5%|▍         | 131/2733 [29:17<9:16:57, 12.84s/it]
  5%|▍         | 132/2733 [29:31<9:31:56, 13.19s/it]
                                                    
{'loss': 1.8029, 'learning_rate': 0.0002963539688568173, 'epoch': 0.05}

  5%|▍         | 132/2733 [29:31<9:31:56, 13.19s/it]
  5%|▍         | 133/2733 [29:45<9:40:35, 13.40s/it]
                                                    
{'loss': 1.8166, 'learning_rate': 0.00029624003038359285, 'epoch': 0.05}

  5%|▍         | 133/2733 [29:45<9:40:35, 13.40s/it]
  5%|▍         | 134/2733 [29:59<9:42:05, 13.44s/it]
                                                    
{'loss': 1.8683, 'learning_rate': 0.0002961260919103684, 'epoch': 0.05}

  5%|▍         | 134/2733 [29:59<9:42:05, 13.44s/it]
  5%|▍         | 135/2733 [30:11<9:22:37, 12.99s/it]
                                                    
{'loss': 1.7531, 'learning_rate': 0.0002960121534371439, 'epoch': 0.05}

  5%|▍         | 135/2733 [30:11<9:22:37, 12.99s/it]
  5%|▍         | 136/2733 [30:25<9:32:52, 13.24s/it]
                                                    
{'loss': 1.8524, 'learning_rate': 0.00029589821496391945, 'epoch': 0.05}

  5%|▍         | 136/2733 [30:25<9:32:52, 13.24s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

  5%|▌         | 137/2733 [30:39<9:47:03, 13.57s/it]
                                                    
{'loss': 1.7765, 'learning_rate': 0.00029578427649069503, 'epoch': 0.05}

  5%|▌         | 137/2733 [30:39<9:47:03, 13.57s/it]
  5%|▌         | 138/2733 [30:53<9:47:02, 13.57s/it]
                                                    
{'loss': 1.7638, 'learning_rate': 0.0002956703380174705, 'epoch': 0.05}

  5%|▌         | 138/2733 [30:53<9:47:02, 13.57s/it]
  5%|▌         | 139/2733 [31:07<9:56:10, 13.79s/it]
                                                    
{'loss': 1.8052, 'learning_rate': 0.0002955563995442461, 'epoch': 0.05}

  5%|▌         | 139/2733 [31:07<9:56:10, 13.79s/it]
  5%|▌         | 140/2733 [31:21<10:02:11, 13.93s/it]
                                                     
{'loss': 1.8282, 'learning_rate': 0.0002954424610710216, 'epoch': 0.05}

  5%|▌         | 140/2733 [31:21<10:02:11, 13.93s/it]
  5%|▌         | 141/2733 [31:35<10:01:54, 13.93s/it]
                                                     
{'loss': 1.8231, 'learning_rate': 0.0002953285225977972, 'epoch': 0.05}

  5%|▌         | 141/2733 [31:35<10:01:54, 13.93s/it]
  5%|▌         | 142/2733 [31:49<10:05:40, 14.03s/it]
                                                     
{'loss': 1.8078, 'learning_rate': 0.0002952145841245727, 'epoch': 0.05}

  5%|▌         | 142/2733 [31:49<10:05:40, 14.03s/it]
  5%|▌         | 143/2733 [32:02<9:50:21, 13.68s/it] 
                                                    
{'loss': 1.8484, 'learning_rate': 0.0002951006456513482, 'epoch': 0.05}

  5%|▌         | 143/2733 [32:02<9:50:21, 13.68s/it]
  5%|▌         | 144/2733 [32:16<9:54:32, 13.78s/it]
                                                    
{'loss': 1.8497, 'learning_rate': 0.0002949867071781238, 'epoch': 0.05}

  5%|▌         | 144/2733 [32:16<9:54:32, 13.78s/it]
  5%|▌         | 145/2733 [32:30<10:00:35, 13.92s/it]
                                                     
{'loss': 1.8988, 'learning_rate': 0.00029487276870489933, 'epoch': 0.05}

  5%|▌         | 145/2733 [32:30<10:00:35, 13.92s/it]
  5%|▌         | 146/2733 [32:44<9:57:04, 13.85s/it] 
                                                    
{'loss': 1.8489, 'learning_rate': 0.00029475883023167486, 'epoch': 0.05}

  5%|▌         | 146/2733 [32:44<9:57:04, 13.85s/it]
  5%|▌         | 147/2733 [32:58<10:02:38, 13.98s/it]
                                                     
{'loss': 1.8742, 'learning_rate': 0.0002946448917584504, 'epoch': 0.05}

  5%|▌         | 147/2733 [32:58<10:02:38, 13.98s/it]
  5%|▌         | 148/2733 [33:11<9:48:53, 13.67s/it] 
                                                    
{'loss': 1.7602, 'learning_rate': 0.00029453095328522597, 'epoch': 0.05}

  5%|▌         | 148/2733 [33:11<9:48:53, 13.67s/it]
  5%|▌         | 149/2733 [33:24<9:38:46, 13.44s/it]
                                                    
{'loss': 1.7668, 'learning_rate': 0.0002944170148120015, 'epoch': 0.05}

  5%|▌         | 149/2733 [33:24<9:38:46, 13.44s/it]
  5%|▌         | 150/2733 [33:37<9:34:43, 13.35s/it]
                                                    
{'loss': 1.6679, 'learning_rate': 0.00029430307633877703, 'epoch': 0.05}

  5%|▌         | 150/2733 [33:37<9:34:43, 13.35s/it]
  6%|▌         | 151/2733 [33:51<9:36:46, 13.40s/it]
                                                    
{'loss': 1.8736, 'learning_rate': 0.00029418913786555256, 'epoch': 0.06}

  6%|▌         | 151/2733 [33:51<9:36:46, 13.40s/it]
  6%|▌         | 152/2733 [34:05<9:40:05, 13.49s/it]
                                                    
{'loss': 1.7311, 'learning_rate': 0.0002940751993923281, 'epoch': 0.06}

  6%|▌         | 152/2733 [34:05<9:40:05, 13.49s/it]
  6%|▌         | 153/2733 [34:18<9:43:23, 13.57s/it]
                                                    
{'loss': 1.8007, 'learning_rate': 0.0002939612609191037, 'epoch': 0.06}

  6%|▌         | 153/2733 [34:18<9:43:23, 13.57s/it]
  6%|▌         | 154/2733 [34:32<9:44:22, 13.60s/it]
                                                    
{'loss': 1.8273, 'learning_rate': 0.0002938473224458792, 'epoch': 0.06}

  6%|▌         | 154/2733 [34:32<9:44:22, 13.60s/it]
  6%|▌         | 155/2733 [34:46<9:44:41, 13.61s/it]
                                                    
{'loss': 1.8953, 'learning_rate': 0.00029373338397265474, 'epoch': 0.06}

  6%|▌         | 155/2733 [34:46<9:44:41, 13.61s/it]
  6%|▌         | 156/2733 [34:59<9:42:28, 13.56s/it]
                                                    
{'loss': 1.8316, 'learning_rate': 0.00029361944549943027, 'epoch': 0.06}

  6%|▌         | 156/2733 [34:59<9:42:28, 13.56s/it]
  6%|▌         | 157/2733 [35:13<9:46:01, 13.65s/it]
                                                    
{'loss': 1.7156, 'learning_rate': 0.0002935055070262058, 'epoch': 0.06}

  6%|▌         | 157/2733 [35:13<9:46:01, 13.65s/it]
  6%|▌         | 158/2733 [35:27<9:51:13, 13.78s/it]
                                                    
{'loss': 1.7559, 'learning_rate': 0.0002933915685529814, 'epoch': 0.06}

  6%|▌         | 158/2733 [35:27<9:51:13, 13.78s/it]
  6%|▌         | 159/2733 [35:40<9:38:13, 13.48s/it]
                                                    
{'loss': 1.8068, 'learning_rate': 0.0002932776300797569, 'epoch': 0.06}

  6%|▌         | 159/2733 [35:40<9:38:13, 13.48s/it]
  6%|▌         | 160/2733 [35:54<9:43:21, 13.60s/it]
                                                    
{'loss': 1.7185, 'learning_rate': 0.00029316369160653245, 'epoch': 0.06}

  6%|▌         | 160/2733 [35:54<9:43:21, 13.60s/it]
  6%|▌         | 161/2733 [36:07<9:43:52, 13.62s/it]
                                                    
{'loss': 1.9019, 'learning_rate': 0.000293049753133308, 'epoch': 0.06}

  6%|▌         | 161/2733 [36:07<9:43:52, 13.62s/it]
  6%|▌         | 162/2733 [36:21<9:43:08, 13.61s/it]
                                                    
{'loss': 1.8152, 'learning_rate': 0.00029293581466008356, 'epoch': 0.06}

  6%|▌         | 162/2733 [36:21<9:43:08, 13.61s/it]
  6%|▌         | 163/2733 [36:34<9:38:49, 13.51s/it]
                                                    
{'loss': 1.7169, 'learning_rate': 0.0002928218761868591, 'epoch': 0.06}

  6%|▌         | 163/2733 [36:34<9:38:49, 13.51s/it]
  6%|▌         | 164/2733 [36:49<9:48:47, 13.75s/it]
                                                    
{'loss': 1.7466, 'learning_rate': 0.0002927079377136346, 'epoch': 0.06}

  6%|▌         | 164/2733 [36:49<9:48:47, 13.75s/it]
  6%|▌         | 165/2733 [37:02<9:45:20, 13.68s/it]
                                                    
{'loss': 1.8351, 'learning_rate': 0.00029259399924041015, 'epoch': 0.06}

  6%|▌         | 165/2733 [37:02<9:45:20, 13.68s/it]
  6%|▌         | 166/2733 [37:16<9:52:40, 13.85s/it]
                                                    
{'loss': 1.8922, 'learning_rate': 0.0002924800607671857, 'epoch': 0.06}

  6%|▌         | 166/2733 [37:16<9:52:40, 13.85s/it]
  6%|▌         | 167/2733 [37:30<9:52:21, 13.85s/it]
                                                    
{'loss': 1.7491, 'learning_rate': 0.00029236612229396127, 'epoch': 0.06}

  6%|▌         | 167/2733 [37:30<9:52:21, 13.85s/it]
  6%|▌         | 168/2733 [37:44<9:50:04, 13.80s/it]
                                                    
{'loss': 1.8249, 'learning_rate': 0.0002922521838207368, 'epoch': 0.06}

  6%|▌         | 168/2733 [37:44<9:50:04, 13.80s/it]
  6%|▌         | 169/2733 [37:58<9:55:47, 13.94s/it]
                                                    
{'loss': 1.7717, 'learning_rate': 0.0002921382453475123, 'epoch': 0.06}

  6%|▌         | 169/2733 [37:58<9:55:47, 13.94s/it]
  6%|▌         | 170/2733 [38:11<9:47:50, 13.76s/it]
                                                    
{'loss': 1.8406, 'learning_rate': 0.00029202430687428786, 'epoch': 0.06}

  6%|▌         | 170/2733 [38:11<9:47:50, 13.76s/it]
  6%|▋         | 171/2733 [38:26<9:54:38, 13.93s/it]
                                                    
{'loss': 1.7473, 'learning_rate': 0.0002919103684010634, 'epoch': 0.06}

  6%|▋         | 171/2733 [38:26<9:54:38, 13.93s/it]
  6%|▋         | 172/2733 [38:39<9:51:56, 13.87s/it]
                                                    
{'loss': 1.7871, 'learning_rate': 0.00029179642992783897, 'epoch': 0.06}

  6%|▋         | 172/2733 [38:40<9:51:56, 13.87s/it]
  6%|▋         | 173/2733 [38:53<9:42:52, 13.66s/it]
                                                    
{'loss': 1.7735, 'learning_rate': 0.0002916824914546145, 'epoch': 0.06}

  6%|▋         | 173/2733 [38:53<9:42:52, 13.66s/it]
  6%|▋         | 174/2733 [39:06<9:43:24, 13.68s/it]
                                                    
{'loss': 1.7892, 'learning_rate': 0.00029156855298139003, 'epoch': 0.06}

  6%|▋         | 174/2733 [39:06<9:43:24, 13.68s/it]
  6%|▋         | 175/2733 [39:20<9:36:49, 13.53s/it]
                                                    
{'loss': 1.6934, 'learning_rate': 0.00029145461450816556, 'epoch': 0.06}

  6%|▋         | 175/2733 [39:20<9:36:49, 13.53s/it]
  6%|▋         | 176/2733 [39:33<9:36:08, 13.52s/it]
                                                    
{'loss': 1.8345, 'learning_rate': 0.00029134067603494115, 'epoch': 0.06}

  6%|▋         | 176/2733 [39:33<9:36:08, 13.52s/it]
  6%|▋         | 177/2733 [39:47<9:39:52, 13.61s/it]
                                                    
{'loss': 1.6992, 'learning_rate': 0.0002912267375617166, 'epoch': 0.06}

  6%|▋         | 177/2733 [39:47<9:39:52, 13.61s/it]
  7%|▋         | 178/2733 [40:01<9:40:08, 13.62s/it]
                                                    
{'loss': 1.8695, 'learning_rate': 0.00029111279908849215, 'epoch': 0.07}

  7%|▋         | 178/2733 [40:01<9:40:08, 13.62s/it]
  7%|▋         | 179/2733 [40:14<9:38:27, 13.59s/it]
                                                    
{'loss': 1.7166, 'learning_rate': 0.00029099886061526774, 'epoch': 0.07}

  7%|▋         | 179/2733 [40:14<9:38:27, 13.59s/it]
  7%|▋         | 180/2733 [40:28<9:44:05, 13.73s/it]
                                                    
{'loss': 1.7892, 'learning_rate': 0.00029088492214204327, 'epoch': 0.07}

  7%|▋         | 180/2733 [40:28<9:44:05, 13.73s/it]
  7%|▋         | 181/2733 [40:42<9:40:22, 13.65s/it]
                                                    
{'loss': 1.8113, 'learning_rate': 0.0002907709836688188, 'epoch': 0.07}

  7%|▋         | 181/2733 [40:42<9:40:22, 13.65s/it]
  7%|▋         | 182/2733 [40:55<9:34:02, 13.50s/it]
                                                    
{'loss': 1.8164, 'learning_rate': 0.00029065704519559433, 'epoch': 0.07}

  7%|▋         | 182/2733 [40:55<9:34:02, 13.50s/it]
  7%|▋         | 183/2733 [41:08<9:31:39, 13.45s/it]
                                                    
{'loss': 1.8739, 'learning_rate': 0.00029054310672236986, 'epoch': 0.07}

  7%|▋         | 183/2733 [41:08<9:31:39, 13.45s/it]
  7%|▋         | 184/2733 [41:22<9:33:05, 13.49s/it]
                                                    
{'loss': 1.8189, 'learning_rate': 0.00029042916824914544, 'epoch': 0.07}

  7%|▋         | 184/2733 [41:22<9:33:05, 13.49s/it]
  7%|▋         | 185/2733 [41:35<9:30:28, 13.43s/it]
                                                    
{'loss': 1.9097, 'learning_rate': 0.000290315229775921, 'epoch': 0.07}

  7%|▋         | 185/2733 [41:35<9:30:28, 13.43s/it]
  7%|▋         | 186/2733 [41:49<9:33:04, 13.50s/it]
                                                    
{'loss': 1.8281, 'learning_rate': 0.0002902012913026965, 'epoch': 0.07}

  7%|▋         | 186/2733 [41:49<9:33:04, 13.50s/it]
  7%|▋         | 187/2733 [42:02<9:32:45, 13.50s/it]
                                                    
{'loss': 1.8256, 'learning_rate': 0.00029008735282947204, 'epoch': 0.07}

  7%|▋         | 187/2733 [42:02<9:32:45, 13.50s/it]
  7%|▋         | 188/2733 [42:15<9:20:06, 13.20s/it]
                                                    
{'loss': 1.7691, 'learning_rate': 0.0002899734143562476, 'epoch': 0.07}

  7%|▋         | 188/2733 [42:15<9:20:06, 13.20s/it]
  7%|▋         | 189/2733 [42:28<9:24:41, 13.32s/it]
                                                    
{'loss': 1.8023, 'learning_rate': 0.00028985947588302315, 'epoch': 0.07}

  7%|▋         | 189/2733 [42:28<9:24:41, 13.32s/it]
  7%|▋         | 190/2733 [42:41<9:12:30, 13.04s/it]
                                                    
{'loss': 1.6979, 'learning_rate': 0.0002897455374097987, 'epoch': 0.07}

  7%|▋         | 190/2733 [42:41<9:12:30, 13.04s/it]
  7%|▋         | 191/2733 [42:55<9:25:14, 13.34s/it]
                                                    
{'loss': 1.6811, 'learning_rate': 0.0002896315989365742, 'epoch': 0.07}

  7%|▋         | 191/2733 [42:55<9:25:14, 13.34s/it]
  7%|▋         | 192/2733 [43:07<9:10:17, 12.99s/it]
                                                    
{'loss': 1.7455, 'learning_rate': 0.00028951766046334974, 'epoch': 0.07}

  7%|▋         | 192/2733 [43:07<9:10:17, 12.99s/it]
  7%|▋         | 193/2733 [43:21<9:25:54, 13.37s/it]
                                                    
{'loss': 1.7268, 'learning_rate': 0.0002894037219901253, 'epoch': 0.07}

  7%|▋         | 193/2733 [43:21<9:25:54, 13.37s/it]
  7%|▋         | 194/2733 [43:35<9:31:15, 13.50s/it]
                                                    
{'loss': 1.8204, 'learning_rate': 0.00028928978351690086, 'epoch': 0.07}

  7%|▋         | 194/2733 [43:35<9:31:15, 13.50s/it]
  7%|▋         | 195/2733 [43:48<9:26:06, 13.38s/it]
                                                    
{'loss': 1.7269, 'learning_rate': 0.0002891758450436764, 'epoch': 0.07}

  7%|▋         | 195/2733 [43:48<9:26:06, 13.38s/it]
  7%|▋         | 196/2733 [44:02<9:30:10, 13.48s/it]
                                                    
{'loss': 1.8462, 'learning_rate': 0.0002890619065704519, 'epoch': 0.07}

  7%|▋         | 196/2733 [44:02<9:30:10, 13.48s/it]
  7%|▋         | 197/2733 [44:15<9:31:38, 13.52s/it]
                                                    
{'loss': 1.8485, 'learning_rate': 0.00028894796809722745, 'epoch': 0.07}

  7%|▋         | 197/2733 [44:15<9:31:38, 13.52s/it]
  7%|▋         | 198/2733 [44:29<9:38:19, 13.69s/it]
                                                    
{'loss': 1.7518, 'learning_rate': 0.00028883402962400303, 'epoch': 0.07}

  7%|▋         | 198/2733 [44:29<9:38:19, 13.69s/it]
  7%|▋         | 199/2733 [44:43<9:32:42, 13.56s/it]
                                                    
{'loss': 1.8535, 'learning_rate': 0.00028872009115077856, 'epoch': 0.07}

  7%|▋         | 199/2733 [44:43<9:32:42, 13.56s/it]
  7%|▋         | 200/2733 [44:56<9:29:18, 13.49s/it]
                                                    
{'loss': 1.7425, 'learning_rate': 0.0002886061526775541, 'epoch': 0.07}

  7%|▋         | 200/2733 [44:56<9:29:18, 13.49s/it]
  7%|▋         | 201/2733 [45:10<9:29:59, 13.51s/it]
                                                    
{'loss': 1.7749, 'learning_rate': 0.0002884922142043296, 'epoch': 0.07}

  7%|▋         | 201/2733 [45:10<9:29:59, 13.51s/it]
  7%|▋         | 202/2733 [45:23<9:24:45, 13.39s/it]
                                                    
{'loss': 1.6604, 'learning_rate': 0.0002883782757311052, 'epoch': 0.07}

  7%|▋         | 202/2733 [45:23<9:24:45, 13.39s/it]
  7%|▋         | 203/2733 [45:37<9:31:50, 13.56s/it]
                                                    
{'loss': 1.7565, 'learning_rate': 0.00028826433725788074, 'epoch': 0.07}

  7%|▋         | 203/2733 [45:37<9:31:50, 13.56s/it]
  7%|▋         | 204/2733 [45:50<9:30:18, 13.53s/it]
                                                    
{'loss': 1.6889, 'learning_rate': 0.00028815039878465627, 'epoch': 0.07}

  7%|▋         | 204/2733 [45:50<9:30:18, 13.53s/it]
  8%|▊         | 205/2733 [46:02<9:12:38, 13.12s/it]
                                                    
{'loss': 1.6938, 'learning_rate': 0.0002880364603114318, 'epoch': 0.08}

  8%|▊         | 205/2733 [46:02<9:12:38, 13.12s/it]
  8%|▊         | 206/2733 [46:17<9:27:57, 13.49s/it]
                                                    
{'loss': 1.8666, 'learning_rate': 0.00028792252183820733, 'epoch': 0.08}

  8%|▊         | 206/2733 [46:17<9:27:57, 13.49s/it]
  8%|▊         | 207/2733 [46:30<9:31:46, 13.58s/it]
                                                    
{'loss': 1.8571, 'learning_rate': 0.0002878085833649829, 'epoch': 0.08}

  8%|▊         | 207/2733 [46:30<9:31:46, 13.58s/it]
  8%|▊         | 208/2733 [46:43<9:25:18, 13.43s/it]
                                                    
{'loss': 1.7461, 'learning_rate': 0.00028769464489175844, 'epoch': 0.08}

  8%|▊         | 208/2733 [46:43<9:25:18, 13.43s/it]
  8%|▊         | 209/2733 [46:57<9:31:57, 13.60s/it]
                                                    
{'loss': 1.7759, 'learning_rate': 0.000287580706418534, 'epoch': 0.08}

  8%|▊         | 209/2733 [46:57<9:31:57, 13.60s/it]
  8%|▊         | 210/2733 [47:12<9:39:01, 13.77s/it]
                                                    
{'loss': 1.7132, 'learning_rate': 0.0002874667679453095, 'epoch': 0.08}

  8%|▊         | 210/2733 [47:12<9:39:01, 13.77s/it]
  8%|▊         | 211/2733 [47:25<9:37:47, 13.75s/it]
                                                    
{'loss': 1.759, 'learning_rate': 0.00028735282947208503, 'epoch': 0.08}

  8%|▊         | 211/2733 [47:25<9:37:47, 13.75s/it]
  8%|▊         | 212/2733 [47:38<9:29:56, 13.56s/it]
                                                    
{'loss': 1.8104, 'learning_rate': 0.0002872388909988606, 'epoch': 0.08}

  8%|▊         | 212/2733 [47:38<9:29:56, 13.56s/it]
  8%|▊         | 213/2733 [47:52<9:23:45, 13.42s/it]
                                                    
{'loss': 1.8068, 'learning_rate': 0.0002871249525256361, 'epoch': 0.08}

  8%|▊         | 213/2733 [47:52<9:23:45, 13.42s/it]
  8%|▊         | 214/2733 [48:05<9:29:20, 13.56s/it]
                                                    
{'loss': 1.8171, 'learning_rate': 0.0002870110140524117, 'epoch': 0.08}

  8%|▊         | 214/2733 [48:05<9:29:20, 13.56s/it]
  8%|▊         | 215/2733 [48:19<9:28:23, 13.54s/it]
                                                    
{'loss': 1.8313, 'learning_rate': 0.0002868970755791872, 'epoch': 0.08}

  8%|▊         | 215/2733 [48:19<9:28:23, 13.54s/it]
  8%|▊         | 216/2733 [48:32<9:23:38, 13.44s/it]
                                                    
{'loss': 1.7714, 'learning_rate': 0.0002867831371059628, 'epoch': 0.08}

  8%|▊         | 216/2733 [48:32<9:23:38, 13.44s/it]
  8%|▊         | 217/2733 [48:44<9:09:43, 13.11s/it]
                                                    
{'loss': 1.6256, 'learning_rate': 0.00028666919863273827, 'epoch': 0.08}

  8%|▊         | 217/2733 [48:44<9:09:43, 13.11s/it]
  8%|▊         | 218/2733 [48:58<9:15:18, 13.25s/it]
                                                    
{'loss': 1.825, 'learning_rate': 0.0002865552601595138, 'epoch': 0.08}

  8%|▊         | 218/2733 [48:58<9:15:18, 13.25s/it]
  8%|▊         | 219/2733 [49:11<9:16:46, 13.29s/it]
                                                    
{'loss': 1.7357, 'learning_rate': 0.0002864413216862894, 'epoch': 0.08}

  8%|▊         | 219/2733 [49:11<9:16:46, 13.29s/it]
  8%|▊         | 220/2733 [49:24<9:10:01, 13.13s/it]
                                                    
{'loss': 1.7803, 'learning_rate': 0.0002863273832130649, 'epoch': 0.08}

  8%|▊         | 220/2733 [49:24<9:10:01, 13.13s/it]
  8%|▊         | 221/2733 [49:38<9:12:57, 13.21s/it]
                                                    
{'loss': 1.7479, 'learning_rate': 0.00028621344473984045, 'epoch': 0.08}

  8%|▊         | 221/2733 [49:38<9:12:57, 13.21s/it]
  8%|▊         | 222/2733 [49:51<9:20:25, 13.39s/it]
                                                    
{'loss': 1.7371, 'learning_rate': 0.000286099506266616, 'epoch': 0.08}

  8%|▊         | 222/2733 [49:51<9:20:25, 13.39s/it]
  8%|▊         | 223/2733 [50:05<9:27:47, 13.57s/it]
                                                    
{'loss': 1.7041, 'learning_rate': 0.00028598556779339156, 'epoch': 0.08}

  8%|▊         | 223/2733 [50:05<9:27:47, 13.57s/it]
  8%|▊         | 224/2733 [50:19<9:28:39, 13.60s/it]
                                                    
{'loss': 1.863, 'learning_rate': 0.0002858716293201671, 'epoch': 0.08}

  8%|▊         | 224/2733 [50:19<9:28:39, 13.60s/it]
  8%|▊         | 225/2733 [50:33<9:28:58, 13.61s/it]
                                                    
{'loss': 1.7709, 'learning_rate': 0.0002857576908469426, 'epoch': 0.08}

  8%|▊         | 225/2733 [50:33<9:28:58, 13.61s/it]
  8%|▊         | 226/2733 [50:46<9:23:31, 13.49s/it]
                                                    
{'loss': 1.778, 'learning_rate': 0.00028564375237371815, 'epoch': 0.08}

  8%|▊         | 226/2733 [50:46<9:23:31, 13.49s/it]
  8%|▊         | 227/2733 [51:00<9:32:47, 13.71s/it]
                                                    
{'loss': 1.8426, 'learning_rate': 0.0002855298139004937, 'epoch': 0.08}

  8%|▊         | 227/2733 [51:00<9:32:47, 13.71s/it]
  8%|▊         | 228/2733 [51:14<9:32:32, 13.71s/it]
                                                    
{'loss': 1.8941, 'learning_rate': 0.00028541587542726927, 'epoch': 0.08}

  8%|▊         | 228/2733 [51:14<9:32:32, 13.71s/it]
  8%|▊         | 229/2733 [51:27<9:19:47, 13.41s/it]
                                                    
{'loss': 1.8019, 'learning_rate': 0.0002853019369540448, 'epoch': 0.08}

  8%|▊         | 229/2733 [51:27<9:19:47, 13.41s/it]
  8%|▊         | 230/2733 [51:41<9:27:37, 13.61s/it]
                                                    
{'loss': 1.817, 'learning_rate': 0.00028518799848082033, 'epoch': 0.08}

  8%|▊         | 230/2733 [51:41<9:27:37, 13.61s/it]
  8%|▊         | 231/2733 [51:54<9:26:31, 13.59s/it]
                                                    
{'loss': 1.7725, 'learning_rate': 0.00028507406000759586, 'epoch': 0.08}

  8%|▊         | 231/2733 [51:54<9:26:31, 13.59s/it]
  8%|▊         | 232/2733 [52:07<9:20:41, 13.45s/it]
                                                    
{'loss': 1.7392, 'learning_rate': 0.0002849601215343714, 'epoch': 0.08}

  8%|▊         | 232/2733 [52:07<9:20:41, 13.45s/it]
  9%|▊         | 233/2733 [52:20<9:17:23, 13.38s/it]
                                                    
{'loss': 1.8028, 'learning_rate': 0.000284846183061147, 'epoch': 0.09}

  9%|▊         | 233/2733 [52:20<9:17:23, 13.38s/it]
  9%|▊         | 234/2733 [52:33<9:10:43, 13.22s/it]
                                                    
{'loss': 1.7354, 'learning_rate': 0.0002847322445879225, 'epoch': 0.09}

  9%|▊         | 234/2733 [52:33<9:10:43, 13.22s/it]
  9%|▊         | 235/2733 [52:47<9:17:16, 13.39s/it]
                                                    
{'loss': 1.8408, 'learning_rate': 0.00028461830611469803, 'epoch': 0.09}

  9%|▊         | 235/2733 [52:47<9:17:16, 13.39s/it]
  9%|▊         | 236/2733 [53:00<9:17:32, 13.40s/it]
                                                    
{'loss': 1.7349, 'learning_rate': 0.00028450436764147356, 'epoch': 0.09}

  9%|▊         | 236/2733 [53:01<9:17:32, 13.40s/it]
  9%|▊         | 237/2733 [53:13<9:12:08, 13.27s/it]
                                                    
{'loss': 1.7493, 'learning_rate': 0.00028439042916824915, 'epoch': 0.09}

  9%|▊         | 237/2733 [53:14<9:12:08, 13.27s/it]
  9%|▊         | 238/2733 [53:27<9:15:04, 13.35s/it]
                                                    
{'loss': 1.7577, 'learning_rate': 0.0002842764906950247, 'epoch': 0.09}

  9%|▊         | 238/2733 [53:27<9:15:04, 13.35s/it]
  9%|▊         | 239/2733 [53:41<9:20:02, 13.47s/it]
                                                    
{'loss': 1.7983, 'learning_rate': 0.0002841625522218002, 'epoch': 0.09}

  9%|▊         | 239/2733 [53:41<9:20:02, 13.47s/it]
  9%|▉         | 240/2733 [53:54<9:19:32, 13.47s/it]
                                                    
{'loss': 1.6857, 'learning_rate': 0.00028404861374857574, 'epoch': 0.09}

  9%|▉         | 240/2733 [53:54<9:19:32, 13.47s/it]
  9%|▉         | 241/2733 [54:08<9:17:39, 13.43s/it]
                                                    
{'loss': 1.8006, 'learning_rate': 0.00028393467527535127, 'epoch': 0.09}

  9%|▉         | 241/2733 [54:08<9:17:39, 13.43s/it]
  9%|▉         | 242/2733 [54:20<9:07:53, 13.20s/it]
                                                    
{'loss': 1.7933, 'learning_rate': 0.00028382073680212686, 'epoch': 0.09}

  9%|▉         | 242/2733 [54:20<9:07:53, 13.20s/it]
  9%|▉         | 243/2733 [54:33<9:06:29, 13.17s/it]
                                                    
{'loss': 1.7771, 'learning_rate': 0.0002837067983289024, 'epoch': 0.09}

  9%|▉         | 243/2733 [54:33<9:06:29, 13.17s/it]
  9%|▉         | 244/2733 [54:46<9:03:25, 13.10s/it]
                                                    
{'loss': 1.7825, 'learning_rate': 0.0002835928598556779, 'epoch': 0.09}

  9%|▉         | 244/2733 [54:46<9:03:25, 13.10s/it]
  9%|▉         | 245/2733 [55:00<9:09:40, 13.26s/it]
                                                    
{'loss': 1.8299, 'learning_rate': 0.00028347892138245345, 'epoch': 0.09}

  9%|▉         | 245/2733 [55:00<9:09:40, 13.26s/it]
  9%|▉         | 246/2733 [55:13<9:06:52, 13.19s/it]
                                                    
{'loss': 1.7099, 'learning_rate': 0.000283364982909229, 'epoch': 0.09}

  9%|▉         | 246/2733 [55:13<9:06:52, 13.19s/it]
  9%|▉         | 247/2733 [55:26<8:59:15, 13.02s/it]
                                                    
{'loss': 1.7146, 'learning_rate': 0.00028325104443600456, 'epoch': 0.09}

  9%|▉         | 247/2733 [55:26<8:59:15, 13.02s/it]
  9%|▉         | 248/2733 [55:39<8:58:48, 13.01s/it]
                                                    
{'loss': 1.8184, 'learning_rate': 0.00028313710596278004, 'epoch': 0.09}

  9%|▉         | 248/2733 [55:39<8:58:48, 13.01s/it]
  9%|▉         | 249/2733 [55:52<9:05:19, 13.17s/it]
                                                    
{'loss': 1.759, 'learning_rate': 0.0002830231674895556, 'epoch': 0.09}

  9%|▉         | 249/2733 [55:52<9:05:19, 13.17s/it]
  9%|▉         | 250/2733 [56:06<9:08:50, 13.26s/it]
                                                    
{'loss': 1.7911, 'learning_rate': 0.00028290922901633115, 'epoch': 0.09}

  9%|▉         | 250/2733 [56:06<9:08:50, 13.26s/it]
  9%|▉         | 251/2733 [56:19<9:06:39, 13.21s/it]
                                                    
{'loss': 1.7403, 'learning_rate': 0.00028279529054310674, 'epoch': 0.09}

  9%|▉         | 251/2733 [56:19<9:06:39, 13.21s/it]
  9%|▉         | 252/2733 [56:32<9:09:11, 13.28s/it]
                                                    
{'loss': 1.7725, 'learning_rate': 0.0002826813520698822, 'epoch': 0.09}

  9%|▉         | 252/2733 [56:32<9:09:11, 13.28s/it]
  9%|▉         | 253/2733 [56:45<9:05:57, 13.21s/it]
                                                    
{'loss': 1.7321, 'learning_rate': 0.00028256741359665774, 'epoch': 0.09}

  9%|▉         | 253/2733 [56:45<9:05:57, 13.21s/it]
  9%|▉         | 254/2733 [56:59<9:12:26, 13.37s/it]
                                                    
{'loss': 1.7941, 'learning_rate': 0.00028245347512343333, 'epoch': 0.09}

  9%|▉         | 254/2733 [56:59<9:12:26, 13.37s/it]
  9%|▉         | 255/2733 [57:13<9:15:57, 13.46s/it]
                                                    
{'loss': 1.8254, 'learning_rate': 0.00028233953665020886, 'epoch': 0.09}

  9%|▉         | 255/2733 [57:13<9:15:57, 13.46s/it]
  9%|▉         | 256/2733 [57:27<9:24:34, 13.68s/it]
                                                    
{'loss': 1.7755, 'learning_rate': 0.0002822255981769844, 'epoch': 0.09}

  9%|▉         | 256/2733 [57:27<9:24:34, 13.68s/it]
  9%|▉         | 257/2733 [57:40<9:22:36, 13.63s/it]
                                                    
{'loss': 1.7618, 'learning_rate': 0.0002821116597037599, 'epoch': 0.09}

  9%|▉         | 257/2733 [57:40<9:22:36, 13.63s/it]
  9%|▉         | 258/2733 [57:53<9:17:17, 13.51s/it]
                                                    
{'loss': 1.7098, 'learning_rate': 0.0002819977212305355, 'epoch': 0.09}

  9%|▉         | 258/2733 [57:54<9:17:17, 13.51s/it]
  9%|▉         | 259/2733 [58:07<9:12:06, 13.39s/it]
                                                    
{'loss': 1.855, 'learning_rate': 0.00028188378275731103, 'epoch': 0.09}

  9%|▉         | 259/2733 [58:07<9:12:06, 13.39s/it]
 10%|▉         | 260/2733 [58:20<9:14:04, 13.44s/it]
                                                    
{'loss': 1.7296, 'learning_rate': 0.00028176984428408656, 'epoch': 0.1}

 10%|▉         | 260/2733 [58:20<9:14:04, 13.44s/it]
 10%|▉         | 261/2733 [58:34<9:23:28, 13.68s/it]
                                                    
{'loss': 1.8668, 'learning_rate': 0.0002816559058108621, 'epoch': 0.1}

 10%|▉         | 261/2733 [58:34<9:23:28, 13.68s/it]
 10%|▉         | 262/2733 [58:48<9:18:48, 13.57s/it]
                                                    
{'loss': 1.7894, 'learning_rate': 0.0002815419673376376, 'epoch': 0.1}

 10%|▉         | 262/2733 [58:48<9:18:48, 13.57s/it]
 10%|▉         | 263/2733 [59:01<9:15:53, 13.50s/it]
                                                    
{'loss': 1.7195, 'learning_rate': 0.0002814280288644132, 'epoch': 0.1}

 10%|▉         | 263/2733 [59:01<9:15:53, 13.50s/it]
 10%|▉         | 264/2733 [59:15<9:15:17, 13.49s/it]
                                                    
{'loss': 1.81, 'learning_rate': 0.00028131409039118874, 'epoch': 0.1}

 10%|▉         | 264/2733 [59:15<9:15:17, 13.49s/it]
 10%|▉         | 265/2733 [59:28<9:12:42, 13.44s/it]
                                                    
{'loss': 1.737, 'learning_rate': 0.00028120015191796427, 'epoch': 0.1}

 10%|▉         | 265/2733 [59:28<9:12:42, 13.44s/it]
 10%|▉         | 266/2733 [59:42<9:15:41, 13.52s/it]
                                                    
{'loss': 1.8219, 'learning_rate': 0.0002810862134447398, 'epoch': 0.1}

 10%|▉         | 266/2733 [59:42<9:15:41, 13.52s/it]
 10%|▉         | 267/2733 [59:55<9:17:41, 13.57s/it]
                                                    
{'loss': 1.7821, 'learning_rate': 0.00028097227497151533, 'epoch': 0.1}

 10%|▉         | 267/2733 [59:55<9:17:41, 13.57s/it]
 10%|▉         | 268/2733 [1:00:09<9:15:48, 13.53s/it]
                                                      
{'loss': 1.7362, 'learning_rate': 0.0002808583364982909, 'epoch': 0.1}

 10%|▉         | 268/2733 [1:00:09<9:15:48, 13.53s/it]
 10%|▉         | 269/2733 [1:00:22<9:17:41, 13.58s/it]
                                                      
{'loss': 1.7531, 'learning_rate': 0.00028074439802506645, 'epoch': 0.1}

 10%|▉         | 269/2733 [1:00:22<9:17:41, 13.58s/it]
 10%|▉         | 270/2733 [1:00:36<9:19:11, 13.62s/it]
                                                      
{'loss': 1.7379, 'learning_rate': 0.000280630459551842, 'epoch': 0.1}

 10%|▉         | 270/2733 [1:00:36<9:19:11, 13.62s/it]
 10%|▉         | 271/2733 [1:00:48<9:04:04, 13.26s/it]
                                                      
{'loss': 1.6965, 'learning_rate': 0.0002805165210786175, 'epoch': 0.1}

 10%|▉         | 271/2733 [1:00:49<9:04:04, 13.26s/it]
 10%|▉         | 272/2733 [1:01:02<9:08:23, 13.37s/it]
                                                      
{'loss': 1.7403, 'learning_rate': 0.0002804025826053931, 'epoch': 0.1}

 10%|▉         | 272/2733 [1:01:02<9:08:23, 13.37s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 10%|▉         | 273/2733 [1:01:17<9:23:21, 13.74s/it]
                                                      
{'loss': 1.7683, 'learning_rate': 0.0002802886441321686, 'epoch': 0.1}

 10%|▉         | 273/2733 [1:01:17<9:23:21, 13.74s/it]
 10%|█         | 274/2733 [1:01:30<9:15:25, 13.55s/it]
                                                      
{'loss': 1.8004, 'learning_rate': 0.00028017470565894415, 'epoch': 0.1}

 10%|█         | 274/2733 [1:01:30<9:15:25, 13.55s/it]
 10%|█         | 275/2733 [1:01:43<9:12:24, 13.48s/it]
                                                      
{'loss': 1.8728, 'learning_rate': 0.0002800607671857197, 'epoch': 0.1}

 10%|█         | 275/2733 [1:01:43<9:12:24, 13.48s/it]
 10%|█         | 276/2733 [1:01:57<9:13:53, 13.53s/it]
                                                      
{'loss': 1.7199, 'learning_rate': 0.0002799468287124952, 'epoch': 0.1}

 10%|█         | 276/2733 [1:01:57<9:13:53, 13.53s/it]
 10%|█         | 277/2733 [1:02:11<9:17:02, 13.61s/it]
                                                      
{'loss': 1.7603, 'learning_rate': 0.0002798328902392708, 'epoch': 0.1}

 10%|█         | 277/2733 [1:02:11<9:17:02, 13.61s/it]
 10%|█         | 278/2733 [1:02:24<9:13:20, 13.52s/it]
                                                      
{'loss': 1.735, 'learning_rate': 0.0002797189517660463, 'epoch': 0.1}

 10%|█         | 278/2733 [1:02:24<9:13:20, 13.52s/it]
 10%|█         | 279/2733 [1:02:36<8:56:46, 13.12s/it]
                                                      
{'loss': 1.6982, 'learning_rate': 0.00027960501329282186, 'epoch': 0.1}

 10%|█         | 279/2733 [1:02:36<8:56:46, 13.12s/it]
 10%|█         | 280/2733 [1:02:49<8:57:42, 13.15s/it]
                                                      
{'loss': 1.7285, 'learning_rate': 0.0002794910748195974, 'epoch': 0.1}

 10%|█         | 280/2733 [1:02:49<8:57:42, 13.15s/it]
 10%|█         | 281/2733 [1:03:03<8:59:02, 13.19s/it]
                                                      
{'loss': 1.7205, 'learning_rate': 0.0002793771363463729, 'epoch': 0.1}

 10%|█         | 281/2733 [1:03:03<8:59:02, 13.19s/it]
 10%|█         | 282/2733 [1:03:15<8:51:49, 13.02s/it]
                                                      
{'loss': 1.7943, 'learning_rate': 0.0002792631978731485, 'epoch': 0.1}

 10%|█         | 282/2733 [1:03:15<8:51:49, 13.02s/it]
 10%|█         | 283/2733 [1:03:29<8:54:59, 13.10s/it]
                                                      
{'loss': 1.8138, 'learning_rate': 0.00027914925939992403, 'epoch': 0.1}

 10%|█         | 283/2733 [1:03:29<8:54:59, 13.10s/it]
 10%|█         | 284/2733 [1:03:41<8:47:53, 12.93s/it]
                                                      
{'loss': 1.685, 'learning_rate': 0.00027903532092669956, 'epoch': 0.1}

 10%|█         | 284/2733 [1:03:41<8:47:53, 12.93s/it]
 10%|█         | 285/2733 [1:03:55<8:59:29, 13.22s/it]
                                                      
{'loss': 1.8198, 'learning_rate': 0.0002789213824534751, 'epoch': 0.1}

 10%|█         | 285/2733 [1:03:55<8:59:29, 13.22s/it]
 10%|█         | 286/2733 [1:04:08<8:54:09, 13.10s/it]
                                                      
{'loss': 1.6721, 'learning_rate': 0.0002788074439802507, 'epoch': 0.1}

 10%|█         | 286/2733 [1:04:08<8:54:09, 13.10s/it]
 11%|█         | 287/2733 [1:04:21<9:01:23, 13.28s/it]
                                                      
{'loss': 1.7606, 'learning_rate': 0.0002786935055070262, 'epoch': 0.11}

 11%|█         | 287/2733 [1:04:21<9:01:23, 13.28s/it]
 11%|█         | 288/2733 [1:04:35<9:01:19, 13.28s/it]
                                                      
{'loss': 1.7744, 'learning_rate': 0.0002785795670338017, 'epoch': 0.11}

 11%|█         | 288/2733 [1:04:35<9:01:19, 13.28s/it]
 11%|█         | 289/2733 [1:04:48<8:55:12, 13.14s/it]
                                                      
{'loss': 1.7444, 'learning_rate': 0.00027846562856057727, 'epoch': 0.11}

 11%|█         | 289/2733 [1:04:48<8:55:12, 13.14s/it]
 11%|█         | 290/2733 [1:05:01<8:59:09, 13.24s/it]
                                                      
{'loss': 1.7105, 'learning_rate': 0.0002783516900873528, 'epoch': 0.11}

 11%|█         | 290/2733 [1:05:01<8:59:09, 13.24s/it]
 11%|█         | 291/2733 [1:05:14<8:59:50, 13.26s/it]
                                                      
{'loss': 1.8416, 'learning_rate': 0.00027823775161412833, 'epoch': 0.11}

 11%|█         | 291/2733 [1:05:14<8:59:50, 13.26s/it]
 11%|█         | 292/2733 [1:05:28<9:02:04, 13.32s/it]
                                                      
{'loss': 1.7632, 'learning_rate': 0.00027812381314090386, 'epoch': 0.11}

 11%|█         | 292/2733 [1:05:28<9:02:04, 13.32s/it]
 11%|█         | 293/2733 [1:05:40<8:53:34, 13.12s/it]
                                                      
{'loss': 1.8289, 'learning_rate': 0.00027800987466767944, 'epoch': 0.11}

 11%|█         | 293/2733 [1:05:40<8:53:34, 13.12s/it]
 11%|█         | 294/2733 [1:05:54<8:55:29, 13.17s/it]
                                                      
{'loss': 1.9107, 'learning_rate': 0.000277895936194455, 'epoch': 0.11}

 11%|█         | 294/2733 [1:05:54<8:55:29, 13.17s/it]
 11%|█         | 295/2733 [1:06:07<9:01:56, 13.34s/it]
                                                      
{'loss': 1.7231, 'learning_rate': 0.0002777819977212305, 'epoch': 0.11}

 11%|█         | 295/2733 [1:06:07<9:01:56, 13.34s/it]
 11%|█         | 296/2733 [1:06:21<9:06:54, 13.47s/it]
                                                      
{'loss': 1.7336, 'learning_rate': 0.00027766805924800604, 'epoch': 0.11}

 11%|█         | 296/2733 [1:06:21<9:06:54, 13.47s/it]
 11%|█         | 297/2733 [1:06:35<9:04:17, 13.41s/it]
                                                      
{'loss': 1.8784, 'learning_rate': 0.00027755412077478157, 'epoch': 0.11}

 11%|█         | 297/2733 [1:06:35<9:04:17, 13.41s/it]
 11%|█         | 298/2733 [1:06:48<9:06:47, 13.47s/it]
                                                      
{'loss': 1.7032, 'learning_rate': 0.00027744018230155715, 'epoch': 0.11}

 11%|█         | 298/2733 [1:06:48<9:06:47, 13.47s/it]
 11%|█         | 299/2733 [1:07:01<9:03:09, 13.39s/it]
                                                      
{'loss': 1.7846, 'learning_rate': 0.0002773262438283327, 'epoch': 0.11}

 11%|█         | 299/2733 [1:07:01<9:03:09, 13.39s/it]
 11%|█         | 300/2733 [1:07:15<9:03:51, 13.41s/it]
                                                      
{'loss': 1.8541, 'learning_rate': 0.0002772123053551082, 'epoch': 0.11}

 11%|█         | 300/2733 [1:07:15<9:03:51, 13.41s/it]
 11%|█         | 301/2733 [1:07:28<9:03:56, 13.42s/it]
                                                      
{'loss': 1.82, 'learning_rate': 0.00027709836688188374, 'epoch': 0.11}

 11%|█         | 301/2733 [1:07:28<9:03:56, 13.42s/it]
 11%|█         | 302/2733 [1:07:41<9:01:23, 13.36s/it]
                                                      
{'loss': 1.6969, 'learning_rate': 0.00027698442840865927, 'epoch': 0.11}

 11%|█         | 302/2733 [1:07:41<9:01:23, 13.36s/it]
 11%|█         | 303/2733 [1:07:56<9:12:12, 13.63s/it]
                                                      
{'loss': 1.7707, 'learning_rate': 0.00027687048993543486, 'epoch': 0.11}

 11%|█         | 303/2733 [1:07:56<9:12:12, 13.63s/it]
 11%|█         | 304/2733 [1:08:09<9:10:12, 13.59s/it]
                                                      
{'loss': 1.615, 'learning_rate': 0.0002767565514622104, 'epoch': 0.11}

 11%|█         | 304/2733 [1:08:09<9:10:12, 13.59s/it]
 11%|█         | 305/2733 [1:08:22<9:03:54, 13.44s/it]
                                                      
{'loss': 1.7696, 'learning_rate': 0.0002766426129889859, 'epoch': 0.11}

 11%|█         | 305/2733 [1:08:22<9:03:54, 13.44s/it]
 11%|█         | 306/2733 [1:08:35<8:59:09, 13.33s/it]
                                                      
{'loss': 1.7753, 'learning_rate': 0.00027652867451576145, 'epoch': 0.11}

 11%|█         | 306/2733 [1:08:35<8:59:09, 13.33s/it]
 11%|█         | 307/2733 [1:08:49<8:56:28, 13.27s/it]
                                                      
{'loss': 1.7281, 'learning_rate': 0.00027641473604253703, 'epoch': 0.11}

 11%|█         | 307/2733 [1:08:49<8:56:28, 13.27s/it]
 11%|█▏        | 308/2733 [1:09:02<9:04:12, 13.46s/it]
                                                      
{'loss': 1.7984, 'learning_rate': 0.00027630079756931256, 'epoch': 0.11}

 11%|█▏        | 308/2733 [1:09:02<9:04:12, 13.46s/it]
 11%|█▏        | 309/2733 [1:09:16<9:04:08, 13.47s/it]
                                                      
{'loss': 1.6804, 'learning_rate': 0.0002761868590960881, 'epoch': 0.11}

 11%|█▏        | 309/2733 [1:09:16<9:04:08, 13.47s/it]
 11%|█▏        | 310/2733 [1:09:29<8:55:42, 13.27s/it]
                                                      
{'loss': 1.8361, 'learning_rate': 0.0002760729206228636, 'epoch': 0.11}

 11%|█▏        | 310/2733 [1:09:29<8:55:42, 13.27s/it]
 11%|█▏        | 311/2733 [1:09:42<8:50:34, 13.14s/it]
                                                      
{'loss': 1.7459, 'learning_rate': 0.00027595898214963915, 'epoch': 0.11}

 11%|█▏        | 311/2733 [1:09:42<8:50:34, 13.14s/it]
 11%|█▏        | 312/2733 [1:09:56<9:03:27, 13.47s/it]
                                                      
{'loss': 1.7444, 'learning_rate': 0.00027584504367641474, 'epoch': 0.11}

 11%|█▏        | 312/2733 [1:09:56<9:03:27, 13.47s/it]
 11%|█▏        | 313/2733 [1:10:09<8:56:31, 13.30s/it]
                                                      
{'loss': 1.6821, 'learning_rate': 0.00027573110520319027, 'epoch': 0.11}

 11%|█▏        | 313/2733 [1:10:09<8:56:31, 13.30s/it]
 11%|█▏        | 314/2733 [1:10:21<8:40:42, 12.92s/it]
                                                      
{'loss': 1.7523, 'learning_rate': 0.0002756171667299658, 'epoch': 0.11}

 11%|█▏        | 314/2733 [1:10:21<8:40:42, 12.92s/it]
 12%|█▏        | 315/2733 [1:10:35<8:51:24, 13.19s/it]
                                                      
{'loss': 1.7483, 'learning_rate': 0.00027550322825674133, 'epoch': 0.12}

 12%|█▏        | 315/2733 [1:10:35<8:51:24, 13.19s/it]
 12%|█▏        | 316/2733 [1:10:48<8:50:55, 13.18s/it]
                                                      
{'loss': 1.7943, 'learning_rate': 0.00027538928978351686, 'epoch': 0.12}

 12%|█▏        | 316/2733 [1:10:48<8:50:55, 13.18s/it]
 12%|█▏        | 317/2733 [1:11:01<8:46:21, 13.07s/it]
                                                      
{'loss': 1.6588, 'learning_rate': 0.00027527535131029244, 'epoch': 0.12}

 12%|█▏        | 317/2733 [1:11:01<8:46:21, 13.07s/it]
 12%|█▏        | 318/2733 [1:11:15<8:59:53, 13.41s/it]
                                                      
{'loss': 1.7867, 'learning_rate': 0.000275161412837068, 'epoch': 0.12}

 12%|█▏        | 318/2733 [1:11:15<8:59:53, 13.41s/it]
 12%|█▏        | 319/2733 [1:11:28<8:57:03, 13.35s/it]
                                                      
{'loss': 1.7509, 'learning_rate': 0.0002750474743638435, 'epoch': 0.12}

 12%|█▏        | 319/2733 [1:11:28<8:57:03, 13.35s/it]
 12%|█▏        | 320/2733 [1:11:42<9:05:15, 13.56s/it]
                                                      
{'loss': 1.807, 'learning_rate': 0.00027493353589061904, 'epoch': 0.12}

 12%|█▏        | 320/2733 [1:11:42<9:05:15, 13.56s/it]
 12%|█▏        | 321/2733 [1:11:56<9:06:54, 13.60s/it]
                                                      
{'loss': 1.7456, 'learning_rate': 0.0002748195974173946, 'epoch': 0.12}

 12%|█▏        | 321/2733 [1:11:56<9:06:54, 13.60s/it]
 12%|█▏        | 322/2733 [1:12:09<9:07:18, 13.62s/it]
                                                      
{'loss': 1.6723, 'learning_rate': 0.00027470565894417015, 'epoch': 0.12}

 12%|█▏        | 322/2733 [1:12:09<9:07:18, 13.62s/it]
 12%|█▏        | 323/2733 [1:12:22<9:00:05, 13.45s/it]
                                                      
{'loss': 1.6345, 'learning_rate': 0.0002745917204709456, 'epoch': 0.12}

 12%|█▏        | 323/2733 [1:12:22<9:00:05, 13.45s/it]
 12%|█▏        | 324/2733 [1:12:36<9:02:10, 13.50s/it]
                                                      
{'loss': 1.7917, 'learning_rate': 0.0002744777819977212, 'epoch': 0.12}

 12%|█▏        | 324/2733 [1:12:36<9:02:10, 13.50s/it]
 12%|█▏        | 325/2733 [1:12:49<8:53:59, 13.31s/it]
                                                      
{'loss': 1.7026, 'learning_rate': 0.00027436384352449674, 'epoch': 0.12}

 12%|█▏        | 325/2733 [1:12:49<8:53:59, 13.31s/it]
 12%|█▏        | 326/2733 [1:13:02<8:51:39, 13.25s/it]
                                                      
{'loss': 1.7266, 'learning_rate': 0.0002742499050512723, 'epoch': 0.12}

 12%|█▏        | 326/2733 [1:13:02<8:51:39, 13.25s/it]
 12%|█▏        | 327/2733 [1:13:16<8:55:55, 13.36s/it]
                                                      
{'loss': 1.7394, 'learning_rate': 0.0002741359665780478, 'epoch': 0.12}

 12%|█▏        | 327/2733 [1:13:16<8:55:55, 13.36s/it]
 12%|█▏        | 328/2733 [1:13:29<8:54:33, 13.34s/it]
                                                      
{'loss': 1.7989, 'learning_rate': 0.0002740220281048234, 'epoch': 0.12}

 12%|█▏        | 328/2733 [1:13:29<8:54:33, 13.34s/it]
 12%|█▏        | 329/2733 [1:13:42<8:48:40, 13.19s/it]
                                                      
{'loss': 1.7434, 'learning_rate': 0.0002739080896315989, 'epoch': 0.12}

 12%|█▏        | 329/2733 [1:13:42<8:48:40, 13.19s/it]
 12%|█▏        | 330/2733 [1:13:55<8:45:13, 13.11s/it]
                                                      
{'loss': 1.7876, 'learning_rate': 0.00027379415115837445, 'epoch': 0.12}

 12%|█▏        | 330/2733 [1:13:55<8:45:13, 13.11s/it]
 12%|█▏        | 331/2733 [1:14:07<8:36:35, 12.90s/it]
                                                      
{'loss': 1.7252, 'learning_rate': 0.00027368021268515, 'epoch': 0.12}

 12%|█▏        | 331/2733 [1:14:07<8:36:35, 12.90s/it]
 12%|█▏        | 332/2733 [1:14:20<8:33:34, 12.83s/it]
                                                      
{'loss': 1.793, 'learning_rate': 0.0002735662742119255, 'epoch': 0.12}

 12%|█▏        | 332/2733 [1:14:20<8:33:34, 12.83s/it]
 12%|█▏        | 333/2733 [1:14:33<8:36:31, 12.91s/it]
                                                      
{'loss': 1.7635, 'learning_rate': 0.0002734523357387011, 'epoch': 0.12}

 12%|█▏        | 333/2733 [1:14:33<8:36:31, 12.91s/it]
 12%|█▏        | 334/2733 [1:14:47<8:46:13, 13.16s/it]
                                                      
{'loss': 1.7527, 'learning_rate': 0.0002733383972654766, 'epoch': 0.12}

 12%|█▏        | 334/2733 [1:14:47<8:46:13, 13.16s/it]
 12%|█▏        | 335/2733 [1:14:59<8:37:57, 12.96s/it]
                                                      
{'loss': 1.7343, 'learning_rate': 0.00027322445879225215, 'epoch': 0.12}

 12%|█▏        | 335/2733 [1:14:59<8:37:57, 12.96s/it]
 12%|█▏        | 336/2733 [1:15:13<8:43:12, 13.10s/it]
                                                      
{'loss': 1.816, 'learning_rate': 0.0002731105203190277, 'epoch': 0.12}

 12%|█▏        | 336/2733 [1:15:13<8:43:12, 13.10s/it]
 12%|█▏        | 337/2733 [1:15:26<8:44:25, 13.13s/it]
                                                      
{'loss': 1.8052, 'learning_rate': 0.0002729965818458032, 'epoch': 0.12}

 12%|█▏        | 337/2733 [1:15:26<8:44:25, 13.13s/it]
 12%|█▏        | 338/2733 [1:15:39<8:45:53, 13.17s/it]
                                                      
{'loss': 1.8104, 'learning_rate': 0.0002728826433725788, 'epoch': 0.12}

 12%|█▏        | 338/2733 [1:15:39<8:45:53, 13.17s/it]
 12%|█▏        | 339/2733 [1:15:52<8:46:48, 13.20s/it]
                                                      
{'loss': 1.7082, 'learning_rate': 0.00027276870489935433, 'epoch': 0.12}

 12%|█▏        | 339/2733 [1:15:52<8:46:48, 13.20s/it]
 12%|█▏        | 340/2733 [1:16:06<8:48:28, 13.25s/it]
                                                      
{'loss': 1.7132, 'learning_rate': 0.00027265476642612986, 'epoch': 0.12}

 12%|█▏        | 340/2733 [1:16:06<8:48:28, 13.25s/it]
 12%|█▏        | 341/2733 [1:16:18<8:40:49, 13.06s/it]
                                                      
{'loss': 1.7723, 'learning_rate': 0.0002725408279529054, 'epoch': 0.12}

 12%|█▏        | 341/2733 [1:16:18<8:40:49, 13.06s/it]
 13%|█▎        | 342/2733 [1:16:31<8:42:12, 13.10s/it]
                                                      
{'loss': 1.759, 'learning_rate': 0.0002724268894796809, 'epoch': 0.13}

 13%|█▎        | 342/2733 [1:16:31<8:42:12, 13.10s/it]
 13%|█▎        | 343/2733 [1:16:45<8:46:00, 13.21s/it]
                                                      
{'loss': 1.8012, 'learning_rate': 0.0002723129510064565, 'epoch': 0.13}

 13%|█▎        | 343/2733 [1:16:45<8:46:00, 13.21s/it]
 13%|█▎        | 344/2733 [1:16:58<8:40:01, 13.06s/it]
                                                      
{'loss': 1.766, 'learning_rate': 0.00027219901253323203, 'epoch': 0.13}

 13%|█▎        | 344/2733 [1:16:58<8:40:01, 13.06s/it]
 13%|█▎        | 345/2733 [1:17:11<8:40:51, 13.09s/it]
                                                      
{'loss': 1.7775, 'learning_rate': 0.00027208507406000757, 'epoch': 0.13}

 13%|█▎        | 345/2733 [1:17:11<8:40:51, 13.09s/it]
 13%|█▎        | 346/2733 [1:17:25<8:50:09, 13.33s/it]
                                                      
{'loss': 1.7736, 'learning_rate': 0.0002719711355867831, 'epoch': 0.13}

 13%|█▎        | 346/2733 [1:17:25<8:50:09, 13.33s/it]
 13%|█▎        | 347/2733 [1:17:39<8:56:49, 13.50s/it]
                                                      
{'loss': 1.8196, 'learning_rate': 0.0002718571971135587, 'epoch': 0.13}

 13%|█▎        | 347/2733 [1:17:39<8:56:49, 13.50s/it]
 13%|█▎        | 348/2733 [1:17:53<9:04:15, 13.69s/it]
                                                      
{'loss': 1.8046, 'learning_rate': 0.0002717432586403342, 'epoch': 0.13}

 13%|█▎        | 348/2733 [1:17:53<9:04:15, 13.69s/it]
 13%|█▎        | 349/2733 [1:18:06<9:01:00, 13.62s/it]
                                                      
{'loss': 1.6524, 'learning_rate': 0.00027162932016710974, 'epoch': 0.13}

 13%|█▎        | 349/2733 [1:18:06<9:01:00, 13.62s/it]
 13%|█▎        | 350/2733 [1:18:20<9:07:58, 13.80s/it]
                                                      
{'loss': 1.6435, 'learning_rate': 0.00027151538169388527, 'epoch': 0.13}

 13%|█▎        | 350/2733 [1:18:20<9:07:58, 13.80s/it]
 13%|█▎        | 351/2733 [1:18:34<9:05:19, 13.74s/it]
                                                      
{'loss': 1.7516, 'learning_rate': 0.0002714014432206608, 'epoch': 0.13}

 13%|█▎        | 351/2733 [1:18:34<9:05:19, 13.74s/it]
 13%|█▎        | 352/2733 [1:18:47<8:54:43, 13.47s/it]
                                                      
{'loss': 1.6681, 'learning_rate': 0.0002712875047474364, 'epoch': 0.13}

 13%|█▎        | 352/2733 [1:18:47<8:54:43, 13.47s/it]
 13%|█▎        | 353/2733 [1:19:00<8:50:37, 13.38s/it]
                                                      
{'loss': 1.7367, 'learning_rate': 0.0002711735662742119, 'epoch': 0.13}

 13%|█▎        | 353/2733 [1:19:00<8:50:37, 13.38s/it]
 13%|█▎        | 354/2733 [1:19:13<8:47:21, 13.30s/it]
                                                      
{'loss': 1.709, 'learning_rate': 0.00027105962780098745, 'epoch': 0.13}

 13%|█▎        | 354/2733 [1:19:13<8:47:21, 13.30s/it]
 13%|█▎        | 355/2733 [1:19:26<8:47:45, 13.32s/it]
                                                      
{'loss': 1.6728, 'learning_rate': 0.000270945689327763, 'epoch': 0.13}

 13%|█▎        | 355/2733 [1:19:26<8:47:45, 13.32s/it]
 13%|█▎        | 356/2733 [1:19:41<8:58:52, 13.60s/it]
                                                      
{'loss': 1.7263, 'learning_rate': 0.0002708317508545385, 'epoch': 0.13}

 13%|█▎        | 356/2733 [1:19:41<8:58:52, 13.60s/it]
 13%|█▎        | 357/2733 [1:19:55<9:04:23, 13.75s/it]
                                                      
{'loss': 1.7112, 'learning_rate': 0.0002707178123813141, 'epoch': 0.13}

 13%|█▎        | 357/2733 [1:19:55<9:04:23, 13.75s/it]
 13%|█▎        | 358/2733 [1:20:08<8:56:16, 13.55s/it]
                                                      
{'loss': 1.7529, 'learning_rate': 0.0002706038739080896, 'epoch': 0.13}

 13%|█▎        | 358/2733 [1:20:08<8:56:16, 13.55s/it]
 13%|█▎        | 359/2733 [1:20:21<8:49:08, 13.37s/it]
                                                      
{'loss': 1.7128, 'learning_rate': 0.00027048993543486515, 'epoch': 0.13}

 13%|█▎        | 359/2733 [1:20:21<8:49:08, 13.37s/it]
 13%|█▎        | 360/2733 [1:20:35<8:57:32, 13.59s/it]
                                                      
{'loss': 1.7162, 'learning_rate': 0.0002703759969616407, 'epoch': 0.13}

 13%|█▎        | 360/2733 [1:20:35<8:57:32, 13.59s/it]
 13%|█▎        | 361/2733 [1:20:49<8:57:40, 13.60s/it]
                                                      
{'loss': 1.8928, 'learning_rate': 0.00027026205848841627, 'epoch': 0.13}

 13%|█▎        | 361/2733 [1:20:49<8:57:40, 13.60s/it]
 13%|█▎        | 362/2733 [1:21:02<9:00:20, 13.67s/it]
                                                      
{'loss': 1.8807, 'learning_rate': 0.0002701481200151918, 'epoch': 0.13}

 13%|█▎        | 362/2733 [1:21:02<9:00:20, 13.67s/it]
 13%|█▎        | 363/2733 [1:21:15<8:52:04, 13.47s/it]
                                                      
{'loss': 1.7158, 'learning_rate': 0.0002700341815419673, 'epoch': 0.13}

 13%|█▎        | 363/2733 [1:21:15<8:52:04, 13.47s/it]
 13%|█▎        | 364/2733 [1:21:29<8:57:24, 13.61s/it]
                                                      
{'loss': 1.7152, 'learning_rate': 0.00026992024306874286, 'epoch': 0.13}

 13%|█▎        | 364/2733 [1:21:29<8:57:24, 13.61s/it]
 13%|█▎        | 365/2733 [1:21:43<8:53:07, 13.51s/it]
                                                      
{'loss': 1.7391, 'learning_rate': 0.0002698063045955184, 'epoch': 0.13}

 13%|█▎        | 365/2733 [1:21:43<8:53:07, 13.51s/it]
 13%|█▎        | 366/2733 [1:21:56<8:50:47, 13.45s/it]
                                                      
{'loss': 1.6036, 'learning_rate': 0.0002696923661222939, 'epoch': 0.13}

 13%|█▎        | 366/2733 [1:21:56<8:50:47, 13.45s/it]
 13%|█▎        | 367/2733 [1:22:10<8:52:54, 13.51s/it]
                                                      
{'loss': 1.6728, 'learning_rate': 0.00026957842764906945, 'epoch': 0.13}

 13%|█▎        | 367/2733 [1:22:10<8:52:54, 13.51s/it]
 13%|█▎        | 368/2733 [1:22:24<8:57:31, 13.64s/it]
                                                      
{'loss': 1.7737, 'learning_rate': 0.00026946448917584503, 'epoch': 0.13}

 13%|█▎        | 368/2733 [1:22:24<8:57:31, 13.64s/it]
 14%|█▎        | 369/2733 [1:22:36<8:48:26, 13.41s/it]
                                                      
{'loss': 1.7454, 'learning_rate': 0.00026935055070262056, 'epoch': 0.14}

 14%|█▎        | 369/2733 [1:22:36<8:48:26, 13.41s/it]
 14%|█▎        | 370/2733 [1:22:50<8:47:15, 13.39s/it]
                                                      
{'loss': 1.6839, 'learning_rate': 0.0002692366122293961, 'epoch': 0.14}

 14%|█▎        | 370/2733 [1:22:50<8:47:15, 13.39s/it]
 14%|█▎        | 371/2733 [1:23:04<8:57:32, 13.65s/it]
                                                      
{'loss': 1.7866, 'learning_rate': 0.0002691226737561716, 'epoch': 0.14}

 14%|█▎        | 371/2733 [1:23:04<8:57:32, 13.65s/it]
 14%|█▎        | 372/2733 [1:23:17<8:53:56, 13.57s/it]
                                                      
{'loss': 1.7883, 'learning_rate': 0.00026900873528294716, 'epoch': 0.14}

 14%|█▎        | 372/2733 [1:23:17<8:53:56, 13.57s/it]
 14%|█▎        | 373/2733 [1:23:30<8:41:44, 13.26s/it]
                                                      
{'loss': 1.6776, 'learning_rate': 0.00026889479680972274, 'epoch': 0.14}

 14%|█▎        | 373/2733 [1:23:30<8:41:44, 13.26s/it]
 14%|█▎        | 374/2733 [1:23:44<8:47:20, 13.41s/it]
                                                      
{'loss': 1.7033, 'learning_rate': 0.00026878085833649827, 'epoch': 0.14}

 14%|█▎        | 374/2733 [1:23:44<8:47:20, 13.41s/it]
 14%|█▎        | 375/2733 [1:23:57<8:46:39, 13.40s/it]
                                                      
{'loss': 1.69, 'learning_rate': 0.0002686669198632738, 'epoch': 0.14}

 14%|█▎        | 375/2733 [1:23:57<8:46:39, 13.40s/it]
 14%|█▍        | 376/2733 [1:24:10<8:43:36, 13.33s/it]
                                                      
{'loss': 1.6859, 'learning_rate': 0.00026855298139004933, 'epoch': 0.14}

 14%|█▍        | 376/2733 [1:24:10<8:43:36, 13.33s/it]
 14%|█▍        | 377/2733 [1:24:23<8:37:46, 13.19s/it]
                                                      
{'loss': 1.7621, 'learning_rate': 0.00026843904291682486, 'epoch': 0.14}

 14%|█▍        | 377/2733 [1:24:23<8:37:46, 13.19s/it]
 14%|█▍        | 378/2733 [1:24:36<8:37:02, 13.17s/it]
                                                      
{'loss': 1.6557, 'learning_rate': 0.00026832510444360045, 'epoch': 0.14}

 14%|█▍        | 378/2733 [1:24:36<8:37:02, 13.17s/it]
 14%|█▍        | 379/2733 [1:24:49<8:31:02, 13.03s/it]
                                                      
{'loss': 1.6025, 'learning_rate': 0.000268211165970376, 'epoch': 0.14}

 14%|█▍        | 379/2733 [1:24:49<8:31:02, 13.03s/it]
 14%|█▍        | 380/2733 [1:25:02<8:30:44, 13.02s/it]
                                                      
{'loss': 1.8824, 'learning_rate': 0.0002680972274971515, 'epoch': 0.14}

 14%|█▍        | 380/2733 [1:25:02<8:30:44, 13.02s/it]
 14%|█▍        | 381/2733 [1:25:16<8:39:25, 13.25s/it]
                                                      
{'loss': 1.7674, 'learning_rate': 0.00026798328902392704, 'epoch': 0.14}

 14%|█▍        | 381/2733 [1:25:16<8:39:25, 13.25s/it]
 14%|█▍        | 382/2733 [1:25:30<8:50:31, 13.54s/it]
                                                      
{'loss': 1.7741, 'learning_rate': 0.0002678693505507026, 'epoch': 0.14}

 14%|█▍        | 382/2733 [1:25:30<8:50:31, 13.54s/it]
 14%|█▍        | 383/2733 [1:25:43<8:44:32, 13.39s/it]
                                                      
{'loss': 1.7553, 'learning_rate': 0.00026775541207747815, 'epoch': 0.14}

 14%|█▍        | 383/2733 [1:25:43<8:44:32, 13.39s/it]
 14%|█▍        | 384/2733 [1:25:56<8:43:46, 13.38s/it]
                                                      
{'loss': 1.7599, 'learning_rate': 0.0002676414736042537, 'epoch': 0.14}

 14%|█▍        | 384/2733 [1:25:56<8:43:46, 13.38s/it]
 14%|█▍        | 385/2733 [1:26:10<8:49:31, 13.53s/it]
                                                      
{'loss': 1.8434, 'learning_rate': 0.0002675275351310292, 'epoch': 0.14}

 14%|█▍        | 385/2733 [1:26:10<8:49:31, 13.53s/it]
 14%|█▍        | 386/2733 [1:26:23<8:38:47, 13.26s/it]
                                                      
{'loss': 1.8118, 'learning_rate': 0.00026741359665780474, 'epoch': 0.14}

 14%|█▍        | 386/2733 [1:26:23<8:38:47, 13.26s/it]
 14%|█▍        | 387/2733 [1:26:36<8:40:10, 13.30s/it]
                                                      
{'loss': 1.7331, 'learning_rate': 0.00026729965818458033, 'epoch': 0.14}

 14%|█▍        | 387/2733 [1:26:36<8:40:10, 13.30s/it]
 14%|█▍        | 388/2733 [1:26:50<8:44:35, 13.42s/it]
                                                      
{'loss': 1.8106, 'learning_rate': 0.00026718571971135586, 'epoch': 0.14}

 14%|█▍        | 388/2733 [1:26:50<8:44:35, 13.42s/it]
 14%|█▍        | 389/2733 [1:27:03<8:35:58, 13.21s/it]
                                                      
{'loss': 1.7549, 'learning_rate': 0.0002670717812381314, 'epoch': 0.14}

 14%|█▍        | 389/2733 [1:27:03<8:35:58, 13.21s/it]
 14%|█▍        | 390/2733 [1:27:16<8:38:44, 13.28s/it]
                                                      
{'loss': 1.6195, 'learning_rate': 0.0002669578427649069, 'epoch': 0.14}

 14%|█▍        | 390/2733 [1:27:16<8:38:44, 13.28s/it]
 14%|█▍        | 391/2733 [1:27:29<8:36:53, 13.24s/it]
                                                      
{'loss': 1.7558, 'learning_rate': 0.00026684390429168245, 'epoch': 0.14}

 14%|█▍        | 391/2733 [1:27:29<8:36:53, 13.24s/it]
 14%|█▍        | 392/2733 [1:27:44<8:48:33, 13.55s/it]
                                                      
{'loss': 1.6831, 'learning_rate': 0.00026672996581845803, 'epoch': 0.14}

 14%|█▍        | 392/2733 [1:27:44<8:48:33, 13.55s/it]
 14%|█▍        | 393/2733 [1:27:57<8:49:23, 13.57s/it]
                                                      
{'loss': 1.8141, 'learning_rate': 0.00026661602734523356, 'epoch': 0.14}

 14%|█▍        | 393/2733 [1:27:57<8:49:23, 13.57s/it]
 14%|█▍        | 394/2733 [1:28:10<8:44:15, 13.45s/it]
                                                      
{'loss': 1.6714, 'learning_rate': 0.0002665020888720091, 'epoch': 0.14}

 14%|█▍        | 394/2733 [1:28:10<8:44:15, 13.45s/it]
 14%|█▍        | 395/2733 [1:28:24<8:43:12, 13.43s/it]
                                                      
{'loss': 1.7242, 'learning_rate': 0.0002663881503987846, 'epoch': 0.14}

 14%|█▍        | 395/2733 [1:28:24<8:43:12, 13.43s/it]
 14%|█▍        | 396/2733 [1:28:37<8:39:23, 13.33s/it]
                                                      
{'loss': 1.795, 'learning_rate': 0.0002662742119255602, 'epoch': 0.14}

 14%|█▍        | 396/2733 [1:28:37<8:39:23, 13.33s/it]
 15%|█▍        | 397/2733 [1:28:50<8:37:43, 13.30s/it]
                                                      
{'loss': 1.6629, 'learning_rate': 0.00026616027345233574, 'epoch': 0.15}

 15%|█▍        | 397/2733 [1:28:50<8:37:43, 13.30s/it]
 15%|█▍        | 398/2733 [1:29:03<8:37:33, 13.30s/it]
                                                      
{'loss': 1.7396, 'learning_rate': 0.0002660463349791112, 'epoch': 0.15}

 15%|█▍        | 398/2733 [1:29:03<8:37:33, 13.30s/it]
 15%|█▍        | 399/2733 [1:29:17<8:42:08, 13.42s/it]
                                                      
{'loss': 1.7649, 'learning_rate': 0.0002659323965058868, 'epoch': 0.15}

 15%|█▍        | 399/2733 [1:29:17<8:42:08, 13.42s/it]
 15%|█▍        | 400/2733 [1:29:30<8:40:20, 13.38s/it]
                                                      
{'loss': 1.6322, 'learning_rate': 0.00026581845803266233, 'epoch': 0.15}

 15%|█▍        | 400/2733 [1:29:30<8:40:20, 13.38s/it]
 15%|█▍        | 401/2733 [1:29:44<8:44:38, 13.50s/it]
                                                      
{'loss': 1.7916, 'learning_rate': 0.0002657045195594379, 'epoch': 0.15}

 15%|█▍        | 401/2733 [1:29:44<8:44:38, 13.50s/it]
 15%|█▍        | 402/2733 [1:29:58<8:54:16, 13.75s/it]
                                                      
{'loss': 1.7476, 'learning_rate': 0.0002655905810862134, 'epoch': 0.15}

 15%|█▍        | 402/2733 [1:29:58<8:54:16, 13.75s/it]
 15%|█▍        | 403/2733 [1:30:12<8:54:56, 13.78s/it]
                                                      
{'loss': 1.8205, 'learning_rate': 0.000265476642612989, 'epoch': 0.15}

 15%|█▍        | 403/2733 [1:30:12<8:54:56, 13.78s/it]
 15%|█▍        | 404/2733 [1:30:25<8:47:07, 13.58s/it]
                                                      
{'loss': 1.6958, 'learning_rate': 0.0002653627041397645, 'epoch': 0.15}

 15%|█▍        | 404/2733 [1:30:25<8:47:07, 13.58s/it]
 15%|█▍        | 405/2733 [1:30:39<8:49:12, 13.64s/it]
                                                      
{'loss': 1.7264, 'learning_rate': 0.00026524876566654004, 'epoch': 0.15}

 15%|█▍        | 405/2733 [1:30:39<8:49:12, 13.64s/it]
 15%|█▍        | 406/2733 [1:30:52<8:44:06, 13.51s/it]
                                                      
{'loss': 1.8257, 'learning_rate': 0.00026513482719331557, 'epoch': 0.15}

 15%|█▍        | 406/2733 [1:30:52<8:44:06, 13.51s/it]
 15%|█▍        | 407/2733 [1:31:07<8:53:04, 13.75s/it]
                                                      
{'loss': 1.7141, 'learning_rate': 0.0002650208887200911, 'epoch': 0.15}

 15%|█▍        | 407/2733 [1:31:07<8:53:04, 13.75s/it]
 15%|█▍        | 408/2733 [1:31:19<8:35:36, 13.31s/it]
                                                      
{'loss': 1.6659, 'learning_rate': 0.0002649069502468667, 'epoch': 0.15}

 15%|█▍        | 408/2733 [1:31:19<8:35:36, 13.31s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 15%|█▍        | 409/2733 [1:31:33<8:39:23, 13.41s/it]
                                                      
{'loss': 1.7887, 'learning_rate': 0.0002647930117736422, 'epoch': 0.15}

 15%|█▍        | 409/2733 [1:31:33<8:39:23, 13.41s/it]
 15%|█▌        | 410/2733 [1:31:47<8:49:14, 13.67s/it]
                                                      
{'loss': 1.795, 'learning_rate': 0.00026467907330041774, 'epoch': 0.15}

 15%|█▌        | 410/2733 [1:31:47<8:49:14, 13.67s/it]
 15%|█▌        | 411/2733 [1:32:01<8:53:14, 13.78s/it]
                                                      
{'loss': 1.7043, 'learning_rate': 0.00026456513482719327, 'epoch': 0.15}

 15%|█▌        | 411/2733 [1:32:01<8:53:14, 13.78s/it]
 15%|█▌        | 412/2733 [1:32:15<8:52:16, 13.76s/it]
                                                      
{'loss': 1.7684, 'learning_rate': 0.0002644511963539688, 'epoch': 0.15}

 15%|█▌        | 412/2733 [1:32:15<8:52:16, 13.76s/it]
 15%|█▌        | 413/2733 [1:32:28<8:49:42, 13.70s/it]
                                                      
{'loss': 1.7651, 'learning_rate': 0.0002643372578807444, 'epoch': 0.15}

 15%|█▌        | 413/2733 [1:32:28<8:49:42, 13.70s/it]
 15%|█▌        | 414/2733 [1:32:42<8:46:20, 13.62s/it]
                                                      
{'loss': 1.7158, 'learning_rate': 0.0002642233194075199, 'epoch': 0.15}

 15%|█▌        | 414/2733 [1:32:42<8:46:20, 13.62s/it]
 15%|█▌        | 415/2733 [1:32:55<8:42:02, 13.51s/it]
                                                      
{'loss': 1.6635, 'learning_rate': 0.00026410938093429545, 'epoch': 0.15}

 15%|█▌        | 415/2733 [1:32:55<8:42:02, 13.51s/it]
 15%|█▌        | 416/2733 [1:33:09<8:44:19, 13.58s/it]
                                                      
{'loss': 1.8075, 'learning_rate': 0.000263995442461071, 'epoch': 0.15}

 15%|█▌        | 416/2733 [1:33:09<8:44:19, 13.58s/it]
 15%|█▌        | 417/2733 [1:33:22<8:44:29, 13.59s/it]
                                                      
{'loss': 1.6785, 'learning_rate': 0.00026388150398784656, 'epoch': 0.15}

 15%|█▌        | 417/2733 [1:33:22<8:44:29, 13.59s/it]
 15%|█▌        | 418/2733 [1:33:34<8:28:33, 13.18s/it]
                                                      
{'loss': 1.6765, 'learning_rate': 0.0002637675655146221, 'epoch': 0.15}

 15%|█▌        | 418/2733 [1:33:34<8:28:33, 13.18s/it]
 15%|█▌        | 419/2733 [1:33:47<8:25:53, 13.12s/it]
                                                      
{'loss': 1.6219, 'learning_rate': 0.0002636536270413976, 'epoch': 0.15}

 15%|█▌        | 419/2733 [1:33:47<8:25:53, 13.12s/it]
 15%|█▌        | 420/2733 [1:34:01<8:30:55, 13.25s/it]
                                                      
{'loss': 1.7289, 'learning_rate': 0.00026353968856817315, 'epoch': 0.15}

 15%|█▌        | 420/2733 [1:34:01<8:30:55, 13.25s/it]
 15%|█▌        | 421/2733 [1:34:14<8:30:16, 13.24s/it]
                                                      
{'loss': 1.8087, 'learning_rate': 0.0002634257500949487, 'epoch': 0.15}

 15%|█▌        | 421/2733 [1:34:14<8:30:16, 13.24s/it]
 15%|█▌        | 422/2733 [1:34:28<8:34:34, 13.36s/it]
                                                      
{'loss': 1.7759, 'learning_rate': 0.00026331181162172427, 'epoch': 0.15}

 15%|█▌        | 422/2733 [1:34:28<8:34:34, 13.36s/it]
 15%|█▌        | 423/2733 [1:34:42<8:43:50, 13.61s/it]
                                                      
{'loss': 1.7997, 'learning_rate': 0.0002631978731484998, 'epoch': 0.15}

 15%|█▌        | 423/2733 [1:34:42<8:43:50, 13.61s/it]
 16%|█▌        | 424/2733 [1:34:55<8:39:38, 13.50s/it]
                                                      
{'loss': 1.6497, 'learning_rate': 0.00026308393467527533, 'epoch': 0.16}

 16%|█▌        | 424/2733 [1:34:55<8:39:38, 13.50s/it]
 16%|█▌        | 425/2733 [1:35:09<8:41:33, 13.56s/it]
                                                      
{'loss': 1.7704, 'learning_rate': 0.00026296999620205086, 'epoch': 0.16}

 16%|█▌        | 425/2733 [1:35:09<8:41:33, 13.56s/it]
 16%|█▌        | 426/2733 [1:35:23<8:43:17, 13.61s/it]
                                                      
{'loss': 1.8058, 'learning_rate': 0.0002628560577288264, 'epoch': 0.16}

 16%|█▌        | 426/2733 [1:35:23<8:43:17, 13.61s/it]
 16%|█▌        | 427/2733 [1:35:37<8:48:45, 13.76s/it]
                                                      
{'loss': 1.6845, 'learning_rate': 0.000262742119255602, 'epoch': 0.16}

 16%|█▌        | 427/2733 [1:35:37<8:48:45, 13.76s/it]
 16%|█▌        | 428/2733 [1:35:51<8:48:34, 13.76s/it]
                                                      
{'loss': 1.6934, 'learning_rate': 0.0002626281807823775, 'epoch': 0.16}

 16%|█▌        | 428/2733 [1:35:51<8:48:34, 13.76s/it]
 16%|█▌        | 429/2733 [1:36:04<8:45:38, 13.69s/it]
                                                      
{'loss': 1.6928, 'learning_rate': 0.00026251424230915304, 'epoch': 0.16}

 16%|█▌        | 429/2733 [1:36:04<8:45:38, 13.69s/it]
 16%|█▌        | 430/2733 [1:36:18<8:42:12, 13.61s/it]
                                                      
{'loss': 1.8058, 'learning_rate': 0.00026240030383592857, 'epoch': 0.16}

 16%|█▌        | 430/2733 [1:36:18<8:42:12, 13.61s/it]
 16%|█▌        | 431/2733 [1:36:31<8:44:39, 13.67s/it]
                                                      
{'loss': 1.7987, 'learning_rate': 0.00026228636536270415, 'epoch': 0.16}

 16%|█▌        | 431/2733 [1:36:31<8:44:39, 13.67s/it]
 16%|█▌        | 432/2733 [1:36:45<8:41:58, 13.61s/it]
                                                      
{'loss': 1.7111, 'learning_rate': 0.0002621724268894797, 'epoch': 0.16}

 16%|█▌        | 432/2733 [1:36:45<8:41:58, 13.61s/it]
 16%|█▌        | 433/2733 [1:36:58<8:32:16, 13.36s/it]
                                                      
{'loss': 1.695, 'learning_rate': 0.0002620584884162552, 'epoch': 0.16}

 16%|█▌        | 433/2733 [1:36:58<8:32:16, 13.36s/it]
 16%|█▌        | 434/2733 [1:37:11<8:28:36, 13.27s/it]
                                                      
{'loss': 1.6744, 'learning_rate': 0.00026194454994303074, 'epoch': 0.16}

 16%|█▌        | 434/2733 [1:37:11<8:28:36, 13.27s/it]
 16%|█▌        | 435/2733 [1:37:24<8:31:21, 13.35s/it]
                                                      
{'loss': 1.7159, 'learning_rate': 0.00026183061146980627, 'epoch': 0.16}

 16%|█▌        | 435/2733 [1:37:24<8:31:21, 13.35s/it]
 16%|█▌        | 436/2733 [1:37:37<8:20:32, 13.07s/it]
                                                      
{'loss': 1.7029, 'learning_rate': 0.00026171667299658186, 'epoch': 0.16}

 16%|█▌        | 436/2733 [1:37:37<8:20:32, 13.07s/it]
 16%|█▌        | 437/2733 [1:37:50<8:21:50, 13.11s/it]
                                                      
{'loss': 1.7344, 'learning_rate': 0.0002616027345233574, 'epoch': 0.16}

 16%|█▌        | 437/2733 [1:37:50<8:21:50, 13.11s/it]
 16%|█▌        | 438/2733 [1:38:04<8:30:47, 13.35s/it]
                                                      
{'loss': 1.7338, 'learning_rate': 0.0002614887960501329, 'epoch': 0.16}

 16%|█▌        | 438/2733 [1:38:04<8:30:47, 13.35s/it]
 16%|█▌        | 439/2733 [1:38:18<8:36:07, 13.50s/it]
                                                      
{'loss': 1.8212, 'learning_rate': 0.00026137485757690845, 'epoch': 0.16}

 16%|█▌        | 439/2733 [1:38:18<8:36:07, 13.50s/it]
 16%|█▌        | 440/2733 [1:38:31<8:35:44, 13.50s/it]
                                                      
{'loss': 1.7386, 'learning_rate': 0.000261260919103684, 'epoch': 0.16}

 16%|█▌        | 440/2733 [1:38:31<8:35:44, 13.50s/it]
 16%|█▌        | 441/2733 [1:38:45<8:42:21, 13.67s/it]
                                                      
{'loss': 1.7747, 'learning_rate': 0.0002611469806304595, 'epoch': 0.16}

 16%|█▌        | 441/2733 [1:38:45<8:42:21, 13.67s/it]
 16%|█▌        | 442/2733 [1:38:58<8:29:34, 13.35s/it]
                                                      
{'loss': 1.6763, 'learning_rate': 0.00026103304215723504, 'epoch': 0.16}

 16%|█▌        | 442/2733 [1:38:58<8:29:34, 13.35s/it]
 16%|█▌        | 443/2733 [1:39:12<8:38:49, 13.59s/it]
                                                      
{'loss': 1.806, 'learning_rate': 0.0002609191036840106, 'epoch': 0.16}

 16%|█▌        | 443/2733 [1:39:12<8:38:49, 13.59s/it]
 16%|█▌        | 444/2733 [1:39:26<8:46:18, 13.80s/it]
                                                      
{'loss': 1.7312, 'learning_rate': 0.00026080516521078615, 'epoch': 0.16}

 16%|█▌        | 444/2733 [1:39:26<8:46:18, 13.80s/it]
 16%|█▋        | 445/2733 [1:39:40<8:41:04, 13.66s/it]
                                                      
{'loss': 1.6886, 'learning_rate': 0.0002606912267375617, 'epoch': 0.16}

 16%|█▋        | 445/2733 [1:39:40<8:41:04, 13.66s/it]
 16%|█▋        | 446/2733 [1:39:53<8:38:29, 13.60s/it]
                                                      
{'loss': 1.6365, 'learning_rate': 0.0002605772882643372, 'epoch': 0.16}

 16%|█▋        | 446/2733 [1:39:53<8:38:29, 13.60s/it]
 16%|█▋        | 447/2733 [1:40:07<8:38:06, 13.60s/it]
                                                      
{'loss': 1.7932, 'learning_rate': 0.00026046334979111274, 'epoch': 0.16}

 16%|█▋        | 447/2733 [1:40:07<8:38:06, 13.60s/it]
 16%|█▋        | 448/2733 [1:40:20<8:34:25, 13.51s/it]
                                                      
{'loss': 1.6953, 'learning_rate': 0.00026034941131788833, 'epoch': 0.16}

 16%|█▋        | 448/2733 [1:40:20<8:34:25, 13.51s/it]
 16%|█▋        | 449/2733 [1:40:33<8:28:36, 13.36s/it]
                                                      
{'loss': 1.673, 'learning_rate': 0.00026023547284466386, 'epoch': 0.16}

 16%|█▋        | 449/2733 [1:40:33<8:28:36, 13.36s/it]
 16%|█▋        | 450/2733 [1:40:46<8:27:05, 13.33s/it]
                                                      
{'loss': 1.7487, 'learning_rate': 0.0002601215343714394, 'epoch': 0.16}

 16%|█▋        | 450/2733 [1:40:46<8:27:05, 13.33s/it]
 17%|█▋        | 451/2733 [1:40:59<8:26:43, 13.32s/it]
                                                      
{'loss': 1.7201, 'learning_rate': 0.0002600075958982149, 'epoch': 0.17}

 17%|█▋        | 451/2733 [1:40:59<8:26:43, 13.32s/it]
 17%|█▋        | 452/2733 [1:41:14<8:37:15, 13.61s/it]
                                                      
{'loss': 1.7014, 'learning_rate': 0.0002598936574249905, 'epoch': 0.17}

 17%|█▋        | 452/2733 [1:41:14<8:37:15, 13.61s/it]
 17%|█▋        | 453/2733 [1:41:27<8:33:41, 13.52s/it]
                                                      
{'loss': 1.8077, 'learning_rate': 0.00025977971895176604, 'epoch': 0.17}

 17%|█▋        | 453/2733 [1:41:27<8:33:41, 13.52s/it]
 17%|█▋        | 454/2733 [1:41:41<8:33:44, 13.53s/it]
                                                      
{'loss': 1.7725, 'learning_rate': 0.00025966578047854157, 'epoch': 0.17}

 17%|█▋        | 454/2733 [1:41:41<8:33:44, 13.53s/it]
 17%|█▋        | 455/2733 [1:41:54<8:36:21, 13.60s/it]
                                                      
{'loss': 1.7719, 'learning_rate': 0.0002595518420053171, 'epoch': 0.17}

 17%|█▋        | 455/2733 [1:41:54<8:36:21, 13.60s/it]
 17%|█▋        | 456/2733 [1:42:08<8:33:49, 13.54s/it]
                                                      
{'loss': 1.717, 'learning_rate': 0.0002594379035320926, 'epoch': 0.17}

 17%|█▋        | 456/2733 [1:42:08<8:33:49, 13.54s/it]
 17%|█▋        | 457/2733 [1:42:21<8:34:46, 13.57s/it]
                                                      
{'loss': 1.8541, 'learning_rate': 0.0002593239650588682, 'epoch': 0.17}

 17%|█▋        | 457/2733 [1:42:21<8:34:46, 13.57s/it]
 17%|█▋        | 458/2733 [1:42:35<8:35:10, 13.59s/it]
                                                      
{'loss': 1.7954, 'learning_rate': 0.00025921002658564374, 'epoch': 0.17}

 17%|█▋        | 458/2733 [1:42:35<8:35:10, 13.59s/it]
 17%|█▋        | 459/2733 [1:42:49<8:35:44, 13.61s/it]
                                                      
{'loss': 1.6423, 'learning_rate': 0.00025909608811241927, 'epoch': 0.17}

 17%|█▋        | 459/2733 [1:42:49<8:35:44, 13.61s/it]
 17%|█▋        | 460/2733 [1:43:02<8:29:29, 13.45s/it]
                                                      
{'loss': 1.8811, 'learning_rate': 0.0002589821496391948, 'epoch': 0.17}

 17%|█▋        | 460/2733 [1:43:02<8:29:29, 13.45s/it]
 17%|█▋        | 461/2733 [1:43:15<8:26:20, 13.37s/it]
                                                      
{'loss': 1.6902, 'learning_rate': 0.00025886821116597033, 'epoch': 0.17}

 17%|█▋        | 461/2733 [1:43:15<8:26:20, 13.37s/it]
 17%|█▋        | 462/2733 [1:43:29<8:32:55, 13.55s/it]
                                                      
{'loss': 1.6623, 'learning_rate': 0.0002587542726927459, 'epoch': 0.17}

 17%|█▋        | 462/2733 [1:43:29<8:32:55, 13.55s/it]
 17%|█▋        | 463/2733 [1:43:42<8:31:38, 13.52s/it]
                                                      
{'loss': 1.7159, 'learning_rate': 0.00025864033421952145, 'epoch': 0.17}

 17%|█▋        | 463/2733 [1:43:42<8:31:38, 13.52s/it]
 17%|█▋        | 464/2733 [1:43:56<8:29:20, 13.47s/it]
                                                      
{'loss': 1.8151, 'learning_rate': 0.000258526395746297, 'epoch': 0.17}

 17%|█▋        | 464/2733 [1:43:56<8:29:20, 13.47s/it]
 17%|█▋        | 465/2733 [1:44:09<8:28:00, 13.44s/it]
                                                      
{'loss': 1.7003, 'learning_rate': 0.0002584124572730725, 'epoch': 0.17}

 17%|█▋        | 465/2733 [1:44:09<8:28:00, 13.44s/it]
 17%|█▋        | 466/2733 [1:44:23<8:30:03, 13.50s/it]
                                                      
{'loss': 1.8336, 'learning_rate': 0.0002582985187998481, 'epoch': 0.17}

 17%|█▋        | 466/2733 [1:44:23<8:30:03, 13.50s/it]
 17%|█▋        | 467/2733 [1:44:36<8:22:31, 13.31s/it]
                                                      
{'loss': 1.6473, 'learning_rate': 0.0002581845803266236, 'epoch': 0.17}

 17%|█▋        | 467/2733 [1:44:36<8:22:31, 13.31s/it]
 17%|█▋        | 468/2733 [1:44:49<8:23:05, 13.33s/it]
                                                      
{'loss': 1.6448, 'learning_rate': 0.00025807064185339915, 'epoch': 0.17}

 17%|█▋        | 468/2733 [1:44:49<8:23:05, 13.33s/it]
 17%|█▋        | 469/2733 [1:45:03<8:27:32, 13.45s/it]
                                                      
{'loss': 1.7582, 'learning_rate': 0.0002579567033801747, 'epoch': 0.17}

 17%|█▋        | 469/2733 [1:45:03<8:27:32, 13.45s/it]
 17%|█▋        | 470/2733 [1:45:16<8:26:44, 13.44s/it]
                                                      
{'loss': 1.7374, 'learning_rate': 0.0002578427649069502, 'epoch': 0.17}

 17%|█▋        | 470/2733 [1:45:16<8:26:44, 13.44s/it]
 17%|█▋        | 471/2733 [1:45:30<8:28:53, 13.50s/it]
                                                      
{'loss': 1.7426, 'learning_rate': 0.0002577288264337258, 'epoch': 0.17}

 17%|█▋        | 471/2733 [1:45:30<8:28:53, 13.50s/it]
 17%|█▋        | 472/2733 [1:45:43<8:22:22, 13.33s/it]
                                                      
{'loss': 1.7993, 'learning_rate': 0.00025761488796050133, 'epoch': 0.17}

 17%|█▋        | 472/2733 [1:45:43<8:22:22, 13.33s/it]
 17%|█▋        | 473/2733 [1:45:56<8:26:00, 13.43s/it]
                                                      
{'loss': 1.7928, 'learning_rate': 0.00025750094948727686, 'epoch': 0.17}

 17%|█▋        | 473/2733 [1:45:56<8:26:00, 13.43s/it]
 17%|█▋        | 474/2733 [1:46:10<8:30:27, 13.56s/it]
                                                      
{'loss': 1.5894, 'learning_rate': 0.0002573870110140524, 'epoch': 0.17}

 17%|█▋        | 474/2733 [1:46:10<8:30:27, 13.56s/it]
 17%|█▋        | 475/2733 [1:46:24<8:30:40, 13.57s/it]
                                                      
{'loss': 1.7897, 'learning_rate': 0.0002572730725408279, 'epoch': 0.17}

 17%|█▋        | 475/2733 [1:46:24<8:30:40, 13.57s/it]
 17%|█▋        | 476/2733 [1:46:37<8:25:03, 13.43s/it]
                                                      
{'loss': 1.7025, 'learning_rate': 0.0002571591340676035, 'epoch': 0.17}

 17%|█▋        | 476/2733 [1:46:37<8:25:03, 13.43s/it]
 17%|█▋        | 477/2733 [1:46:50<8:17:28, 13.23s/it]
                                                      
{'loss': 1.7385, 'learning_rate': 0.000257045195594379, 'epoch': 0.17}

 17%|█▋        | 477/2733 [1:46:50<8:17:28, 13.23s/it]
 17%|█▋        | 478/2733 [1:47:03<8:22:15, 13.36s/it]
                                                      
{'loss': 1.6996, 'learning_rate': 0.00025693125712115456, 'epoch': 0.17}

 17%|█▋        | 478/2733 [1:47:03<8:22:15, 13.36s/it]
 18%|█▊        | 479/2733 [1:47:17<8:26:48, 13.49s/it]
                                                      
{'loss': 1.778, 'learning_rate': 0.0002568173186479301, 'epoch': 0.18}

 18%|█▊        | 479/2733 [1:47:17<8:26:48, 13.49s/it]
 18%|█▊        | 480/2733 [1:47:31<8:28:44, 13.55s/it]
                                                      
{'loss': 1.7474, 'learning_rate': 0.0002567033801747057, 'epoch': 0.18}

 18%|█▊        | 480/2733 [1:47:31<8:28:44, 13.55s/it]
 18%|█▊        | 481/2733 [1:47:44<8:30:03, 13.59s/it]
                                                      
{'loss': 1.7732, 'learning_rate': 0.00025658944170148116, 'epoch': 0.18}

 18%|█▊        | 481/2733 [1:47:45<8:30:03, 13.59s/it]
 18%|█▊        | 482/2733 [1:47:58<8:27:38, 13.53s/it]
                                                      
{'loss': 1.6797, 'learning_rate': 0.0002564755032282567, 'epoch': 0.18}

 18%|█▊        | 482/2733 [1:47:58<8:27:38, 13.53s/it]
 18%|█▊        | 483/2733 [1:48:12<8:31:39, 13.64s/it]
                                                      
{'loss': 1.7999, 'learning_rate': 0.00025636156475503227, 'epoch': 0.18}

 18%|█▊        | 483/2733 [1:48:12<8:31:39, 13.64s/it]
 18%|█▊        | 484/2733 [1:48:24<8:18:17, 13.29s/it]
                                                      
{'loss': 1.6878, 'learning_rate': 0.0002562476262818078, 'epoch': 0.18}

 18%|█▊        | 484/2733 [1:48:24<8:18:17, 13.29s/it]
 18%|█▊        | 485/2733 [1:48:38<8:21:50, 13.39s/it]
                                                      
{'loss': 1.7464, 'learning_rate': 0.00025613368780858333, 'epoch': 0.18}

 18%|█▊        | 485/2733 [1:48:38<8:21:50, 13.39s/it]
 18%|█▊        | 486/2733 [1:48:51<8:23:33, 13.45s/it]
                                                      
{'loss': 1.7269, 'learning_rate': 0.00025601974933535886, 'epoch': 0.18}

 18%|█▊        | 486/2733 [1:48:51<8:23:33, 13.45s/it]
 18%|█▊        | 487/2733 [1:49:05<8:25:54, 13.51s/it]
                                                      
{'loss': 1.7049, 'learning_rate': 0.00025590581086213445, 'epoch': 0.18}

 18%|█▊        | 487/2733 [1:49:05<8:25:54, 13.51s/it]
 18%|█▊        | 488/2733 [1:49:19<8:34:02, 13.74s/it]
                                                      
{'loss': 1.7346, 'learning_rate': 0.00025579187238891, 'epoch': 0.18}

 18%|█▊        | 488/2733 [1:49:19<8:34:02, 13.74s/it]
 18%|█▊        | 489/2733 [1:49:33<8:29:28, 13.62s/it]
                                                      
{'loss': 1.772, 'learning_rate': 0.0002556779339156855, 'epoch': 0.18}

 18%|█▊        | 489/2733 [1:49:33<8:29:28, 13.62s/it]
 18%|█▊        | 490/2733 [1:49:46<8:30:07, 13.65s/it]
                                                      
{'loss': 1.5743, 'learning_rate': 0.00025556399544246104, 'epoch': 0.18}

 18%|█▊        | 490/2733 [1:49:46<8:30:07, 13.65s/it]
 18%|█▊        | 491/2733 [1:50:00<8:28:46, 13.62s/it]
                                                      
{'loss': 1.8006, 'learning_rate': 0.00025545005696923657, 'epoch': 0.18}

 18%|█▊        | 491/2733 [1:50:00<8:28:46, 13.62s/it]
 18%|█▊        | 492/2733 [1:50:13<8:25:20, 13.53s/it]
                                                      
{'loss': 1.7391, 'learning_rate': 0.00025533611849601215, 'epoch': 0.18}

 18%|█▊        | 492/2733 [1:50:13<8:25:20, 13.53s/it]
 18%|█▊        | 493/2733 [1:50:26<8:19:16, 13.37s/it]
                                                      
{'loss': 1.7438, 'learning_rate': 0.0002552221800227877, 'epoch': 0.18}

 18%|█▊        | 493/2733 [1:50:26<8:19:16, 13.37s/it]
 18%|█▊        | 494/2733 [1:50:39<8:11:06, 13.16s/it]
                                                      
{'loss': 1.6739, 'learning_rate': 0.0002551082415495632, 'epoch': 0.18}

 18%|█▊        | 494/2733 [1:50:39<8:11:06, 13.16s/it]
 18%|█▊        | 495/2733 [1:50:53<8:23:45, 13.51s/it]
                                                      
{'loss': 1.6893, 'learning_rate': 0.00025499430307633874, 'epoch': 0.18}

 18%|█▊        | 495/2733 [1:50:53<8:23:45, 13.51s/it]
 18%|█▊        | 496/2733 [1:51:07<8:28:48, 13.65s/it]
                                                      
{'loss': 1.7145, 'learning_rate': 0.0002548803646031143, 'epoch': 0.18}

 18%|█▊        | 496/2733 [1:51:07<8:28:48, 13.65s/it]
 18%|█▊        | 497/2733 [1:51:21<8:29:04, 13.66s/it]
                                                      
{'loss': 1.6333, 'learning_rate': 0.00025476642612988986, 'epoch': 0.18}

 18%|█▊        | 497/2733 [1:51:21<8:29:04, 13.66s/it]
 18%|█▊        | 498/2733 [1:51:35<8:29:04, 13.67s/it]
                                                      
{'loss': 1.7123, 'learning_rate': 0.0002546524876566654, 'epoch': 0.18}

 18%|█▊        | 498/2733 [1:51:35<8:29:04, 13.67s/it]
 18%|█▊        | 499/2733 [1:51:48<8:29:55, 13.70s/it]
                                                      
{'loss': 1.8118, 'learning_rate': 0.0002545385491834409, 'epoch': 0.18}

 18%|█▊        | 499/2733 [1:51:48<8:29:55, 13.70s/it]
 18%|█▊        | 500/2733 [1:52:02<8:33:14, 13.79s/it]
                                                      
{'loss': 1.73, 'learning_rate': 0.00025442461071021645, 'epoch': 0.18}

 18%|█▊        | 500/2733 [1:52:02<8:33:14, 13.79s/it]
 18%|█▊        | 501/2733 [1:52:16<8:31:46, 13.76s/it]
                                                      
{'loss': 1.6612, 'learning_rate': 0.000254310672236992, 'epoch': 0.18}

 18%|█▊        | 501/2733 [1:52:16<8:31:46, 13.76s/it]
 18%|█▊        | 502/2733 [1:52:30<8:30:01, 13.72s/it]
                                                      
{'loss': 1.7561, 'learning_rate': 0.00025419673376376756, 'epoch': 0.18}

 18%|█▊        | 502/2733 [1:52:30<8:30:01, 13.72s/it]
 18%|█▊        | 503/2733 [1:52:43<8:25:26, 13.60s/it]
                                                      
{'loss': 1.7361, 'learning_rate': 0.0002540827952905431, 'epoch': 0.18}

 18%|█▊        | 503/2733 [1:52:43<8:25:26, 13.60s/it]
 18%|█▊        | 504/2733 [1:52:56<8:22:55, 13.54s/it]
                                                      
{'loss': 1.6621, 'learning_rate': 0.0002539688568173186, 'epoch': 0.18}

 18%|█▊        | 504/2733 [1:52:56<8:22:55, 13.54s/it]
 18%|█▊        | 505/2733 [1:53:09<8:15:43, 13.35s/it]
                                                      
{'loss': 1.6287, 'learning_rate': 0.00025385491834409416, 'epoch': 0.18}

 18%|█▊        | 505/2733 [1:53:09<8:15:43, 13.35s/it]
 19%|█▊        | 506/2733 [1:53:22<8:03:32, 13.03s/it]
                                                      
{'loss': 1.6988, 'learning_rate': 0.00025374097987086974, 'epoch': 0.19}

 19%|█▊        | 506/2733 [1:53:22<8:03:32, 13.03s/it]
 19%|█▊        | 507/2733 [1:53:35<8:05:53, 13.10s/it]
                                                      
{'loss': 1.6965, 'learning_rate': 0.00025362704139764527, 'epoch': 0.19}

 19%|█▊        | 507/2733 [1:53:35<8:05:53, 13.10s/it]
 19%|█▊        | 508/2733 [1:53:49<8:15:41, 13.37s/it]
                                                      
{'loss': 1.7382, 'learning_rate': 0.0002535131029244208, 'epoch': 0.19}

 19%|█▊        | 508/2733 [1:53:49<8:15:41, 13.37s/it]
 19%|█▊        | 509/2733 [1:54:03<8:25:18, 13.63s/it]
                                                      
{'loss': 1.7938, 'learning_rate': 0.00025339916445119633, 'epoch': 0.19}

 19%|█▊        | 509/2733 [1:54:03<8:25:18, 13.63s/it]
 19%|█▊        | 510/2733 [1:54:17<8:22:53, 13.57s/it]
                                                      
{'loss': 1.7396, 'learning_rate': 0.00025328522597797186, 'epoch': 0.19}

 19%|█▊        | 510/2733 [1:54:17<8:22:53, 13.57s/it]
 19%|█▊        | 511/2733 [1:54:30<8:25:25, 13.65s/it]
                                                      
{'loss': 1.6342, 'learning_rate': 0.00025317128750474745, 'epoch': 0.19}

 19%|█▊        | 511/2733 [1:54:30<8:25:25, 13.65s/it]
 19%|█▊        | 512/2733 [1:54:43<8:13:50, 13.34s/it]
                                                      
{'loss': 1.6908, 'learning_rate': 0.000253057349031523, 'epoch': 0.19}

 19%|█▊        | 512/2733 [1:54:43<8:13:50, 13.34s/it]
 19%|█▉        | 513/2733 [1:54:56<8:09:57, 13.24s/it]
                                                      
{'loss': 1.8238, 'learning_rate': 0.0002529434105582985, 'epoch': 0.19}

 19%|█▉        | 513/2733 [1:54:56<8:09:57, 13.24s/it]
 19%|█▉        | 514/2733 [1:55:10<8:20:27, 13.53s/it]
                                                      
{'loss': 1.6934, 'learning_rate': 0.00025282947208507404, 'epoch': 0.19}

 19%|█▉        | 514/2733 [1:55:10<8:20:27, 13.53s/it]
 19%|█▉        | 515/2733 [1:55:24<8:18:17, 13.48s/it]
                                                      
{'loss': 1.7353, 'learning_rate': 0.00025271553361184957, 'epoch': 0.19}

 19%|█▉        | 515/2733 [1:55:24<8:18:17, 13.48s/it]
 19%|█▉        | 516/2733 [1:55:37<8:19:17, 13.51s/it]
                                                      
{'loss': 1.7053, 'learning_rate': 0.0002526015951386251, 'epoch': 0.19}

 19%|█▉        | 516/2733 [1:55:37<8:19:17, 13.51s/it]
 19%|█▉        | 517/2733 [1:55:50<8:14:46, 13.40s/it]
                                                      
{'loss': 1.6355, 'learning_rate': 0.00025248765666540063, 'epoch': 0.19}

 19%|█▉        | 517/2733 [1:55:50<8:14:46, 13.40s/it]
 19%|█▉        | 518/2733 [1:56:04<8:16:32, 13.45s/it]
                                                      
{'loss': 1.5814, 'learning_rate': 0.0002523737181921762, 'epoch': 0.19}

 19%|█▉        | 518/2733 [1:56:04<8:16:32, 13.45s/it]
 19%|█▉        | 519/2733 [1:56:18<8:24:44, 13.68s/it]
                                                      
{'loss': 1.6949, 'learning_rate': 0.00025225977971895174, 'epoch': 0.19}

 19%|█▉        | 519/2733 [1:56:18<8:24:44, 13.68s/it]
 19%|█▉        | 520/2733 [1:56:31<8:12:58, 13.37s/it]
                                                      
{'loss': 1.7471, 'learning_rate': 0.0002521458412457273, 'epoch': 0.19}

 19%|█▉        | 520/2733 [1:56:31<8:12:58, 13.37s/it]
 19%|█▉        | 521/2733 [1:56:44<8:15:21, 13.44s/it]
                                                      
{'loss': 1.7479, 'learning_rate': 0.0002520319027725028, 'epoch': 0.19}

 19%|█▉        | 521/2733 [1:56:44<8:15:21, 13.44s/it]
 19%|█▉        | 522/2733 [1:56:59<8:24:07, 13.68s/it]
                                                      
{'loss': 1.7954, 'learning_rate': 0.00025191796429927833, 'epoch': 0.19}

 19%|█▉        | 522/2733 [1:56:59<8:24:07, 13.68s/it]
 19%|█▉        | 523/2733 [1:57:12<8:24:26, 13.70s/it]
                                                      
{'loss': 1.7424, 'learning_rate': 0.0002518040258260539, 'epoch': 0.19}

 19%|█▉        | 523/2733 [1:57:12<8:24:26, 13.70s/it]
 19%|█▉        | 524/2733 [1:57:26<8:25:53, 13.74s/it]
                                                      
{'loss': 1.8042, 'learning_rate': 0.00025169008735282945, 'epoch': 0.19}

 19%|█▉        | 524/2733 [1:57:26<8:25:53, 13.74s/it]
 19%|█▉        | 525/2733 [1:57:40<8:27:28, 13.79s/it]
                                                      
{'loss': 1.7329, 'learning_rate': 0.000251576148879605, 'epoch': 0.19}

 19%|█▉        | 525/2733 [1:57:40<8:27:28, 13.79s/it]
 19%|█▉        | 526/2733 [1:57:53<8:13:17, 13.41s/it]
                                                      
{'loss': 1.6221, 'learning_rate': 0.0002514622104063805, 'epoch': 0.19}

 19%|█▉        | 526/2733 [1:57:53<8:13:17, 13.41s/it]
 19%|█▉        | 527/2733 [1:58:07<8:22:30, 13.67s/it]
                                                      
{'loss': 1.8302, 'learning_rate': 0.0002513482719331561, 'epoch': 0.19}

 19%|█▉        | 527/2733 [1:58:07<8:22:30, 13.67s/it]
 19%|█▉        | 528/2733 [1:58:20<8:17:06, 13.53s/it]
                                                      
{'loss': 1.7262, 'learning_rate': 0.0002512343334599316, 'epoch': 0.19}

 19%|█▉        | 528/2733 [1:58:20<8:17:06, 13.53s/it]
 19%|█▉        | 529/2733 [1:58:34<8:18:10, 13.56s/it]
                                                      
{'loss': 1.8259, 'learning_rate': 0.00025112039498670715, 'epoch': 0.19}

 19%|█▉        | 529/2733 [1:58:34<8:18:10, 13.56s/it]
 19%|█▉        | 530/2733 [1:58:48<8:22:05, 13.67s/it]
                                                      
{'loss': 1.6517, 'learning_rate': 0.0002510064565134827, 'epoch': 0.19}

 19%|█▉        | 530/2733 [1:58:48<8:22:05, 13.67s/it]
 19%|█▉        | 531/2733 [1:59:01<8:18:41, 13.59s/it]
                                                      
{'loss': 1.7074, 'learning_rate': 0.0002508925180402582, 'epoch': 0.19}

 19%|█▉        | 531/2733 [1:59:01<8:18:41, 13.59s/it]
 19%|█▉        | 532/2733 [1:59:14<8:13:53, 13.46s/it]
                                                      
{'loss': 1.7153, 'learning_rate': 0.0002507785795670338, 'epoch': 0.19}

 19%|█▉        | 532/2733 [1:59:14<8:13:53, 13.46s/it]
 20%|█▉        | 533/2733 [1:59:28<8:12:40, 13.44s/it]
                                                      
{'loss': 1.6822, 'learning_rate': 0.00025066464109380933, 'epoch': 0.2}

 20%|█▉        | 533/2733 [1:59:28<8:12:40, 13.44s/it]
 20%|█▉        | 534/2733 [1:59:40<8:04:39, 13.22s/it]
                                                      
{'loss': 1.7443, 'learning_rate': 0.00025055070262058486, 'epoch': 0.2}

 20%|█▉        | 534/2733 [1:59:40<8:04:39, 13.22s/it]
 20%|█▉        | 535/2733 [1:59:54<8:05:49, 13.26s/it]
                                                      
{'loss': 1.7663, 'learning_rate': 0.0002504367641473604, 'epoch': 0.2}

 20%|█▉        | 535/2733 [1:59:54<8:05:49, 13.26s/it]
 20%|█▉        | 536/2733 [2:00:07<8:09:55, 13.38s/it]
                                                      
{'loss': 1.6835, 'learning_rate': 0.0002503228256741359, 'epoch': 0.2}

 20%|█▉        | 536/2733 [2:00:07<8:09:55, 13.38s/it]
 20%|█▉        | 537/2733 [2:00:21<8:12:53, 13.47s/it]
                                                      
{'loss': 1.7685, 'learning_rate': 0.0002502088872009115, 'epoch': 0.2}

 20%|█▉        | 537/2733 [2:00:21<8:12:53, 13.47s/it]
 20%|█▉        | 538/2733 [2:00:34<8:09:30, 13.38s/it]
                                                      
{'loss': 1.7151, 'learning_rate': 0.00025009494872768704, 'epoch': 0.2}

 20%|█▉        | 538/2733 [2:00:34<8:09:30, 13.38s/it]
 20%|█▉        | 539/2733 [2:00:47<8:03:34, 13.22s/it]
                                                      
{'loss': 1.686, 'learning_rate': 0.00024998101025446257, 'epoch': 0.2}

 20%|█▉        | 539/2733 [2:00:47<8:03:34, 13.22s/it]
 20%|█▉        | 540/2733 [2:01:01<8:14:49, 13.54s/it]
                                                      
{'loss': 1.8126, 'learning_rate': 0.0002498670717812381, 'epoch': 0.2}

 20%|█▉        | 540/2733 [2:01:01<8:14:49, 13.54s/it]
 20%|█▉        | 541/2733 [2:01:15<8:16:18, 13.59s/it]
                                                      
{'loss': 1.6498, 'learning_rate': 0.0002497531333080137, 'epoch': 0.2}

 20%|█▉        | 541/2733 [2:01:15<8:16:18, 13.59s/it]
 20%|█▉        | 542/2733 [2:01:29<8:19:06, 13.67s/it]
                                                      
{'loss': 1.7355, 'learning_rate': 0.0002496391948347892, 'epoch': 0.2}

 20%|█▉        | 542/2733 [2:01:29<8:19:06, 13.67s/it]
 20%|█▉        | 543/2733 [2:01:43<8:20:34, 13.71s/it]
                                                      
{'loss': 1.7893, 'learning_rate': 0.00024952525636156474, 'epoch': 0.2}

 20%|█▉        | 543/2733 [2:01:43<8:20:34, 13.71s/it]
 20%|█▉        | 544/2733 [2:01:56<8:12:38, 13.50s/it]
                                                      
{'loss': 1.6953, 'learning_rate': 0.00024941131788834027, 'epoch': 0.2}

 20%|█▉        | 544/2733 [2:01:56<8:12:38, 13.50s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 20%|█▉        | 545/2733 [2:02:10<8:19:24, 13.70s/it]
                                                      
{'loss': 1.7212, 'learning_rate': 0.0002492973794151158, 'epoch': 0.2}

 20%|█▉        | 545/2733 [2:02:10<8:19:24, 13.70s/it]
 20%|█▉        | 546/2733 [2:02:23<8:12:55, 13.52s/it]
                                                      
{'loss': 1.6806, 'learning_rate': 0.0002491834409418914, 'epoch': 0.2}

 20%|█▉        | 546/2733 [2:02:23<8:12:55, 13.52s/it]
 20%|██        | 547/2733 [2:02:36<8:12:06, 13.51s/it]
                                                      
{'loss': 1.7023, 'learning_rate': 0.0002490695024686669, 'epoch': 0.2}

 20%|██        | 547/2733 [2:02:36<8:12:06, 13.51s/it]
 20%|██        | 548/2733 [2:02:50<8:13:46, 13.56s/it]
                                                      
{'loss': 1.6882, 'learning_rate': 0.00024895556399544245, 'epoch': 0.2}

 20%|██        | 548/2733 [2:02:50<8:13:46, 13.56s/it]
 20%|██        | 549/2733 [2:03:04<8:15:02, 13.60s/it]
                                                      
{'loss': 1.8668, 'learning_rate': 0.000248841625522218, 'epoch': 0.2}

 20%|██        | 549/2733 [2:03:04<8:15:02, 13.60s/it]
 20%|██        | 550/2733 [2:03:18<8:19:52, 13.74s/it]
                                                      
{'loss': 1.703, 'learning_rate': 0.0002487276870489935, 'epoch': 0.2}

 20%|██        | 550/2733 [2:03:18<8:19:52, 13.74s/it]
 20%|██        | 551/2733 [2:03:32<8:22:49, 13.83s/it]
                                                      
{'loss': 1.5986, 'learning_rate': 0.0002486137485757691, 'epoch': 0.2}

 20%|██        | 551/2733 [2:03:32<8:22:49, 13.83s/it]
 20%|██        | 552/2733 [2:03:45<8:09:37, 13.47s/it]
                                                      
{'loss': 1.6984, 'learning_rate': 0.00024849981010254457, 'epoch': 0.2}

 20%|██        | 552/2733 [2:03:45<8:09:37, 13.47s/it]
 20%|██        | 553/2733 [2:03:58<8:10:59, 13.51s/it]
                                                      
{'loss': 1.7819, 'learning_rate': 0.00024838587162932015, 'epoch': 0.2}

 20%|██        | 553/2733 [2:03:58<8:10:59, 13.51s/it]
 20%|██        | 554/2733 [2:04:12<8:10:27, 13.50s/it]
                                                      
{'loss': 1.6893, 'learning_rate': 0.0002482719331560957, 'epoch': 0.2}

 20%|██        | 554/2733 [2:04:12<8:10:27, 13.50s/it]
 20%|██        | 555/2733 [2:04:25<8:06:33, 13.40s/it]
                                                      
{'loss': 1.7545, 'learning_rate': 0.0002481579946828712, 'epoch': 0.2}

 20%|██        | 555/2733 [2:04:25<8:06:33, 13.40s/it]
 20%|██        | 556/2733 [2:04:38<8:03:16, 13.32s/it]
                                                      
{'loss': 1.6327, 'learning_rate': 0.00024804405620964675, 'epoch': 0.2}

 20%|██        | 556/2733 [2:04:38<8:03:16, 13.32s/it]
 20%|██        | 557/2733 [2:04:52<8:07:31, 13.44s/it]
                                                      
{'loss': 1.7007, 'learning_rate': 0.0002479301177364223, 'epoch': 0.2}

 20%|██        | 557/2733 [2:04:52<8:07:31, 13.44s/it]
 20%|██        | 558/2733 [2:05:06<8:13:29, 13.61s/it]
                                                      
{'loss': 1.7247, 'learning_rate': 0.00024781617926319786, 'epoch': 0.2}

 20%|██        | 558/2733 [2:05:06<8:13:29, 13.61s/it]
 20%|██        | 559/2733 [2:05:19<8:15:01, 13.66s/it]
                                                      
{'loss': 1.7796, 'learning_rate': 0.0002477022407899734, 'epoch': 0.2}

 20%|██        | 559/2733 [2:05:19<8:15:01, 13.66s/it]
 20%|██        | 560/2733 [2:05:33<8:09:10, 13.51s/it]
                                                      
{'loss': 1.7025, 'learning_rate': 0.0002475883023167489, 'epoch': 0.2}

 20%|██        | 560/2733 [2:05:33<8:09:10, 13.51s/it]
 21%|██        | 561/2733 [2:05:46<8:07:49, 13.48s/it]
                                                      
{'loss': 1.7913, 'learning_rate': 0.00024747436384352445, 'epoch': 0.21}

 21%|██        | 561/2733 [2:05:46<8:07:49, 13.48s/it]
 21%|██        | 562/2733 [2:05:59<8:04:12, 13.38s/it]
                                                      
{'loss': 1.7159, 'learning_rate': 0.00024736042537030004, 'epoch': 0.21}

 21%|██        | 562/2733 [2:05:59<8:04:12, 13.38s/it]
 21%|██        | 563/2733 [2:06:12<7:58:21, 13.23s/it]
                                                      
{'loss': 1.6353, 'learning_rate': 0.00024724648689707557, 'epoch': 0.21}

 21%|██        | 563/2733 [2:06:12<7:58:21, 13.23s/it]
 21%|██        | 564/2733 [2:06:25<7:56:18, 13.18s/it]
                                                      
{'loss': 1.7472, 'learning_rate': 0.0002471325484238511, 'epoch': 0.21}

 21%|██        | 564/2733 [2:06:25<7:56:18, 13.18s/it]
 21%|██        | 565/2733 [2:06:38<7:51:15, 13.04s/it]
                                                      
{'loss': 1.751, 'learning_rate': 0.0002470186099506266, 'epoch': 0.21}

 21%|██        | 565/2733 [2:06:38<7:51:15, 13.04s/it]
 21%|██        | 566/2733 [2:06:51<7:50:10, 13.02s/it]
                                                      
{'loss': 1.7359, 'learning_rate': 0.00024690467147740216, 'epoch': 0.21}

 21%|██        | 566/2733 [2:06:51<7:50:10, 13.02s/it]
 21%|██        | 567/2733 [2:07:03<7:43:51, 12.85s/it]
                                                      
{'loss': 1.7612, 'learning_rate': 0.00024679073300417774, 'epoch': 0.21}

 21%|██        | 567/2733 [2:07:03<7:43:51, 12.85s/it]
 21%|██        | 568/2733 [2:07:17<7:51:50, 13.08s/it]
                                                      
{'loss': 1.7504, 'learning_rate': 0.00024667679453095327, 'epoch': 0.21}

 21%|██        | 568/2733 [2:07:17<7:51:50, 13.08s/it]
 21%|██        | 569/2733 [2:07:30<7:51:54, 13.08s/it]
                                                      
{'loss': 1.7444, 'learning_rate': 0.0002465628560577288, 'epoch': 0.21}

 21%|██        | 569/2733 [2:07:30<7:51:54, 13.08s/it]
 21%|██        | 570/2733 [2:07:44<8:04:09, 13.43s/it]
                                                      
{'loss': 1.7773, 'learning_rate': 0.00024644891758450433, 'epoch': 0.21}

 21%|██        | 570/2733 [2:07:44<8:04:09, 13.43s/it]
 21%|██        | 571/2733 [2:07:58<8:05:56, 13.49s/it]
                                                      
{'loss': 1.6905, 'learning_rate': 0.00024633497911127986, 'epoch': 0.21}

 21%|██        | 571/2733 [2:07:58<8:05:56, 13.49s/it]
 21%|██        | 572/2733 [2:08:11<8:00:45, 13.35s/it]
                                                      
{'loss': 1.7419, 'learning_rate': 0.00024622104063805545, 'epoch': 0.21}

 21%|██        | 572/2733 [2:08:11<8:00:45, 13.35s/it]
 21%|██        | 573/2733 [2:08:25<8:07:10, 13.53s/it]
                                                      
{'loss': 1.7796, 'learning_rate': 0.000246107102164831, 'epoch': 0.21}

 21%|██        | 573/2733 [2:08:25<8:07:10, 13.53s/it]
 21%|██        | 574/2733 [2:08:38<8:02:33, 13.41s/it]
                                                      
{'loss': 1.705, 'learning_rate': 0.0002459931636916065, 'epoch': 0.21}

 21%|██        | 574/2733 [2:08:38<8:02:33, 13.41s/it]
 21%|██        | 575/2733 [2:08:52<8:04:57, 13.48s/it]
                                                      
{'loss': 1.694, 'learning_rate': 0.00024587922521838204, 'epoch': 0.21}

 21%|██        | 575/2733 [2:08:52<8:04:57, 13.48s/it]
 21%|██        | 576/2733 [2:09:04<7:57:21, 13.28s/it]
                                                      
{'loss': 1.6514, 'learning_rate': 0.0002457652867451576, 'epoch': 0.21}

 21%|██        | 576/2733 [2:09:04<7:57:21, 13.28s/it]
 21%|██        | 577/2733 [2:09:18<8:01:11, 13.39s/it]
                                                      
{'loss': 1.7308, 'learning_rate': 0.00024565134827193315, 'epoch': 0.21}

 21%|██        | 577/2733 [2:09:18<8:01:11, 13.39s/it]
 21%|██        | 578/2733 [2:09:31<7:59:16, 13.34s/it]
                                                      
{'loss': 1.7587, 'learning_rate': 0.0002455374097987087, 'epoch': 0.21}

 21%|██        | 578/2733 [2:09:31<7:59:16, 13.34s/it]
 21%|██        | 579/2733 [2:09:45<7:58:14, 13.32s/it]
                                                      
{'loss': 1.7224, 'learning_rate': 0.0002454234713254842, 'epoch': 0.21}

 21%|██        | 579/2733 [2:09:45<7:58:14, 13.32s/it]
 21%|██        | 580/2733 [2:09:58<8:01:15, 13.41s/it]
                                                      
{'loss': 1.703, 'learning_rate': 0.00024530953285225974, 'epoch': 0.21}

 21%|██        | 580/2733 [2:09:58<8:01:15, 13.41s/it]
 21%|██▏       | 581/2733 [2:10:12<8:05:49, 13.55s/it]
                                                      
{'loss': 1.7257, 'learning_rate': 0.00024519559437903533, 'epoch': 0.21}

 21%|██▏       | 581/2733 [2:10:12<8:05:49, 13.55s/it]
 21%|██▏       | 582/2733 [2:10:26<8:06:33, 13.57s/it]
                                                      
{'loss': 1.7029, 'learning_rate': 0.00024508165590581086, 'epoch': 0.21}

 21%|██▏       | 582/2733 [2:10:26<8:06:33, 13.57s/it]
 21%|██▏       | 583/2733 [2:10:39<8:01:31, 13.44s/it]
                                                      
{'loss': 1.7045, 'learning_rate': 0.0002449677174325864, 'epoch': 0.21}

 21%|██▏       | 583/2733 [2:10:39<8:01:31, 13.44s/it]
 21%|██▏       | 584/2733 [2:10:52<7:58:24, 13.36s/it]
                                                      
{'loss': 1.6928, 'learning_rate': 0.0002448537789593619, 'epoch': 0.21}

 21%|██▏       | 584/2733 [2:10:52<7:58:24, 13.36s/it]
 21%|██▏       | 585/2733 [2:11:05<7:51:45, 13.18s/it]
                                                      
{'loss': 1.6506, 'learning_rate': 0.00024473984048613745, 'epoch': 0.21}

 21%|██▏       | 585/2733 [2:11:05<7:51:45, 13.18s/it]
 21%|██▏       | 586/2733 [2:11:18<7:49:03, 13.11s/it]
                                                      
{'loss': 1.5779, 'learning_rate': 0.00024462590201291303, 'epoch': 0.21}

 21%|██▏       | 586/2733 [2:11:18<7:49:03, 13.11s/it]
 21%|██▏       | 587/2733 [2:11:31<7:55:24, 13.29s/it]
                                                      
{'loss': 1.6211, 'learning_rate': 0.0002445119635396885, 'epoch': 0.21}

 21%|██▏       | 587/2733 [2:11:31<7:55:24, 13.29s/it]
 22%|██▏       | 588/2733 [2:11:45<7:59:33, 13.41s/it]
                                                      
{'loss': 1.7122, 'learning_rate': 0.0002443980250664641, 'epoch': 0.22}

 22%|██▏       | 588/2733 [2:11:45<7:59:33, 13.41s/it]
 22%|██▏       | 589/2733 [2:11:59<8:02:39, 13.51s/it]
                                                      
{'loss': 1.8481, 'learning_rate': 0.0002442840865932396, 'epoch': 0.22}

 22%|██▏       | 589/2733 [2:11:59<8:02:39, 13.51s/it]
 22%|██▏       | 590/2733 [2:12:12<7:59:03, 13.41s/it]
                                                      
{'loss': 1.6451, 'learning_rate': 0.0002441701481200152, 'epoch': 0.22}

 22%|██▏       | 590/2733 [2:12:12<7:59:03, 13.41s/it]
 22%|██▏       | 591/2733 [2:12:26<8:04:41, 13.58s/it]
                                                      
{'loss': 1.7901, 'learning_rate': 0.00024405620964679071, 'epoch': 0.22}

 22%|██▏       | 591/2733 [2:12:26<8:04:41, 13.58s/it]
 22%|██▏       | 592/2733 [2:12:40<8:10:45, 13.75s/it]
                                                      
{'loss': 1.7588, 'learning_rate': 0.00024394227117356624, 'epoch': 0.22}

 22%|██▏       | 592/2733 [2:12:40<8:10:45, 13.75s/it]
 22%|██▏       | 593/2733 [2:12:53<7:57:28, 13.39s/it]
                                                      
{'loss': 1.7294, 'learning_rate': 0.0002438283327003418, 'epoch': 0.22}

 22%|██▏       | 593/2733 [2:12:53<7:57:28, 13.39s/it]
 22%|██▏       | 594/2733 [2:13:07<8:03:29, 13.56s/it]
                                                      
{'loss': 1.7364, 'learning_rate': 0.00024371439422711733, 'epoch': 0.22}

 22%|██▏       | 594/2733 [2:13:07<8:03:29, 13.56s/it]
 22%|██▏       | 595/2733 [2:13:20<7:59:59, 13.47s/it]
                                                      
{'loss': 1.6971, 'learning_rate': 0.0002436004557538929, 'epoch': 0.22}

 22%|██▏       | 595/2733 [2:13:20<7:59:59, 13.47s/it]
 22%|██▏       | 596/2733 [2:13:34<8:08:05, 13.70s/it]
                                                      
{'loss': 1.8478, 'learning_rate': 0.00024348651728066842, 'epoch': 0.22}

 22%|██▏       | 596/2733 [2:13:34<8:08:05, 13.70s/it]
 22%|██▏       | 597/2733 [2:13:47<8:04:02, 13.60s/it]
                                                      
{'loss': 1.6228, 'learning_rate': 0.00024337257880744398, 'epoch': 0.22}

 22%|██▏       | 597/2733 [2:13:47<8:04:02, 13.60s/it]
 22%|██▏       | 598/2733 [2:14:01<8:06:33, 13.67s/it]
                                                      
{'loss': 1.7318, 'learning_rate': 0.0002432586403342195, 'epoch': 0.22}

 22%|██▏       | 598/2733 [2:14:01<8:06:33, 13.67s/it]
 22%|██▏       | 599/2733 [2:14:15<8:01:37, 13.54s/it]
                                                      
{'loss': 1.7089, 'learning_rate': 0.00024314470186099504, 'epoch': 0.22}

 22%|██▏       | 599/2733 [2:14:15<8:01:37, 13.54s/it]
 22%|██▏       | 600/2733 [2:14:29<8:07:18, 13.71s/it]
                                                      
{'loss': 1.7708, 'learning_rate': 0.0002430307633877706, 'epoch': 0.22}

 22%|██▏       | 600/2733 [2:14:29<8:07:18, 13.71s/it]
 22%|██▏       | 601/2733 [2:14:41<7:57:57, 13.45s/it]
                                                      
{'loss': 1.6623, 'learning_rate': 0.00024291682491454613, 'epoch': 0.22}

 22%|██▏       | 601/2733 [2:14:41<7:57:57, 13.45s/it]
 22%|██▏       | 602/2733 [2:14:55<7:59:57, 13.51s/it]
                                                      
{'loss': 1.8118, 'learning_rate': 0.00024280288644132168, 'epoch': 0.22}

 22%|██▏       | 602/2733 [2:14:55<7:59:57, 13.51s/it]
 22%|██▏       | 603/2733 [2:15:09<8:02:31, 13.59s/it]
                                                      
{'loss': 1.6695, 'learning_rate': 0.0002426889479680972, 'epoch': 0.22}

 22%|██▏       | 603/2733 [2:15:09<8:02:31, 13.59s/it]
 22%|██▏       | 604/2733 [2:15:22<8:00:39, 13.55s/it]
                                                      
{'loss': 1.6987, 'learning_rate': 0.00024257500949487277, 'epoch': 0.22}

 22%|██▏       | 604/2733 [2:15:22<8:00:39, 13.55s/it]
 22%|██▏       | 605/2733 [2:15:35<7:53:30, 13.35s/it]
                                                      
{'loss': 1.7594, 'learning_rate': 0.0002424610710216483, 'epoch': 0.22}

 22%|██▏       | 605/2733 [2:15:35<7:53:30, 13.35s/it]
 22%|██▏       | 606/2733 [2:15:49<7:55:33, 13.41s/it]
                                                      
{'loss': 1.734, 'learning_rate': 0.0002423471325484238, 'epoch': 0.22}

 22%|██▏       | 606/2733 [2:15:49<7:55:33, 13.41s/it]
 22%|██▏       | 607/2733 [2:16:02<7:49:28, 13.25s/it]
                                                      
{'loss': 1.7443, 'learning_rate': 0.0002422331940751994, 'epoch': 0.22}

 22%|██▏       | 607/2733 [2:16:02<7:49:28, 13.25s/it]
 22%|██▏       | 608/2733 [2:16:15<7:54:24, 13.40s/it]
                                                      
{'loss': 1.6828, 'learning_rate': 0.0002421192556019749, 'epoch': 0.22}

 22%|██▏       | 608/2733 [2:16:15<7:54:24, 13.40s/it]
 22%|██▏       | 609/2733 [2:16:29<7:59:19, 13.54s/it]
                                                      
{'loss': 1.6354, 'learning_rate': 0.00024200531712875048, 'epoch': 0.22}

 22%|██▏       | 609/2733 [2:16:29<7:59:19, 13.54s/it]
 22%|██▏       | 610/2733 [2:16:43<7:56:47, 13.47s/it]
                                                      
{'loss': 1.6166, 'learning_rate': 0.00024189137865552598, 'epoch': 0.22}

 22%|██▏       | 610/2733 [2:16:43<7:56:47, 13.47s/it]
 22%|██▏       | 611/2733 [2:16:56<8:00:14, 13.58s/it]
                                                      
{'loss': 1.7297, 'learning_rate': 0.00024177744018230156, 'epoch': 0.22}

 22%|██▏       | 611/2733 [2:16:56<8:00:14, 13.58s/it]
 22%|██▏       | 612/2733 [2:17:10<7:58:18, 13.53s/it]
                                                      
{'loss': 1.6397, 'learning_rate': 0.00024166350170907707, 'epoch': 0.22}

 22%|██▏       | 612/2733 [2:17:10<7:58:18, 13.53s/it]
 22%|██▏       | 613/2733 [2:17:23<7:58:21, 13.54s/it]
                                                      
{'loss': 1.8557, 'learning_rate': 0.0002415495632358526, 'epoch': 0.22}

 22%|██▏       | 613/2733 [2:17:23<7:58:21, 13.54s/it]
 22%|██▏       | 614/2733 [2:17:36<7:45:15, 13.17s/it]
                                                      
{'loss': 1.6605, 'learning_rate': 0.00024143562476262816, 'epoch': 0.22}

 22%|██▏       | 614/2733 [2:17:36<7:45:15, 13.17s/it]
 23%|██▎       | 615/2733 [2:17:49<7:42:14, 13.09s/it]
                                                      
{'loss': 1.7558, 'learning_rate': 0.00024132168628940369, 'epoch': 0.23}

 23%|██▎       | 615/2733 [2:17:49<7:42:14, 13.09s/it]
 23%|██▎       | 616/2733 [2:18:02<7:47:16, 13.24s/it]
                                                      
{'loss': 1.704, 'learning_rate': 0.00024120774781617924, 'epoch': 0.23}

 23%|██▎       | 616/2733 [2:18:02<7:47:16, 13.24s/it]
 23%|██▎       | 617/2733 [2:18:15<7:46:50, 13.24s/it]
                                                      
{'loss': 1.6188, 'learning_rate': 0.00024109380934295477, 'epoch': 0.23}

 23%|██▎       | 617/2733 [2:18:15<7:46:50, 13.24s/it]
 23%|██▎       | 618/2733 [2:18:29<7:45:17, 13.20s/it]
                                                      
{'loss': 1.6626, 'learning_rate': 0.00024097987086973033, 'epoch': 0.23}

 23%|██▎       | 618/2733 [2:18:29<7:45:17, 13.20s/it]
 23%|██▎       | 619/2733 [2:18:41<7:38:06, 13.00s/it]
                                                      
{'loss': 1.6949, 'learning_rate': 0.00024086593239650586, 'epoch': 0.23}

 23%|██▎       | 619/2733 [2:18:41<7:38:06, 13.00s/it]
 23%|██▎       | 620/2733 [2:18:54<7:39:51, 13.06s/it]
                                                      
{'loss': 1.6601, 'learning_rate': 0.0002407519939232814, 'epoch': 0.23}

 23%|██▎       | 620/2733 [2:18:54<7:39:51, 13.06s/it]
 23%|██▎       | 621/2733 [2:19:07<7:41:03, 13.10s/it]
                                                      
{'loss': 1.7602, 'learning_rate': 0.00024063805545005695, 'epoch': 0.23}

 23%|██▎       | 621/2733 [2:19:08<7:41:03, 13.10s/it]
 23%|██▎       | 622/2733 [2:19:21<7:45:09, 13.22s/it]
                                                      
{'loss': 1.7158, 'learning_rate': 0.00024052411697683248, 'epoch': 0.23}

 23%|██▎       | 622/2733 [2:19:21<7:45:09, 13.22s/it]
 23%|██▎       | 623/2733 [2:19:35<7:49:43, 13.36s/it]
                                                      
{'loss': 1.7606, 'learning_rate': 0.00024041017850360804, 'epoch': 0.23}

 23%|██▎       | 623/2733 [2:19:35<7:49:43, 13.36s/it]
 23%|██▎       | 624/2733 [2:19:47<7:43:08, 13.18s/it]
                                                      
{'loss': 1.5455, 'learning_rate': 0.00024029624003038357, 'epoch': 0.23}

 23%|██▎       | 624/2733 [2:19:47<7:43:08, 13.18s/it]
 23%|██▎       | 625/2733 [2:20:01<7:48:32, 13.34s/it]
                                                      
{'loss': 1.7705, 'learning_rate': 0.00024018230155715912, 'epoch': 0.23}

 23%|██▎       | 625/2733 [2:20:01<7:48:32, 13.34s/it]
 23%|██▎       | 626/2733 [2:20:14<7:46:43, 13.29s/it]
                                                      
{'loss': 1.6715, 'learning_rate': 0.00024006836308393466, 'epoch': 0.23}

 23%|██▎       | 626/2733 [2:20:14<7:46:43, 13.29s/it]
 23%|██▎       | 627/2733 [2:20:28<7:52:03, 13.45s/it]
                                                      
{'loss': 1.6549, 'learning_rate': 0.00023995442461071019, 'epoch': 0.23}

 23%|██▎       | 627/2733 [2:20:28<7:52:03, 13.45s/it]
 23%|██▎       | 628/2733 [2:20:42<7:57:05, 13.60s/it]
                                                      
{'loss': 1.7083, 'learning_rate': 0.00023984048613748574, 'epoch': 0.23}

 23%|██▎       | 628/2733 [2:20:42<7:57:05, 13.60s/it]
 23%|██▎       | 629/2733 [2:20:56<8:01:56, 13.74s/it]
                                                      
{'loss': 1.6629, 'learning_rate': 0.00023972654766426127, 'epoch': 0.23}

 23%|██▎       | 629/2733 [2:20:56<8:01:56, 13.74s/it]
 23%|██▎       | 630/2733 [2:21:10<8:00:26, 13.71s/it]
                                                      
{'loss': 1.6889, 'learning_rate': 0.00023961260919103683, 'epoch': 0.23}

 23%|██▎       | 630/2733 [2:21:10<8:00:26, 13.71s/it]
 23%|██▎       | 631/2733 [2:21:24<8:00:34, 13.72s/it]
                                                      
{'loss': 1.7726, 'learning_rate': 0.00023949867071781236, 'epoch': 0.23}

 23%|██▎       | 631/2733 [2:21:24<8:00:34, 13.72s/it]
 23%|██▎       | 632/2733 [2:21:37<8:00:08, 13.71s/it]
                                                      
{'loss': 1.7724, 'learning_rate': 0.00023938473224458792, 'epoch': 0.23}

 23%|██▎       | 632/2733 [2:21:37<8:00:08, 13.71s/it]
 23%|██▎       | 633/2733 [2:21:51<7:56:36, 13.62s/it]
                                                      
{'loss': 1.671, 'learning_rate': 0.00023927079377136345, 'epoch': 0.23}

 23%|██▎       | 633/2733 [2:21:51<7:56:36, 13.62s/it]
 23%|██▎       | 634/2733 [2:22:03<7:42:49, 13.23s/it]
                                                      
{'loss': 1.6709, 'learning_rate': 0.00023915685529813898, 'epoch': 0.23}

 23%|██▎       | 634/2733 [2:22:03<7:42:49, 13.23s/it]
 23%|██▎       | 635/2733 [2:22:16<7:42:44, 13.23s/it]
                                                      
{'loss': 1.6903, 'learning_rate': 0.00023904291682491454, 'epoch': 0.23}

 23%|██▎       | 635/2733 [2:22:16<7:42:44, 13.23s/it]
 23%|██▎       | 636/2733 [2:22:29<7:42:17, 13.23s/it]
                                                      
{'loss': 1.6633, 'learning_rate': 0.00023892897835169007, 'epoch': 0.23}

 23%|██▎       | 636/2733 [2:22:29<7:42:17, 13.23s/it]
 23%|██▎       | 637/2733 [2:22:43<7:46:31, 13.35s/it]
                                                      
{'loss': 1.7257, 'learning_rate': 0.00023881503987846562, 'epoch': 0.23}

 23%|██▎       | 637/2733 [2:22:43<7:46:31, 13.35s/it]
 23%|██▎       | 638/2733 [2:22:57<7:53:36, 13.56s/it]
                                                      
{'loss': 1.6974, 'learning_rate': 0.00023870110140524116, 'epoch': 0.23}

 23%|██▎       | 638/2733 [2:22:57<7:53:36, 13.56s/it]
 23%|██▎       | 639/2733 [2:23:11<7:58:12, 13.70s/it]
                                                      
{'loss': 1.7143, 'learning_rate': 0.0002385871629320167, 'epoch': 0.23}

 23%|██▎       | 639/2733 [2:23:11<7:58:12, 13.70s/it]
 23%|██▎       | 640/2733 [2:23:24<7:48:19, 13.43s/it]
                                                      
{'loss': 1.7178, 'learning_rate': 0.00023847322445879224, 'epoch': 0.23}

 23%|██▎       | 640/2733 [2:23:24<7:48:19, 13.43s/it]
 23%|██▎       | 641/2733 [2:23:37<7:45:51, 13.36s/it]
                                                      
{'loss': 1.6958, 'learning_rate': 0.00023835928598556777, 'epoch': 0.23}

 23%|██▎       | 641/2733 [2:23:37<7:45:51, 13.36s/it]
 23%|██▎       | 642/2733 [2:23:49<7:33:15, 13.01s/it]
                                                      
{'loss': 1.7723, 'learning_rate': 0.00023824534751234333, 'epoch': 0.23}

 23%|██▎       | 642/2733 [2:23:49<7:33:15, 13.01s/it]
 24%|██▎       | 643/2733 [2:24:03<7:37:41, 13.14s/it]
                                                      
{'loss': 1.7038, 'learning_rate': 0.00023813140903911886, 'epoch': 0.24}

 24%|██▎       | 643/2733 [2:24:03<7:37:41, 13.14s/it]
 24%|██▎       | 644/2733 [2:24:16<7:34:35, 13.06s/it]
                                                      
{'loss': 1.7792, 'learning_rate': 0.00023801747056589442, 'epoch': 0.24}

 24%|██▎       | 644/2733 [2:24:16<7:34:35, 13.06s/it]
 24%|██▎       | 645/2733 [2:24:29<7:40:12, 13.22s/it]
                                                      
{'loss': 1.6772, 'learning_rate': 0.00023790353209266992, 'epoch': 0.24}

 24%|██▎       | 645/2733 [2:24:29<7:40:12, 13.22s/it]
 24%|██▎       | 646/2733 [2:24:43<7:41:29, 13.27s/it]
                                                      
{'loss': 1.6533, 'learning_rate': 0.00023778959361944545, 'epoch': 0.24}

 24%|██▎       | 646/2733 [2:24:43<7:41:29, 13.27s/it]
 24%|██▎       | 647/2733 [2:24:56<7:44:23, 13.36s/it]
                                                      
{'loss': 1.6613, 'learning_rate': 0.000237675655146221, 'epoch': 0.24}

 24%|██▎       | 647/2733 [2:24:56<7:44:23, 13.36s/it]
 24%|██▎       | 648/2733 [2:25:09<7:38:47, 13.20s/it]
                                                      
{'loss': 1.7594, 'learning_rate': 0.00023756171667299654, 'epoch': 0.24}

 24%|██▎       | 648/2733 [2:25:09<7:38:47, 13.20s/it]
 24%|██▎       | 649/2733 [2:25:23<7:42:45, 13.32s/it]
                                                      
{'loss': 1.6795, 'learning_rate': 0.0002374477781997721, 'epoch': 0.24}

 24%|██▎       | 649/2733 [2:25:23<7:42:45, 13.32s/it]
 24%|██▍       | 650/2733 [2:25:36<7:45:43, 13.42s/it]
                                                      
{'loss': 1.7478, 'learning_rate': 0.00023733383972654763, 'epoch': 0.24}

 24%|██▍       | 650/2733 [2:25:36<7:45:43, 13.42s/it]
 24%|██▍       | 651/2733 [2:25:49<7:38:21, 13.21s/it]
                                                      
{'loss': 1.6648, 'learning_rate': 0.00023721990125332319, 'epoch': 0.24}

 24%|██▍       | 651/2733 [2:25:49<7:38:21, 13.21s/it]
 24%|██▍       | 652/2733 [2:26:02<7:39:53, 13.26s/it]
                                                      
{'loss': 1.7586, 'learning_rate': 0.00023710596278009872, 'epoch': 0.24}

 24%|██▍       | 652/2733 [2:26:02<7:39:53, 13.26s/it]
 24%|██▍       | 653/2733 [2:26:15<7:37:16, 13.19s/it]
                                                      
{'loss': 1.6423, 'learning_rate': 0.00023699202430687425, 'epoch': 0.24}

 24%|██▍       | 653/2733 [2:26:15<7:37:16, 13.19s/it]
 24%|██▍       | 654/2733 [2:26:29<7:38:58, 13.25s/it]
                                                      
{'loss': 1.8085, 'learning_rate': 0.0002368780858336498, 'epoch': 0.24}

 24%|██▍       | 654/2733 [2:26:29<7:38:58, 13.25s/it]
 24%|██▍       | 655/2733 [2:26:42<7:43:40, 13.39s/it]
                                                      
{'loss': 1.581, 'learning_rate': 0.00023676414736042533, 'epoch': 0.24}

 24%|██▍       | 655/2733 [2:26:42<7:43:40, 13.39s/it]
 24%|██▍       | 656/2733 [2:26:55<7:39:06, 13.26s/it]
                                                      
{'loss': 1.6439, 'learning_rate': 0.0002366502088872009, 'epoch': 0.24}

 24%|██▍       | 656/2733 [2:26:55<7:39:06, 13.26s/it]
 24%|██▍       | 657/2733 [2:27:08<7:34:22, 13.13s/it]
                                                      
{'loss': 1.7302, 'learning_rate': 0.00023653627041397642, 'epoch': 0.24}

 24%|██▍       | 657/2733 [2:27:08<7:34:22, 13.13s/it]
 24%|██▍       | 658/2733 [2:27:21<7:32:22, 13.08s/it]
                                                      
{'loss': 1.7592, 'learning_rate': 0.00023642233194075198, 'epoch': 0.24}

 24%|██▍       | 658/2733 [2:27:21<7:32:22, 13.08s/it]
 24%|██▍       | 659/2733 [2:27:34<7:30:20, 13.03s/it]
                                                      
{'loss': 1.6556, 'learning_rate': 0.0002363083934675275, 'epoch': 0.24}

 24%|██▍       | 659/2733 [2:27:34<7:30:20, 13.03s/it]
 24%|██▍       | 660/2733 [2:27:47<7:33:00, 13.11s/it]
                                                      
{'loss': 1.7294, 'learning_rate': 0.00023619445499430304, 'epoch': 0.24}

 24%|██▍       | 660/2733 [2:27:47<7:33:00, 13.11s/it]
 24%|██▍       | 661/2733 [2:28:01<7:33:16, 13.13s/it]
                                                      
{'loss': 1.7539, 'learning_rate': 0.0002360805165210786, 'epoch': 0.24}

 24%|██▍       | 661/2733 [2:28:01<7:33:16, 13.13s/it]
 24%|██▍       | 662/2733 [2:28:14<7:35:03, 13.18s/it]
                                                      
{'loss': 1.6801, 'learning_rate': 0.00023596657804785413, 'epoch': 0.24}

 24%|██▍       | 662/2733 [2:28:14<7:35:03, 13.18s/it]
 24%|██▍       | 663/2733 [2:28:28<7:39:34, 13.32s/it]
                                                      
{'loss': 1.7315, 'learning_rate': 0.00023585263957462968, 'epoch': 0.24}

 24%|██▍       | 663/2733 [2:28:28<7:39:34, 13.32s/it]
 24%|██▍       | 664/2733 [2:28:41<7:35:44, 13.22s/it]
                                                      
{'loss': 1.6861, 'learning_rate': 0.00023573870110140522, 'epoch': 0.24}

 24%|██▍       | 664/2733 [2:28:41<7:35:44, 13.22s/it]
 24%|██▍       | 665/2733 [2:28:54<7:43:02, 13.43s/it]
                                                      
{'loss': 1.7857, 'learning_rate': 0.00023562476262818077, 'epoch': 0.24}

 24%|██▍       | 665/2733 [2:28:54<7:43:02, 13.43s/it]
 24%|██▍       | 666/2733 [2:29:08<7:41:49, 13.41s/it]
                                                      
{'loss': 1.7813, 'learning_rate': 0.0002355108241549563, 'epoch': 0.24}

 24%|██▍       | 666/2733 [2:29:08<7:41:49, 13.41s/it]
 24%|██▍       | 667/2733 [2:29:21<7:44:04, 13.48s/it]
                                                      
{'loss': 1.7645, 'learning_rate': 0.00023539688568173183, 'epoch': 0.24}

 24%|██▍       | 667/2733 [2:29:21<7:44:04, 13.48s/it]
 24%|██▍       | 668/2733 [2:29:35<7:42:38, 13.44s/it]
                                                      
{'loss': 1.7598, 'learning_rate': 0.0002352829472085074, 'epoch': 0.24}

 24%|██▍       | 668/2733 [2:29:35<7:42:38, 13.44s/it]
 24%|██▍       | 669/2733 [2:29:49<7:47:00, 13.58s/it]
                                                      
{'loss': 1.776, 'learning_rate': 0.00023516900873528292, 'epoch': 0.24}

 24%|██▍       | 669/2733 [2:29:49<7:47:00, 13.58s/it]
 25%|██▍       | 670/2733 [2:30:03<7:53:45, 13.78s/it]
                                                      
{'loss': 1.7032, 'learning_rate': 0.00023505507026205848, 'epoch': 0.25}

 25%|██▍       | 670/2733 [2:30:03<7:53:45, 13.78s/it]
 25%|██▍       | 671/2733 [2:30:16<7:50:34, 13.69s/it]
                                                      
{'loss': 1.6377, 'learning_rate': 0.000234941131788834, 'epoch': 0.25}

 25%|██▍       | 671/2733 [2:30:16<7:50:34, 13.69s/it]
 25%|██▍       | 672/2733 [2:30:30<7:48:47, 13.65s/it]
                                                      
{'loss': 1.6848, 'learning_rate': 0.00023482719331560957, 'epoch': 0.25}

 25%|██▍       | 672/2733 [2:30:30<7:48:47, 13.65s/it]
 25%|██▍       | 673/2733 [2:30:44<7:54:40, 13.83s/it]
                                                      
{'loss': 1.785, 'learning_rate': 0.0002347132548423851, 'epoch': 0.25}

 25%|██▍       | 673/2733 [2:30:44<7:54:40, 13.83s/it]
 25%|██▍       | 674/2733 [2:30:58<7:48:48, 13.66s/it]
                                                      
{'loss': 1.705, 'learning_rate': 0.00023459931636916063, 'epoch': 0.25}

 25%|██▍       | 674/2733 [2:30:58<7:48:48, 13.66s/it]
 25%|██▍       | 675/2733 [2:31:11<7:43:55, 13.53s/it]
                                                      
{'loss': 1.6737, 'learning_rate': 0.00023448537789593618, 'epoch': 0.25}

 25%|██▍       | 675/2733 [2:31:11<7:43:55, 13.53s/it]
 25%|██▍       | 676/2733 [2:31:24<7:44:43, 13.56s/it]
                                                      
{'loss': 1.6182, 'learning_rate': 0.00023437143942271171, 'epoch': 0.25}

 25%|██▍       | 676/2733 [2:31:24<7:44:43, 13.56s/it]
 25%|██▍       | 677/2733 [2:31:37<7:37:43, 13.36s/it]
                                                      
{'loss': 1.6768, 'learning_rate': 0.00023425750094948727, 'epoch': 0.25}

 25%|██▍       | 677/2733 [2:31:37<7:37:43, 13.36s/it]
 25%|██▍       | 678/2733 [2:31:51<7:44:55, 13.57s/it]
                                                      
{'loss': 1.6673, 'learning_rate': 0.0002341435624762628, 'epoch': 0.25}

 25%|██▍       | 678/2733 [2:31:51<7:44:55, 13.57s/it]
 25%|██▍       | 679/2733 [2:32:04<7:33:57, 13.26s/it]
                                                      
{'loss': 1.727, 'learning_rate': 0.00023402962400303836, 'epoch': 0.25}

 25%|██▍       | 679/2733 [2:32:04<7:33:57, 13.26s/it]
 25%|██▍       | 680/2733 [2:32:17<7:37:03, 13.36s/it]
                                                      
{'loss': 1.5915, 'learning_rate': 0.0002339156855298139, 'epoch': 0.25}

 25%|██▍       | 680/2733 [2:32:17<7:37:03, 13.36s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 25%|██▍       | 681/2733 [2:32:32<7:45:16, 13.60s/it]
                                                      
{'loss': 1.6751, 'learning_rate': 0.0002338017470565894, 'epoch': 0.25}

 25%|██▍       | 681/2733 [2:32:32<7:45:16, 13.60s/it]
 25%|██▍       | 682/2733 [2:32:45<7:47:08, 13.67s/it]
                                                      
{'loss': 1.699, 'learning_rate': 0.00023368780858336498, 'epoch': 0.25}

 25%|██▍       | 682/2733 [2:32:45<7:47:08, 13.67s/it]
 25%|██▍       | 683/2733 [2:32:59<7:47:35, 13.69s/it]
                                                      
{'loss': 1.8164, 'learning_rate': 0.00023357387011014048, 'epoch': 0.25}

 25%|██▍       | 683/2733 [2:32:59<7:47:35, 13.69s/it]
 25%|██▌       | 684/2733 [2:33:13<7:48:37, 13.72s/it]
                                                      
{'loss': 1.6877, 'learning_rate': 0.00023345993163691607, 'epoch': 0.25}

 25%|██▌       | 684/2733 [2:33:13<7:48:37, 13.72s/it]
 25%|██▌       | 685/2733 [2:33:27<7:54:22, 13.90s/it]
                                                      
{'loss': 1.7085, 'learning_rate': 0.00023334599316369157, 'epoch': 0.25}

 25%|██▌       | 685/2733 [2:33:27<7:54:22, 13.90s/it]
 25%|██▌       | 686/2733 [2:33:41<7:49:50, 13.77s/it]
                                                      
{'loss': 1.6194, 'learning_rate': 0.00023323205469046715, 'epoch': 0.25}

 25%|██▌       | 686/2733 [2:33:41<7:49:50, 13.77s/it]
 25%|██▌       | 687/2733 [2:33:54<7:43:24, 13.59s/it]
                                                      
{'loss': 1.7491, 'learning_rate': 0.00023311811621724266, 'epoch': 0.25}

 25%|██▌       | 687/2733 [2:33:54<7:43:24, 13.59s/it]
 25%|██▌       | 688/2733 [2:34:07<7:41:00, 13.53s/it]
                                                      
{'loss': 1.7545, 'learning_rate': 0.0002330041777440182, 'epoch': 0.25}

 25%|██▌       | 688/2733 [2:34:07<7:41:00, 13.53s/it]
 25%|██▌       | 689/2733 [2:34:21<7:38:46, 13.47s/it]
                                                      
{'loss': 1.6017, 'learning_rate': 0.00023289023927079374, 'epoch': 0.25}

 25%|██▌       | 689/2733 [2:34:21<7:38:46, 13.47s/it]
 25%|██▌       | 690/2733 [2:34:34<7:40:03, 13.51s/it]
                                                      
{'loss': 1.7257, 'learning_rate': 0.00023277630079756928, 'epoch': 0.25}

 25%|██▌       | 690/2733 [2:34:34<7:40:03, 13.51s/it]
 25%|██▌       | 691/2733 [2:34:47<7:36:02, 13.40s/it]
                                                      
{'loss': 1.6121, 'learning_rate': 0.00023266236232434483, 'epoch': 0.25}

 25%|██▌       | 691/2733 [2:34:47<7:36:02, 13.40s/it]
 25%|██▌       | 692/2733 [2:35:01<7:40:49, 13.55s/it]
                                                      
{'loss': 1.6641, 'learning_rate': 0.00023254842385112036, 'epoch': 0.25}

 25%|██▌       | 692/2733 [2:35:01<7:40:49, 13.55s/it]
 25%|██▌       | 693/2733 [2:35:15<7:43:00, 13.62s/it]
                                                      
{'loss': 1.7575, 'learning_rate': 0.00023243448537789592, 'epoch': 0.25}

 25%|██▌       | 693/2733 [2:35:15<7:43:00, 13.62s/it]
 25%|██▌       | 694/2733 [2:35:29<7:43:08, 13.63s/it]
                                                      
{'loss': 1.6922, 'learning_rate': 0.00023232054690467145, 'epoch': 0.25}

 25%|██▌       | 694/2733 [2:35:29<7:43:08, 13.63s/it]
 25%|██▌       | 695/2733 [2:35:42<7:39:13, 13.52s/it]
                                                      
{'loss': 1.7697, 'learning_rate': 0.00023220660843144698, 'epoch': 0.25}

 25%|██▌       | 695/2733 [2:35:42<7:39:13, 13.52s/it]
 25%|██▌       | 696/2733 [2:35:56<7:43:05, 13.64s/it]
                                                      
{'loss': 1.7208, 'learning_rate': 0.00023209266995822254, 'epoch': 0.25}

 25%|██▌       | 696/2733 [2:35:56<7:43:05, 13.64s/it]
 26%|██▌       | 697/2733 [2:36:10<7:48:58, 13.82s/it]
                                                      
{'loss': 1.6709, 'learning_rate': 0.00023197873148499807, 'epoch': 0.26}

 26%|██▌       | 697/2733 [2:36:10<7:48:58, 13.82s/it]
 26%|██▌       | 698/2733 [2:36:24<7:46:26, 13.75s/it]
                                                      
{'loss': 1.6053, 'learning_rate': 0.00023186479301177363, 'epoch': 0.26}

 26%|██▌       | 698/2733 [2:36:24<7:46:26, 13.75s/it]
 26%|██▌       | 699/2733 [2:36:37<7:42:40, 13.65s/it]
                                                      
{'loss': 1.6708, 'learning_rate': 0.00023175085453854916, 'epoch': 0.26}

 26%|██▌       | 699/2733 [2:36:37<7:42:40, 13.65s/it]
 26%|██▌       | 700/2733 [2:36:51<7:39:52, 13.57s/it]
                                                      
{'loss': 1.6298, 'learning_rate': 0.00023163691606532471, 'epoch': 0.26}

 26%|██▌       | 700/2733 [2:36:51<7:39:52, 13.57s/it]
 26%|██▌       | 701/2733 [2:37:04<7:36:50, 13.49s/it]
                                                      
{'loss': 1.6592, 'learning_rate': 0.00023152297759210024, 'epoch': 0.26}

 26%|██▌       | 701/2733 [2:37:04<7:36:50, 13.49s/it]
 26%|██▌       | 702/2733 [2:37:18<7:39:33, 13.58s/it]
                                                      
{'loss': 1.7966, 'learning_rate': 0.00023140903911887577, 'epoch': 0.26}

 26%|██▌       | 702/2733 [2:37:18<7:39:33, 13.58s/it]
 26%|██▌       | 703/2733 [2:37:31<7:37:53, 13.53s/it]
                                                      
{'loss': 1.7965, 'learning_rate': 0.00023129510064565133, 'epoch': 0.26}

 26%|██▌       | 703/2733 [2:37:31<7:37:53, 13.53s/it]
 26%|██▌       | 704/2733 [2:37:45<7:42:48, 13.69s/it]
                                                      
{'loss': 1.66, 'learning_rate': 0.00023118116217242686, 'epoch': 0.26}

 26%|██▌       | 704/2733 [2:37:45<7:42:48, 13.69s/it]
 26%|██▌       | 705/2733 [2:37:58<7:37:45, 13.54s/it]
                                                      
{'loss': 1.7823, 'learning_rate': 0.00023106722369920242, 'epoch': 0.26}

 26%|██▌       | 705/2733 [2:37:58<7:37:45, 13.54s/it]
 26%|██▌       | 706/2733 [2:38:12<7:40:58, 13.65s/it]
                                                      
{'loss': 1.8306, 'learning_rate': 0.00023095328522597795, 'epoch': 0.26}

 26%|██▌       | 706/2733 [2:38:12<7:40:58, 13.65s/it]
 26%|██▌       | 707/2733 [2:38:26<7:41:24, 13.66s/it]
                                                      
{'loss': 1.6608, 'learning_rate': 0.0002308393467527535, 'epoch': 0.26}

 26%|██▌       | 707/2733 [2:38:26<7:41:24, 13.66s/it]
 26%|██▌       | 708/2733 [2:38:40<7:42:19, 13.70s/it]
                                                      
{'loss': 1.6983, 'learning_rate': 0.00023072540827952904, 'epoch': 0.26}

 26%|██▌       | 708/2733 [2:38:40<7:42:19, 13.70s/it]
 26%|██▌       | 709/2733 [2:38:53<7:34:15, 13.47s/it]
                                                      
{'loss': 1.6895, 'learning_rate': 0.00023061146980630457, 'epoch': 0.26}

 26%|██▌       | 709/2733 [2:38:53<7:34:15, 13.47s/it]
 26%|██▌       | 710/2733 [2:39:07<7:42:02, 13.70s/it]
                                                      
{'loss': 1.7002, 'learning_rate': 0.00023049753133308013, 'epoch': 0.26}

 26%|██▌       | 710/2733 [2:39:07<7:42:02, 13.70s/it]
 26%|██▌       | 711/2733 [2:39:21<7:42:30, 13.72s/it]
                                                      
{'loss': 1.695, 'learning_rate': 0.00023038359285985566, 'epoch': 0.26}

 26%|██▌       | 711/2733 [2:39:21<7:42:30, 13.72s/it]
 26%|██▌       | 712/2733 [2:39:34<7:37:52, 13.59s/it]
                                                      
{'loss': 1.7987, 'learning_rate': 0.00023026965438663121, 'epoch': 0.26}

 26%|██▌       | 712/2733 [2:39:34<7:37:52, 13.59s/it]
 26%|██▌       | 713/2733 [2:39:47<7:36:37, 13.56s/it]
                                                      
{'loss': 1.6952, 'learning_rate': 0.00023015571591340674, 'epoch': 0.26}

 26%|██▌       | 713/2733 [2:39:47<7:36:37, 13.56s/it]
 26%|██▌       | 714/2733 [2:40:00<7:29:19, 13.35s/it]
                                                      
{'loss': 1.6724, 'learning_rate': 0.0002300417774401823, 'epoch': 0.26}

 26%|██▌       | 714/2733 [2:40:00<7:29:19, 13.35s/it]
 26%|██▌       | 715/2733 [2:40:13<7:25:20, 13.24s/it]
                                                      
{'loss': 1.6327, 'learning_rate': 0.00022992783896695783, 'epoch': 0.26}

 26%|██▌       | 715/2733 [2:40:13<7:25:20, 13.24s/it]
 26%|██▌       | 716/2733 [2:40:26<7:17:51, 13.02s/it]
                                                      
{'loss': 1.6495, 'learning_rate': 0.00022981390049373336, 'epoch': 0.26}

 26%|██▌       | 716/2733 [2:40:26<7:17:51, 13.02s/it]
 26%|██▌       | 717/2733 [2:40:39<7:21:10, 13.13s/it]
                                                      
{'loss': 1.6115, 'learning_rate': 0.00022969996202050892, 'epoch': 0.26}

 26%|██▌       | 717/2733 [2:40:39<7:21:10, 13.13s/it]
 26%|██▋       | 718/2733 [2:40:52<7:18:16, 13.05s/it]
                                                      
{'loss': 1.7497, 'learning_rate': 0.00022958602354728445, 'epoch': 0.26}

 26%|██▋       | 718/2733 [2:40:52<7:18:16, 13.05s/it]
 26%|██▋       | 719/2733 [2:41:05<7:21:10, 13.14s/it]
                                                      
{'loss': 1.6456, 'learning_rate': 0.00022947208507406, 'epoch': 0.26}

 26%|██▋       | 719/2733 [2:41:05<7:21:10, 13.14s/it]
 26%|██▋       | 720/2733 [2:41:19<7:21:53, 13.17s/it]
                                                      
{'loss': 1.7278, 'learning_rate': 0.0002293581466008355, 'epoch': 0.26}

 26%|██▋       | 720/2733 [2:41:19<7:21:53, 13.17s/it]
 26%|██▋       | 721/2733 [2:41:32<7:25:11, 13.28s/it]
                                                      
{'loss': 1.7996, 'learning_rate': 0.0002292442081276111, 'epoch': 0.26}

 26%|██▋       | 721/2733 [2:41:32<7:25:11, 13.28s/it]
 26%|██▋       | 722/2733 [2:41:46<7:35:13, 13.58s/it]
                                                      
{'loss': 1.8006, 'learning_rate': 0.0002291302696543866, 'epoch': 0.26}

 26%|██▋       | 722/2733 [2:41:46<7:35:13, 13.58s/it]
 26%|██▋       | 723/2733 [2:42:00<7:36:57, 13.64s/it]
                                                      
{'loss': 1.7325, 'learning_rate': 0.00022901633118116213, 'epoch': 0.26}

 26%|██▋       | 723/2733 [2:42:00<7:36:57, 13.64s/it]
 26%|██▋       | 724/2733 [2:42:14<7:40:35, 13.76s/it]
                                                      
{'loss': 1.5981, 'learning_rate': 0.00022890239270793769, 'epoch': 0.26}

 26%|██▋       | 724/2733 [2:42:14<7:40:35, 13.76s/it]
 27%|██▋       | 725/2733 [2:42:27<7:33:12, 13.54s/it]
                                                      
{'loss': 1.7266, 'learning_rate': 0.00022878845423471322, 'epoch': 0.27}

 27%|██▋       | 725/2733 [2:42:27<7:33:12, 13.54s/it]
 27%|██▋       | 726/2733 [2:42:40<7:28:26, 13.41s/it]
                                                      
{'loss': 1.696, 'learning_rate': 0.00022867451576148877, 'epoch': 0.27}

 27%|██▋       | 726/2733 [2:42:40<7:28:26, 13.41s/it]
 27%|██▋       | 727/2733 [2:42:53<7:22:11, 13.23s/it]
                                                      
{'loss': 1.716, 'learning_rate': 0.0002285605772882643, 'epoch': 0.27}

 27%|██▋       | 727/2733 [2:42:53<7:22:11, 13.23s/it]
 27%|██▋       | 728/2733 [2:43:07<7:32:39, 13.55s/it]
                                                      
{'loss': 1.6541, 'learning_rate': 0.00022844663881503986, 'epoch': 0.27}

 27%|██▋       | 728/2733 [2:43:07<7:32:39, 13.55s/it]
 27%|██▋       | 729/2733 [2:43:22<7:40:06, 13.78s/it]
                                                      
{'loss': 1.818, 'learning_rate': 0.0002283327003418154, 'epoch': 0.27}

 27%|██▋       | 729/2733 [2:43:22<7:40:06, 13.78s/it]
 27%|██▋       | 730/2733 [2:43:35<7:35:10, 13.63s/it]
                                                      
{'loss': 1.6784, 'learning_rate': 0.00022821876186859092, 'epoch': 0.27}

 27%|██▋       | 730/2733 [2:43:35<7:35:10, 13.63s/it]
 27%|██▋       | 731/2733 [2:43:48<7:32:23, 13.56s/it]
                                                      
{'loss': 1.6782, 'learning_rate': 0.00022810482339536648, 'epoch': 0.27}

 27%|██▋       | 731/2733 [2:43:48<7:32:23, 13.56s/it]
 27%|██▋       | 732/2733 [2:44:02<7:31:59, 13.55s/it]
                                                      
{'loss': 1.727, 'learning_rate': 0.000227990884922142, 'epoch': 0.27}

 27%|██▋       | 732/2733 [2:44:02<7:31:59, 13.55s/it]
 27%|██▋       | 733/2733 [2:44:16<7:37:48, 13.73s/it]
                                                      
{'loss': 1.6449, 'learning_rate': 0.00022787694644891757, 'epoch': 0.27}

 27%|██▋       | 733/2733 [2:44:16<7:37:48, 13.73s/it]
 27%|██▋       | 734/2733 [2:44:30<7:37:09, 13.72s/it]
                                                      
{'loss': 1.7513, 'learning_rate': 0.0002277630079756931, 'epoch': 0.27}

 27%|██▋       | 734/2733 [2:44:30<7:37:09, 13.72s/it]
 27%|██▋       | 735/2733 [2:44:43<7:35:51, 13.69s/it]
                                                      
{'loss': 1.7256, 'learning_rate': 0.00022764906950246866, 'epoch': 0.27}

 27%|██▋       | 735/2733 [2:44:43<7:35:51, 13.69s/it]
 27%|██▋       | 736/2733 [2:44:57<7:36:15, 13.71s/it]
                                                      
{'loss': 1.7009, 'learning_rate': 0.00022753513102924419, 'epoch': 0.27}

 27%|██▋       | 736/2733 [2:44:57<7:36:15, 13.71s/it]
 27%|██▋       | 737/2733 [2:45:11<7:37:21, 13.75s/it]
                                                      
{'loss': 1.7909, 'learning_rate': 0.00022742119255601972, 'epoch': 0.27}

 27%|██▋       | 737/2733 [2:45:11<7:37:21, 13.75s/it]
 27%|██▋       | 738/2733 [2:45:24<7:31:22, 13.58s/it]
                                                      
{'loss': 1.668, 'learning_rate': 0.00022730725408279527, 'epoch': 0.27}

 27%|██▋       | 738/2733 [2:45:24<7:31:22, 13.58s/it]
 27%|██▋       | 739/2733 [2:45:38<7:37:26, 13.76s/it]
                                                      
{'loss': 1.7156, 'learning_rate': 0.0002271933156095708, 'epoch': 0.27}

 27%|██▋       | 739/2733 [2:45:38<7:37:26, 13.76s/it]
 27%|██▋       | 740/2733 [2:45:51<7:28:56, 13.52s/it]
                                                      
{'loss': 1.6482, 'learning_rate': 0.00022707937713634636, 'epoch': 0.27}

 27%|██▋       | 740/2733 [2:45:51<7:28:56, 13.52s/it]
 27%|██▋       | 741/2733 [2:46:05<7:28:34, 13.51s/it]
                                                      
{'loss': 1.7307, 'learning_rate': 0.0002269654386631219, 'epoch': 0.27}

 27%|██▋       | 741/2733 [2:46:05<7:28:34, 13.51s/it]
 27%|██▋       | 742/2733 [2:46:17<7:14:34, 13.10s/it]
                                                      
{'loss': 1.666, 'learning_rate': 0.00022685150018989745, 'epoch': 0.27}

 27%|██▋       | 742/2733 [2:46:17<7:14:34, 13.10s/it]
 27%|██▋       | 743/2733 [2:46:30<7:18:35, 13.22s/it]
                                                      
{'loss': 1.7831, 'learning_rate': 0.00022673756171667298, 'epoch': 0.27}

 27%|██▋       | 743/2733 [2:46:31<7:18:35, 13.22s/it]
 27%|██▋       | 744/2733 [2:46:44<7:23:55, 13.39s/it]
                                                      
{'loss': 1.7187, 'learning_rate': 0.0002266236232434485, 'epoch': 0.27}

 27%|██▋       | 744/2733 [2:46:44<7:23:55, 13.39s/it]
 27%|██▋       | 745/2733 [2:46:59<7:32:15, 13.65s/it]
                                                      
{'loss': 1.7674, 'learning_rate': 0.00022650968477022407, 'epoch': 0.27}

 27%|██▋       | 745/2733 [2:46:59<7:32:15, 13.65s/it]
 27%|██▋       | 746/2733 [2:47:12<7:26:02, 13.47s/it]
                                                      
{'loss': 1.6133, 'learning_rate': 0.0002263957462969996, 'epoch': 0.27}

 27%|██▋       | 746/2733 [2:47:12<7:26:02, 13.47s/it]
 27%|██▋       | 747/2733 [2:47:26<7:31:23, 13.64s/it]
                                                      
{'loss': 1.5907, 'learning_rate': 0.00022628180782377516, 'epoch': 0.27}

 27%|██▋       | 747/2733 [2:47:26<7:31:23, 13.64s/it]
 27%|██▋       | 748/2733 [2:47:39<7:23:47, 13.41s/it]
                                                      
{'loss': 1.7379, 'learning_rate': 0.00022616786935055069, 'epoch': 0.27}

 27%|██▋       | 748/2733 [2:47:39<7:23:47, 13.41s/it]
 27%|██▋       | 749/2733 [2:47:52<7:19:56, 13.30s/it]
                                                      
{'loss': 1.6437, 'learning_rate': 0.00022605393087732624, 'epoch': 0.27}

 27%|██▋       | 749/2733 [2:47:52<7:19:56, 13.30s/it]
 27%|██▋       | 750/2733 [2:48:05<7:23:07, 13.41s/it]
                                                      
{'loss': 1.6408, 'learning_rate': 0.00022593999240410177, 'epoch': 0.27}

 27%|██▋       | 750/2733 [2:48:05<7:23:07, 13.41s/it]
 27%|██▋       | 751/2733 [2:48:18<7:17:42, 13.25s/it]
                                                      
{'loss': 1.601, 'learning_rate': 0.0002258260539308773, 'epoch': 0.27}

 27%|██▋       | 751/2733 [2:48:18<7:17:42, 13.25s/it]
 28%|██▊       | 752/2733 [2:48:32<7:26:30, 13.52s/it]
                                                      
{'loss': 1.747, 'learning_rate': 0.00022571211545765286, 'epoch': 0.28}

 28%|██▊       | 752/2733 [2:48:32<7:26:30, 13.52s/it]
 28%|██▊       | 753/2733 [2:48:45<7:18:51, 13.30s/it]
                                                      
{'loss': 1.6394, 'learning_rate': 0.0002255981769844284, 'epoch': 0.28}

 28%|██▊       | 753/2733 [2:48:45<7:18:51, 13.30s/it]
 28%|██▊       | 754/2733 [2:48:59<7:27:58, 13.58s/it]
                                                      
{'loss': 1.7464, 'learning_rate': 0.00022548423851120395, 'epoch': 0.28}

 28%|██▊       | 754/2733 [2:48:59<7:27:58, 13.58s/it]
 28%|██▊       | 755/2733 [2:49:13<7:28:10, 13.59s/it]
                                                      
{'loss': 1.7322, 'learning_rate': 0.00022537030003797948, 'epoch': 0.28}

 28%|██▊       | 755/2733 [2:49:13<7:28:10, 13.59s/it]
 28%|██▊       | 756/2733 [2:49:25<7:15:21, 13.21s/it]
                                                      
{'loss': 1.587, 'learning_rate': 0.00022525636156475504, 'epoch': 0.28}

 28%|██▊       | 756/2733 [2:49:25<7:15:21, 13.21s/it]
 28%|██▊       | 757/2733 [2:49:38<7:13:20, 13.16s/it]
                                                      
{'loss': 1.6802, 'learning_rate': 0.00022514242309153057, 'epoch': 0.28}

 28%|██▊       | 757/2733 [2:49:38<7:13:20, 13.16s/it]
 28%|██▊       | 758/2733 [2:49:51<7:11:49, 13.12s/it]
                                                      
{'loss': 1.6957, 'learning_rate': 0.00022502848461830607, 'epoch': 0.28}

 28%|██▊       | 758/2733 [2:49:51<7:11:49, 13.12s/it]
 28%|██▊       | 759/2733 [2:50:05<7:19:42, 13.36s/it]
                                                      
{'loss': 1.6183, 'learning_rate': 0.00022491454614508166, 'epoch': 0.28}

 28%|██▊       | 759/2733 [2:50:05<7:19:42, 13.36s/it]
 28%|██▊       | 760/2733 [2:50:18<7:18:44, 13.34s/it]
                                                      
{'loss': 1.7132, 'learning_rate': 0.00022480060767185716, 'epoch': 0.28}

 28%|██▊       | 760/2733 [2:50:19<7:18:44, 13.34s/it]
 28%|██▊       | 761/2733 [2:50:32<7:22:39, 13.47s/it]
                                                      
{'loss': 1.7256, 'learning_rate': 0.00022468666919863274, 'epoch': 0.28}

 28%|██▊       | 761/2733 [2:50:32<7:22:39, 13.47s/it]
 28%|██▊       | 762/2733 [2:50:46<7:25:04, 13.55s/it]
                                                      
{'loss': 1.6416, 'learning_rate': 0.00022457273072540825, 'epoch': 0.28}

 28%|██▊       | 762/2733 [2:50:46<7:25:04, 13.55s/it]
 28%|██▊       | 763/2733 [2:50:59<7:22:03, 13.46s/it]
                                                      
{'loss': 1.662, 'learning_rate': 0.0002244587922521838, 'epoch': 0.28}

 28%|██▊       | 763/2733 [2:50:59<7:22:03, 13.46s/it]
 28%|██▊       | 764/2733 [2:51:13<7:23:33, 13.52s/it]
                                                      
{'loss': 1.738, 'learning_rate': 0.00022434485377895933, 'epoch': 0.28}

 28%|██▊       | 764/2733 [2:51:13<7:23:33, 13.52s/it]
 28%|██▊       | 765/2733 [2:51:26<7:15:22, 13.27s/it]
                                                      
{'loss': 1.5411, 'learning_rate': 0.00022423091530573486, 'epoch': 0.28}

 28%|██▊       | 765/2733 [2:51:26<7:15:22, 13.27s/it]
 28%|██▊       | 766/2733 [2:51:39<7:21:07, 13.46s/it]
                                                      
{'loss': 1.741, 'learning_rate': 0.00022411697683251042, 'epoch': 0.28}

 28%|██▊       | 766/2733 [2:51:40<7:21:07, 13.46s/it]
 28%|██▊       | 767/2733 [2:51:53<7:23:26, 13.53s/it]
                                                      
{'loss': 1.6789, 'learning_rate': 0.00022400303835928595, 'epoch': 0.28}

 28%|██▊       | 767/2733 [2:51:53<7:23:26, 13.53s/it]
 28%|██▊       | 768/2733 [2:52:06<7:18:44, 13.40s/it]
                                                      
{'loss': 1.7631, 'learning_rate': 0.0002238890998860615, 'epoch': 0.28}

 28%|██▊       | 768/2733 [2:52:06<7:18:44, 13.40s/it]
 28%|██▊       | 769/2733 [2:52:20<7:21:03, 13.47s/it]
                                                      
{'loss': 1.6915, 'learning_rate': 0.00022377516141283704, 'epoch': 0.28}

 28%|██▊       | 769/2733 [2:52:20<7:21:03, 13.47s/it]
 28%|██▊       | 770/2733 [2:52:34<7:22:41, 13.53s/it]
                                                      
{'loss': 1.7349, 'learning_rate': 0.0002236612229396126, 'epoch': 0.28}

 28%|██▊       | 770/2733 [2:52:34<7:22:41, 13.53s/it]
 28%|██▊       | 771/2733 [2:52:47<7:23:07, 13.55s/it]
                                                      
{'loss': 1.781, 'learning_rate': 0.00022354728446638813, 'epoch': 0.28}

 28%|██▊       | 771/2733 [2:52:47<7:23:07, 13.55s/it]
 28%|██▊       | 772/2733 [2:53:01<7:24:43, 13.61s/it]
                                                      
{'loss': 1.6673, 'learning_rate': 0.00022343334599316366, 'epoch': 0.28}

 28%|██▊       | 772/2733 [2:53:01<7:24:43, 13.61s/it]
 28%|██▊       | 773/2733 [2:53:14<7:22:10, 13.54s/it]
                                                      
{'loss': 1.6616, 'learning_rate': 0.00022331940751993922, 'epoch': 0.28}

 28%|██▊       | 773/2733 [2:53:14<7:22:10, 13.54s/it]
 28%|██▊       | 774/2733 [2:53:27<7:13:19, 13.27s/it]
                                                      
{'loss': 1.6773, 'learning_rate': 0.00022320546904671475, 'epoch': 0.28}

 28%|██▊       | 774/2733 [2:53:27<7:13:19, 13.27s/it]
 28%|██▊       | 775/2733 [2:53:41<7:20:42, 13.50s/it]
                                                      
{'loss': 1.6698, 'learning_rate': 0.0002230915305734903, 'epoch': 0.28}

 28%|██▊       | 775/2733 [2:53:41<7:20:42, 13.50s/it]
 28%|██▊       | 776/2733 [2:53:54<7:16:48, 13.39s/it]
                                                      
{'loss': 1.6158, 'learning_rate': 0.00022297759210026583, 'epoch': 0.28}

 28%|██▊       | 776/2733 [2:53:54<7:16:48, 13.39s/it]
 28%|██▊       | 777/2733 [2:54:07<7:10:19, 13.20s/it]
                                                      
{'loss': 1.5836, 'learning_rate': 0.0002228636536270414, 'epoch': 0.28}

 28%|██▊       | 777/2733 [2:54:07<7:10:19, 13.20s/it]
 28%|██▊       | 778/2733 [2:54:20<7:06:33, 13.09s/it]
                                                      
{'loss': 1.6778, 'learning_rate': 0.00022274971515381692, 'epoch': 0.28}

 28%|██▊       | 778/2733 [2:54:20<7:06:33, 13.09s/it]
 29%|██▊       | 779/2733 [2:54:33<7:08:54, 13.17s/it]
                                                      
{'loss': 1.6715, 'learning_rate': 0.00022263577668059245, 'epoch': 0.29}

 29%|██▊       | 779/2733 [2:54:33<7:08:54, 13.17s/it]
 29%|██▊       | 780/2733 [2:54:47<7:12:59, 13.30s/it]
                                                      
{'loss': 1.737, 'learning_rate': 0.000222521838207368, 'epoch': 0.29}

 29%|██▊       | 780/2733 [2:54:47<7:12:59, 13.30s/it]
 29%|██▊       | 781/2733 [2:55:00<7:10:41, 13.24s/it]
                                                      
{'loss': 1.6227, 'learning_rate': 0.00022240789973414354, 'epoch': 0.29}

 29%|██▊       | 781/2733 [2:55:00<7:10:41, 13.24s/it]
 29%|██▊       | 782/2733 [2:55:14<7:15:58, 13.41s/it]
                                                      
{'loss': 1.6836, 'learning_rate': 0.0002222939612609191, 'epoch': 0.29}

 29%|██▊       | 782/2733 [2:55:14<7:15:58, 13.41s/it]
 29%|██▊       | 783/2733 [2:55:27<7:19:24, 13.52s/it]
                                                      
{'loss': 1.6992, 'learning_rate': 0.00022218002278769463, 'epoch': 0.29}

 29%|██▊       | 783/2733 [2:55:27<7:19:24, 13.52s/it]
 29%|██▊       | 784/2733 [2:55:41<7:21:51, 13.60s/it]
                                                      
{'loss': 1.7141, 'learning_rate': 0.00022206608431447018, 'epoch': 0.29}

 29%|██▊       | 784/2733 [2:55:41<7:21:51, 13.60s/it]
 29%|██▊       | 785/2733 [2:55:54<7:16:57, 13.46s/it]
                                                      
{'loss': 1.6808, 'learning_rate': 0.00022195214584124572, 'epoch': 0.29}

 29%|██▊       | 785/2733 [2:55:54<7:16:57, 13.46s/it]
 29%|██▉       | 786/2733 [2:56:08<7:15:46, 13.43s/it]
                                                      
{'loss': 1.6698, 'learning_rate': 0.00022183820736802125, 'epoch': 0.29}

 29%|██▉       | 786/2733 [2:56:08<7:15:46, 13.43s/it]
 29%|██▉       | 787/2733 [2:56:21<7:17:27, 13.49s/it]
                                                      
{'loss': 1.6098, 'learning_rate': 0.0002217242688947968, 'epoch': 0.29}

 29%|██▉       | 787/2733 [2:56:21<7:17:27, 13.49s/it]
 29%|██▉       | 788/2733 [2:56:34<7:12:24, 13.34s/it]
                                                      
{'loss': 1.6514, 'learning_rate': 0.00022161033042157233, 'epoch': 0.29}

 29%|██▉       | 788/2733 [2:56:34<7:12:24, 13.34s/it]
 29%|██▉       | 789/2733 [2:56:47<7:10:35, 13.29s/it]
                                                      
{'loss': 1.7195, 'learning_rate': 0.0002214963919483479, 'epoch': 0.29}

 29%|██▉       | 789/2733 [2:56:47<7:10:35, 13.29s/it]
 29%|██▉       | 790/2733 [2:57:02<7:20:06, 13.59s/it]
                                                      
{'loss': 1.7209, 'learning_rate': 0.00022138245347512342, 'epoch': 0.29}

 29%|██▉       | 790/2733 [2:57:02<7:20:06, 13.59s/it]
 29%|██▉       | 791/2733 [2:57:14<7:11:54, 13.34s/it]
                                                      
{'loss': 1.6629, 'learning_rate': 0.00022126851500189898, 'epoch': 0.29}

 29%|██▉       | 791/2733 [2:57:15<7:11:54, 13.34s/it]
 29%|██▉       | 792/2733 [2:57:29<7:20:12, 13.61s/it]
                                                      
{'loss': 1.7242, 'learning_rate': 0.0002211545765286745, 'epoch': 0.29}

 29%|██▉       | 792/2733 [2:57:29<7:20:12, 13.61s/it]
 29%|██▉       | 793/2733 [2:57:42<7:14:46, 13.45s/it]
                                                      
{'loss': 1.6999, 'learning_rate': 0.00022104063805545, 'epoch': 0.29}

 29%|██▉       | 793/2733 [2:57:42<7:14:46, 13.45s/it]
 29%|██▉       | 794/2733 [2:57:56<7:21:35, 13.66s/it]
                                                      
{'loss': 1.736, 'learning_rate': 0.0002209266995822256, 'epoch': 0.29}

 29%|██▉       | 794/2733 [2:57:56<7:21:35, 13.66s/it]
 29%|██▉       | 795/2733 [2:58:10<7:22:16, 13.69s/it]
                                                      
{'loss': 1.7817, 'learning_rate': 0.0002208127611090011, 'epoch': 0.29}

 29%|██▉       | 795/2733 [2:58:10<7:22:16, 13.69s/it]
 29%|██▉       | 796/2733 [2:58:23<7:21:43, 13.68s/it]
                                                      
{'loss': 1.7654, 'learning_rate': 0.00022069882263577668, 'epoch': 0.29}

 29%|██▉       | 796/2733 [2:58:23<7:21:43, 13.68s/it]
 29%|██▉       | 797/2733 [2:58:37<7:20:27, 13.65s/it]
                                                      
{'loss': 1.6294, 'learning_rate': 0.0002205848841625522, 'epoch': 0.29}

 29%|██▉       | 797/2733 [2:58:37<7:20:27, 13.65s/it]
 29%|██▉       | 798/2733 [2:58:50<7:10:09, 13.34s/it]
                                                      
{'loss': 1.6721, 'learning_rate': 0.00022047094568932772, 'epoch': 0.29}

 29%|██▉       | 798/2733 [2:58:50<7:10:09, 13.34s/it]
 29%|██▉       | 799/2733 [2:59:03<7:13:04, 13.44s/it]
                                                      
{'loss': 1.6306, 'learning_rate': 0.00022035700721610328, 'epoch': 0.29}

 29%|██▉       | 799/2733 [2:59:03<7:13:04, 13.44s/it]
 29%|██▉       | 800/2733 [2:59:17<7:14:28, 13.49s/it]
                                                      
{'loss': 1.6328, 'learning_rate': 0.0002202430687428788, 'epoch': 0.29}

 29%|██▉       | 800/2733 [2:59:17<7:14:28, 13.49s/it]
 29%|██▉       | 801/2733 [2:59:30<7:14:27, 13.49s/it]
                                                      
{'loss': 1.7297, 'learning_rate': 0.00022012913026965436, 'epoch': 0.29}

 29%|██▉       | 801/2733 [2:59:30<7:14:27, 13.49s/it]
 29%|██▉       | 802/2733 [2:59:43<7:09:17, 13.34s/it]
                                                      
{'loss': 1.6662, 'learning_rate': 0.0002200151917964299, 'epoch': 0.29}

 29%|██▉       | 802/2733 [2:59:43<7:09:17, 13.34s/it]
 29%|██▉       | 803/2733 [2:59:56<7:00:00, 13.06s/it]
                                                      
{'loss': 1.6324, 'learning_rate': 0.00021990125332320545, 'epoch': 0.29}

 29%|██▉       | 803/2733 [2:59:56<7:00:00, 13.06s/it]
 29%|██▉       | 804/2733 [3:00:09<7:00:32, 13.08s/it]
                                                      
{'loss': 1.6435, 'learning_rate': 0.00021978731484998098, 'epoch': 0.29}

 29%|██▉       | 804/2733 [3:00:09<7:00:32, 13.08s/it]
 29%|██▉       | 805/2733 [3:00:23<7:07:57, 13.32s/it]
                                                      
{'loss': 1.6258, 'learning_rate': 0.0002196733763767565, 'epoch': 0.29}

 29%|██▉       | 805/2733 [3:00:23<7:07:57, 13.32s/it]
 29%|██▉       | 806/2733 [3:00:36<7:09:50, 13.38s/it]
                                                      
{'loss': 1.7256, 'learning_rate': 0.00021955943790353207, 'epoch': 0.29}

 29%|██▉       | 806/2733 [3:00:36<7:09:50, 13.38s/it]
 30%|██▉       | 807/2733 [3:00:50<7:14:44, 13.54s/it]
                                                      
{'loss': 1.7165, 'learning_rate': 0.0002194454994303076, 'epoch': 0.3}

 30%|██▉       | 807/2733 [3:00:50<7:14:44, 13.54s/it]
 30%|██▉       | 808/2733 [3:01:04<7:13:16, 13.50s/it]
                                                      
{'loss': 1.6558, 'learning_rate': 0.00021933156095708316, 'epoch': 0.3}

 30%|██▉       | 808/2733 [3:01:04<7:13:16, 13.50s/it]
 30%|██▉       | 809/2733 [3:01:17<7:12:39, 13.49s/it]
                                                      
{'loss': 1.7043, 'learning_rate': 0.0002192176224838587, 'epoch': 0.3}

 30%|██▉       | 809/2733 [3:01:17<7:12:39, 13.49s/it]
 30%|██▉       | 810/2733 [3:01:31<7:12:29, 13.49s/it]
                                                      
{'loss': 1.7314, 'learning_rate': 0.00021910368401063424, 'epoch': 0.3}

 30%|██▉       | 810/2733 [3:01:31<7:12:29, 13.49s/it]
 30%|██▉       | 811/2733 [3:01:44<7:09:38, 13.41s/it]
                                                      
{'loss': 1.67, 'learning_rate': 0.00021898974553740978, 'epoch': 0.3}

 30%|██▉       | 811/2733 [3:01:44<7:09:38, 13.41s/it]
 30%|██▉       | 812/2733 [3:01:57<7:07:24, 13.35s/it]
                                                      
{'loss': 1.759, 'learning_rate': 0.0002188758070641853, 'epoch': 0.3}

 30%|██▉       | 812/2733 [3:01:57<7:07:24, 13.35s/it]
 30%|██▉       | 813/2733 [3:02:11<7:13:43, 13.55s/it]
                                                      
{'loss': 1.7273, 'learning_rate': 0.00021876186859096086, 'epoch': 0.3}

 30%|██▉       | 813/2733 [3:02:11<7:13:43, 13.55s/it]
 30%|██▉       | 814/2733 [3:02:25<7:15:50, 13.63s/it]
                                                      
{'loss': 1.5972, 'learning_rate': 0.0002186479301177364, 'epoch': 0.3}

 30%|██▉       | 814/2733 [3:02:25<7:15:50, 13.63s/it]
 30%|██▉       | 815/2733 [3:02:38<7:15:21, 13.62s/it]
                                                      
{'loss': 1.5688, 'learning_rate': 0.00021853399164451195, 'epoch': 0.3}

 30%|██▉       | 815/2733 [3:02:38<7:15:21, 13.62s/it]
 30%|██▉       | 816/2733 [3:02:52<7:19:18, 13.75s/it]
                                                      
{'loss': 1.6131, 'learning_rate': 0.00021842005317128748, 'epoch': 0.3}

 30%|██▉       | 816/2733 [3:02:52<7:19:18, 13.75s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 30%|██▉       | 817/2733 [3:03:07<7:25:01, 13.94s/it]
                                                      
{'loss': 1.7199, 'learning_rate': 0.00021830611469806304, 'epoch': 0.3}

 30%|██▉       | 817/2733 [3:03:07<7:25:01, 13.94s/it]
 30%|██▉       | 818/2733 [3:03:21<7:26:51, 14.00s/it]
                                                      
{'loss': 1.8113, 'learning_rate': 0.00021819217622483857, 'epoch': 0.3}

 30%|██▉       | 818/2733 [3:03:21<7:26:51, 14.00s/it]
 30%|██▉       | 819/2733 [3:03:34<7:21:39, 13.85s/it]
                                                      
{'loss': 1.7232, 'learning_rate': 0.0002180782377516141, 'epoch': 0.3}

 30%|██▉       | 819/2733 [3:03:34<7:21:39, 13.85s/it]
 30%|███       | 820/2733 [3:03:48<7:19:08, 13.77s/it]
                                                      
{'loss': 1.6788, 'learning_rate': 0.00021796429927838966, 'epoch': 0.3}

 30%|███       | 820/2733 [3:03:48<7:19:08, 13.77s/it]
 30%|███       | 821/2733 [3:04:01<7:10:51, 13.52s/it]
                                                      
{'loss': 1.7671, 'learning_rate': 0.0002178503608051652, 'epoch': 0.3}

 30%|███       | 821/2733 [3:04:01<7:10:51, 13.52s/it]
 30%|███       | 822/2733 [3:04:14<7:06:17, 13.38s/it]
                                                      
{'loss': 1.6745, 'learning_rate': 0.00021773642233194074, 'epoch': 0.3}

 30%|███       | 822/2733 [3:04:14<7:06:17, 13.38s/it]
 30%|███       | 823/2733 [3:04:27<7:04:16, 13.33s/it]
                                                      
{'loss': 1.69, 'learning_rate': 0.00021762248385871627, 'epoch': 0.3}

 30%|███       | 823/2733 [3:04:27<7:04:16, 13.33s/it]
 30%|███       | 824/2733 [3:04:40<7:02:57, 13.29s/it]
                                                      
{'loss': 1.6577, 'learning_rate': 0.00021750854538549183, 'epoch': 0.3}

 30%|███       | 824/2733 [3:04:41<7:02:57, 13.29s/it]
 30%|███       | 825/2733 [3:04:53<6:58:38, 13.16s/it]
                                                      
{'loss': 1.7077, 'learning_rate': 0.00021739460691226736, 'epoch': 0.3}

 30%|███       | 825/2733 [3:04:53<6:58:38, 13.16s/it]
 30%|███       | 826/2733 [3:05:07<7:04:36, 13.36s/it]
                                                      
{'loss': 1.6517, 'learning_rate': 0.0002172806684390429, 'epoch': 0.3}

 30%|███       | 826/2733 [3:05:07<7:04:36, 13.36s/it]
 30%|███       | 827/2733 [3:05:20<6:59:52, 13.22s/it]
                                                      
{'loss': 1.651, 'learning_rate': 0.00021716672996581845, 'epoch': 0.3}

 30%|███       | 827/2733 [3:05:20<6:59:52, 13.22s/it]
 30%|███       | 828/2733 [3:05:33<7:01:02, 13.26s/it]
                                                      
{'loss': 1.6841, 'learning_rate': 0.00021705279149259398, 'epoch': 0.3}

 30%|███       | 828/2733 [3:05:33<7:01:02, 13.26s/it]
 30%|███       | 829/2733 [3:05:47<7:04:34, 13.38s/it]
                                                      
{'loss': 1.6826, 'learning_rate': 0.00021693885301936954, 'epoch': 0.3}

 30%|███       | 829/2733 [3:05:47<7:04:34, 13.38s/it]
 30%|███       | 830/2733 [3:06:01<7:05:01, 13.40s/it]
                                                      
{'loss': 1.6375, 'learning_rate': 0.00021682491454614507, 'epoch': 0.3}

 30%|███       | 830/2733 [3:06:01<7:05:01, 13.40s/it]
 30%|███       | 831/2733 [3:06:12<6:51:20, 12.98s/it]
                                                      
{'loss': 1.6664, 'learning_rate': 0.00021671097607292063, 'epoch': 0.3}

 30%|███       | 831/2733 [3:06:13<6:51:20, 12.98s/it]
 30%|███       | 832/2733 [3:06:26<6:53:02, 13.04s/it]
                                                      
{'loss': 1.6227, 'learning_rate': 0.00021659703759969616, 'epoch': 0.3}

 30%|███       | 832/2733 [3:06:26<6:53:02, 13.04s/it]
 30%|███       | 833/2733 [3:06:39<6:57:56, 13.20s/it]
                                                      
{'loss': 1.6828, 'learning_rate': 0.00021648309912647166, 'epoch': 0.3}

 30%|███       | 833/2733 [3:06:39<6:57:56, 13.20s/it]
 31%|███       | 834/2733 [3:06:52<6:56:33, 13.16s/it]
                                                      
{'loss': 1.748, 'learning_rate': 0.00021636916065324724, 'epoch': 0.31}

 31%|███       | 834/2733 [3:06:52<6:56:33, 13.16s/it]
 31%|███       | 835/2733 [3:07:05<6:54:49, 13.11s/it]
                                                      
{'loss': 1.6716, 'learning_rate': 0.00021625522218002275, 'epoch': 0.31}

 31%|███       | 835/2733 [3:07:05<6:54:49, 13.11s/it]
 31%|███       | 836/2733 [3:07:18<6:54:37, 13.11s/it]
                                                      
{'loss': 1.7301, 'learning_rate': 0.0002161412837067983, 'epoch': 0.31}

 31%|███       | 836/2733 [3:07:18<6:54:37, 13.11s/it]
 31%|███       | 837/2733 [3:07:31<6:50:37, 12.99s/it]
                                                      
{'loss': 1.742, 'learning_rate': 0.00021602734523357384, 'epoch': 0.31}

 31%|███       | 837/2733 [3:07:31<6:50:37, 12.99s/it]
 31%|███       | 838/2733 [3:07:45<7:02:17, 13.37s/it]
                                                      
{'loss': 1.6892, 'learning_rate': 0.0002159134067603494, 'epoch': 0.31}

 31%|███       | 838/2733 [3:07:45<7:02:17, 13.37s/it]
 31%|███       | 839/2733 [3:07:59<7:05:16, 13.47s/it]
                                                      
{'loss': 1.7392, 'learning_rate': 0.00021579946828712492, 'epoch': 0.31}

 31%|███       | 839/2733 [3:07:59<7:05:16, 13.47s/it]
 31%|███       | 840/2733 [3:08:12<7:03:39, 13.43s/it]
                                                      
{'loss': 1.7205, 'learning_rate': 0.00021568552981390045, 'epoch': 0.31}

 31%|███       | 840/2733 [3:08:12<7:03:39, 13.43s/it]
 31%|███       | 841/2733 [3:08:26<7:02:13, 13.39s/it]
                                                      
{'loss': 1.7734, 'learning_rate': 0.000215571591340676, 'epoch': 0.31}

 31%|███       | 841/2733 [3:08:26<7:02:13, 13.39s/it]
 31%|███       | 842/2733 [3:08:39<6:58:27, 13.28s/it]
                                                      
{'loss': 1.6477, 'learning_rate': 0.00021545765286745154, 'epoch': 0.31}

 31%|███       | 842/2733 [3:08:39<6:58:27, 13.28s/it]
 31%|███       | 843/2733 [3:08:53<7:02:38, 13.42s/it]
                                                      
{'loss': 1.6904, 'learning_rate': 0.0002153437143942271, 'epoch': 0.31}

 31%|███       | 843/2733 [3:08:53<7:02:38, 13.42s/it]
 31%|███       | 844/2733 [3:09:06<7:02:07, 13.41s/it]
                                                      
{'loss': 1.8045, 'learning_rate': 0.00021522977592100263, 'epoch': 0.31}

 31%|███       | 844/2733 [3:09:06<7:02:07, 13.41s/it]
 31%|███       | 845/2733 [3:09:19<6:58:29, 13.30s/it]
                                                      
{'loss': 1.6577, 'learning_rate': 0.0002151158374477782, 'epoch': 0.31}

 31%|███       | 845/2733 [3:09:19<6:58:29, 13.30s/it]
 31%|███       | 846/2733 [3:09:33<7:07:04, 13.58s/it]
                                                      
{'loss': 1.7364, 'learning_rate': 0.00021500189897455372, 'epoch': 0.31}

 31%|███       | 846/2733 [3:09:33<7:07:04, 13.58s/it]
 31%|███       | 847/2733 [3:09:47<7:11:10, 13.72s/it]
                                                      
{'loss': 1.7285, 'learning_rate': 0.00021488796050132925, 'epoch': 0.31}

 31%|███       | 847/2733 [3:09:47<7:11:10, 13.72s/it]
 31%|███       | 848/2733 [3:10:00<7:06:01, 13.56s/it]
                                                      
{'loss': 1.6607, 'learning_rate': 0.0002147740220281048, 'epoch': 0.31}

 31%|███       | 848/2733 [3:10:00<7:06:01, 13.56s/it]
 31%|███       | 849/2733 [3:10:14<7:05:46, 13.56s/it]
                                                      
{'loss': 1.632, 'learning_rate': 0.00021466008355488034, 'epoch': 0.31}

 31%|███       | 849/2733 [3:10:14<7:05:46, 13.56s/it]
 31%|███       | 850/2733 [3:10:27<7:02:34, 13.46s/it]
                                                      
{'loss': 1.7384, 'learning_rate': 0.0002145461450816559, 'epoch': 0.31}

 31%|███       | 850/2733 [3:10:27<7:02:34, 13.46s/it]
 31%|███       | 851/2733 [3:10:41<7:00:53, 13.42s/it]
                                                      
{'loss': 1.6439, 'learning_rate': 0.00021443220660843142, 'epoch': 0.31}

 31%|███       | 851/2733 [3:10:41<7:00:53, 13.42s/it]
 31%|███       | 852/2733 [3:10:55<7:08:00, 13.65s/it]
                                                      
{'loss': 1.7105, 'learning_rate': 0.00021431826813520698, 'epoch': 0.31}

 31%|███       | 852/2733 [3:10:55<7:08:00, 13.65s/it]
 31%|███       | 853/2733 [3:11:08<7:02:22, 13.48s/it]
                                                      
{'loss': 1.6152, 'learning_rate': 0.0002142043296619825, 'epoch': 0.31}

 31%|███       | 853/2733 [3:11:08<7:02:22, 13.48s/it]
 31%|███       | 854/2733 [3:11:21<6:59:41, 13.40s/it]
                                                      
{'loss': 1.7125, 'learning_rate': 0.00021409039118875804, 'epoch': 0.31}

 31%|███       | 854/2733 [3:11:21<6:59:41, 13.40s/it]
 31%|███▏      | 855/2733 [3:11:35<7:00:32, 13.44s/it]
                                                      
{'loss': 1.6829, 'learning_rate': 0.0002139764527155336, 'epoch': 0.31}

 31%|███▏      | 855/2733 [3:11:35<7:00:32, 13.44s/it]
 31%|███▏      | 856/2733 [3:11:49<7:08:47, 13.71s/it]
                                                      
{'loss': 1.7629, 'learning_rate': 0.00021386251424230913, 'epoch': 0.31}

 31%|███▏      | 856/2733 [3:11:49<7:08:47, 13.71s/it]
 31%|███▏      | 857/2733 [3:12:03<7:08:31, 13.71s/it]
                                                      
{'loss': 1.647, 'learning_rate': 0.00021374857576908469, 'epoch': 0.31}

 31%|███▏      | 857/2733 [3:12:03<7:08:31, 13.71s/it]
 31%|███▏      | 858/2733 [3:12:16<7:06:04, 13.63s/it]
                                                      
{'loss': 1.6305, 'learning_rate': 0.00021363463729586022, 'epoch': 0.31}

 31%|███▏      | 858/2733 [3:12:16<7:06:04, 13.63s/it]
 31%|███▏      | 859/2733 [3:12:30<7:06:40, 13.66s/it]
                                                      
{'loss': 1.7115, 'learning_rate': 0.00021352069882263577, 'epoch': 0.31}

 31%|███▏      | 859/2733 [3:12:30<7:06:40, 13.66s/it]
 31%|███▏      | 860/2733 [3:12:43<7:02:41, 13.54s/it]
                                                      
{'loss': 1.5164, 'learning_rate': 0.0002134067603494113, 'epoch': 0.31}

 31%|███▏      | 860/2733 [3:12:43<7:02:41, 13.54s/it]
 32%|███▏      | 861/2733 [3:12:56<6:54:00, 13.27s/it]
                                                      
{'loss': 1.7454, 'learning_rate': 0.00021329282187618683, 'epoch': 0.32}

 32%|███▏      | 861/2733 [3:12:56<6:54:00, 13.27s/it]
 32%|███▏      | 862/2733 [3:13:09<6:50:31, 13.17s/it]
                                                      
{'loss': 1.7106, 'learning_rate': 0.0002131788834029624, 'epoch': 0.32}

 32%|███▏      | 862/2733 [3:13:09<6:50:31, 13.17s/it]
 32%|███▏      | 863/2733 [3:13:23<6:59:40, 13.47s/it]
                                                      
{'loss': 1.6554, 'learning_rate': 0.00021306494492973792, 'epoch': 0.32}

 32%|███▏      | 863/2733 [3:13:23<6:59:40, 13.47s/it]
 32%|███▏      | 864/2733 [3:13:36<6:56:54, 13.38s/it]
                                                      
{'loss': 1.6913, 'learning_rate': 0.00021295100645651348, 'epoch': 0.32}

 32%|███▏      | 864/2733 [3:13:36<6:56:54, 13.38s/it]
 32%|███▏      | 865/2733 [3:13:50<7:04:17, 13.63s/it]
                                                      
{'loss': 1.6753, 'learning_rate': 0.000212837067983289, 'epoch': 0.32}

 32%|███▏      | 865/2733 [3:13:50<7:04:17, 13.63s/it]
 32%|███▏      | 866/2733 [3:14:04<7:09:52, 13.82s/it]
                                                      
{'loss': 1.7517, 'learning_rate': 0.00021272312951006457, 'epoch': 0.32}

 32%|███▏      | 866/2733 [3:14:04<7:09:52, 13.82s/it]
 32%|███▏      | 867/2733 [3:14:18<7:05:11, 13.67s/it]
                                                      
{'loss': 1.613, 'learning_rate': 0.0002126091910368401, 'epoch': 0.32}

 32%|███▏      | 867/2733 [3:14:18<7:05:11, 13.67s/it]
 32%|███▏      | 868/2733 [3:14:31<7:00:09, 13.52s/it]
                                                      
{'loss': 1.7434, 'learning_rate': 0.0002124952525636156, 'epoch': 0.32}

 32%|███▏      | 868/2733 [3:14:31<7:00:09, 13.52s/it]
 32%|███▏      | 869/2733 [3:14:44<6:59:51, 13.51s/it]
                                                      
{'loss': 1.6926, 'learning_rate': 0.00021238131409039119, 'epoch': 0.32}

 32%|███▏      | 869/2733 [3:14:44<6:59:51, 13.51s/it]
 32%|███▏      | 870/2733 [3:14:58<7:00:38, 13.55s/it]
                                                      
{'loss': 1.7061, 'learning_rate': 0.0002122673756171667, 'epoch': 0.32}

 32%|███▏      | 870/2733 [3:14:58<7:00:38, 13.55s/it]
 32%|███▏      | 871/2733 [3:15:12<7:00:56, 13.56s/it]
                                                      
{'loss': 1.7415, 'learning_rate': 0.00021215343714394227, 'epoch': 0.32}

 32%|███▏      | 871/2733 [3:15:12<7:00:56, 13.56s/it]
 32%|███▏      | 872/2733 [3:15:25<7:01:07, 13.58s/it]
                                                      
{'loss': 1.7005, 'learning_rate': 0.00021203949867071778, 'epoch': 0.32}

 32%|███▏      | 872/2733 [3:15:25<7:01:07, 13.58s/it]
 32%|███▏      | 873/2733 [3:15:39<7:02:17, 13.62s/it]
                                                      
{'loss': 1.7058, 'learning_rate': 0.00021192556019749336, 'epoch': 0.32}

 32%|███▏      | 873/2733 [3:15:39<7:02:17, 13.62s/it]
 32%|███▏      | 874/2733 [3:15:52<6:57:41, 13.48s/it]
                                                      
{'loss': 1.7357, 'learning_rate': 0.00021181162172426886, 'epoch': 0.32}

 32%|███▏      | 874/2733 [3:15:52<6:57:41, 13.48s/it]
 32%|███▏      | 875/2733 [3:16:06<6:59:32, 13.55s/it]
                                                      
{'loss': 1.7008, 'learning_rate': 0.0002116976832510444, 'epoch': 0.32}

 32%|███▏      | 875/2733 [3:16:06<6:59:32, 13.55s/it]
 32%|███▏      | 876/2733 [3:16:19<6:56:09, 13.45s/it]
                                                      
{'loss': 1.6861, 'learning_rate': 0.00021158374477781995, 'epoch': 0.32}

 32%|███▏      | 876/2733 [3:16:19<6:56:09, 13.45s/it]
 32%|███▏      | 877/2733 [3:16:33<6:57:50, 13.51s/it]
                                                      
{'loss': 1.5469, 'learning_rate': 0.00021146980630459548, 'epoch': 0.32}

 32%|███▏      | 877/2733 [3:16:33<6:57:50, 13.51s/it]
 32%|███▏      | 878/2733 [3:16:46<6:54:33, 13.41s/it]
                                                      
{'loss': 1.6825, 'learning_rate': 0.00021135586783137104, 'epoch': 0.32}

 32%|███▏      | 878/2733 [3:16:46<6:54:33, 13.41s/it]
 32%|███▏      | 879/2733 [3:17:00<7:01:10, 13.63s/it]
                                                      
{'loss': 1.7915, 'learning_rate': 0.00021124192935814657, 'epoch': 0.32}

 32%|███▏      | 879/2733 [3:17:00<7:01:10, 13.63s/it]
 32%|███▏      | 880/2733 [3:17:13<6:51:59, 13.34s/it]
                                                      
{'loss': 1.6286, 'learning_rate': 0.00021112799088492213, 'epoch': 0.32}

 32%|███▏      | 880/2733 [3:17:13<6:51:59, 13.34s/it]
 32%|███▏      | 881/2733 [3:17:26<6:51:41, 13.34s/it]
                                                      
{'loss': 1.6853, 'learning_rate': 0.00021101405241169766, 'epoch': 0.32}

 32%|███▏      | 881/2733 [3:17:26<6:51:41, 13.34s/it]
 32%|███▏      | 882/2733 [3:17:40<6:56:45, 13.51s/it]
                                                      
{'loss': 1.7188, 'learning_rate': 0.0002109001139384732, 'epoch': 0.32}

 32%|███▏      | 882/2733 [3:17:40<6:56:45, 13.51s/it]
 32%|███▏      | 883/2733 [3:17:53<6:56:36, 13.51s/it]
                                                      
{'loss': 1.624, 'learning_rate': 0.00021078617546524875, 'epoch': 0.32}

 32%|███▏      | 883/2733 [3:17:53<6:56:36, 13.51s/it]
 32%|███▏      | 884/2733 [3:18:07<7:00:18, 13.64s/it]
                                                      
{'loss': 1.718, 'learning_rate': 0.00021067223699202428, 'epoch': 0.32}

 32%|███▏      | 884/2733 [3:18:07<7:00:18, 13.64s/it]
 32%|███▏      | 885/2733 [3:18:21<6:59:56, 13.63s/it]
                                                      
{'loss': 1.6378, 'learning_rate': 0.00021055829851879983, 'epoch': 0.32}

 32%|███▏      | 885/2733 [3:18:21<6:59:56, 13.63s/it]
 32%|███▏      | 886/2733 [3:18:35<7:00:59, 13.68s/it]
                                                      
{'loss': 1.6756, 'learning_rate': 0.00021044436004557536, 'epoch': 0.32}

 32%|███▏      | 886/2733 [3:18:35<7:00:59, 13.68s/it]
 32%|███▏      | 887/2733 [3:18:48<6:59:35, 13.64s/it]
                                                      
{'loss': 1.791, 'learning_rate': 0.00021033042157235092, 'epoch': 0.32}

 32%|███▏      | 887/2733 [3:18:48<6:59:35, 13.64s/it]
 32%|███▏      | 888/2733 [3:19:01<6:49:47, 13.33s/it]
                                                      
{'loss': 1.5639, 'learning_rate': 0.00021021648309912645, 'epoch': 0.32}

 32%|███▏      | 888/2733 [3:19:01<6:49:47, 13.33s/it]
 33%|███▎      | 889/2733 [3:19:15<6:52:46, 13.43s/it]
                                                      
{'loss': 1.7219, 'learning_rate': 0.00021010254462590198, 'epoch': 0.33}

 33%|███▎      | 889/2733 [3:19:15<6:52:46, 13.43s/it]
 33%|███▎      | 890/2733 [3:19:29<6:59:10, 13.65s/it]
                                                      
{'loss': 1.7046, 'learning_rate': 0.00020998860615267754, 'epoch': 0.33}

 33%|███▎      | 890/2733 [3:19:29<6:59:10, 13.65s/it]
 33%|███▎      | 891/2733 [3:19:43<7:04:17, 13.82s/it]
                                                      
{'loss': 1.7952, 'learning_rate': 0.00020987466767945307, 'epoch': 0.33}

 33%|███▎      | 891/2733 [3:19:43<7:04:17, 13.82s/it]
 33%|███▎      | 892/2733 [3:19:57<7:06:46, 13.91s/it]
                                                      
{'loss': 1.7891, 'learning_rate': 0.00020976072920622863, 'epoch': 0.33}

 33%|███▎      | 892/2733 [3:19:57<7:06:46, 13.91s/it]
 33%|███▎      | 893/2733 [3:20:11<7:06:17, 13.90s/it]
                                                      
{'loss': 1.7492, 'learning_rate': 0.00020964679073300416, 'epoch': 0.33}

 33%|███▎      | 893/2733 [3:20:11<7:06:17, 13.90s/it]
 33%|███▎      | 894/2733 [3:20:24<6:56:16, 13.58s/it]
                                                      
{'loss': 1.7152, 'learning_rate': 0.00020953285225977972, 'epoch': 0.33}

 33%|███▎      | 894/2733 [3:20:24<6:56:16, 13.58s/it]
 33%|███▎      | 895/2733 [3:20:37<6:55:07, 13.55s/it]
                                                      
{'loss': 1.7417, 'learning_rate': 0.00020941891378655525, 'epoch': 0.33}

 33%|███▎      | 895/2733 [3:20:37<6:55:07, 13.55s/it]
 33%|███▎      | 896/2733 [3:20:50<6:47:01, 13.29s/it]
                                                      
{'loss': 1.6676, 'learning_rate': 0.00020930497531333078, 'epoch': 0.33}

 33%|███▎      | 896/2733 [3:20:50<6:47:01, 13.29s/it]
 33%|███▎      | 897/2733 [3:21:04<6:51:02, 13.43s/it]
                                                      
{'loss': 1.6892, 'learning_rate': 0.00020919103684010633, 'epoch': 0.33}

 33%|███▎      | 897/2733 [3:21:04<6:51:02, 13.43s/it]
 33%|███▎      | 898/2733 [3:21:18<6:57:26, 13.65s/it]
                                                      
{'loss': 1.6792, 'learning_rate': 0.00020907709836688186, 'epoch': 0.33}

 33%|███▎      | 898/2733 [3:21:18<6:57:26, 13.65s/it]
 33%|███▎      | 899/2733 [3:21:32<7:02:33, 13.82s/it]
                                                      
{'loss': 1.8325, 'learning_rate': 0.00020896315989365742, 'epoch': 0.33}

 33%|███▎      | 899/2733 [3:21:32<7:02:33, 13.82s/it]
 33%|███▎      | 900/2733 [3:21:46<7:05:13, 13.92s/it]
                                                      
{'loss': 1.6716, 'learning_rate': 0.00020884922142043295, 'epoch': 0.33}

 33%|███▎      | 900/2733 [3:21:46<7:05:13, 13.92s/it]
 33%|███▎      | 901/2733 [3:22:00<7:07:32, 14.00s/it]
                                                      
{'loss': 1.6572, 'learning_rate': 0.0002087352829472085, 'epoch': 0.33}

 33%|███▎      | 901/2733 [3:22:00<7:07:32, 14.00s/it]
 33%|███▎      | 902/2733 [3:22:14<7:00:03, 13.76s/it]
                                                      
{'loss': 1.7695, 'learning_rate': 0.00020862134447398404, 'epoch': 0.33}

 33%|███▎      | 902/2733 [3:22:14<7:00:03, 13.76s/it]
 33%|███▎      | 903/2733 [3:22:28<7:02:12, 13.84s/it]
                                                      
{'loss': 1.6966, 'learning_rate': 0.00020850740600075957, 'epoch': 0.33}

 33%|███▎      | 903/2733 [3:22:28<7:02:12, 13.84s/it]
 33%|███▎      | 904/2733 [3:22:41<6:57:40, 13.70s/it]
                                                      
{'loss': 1.7966, 'learning_rate': 0.00020839346752753513, 'epoch': 0.33}

 33%|███▎      | 904/2733 [3:22:41<6:57:40, 13.70s/it]
 33%|███▎      | 905/2733 [3:22:55<7:02:06, 13.85s/it]
                                                      
{'loss': 1.6755, 'learning_rate': 0.00020827952905431066, 'epoch': 0.33}

 33%|███▎      | 905/2733 [3:22:55<7:02:06, 13.85s/it]
 33%|███▎      | 906/2733 [3:23:09<6:59:00, 13.76s/it]
                                                      
{'loss': 1.7377, 'learning_rate': 0.00020816559058108622, 'epoch': 0.33}

 33%|███▎      | 906/2733 [3:23:09<6:59:00, 13.76s/it]
 33%|███▎      | 907/2733 [3:23:22<6:57:23, 13.71s/it]
                                                      
{'loss': 1.8017, 'learning_rate': 0.00020805165210786175, 'epoch': 0.33}

 33%|███▎      | 907/2733 [3:23:22<6:57:23, 13.71s/it]
 33%|███▎      | 908/2733 [3:23:36<6:55:37, 13.66s/it]
                                                      
{'loss': 1.643, 'learning_rate': 0.0002079377136346373, 'epoch': 0.33}

 33%|███▎      | 908/2733 [3:23:36<6:55:37, 13.66s/it]
 33%|███▎      | 909/2733 [3:23:49<6:46:48, 13.38s/it]
                                                      
{'loss': 1.6278, 'learning_rate': 0.0002078237751614128, 'epoch': 0.33}

 33%|███▎      | 909/2733 [3:23:49<6:46:48, 13.38s/it]
 33%|███▎      | 910/2733 [3:24:02<6:46:01, 13.36s/it]
                                                      
{'loss': 1.6667, 'learning_rate': 0.00020770983668818834, 'epoch': 0.33}

 33%|███▎      | 910/2733 [3:24:02<6:46:01, 13.36s/it]
 33%|███▎      | 911/2733 [3:24:15<6:39:16, 13.15s/it]
                                                      
{'loss': 1.7385, 'learning_rate': 0.0002075958982149639, 'epoch': 0.33}

 33%|███▎      | 911/2733 [3:24:15<6:39:16, 13.15s/it]
 33%|███▎      | 912/2733 [3:24:28<6:40:53, 13.21s/it]
                                                      
{'loss': 1.683, 'learning_rate': 0.00020748195974173942, 'epoch': 0.33}

 33%|███▎      | 912/2733 [3:24:28<6:40:53, 13.21s/it]
 33%|███▎      | 913/2733 [3:24:41<6:38:51, 13.15s/it]
                                                      
{'loss': 1.6809, 'learning_rate': 0.00020736802126851498, 'epoch': 0.33}

 33%|███▎      | 913/2733 [3:24:41<6:38:51, 13.15s/it]
 33%|███▎      | 914/2733 [3:24:55<6:46:02, 13.39s/it]
                                                      
{'loss': 1.6463, 'learning_rate': 0.0002072540827952905, 'epoch': 0.33}

 33%|███▎      | 914/2733 [3:24:55<6:46:02, 13.39s/it]
 33%|███▎      | 915/2733 [3:25:08<6:41:25, 13.25s/it]
                                                      
{'loss': 1.7108, 'learning_rate': 0.00020714014432206607, 'epoch': 0.33}

 33%|███▎      | 915/2733 [3:25:08<6:41:25, 13.25s/it]
 34%|███▎      | 916/2733 [3:25:22<6:50:02, 13.54s/it]
                                                      
{'loss': 1.7095, 'learning_rate': 0.0002070262058488416, 'epoch': 0.34}

 34%|███▎      | 916/2733 [3:25:22<6:50:02, 13.54s/it]
 34%|███▎      | 917/2733 [3:25:36<6:51:33, 13.60s/it]
                                                      
{'loss': 1.804, 'learning_rate': 0.00020691226737561713, 'epoch': 0.34}

 34%|███▎      | 917/2733 [3:25:36<6:51:33, 13.60s/it]
 34%|███▎      | 918/2733 [3:25:49<6:49:20, 13.53s/it]
                                                      
{'loss': 1.7234, 'learning_rate': 0.0002067983289023927, 'epoch': 0.34}

 34%|███▎      | 918/2733 [3:25:49<6:49:20, 13.53s/it]
 34%|███▎      | 919/2733 [3:26:03<6:49:57, 13.56s/it]
                                                      
{'loss': 1.7689, 'learning_rate': 0.00020668439042916822, 'epoch': 0.34}

 34%|███▎      | 919/2733 [3:26:03<6:49:57, 13.56s/it]
 34%|███▎      | 920/2733 [3:26:16<6:41:47, 13.30s/it]
                                                      
{'loss': 1.7503, 'learning_rate': 0.00020657045195594378, 'epoch': 0.34}

 34%|███▎      | 920/2733 [3:26:16<6:41:47, 13.30s/it]
 34%|███▎      | 921/2733 [3:26:29<6:44:52, 13.41s/it]
                                                      
{'loss': 1.6906, 'learning_rate': 0.0002064565134827193, 'epoch': 0.34}

 34%|███▎      | 921/2733 [3:26:29<6:44:52, 13.41s/it]
 34%|███▎      | 922/2733 [3:26:42<6:39:09, 13.22s/it]
                                                      
{'loss': 1.7464, 'learning_rate': 0.00020634257500949486, 'epoch': 0.34}

 34%|███▎      | 922/2733 [3:26:42<6:39:09, 13.22s/it]
 34%|███▍      | 923/2733 [3:26:55<6:37:25, 13.17s/it]
                                                      
{'loss': 1.6927, 'learning_rate': 0.0002062286365362704, 'epoch': 0.34}

 34%|███▍      | 923/2733 [3:26:55<6:37:25, 13.17s/it]
 34%|███▍      | 924/2733 [3:27:09<6:42:39, 13.36s/it]
                                                      
{'loss': 1.8119, 'learning_rate': 0.00020611469806304592, 'epoch': 0.34}

 34%|███▍      | 924/2733 [3:27:09<6:42:39, 13.36s/it]
 34%|███▍      | 925/2733 [3:27:23<6:47:31, 13.52s/it]
                                                      
{'loss': 1.6555, 'learning_rate': 0.00020600075958982148, 'epoch': 0.34}

 34%|███▍      | 925/2733 [3:27:23<6:47:31, 13.52s/it]
 34%|███▍      | 926/2733 [3:27:36<6:49:31, 13.60s/it]
                                                      
{'loss': 1.7871, 'learning_rate': 0.000205886821116597, 'epoch': 0.34}

 34%|███▍      | 926/2733 [3:27:37<6:49:31, 13.60s/it]
 34%|███▍      | 927/2733 [3:27:49<6:43:38, 13.41s/it]
                                                      
{'loss': 1.6662, 'learning_rate': 0.00020577288264337257, 'epoch': 0.34}

 34%|███▍      | 927/2733 [3:27:49<6:43:38, 13.41s/it]
 34%|███▍      | 928/2733 [3:28:03<6:45:39, 13.48s/it]
                                                      
{'loss': 1.7104, 'learning_rate': 0.0002056589441701481, 'epoch': 0.34}

 34%|███▍      | 928/2733 [3:28:03<6:45:39, 13.48s/it]
 34%|███▍      | 929/2733 [3:28:16<6:41:44, 13.36s/it]
                                                      
{'loss': 1.6483, 'learning_rate': 0.00020554500569692366, 'epoch': 0.34}

 34%|███▍      | 929/2733 [3:28:16<6:41:44, 13.36s/it]
 34%|███▍      | 930/2733 [3:28:30<6:42:39, 13.40s/it]
                                                      
{'loss': 1.6784, 'learning_rate': 0.0002054310672236992, 'epoch': 0.34}

 34%|███▍      | 930/2733 [3:28:30<6:42:39, 13.40s/it]
 34%|███▍      | 931/2733 [3:28:44<6:49:51, 13.65s/it]
                                                      
{'loss': 1.7466, 'learning_rate': 0.00020531712875047472, 'epoch': 0.34}

 34%|███▍      | 931/2733 [3:28:44<6:49:51, 13.65s/it]
 34%|███▍      | 932/2733 [3:28:57<6:44:23, 13.47s/it]
                                                      
{'loss': 1.6708, 'learning_rate': 0.00020520319027725028, 'epoch': 0.34}

 34%|███▍      | 932/2733 [3:28:57<6:44:23, 13.47s/it]
 34%|███▍      | 933/2733 [3:29:11<6:48:36, 13.62s/it]
                                                      
{'loss': 1.7096, 'learning_rate': 0.0002050892518040258, 'epoch': 0.34}

 34%|███▍      | 933/2733 [3:29:11<6:48:36, 13.62s/it]
 34%|███▍      | 934/2733 [3:29:24<6:46:58, 13.57s/it]
                                                      
{'loss': 1.7189, 'learning_rate': 0.00020497531333080136, 'epoch': 0.34}

 34%|███▍      | 934/2733 [3:29:24<6:46:58, 13.57s/it]
 34%|███▍      | 935/2733 [3:29:37<6:40:56, 13.38s/it]
                                                      
{'loss': 1.6377, 'learning_rate': 0.0002048613748575769, 'epoch': 0.34}

 34%|███▍      | 935/2733 [3:29:37<6:40:56, 13.38s/it]
 34%|███▍      | 936/2733 [3:29:52<6:48:25, 13.64s/it]
                                                      
{'loss': 1.7085, 'learning_rate': 0.00020474743638435245, 'epoch': 0.34}

 34%|███▍      | 936/2733 [3:29:52<6:48:25, 13.64s/it]
 34%|███▍      | 937/2733 [3:30:04<6:39:12, 13.34s/it]
                                                      
{'loss': 1.6687, 'learning_rate': 0.00020463349791112798, 'epoch': 0.34}

 34%|███▍      | 937/2733 [3:30:04<6:39:12, 13.34s/it]
 34%|███▍      | 938/2733 [3:30:18<6:41:48, 13.43s/it]
                                                      
{'loss': 1.6911, 'learning_rate': 0.0002045195594379035, 'epoch': 0.34}

 34%|███▍      | 938/2733 [3:30:18<6:41:48, 13.43s/it]
 34%|███▍      | 939/2733 [3:30:31<6:38:32, 13.33s/it]
                                                      
{'loss': 1.7022, 'learning_rate': 0.00020440562096467907, 'epoch': 0.34}

 34%|███▍      | 939/2733 [3:30:31<6:38:32, 13.33s/it]
 34%|███▍      | 940/2733 [3:30:44<6:38:22, 13.33s/it]
                                                      
{'loss': 1.711, 'learning_rate': 0.0002042916824914546, 'epoch': 0.34}

 34%|███▍      | 940/2733 [3:30:44<6:38:22, 13.33s/it]
 34%|███▍      | 941/2733 [3:30:57<6:37:02, 13.29s/it]
                                                      
{'loss': 1.6762, 'learning_rate': 0.00020417774401823016, 'epoch': 0.34}

 34%|███▍      | 941/2733 [3:30:58<6:37:02, 13.29s/it]
 34%|███▍      | 942/2733 [3:31:11<6:37:29, 13.32s/it]
                                                      
{'loss': 1.6287, 'learning_rate': 0.0002040638055450057, 'epoch': 0.34}

 34%|███▍      | 942/2733 [3:31:11<6:37:29, 13.32s/it]
 35%|███▍      | 943/2733 [3:31:25<6:40:19, 13.42s/it]
                                                      
{'loss': 1.6635, 'learning_rate': 0.00020394986707178124, 'epoch': 0.35}

 35%|███▍      | 943/2733 [3:31:25<6:40:19, 13.42s/it]
 35%|███▍      | 944/2733 [3:31:38<6:39:52, 13.41s/it]
                                                      
{'loss': 1.687, 'learning_rate': 0.00020383592859855678, 'epoch': 0.35}

 35%|███▍      | 944/2733 [3:31:38<6:39:52, 13.41s/it]
 35%|███▍      | 945/2733 [3:31:52<6:41:25, 13.47s/it]
                                                      
{'loss': 1.749, 'learning_rate': 0.00020372199012533228, 'epoch': 0.35}

 35%|███▍      | 945/2733 [3:31:52<6:41:25, 13.47s/it]
 35%|███▍      | 946/2733 [3:32:05<6:43:25, 13.55s/it]
                                                      
{'loss': 1.7634, 'learning_rate': 0.00020360805165210786, 'epoch': 0.35}

 35%|███▍      | 946/2733 [3:32:05<6:43:25, 13.55s/it]
 35%|███▍      | 947/2733 [3:32:20<6:49:37, 13.76s/it]
                                                      
{'loss': 1.7194, 'learning_rate': 0.00020349411317888337, 'epoch': 0.35}

 35%|███▍      | 947/2733 [3:32:20<6:49:37, 13.76s/it]
 35%|███▍      | 948/2733 [3:32:33<6:48:30, 13.73s/it]
                                                      
{'loss': 1.761, 'learning_rate': 0.00020338017470565895, 'epoch': 0.35}

 35%|███▍      | 948/2733 [3:32:33<6:48:30, 13.73s/it]
 35%|███▍      | 949/2733 [3:32:47<6:49:25, 13.77s/it]
                                                      
{'loss': 1.7483, 'learning_rate': 0.00020326623623243445, 'epoch': 0.35}

 35%|███▍      | 949/2733 [3:32:47<6:49:25, 13.77s/it]
 35%|███▍      | 950/2733 [3:33:01<6:47:20, 13.71s/it]
                                                      
{'loss': 1.6647, 'learning_rate': 0.00020315229775921004, 'epoch': 0.35}

 35%|███▍      | 950/2733 [3:33:01<6:47:20, 13.71s/it]
 35%|███▍      | 951/2733 [3:33:14<6:41:01, 13.50s/it]
                                                      
{'loss': 1.6521, 'learning_rate': 0.00020303835928598554, 'epoch': 0.35}

 35%|███▍      | 951/2733 [3:33:14<6:41:01, 13.50s/it]
 35%|███▍      | 952/2733 [3:33:26<6:32:57, 13.24s/it]
                                                      
{'loss': 1.6659, 'learning_rate': 0.00020292442081276107, 'epoch': 0.35}

 35%|███▍      | 952/2733 [3:33:26<6:32:57, 13.24s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 35%|███▍      | 953/2733 [3:33:39<6:32:45, 13.24s/it]
                                                      
{'loss': 1.7851, 'learning_rate': 0.00020281048233953663, 'epoch': 0.35}

 35%|███▍      | 953/2733 [3:33:39<6:32:45, 13.24s/it]
 35%|███▍      | 954/2733 [3:33:53<6:34:55, 13.32s/it]
                                                      
{'loss': 1.7587, 'learning_rate': 0.00020269654386631216, 'epoch': 0.35}

 35%|███▍      | 954/2733 [3:33:53<6:34:55, 13.32s/it]
 35%|███▍      | 955/2733 [3:34:06<6:35:50, 13.36s/it]
                                                      
{'loss': 1.7519, 'learning_rate': 0.00020258260539308772, 'epoch': 0.35}

 35%|███▍      | 955/2733 [3:34:06<6:35:50, 13.36s/it]
 35%|███▍      | 956/2733 [3:34:20<6:36:37, 13.39s/it]
                                                      
{'loss': 1.723, 'learning_rate': 0.00020246866691986325, 'epoch': 0.35}

 35%|███▍      | 956/2733 [3:34:20<6:36:37, 13.39s/it]
 35%|███▌      | 957/2733 [3:34:34<6:38:20, 13.46s/it]
                                                      
{'loss': 1.7924, 'learning_rate': 0.00020235472844663878, 'epoch': 0.35}

 35%|███▌      | 957/2733 [3:34:34<6:38:20, 13.46s/it]
 35%|███▌      | 958/2733 [3:34:46<6:31:31, 13.23s/it]
                                                      
{'loss': 1.6819, 'learning_rate': 0.00020224078997341434, 'epoch': 0.35}

 35%|███▌      | 958/2733 [3:34:46<6:31:31, 13.23s/it]
 35%|███▌      | 959/2733 [3:34:59<6:24:08, 12.99s/it]
                                                      
{'loss': 1.7369, 'learning_rate': 0.00020212685150018987, 'epoch': 0.35}

 35%|███▌      | 959/2733 [3:34:59<6:24:08, 12.99s/it]
 35%|███▌      | 960/2733 [3:35:12<6:26:25, 13.08s/it]
                                                      
{'loss': 1.6917, 'learning_rate': 0.00020201291302696542, 'epoch': 0.35}

 35%|███▌      | 960/2733 [3:35:12<6:26:25, 13.08s/it]
 35%|███▌      | 961/2733 [3:35:26<6:31:53, 13.27s/it]
                                                      
{'loss': 1.7749, 'learning_rate': 0.00020189897455374095, 'epoch': 0.35}

 35%|███▌      | 961/2733 [3:35:26<6:31:53, 13.27s/it]
 35%|███▌      | 962/2733 [3:35:40<6:40:42, 13.58s/it]
                                                      
{'loss': 1.7799, 'learning_rate': 0.0002017850360805165, 'epoch': 0.35}

 35%|███▌      | 962/2733 [3:35:40<6:40:42, 13.58s/it]
 35%|███▌      | 963/2733 [3:35:54<6:40:30, 13.58s/it]
                                                      
{'loss': 1.7187, 'learning_rate': 0.00020167109760729204, 'epoch': 0.35}

 35%|███▌      | 963/2733 [3:35:54<6:40:30, 13.58s/it]
 35%|███▌      | 964/2733 [3:36:07<6:42:03, 13.64s/it]
                                                      
{'loss': 1.7414, 'learning_rate': 0.00020155715913406757, 'epoch': 0.35}

 35%|███▌      | 964/2733 [3:36:07<6:42:03, 13.64s/it]
 35%|███▌      | 965/2733 [3:36:20<6:32:52, 13.33s/it]
                                                      
{'loss': 1.7714, 'learning_rate': 0.00020144322066084313, 'epoch': 0.35}

 35%|███▌      | 965/2733 [3:36:20<6:32:52, 13.33s/it]
 35%|███▌      | 966/2733 [3:36:34<6:36:56, 13.48s/it]
                                                      
{'loss': 1.6987, 'learning_rate': 0.00020132928218761866, 'epoch': 0.35}

 35%|███▌      | 966/2733 [3:36:34<6:36:56, 13.48s/it]
 35%|███▌      | 967/2733 [3:36:48<6:40:54, 13.62s/it]
                                                      
{'loss': 1.6746, 'learning_rate': 0.00020121534371439422, 'epoch': 0.35}

 35%|███▌      | 967/2733 [3:36:48<6:40:54, 13.62s/it]
 35%|███▌      | 968/2733 [3:37:01<6:38:42, 13.55s/it]
                                                      
{'loss': 1.765, 'learning_rate': 0.00020110140524116975, 'epoch': 0.35}

 35%|███▌      | 968/2733 [3:37:01<6:38:42, 13.55s/it]
 35%|███▌      | 969/2733 [3:37:14<6:34:28, 13.42s/it]
                                                      
{'loss': 1.7471, 'learning_rate': 0.0002009874667679453, 'epoch': 0.35}

 35%|███▌      | 969/2733 [3:37:14<6:34:28, 13.42s/it]
 35%|███▌      | 970/2733 [3:37:28<6:36:35, 13.50s/it]
                                                      
{'loss': 1.7381, 'learning_rate': 0.00020087352829472084, 'epoch': 0.35}

 35%|███▌      | 970/2733 [3:37:28<6:36:35, 13.50s/it]
 36%|███▌      | 971/2733 [3:37:41<6:36:49, 13.51s/it]
                                                      
{'loss': 1.6662, 'learning_rate': 0.00020075958982149637, 'epoch': 0.36}

 36%|███▌      | 971/2733 [3:37:41<6:36:49, 13.51s/it]
 36%|███▌      | 972/2733 [3:37:54<6:31:07, 13.33s/it]
                                                      
{'loss': 1.792, 'learning_rate': 0.00020064565134827192, 'epoch': 0.36}

 36%|███▌      | 972/2733 [3:37:54<6:31:07, 13.33s/it]
 36%|███▌      | 973/2733 [3:38:07<6:27:21, 13.21s/it]
                                                      
{'loss': 1.6566, 'learning_rate': 0.00020053171287504745, 'epoch': 0.36}

 36%|███▌      | 973/2733 [3:38:07<6:27:21, 13.21s/it]
 36%|███▌      | 974/2733 [3:38:20<6:25:48, 13.16s/it]
                                                      
{'loss': 1.7468, 'learning_rate': 0.000200417774401823, 'epoch': 0.36}

 36%|███▌      | 974/2733 [3:38:20<6:25:48, 13.16s/it]
 36%|███▌      | 975/2733 [3:38:34<6:29:41, 13.30s/it]
                                                      
{'loss': 1.6817, 'learning_rate': 0.00020030383592859854, 'epoch': 0.36}

 36%|███▌      | 975/2733 [3:38:34<6:29:41, 13.30s/it]
 36%|███▌      | 976/2733 [3:38:47<6:27:49, 13.24s/it]
                                                      
{'loss': 1.6819, 'learning_rate': 0.0002001898974553741, 'epoch': 0.36}

 36%|███▌      | 976/2733 [3:38:47<6:27:49, 13.24s/it]
 36%|███▌      | 977/2733 [3:39:01<6:35:50, 13.53s/it]
                                                      
{'loss': 1.7715, 'learning_rate': 0.00020007595898214963, 'epoch': 0.36}

 36%|███▌      | 977/2733 [3:39:01<6:35:50, 13.53s/it]
 36%|███▌      | 978/2733 [3:39:15<6:35:24, 13.52s/it]
                                                      
{'loss': 1.7546, 'learning_rate': 0.00019996202050892516, 'epoch': 0.36}

 36%|███▌      | 978/2733 [3:39:15<6:35:24, 13.52s/it]
 36%|███▌      | 979/2733 [3:39:28<6:34:44, 13.50s/it]
                                                      
{'loss': 1.7356, 'learning_rate': 0.00019984808203570072, 'epoch': 0.36}

 36%|███▌      | 979/2733 [3:39:28<6:34:44, 13.50s/it]
 36%|███▌      | 980/2733 [3:39:42<6:35:37, 13.54s/it]
                                                      
{'loss': 1.6318, 'learning_rate': 0.00019973414356247625, 'epoch': 0.36}

 36%|███▌      | 980/2733 [3:39:42<6:35:37, 13.54s/it]
 36%|███▌      | 981/2733 [3:39:55<6:35:54, 13.56s/it]
                                                      
{'loss': 1.675, 'learning_rate': 0.0001996202050892518, 'epoch': 0.36}

 36%|███▌      | 981/2733 [3:39:55<6:35:54, 13.56s/it]
 36%|███▌      | 982/2733 [3:40:09<6:33:05, 13.47s/it]
                                                      
{'loss': 1.6391, 'learning_rate': 0.00019950626661602733, 'epoch': 0.36}

 36%|███▌      | 982/2733 [3:40:09<6:33:05, 13.47s/it]
 36%|███▌      | 983/2733 [3:40:23<6:39:33, 13.70s/it]
                                                      
{'loss': 1.6307, 'learning_rate': 0.0001993923281428029, 'epoch': 0.36}

 36%|███▌      | 983/2733 [3:40:23<6:39:33, 13.70s/it]
 36%|███▌      | 984/2733 [3:40:35<6:27:19, 13.29s/it]
                                                      
{'loss': 1.711, 'learning_rate': 0.0001992783896695784, 'epoch': 0.36}

 36%|███▌      | 984/2733 [3:40:35<6:27:19, 13.29s/it]
 36%|███▌      | 985/2733 [3:40:48<6:20:55, 13.08s/it]
                                                      
{'loss': 1.6636, 'learning_rate': 0.00019916445119635393, 'epoch': 0.36}

 36%|███▌      | 985/2733 [3:40:48<6:20:55, 13.08s/it]
 36%|███▌      | 986/2733 [3:41:01<6:25:55, 13.25s/it]
                                                      
{'loss': 1.6608, 'learning_rate': 0.00019905051272312948, 'epoch': 0.36}

 36%|███▌      | 986/2733 [3:41:02<6:25:55, 13.25s/it]
 36%|███▌      | 987/2733 [3:41:15<6:26:54, 13.30s/it]
                                                      
{'loss': 1.6504, 'learning_rate': 0.00019893657424990501, 'epoch': 0.36}

 36%|███▌      | 987/2733 [3:41:15<6:26:54, 13.30s/it]
 36%|███▌      | 988/2733 [3:41:29<6:32:45, 13.50s/it]
                                                      
{'loss': 1.6512, 'learning_rate': 0.00019882263577668057, 'epoch': 0.36}

 36%|███▌      | 988/2733 [3:41:29<6:32:45, 13.50s/it]
 36%|███▌      | 989/2733 [3:41:42<6:29:08, 13.39s/it]
                                                      
{'loss': 1.6401, 'learning_rate': 0.0001987086973034561, 'epoch': 0.36}

 36%|███▌      | 989/2733 [3:41:42<6:29:08, 13.39s/it]
 36%|███▌      | 990/2733 [3:41:55<6:25:35, 13.27s/it]
                                                      
{'loss': 1.6297, 'learning_rate': 0.00019859475883023166, 'epoch': 0.36}

 36%|███▌      | 990/2733 [3:41:55<6:25:35, 13.27s/it]
 36%|███▋      | 991/2733 [3:42:08<6:25:59, 13.29s/it]
                                                      
{'loss': 1.666, 'learning_rate': 0.0001984808203570072, 'epoch': 0.36}

 36%|███▋      | 991/2733 [3:42:08<6:25:59, 13.29s/it]
 36%|███▋      | 992/2733 [3:42:22<6:31:14, 13.48s/it]
                                                      
{'loss': 1.8156, 'learning_rate': 0.00019836688188378272, 'epoch': 0.36}

 36%|███▋      | 992/2733 [3:42:22<6:31:14, 13.48s/it]
 36%|███▋      | 993/2733 [3:42:34<6:19:08, 13.07s/it]
                                                      
{'loss': 1.6359, 'learning_rate': 0.00019825294341055828, 'epoch': 0.36}

 36%|███▋      | 993/2733 [3:42:34<6:19:08, 13.07s/it]
 36%|███▋      | 994/2733 [3:42:47<6:16:09, 12.98s/it]
                                                      
{'loss': 1.6404, 'learning_rate': 0.0001981390049373338, 'epoch': 0.36}

 36%|███▋      | 994/2733 [3:42:47<6:16:09, 12.98s/it]
 36%|███▋      | 995/2733 [3:43:00<6:17:39, 13.04s/it]
                                                      
{'loss': 1.6754, 'learning_rate': 0.00019802506646410936, 'epoch': 0.36}

 36%|███▋      | 995/2733 [3:43:00<6:17:39, 13.04s/it]
 36%|███▋      | 996/2733 [3:43:14<6:19:12, 13.10s/it]
                                                      
{'loss': 1.7621, 'learning_rate': 0.0001979111279908849, 'epoch': 0.36}

 36%|███▋      | 996/2733 [3:43:14<6:19:12, 13.10s/it]
 36%|███▋      | 997/2733 [3:43:26<6:15:01, 12.96s/it]
                                                      
{'loss': 1.6105, 'learning_rate': 0.00019779718951766045, 'epoch': 0.36}

 36%|███▋      | 997/2733 [3:43:26<6:15:01, 12.96s/it]
 37%|███▋      | 998/2733 [3:43:40<6:18:48, 13.10s/it]
                                                      
{'loss': 1.6278, 'learning_rate': 0.00019768325104443598, 'epoch': 0.37}

 37%|███▋      | 998/2733 [3:43:40<6:18:48, 13.10s/it]
 37%|███▋      | 999/2733 [3:43:53<6:18:10, 13.09s/it]
                                                      
{'loss': 1.6653, 'learning_rate': 0.0001975693125712115, 'epoch': 0.37}

 37%|███▋      | 999/2733 [3:43:53<6:18:10, 13.09s/it]
 37%|███▋      | 1000/2733 [3:44:06<6:21:50, 13.22s/it]
                                                       
{'loss': 1.7183, 'learning_rate': 0.00019745537409798707, 'epoch': 0.37}

 37%|███▋      | 1000/2733 [3:44:06<6:21:50, 13.22s/it]
 37%|███▋      | 1001/2733 [3:44:20<6:23:29, 13.28s/it]
                                                       
{'loss': 1.6499, 'learning_rate': 0.0001973414356247626, 'epoch': 0.37}

 37%|███▋      | 1001/2733 [3:44:20<6:23:29, 13.28s/it]
 37%|███▋      | 1002/2733 [3:44:34<6:30:19, 13.53s/it]
                                                       
{'loss': 1.727, 'learning_rate': 0.00019722749715153816, 'epoch': 0.37}

 37%|███▋      | 1002/2733 [3:44:34<6:30:19, 13.53s/it]
 37%|███▋      | 1003/2733 [3:44:47<6:27:46, 13.45s/it]
                                                       
{'loss': 1.7023, 'learning_rate': 0.0001971135586783137, 'epoch': 0.37}

 37%|███▋      | 1003/2733 [3:44:47<6:27:46, 13.45s/it]
 37%|███▋      | 1004/2733 [3:45:01<6:28:52, 13.49s/it]
                                                       
{'loss': 1.7306, 'learning_rate': 0.00019699962020508925, 'epoch': 0.37}

 37%|███▋      | 1004/2733 [3:45:01<6:28:52, 13.49s/it]
 37%|███▋      | 1005/2733 [3:45:14<6:26:59, 13.44s/it]
                                                       
{'loss': 1.7494, 'learning_rate': 0.00019688568173186478, 'epoch': 0.37}

 37%|███▋      | 1005/2733 [3:45:14<6:26:59, 13.44s/it]
 37%|███▋      | 1006/2733 [3:45:28<6:31:33, 13.60s/it]
                                                       
{'loss': 1.6776, 'learning_rate': 0.0001967717432586403, 'epoch': 0.37}

 37%|███▋      | 1006/2733 [3:45:28<6:31:33, 13.60s/it]
 37%|███▋      | 1007/2733 [3:45:41<6:23:08, 13.32s/it]
                                                       
{'loss': 1.6606, 'learning_rate': 0.00019665780478541586, 'epoch': 0.37}

 37%|███▋      | 1007/2733 [3:45:41<6:23:08, 13.32s/it]
 37%|███▋      | 1008/2733 [3:45:54<6:28:03, 13.50s/it]
                                                       
{'loss': 1.6941, 'learning_rate': 0.0001965438663121914, 'epoch': 0.37}

 37%|███▋      | 1008/2733 [3:45:54<6:28:03, 13.50s/it]
 37%|███▋      | 1009/2733 [3:46:08<6:31:15, 13.62s/it]
                                                       
{'loss': 1.6828, 'learning_rate': 0.00019642992783896695, 'epoch': 0.37}

 37%|███▋      | 1009/2733 [3:46:08<6:31:15, 13.62s/it]
 37%|███▋      | 1010/2733 [3:46:22<6:29:54, 13.58s/it]
                                                       
{'loss': 1.7325, 'learning_rate': 0.00019631598936574248, 'epoch': 0.37}

 37%|███▋      | 1010/2733 [3:46:22<6:29:54, 13.58s/it]
 37%|███▋      | 1011/2733 [3:46:35<6:28:10, 13.53s/it]
                                                       
{'loss': 1.6702, 'learning_rate': 0.00019620205089251804, 'epoch': 0.37}

 37%|███▋      | 1011/2733 [3:46:35<6:28:10, 13.53s/it]
 37%|███▋      | 1012/2733 [3:46:49<6:32:23, 13.68s/it]
                                                       
{'loss': 1.6196, 'learning_rate': 0.00019608811241929357, 'epoch': 0.37}

 37%|███▋      | 1012/2733 [3:46:49<6:32:23, 13.68s/it]
 37%|███▋      | 1013/2733 [3:47:02<6:24:35, 13.42s/it]
                                                       
{'loss': 1.7267, 'learning_rate': 0.0001959741739460691, 'epoch': 0.37}

 37%|███▋      | 1013/2733 [3:47:02<6:24:35, 13.42s/it]
 37%|███▋      | 1014/2733 [3:47:16<6:24:26, 13.42s/it]
                                                       
{'loss': 1.7492, 'learning_rate': 0.00019586023547284466, 'epoch': 0.37}

 37%|███▋      | 1014/2733 [3:47:16<6:24:26, 13.42s/it]
 37%|███▋      | 1015/2733 [3:47:29<6:23:43, 13.40s/it]
                                                       
{'loss': 1.719, 'learning_rate': 0.0001957462969996202, 'epoch': 0.37}

 37%|███▋      | 1015/2733 [3:47:29<6:23:43, 13.40s/it]
 37%|███▋      | 1016/2733 [3:47:42<6:24:08, 13.42s/it]
                                                       
{'loss': 1.6358, 'learning_rate': 0.00019563235852639575, 'epoch': 0.37}

 37%|███▋      | 1016/2733 [3:47:42<6:24:08, 13.42s/it]
 37%|███▋      | 1017/2733 [3:47:56<6:25:15, 13.47s/it]
                                                       
{'loss': 1.6783, 'learning_rate': 0.00019551842005317128, 'epoch': 0.37}

 37%|███▋      | 1017/2733 [3:47:56<6:25:15, 13.47s/it]
 37%|███▋      | 1018/2733 [3:48:09<6:19:00, 13.26s/it]
                                                       
{'loss': 1.7822, 'learning_rate': 0.00019540448157994683, 'epoch': 0.37}

 37%|███▋      | 1018/2733 [3:48:09<6:19:00, 13.26s/it]
 37%|███▋      | 1019/2733 [3:48:23<6:25:34, 13.50s/it]
                                                       
{'loss': 1.757, 'learning_rate': 0.00019529054310672236, 'epoch': 0.37}

 37%|███▋      | 1019/2733 [3:48:23<6:25:34, 13.50s/it]
 37%|███▋      | 1020/2733 [3:48:35<6:17:40, 13.23s/it]
                                                       
{'loss': 1.597, 'learning_rate': 0.00019517660463349787, 'epoch': 0.37}

 37%|███▋      | 1020/2733 [3:48:35<6:17:40, 13.23s/it]
 37%|███▋      | 1021/2733 [3:48:49<6:19:02, 13.28s/it]
                                                       
{'loss': 1.6719, 'learning_rate': 0.00019506266616027345, 'epoch': 0.37}

 37%|███▋      | 1021/2733 [3:48:49<6:19:02, 13.28s/it]
 37%|███▋      | 1022/2733 [3:49:02<6:22:26, 13.41s/it]
                                                       
{'loss': 1.6663, 'learning_rate': 0.00019494872768704896, 'epoch': 0.37}

 37%|███▋      | 1022/2733 [3:49:02<6:22:26, 13.41s/it]
 37%|███▋      | 1023/2733 [3:49:16<6:27:09, 13.58s/it]
                                                       
{'loss': 1.6675, 'learning_rate': 0.00019483478921382454, 'epoch': 0.37}

 37%|███▋      | 1023/2733 [3:49:16<6:27:09, 13.58s/it]
 37%|███▋      | 1024/2733 [3:49:29<6:19:17, 13.32s/it]
                                                       
{'loss': 1.6053, 'learning_rate': 0.00019472085074060004, 'epoch': 0.37}

 37%|███▋      | 1024/2733 [3:49:29<6:19:17, 13.32s/it]
 38%|███▊      | 1025/2733 [3:49:42<6:18:32, 13.30s/it]
                                                       
{'loss': 1.6159, 'learning_rate': 0.00019460691226737563, 'epoch': 0.38}

 38%|███▊      | 1025/2733 [3:49:42<6:18:32, 13.30s/it]
 38%|███▊      | 1026/2733 [3:49:56<6:24:40, 13.52s/it]
                                                       
{'loss': 1.5865, 'learning_rate': 0.00019449297379415113, 'epoch': 0.38}

 38%|███▊      | 1026/2733 [3:49:56<6:24:40, 13.52s/it]
 38%|███▊      | 1027/2733 [3:50:09<6:18:07, 13.30s/it]
                                                       
{'loss': 1.7044, 'learning_rate': 0.00019437903532092666, 'epoch': 0.38}

 38%|███▊      | 1027/2733 [3:50:09<6:18:07, 13.30s/it]
 38%|███▊      | 1028/2733 [3:50:22<6:09:26, 13.00s/it]
                                                       
{'loss': 1.598, 'learning_rate': 0.00019426509684770222, 'epoch': 0.38}

 38%|███▊      | 1028/2733 [3:50:22<6:09:26, 13.00s/it]
 38%|███▊      | 1029/2733 [3:50:35<6:10:51, 13.06s/it]
                                                       
{'loss': 1.7034, 'learning_rate': 0.00019415115837447775, 'epoch': 0.38}

 38%|███▊      | 1029/2733 [3:50:35<6:10:51, 13.06s/it]
 38%|███▊      | 1030/2733 [3:50:49<6:20:52, 13.42s/it]
                                                       
{'loss': 1.7589, 'learning_rate': 0.0001940372199012533, 'epoch': 0.38}

 38%|███▊      | 1030/2733 [3:50:49<6:20:52, 13.42s/it]
 38%|███▊      | 1031/2733 [3:51:03<6:27:33, 13.66s/it]
                                                       
{'loss': 1.6929, 'learning_rate': 0.00019392328142802884, 'epoch': 0.38}

 38%|███▊      | 1031/2733 [3:51:03<6:27:33, 13.66s/it]
 38%|███▊      | 1032/2733 [3:51:16<6:23:45, 13.54s/it]
                                                       
{'loss': 1.746, 'learning_rate': 0.0001938093429548044, 'epoch': 0.38}

 38%|███▊      | 1032/2733 [3:51:16<6:23:45, 13.54s/it]
 38%|███▊      | 1033/2733 [3:51:29<6:16:39, 13.29s/it]
                                                       
{'loss': 1.6318, 'learning_rate': 0.00019369540448157992, 'epoch': 0.38}

 38%|███▊      | 1033/2733 [3:51:29<6:16:39, 13.29s/it]
 38%|███▊      | 1034/2733 [3:51:42<6:13:23, 13.19s/it]
                                                       
{'loss': 1.6131, 'learning_rate': 0.00019358146600835545, 'epoch': 0.38}

 38%|███▊      | 1034/2733 [3:51:42<6:13:23, 13.19s/it]
 38%|███▊      | 1035/2733 [3:51:56<6:17:10, 13.33s/it]
                                                       
{'loss': 1.7733, 'learning_rate': 0.000193467527535131, 'epoch': 0.38}

 38%|███▊      | 1035/2733 [3:51:56<6:17:10, 13.33s/it]
 38%|███▊      | 1036/2733 [3:52:09<6:16:20, 13.31s/it]
                                                       
{'loss': 1.636, 'learning_rate': 0.00019335358906190654, 'epoch': 0.38}

 38%|███▊      | 1036/2733 [3:52:09<6:16:20, 13.31s/it]
 38%|███▊      | 1037/2733 [3:52:22<6:15:14, 13.28s/it]
                                                       
{'loss': 1.6885, 'learning_rate': 0.0001932396505886821, 'epoch': 0.38}

 38%|███▊      | 1037/2733 [3:52:22<6:15:14, 13.28s/it]
 38%|███▊      | 1038/2733 [3:52:35<6:13:35, 13.22s/it]
                                                       
{'loss': 1.7616, 'learning_rate': 0.00019312571211545763, 'epoch': 0.38}

 38%|███▊      | 1038/2733 [3:52:35<6:13:35, 13.22s/it]
 38%|███▊      | 1039/2733 [3:52:48<6:11:17, 13.15s/it]
                                                       
{'loss': 1.684, 'learning_rate': 0.0001930117736422332, 'epoch': 0.38}

 38%|███▊      | 1039/2733 [3:52:48<6:11:17, 13.15s/it]
 38%|███▊      | 1040/2733 [3:53:02<6:13:07, 13.22s/it]
                                                       
{'loss': 1.7465, 'learning_rate': 0.00019289783516900872, 'epoch': 0.38}

 38%|███▊      | 1040/2733 [3:53:02<6:13:07, 13.22s/it]
 38%|███▊      | 1041/2733 [3:53:14<6:08:30, 13.07s/it]
                                                       
{'loss': 1.6334, 'learning_rate': 0.00019278389669578425, 'epoch': 0.38}

 38%|███▊      | 1041/2733 [3:53:14<6:08:30, 13.07s/it]
 38%|███▊      | 1042/2733 [3:53:28<6:12:51, 13.23s/it]
                                                       
{'loss': 1.7501, 'learning_rate': 0.0001926699582225598, 'epoch': 0.38}

 38%|███▊      | 1042/2733 [3:53:28<6:12:51, 13.23s/it]
 38%|███▊      | 1043/2733 [3:53:42<6:16:40, 13.37s/it]
                                                       
{'loss': 1.6535, 'learning_rate': 0.00019255601974933534, 'epoch': 0.38}

 38%|███▊      | 1043/2733 [3:53:42<6:16:40, 13.37s/it]
 38%|███▊      | 1044/2733 [3:53:56<6:19:53, 13.50s/it]
                                                       
{'loss': 1.7754, 'learning_rate': 0.0001924420812761109, 'epoch': 0.38}

 38%|███▊      | 1044/2733 [3:53:56<6:19:53, 13.50s/it]
 38%|███▊      | 1045/2733 [3:54:09<6:18:25, 13.45s/it]
                                                       
{'loss': 1.7158, 'learning_rate': 0.00019232814280288642, 'epoch': 0.38}

 38%|███▊      | 1045/2733 [3:54:09<6:18:25, 13.45s/it]
 38%|███▊      | 1046/2733 [3:54:22<6:18:58, 13.48s/it]
                                                       
{'loss': 1.6515, 'learning_rate': 0.00019221420432966198, 'epoch': 0.38}

 38%|███▊      | 1046/2733 [3:54:22<6:18:58, 13.48s/it]
 38%|███▊      | 1047/2733 [3:54:34<6:06:56, 13.06s/it]
                                                       
{'loss': 1.6939, 'learning_rate': 0.0001921002658564375, 'epoch': 0.38}

 38%|███▊      | 1047/2733 [3:54:35<6:06:56, 13.06s/it]
 38%|███▊      | 1048/2733 [3:54:48<6:13:19, 13.29s/it]
                                                       
{'loss': 1.6853, 'learning_rate': 0.00019198632738321304, 'epoch': 0.38}

 38%|███▊      | 1048/2733 [3:54:48<6:13:19, 13.29s/it]
 38%|███▊      | 1049/2733 [3:55:02<6:18:19, 13.48s/it]
                                                       
{'loss': 1.7203, 'learning_rate': 0.0001918723889099886, 'epoch': 0.38}

 38%|███▊      | 1049/2733 [3:55:02<6:18:19, 13.48s/it]
 38%|███▊      | 1050/2733 [3:55:16<6:17:58, 13.47s/it]
                                                       
{'loss': 1.7444, 'learning_rate': 0.00019175845043676413, 'epoch': 0.38}

 38%|███▊      | 1050/2733 [3:55:16<6:17:58, 13.47s/it]
 38%|███▊      | 1051/2733 [3:55:29<6:13:45, 13.33s/it]
                                                       
{'loss': 1.7092, 'learning_rate': 0.0001916445119635397, 'epoch': 0.38}

 38%|███▊      | 1051/2733 [3:55:29<6:13:45, 13.33s/it]
 38%|███▊      | 1052/2733 [3:55:43<6:18:31, 13.51s/it]
                                                       
{'loss': 1.6975, 'learning_rate': 0.00019153057349031522, 'epoch': 0.38}

 38%|███▊      | 1052/2733 [3:55:43<6:18:31, 13.51s/it]
 39%|███▊      | 1053/2733 [3:55:56<6:19:17, 13.55s/it]
                                                       
{'loss': 1.6107, 'learning_rate': 0.00019141663501709078, 'epoch': 0.39}

 39%|███▊      | 1053/2733 [3:55:56<6:19:17, 13.55s/it]
 39%|███▊      | 1054/2733 [3:56:11<6:24:57, 13.76s/it]
                                                       
{'loss': 1.7273, 'learning_rate': 0.0001913026965438663, 'epoch': 0.39}

 39%|███▊      | 1054/2733 [3:56:11<6:24:57, 13.76s/it]
 39%|███▊      | 1055/2733 [3:56:24<6:23:04, 13.70s/it]
                                                       
{'loss': 1.5877, 'learning_rate': 0.00019118875807064184, 'epoch': 0.39}

 39%|███▊      | 1055/2733 [3:56:24<6:23:04, 13.70s/it]
 39%|███▊      | 1056/2733 [3:56:37<6:20:32, 13.62s/it]
                                                       
{'loss': 1.6782, 'learning_rate': 0.0001910748195974174, 'epoch': 0.39}

 39%|███▊      | 1056/2733 [3:56:38<6:20:32, 13.62s/it]
 39%|███▊      | 1057/2733 [3:56:51<6:15:43, 13.45s/it]
                                                       
{'loss': 1.7444, 'learning_rate': 0.0001909608811241929, 'epoch': 0.39}

 39%|███▊      | 1057/2733 [3:56:51<6:15:43, 13.45s/it]
 39%|███▊      | 1058/2733 [3:57:03<6:07:47, 13.17s/it]
                                                       
{'loss': 1.709, 'learning_rate': 0.00019084694265096848, 'epoch': 0.39}

 39%|███▊      | 1058/2733 [3:57:03<6:07:47, 13.17s/it]
 39%|███▊      | 1059/2733 [3:57:17<6:16:52, 13.51s/it]
                                                       
{'loss': 1.6615, 'learning_rate': 0.00019073300417774398, 'epoch': 0.39}

 39%|███▊      | 1059/2733 [3:57:17<6:16:52, 13.51s/it]
 39%|███▉      | 1060/2733 [3:57:31<6:18:21, 13.57s/it]
                                                       
{'loss': 1.7221, 'learning_rate': 0.00019061906570451957, 'epoch': 0.39}

 39%|███▉      | 1060/2733 [3:57:31<6:18:21, 13.57s/it]
 39%|███▉      | 1061/2733 [3:57:45<6:21:21, 13.69s/it]
                                                       
{'loss': 1.7563, 'learning_rate': 0.00019050512723129507, 'epoch': 0.39}

 39%|███▉      | 1061/2733 [3:57:45<6:21:21, 13.69s/it]
 39%|███▉      | 1062/2733 [3:57:59<6:19:31, 13.63s/it]
                                                       
{'loss': 1.7196, 'learning_rate': 0.0001903911887580706, 'epoch': 0.39}

 39%|███▉      | 1062/2733 [3:57:59<6:19:31, 13.63s/it]
 39%|███▉      | 1063/2733 [3:58:13<6:24:11, 13.80s/it]
                                                       
{'loss': 1.6806, 'learning_rate': 0.00019027725028484616, 'epoch': 0.39}

 39%|███▉      | 1063/2733 [3:58:13<6:24:11, 13.80s/it]
 39%|███▉      | 1064/2733 [3:58:26<6:20:32, 13.68s/it]
                                                       
{'loss': 1.6825, 'learning_rate': 0.0001901633118116217, 'epoch': 0.39}

 39%|███▉      | 1064/2733 [3:58:26<6:20:32, 13.68s/it]
 39%|███▉      | 1065/2733 [3:58:40<6:21:24, 13.72s/it]
                                                       
{'loss': 1.6861, 'learning_rate': 0.00019004937333839725, 'epoch': 0.39}

 39%|███▉      | 1065/2733 [3:58:40<6:21:24, 13.72s/it]
 39%|███▉      | 1066/2733 [3:58:52<6:05:21, 13.15s/it]
                                                       
{'loss': 1.5475, 'learning_rate': 0.00018993543486517278, 'epoch': 0.39}

 39%|███▉      | 1066/2733 [3:58:52<6:05:21, 13.15s/it]
 39%|███▉      | 1067/2733 [3:59:06<6:10:30, 13.34s/it]
                                                       
{'loss': 1.7199, 'learning_rate': 0.00018982149639194834, 'epoch': 0.39}

 39%|███▉      | 1067/2733 [3:59:06<6:10:30, 13.34s/it]
 39%|███▉      | 1068/2733 [3:59:19<6:12:47, 13.43s/it]
                                                       
{'loss': 1.6419, 'learning_rate': 0.00018970755791872387, 'epoch': 0.39}

 39%|███▉      | 1068/2733 [3:59:19<6:12:47, 13.43s/it]
 39%|███▉      | 1069/2733 [3:59:33<6:16:22, 13.57s/it]
                                                       
{'loss': 1.6637, 'learning_rate': 0.0001895936194454994, 'epoch': 0.39}

 39%|███▉      | 1069/2733 [3:59:33<6:16:22, 13.57s/it]
 39%|███▉      | 1070/2733 [3:59:46<6:11:41, 13.41s/it]
                                                       
{'loss': 1.663, 'learning_rate': 0.00018947968097227495, 'epoch': 0.39}

 39%|███▉      | 1070/2733 [3:59:46<6:11:41, 13.41s/it]
 39%|███▉      | 1071/2733 [4:00:00<6:17:59, 13.65s/it]
                                                       
{'loss': 1.7471, 'learning_rate': 0.00018936574249905048, 'epoch': 0.39}

 39%|███▉      | 1071/2733 [4:00:00<6:17:59, 13.65s/it]
 39%|███▉      | 1072/2733 [4:00:13<6:13:04, 13.48s/it]
                                                       
{'loss': 1.8075, 'learning_rate': 0.00018925180402582604, 'epoch': 0.39}

 39%|███▉      | 1072/2733 [4:00:13<6:13:04, 13.48s/it]
 39%|███▉      | 1073/2733 [4:00:26<6:08:58, 13.34s/it]
                                                       
{'loss': 1.6699, 'learning_rate': 0.00018913786555260157, 'epoch': 0.39}

 39%|███▉      | 1073/2733 [4:00:26<6:08:58, 13.34s/it]
 39%|███▉      | 1074/2733 [4:00:41<6:16:28, 13.62s/it]
                                                       
{'loss': 1.7432, 'learning_rate': 0.00018902392707937713, 'epoch': 0.39}

 39%|███▉      | 1074/2733 [4:00:41<6:16:28, 13.62s/it]
 39%|███▉      | 1075/2733 [4:00:54<6:15:34, 13.59s/it]
                                                       
{'loss': 1.6696, 'learning_rate': 0.00018890998860615266, 'epoch': 0.39}

 39%|███▉      | 1075/2733 [4:00:54<6:15:34, 13.59s/it]
 39%|███▉      | 1076/2733 [4:01:08<6:13:47, 13.53s/it]
                                                       
{'loss': 1.6137, 'learning_rate': 0.0001887960501329282, 'epoch': 0.39}

 39%|███▉      | 1076/2733 [4:01:08<6:13:47, 13.53s/it]
 39%|███▉      | 1077/2733 [4:01:21<6:11:18, 13.45s/it]
                                                       
{'loss': 1.7155, 'learning_rate': 0.00018868211165970375, 'epoch': 0.39}

 39%|███▉      | 1077/2733 [4:01:21<6:11:18, 13.45s/it]
 39%|███▉      | 1078/2733 [4:01:35<6:17:47, 13.70s/it]
                                                       
{'loss': 1.7225, 'learning_rate': 0.00018856817318647928, 'epoch': 0.39}

 39%|███▉      | 1078/2733 [4:01:35<6:17:47, 13.70s/it]
 39%|███▉      | 1079/2733 [4:01:49<6:17:18, 13.69s/it]
                                                       
{'loss': 1.6972, 'learning_rate': 0.00018845423471325484, 'epoch': 0.39}

 39%|███▉      | 1079/2733 [4:01:49<6:17:18, 13.69s/it]
 40%|███▉      | 1080/2733 [4:02:03<6:19:54, 13.79s/it]
                                                       
{'loss': 1.7361, 'learning_rate': 0.00018834029624003037, 'epoch': 0.4}

 40%|███▉      | 1080/2733 [4:02:03<6:19:54, 13.79s/it]
 40%|███▉      | 1081/2733 [4:02:17<6:19:23, 13.78s/it]
                                                       
{'loss': 1.6704, 'learning_rate': 0.00018822635776680592, 'epoch': 0.4}

 40%|███▉      | 1081/2733 [4:02:17<6:19:23, 13.78s/it]
 40%|███▉      | 1082/2733 [4:02:31<6:22:46, 13.91s/it]
                                                       
{'loss': 1.7325, 'learning_rate': 0.00018811241929358145, 'epoch': 0.4}

 40%|███▉      | 1082/2733 [4:02:31<6:22:46, 13.91s/it]
 40%|███▉      | 1083/2733 [4:02:45<6:20:53, 13.85s/it]
                                                       
{'loss': 1.6767, 'learning_rate': 0.00018799848082035698, 'epoch': 0.4}

 40%|███▉      | 1083/2733 [4:02:45<6:20:53, 13.85s/it]
 40%|███▉      | 1084/2733 [4:02:59<6:24:13, 13.98s/it]
                                                       
{'loss': 1.8165, 'learning_rate': 0.00018788454234713254, 'epoch': 0.4}

 40%|███▉      | 1084/2733 [4:02:59<6:24:13, 13.98s/it]
 40%|███▉      | 1085/2733 [4:03:12<6:20:52, 13.87s/it]
                                                       
{'loss': 1.6939, 'learning_rate': 0.00018777060387390807, 'epoch': 0.4}

 40%|███▉      | 1085/2733 [4:03:12<6:20:52, 13.87s/it]
 40%|███▉      | 1086/2733 [4:03:25<6:11:27, 13.53s/it]
                                                       
{'loss': 1.6802, 'learning_rate': 0.00018765666540068363, 'epoch': 0.4}

 40%|███▉      | 1086/2733 [4:03:25<6:11:27, 13.53s/it]
 40%|███▉      | 1087/2733 [4:03:39<6:11:30, 13.54s/it]
                                                       
{'loss': 1.6993, 'learning_rate': 0.00018754272692745916, 'epoch': 0.4}

 40%|███▉      | 1087/2733 [4:03:39<6:11:30, 13.54s/it]
 40%|███▉      | 1088/2733 [4:03:51<6:03:08, 13.25s/it]
                                                       
{'loss': 1.7212, 'learning_rate': 0.00018742878845423472, 'epoch': 0.4}

 40%|███▉      | 1088/2733 [4:03:51<6:03:08, 13.25s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 40%|███▉      | 1089/2733 [4:04:05<6:05:59, 13.36s/it]
                                                       
{'loss': 1.7297, 'learning_rate': 0.00018731484998101025, 'epoch': 0.4}

 40%|███▉      | 1089/2733 [4:04:05<6:05:59, 13.36s/it]
 40%|███▉      | 1090/2733 [4:04:18<6:05:46, 13.36s/it]
                                                       
{'loss': 1.6481, 'learning_rate': 0.00018720091150778578, 'epoch': 0.4}

 40%|███▉      | 1090/2733 [4:04:18<6:05:46, 13.36s/it]
 40%|███▉      | 1091/2733 [4:04:32<6:07:32, 13.43s/it]
                                                       
{'loss': 1.6929, 'learning_rate': 0.00018708697303456134, 'epoch': 0.4}

 40%|███▉      | 1091/2733 [4:04:32<6:07:32, 13.43s/it]
 40%|███▉      | 1092/2733 [4:04:46<6:13:44, 13.66s/it]
                                                       
{'loss': 1.6865, 'learning_rate': 0.00018697303456133687, 'epoch': 0.4}

 40%|███▉      | 1092/2733 [4:04:46<6:13:44, 13.66s/it]
 40%|███▉      | 1093/2733 [4:05:00<6:12:56, 13.64s/it]
                                                       
{'loss': 1.7433, 'learning_rate': 0.00018685909608811242, 'epoch': 0.4}

 40%|███▉      | 1093/2733 [4:05:00<6:12:56, 13.64s/it]
 40%|████      | 1094/2733 [4:05:14<6:17:10, 13.81s/it]
                                                       
{'loss': 1.7195, 'learning_rate': 0.00018674515761488795, 'epoch': 0.4}

 40%|████      | 1094/2733 [4:05:14<6:17:10, 13.81s/it]
 40%|████      | 1095/2733 [4:05:27<6:07:36, 13.47s/it]
                                                       
{'loss': 1.69, 'learning_rate': 0.0001866312191416635, 'epoch': 0.4}

 40%|████      | 1095/2733 [4:05:27<6:07:36, 13.47s/it]
 40%|████      | 1096/2733 [4:05:40<6:04:31, 13.36s/it]
                                                       
{'loss': 1.6251, 'learning_rate': 0.00018651728066843904, 'epoch': 0.4}

 40%|████      | 1096/2733 [4:05:40<6:04:31, 13.36s/it]
 40%|████      | 1097/2733 [4:05:53<6:02:24, 13.29s/it]
                                                       
{'loss': 1.6291, 'learning_rate': 0.00018640334219521454, 'epoch': 0.4}

 40%|████      | 1097/2733 [4:05:53<6:02:24, 13.29s/it]
 40%|████      | 1098/2733 [4:06:06<6:05:31, 13.41s/it]
                                                       
{'loss': 1.7143, 'learning_rate': 0.00018628940372199013, 'epoch': 0.4}

 40%|████      | 1098/2733 [4:06:07<6:05:31, 13.41s/it]
 40%|████      | 1099/2733 [4:06:20<6:07:53, 13.51s/it]
                                                       
{'loss': 1.7991, 'learning_rate': 0.00018617546524876563, 'epoch': 0.4}

 40%|████      | 1099/2733 [4:06:20<6:07:53, 13.51s/it]
 40%|████      | 1100/2733 [4:06:34<6:09:43, 13.58s/it]
                                                       
{'loss': 1.656, 'learning_rate': 0.0001860615267755412, 'epoch': 0.4}

 40%|████      | 1100/2733 [4:06:34<6:09:43, 13.58s/it]
 40%|████      | 1101/2733 [4:06:48<6:13:50, 13.74s/it]
                                                       
{'loss': 1.6789, 'learning_rate': 0.00018594758830231672, 'epoch': 0.4}

 40%|████      | 1101/2733 [4:06:48<6:13:50, 13.74s/it]
 40%|████      | 1102/2733 [4:07:01<6:09:52, 13.61s/it]
                                                       
{'loss': 1.7317, 'learning_rate': 0.00018583364982909228, 'epoch': 0.4}

 40%|████      | 1102/2733 [4:07:01<6:09:52, 13.61s/it]
 40%|████      | 1103/2733 [4:07:15<6:08:35, 13.57s/it]
                                                       
{'loss': 1.7439, 'learning_rate': 0.0001857197113558678, 'epoch': 0.4}

 40%|████      | 1103/2733 [4:07:15<6:08:35, 13.57s/it]
 40%|████      | 1104/2733 [4:07:28<6:02:36, 13.36s/it]
                                                       
{'loss': 1.6394, 'learning_rate': 0.00018560577288264334, 'epoch': 0.4}

 40%|████      | 1104/2733 [4:07:28<6:02:36, 13.36s/it]
 40%|████      | 1105/2733 [4:07:41<5:58:01, 13.20s/it]
                                                       
{'loss': 1.7464, 'learning_rate': 0.0001854918344094189, 'epoch': 0.4}

 40%|████      | 1105/2733 [4:07:41<5:58:01, 13.20s/it]
 40%|████      | 1106/2733 [4:07:54<5:58:06, 13.21s/it]
                                                       
{'loss': 1.7868, 'learning_rate': 0.00018537789593619443, 'epoch': 0.4}

 40%|████      | 1106/2733 [4:07:54<5:58:06, 13.21s/it]
 41%|████      | 1107/2733 [4:08:07<5:57:32, 13.19s/it]
                                                       
{'loss': 1.6564, 'learning_rate': 0.00018526395746296998, 'epoch': 0.41}

 41%|████      | 1107/2733 [4:08:07<5:57:32, 13.19s/it]
 41%|████      | 1108/2733 [4:08:20<5:58:28, 13.24s/it]
                                                       
{'loss': 1.6495, 'learning_rate': 0.00018515001898974551, 'epoch': 0.41}

 41%|████      | 1108/2733 [4:08:20<5:58:28, 13.24s/it]
 41%|████      | 1109/2733 [4:08:34<6:00:25, 13.32s/it]
                                                       
{'loss': 1.7519, 'learning_rate': 0.00018503608051652104, 'epoch': 0.41}

 41%|████      | 1109/2733 [4:08:34<6:00:25, 13.32s/it]
 41%|████      | 1110/2733 [4:08:48<6:03:39, 13.44s/it]
                                                       
{'loss': 1.6911, 'learning_rate': 0.0001849221420432966, 'epoch': 0.41}

 41%|████      | 1110/2733 [4:08:48<6:03:39, 13.44s/it]
 41%|████      | 1111/2733 [4:09:01<5:59:52, 13.31s/it]
                                                       
{'loss': 1.7, 'learning_rate': 0.00018480820357007213, 'epoch': 0.41}

 41%|████      | 1111/2733 [4:09:01<5:59:52, 13.31s/it]
 41%|████      | 1112/2733 [4:09:14<5:57:35, 13.24s/it]
                                                       
{'loss': 1.7085, 'learning_rate': 0.0001846942650968477, 'epoch': 0.41}

 41%|████      | 1112/2733 [4:09:14<5:57:35, 13.24s/it]
 41%|████      | 1113/2733 [4:09:27<6:01:38, 13.39s/it]
                                                       
{'loss': 1.6886, 'learning_rate': 0.00018458032662362322, 'epoch': 0.41}

 41%|████      | 1113/2733 [4:09:27<6:01:38, 13.39s/it]
 41%|████      | 1114/2733 [4:09:41<5:59:40, 13.33s/it]
                                                       
{'loss': 1.5864, 'learning_rate': 0.00018446638815039878, 'epoch': 0.41}

 41%|████      | 1114/2733 [4:09:41<5:59:40, 13.33s/it]
 41%|████      | 1115/2733 [4:09:55<6:06:16, 13.58s/it]
                                                       
{'loss': 1.6447, 'learning_rate': 0.0001843524496771743, 'epoch': 0.41}

 41%|████      | 1115/2733 [4:09:55<6:06:16, 13.58s/it]
 41%|████      | 1116/2733 [4:10:07<5:57:55, 13.28s/it]
                                                       
{'loss': 1.7251, 'learning_rate': 0.00018423851120394984, 'epoch': 0.41}

 41%|████      | 1116/2733 [4:10:07<5:57:55, 13.28s/it]
 41%|████      | 1117/2733 [4:10:20<5:56:59, 13.25s/it]
                                                       
{'loss': 1.604, 'learning_rate': 0.0001841245727307254, 'epoch': 0.41}

 41%|████      | 1117/2733 [4:10:20<5:56:59, 13.25s/it]
 41%|████      | 1118/2733 [4:10:34<6:01:02, 13.41s/it]
                                                       
{'loss': 1.6875, 'learning_rate': 0.00018401063425750093, 'epoch': 0.41}

 41%|████      | 1118/2733 [4:10:34<6:01:02, 13.41s/it]
 41%|████      | 1119/2733 [4:10:48<5:59:56, 13.38s/it]
                                                       
{'loss': 1.7456, 'learning_rate': 0.00018389669578427648, 'epoch': 0.41}

 41%|████      | 1119/2733 [4:10:48<5:59:56, 13.38s/it]
 41%|████      | 1120/2733 [4:11:02<6:06:21, 13.63s/it]
                                                       
{'loss': 1.6844, 'learning_rate': 0.000183782757311052, 'epoch': 0.41}

 41%|████      | 1120/2733 [4:11:02<6:06:21, 13.63s/it]
 41%|████      | 1121/2733 [4:11:16<6:09:31, 13.75s/it]
                                                       
{'loss': 1.6576, 'learning_rate': 0.00018366881883782757, 'epoch': 0.41}

 41%|████      | 1121/2733 [4:11:16<6:09:31, 13.75s/it]
 41%|████      | 1122/2733 [4:11:30<6:11:38, 13.84s/it]
                                                       
{'loss': 1.6349, 'learning_rate': 0.0001835548803646031, 'epoch': 0.41}

 41%|████      | 1122/2733 [4:11:30<6:11:38, 13.84s/it]
 41%|████      | 1123/2733 [4:11:44<6:10:25, 13.80s/it]
                                                       
{'loss': 1.6832, 'learning_rate': 0.00018344094189137863, 'epoch': 0.41}

 41%|████      | 1123/2733 [4:11:44<6:10:25, 13.80s/it]
 41%|████      | 1124/2733 [4:11:57<6:07:01, 13.69s/it]
                                                       
{'loss': 1.6532, 'learning_rate': 0.0001833270034181542, 'epoch': 0.41}

 41%|████      | 1124/2733 [4:11:57<6:07:01, 13.69s/it]
 41%|████      | 1125/2733 [4:12:11<6:07:05, 13.70s/it]
                                                       
{'loss': 1.7419, 'learning_rate': 0.00018321306494492972, 'epoch': 0.41}

 41%|████      | 1125/2733 [4:12:11<6:07:05, 13.70s/it]
 41%|████      | 1126/2733 [4:12:24<6:05:56, 13.66s/it]
                                                       
{'loss': 1.6684, 'learning_rate': 0.00018309912647170528, 'epoch': 0.41}

 41%|████      | 1126/2733 [4:12:24<6:05:56, 13.66s/it]
 41%|████      | 1127/2733 [4:12:37<5:57:34, 13.36s/it]
                                                       
{'loss': 1.5529, 'learning_rate': 0.0001829851879984808, 'epoch': 0.41}

 41%|████      | 1127/2733 [4:12:37<5:57:34, 13.36s/it]
 41%|████▏     | 1128/2733 [4:12:51<5:59:27, 13.44s/it]
                                                       
{'loss': 1.6393, 'learning_rate': 0.00018287124952525636, 'epoch': 0.41}

 41%|████▏     | 1128/2733 [4:12:51<5:59:27, 13.44s/it]
 41%|████▏     | 1129/2733 [4:13:04<5:55:46, 13.31s/it]
                                                       
{'loss': 1.6765, 'learning_rate': 0.0001827573110520319, 'epoch': 0.41}

 41%|████▏     | 1129/2733 [4:13:04<5:55:46, 13.31s/it]
 41%|████▏     | 1130/2733 [4:13:17<5:58:57, 13.44s/it]
                                                       
{'loss': 1.6868, 'learning_rate': 0.00018264337257880743, 'epoch': 0.41}

 41%|████▏     | 1130/2733 [4:13:17<5:58:57, 13.44s/it]
 41%|████▏     | 1131/2733 [4:13:31<5:58:38, 13.43s/it]
                                                       
{'loss': 1.6629, 'learning_rate': 0.00018252943410558298, 'epoch': 0.41}

 41%|████▏     | 1131/2733 [4:13:31<5:58:38, 13.43s/it]
 41%|████▏     | 1132/2733 [4:13:44<5:59:40, 13.48s/it]
                                                       
{'loss': 1.7082, 'learning_rate': 0.00018241549563235849, 'epoch': 0.41}

 41%|████▏     | 1132/2733 [4:13:44<5:59:40, 13.48s/it]
 41%|████▏     | 1133/2733 [4:13:57<5:54:32, 13.30s/it]
                                                       
{'loss': 1.7012, 'learning_rate': 0.00018230155715913407, 'epoch': 0.41}

 41%|████▏     | 1133/2733 [4:13:57<5:54:32, 13.30s/it]
 41%|████▏     | 1134/2733 [4:14:09<5:45:16, 12.96s/it]
                                                       
{'loss': 1.5939, 'learning_rate': 0.00018218761868590957, 'epoch': 0.41}

 41%|████▏     | 1134/2733 [4:14:09<5:45:16, 12.96s/it]
 42%|████▏     | 1135/2733 [4:14:23<5:48:29, 13.08s/it]
                                                       
{'loss': 1.6218, 'learning_rate': 0.00018207368021268516, 'epoch': 0.42}

 42%|████▏     | 1135/2733 [4:14:23<5:48:29, 13.08s/it]
 42%|████▏     | 1136/2733 [4:14:37<5:54:14, 13.31s/it]
                                                       
{'loss': 1.7508, 'learning_rate': 0.00018195974173946066, 'epoch': 0.42}

 42%|████▏     | 1136/2733 [4:14:37<5:54:14, 13.31s/it]
 42%|████▏     | 1137/2733 [4:14:50<5:54:40, 13.33s/it]
                                                       
{'loss': 1.755, 'learning_rate': 0.0001818458032662362, 'epoch': 0.42}

 42%|████▏     | 1137/2733 [4:14:50<5:54:40, 13.33s/it]
 42%|████▏     | 1138/2733 [4:15:04<5:56:25, 13.41s/it]
                                                       
{'loss': 1.6252, 'learning_rate': 0.00018173186479301175, 'epoch': 0.42}

 42%|████▏     | 1138/2733 [4:15:04<5:56:25, 13.41s/it]
 42%|████▏     | 1139/2733 [4:15:18<6:02:59, 13.66s/it]
                                                       
{'loss': 1.7494, 'learning_rate': 0.00018161792631978728, 'epoch': 0.42}

 42%|████▏     | 1139/2733 [4:15:18<6:02:59, 13.66s/it]
 42%|████▏     | 1140/2733 [4:15:31<6:02:38, 13.66s/it]
                                                       
{'loss': 1.6462, 'learning_rate': 0.00018150398784656284, 'epoch': 0.42}

 42%|████▏     | 1140/2733 [4:15:31<6:02:38, 13.66s/it]
 42%|████▏     | 1141/2733 [4:15:45<6:02:24, 13.66s/it]
                                                       
{'loss': 1.7782, 'learning_rate': 0.00018139004937333837, 'epoch': 0.42}

 42%|████▏     | 1141/2733 [4:15:45<6:02:24, 13.66s/it]
 42%|████▏     | 1142/2733 [4:15:59<6:02:47, 13.68s/it]
                                                       
{'loss': 1.7161, 'learning_rate': 0.00018127611090011393, 'epoch': 0.42}

 42%|████▏     | 1142/2733 [4:15:59<6:02:47, 13.68s/it]
 42%|████▏     | 1143/2733 [4:16:13<6:02:54, 13.69s/it]
                                                       
{'loss': 1.6508, 'learning_rate': 0.00018116217242688946, 'epoch': 0.42}

 42%|████▏     | 1143/2733 [4:16:13<6:02:54, 13.69s/it]
 42%|████▏     | 1144/2733 [4:16:27<6:07:13, 13.87s/it]
                                                       
{'loss': 1.7537, 'learning_rate': 0.00018104823395366499, 'epoch': 0.42}

 42%|████▏     | 1144/2733 [4:16:27<6:07:13, 13.87s/it]
 42%|████▏     | 1145/2733 [4:16:41<6:07:30, 13.89s/it]
                                                       
{'loss': 1.7967, 'learning_rate': 0.00018093429548044054, 'epoch': 0.42}

 42%|████▏     | 1145/2733 [4:16:41<6:07:30, 13.89s/it]
 42%|████▏     | 1146/2733 [4:16:54<6:03:35, 13.75s/it]
                                                       
{'loss': 1.6674, 'learning_rate': 0.00018082035700721607, 'epoch': 0.42}

 42%|████▏     | 1146/2733 [4:16:54<6:03:35, 13.75s/it]
 42%|████▏     | 1147/2733 [4:17:08<6:02:50, 13.73s/it]
                                                       
{'loss': 1.6941, 'learning_rate': 0.00018070641853399163, 'epoch': 0.42}

 42%|████▏     | 1147/2733 [4:17:08<6:02:50, 13.73s/it]
 42%|████▏     | 1148/2733 [4:17:22<6:02:27, 13.72s/it]
                                                       
{'loss': 1.7176, 'learning_rate': 0.00018059248006076716, 'epoch': 0.42}

 42%|████▏     | 1148/2733 [4:17:22<6:02:27, 13.72s/it]
 42%|████▏     | 1149/2733 [4:17:35<6:00:08, 13.64s/it]
                                                       
{'loss': 1.6226, 'learning_rate': 0.00018047854158754272, 'epoch': 0.42}

 42%|████▏     | 1149/2733 [4:17:35<6:00:08, 13.64s/it]
 42%|████▏     | 1150/2733 [4:17:49<6:05:01, 13.84s/it]
                                                       
{'loss': 1.7088, 'learning_rate': 0.00018036460311431825, 'epoch': 0.42}

 42%|████▏     | 1150/2733 [4:17:49<6:05:01, 13.84s/it]
 42%|████▏     | 1151/2733 [4:18:04<6:08:53, 13.99s/it]
                                                       
{'loss': 1.7542, 'learning_rate': 0.00018025066464109378, 'epoch': 0.42}

 42%|████▏     | 1151/2733 [4:18:04<6:08:53, 13.99s/it]
 42%|████▏     | 1152/2733 [4:18:17<6:07:10, 13.93s/it]
                                                       
{'loss': 1.6727, 'learning_rate': 0.00018013672616786934, 'epoch': 0.42}

 42%|████▏     | 1152/2733 [4:18:17<6:07:10, 13.93s/it]
 42%|████▏     | 1153/2733 [4:18:30<5:56:19, 13.53s/it]
                                                       
{'loss': 1.5987, 'learning_rate': 0.00018002278769464487, 'epoch': 0.42}

 42%|████▏     | 1153/2733 [4:18:30<5:56:19, 13.53s/it]
 42%|████▏     | 1154/2733 [4:18:43<5:55:08, 13.49s/it]
                                                       
{'loss': 1.6287, 'learning_rate': 0.00017990884922142042, 'epoch': 0.42}

 42%|████▏     | 1154/2733 [4:18:43<5:55:08, 13.49s/it]
 42%|████▏     | 1155/2733 [4:18:57<5:57:22, 13.59s/it]
                                                       
{'loss': 1.6722, 'learning_rate': 0.00017979491074819596, 'epoch': 0.42}

 42%|████▏     | 1155/2733 [4:18:57<5:57:22, 13.59s/it]
 42%|████▏     | 1156/2733 [4:19:10<5:51:12, 13.36s/it]
                                                       
{'loss': 1.6183, 'learning_rate': 0.0001796809722749715, 'epoch': 0.42}

 42%|████▏     | 1156/2733 [4:19:10<5:51:12, 13.36s/it]
 42%|████▏     | 1157/2733 [4:19:24<5:51:31, 13.38s/it]
                                                       
{'loss': 1.6571, 'learning_rate': 0.00017956703380174704, 'epoch': 0.42}

 42%|████▏     | 1157/2733 [4:19:24<5:51:31, 13.38s/it]
 42%|████▏     | 1158/2733 [4:19:37<5:55:21, 13.54s/it]
                                                       
{'loss': 1.8273, 'learning_rate': 0.00017945309532852257, 'epoch': 0.42}

 42%|████▏     | 1158/2733 [4:19:37<5:55:21, 13.54s/it]
 42%|████▏     | 1159/2733 [4:19:51<5:57:31, 13.63s/it]
                                                       
{'loss': 1.7542, 'learning_rate': 0.00017933915685529813, 'epoch': 0.42}

 42%|████▏     | 1159/2733 [4:19:51<5:57:31, 13.63s/it]
 42%|████▏     | 1160/2733 [4:20:05<5:56:47, 13.61s/it]
                                                       
{'loss': 1.6894, 'learning_rate': 0.00017922521838207366, 'epoch': 0.42}

 42%|████▏     | 1160/2733 [4:20:05<5:56:47, 13.61s/it]
 42%|████▏     | 1161/2733 [4:20:18<5:56:45, 13.62s/it]
                                                       
{'loss': 1.6068, 'learning_rate': 0.00017911127990884922, 'epoch': 0.42}

 42%|████▏     | 1161/2733 [4:20:18<5:56:45, 13.62s/it]
 43%|████▎     | 1162/2733 [4:20:32<5:55:06, 13.56s/it]
                                                       
{'loss': 1.7623, 'learning_rate': 0.00017899734143562475, 'epoch': 0.43}

 43%|████▎     | 1162/2733 [4:20:32<5:55:06, 13.56s/it]
 43%|████▎     | 1163/2733 [4:20:45<5:52:53, 13.49s/it]
                                                       
{'loss': 1.6641, 'learning_rate': 0.0001788834029624003, 'epoch': 0.43}

 43%|████▎     | 1163/2733 [4:20:45<5:52:53, 13.49s/it]
 43%|████▎     | 1164/2733 [4:20:59<5:58:40, 13.72s/it]
                                                       
{'loss': 1.7416, 'learning_rate': 0.00017876946448917584, 'epoch': 0.43}

 43%|████▎     | 1164/2733 [4:20:59<5:58:40, 13.72s/it]
 43%|████▎     | 1165/2733 [4:21:12<5:52:58, 13.51s/it]
                                                       
{'loss': 1.6919, 'learning_rate': 0.00017865552601595137, 'epoch': 0.43}

 43%|████▎     | 1165/2733 [4:21:13<5:52:58, 13.51s/it]
 43%|████▎     | 1166/2733 [4:21:26<5:50:52, 13.43s/it]
                                                       
{'loss': 1.5959, 'learning_rate': 0.00017854158754272692, 'epoch': 0.43}

 43%|████▎     | 1166/2733 [4:21:26<5:50:52, 13.43s/it]
 43%|████▎     | 1167/2733 [4:21:39<5:45:56, 13.25s/it]
                                                       
{'loss': 1.6811, 'learning_rate': 0.00017842764906950245, 'epoch': 0.43}

 43%|████▎     | 1167/2733 [4:21:39<5:45:56, 13.25s/it]
 43%|████▎     | 1168/2733 [4:21:52<5:47:56, 13.34s/it]
                                                       
{'loss': 1.6126, 'learning_rate': 0.000178313710596278, 'epoch': 0.43}

 43%|████▎     | 1168/2733 [4:21:52<5:47:56, 13.34s/it]
 43%|████▎     | 1169/2733 [4:22:05<5:44:17, 13.21s/it]
                                                       
{'loss': 1.7188, 'learning_rate': 0.00017819977212305354, 'epoch': 0.43}

 43%|████▎     | 1169/2733 [4:22:05<5:44:17, 13.21s/it]
 43%|████▎     | 1170/2733 [4:22:19<5:48:53, 13.39s/it]
                                                       
{'loss': 1.7816, 'learning_rate': 0.0001780858336498291, 'epoch': 0.43}

 43%|████▎     | 1170/2733 [4:22:19<5:48:53, 13.39s/it]
 43%|████▎     | 1171/2733 [4:22:32<5:50:02, 13.45s/it]
                                                       
{'loss': 1.6446, 'learning_rate': 0.00017797189517660463, 'epoch': 0.43}

 43%|████▎     | 1171/2733 [4:22:32<5:50:02, 13.45s/it]
 43%|████▎     | 1172/2733 [4:22:47<5:56:06, 13.69s/it]
                                                       
{'loss': 1.7498, 'learning_rate': 0.00017785795670338013, 'epoch': 0.43}

 43%|████▎     | 1172/2733 [4:22:47<5:56:06, 13.69s/it]
 43%|████▎     | 1173/2733 [4:23:00<5:55:55, 13.69s/it]
                                                       
{'loss': 1.7247, 'learning_rate': 0.00017774401823015572, 'epoch': 0.43}

 43%|████▎     | 1173/2733 [4:23:00<5:55:55, 13.69s/it]
 43%|████▎     | 1174/2733 [4:23:14<5:55:03, 13.66s/it]
                                                       
{'loss': 1.6642, 'learning_rate': 0.00017763007975693122, 'epoch': 0.43}

 43%|████▎     | 1174/2733 [4:23:14<5:55:03, 13.66s/it]
 43%|████▎     | 1175/2733 [4:23:27<5:53:20, 13.61s/it]
                                                       
{'loss': 1.741, 'learning_rate': 0.00017751614128370678, 'epoch': 0.43}

 43%|████▎     | 1175/2733 [4:23:27<5:53:20, 13.61s/it]
 43%|████▎     | 1176/2733 [4:23:41<5:51:43, 13.55s/it]
                                                       
{'loss': 1.6628, 'learning_rate': 0.0001774022028104823, 'epoch': 0.43}

 43%|████▎     | 1176/2733 [4:23:41<5:51:43, 13.55s/it]
 43%|████▎     | 1177/2733 [4:23:54<5:51:53, 13.57s/it]
                                                       
{'loss': 1.707, 'learning_rate': 0.00017728826433725787, 'epoch': 0.43}

 43%|████▎     | 1177/2733 [4:23:54<5:51:53, 13.57s/it]
 43%|████▎     | 1178/2733 [4:24:08<5:50:38, 13.53s/it]
                                                       
{'loss': 1.7058, 'learning_rate': 0.0001771743258640334, 'epoch': 0.43}

 43%|████▎     | 1178/2733 [4:24:08<5:50:38, 13.53s/it]
 43%|████▎     | 1179/2733 [4:24:22<5:51:33, 13.57s/it]
                                                       
{'loss': 1.6219, 'learning_rate': 0.00017706038739080893, 'epoch': 0.43}

 43%|████▎     | 1179/2733 [4:24:22<5:51:33, 13.57s/it]
 43%|████▎     | 1180/2733 [4:24:35<5:48:23, 13.46s/it]
                                                       
{'loss': 1.703, 'learning_rate': 0.00017694644891758448, 'epoch': 0.43}

 43%|████▎     | 1180/2733 [4:24:35<5:48:23, 13.46s/it]
 43%|████▎     | 1181/2733 [4:24:48<5:49:47, 13.52s/it]
                                                       
{'loss': 1.7948, 'learning_rate': 0.00017683251044436002, 'epoch': 0.43}

 43%|████▎     | 1181/2733 [4:24:48<5:49:47, 13.52s/it]
 43%|████▎     | 1182/2733 [4:25:01<5:43:34, 13.29s/it]
                                                       
{'loss': 1.6321, 'learning_rate': 0.00017671857197113557, 'epoch': 0.43}

 43%|████▎     | 1182/2733 [4:25:01<5:43:34, 13.29s/it]
 43%|████▎     | 1183/2733 [4:25:15<5:46:18, 13.41s/it]
                                                       
{'loss': 1.768, 'learning_rate': 0.0001766046334979111, 'epoch': 0.43}

 43%|████▎     | 1183/2733 [4:25:15<5:46:18, 13.41s/it]
 43%|████▎     | 1184/2733 [4:25:28<5:44:11, 13.33s/it]
                                                       
{'loss': 1.645, 'learning_rate': 0.00017649069502468666, 'epoch': 0.43}

 43%|████▎     | 1184/2733 [4:25:28<5:44:11, 13.33s/it]
 43%|████▎     | 1185/2733 [4:25:42<5:45:24, 13.39s/it]
                                                       
{'loss': 1.6707, 'learning_rate': 0.0001763767565514622, 'epoch': 0.43}

 43%|████▎     | 1185/2733 [4:25:42<5:45:24, 13.39s/it]
 43%|████▎     | 1186/2733 [4:25:55<5:45:41, 13.41s/it]
                                                       
{'loss': 1.669, 'learning_rate': 0.00017626281807823772, 'epoch': 0.43}

 43%|████▎     | 1186/2733 [4:25:55<5:45:41, 13.41s/it]
 43%|████▎     | 1187/2733 [4:26:07<5:38:18, 13.13s/it]
                                                       
{'loss': 1.6151, 'learning_rate': 0.00017614887960501328, 'epoch': 0.43}

 43%|████▎     | 1187/2733 [4:26:08<5:38:18, 13.13s/it]
 43%|████▎     | 1188/2733 [4:26:21<5:41:19, 13.26s/it]
                                                       
{'loss': 1.6513, 'learning_rate': 0.0001760349411317888, 'epoch': 0.43}

 43%|████▎     | 1188/2733 [4:26:21<5:41:19, 13.26s/it]
 44%|████▎     | 1189/2733 [4:26:33<5:34:58, 13.02s/it]
                                                       
{'loss': 1.6061, 'learning_rate': 0.00017592100265856437, 'epoch': 0.44}

 44%|████▎     | 1189/2733 [4:26:34<5:34:58, 13.02s/it]
 44%|████▎     | 1190/2733 [4:26:48<5:42:36, 13.32s/it]
                                                       
{'loss': 1.7891, 'learning_rate': 0.0001758070641853399, 'epoch': 0.44}

 44%|████▎     | 1190/2733 [4:26:48<5:42:36, 13.32s/it]
 44%|████▎     | 1191/2733 [4:27:01<5:40:24, 13.25s/it]
                                                       
{'loss': 1.8052, 'learning_rate': 0.00017569312571211545, 'epoch': 0.44}

 44%|████▎     | 1191/2733 [4:27:01<5:40:24, 13.25s/it]
 44%|████▎     | 1192/2733 [4:27:15<5:47:32, 13.53s/it]
                                                       
{'loss': 1.6427, 'learning_rate': 0.00017557918723889098, 'epoch': 0.44}

 44%|████▎     | 1192/2733 [4:27:15<5:47:32, 13.53s/it]
 44%|████▎     | 1193/2733 [4:27:27<5:40:24, 13.26s/it]
                                                       
{'loss': 1.5943, 'learning_rate': 0.00017546524876566651, 'epoch': 0.44}

 44%|████▎     | 1193/2733 [4:27:27<5:40:24, 13.26s/it]
 44%|████▎     | 1194/2733 [4:27:41<5:42:29, 13.35s/it]
                                                       
{'loss': 1.6396, 'learning_rate': 0.00017535131029244207, 'epoch': 0.44}

 44%|████▎     | 1194/2733 [4:27:41<5:42:29, 13.35s/it]
 44%|████▎     | 1195/2733 [4:27:54<5:39:35, 13.25s/it]
                                                       
{'loss': 1.6898, 'learning_rate': 0.0001752373718192176, 'epoch': 0.44}

 44%|████▎     | 1195/2733 [4:27:54<5:39:35, 13.25s/it]
 44%|████▍     | 1196/2733 [4:28:07<5:39:10, 13.24s/it]
                                                       
{'loss': 1.7314, 'learning_rate': 0.00017512343334599316, 'epoch': 0.44}

 44%|████▍     | 1196/2733 [4:28:07<5:39:10, 13.24s/it]
 44%|████▍     | 1197/2733 [4:28:20<5:38:19, 13.22s/it]
                                                       
{'loss': 1.658, 'learning_rate': 0.0001750094948727687, 'epoch': 0.44}

 44%|████▍     | 1197/2733 [4:28:20<5:38:19, 13.22s/it]
 44%|████▍     | 1198/2733 [4:28:35<5:47:06, 13.57s/it]
                                                       
{'loss': 1.7381, 'learning_rate': 0.00017489555639954425, 'epoch': 0.44}

 44%|████▍     | 1198/2733 [4:28:35<5:47:06, 13.57s/it]
 44%|████▍     | 1199/2733 [4:28:48<5:42:39, 13.40s/it]
                                                       
{'loss': 1.6723, 'learning_rate': 0.00017478161792631978, 'epoch': 0.44}

 44%|████▍     | 1199/2733 [4:28:48<5:42:39, 13.40s/it]
 44%|████▍     | 1200/2733 [4:29:02<5:46:11, 13.55s/it]
                                                       
{'loss': 1.8172, 'learning_rate': 0.0001746676794530953, 'epoch': 0.44}

 44%|████▍     | 1200/2733 [4:29:02<5:46:11, 13.55s/it]
 44%|████▍     | 1201/2733 [4:29:15<5:45:38, 13.54s/it]
                                                       
{'loss': 1.8764, 'learning_rate': 0.00017455374097987087, 'epoch': 0.44}

 44%|████▍     | 1201/2733 [4:29:15<5:45:38, 13.54s/it]
 44%|████▍     | 1202/2733 [4:29:28<5:43:22, 13.46s/it]
                                                       
{'loss': 1.7253, 'learning_rate': 0.0001744398025066464, 'epoch': 0.44}

 44%|████▍     | 1202/2733 [4:29:28<5:43:22, 13.46s/it]
 44%|████▍     | 1203/2733 [4:29:42<5:44:53, 13.52s/it]
                                                       
{'loss': 1.7529, 'learning_rate': 0.00017432586403342195, 'epoch': 0.44}

 44%|████▍     | 1203/2733 [4:29:42<5:44:53, 13.52s/it]
 44%|████▍     | 1204/2733 [4:29:56<5:45:24, 13.55s/it]
                                                       
{'loss': 1.8036, 'learning_rate': 0.00017421192556019748, 'epoch': 0.44}

 44%|████▍     | 1204/2733 [4:29:56<5:45:24, 13.55s/it]
 44%|████▍     | 1205/2733 [4:30:09<5:41:48, 13.42s/it]
                                                       
{'loss': 1.6625, 'learning_rate': 0.00017409798708697304, 'epoch': 0.44}

 44%|████▍     | 1205/2733 [4:30:09<5:41:48, 13.42s/it]
 44%|████▍     | 1206/2733 [4:30:22<5:43:05, 13.48s/it]
                                                       
{'loss': 1.64, 'learning_rate': 0.00017398404861374857, 'epoch': 0.44}

 44%|████▍     | 1206/2733 [4:30:23<5:43:05, 13.48s/it]
 44%|████▍     | 1207/2733 [4:30:36<5:39:18, 13.34s/it]
                                                       
{'loss': 1.5685, 'learning_rate': 0.00017387011014052408, 'epoch': 0.44}

 44%|████▍     | 1207/2733 [4:30:36<5:39:18, 13.34s/it]
 44%|████▍     | 1208/2733 [4:30:48<5:36:21, 13.23s/it]
                                                       
{'loss': 1.5932, 'learning_rate': 0.00017375617166729966, 'epoch': 0.44}

 44%|████▍     | 1208/2733 [4:30:49<5:36:21, 13.23s/it]
 44%|████▍     | 1209/2733 [4:31:02<5:38:38, 13.33s/it]
                                                       
{'loss': 1.6394, 'learning_rate': 0.00017364223319407516, 'epoch': 0.44}

 44%|████▍     | 1209/2733 [4:31:02<5:38:38, 13.33s/it]
 44%|████▍     | 1210/2733 [4:31:15<5:36:44, 13.27s/it]
                                                       
{'loss': 1.7264, 'learning_rate': 0.00017352829472085075, 'epoch': 0.44}

 44%|████▍     | 1210/2733 [4:31:15<5:36:44, 13.27s/it]
 44%|████▍     | 1211/2733 [4:31:29<5:38:55, 13.36s/it]
                                                       
{'loss': 1.8288, 'learning_rate': 0.00017341435624762625, 'epoch': 0.44}

 44%|████▍     | 1211/2733 [4:31:29<5:38:55, 13.36s/it]
 44%|████▍     | 1212/2733 [4:31:42<5:39:34, 13.40s/it]
                                                       
{'loss': 1.6841, 'learning_rate': 0.00017330041777440184, 'epoch': 0.44}

 44%|████▍     | 1212/2733 [4:31:42<5:39:34, 13.40s/it]
 44%|████▍     | 1213/2733 [4:31:55<5:34:29, 13.20s/it]
                                                       
{'loss': 1.6633, 'learning_rate': 0.00017318647930117734, 'epoch': 0.44}

 44%|████▍     | 1213/2733 [4:31:55<5:34:29, 13.20s/it]
 44%|████▍     | 1214/2733 [4:32:09<5:37:40, 13.34s/it]
                                                       
{'loss': 1.7661, 'learning_rate': 0.00017307254082795287, 'epoch': 0.44}

 44%|████▍     | 1214/2733 [4:32:09<5:37:40, 13.34s/it]
 44%|████▍     | 1215/2733 [4:32:22<5:39:00, 13.40s/it]
                                                       
{'loss': 1.7237, 'learning_rate': 0.00017295860235472843, 'epoch': 0.44}

 44%|████▍     | 1215/2733 [4:32:22<5:39:00, 13.40s/it]
 44%|████▍     | 1216/2733 [4:32:36<5:39:28, 13.43s/it]
                                                       
{'loss': 1.5758, 'learning_rate': 0.00017284466388150396, 'epoch': 0.44}

 44%|████▍     | 1216/2733 [4:32:36<5:39:28, 13.43s/it]
 45%|████▍     | 1217/2733 [4:32:49<5:36:01, 13.30s/it]
                                                       
{'loss': 1.6491, 'learning_rate': 0.00017273072540827951, 'epoch': 0.45}

 45%|████▍     | 1217/2733 [4:32:49<5:36:01, 13.30s/it]
 45%|████▍     | 1218/2733 [4:33:02<5:36:02, 13.31s/it]
                                                       
{'loss': 1.6214, 'learning_rate': 0.00017261678693505504, 'epoch': 0.45}

 45%|████▍     | 1218/2733 [4:33:02<5:36:02, 13.31s/it]
 45%|████▍     | 1219/2733 [4:33:15<5:36:46, 13.35s/it]
                                                       
{'loss': 1.6552, 'learning_rate': 0.0001725028484618306, 'epoch': 0.45}

 45%|████▍     | 1219/2733 [4:33:15<5:36:46, 13.35s/it]
 45%|████▍     | 1220/2733 [4:33:28<5:31:07, 13.13s/it]
                                                       
{'loss': 1.6486, 'learning_rate': 0.00017238890998860613, 'epoch': 0.45}

 45%|████▍     | 1220/2733 [4:33:28<5:31:07, 13.13s/it]
 45%|████▍     | 1221/2733 [4:33:42<5:34:32, 13.28s/it]
                                                       
{'loss': 1.6827, 'learning_rate': 0.00017227497151538166, 'epoch': 0.45}

 45%|████▍     | 1221/2733 [4:33:42<5:34:32, 13.28s/it]
 45%|████▍     | 1222/2733 [4:33:55<5:35:36, 13.33s/it]
                                                       
{'loss': 1.6699, 'learning_rate': 0.00017216103304215722, 'epoch': 0.45}

 45%|████▍     | 1222/2733 [4:33:55<5:35:36, 13.33s/it]
 45%|████▍     | 1223/2733 [4:34:08<5:31:33, 13.17s/it]
                                                       
{'loss': 1.7866, 'learning_rate': 0.00017204709456893275, 'epoch': 0.45}

 45%|████▍     | 1223/2733 [4:34:08<5:31:33, 13.17s/it]
 45%|████▍     | 1224/2733 [4:34:22<5:35:07, 13.32s/it]
                                                       
{'loss': 1.6904, 'learning_rate': 0.0001719331560957083, 'epoch': 0.45}

 45%|████▍     | 1224/2733 [4:34:22<5:35:07, 13.32s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 45%|████▍     | 1225/2733 [4:34:34<5:29:04, 13.09s/it]
                                                       
{'loss': 1.6883, 'learning_rate': 0.00017181921762248384, 'epoch': 0.45}

 45%|████▍     | 1225/2733 [4:34:34<5:29:04, 13.09s/it]
 45%|████▍     | 1226/2733 [4:34:47<5:26:07, 12.98s/it]
                                                       
{'loss': 1.6699, 'learning_rate': 0.0001717052791492594, 'epoch': 0.45}

 45%|████▍     | 1226/2733 [4:34:47<5:26:07, 12.98s/it]
 45%|████▍     | 1227/2733 [4:35:01<5:31:57, 13.23s/it]
                                                       
{'loss': 1.745, 'learning_rate': 0.00017159134067603493, 'epoch': 0.45}

 45%|████▍     | 1227/2733 [4:35:01<5:31:57, 13.23s/it]
 45%|████▍     | 1228/2733 [4:35:14<5:34:23, 13.33s/it]
                                                       
{'loss': 1.7072, 'learning_rate': 0.00017147740220281046, 'epoch': 0.45}

 45%|████▍     | 1228/2733 [4:35:14<5:34:23, 13.33s/it]
 45%|████▍     | 1229/2733 [4:35:28<5:34:15, 13.33s/it]
                                                       
{'loss': 1.6401, 'learning_rate': 0.00017136346372958601, 'epoch': 0.45}

 45%|████▍     | 1229/2733 [4:35:28<5:34:15, 13.33s/it]
 45%|████▌     | 1230/2733 [4:35:42<5:40:43, 13.60s/it]
                                                       
{'loss': 1.7756, 'learning_rate': 0.00017124952525636154, 'epoch': 0.45}

 45%|████▌     | 1230/2733 [4:35:42<5:40:43, 13.60s/it]
 45%|████▌     | 1231/2733 [4:35:55<5:38:45, 13.53s/it]
                                                       
{'loss': 1.7804, 'learning_rate': 0.0001711355867831371, 'epoch': 0.45}

 45%|████▌     | 1231/2733 [4:35:55<5:38:45, 13.53s/it]
 45%|████▌     | 1232/2733 [4:36:09<5:38:02, 13.51s/it]
                                                       
{'loss': 1.6441, 'learning_rate': 0.00017102164830991263, 'epoch': 0.45}

 45%|████▌     | 1232/2733 [4:36:09<5:38:02, 13.51s/it]
 45%|████▌     | 1233/2733 [4:36:23<5:41:43, 13.67s/it]
                                                       
{'loss': 1.6821, 'learning_rate': 0.0001709077098366882, 'epoch': 0.45}

 45%|████▌     | 1233/2733 [4:36:23<5:41:43, 13.67s/it]
 45%|████▌     | 1234/2733 [4:36:36<5:38:47, 13.56s/it]
                                                       
{'loss': 1.7627, 'learning_rate': 0.00017079377136346372, 'epoch': 0.45}

 45%|████▌     | 1234/2733 [4:36:36<5:38:47, 13.56s/it]
 45%|████▌     | 1235/2733 [4:36:49<5:36:24, 13.47s/it]
                                                       
{'loss': 1.6385, 'learning_rate': 0.00017067983289023925, 'epoch': 0.45}

 45%|████▌     | 1235/2733 [4:36:49<5:36:24, 13.47s/it]
 45%|████▌     | 1236/2733 [4:37:03<5:41:39, 13.69s/it]
                                                       
{'loss': 1.7415, 'learning_rate': 0.0001705658944170148, 'epoch': 0.45}

 45%|████▌     | 1236/2733 [4:37:04<5:41:39, 13.69s/it]
 45%|████▌     | 1237/2733 [4:37:17<5:39:02, 13.60s/it]
                                                       
{'loss': 1.6119, 'learning_rate': 0.00017045195594379034, 'epoch': 0.45}

 45%|████▌     | 1237/2733 [4:37:17<5:39:02, 13.60s/it]
 45%|████▌     | 1238/2733 [4:37:31<5:42:44, 13.76s/it]
                                                       
{'loss': 1.6669, 'learning_rate': 0.0001703380174705659, 'epoch': 0.45}

 45%|████▌     | 1238/2733 [4:37:31<5:42:44, 13.76s/it]
 45%|████▌     | 1239/2733 [4:37:45<5:44:53, 13.85s/it]
                                                       
{'loss': 1.5946, 'learning_rate': 0.00017022407899734143, 'epoch': 0.45}

 45%|████▌     | 1239/2733 [4:37:45<5:44:53, 13.85s/it]
 45%|████▌     | 1240/2733 [4:37:59<5:43:10, 13.79s/it]
                                                       
{'loss': 1.7243, 'learning_rate': 0.00017011014052411698, 'epoch': 0.45}

 45%|████▌     | 1240/2733 [4:37:59<5:43:10, 13.79s/it]
 45%|████▌     | 1241/2733 [4:38:12<5:36:01, 13.51s/it]
                                                       
{'loss': 1.6021, 'learning_rate': 0.00016999620205089251, 'epoch': 0.45}

 45%|████▌     | 1241/2733 [4:38:12<5:36:01, 13.51s/it]
 45%|████▌     | 1242/2733 [4:38:24<5:30:46, 13.31s/it]
                                                       
{'loss': 1.5712, 'learning_rate': 0.00016988226357766804, 'epoch': 0.45}

 45%|████▌     | 1242/2733 [4:38:24<5:30:46, 13.31s/it]
 45%|████▌     | 1243/2733 [4:38:37<5:26:53, 13.16s/it]
                                                       
{'loss': 1.6813, 'learning_rate': 0.0001697683251044436, 'epoch': 0.45}

 45%|████▌     | 1243/2733 [4:38:37<5:26:53, 13.16s/it]
 46%|████▌     | 1244/2733 [4:38:50<5:25:11, 13.10s/it]
                                                       
{'loss': 1.6627, 'learning_rate': 0.00016965438663121913, 'epoch': 0.46}

 46%|████▌     | 1244/2733 [4:38:50<5:25:11, 13.10s/it]
 46%|████▌     | 1245/2733 [4:39:04<5:26:34, 13.17s/it]
                                                       
{'loss': 1.7269, 'learning_rate': 0.0001695404481579947, 'epoch': 0.46}

 46%|████▌     | 1245/2733 [4:39:04<5:26:34, 13.17s/it]
 46%|████▌     | 1246/2733 [4:39:18<5:34:27, 13.50s/it]
                                                       
{'loss': 1.6076, 'learning_rate': 0.00016942650968477022, 'epoch': 0.46}

 46%|████▌     | 1246/2733 [4:39:18<5:34:27, 13.50s/it]
 46%|████▌     | 1247/2733 [4:39:32<5:36:31, 13.59s/it]
                                                       
{'loss': 1.7235, 'learning_rate': 0.00016931257121154578, 'epoch': 0.46}

 46%|████▌     | 1247/2733 [4:39:32<5:36:31, 13.59s/it]
 46%|████▌     | 1248/2733 [4:39:44<5:28:40, 13.28s/it]
                                                       
{'loss': 1.6926, 'learning_rate': 0.00016919863273832128, 'epoch': 0.46}

 46%|████▌     | 1248/2733 [4:39:44<5:28:40, 13.28s/it]
 46%|████▌     | 1249/2733 [4:39:57<5:26:40, 13.21s/it]
                                                       
{'loss': 1.6903, 'learning_rate': 0.0001690846942650968, 'epoch': 0.46}

 46%|████▌     | 1249/2733 [4:39:57<5:26:40, 13.21s/it]
 46%|████▌     | 1250/2733 [4:40:11<5:33:12, 13.48s/it]
                                                       
{'loss': 1.6862, 'learning_rate': 0.00016897075579187237, 'epoch': 0.46}

 46%|████▌     | 1250/2733 [4:40:11<5:33:12, 13.48s/it]
 46%|████▌     | 1251/2733 [4:40:25<5:32:00, 13.44s/it]
                                                       
{'loss': 1.7143, 'learning_rate': 0.0001688568173186479, 'epoch': 0.46}

 46%|████▌     | 1251/2733 [4:40:25<5:32:00, 13.44s/it]
 46%|████▌     | 1252/2733 [4:40:37<5:21:50, 13.04s/it]
                                                       
{'loss': 1.6255, 'learning_rate': 0.00016874287884542346, 'epoch': 0.46}

 46%|████▌     | 1252/2733 [4:40:37<5:21:50, 13.04s/it]
 46%|████▌     | 1253/2733 [4:40:50<5:22:36, 13.08s/it]
                                                       
{'loss': 1.6813, 'learning_rate': 0.00016862894037219899, 'epoch': 0.46}

 46%|████▌     | 1253/2733 [4:40:50<5:22:36, 13.08s/it]
 46%|████▌     | 1254/2733 [4:41:03<5:25:31, 13.21s/it]
                                                       
{'loss': 1.6405, 'learning_rate': 0.00016851500189897454, 'epoch': 0.46}

 46%|████▌     | 1254/2733 [4:41:03<5:25:31, 13.21s/it]
 46%|████▌     | 1255/2733 [4:41:17<5:26:23, 13.25s/it]
                                                       
{'loss': 1.6658, 'learning_rate': 0.00016840106342575007, 'epoch': 0.46}

 46%|████▌     | 1255/2733 [4:41:17<5:26:23, 13.25s/it]
 46%|████▌     | 1256/2733 [4:41:30<5:27:34, 13.31s/it]
                                                       
{'loss': 1.6507, 'learning_rate': 0.0001682871249525256, 'epoch': 0.46}

 46%|████▌     | 1256/2733 [4:41:30<5:27:34, 13.31s/it]
 46%|████▌     | 1257/2733 [4:41:44<5:29:21, 13.39s/it]
                                                       
{'loss': 1.6403, 'learning_rate': 0.00016817318647930116, 'epoch': 0.46}

 46%|████▌     | 1257/2733 [4:41:44<5:29:21, 13.39s/it]
 46%|████▌     | 1258/2733 [4:41:57<5:26:17, 13.27s/it]
                                                       
{'loss': 1.7554, 'learning_rate': 0.0001680592480060767, 'epoch': 0.46}

 46%|████▌     | 1258/2733 [4:41:57<5:26:17, 13.27s/it]
 46%|████▌     | 1259/2733 [4:42:11<5:29:38, 13.42s/it]
                                                       
{'loss': 1.6943, 'learning_rate': 0.00016794530953285225, 'epoch': 0.46}

 46%|████▌     | 1259/2733 [4:42:11<5:29:38, 13.42s/it]
 46%|████▌     | 1260/2733 [4:42:24<5:27:43, 13.35s/it]
                                                       
{'loss': 1.71, 'learning_rate': 0.00016783137105962778, 'epoch': 0.46}

 46%|████▌     | 1260/2733 [4:42:24<5:27:43, 13.35s/it]
 46%|████▌     | 1261/2733 [4:42:38<5:30:54, 13.49s/it]
                                                       
{'loss': 1.687, 'learning_rate': 0.00016771743258640334, 'epoch': 0.46}

 46%|████▌     | 1261/2733 [4:42:38<5:30:54, 13.49s/it]
 46%|████▌     | 1262/2733 [4:42:52<5:36:11, 13.71s/it]
                                                       
{'loss': 1.7507, 'learning_rate': 0.00016760349411317887, 'epoch': 0.46}

 46%|████▌     | 1262/2733 [4:42:52<5:36:11, 13.71s/it]
 46%|████▌     | 1263/2733 [4:43:05<5:32:01, 13.55s/it]
                                                       
{'loss': 1.7387, 'learning_rate': 0.0001674895556399544, 'epoch': 0.46}

 46%|████▌     | 1263/2733 [4:43:05<5:32:01, 13.55s/it]
 46%|████▌     | 1264/2733 [4:43:19<5:33:53, 13.64s/it]
                                                       
{'loss': 1.6701, 'learning_rate': 0.00016737561716672996, 'epoch': 0.46}

 46%|████▌     | 1264/2733 [4:43:19<5:33:53, 13.64s/it]
 46%|████▋     | 1265/2733 [4:43:32<5:31:53, 13.57s/it]
                                                       
{'loss': 1.8003, 'learning_rate': 0.00016726167869350549, 'epoch': 0.46}

 46%|████▋     | 1265/2733 [4:43:32<5:31:53, 13.57s/it]
 46%|████▋     | 1266/2733 [4:43:46<5:31:26, 13.56s/it]
                                                       
{'loss': 1.6285, 'learning_rate': 0.00016714774022028104, 'epoch': 0.46}

 46%|████▋     | 1266/2733 [4:43:46<5:31:26, 13.56s/it]
 46%|████▋     | 1267/2733 [4:43:59<5:30:43, 13.54s/it]
                                                       
{'loss': 1.6799, 'learning_rate': 0.00016703380174705657, 'epoch': 0.46}

 46%|████▋     | 1267/2733 [4:43:59<5:30:43, 13.54s/it]
 46%|████▋     | 1268/2733 [4:44:13<5:33:10, 13.65s/it]
                                                       
{'loss': 1.6332, 'learning_rate': 0.0001669198632738321, 'epoch': 0.46}

 46%|████▋     | 1268/2733 [4:44:13<5:33:10, 13.65s/it]
 46%|████▋     | 1269/2733 [4:44:27<5:32:17, 13.62s/it]
                                                       
{'loss': 1.7057, 'learning_rate': 0.00016680592480060766, 'epoch': 0.46}

 46%|████▋     | 1269/2733 [4:44:27<5:32:17, 13.62s/it]
 46%|████▋     | 1270/2733 [4:44:40<5:32:21, 13.63s/it]
                                                       
{'loss': 1.6992, 'learning_rate': 0.0001666919863273832, 'epoch': 0.46}

 46%|████▋     | 1270/2733 [4:44:40<5:32:21, 13.63s/it]
 47%|████▋     | 1271/2733 [4:44:53<5:27:42, 13.45s/it]
                                                       
{'loss': 1.5843, 'learning_rate': 0.00016657804785415875, 'epoch': 0.47}

 47%|████▋     | 1271/2733 [4:44:53<5:27:42, 13.45s/it]
 47%|████▋     | 1272/2733 [4:45:07<5:29:44, 13.54s/it]
                                                       
{'loss': 1.6953, 'learning_rate': 0.00016646410938093428, 'epoch': 0.47}

 47%|████▋     | 1272/2733 [4:45:07<5:29:44, 13.54s/it]
 47%|████▋     | 1273/2733 [4:45:20<5:27:28, 13.46s/it]
                                                       
{'loss': 1.6851, 'learning_rate': 0.00016635017090770984, 'epoch': 0.47}

 47%|████▋     | 1273/2733 [4:45:20<5:27:28, 13.46s/it]
 47%|████▋     | 1274/2733 [4:45:34<5:27:20, 13.46s/it]
                                                       
{'loss': 1.6972, 'learning_rate': 0.00016623623243448537, 'epoch': 0.47}

 47%|████▋     | 1274/2733 [4:45:34<5:27:20, 13.46s/it]
 47%|████▋     | 1275/2733 [4:45:47<5:25:40, 13.40s/it]
                                                       
{'loss': 1.6571, 'learning_rate': 0.0001661222939612609, 'epoch': 0.47}

 47%|████▋     | 1275/2733 [4:45:47<5:25:40, 13.40s/it]
 47%|████▋     | 1276/2733 [4:46:01<5:31:29, 13.65s/it]
                                                       
{'loss': 1.6882, 'learning_rate': 0.00016600835548803646, 'epoch': 0.47}

 47%|████▋     | 1276/2733 [4:46:01<5:31:29, 13.65s/it]
 47%|████▋     | 1277/2733 [4:46:15<5:29:45, 13.59s/it]
                                                       
{'loss': 1.7159, 'learning_rate': 0.00016589441701481199, 'epoch': 0.47}

 47%|████▋     | 1277/2733 [4:46:15<5:29:45, 13.59s/it]
 47%|████▋     | 1278/2733 [4:46:29<5:32:23, 13.71s/it]
                                                       
{'loss': 1.7186, 'learning_rate': 0.00016578047854158754, 'epoch': 0.47}

 47%|████▋     | 1278/2733 [4:46:29<5:32:23, 13.71s/it]
 47%|████▋     | 1279/2733 [4:46:42<5:31:29, 13.68s/it]
                                                       
{'loss': 1.7137, 'learning_rate': 0.00016566654006836307, 'epoch': 0.47}

 47%|████▋     | 1279/2733 [4:46:42<5:31:29, 13.68s/it]
 47%|████▋     | 1280/2733 [4:46:57<5:35:16, 13.85s/it]
                                                       
{'loss': 1.7109, 'learning_rate': 0.00016555260159513863, 'epoch': 0.47}

 47%|████▋     | 1280/2733 [4:46:57<5:35:16, 13.85s/it]
 47%|████▋     | 1281/2733 [4:47:11<5:36:25, 13.90s/it]
                                                       
{'loss': 1.7003, 'learning_rate': 0.00016543866312191416, 'epoch': 0.47}

 47%|████▋     | 1281/2733 [4:47:11<5:36:25, 13.90s/it]
 47%|████▋     | 1282/2733 [4:47:25<5:37:21, 13.95s/it]
                                                       
{'loss': 1.6717, 'learning_rate': 0.00016532472464868966, 'epoch': 0.47}

 47%|████▋     | 1282/2733 [4:47:25<5:37:21, 13.95s/it]
 47%|████▋     | 1283/2733 [4:47:38<5:33:43, 13.81s/it]
                                                       
{'loss': 1.733, 'learning_rate': 0.00016521078617546525, 'epoch': 0.47}

 47%|████▋     | 1283/2733 [4:47:38<5:33:43, 13.81s/it]
 47%|████▋     | 1284/2733 [4:47:52<5:32:34, 13.77s/it]
                                                       
{'loss': 1.6737, 'learning_rate': 0.00016509684770224075, 'epoch': 0.47}

 47%|████▋     | 1284/2733 [4:47:52<5:32:34, 13.77s/it]
 47%|████▋     | 1285/2733 [4:48:05<5:28:22, 13.61s/it]
                                                       
{'loss': 1.7338, 'learning_rate': 0.00016498290922901634, 'epoch': 0.47}

 47%|████▋     | 1285/2733 [4:48:05<5:28:22, 13.61s/it]
 47%|████▋     | 1286/2733 [4:48:19<5:32:32, 13.79s/it]
                                                       
{'loss': 1.7146, 'learning_rate': 0.00016486897075579184, 'epoch': 0.47}

 47%|████▋     | 1286/2733 [4:48:19<5:32:32, 13.79s/it]
 47%|████▋     | 1287/2733 [4:48:32<5:24:35, 13.47s/it]
                                                       
{'loss': 1.6381, 'learning_rate': 0.00016475503228256742, 'epoch': 0.47}

 47%|████▋     | 1287/2733 [4:48:32<5:24:35, 13.47s/it]
 47%|████▋     | 1288/2733 [4:48:45<5:20:24, 13.30s/it]
                                                       
{'loss': 1.5658, 'learning_rate': 0.00016464109380934293, 'epoch': 0.47}

 47%|████▋     | 1288/2733 [4:48:45<5:20:24, 13.30s/it]
 47%|████▋     | 1289/2733 [4:48:58<5:16:01, 13.13s/it]
                                                       
{'loss': 1.716, 'learning_rate': 0.00016452715533611846, 'epoch': 0.47}

 47%|████▋     | 1289/2733 [4:48:58<5:16:01, 13.13s/it]
 47%|████▋     | 1290/2733 [4:49:12<5:20:56, 13.35s/it]
                                                       
{'loss': 1.6318, 'learning_rate': 0.00016441321686289402, 'epoch': 0.47}

 47%|████▋     | 1290/2733 [4:49:12<5:20:56, 13.35s/it]
 47%|████▋     | 1291/2733 [4:49:25<5:18:14, 13.24s/it]
                                                       
{'loss': 1.673, 'learning_rate': 0.00016429927838966955, 'epoch': 0.47}

 47%|████▋     | 1291/2733 [4:49:25<5:18:14, 13.24s/it]
 47%|████▋     | 1292/2733 [4:49:38<5:17:54, 13.24s/it]
                                                       
{'loss': 1.6595, 'learning_rate': 0.0001641853399164451, 'epoch': 0.47}

 47%|████▋     | 1292/2733 [4:49:38<5:17:54, 13.24s/it]
 47%|████▋     | 1293/2733 [4:49:51<5:20:31, 13.35s/it]
                                                       
{'loss': 1.6172, 'learning_rate': 0.00016407140144322063, 'epoch': 0.47}

 47%|████▋     | 1293/2733 [4:49:51<5:20:31, 13.35s/it]
 47%|████▋     | 1294/2733 [4:50:05<5:21:21, 13.40s/it]
                                                       
{'loss': 1.6671, 'learning_rate': 0.0001639574629699962, 'epoch': 0.47}

 47%|████▋     | 1294/2733 [4:50:05<5:21:21, 13.40s/it]
 47%|████▋     | 1295/2733 [4:50:18<5:21:30, 13.41s/it]
                                                       
{'loss': 1.7621, 'learning_rate': 0.00016384352449677172, 'epoch': 0.47}

 47%|████▋     | 1295/2733 [4:50:18<5:21:30, 13.41s/it]
 47%|████▋     | 1296/2733 [4:50:32<5:26:11, 13.62s/it]
                                                       
{'loss': 1.7758, 'learning_rate': 0.00016372958602354725, 'epoch': 0.47}

 47%|████▋     | 1296/2733 [4:50:32<5:26:11, 13.62s/it]
 47%|████▋     | 1297/2733 [4:50:45<5:21:40, 13.44s/it]
                                                       
{'loss': 1.5748, 'learning_rate': 0.0001636156475503228, 'epoch': 0.47}

 47%|████▋     | 1297/2733 [4:50:46<5:21:40, 13.44s/it]
 47%|████▋     | 1298/2733 [4:50:59<5:24:57, 13.59s/it]
                                                       
{'loss': 1.7214, 'learning_rate': 0.00016350170907709834, 'epoch': 0.47}

 47%|████▋     | 1298/2733 [4:50:59<5:24:57, 13.59s/it]
 48%|████▊     | 1299/2733 [4:51:14<5:29:29, 13.79s/it]
                                                       
{'loss': 1.6283, 'learning_rate': 0.0001633877706038739, 'epoch': 0.48}

 48%|████▊     | 1299/2733 [4:51:14<5:29:29, 13.79s/it]
 48%|████▊     | 1300/2733 [4:51:27<5:29:09, 13.78s/it]
                                                       
{'loss': 1.6494, 'learning_rate': 0.00016327383213064943, 'epoch': 0.48}

 48%|████▊     | 1300/2733 [4:51:27<5:29:09, 13.78s/it]
 48%|████▊     | 1301/2733 [4:51:40<5:19:19, 13.38s/it]
                                                       
{'loss': 1.6786, 'learning_rate': 0.00016315989365742498, 'epoch': 0.48}

 48%|████▊     | 1301/2733 [4:51:40<5:19:19, 13.38s/it]
 48%|████▊     | 1302/2733 [4:51:53<5:15:49, 13.24s/it]
                                                       
{'loss': 1.682, 'learning_rate': 0.00016304595518420052, 'epoch': 0.48}

 48%|████▊     | 1302/2733 [4:51:53<5:15:49, 13.24s/it]
 48%|████▊     | 1303/2733 [4:52:07<5:19:38, 13.41s/it]
                                                       
{'loss': 1.771, 'learning_rate': 0.00016293201671097605, 'epoch': 0.48}

 48%|████▊     | 1303/2733 [4:52:07<5:19:38, 13.41s/it]
 48%|████▊     | 1304/2733 [4:52:20<5:17:20, 13.32s/it]
                                                       
{'loss': 1.7238, 'learning_rate': 0.0001628180782377516, 'epoch': 0.48}

 48%|████▊     | 1304/2733 [4:52:20<5:17:20, 13.32s/it]
 48%|████▊     | 1305/2733 [4:52:32<5:12:47, 13.14s/it]
                                                       
{'loss': 1.6745, 'learning_rate': 0.00016270413976452713, 'epoch': 0.48}

 48%|████▊     | 1305/2733 [4:52:32<5:12:47, 13.14s/it]
 48%|████▊     | 1306/2733 [4:52:46<5:17:31, 13.35s/it]
                                                       
{'loss': 1.5712, 'learning_rate': 0.0001625902012913027, 'epoch': 0.48}

 48%|████▊     | 1306/2733 [4:52:46<5:17:31, 13.35s/it]
 48%|████▊     | 1307/2733 [4:53:00<5:21:00, 13.51s/it]
                                                       
{'loss': 1.6971, 'learning_rate': 0.00016247626281807822, 'epoch': 0.48}

 48%|████▊     | 1307/2733 [4:53:00<5:21:00, 13.51s/it]
 48%|████▊     | 1308/2733 [4:53:13<5:19:44, 13.46s/it]
                                                       
{'loss': 1.6912, 'learning_rate': 0.00016236232434485378, 'epoch': 0.48}

 48%|████▊     | 1308/2733 [4:53:14<5:19:44, 13.46s/it]
 48%|████▊     | 1309/2733 [4:53:28<5:24:52, 13.69s/it]
                                                       
{'loss': 1.563, 'learning_rate': 0.0001622483858716293, 'epoch': 0.48}

 48%|████▊     | 1309/2733 [4:53:28<5:24:52, 13.69s/it]
 48%|████▊     | 1310/2733 [4:53:42<5:26:28, 13.77s/it]
                                                       
{'loss': 1.5937, 'learning_rate': 0.00016213444739840484, 'epoch': 0.48}

 48%|████▊     | 1310/2733 [4:53:42<5:26:28, 13.77s/it]
 48%|████▊     | 1311/2733 [4:53:55<5:22:00, 13.59s/it]
                                                       
{'loss': 1.7407, 'learning_rate': 0.0001620205089251804, 'epoch': 0.48}

 48%|████▊     | 1311/2733 [4:53:55<5:22:00, 13.59s/it]
 48%|████▊     | 1312/2733 [4:54:08<5:18:27, 13.45s/it]
                                                       
{'loss': 1.7056, 'learning_rate': 0.00016190657045195593, 'epoch': 0.48}

 48%|████▊     | 1312/2733 [4:54:08<5:18:27, 13.45s/it]
 48%|████▊     | 1313/2733 [4:54:22<5:19:59, 13.52s/it]
                                                       
{'loss': 1.6897, 'learning_rate': 0.00016179263197873148, 'epoch': 0.48}

 48%|████▊     | 1313/2733 [4:54:22<5:19:59, 13.52s/it]
 48%|████▊     | 1314/2733 [4:54:36<5:22:58, 13.66s/it]
                                                       
{'loss': 1.7697, 'learning_rate': 0.00016167869350550701, 'epoch': 0.48}

 48%|████▊     | 1314/2733 [4:54:36<5:22:58, 13.66s/it]
 48%|████▊     | 1315/2733 [4:54:49<5:23:07, 13.67s/it]
                                                       
{'loss': 1.688, 'learning_rate': 0.00016156475503228257, 'epoch': 0.48}

 48%|████▊     | 1315/2733 [4:54:49<5:23:07, 13.67s/it]
 48%|████▊     | 1316/2733 [4:55:03<5:24:50, 13.76s/it]
                                                       
{'loss': 1.7311, 'learning_rate': 0.0001614508165590581, 'epoch': 0.48}

 48%|████▊     | 1316/2733 [4:55:03<5:24:50, 13.76s/it]
 48%|████▊     | 1317/2733 [4:55:16<5:19:29, 13.54s/it]
                                                       
{'loss': 1.6566, 'learning_rate': 0.00016133687808583363, 'epoch': 0.48}

 48%|████▊     | 1317/2733 [4:55:16<5:19:29, 13.54s/it]
 48%|████▊     | 1318/2733 [4:55:29<5:13:28, 13.29s/it]
                                                       
{'loss': 1.6025, 'learning_rate': 0.0001612229396126092, 'epoch': 0.48}

 48%|████▊     | 1318/2733 [4:55:29<5:13:28, 13.29s/it]
 48%|████▊     | 1319/2733 [4:55:43<5:20:13, 13.59s/it]
                                                       
{'loss': 1.7297, 'learning_rate': 0.00016110900113938472, 'epoch': 0.48}

 48%|████▊     | 1319/2733 [4:55:43<5:20:13, 13.59s/it]
 48%|████▊     | 1320/2733 [4:55:56<5:14:58, 13.37s/it]
                                                       
{'loss': 1.6549, 'learning_rate': 0.00016099506266616028, 'epoch': 0.48}

 48%|████▊     | 1320/2733 [4:55:56<5:14:58, 13.37s/it]
 48%|████▊     | 1321/2733 [4:56:09<5:11:05, 13.22s/it]
                                                       
{'loss': 1.558, 'learning_rate': 0.0001608811241929358, 'epoch': 0.48}

 48%|████▊     | 1321/2733 [4:56:09<5:11:05, 13.22s/it]
 48%|████▊     | 1322/2733 [4:56:22<5:08:52, 13.13s/it]
                                                       
{'loss': 1.6736, 'learning_rate': 0.00016076718571971137, 'epoch': 0.48}

 48%|████▊     | 1322/2733 [4:56:22<5:08:52, 13.13s/it]
 48%|████▊     | 1323/2733 [4:56:36<5:12:25, 13.29s/it]
                                                       
{'loss': 1.6874, 'learning_rate': 0.00016065324724648687, 'epoch': 0.48}

 48%|████▊     | 1323/2733 [4:56:36<5:12:25, 13.29s/it]
 48%|████▊     | 1324/2733 [4:56:49<5:14:22, 13.39s/it]
                                                       
{'loss': 1.6726, 'learning_rate': 0.0001605393087732624, 'epoch': 0.48}

 48%|████▊     | 1324/2733 [4:56:49<5:14:22, 13.39s/it]
 48%|████▊     | 1325/2733 [4:57:03<5:15:33, 13.45s/it]
                                                       
{'loss': 1.725, 'learning_rate': 0.00016042537030003796, 'epoch': 0.48}

 48%|████▊     | 1325/2733 [4:57:03<5:15:33, 13.45s/it]
 49%|████▊     | 1326/2733 [4:57:16<5:13:17, 13.36s/it]
                                                       
{'loss': 1.7019, 'learning_rate': 0.0001603114318268135, 'epoch': 0.49}

 49%|████▊     | 1326/2733 [4:57:16<5:13:17, 13.36s/it]
 49%|████▊     | 1327/2733 [4:57:29<5:08:29, 13.16s/it]
                                                       
{'loss': 1.6005, 'learning_rate': 0.00016019749335358905, 'epoch': 0.49}

 49%|████▊     | 1327/2733 [4:57:29<5:08:29, 13.16s/it]
 49%|████▊     | 1328/2733 [4:57:41<5:04:15, 12.99s/it]
                                                       
{'loss': 1.6405, 'learning_rate': 0.00016008355488036458, 'epoch': 0.49}

 49%|████▊     | 1328/2733 [4:57:41<5:04:15, 12.99s/it]
 49%|████▊     | 1329/2733 [4:57:55<5:12:30, 13.36s/it]
                                                       
{'loss': 1.6372, 'learning_rate': 0.00015996961640714013, 'epoch': 0.49}

 49%|████▊     | 1329/2733 [4:57:56<5:12:30, 13.36s/it]
 49%|████▊     | 1330/2733 [4:58:09<5:10:53, 13.30s/it]
                                                       
{'loss': 1.6116, 'learning_rate': 0.00015985567793391566, 'epoch': 0.49}

 49%|████▊     | 1330/2733 [4:58:09<5:10:53, 13.30s/it]
 49%|████▊     | 1331/2733 [4:58:22<5:12:15, 13.36s/it]
                                                       
{'loss': 1.7209, 'learning_rate': 0.0001597417394606912, 'epoch': 0.49}

 49%|████▊     | 1331/2733 [4:58:22<5:12:15, 13.36s/it]
 49%|████▊     | 1332/2733 [4:58:35<5:06:54, 13.14s/it]
                                                       
{'loss': 1.6762, 'learning_rate': 0.00015962780098746675, 'epoch': 0.49}

 49%|████▊     | 1332/2733 [4:58:35<5:06:54, 13.14s/it]
 49%|████▉     | 1333/2733 [4:58:48<5:08:36, 13.23s/it]
                                                       
{'loss': 1.7543, 'learning_rate': 0.00015951386251424228, 'epoch': 0.49}

 49%|████▉     | 1333/2733 [4:58:48<5:08:36, 13.23s/it]
 49%|████▉     | 1334/2733 [4:59:02<5:11:32, 13.36s/it]
                                                       
{'loss': 1.6849, 'learning_rate': 0.00015939992404101784, 'epoch': 0.49}

 49%|████▉     | 1334/2733 [4:59:02<5:11:32, 13.36s/it]
 49%|████▉     | 1335/2733 [4:59:15<5:12:40, 13.42s/it]
                                                       
{'loss': 1.615, 'learning_rate': 0.00015928598556779337, 'epoch': 0.49}

 49%|████▉     | 1335/2733 [4:59:15<5:12:40, 13.42s/it]
 49%|████▉     | 1336/2733 [4:59:29<5:11:16, 13.37s/it]
                                                       
{'loss': 1.739, 'learning_rate': 0.00015917204709456893, 'epoch': 0.49}

 49%|████▉     | 1336/2733 [4:59:29<5:11:16, 13.37s/it]
 49%|████▉     | 1337/2733 [4:59:42<5:12:42, 13.44s/it]
                                                       
{'loss': 1.7029, 'learning_rate': 0.00015905810862134446, 'epoch': 0.49}

 49%|████▉     | 1337/2733 [4:59:42<5:12:42, 13.44s/it]
 49%|████▉     | 1338/2733 [4:59:55<5:09:31, 13.31s/it]
                                                       
{'loss': 1.7298, 'learning_rate': 0.00015894417014812, 'epoch': 0.49}

 49%|████▉     | 1338/2733 [4:59:55<5:09:31, 13.31s/it]
 49%|████▉     | 1339/2733 [5:00:09<5:10:23, 13.36s/it]
                                                       
{'loss': 1.7889, 'learning_rate': 0.00015883023167489554, 'epoch': 0.49}

 49%|████▉     | 1339/2733 [5:00:09<5:10:23, 13.36s/it]
 49%|████▉     | 1340/2733 [5:00:23<5:15:22, 13.58s/it]
                                                       
{'loss': 1.6758, 'learning_rate': 0.00015871629320167108, 'epoch': 0.49}

 49%|████▉     | 1340/2733 [5:00:23<5:15:22, 13.58s/it]
 49%|████▉     | 1341/2733 [5:00:37<5:19:36, 13.78s/it]
                                                       
{'loss': 1.7675, 'learning_rate': 0.00015860235472844663, 'epoch': 0.49}

 49%|████▉     | 1341/2733 [5:00:37<5:19:36, 13.78s/it]
 49%|████▉     | 1342/2733 [5:00:50<5:15:47, 13.62s/it]
                                                       
{'loss': 1.6066, 'learning_rate': 0.00015848841625522216, 'epoch': 0.49}

 49%|████▉     | 1342/2733 [5:00:50<5:15:47, 13.62s/it]
 49%|████▉     | 1343/2733 [5:01:04<5:13:07, 13.52s/it]
                                                       
{'loss': 1.6327, 'learning_rate': 0.00015837447778199772, 'epoch': 0.49}

 49%|████▉     | 1343/2733 [5:01:04<5:13:07, 13.52s/it]
 49%|████▉     | 1344/2733 [5:01:17<5:12:17, 13.49s/it]
                                                       
{'loss': 1.702, 'learning_rate': 0.00015826053930877325, 'epoch': 0.49}

 49%|████▉     | 1344/2733 [5:01:17<5:12:17, 13.49s/it]
 49%|████▉     | 1345/2733 [5:01:30<5:06:49, 13.26s/it]
                                                       
{'loss': 1.7193, 'learning_rate': 0.00015814660083554878, 'epoch': 0.49}

 49%|████▉     | 1345/2733 [5:01:30<5:06:49, 13.26s/it]
 49%|████▉     | 1346/2733 [5:01:43<5:04:06, 13.16s/it]
                                                       
{'loss': 1.7391, 'learning_rate': 0.00015803266236232434, 'epoch': 0.49}

 49%|████▉     | 1346/2733 [5:01:43<5:04:06, 13.16s/it]
 49%|████▉     | 1347/2733 [5:01:57<5:11:26, 13.48s/it]
                                                       
{'loss': 1.6622, 'learning_rate': 0.00015791872388909987, 'epoch': 0.49}

 49%|████▉     | 1347/2733 [5:01:57<5:11:26, 13.48s/it]
 49%|████▉     | 1348/2733 [5:02:11<5:12:04, 13.52s/it]
                                                       
{'loss': 1.7693, 'learning_rate': 0.00015780478541587543, 'epoch': 0.49}

 49%|████▉     | 1348/2733 [5:02:11<5:12:04, 13.52s/it]
 49%|████▉     | 1349/2733 [5:02:25<5:16:55, 13.74s/it]
                                                       
{'loss': 1.6655, 'learning_rate': 0.00015769084694265096, 'epoch': 0.49}

 49%|████▉     | 1349/2733 [5:02:25<5:16:55, 13.74s/it]
 49%|████▉     | 1350/2733 [5:02:38<5:16:12, 13.72s/it]
                                                       
{'loss': 1.6919, 'learning_rate': 0.00015757690846942651, 'epoch': 0.49}

 49%|████▉     | 1350/2733 [5:02:39<5:16:12, 13.72s/it]
 49%|████▉     | 1351/2733 [5:02:52<5:17:32, 13.79s/it]
                                                       
{'loss': 1.6522, 'learning_rate': 0.00015746296999620204, 'epoch': 0.49}

 49%|████▉     | 1351/2733 [5:02:52<5:17:32, 13.79s/it]
 49%|████▉     | 1352/2733 [5:03:06<5:16:16, 13.74s/it]
                                                       
{'loss': 1.5505, 'learning_rate': 0.00015734903152297757, 'epoch': 0.49}

 49%|████▉     | 1352/2733 [5:03:06<5:16:16, 13.74s/it]
 50%|████▉     | 1353/2733 [5:03:20<5:16:18, 13.75s/it]
                                                       
{'loss': 1.7082, 'learning_rate': 0.00015723509304975313, 'epoch': 0.5}

 50%|████▉     | 1353/2733 [5:03:20<5:16:18, 13.75s/it]
 50%|████▉     | 1354/2733 [5:03:34<5:17:36, 13.82s/it]
                                                       
{'loss': 1.6706, 'learning_rate': 0.00015712115457652866, 'epoch': 0.5}

 50%|████▉     | 1354/2733 [5:03:34<5:17:36, 13.82s/it]
 50%|████▉     | 1355/2733 [5:03:48<5:18:17, 13.86s/it]
                                                       
{'loss': 1.7388, 'learning_rate': 0.00015700721610330422, 'epoch': 0.5}

 50%|████▉     | 1355/2733 [5:03:48<5:18:17, 13.86s/it]
 50%|████▉     | 1356/2733 [5:04:00<5:09:55, 13.50s/it]
                                                       
{'loss': 1.6923, 'learning_rate': 0.00015689327763007975, 'epoch': 0.5}

 50%|████▉     | 1356/2733 [5:04:00<5:09:55, 13.50s/it]
 50%|████▉     | 1357/2733 [5:04:14<5:06:39, 13.37s/it]
                                                       
{'loss': 1.6767, 'learning_rate': 0.0001567793391568553, 'epoch': 0.5}

 50%|████▉     | 1357/2733 [5:04:14<5:06:39, 13.37s/it]
 50%|████▉     | 1358/2733 [5:04:27<5:07:23, 13.41s/it]
                                                       
{'loss': 1.7602, 'learning_rate': 0.00015666540068363084, 'epoch': 0.5}

 50%|████▉     | 1358/2733 [5:04:27<5:07:23, 13.41s/it]
 50%|████▉     | 1359/2733 [5:04:41<5:11:01, 13.58s/it]
                                                       
{'loss': 1.6881, 'learning_rate': 0.00015655146221040634, 'epoch': 0.5}

 50%|████▉     | 1359/2733 [5:04:41<5:11:01, 13.58s/it]
 50%|████▉     | 1360/2733 [5:04:54<5:05:02, 13.33s/it]
                                                       
{'loss': 1.6714, 'learning_rate': 0.00015643752373718193, 'epoch': 0.5}

 50%|████▉     | 1360/2733 [5:04:54<5:05:02, 13.33s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 50%|████▉     | 1361/2733 [5:05:07<5:03:57, 13.29s/it]
                                                       
{'loss': 1.6347, 'learning_rate': 0.00015632358526395743, 'epoch': 0.5}

 50%|████▉     | 1361/2733 [5:05:07<5:03:57, 13.29s/it]
 50%|████▉     | 1362/2733 [5:05:21<5:09:52, 13.56s/it]
                                                       
{'loss': 1.7012, 'learning_rate': 0.00015620964679073301, 'epoch': 0.5}

 50%|████▉     | 1362/2733 [5:05:21<5:09:52, 13.56s/it]
 50%|████▉     | 1363/2733 [5:05:35<5:08:36, 13.52s/it]
                                                       
{'loss': 1.5991, 'learning_rate': 0.00015609570831750852, 'epoch': 0.5}

 50%|████▉     | 1363/2733 [5:05:35<5:08:36, 13.52s/it]
 50%|████▉     | 1364/2733 [5:05:48<5:05:11, 13.38s/it]
                                                       
{'loss': 1.6982, 'learning_rate': 0.00015598176984428407, 'epoch': 0.5}

 50%|████▉     | 1364/2733 [5:05:48<5:05:11, 13.38s/it]
 50%|████▉     | 1365/2733 [5:06:01<5:07:11, 13.47s/it]
                                                       
{'loss': 1.7146, 'learning_rate': 0.0001558678313710596, 'epoch': 0.5}

 50%|████▉     | 1365/2733 [5:06:01<5:07:11, 13.47s/it]
 50%|████▉     | 1366/2733 [5:06:15<5:10:39, 13.64s/it]
                                                       
{'loss': 1.6819, 'learning_rate': 0.00015575389289783514, 'epoch': 0.5}

 50%|████▉     | 1366/2733 [5:06:15<5:10:39, 13.64s/it]
 50%|█████     | 1367/2733 [5:06:28<5:04:08, 13.36s/it]
                                                       
{'loss': 1.6567, 'learning_rate': 0.0001556399544246107, 'epoch': 0.5}

 50%|█████     | 1367/2733 [5:06:28<5:04:08, 13.36s/it]
 50%|█████     | 1368/2733 [5:06:41<5:00:44, 13.22s/it]
                                                       
{'loss': 1.742, 'learning_rate': 0.00015552601595138622, 'epoch': 0.5}

 50%|█████     | 1368/2733 [5:06:41<5:00:44, 13.22s/it]
 50%|█████     | 1369/2733 [5:06:54<4:59:57, 13.19s/it]
                                                       
{'loss': 1.8275, 'learning_rate': 0.00015541207747816178, 'epoch': 0.5}

 50%|█████     | 1369/2733 [5:06:54<4:59:57, 13.19s/it]
 50%|█████     | 1370/2733 [5:07:08<5:06:55, 13.51s/it]
                                                       
{'loss': 1.7218, 'learning_rate': 0.0001552981390049373, 'epoch': 0.5}

 50%|█████     | 1370/2733 [5:07:08<5:06:55, 13.51s/it]
 50%|█████     | 1371/2733 [5:07:22<5:06:22, 13.50s/it]
                                                       
{'loss': 1.5844, 'learning_rate': 0.00015518420053171287, 'epoch': 0.5}

 50%|█████     | 1371/2733 [5:07:22<5:06:22, 13.50s/it]
 50%|█████     | 1372/2733 [5:07:35<5:05:51, 13.48s/it]
                                                       
{'loss': 1.6613, 'learning_rate': 0.0001550702620584884, 'epoch': 0.5}

 50%|█████     | 1372/2733 [5:07:35<5:05:51, 13.48s/it]
 50%|█████     | 1373/2733 [5:07:48<5:02:20, 13.34s/it]
                                                       
{'loss': 1.5846, 'learning_rate': 0.00015495632358526393, 'epoch': 0.5}

 50%|█████     | 1373/2733 [5:07:48<5:02:20, 13.34s/it]
 50%|█████     | 1374/2733 [5:08:02<5:06:08, 13.52s/it]
                                                       
{'loss': 1.7516, 'learning_rate': 0.00015484238511203949, 'epoch': 0.5}

 50%|█████     | 1374/2733 [5:08:02<5:06:08, 13.52s/it]
 50%|█████     | 1375/2733 [5:08:16<5:07:42, 13.60s/it]
                                                       
{'loss': 1.8035, 'learning_rate': 0.00015472844663881502, 'epoch': 0.5}

 50%|█████     | 1375/2733 [5:08:16<5:07:42, 13.60s/it]
 50%|█████     | 1376/2733 [5:08:30<5:11:16, 13.76s/it]
                                                       
{'loss': 1.6552, 'learning_rate': 0.00015461450816559057, 'epoch': 0.5}

 50%|█████     | 1376/2733 [5:08:30<5:11:16, 13.76s/it]
 50%|█████     | 1377/2733 [5:08:43<5:06:35, 13.57s/it]
                                                       
{'loss': 1.6243, 'learning_rate': 0.0001545005696923661, 'epoch': 0.5}

 50%|█████     | 1377/2733 [5:08:43<5:06:35, 13.57s/it]
 50%|█████     | 1378/2733 [5:08:56<5:00:36, 13.31s/it]
                                                       
{'loss': 1.5832, 'learning_rate': 0.00015438663121914166, 'epoch': 0.5}

 50%|█████     | 1378/2733 [5:08:56<5:00:36, 13.31s/it]
 50%|█████     | 1379/2733 [5:09:09<5:00:29, 13.32s/it]
                                                       
{'loss': 1.661, 'learning_rate': 0.0001542726927459172, 'epoch': 0.5}

 50%|█████     | 1379/2733 [5:09:09<5:00:29, 13.32s/it]
 50%|█████     | 1380/2733 [5:09:23<5:03:20, 13.45s/it]
                                                       
{'loss': 1.6993, 'learning_rate': 0.00015415875427269272, 'epoch': 0.5}

 50%|█████     | 1380/2733 [5:09:23<5:03:20, 13.45s/it]
 51%|█████     | 1381/2733 [5:09:36<4:59:22, 13.29s/it]
                                                       
{'loss': 1.6669, 'learning_rate': 0.00015404481579946828, 'epoch': 0.51}

 51%|█████     | 1381/2733 [5:09:36<4:59:22, 13.29s/it]
 51%|█████     | 1382/2733 [5:09:50<5:05:12, 13.55s/it]
                                                       
{'loss': 1.7356, 'learning_rate': 0.0001539308773262438, 'epoch': 0.51}

 51%|█████     | 1382/2733 [5:09:50<5:05:12, 13.55s/it]
 51%|█████     | 1383/2733 [5:10:03<5:03:51, 13.50s/it]
                                                       
{'loss': 1.6296, 'learning_rate': 0.00015381693885301937, 'epoch': 0.51}

 51%|█████     | 1383/2733 [5:10:03<5:03:51, 13.50s/it]
 51%|█████     | 1384/2733 [5:10:17<5:01:32, 13.41s/it]
                                                       
{'loss': 1.6702, 'learning_rate': 0.0001537030003797949, 'epoch': 0.51}

 51%|█████     | 1384/2733 [5:10:17<5:01:32, 13.41s/it]
 51%|█████     | 1385/2733 [5:10:31<5:06:46, 13.65s/it]
                                                       
{'loss': 1.7136, 'learning_rate': 0.00015358906190657046, 'epoch': 0.51}

 51%|█████     | 1385/2733 [5:10:31<5:06:46, 13.65s/it]
 51%|█████     | 1386/2733 [5:10:44<5:03:38, 13.53s/it]
                                                       
{'loss': 1.7644, 'learning_rate': 0.00015347512343334599, 'epoch': 0.51}

 51%|█████     | 1386/2733 [5:10:44<5:03:38, 13.53s/it]
 51%|█████     | 1387/2733 [5:10:57<5:00:25, 13.39s/it]
                                                       
{'loss': 1.6549, 'learning_rate': 0.00015336118496012152, 'epoch': 0.51}

 51%|█████     | 1387/2733 [5:10:57<5:00:25, 13.39s/it]
 51%|█████     | 1388/2733 [5:11:10<4:58:05, 13.30s/it]
                                                       
{'loss': 1.6931, 'learning_rate': 0.00015324724648689707, 'epoch': 0.51}

 51%|█████     | 1388/2733 [5:11:10<4:58:05, 13.30s/it]
 51%|█████     | 1389/2733 [5:11:24<4:59:36, 13.38s/it]
                                                       
{'loss': 1.6988, 'learning_rate': 0.0001531333080136726, 'epoch': 0.51}

 51%|█████     | 1389/2733 [5:11:24<4:59:36, 13.38s/it]
 51%|█████     | 1390/2733 [5:11:37<4:55:58, 13.22s/it]
                                                       
{'loss': 1.6477, 'learning_rate': 0.00015301936954044816, 'epoch': 0.51}

 51%|█████     | 1390/2733 [5:11:37<4:55:58, 13.22s/it]
 51%|█████     | 1391/2733 [5:11:50<4:59:07, 13.37s/it]
                                                       
{'loss': 1.6776, 'learning_rate': 0.0001529054310672237, 'epoch': 0.51}

 51%|█████     | 1391/2733 [5:11:50<4:59:07, 13.37s/it]
 51%|█████     | 1392/2733 [5:12:03<4:56:19, 13.26s/it]
                                                       
{'loss': 1.7103, 'learning_rate': 0.00015279149259399925, 'epoch': 0.51}

 51%|█████     | 1392/2733 [5:12:03<4:56:19, 13.26s/it]
 51%|█████     | 1393/2733 [5:12:18<5:02:40, 13.55s/it]
                                                       
{'loss': 1.7184, 'learning_rate': 0.00015267755412077478, 'epoch': 0.51}

 51%|█████     | 1393/2733 [5:12:18<5:02:40, 13.55s/it]
 51%|█████     | 1394/2733 [5:12:31<5:03:20, 13.59s/it]
                                                       
{'loss': 1.6268, 'learning_rate': 0.0001525636156475503, 'epoch': 0.51}

 51%|█████     | 1394/2733 [5:12:31<5:03:20, 13.59s/it]
 51%|█████     | 1395/2733 [5:12:44<4:55:57, 13.27s/it]
                                                       
{'loss': 1.611, 'learning_rate': 0.00015244967717432587, 'epoch': 0.51}

 51%|█████     | 1395/2733 [5:12:44<4:55:57, 13.27s/it]
 51%|█████     | 1396/2733 [5:12:57<4:53:42, 13.18s/it]
                                                       
{'loss': 1.6399, 'learning_rate': 0.00015233573870110137, 'epoch': 0.51}

 51%|█████     | 1396/2733 [5:12:57<4:53:42, 13.18s/it]
 51%|█████     | 1397/2733 [5:13:10<4:55:23, 13.27s/it]
                                                       
{'loss': 1.7393, 'learning_rate': 0.00015222180022787696, 'epoch': 0.51}

 51%|█████     | 1397/2733 [5:13:10<4:55:23, 13.27s/it]
 51%|█████     | 1398/2733 [5:13:24<4:57:26, 13.37s/it]
                                                       
{'loss': 1.6634, 'learning_rate': 0.00015210786175465246, 'epoch': 0.51}

 51%|█████     | 1398/2733 [5:13:24<4:57:26, 13.37s/it]
 51%|█████     | 1399/2733 [5:13:37<4:53:35, 13.20s/it]
                                                       
{'loss': 1.7192, 'learning_rate': 0.00015199392328142804, 'epoch': 0.51}

 51%|█████     | 1399/2733 [5:13:37<4:53:35, 13.20s/it]
 51%|█████     | 1400/2733 [5:13:51<4:57:37, 13.40s/it]
                                                       
{'loss': 1.5686, 'learning_rate': 0.00015187998480820355, 'epoch': 0.51}

 51%|█████     | 1400/2733 [5:13:51<4:57:37, 13.40s/it]
 51%|█████▏    | 1401/2733 [5:14:04<4:56:42, 13.37s/it]
                                                       
{'loss': 1.6801, 'learning_rate': 0.00015176604633497908, 'epoch': 0.51}

 51%|█████▏    | 1401/2733 [5:14:04<4:56:42, 13.37s/it]
 51%|█████▏    | 1402/2733 [5:14:18<5:02:48, 13.65s/it]
                                                       
{'loss': 1.6851, 'learning_rate': 0.00015165210786175463, 'epoch': 0.51}

 51%|█████▏    | 1402/2733 [5:14:18<5:02:48, 13.65s/it]
 51%|█████▏    | 1403/2733 [5:14:31<4:58:55, 13.48s/it]
                                                       
{'loss': 1.6652, 'learning_rate': 0.00015153816938853016, 'epoch': 0.51}

 51%|█████▏    | 1403/2733 [5:14:31<4:58:55, 13.48s/it]
 51%|█████▏    | 1404/2733 [5:14:45<4:57:53, 13.45s/it]
                                                       
{'loss': 1.6779, 'learning_rate': 0.00015142423091530572, 'epoch': 0.51}

 51%|█████▏    | 1404/2733 [5:14:45<4:57:53, 13.45s/it]
 51%|█████▏    | 1405/2733 [5:14:58<4:57:18, 13.43s/it]
                                                       
{'loss': 1.6267, 'learning_rate': 0.00015131029244208125, 'epoch': 0.51}

 51%|█████▏    | 1405/2733 [5:14:58<4:57:18, 13.43s/it]
 51%|█████▏    | 1406/2733 [5:15:12<4:58:14, 13.48s/it]
                                                       
{'loss': 1.7548, 'learning_rate': 0.0001511963539688568, 'epoch': 0.51}

 51%|█████▏    | 1406/2733 [5:15:12<4:58:14, 13.48s/it]
 51%|█████▏    | 1407/2733 [5:15:25<4:58:30, 13.51s/it]
                                                       
{'loss': 1.6893, 'learning_rate': 0.00015108241549563234, 'epoch': 0.51}

 51%|█████▏    | 1407/2733 [5:15:25<4:58:30, 13.51s/it]
 52%|█████▏    | 1408/2733 [5:15:40<5:03:36, 13.75s/it]
                                                       
{'loss': 1.7225, 'learning_rate': 0.00015096847702240787, 'epoch': 0.52}

 52%|█████▏    | 1408/2733 [5:15:40<5:03:36, 13.75s/it]
 52%|█████▏    | 1409/2733 [5:15:53<5:01:09, 13.65s/it]
                                                       
{'loss': 1.6557, 'learning_rate': 0.00015085453854918343, 'epoch': 0.52}

 52%|█████▏    | 1409/2733 [5:15:53<5:01:09, 13.65s/it]
 52%|█████▏    | 1410/2733 [5:16:06<4:58:30, 13.54s/it]
                                                       
{'loss': 1.5614, 'learning_rate': 0.00015074060007595896, 'epoch': 0.52}

 52%|█████▏    | 1410/2733 [5:16:06<4:58:30, 13.54s/it]
 52%|█████▏    | 1411/2733 [5:16:19<4:54:48, 13.38s/it]
                                                       
{'loss': 1.6085, 'learning_rate': 0.00015062666160273452, 'epoch': 0.52}

 52%|█████▏    | 1411/2733 [5:16:19<4:54:48, 13.38s/it]
 52%|█████▏    | 1412/2733 [5:16:33<4:58:30, 13.56s/it]
                                                       
{'loss': 1.6384, 'learning_rate': 0.00015051272312951005, 'epoch': 0.52}

 52%|█████▏    | 1412/2733 [5:16:33<4:58:30, 13.56s/it]
 52%|█████▏    | 1413/2733 [5:16:47<4:57:57, 13.54s/it]
                                                       
{'loss': 1.6393, 'learning_rate': 0.0001503987846562856, 'epoch': 0.52}

 52%|█████▏    | 1413/2733 [5:16:47<4:57:57, 13.54s/it]
 52%|█████▏    | 1414/2733 [5:16:59<4:51:59, 13.28s/it]
                                                       
{'loss': 1.6997, 'learning_rate': 0.00015028484618306113, 'epoch': 0.52}

 52%|█████▏    | 1414/2733 [5:16:59<4:51:59, 13.28s/it]
 52%|█████▏    | 1415/2733 [5:17:13<4:53:05, 13.34s/it]
                                                       
{'loss': 1.6677, 'learning_rate': 0.00015017090770983666, 'epoch': 0.52}

 52%|█████▏    | 1415/2733 [5:17:13<4:53:05, 13.34s/it]
 52%|█████▏    | 1416/2733 [5:17:27<4:56:18, 13.50s/it]
                                                       
{'loss': 1.7175, 'learning_rate': 0.00015005696923661222, 'epoch': 0.52}

 52%|█████▏    | 1416/2733 [5:17:27<4:56:18, 13.50s/it]
 52%|█████▏    | 1417/2733 [5:17:40<4:53:31, 13.38s/it]
                                                       
{'loss': 1.761, 'learning_rate': 0.00014994303076338775, 'epoch': 0.52}

 52%|█████▏    | 1417/2733 [5:17:40<4:53:31, 13.38s/it]
 52%|█████▏    | 1418/2733 [5:17:54<4:56:52, 13.55s/it]
                                                       
{'loss': 1.681, 'learning_rate': 0.00014982909229016328, 'epoch': 0.52}

 52%|█████▏    | 1418/2733 [5:17:54<4:56:52, 13.55s/it]
 52%|█████▏    | 1419/2733 [5:18:08<4:58:22, 13.62s/it]
                                                       
{'loss': 1.7134, 'learning_rate': 0.00014971515381693884, 'epoch': 0.52}

 52%|█████▏    | 1419/2733 [5:18:08<4:58:22, 13.62s/it]
 52%|█████▏    | 1420/2733 [5:18:21<4:56:23, 13.54s/it]
                                                       
{'loss': 1.5693, 'learning_rate': 0.00014960121534371437, 'epoch': 0.52}

 52%|█████▏    | 1420/2733 [5:18:21<4:56:23, 13.54s/it]
 52%|█████▏    | 1421/2733 [5:18:34<4:54:31, 13.47s/it]
                                                       
{'loss': 1.7092, 'learning_rate': 0.00014948727687048993, 'epoch': 0.52}

 52%|█████▏    | 1421/2733 [5:18:34<4:54:31, 13.47s/it]
 52%|█████▏    | 1422/2733 [5:18:48<4:59:16, 13.70s/it]
                                                       
{'loss': 1.6868, 'learning_rate': 0.00014937333839726546, 'epoch': 0.52}

 52%|█████▏    | 1422/2733 [5:18:48<4:59:16, 13.70s/it]
 52%|█████▏    | 1423/2733 [5:19:02<4:55:09, 13.52s/it]
                                                       
{'loss': 1.7222, 'learning_rate': 0.00014925939992404102, 'epoch': 0.52}

 52%|█████▏    | 1423/2733 [5:19:02<4:55:09, 13.52s/it]
 52%|█████▏    | 1424/2733 [5:19:15<4:57:07, 13.62s/it]
                                                       
{'loss': 1.6851, 'learning_rate': 0.00014914546145081655, 'epoch': 0.52}

 52%|█████▏    | 1424/2733 [5:19:15<4:57:07, 13.62s/it]
 52%|█████▏    | 1425/2733 [5:19:30<5:00:37, 13.79s/it]
                                                       
{'loss': 1.6298, 'learning_rate': 0.00014903152297759208, 'epoch': 0.52}

 52%|█████▏    | 1425/2733 [5:19:30<5:00:37, 13.79s/it]
 52%|█████▏    | 1426/2733 [5:19:42<4:53:58, 13.50s/it]
                                                       
{'loss': 1.729, 'learning_rate': 0.00014891758450436763, 'epoch': 0.52}

 52%|█████▏    | 1426/2733 [5:19:42<4:53:58, 13.50s/it]
 52%|█████▏    | 1427/2733 [5:19:56<4:55:57, 13.60s/it]
                                                       
{'loss': 1.718, 'learning_rate': 0.00014880364603114316, 'epoch': 0.52}

 52%|█████▏    | 1427/2733 [5:19:56<4:55:57, 13.60s/it]
 52%|█████▏    | 1428/2733 [5:20:09<4:50:20, 13.35s/it]
                                                       
{'loss': 1.583, 'learning_rate': 0.00014868970755791872, 'epoch': 0.52}

 52%|█████▏    | 1428/2733 [5:20:09<4:50:20, 13.35s/it]
 52%|█████▏    | 1429/2733 [5:20:23<4:52:35, 13.46s/it]
                                                       
{'loss': 1.7564, 'learning_rate': 0.00014857576908469425, 'epoch': 0.52}

 52%|█████▏    | 1429/2733 [5:20:23<4:52:35, 13.46s/it]
 52%|█████▏    | 1430/2733 [5:20:36<4:53:29, 13.51s/it]
                                                       
{'loss': 1.7336, 'learning_rate': 0.0001484618306114698, 'epoch': 0.52}

 52%|█████▏    | 1430/2733 [5:20:36<4:53:29, 13.51s/it]
 52%|█████▏    | 1431/2733 [5:20:50<4:54:45, 13.58s/it]
                                                       
{'loss': 1.706, 'learning_rate': 0.00014834789213824534, 'epoch': 0.52}

 52%|█████▏    | 1431/2733 [5:20:50<4:54:45, 13.58s/it]
 52%|█████▏    | 1432/2733 [5:21:03<4:52:57, 13.51s/it]
                                                       
{'loss': 1.655, 'learning_rate': 0.00014823395366502087, 'epoch': 0.52}

 52%|█████▏    | 1432/2733 [5:21:03<4:52:57, 13.51s/it]
 52%|█████▏    | 1433/2733 [5:21:17<4:53:37, 13.55s/it]
                                                       
{'loss': 1.6878, 'learning_rate': 0.00014812001519179643, 'epoch': 0.52}

 52%|█████▏    | 1433/2733 [5:21:17<4:53:37, 13.55s/it]
 52%|█████▏    | 1434/2733 [5:21:30<4:50:12, 13.40s/it]
                                                       
{'loss': 1.5996, 'learning_rate': 0.00014800607671857196, 'epoch': 0.52}

 52%|█████▏    | 1434/2733 [5:21:30<4:50:12, 13.40s/it]
 53%|█████▎    | 1435/2733 [5:21:43<4:47:30, 13.29s/it]
                                                       
{'loss': 1.6261, 'learning_rate': 0.00014789213824534752, 'epoch': 0.53}

 53%|█████▎    | 1435/2733 [5:21:43<4:47:30, 13.29s/it]
 53%|█████▎    | 1436/2733 [5:21:56<4:46:02, 13.23s/it]
                                                       
{'loss': 1.6263, 'learning_rate': 0.00014777819977212305, 'epoch': 0.53}

 53%|█████▎    | 1436/2733 [5:21:56<4:46:02, 13.23s/it]
 53%|█████▎    | 1437/2733 [5:22:10<4:47:35, 13.31s/it]
                                                       
{'loss': 1.6658, 'learning_rate': 0.0001476642612988986, 'epoch': 0.53}

 53%|█████▎    | 1437/2733 [5:22:10<4:47:35, 13.31s/it]
 53%|█████▎    | 1438/2733 [5:22:23<4:44:48, 13.20s/it]
                                                       
{'loss': 1.6015, 'learning_rate': 0.0001475503228256741, 'epoch': 0.53}

 53%|█████▎    | 1438/2733 [5:22:23<4:44:48, 13.20s/it]
 53%|█████▎    | 1439/2733 [5:22:36<4:47:06, 13.31s/it]
                                                       
{'loss': 1.7759, 'learning_rate': 0.00014743638435244966, 'epoch': 0.53}

 53%|█████▎    | 1439/2733 [5:22:36<4:47:06, 13.31s/it]
 53%|█████▎    | 1440/2733 [5:22:50<4:48:58, 13.41s/it]
                                                       
{'loss': 1.7291, 'learning_rate': 0.0001473224458792252, 'epoch': 0.53}

 53%|█████▎    | 1440/2733 [5:22:50<4:48:58, 13.41s/it]
 53%|█████▎    | 1441/2733 [5:23:04<4:53:59, 13.65s/it]
                                                       
{'loss': 1.7254, 'learning_rate': 0.00014720850740600075, 'epoch': 0.53}

 53%|█████▎    | 1441/2733 [5:23:04<4:53:59, 13.65s/it]
 53%|█████▎    | 1442/2733 [5:23:18<4:53:20, 13.63s/it]
                                                       
{'loss': 1.7291, 'learning_rate': 0.00014709456893277628, 'epoch': 0.53}

 53%|█████▎    | 1442/2733 [5:23:18<4:53:20, 13.63s/it]
 53%|█████▎    | 1443/2733 [5:23:30<4:46:46, 13.34s/it]
                                                       
{'loss': 1.6377, 'learning_rate': 0.00014698063045955184, 'epoch': 0.53}

 53%|█████▎    | 1443/2733 [5:23:30<4:46:46, 13.34s/it]
 53%|█████▎    | 1444/2733 [5:23:44<4:48:19, 13.42s/it]
                                                       
{'loss': 1.6817, 'learning_rate': 0.00014686669198632737, 'epoch': 0.53}

 53%|█████▎    | 1444/2733 [5:23:44<4:48:19, 13.42s/it]
 53%|█████▎    | 1445/2733 [5:23:56<4:42:00, 13.14s/it]
                                                       
{'loss': 1.6341, 'learning_rate': 0.0001467527535131029, 'epoch': 0.53}

 53%|█████▎    | 1445/2733 [5:23:56<4:42:00, 13.14s/it]
 53%|█████▎    | 1446/2733 [5:24:10<4:45:29, 13.31s/it]
                                                       
{'loss': 1.5797, 'learning_rate': 0.00014663881503987846, 'epoch': 0.53}

 53%|█████▎    | 1446/2733 [5:24:10<4:45:29, 13.31s/it]
 53%|█████▎    | 1447/2733 [5:24:23<4:40:17, 13.08s/it]
                                                       
{'loss': 1.6454, 'learning_rate': 0.000146524876566654, 'epoch': 0.53}

 53%|█████▎    | 1447/2733 [5:24:23<4:40:17, 13.08s/it]
 53%|█████▎    | 1448/2733 [5:24:36<4:43:37, 13.24s/it]
                                                       
{'loss': 1.733, 'learning_rate': 0.00014641093809342955, 'epoch': 0.53}

 53%|█████▎    | 1448/2733 [5:24:36<4:43:37, 13.24s/it]
 53%|█████▎    | 1449/2733 [5:24:49<4:41:33, 13.16s/it]
                                                       
{'loss': 1.5954, 'learning_rate': 0.00014629699962020508, 'epoch': 0.53}

 53%|█████▎    | 1449/2733 [5:24:49<4:41:33, 13.16s/it]
 53%|█████▎    | 1450/2733 [5:25:02<4:40:01, 13.10s/it]
                                                       
{'loss': 1.7012, 'learning_rate': 0.00014618306114698063, 'epoch': 0.53}

 53%|█████▎    | 1450/2733 [5:25:02<4:40:01, 13.10s/it]
 53%|█████▎    | 1451/2733 [5:25:17<4:47:10, 13.44s/it]
                                                       
{'loss': 1.6304, 'learning_rate': 0.00014606912267375616, 'epoch': 0.53}

 53%|█████▎    | 1451/2733 [5:25:17<4:47:10, 13.44s/it]
 53%|█████▎    | 1452/2733 [5:25:29<4:42:31, 13.23s/it]
                                                       
{'loss': 1.5907, 'learning_rate': 0.0001459551842005317, 'epoch': 0.53}

 53%|█████▎    | 1452/2733 [5:25:29<4:42:31, 13.23s/it]
 53%|█████▎    | 1453/2733 [5:25:43<4:43:49, 13.30s/it]
                                                       
{'loss': 1.6709, 'learning_rate': 0.00014584124572730725, 'epoch': 0.53}

 53%|█████▎    | 1453/2733 [5:25:43<4:43:49, 13.30s/it]
 53%|█████▎    | 1454/2733 [5:25:57<4:48:16, 13.52s/it]
                                                       
{'loss': 1.7622, 'learning_rate': 0.00014572730725408278, 'epoch': 0.53}

 53%|█████▎    | 1454/2733 [5:25:57<4:48:16, 13.52s/it]
 53%|█████▎    | 1455/2733 [5:26:10<4:44:39, 13.36s/it]
                                                       
{'loss': 1.6515, 'learning_rate': 0.0001456133687808583, 'epoch': 0.53}

 53%|█████▎    | 1455/2733 [5:26:10<4:44:39, 13.36s/it]
 53%|█████▎    | 1456/2733 [5:26:23<4:40:57, 13.20s/it]
                                                       
{'loss': 1.667, 'learning_rate': 0.00014549943030763387, 'epoch': 0.53}

 53%|█████▎    | 1456/2733 [5:26:23<4:40:57, 13.20s/it]
 53%|█████▎    | 1457/2733 [5:26:36<4:44:20, 13.37s/it]
                                                       
{'loss': 1.683, 'learning_rate': 0.0001453854918344094, 'epoch': 0.53}

 53%|█████▎    | 1457/2733 [5:26:36<4:44:20, 13.37s/it]
 53%|█████▎    | 1458/2733 [5:26:51<4:49:56, 13.64s/it]
                                                       
{'loss': 1.6981, 'learning_rate': 0.00014527155336118493, 'epoch': 0.53}

 53%|█████▎    | 1458/2733 [5:26:51<4:49:56, 13.64s/it]
 53%|█████▎    | 1459/2733 [5:27:04<4:48:15, 13.58s/it]
                                                       
{'loss': 1.6768, 'learning_rate': 0.0001451576148879605, 'epoch': 0.53}

 53%|█████▎    | 1459/2733 [5:27:04<4:48:15, 13.58s/it]
 53%|█████▎    | 1460/2733 [5:27:17<4:45:59, 13.48s/it]
                                                       
{'loss': 1.7394, 'learning_rate': 0.00014504367641473602, 'epoch': 0.53}

 53%|█████▎    | 1460/2733 [5:27:17<4:45:59, 13.48s/it]
 53%|█████▎    | 1461/2733 [5:27:32<4:50:26, 13.70s/it]
                                                       
{'loss': 1.6645, 'learning_rate': 0.00014492973794151158, 'epoch': 0.53}

 53%|█████▎    | 1461/2733 [5:27:32<4:50:26, 13.70s/it]
 53%|█████▎    | 1462/2733 [5:27:45<4:49:29, 13.67s/it]
                                                       
{'loss': 1.6375, 'learning_rate': 0.0001448157994682871, 'epoch': 0.53}

 53%|█████▎    | 1462/2733 [5:27:45<4:49:29, 13.67s/it]
 54%|█████▎    | 1463/2733 [5:27:58<4:44:55, 13.46s/it]
                                                       
{'loss': 1.7209, 'learning_rate': 0.00014470186099506266, 'epoch': 0.54}

 54%|█████▎    | 1463/2733 [5:27:58<4:44:55, 13.46s/it]
 54%|█████▎    | 1464/2733 [5:28:12<4:46:11, 13.53s/it]
                                                       
{'loss': 1.7622, 'learning_rate': 0.0001445879225218382, 'epoch': 0.54}

 54%|█████▎    | 1464/2733 [5:28:12<4:46:11, 13.53s/it]
 54%|█████▎    | 1465/2733 [5:28:26<4:50:35, 13.75s/it]
                                                       
{'loss': 1.7458, 'learning_rate': 0.00014447398404861372, 'epoch': 0.54}

 54%|█████▎    | 1465/2733 [5:28:26<4:50:35, 13.75s/it]
 54%|█████▎    | 1466/2733 [5:28:40<4:49:37, 13.72s/it]
                                                       
{'loss': 1.631, 'learning_rate': 0.00014436004557538928, 'epoch': 0.54}

 54%|█████▎    | 1466/2733 [5:28:40<4:49:37, 13.72s/it]
 54%|█████▎    | 1467/2733 [5:28:53<4:49:27, 13.72s/it]
                                                       
{'loss': 1.7032, 'learning_rate': 0.0001442461071021648, 'epoch': 0.54}

 54%|█████▎    | 1467/2733 [5:28:53<4:49:27, 13.72s/it]
 54%|█████▎    | 1468/2733 [5:29:07<4:48:31, 13.68s/it]
                                                       
{'loss': 1.6634, 'learning_rate': 0.00014413216862894037, 'epoch': 0.54}

 54%|█████▎    | 1468/2733 [5:29:07<4:48:31, 13.68s/it]
 54%|█████▍    | 1469/2733 [5:29:21<4:51:31, 13.84s/it]
                                                       
{'loss': 1.8177, 'learning_rate': 0.0001440182301557159, 'epoch': 0.54}

 54%|█████▍    | 1469/2733 [5:29:21<4:51:31, 13.84s/it]
 54%|█████▍    | 1470/2733 [5:29:35<4:49:45, 13.77s/it]
                                                       
{'loss': 1.7117, 'learning_rate': 0.00014390429168249146, 'epoch': 0.54}

 54%|█████▍    | 1470/2733 [5:29:35<4:49:45, 13.77s/it]
 54%|█████▍    | 1471/2733 [5:29:48<4:46:10, 13.61s/it]
                                                       
{'loss': 1.5717, 'learning_rate': 0.000143790353209267, 'epoch': 0.54}

 54%|█████▍    | 1471/2733 [5:29:48<4:46:10, 13.61s/it]
 54%|█████▍    | 1472/2733 [5:30:02<4:47:35, 13.68s/it]
                                                       
{'loss': 1.6623, 'learning_rate': 0.00014367641473604252, 'epoch': 0.54}

 54%|█████▍    | 1472/2733 [5:30:02<4:47:35, 13.68s/it]
 54%|█████▍    | 1473/2733 [5:30:15<4:40:49, 13.37s/it]
                                                       
{'loss': 1.6663, 'learning_rate': 0.00014356247626281805, 'epoch': 0.54}

 54%|█████▍    | 1473/2733 [5:30:15<4:40:49, 13.37s/it]
 54%|█████▍    | 1474/2733 [5:30:28<4:42:12, 13.45s/it]
                                                       
{'loss': 1.6385, 'learning_rate': 0.0001434485377895936, 'epoch': 0.54}

 54%|█████▍    | 1474/2733 [5:30:28<4:42:12, 13.45s/it]
 54%|█████▍    | 1475/2733 [5:30:41<4:38:05, 13.26s/it]
                                                       
{'loss': 1.5952, 'learning_rate': 0.00014333459931636914, 'epoch': 0.54}

 54%|█████▍    | 1475/2733 [5:30:41<4:38:05, 13.26s/it]
 54%|█████▍    | 1476/2733 [5:30:54<4:36:38, 13.21s/it]
                                                       
{'loss': 1.8436, 'learning_rate': 0.0001432206608431447, 'epoch': 0.54}

 54%|█████▍    | 1476/2733 [5:30:54<4:36:38, 13.21s/it]
 54%|█████▍    | 1477/2733 [5:31:07<4:35:20, 13.15s/it]
                                                       
{'loss': 1.7363, 'learning_rate': 0.00014310672236992022, 'epoch': 0.54}

 54%|█████▍    | 1477/2733 [5:31:07<4:35:20, 13.15s/it]
 54%|█████▍    | 1478/2733 [5:31:21<4:40:19, 13.40s/it]
                                                       
{'loss': 1.6407, 'learning_rate': 0.00014299278389669578, 'epoch': 0.54}

 54%|█████▍    | 1478/2733 [5:31:21<4:40:19, 13.40s/it]
 54%|█████▍    | 1479/2733 [5:31:34<4:38:26, 13.32s/it]
                                                       
{'loss': 1.688, 'learning_rate': 0.0001428788454234713, 'epoch': 0.54}

 54%|█████▍    | 1479/2733 [5:31:34<4:38:26, 13.32s/it]
 54%|█████▍    | 1480/2733 [5:31:47<4:35:04, 13.17s/it]
                                                       
{'loss': 1.6769, 'learning_rate': 0.00014276490695024684, 'epoch': 0.54}

 54%|█████▍    | 1480/2733 [5:31:47<4:35:04, 13.17s/it]
 54%|█████▍    | 1481/2733 [5:32:01<4:37:28, 13.30s/it]
                                                       
{'loss': 1.6559, 'learning_rate': 0.0001426509684770224, 'epoch': 0.54}

 54%|█████▍    | 1481/2733 [5:32:01<4:37:28, 13.30s/it]
 54%|█████▍    | 1482/2733 [5:32:14<4:37:32, 13.31s/it]
                                                       
{'loss': 1.602, 'learning_rate': 0.00014253703000379793, 'epoch': 0.54}

 54%|█████▍    | 1482/2733 [5:32:14<4:37:32, 13.31s/it]
 54%|█████▍    | 1483/2733 [5:32:27<4:32:56, 13.10s/it]
                                                       
{'loss': 1.5314, 'learning_rate': 0.0001424230915305735, 'epoch': 0.54}

 54%|█████▍    | 1483/2733 [5:32:27<4:32:56, 13.10s/it]
 54%|█████▍    | 1484/2733 [5:32:41<4:39:27, 13.43s/it]
                                                       
{'loss': 1.6849, 'learning_rate': 0.00014230915305734902, 'epoch': 0.54}

 54%|█████▍    | 1484/2733 [5:32:41<4:39:27, 13.43s/it]
 54%|█████▍    | 1485/2733 [5:32:54<4:37:16, 13.33s/it]
                                                       
{'loss': 1.6659, 'learning_rate': 0.00014219521458412457, 'epoch': 0.54}

 54%|█████▍    | 1485/2733 [5:32:54<4:37:16, 13.33s/it]
 54%|█████▍    | 1486/2733 [5:33:07<4:38:43, 13.41s/it]
                                                       
{'loss': 1.6112, 'learning_rate': 0.0001420812761109001, 'epoch': 0.54}

 54%|█████▍    | 1486/2733 [5:33:07<4:38:43, 13.41s/it]
 54%|█████▍    | 1487/2733 [5:33:21<4:38:03, 13.39s/it]
                                                       
{'loss': 1.5453, 'learning_rate': 0.00014196733763767564, 'epoch': 0.54}

 54%|█████▍    | 1487/2733 [5:33:21<4:38:03, 13.39s/it]
 54%|█████▍    | 1488/2733 [5:33:34<4:35:25, 13.27s/it]
                                                       
{'loss': 1.6198, 'learning_rate': 0.0001418533991644512, 'epoch': 0.54}

 54%|█████▍    | 1488/2733 [5:33:34<4:35:25, 13.27s/it]
 54%|█████▍    | 1489/2733 [5:33:47<4:33:57, 13.21s/it]
                                                       
{'loss': 1.6152, 'learning_rate': 0.00014173946069122672, 'epoch': 0.54}

 54%|█████▍    | 1489/2733 [5:33:47<4:33:57, 13.21s/it]
 55%|█████▍    | 1490/2733 [5:34:01<4:36:16, 13.34s/it]
                                                       
{'loss': 1.6667, 'learning_rate': 0.00014162552221800228, 'epoch': 0.55}

 55%|█████▍    | 1490/2733 [5:34:01<4:36:16, 13.34s/it]
 55%|█████▍    | 1491/2733 [5:34:14<4:35:53, 13.33s/it]
                                                       
{'loss': 1.7455, 'learning_rate': 0.0001415115837447778, 'epoch': 0.55}

 55%|█████▍    | 1491/2733 [5:34:14<4:35:53, 13.33s/it]
 55%|█████▍    | 1492/2733 [5:34:28<4:40:05, 13.54s/it]
                                                       
{'loss': 1.6435, 'learning_rate': 0.00014139764527155337, 'epoch': 0.55}

 55%|█████▍    | 1492/2733 [5:34:28<4:40:05, 13.54s/it]
 55%|█████▍    | 1493/2733 [5:34:41<4:36:41, 13.39s/it]
                                                       
{'loss': 1.7207, 'learning_rate': 0.00014128370679832887, 'epoch': 0.55}

 55%|█████▍    | 1493/2733 [5:34:41<4:36:41, 13.39s/it]
 55%|█████▍    | 1494/2733 [5:34:54<4:33:53, 13.26s/it]
                                                       
{'loss': 1.6263, 'learning_rate': 0.00014116976832510443, 'epoch': 0.55}

 55%|█████▍    | 1494/2733 [5:34:54<4:33:53, 13.26s/it]
 55%|█████▍    | 1495/2733 [5:35:07<4:32:52, 13.23s/it]
                                                       
{'loss': 1.6943, 'learning_rate': 0.00014105582985187996, 'epoch': 0.55}

 55%|█████▍    | 1495/2733 [5:35:07<4:32:52, 13.23s/it]
 55%|█████▍    | 1496/2733 [5:35:20<4:28:50, 13.04s/it]
                                                       
{'loss': 1.7175, 'learning_rate': 0.00014094189137865552, 'epoch': 0.55}

 55%|█████▍    | 1496/2733 [5:35:20<4:28:50, 13.04s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 55%|█████▍    | 1497/2733 [5:35:34<4:34:15, 13.31s/it]
                                                       
{'loss': 1.6966, 'learning_rate': 0.00014082795290543105, 'epoch': 0.55}

 55%|█████▍    | 1497/2733 [5:35:34<4:34:15, 13.31s/it]
 55%|█████▍    | 1498/2733 [5:35:48<4:39:54, 13.60s/it]
                                                       
{'loss': 1.7793, 'learning_rate': 0.0001407140144322066, 'epoch': 0.55}

 55%|█████▍    | 1498/2733 [5:35:48<4:39:54, 13.60s/it]
 55%|█████▍    | 1499/2733 [5:36:01<4:37:28, 13.49s/it]
                                                       
{'loss': 1.7045, 'learning_rate': 0.00014060007595898213, 'epoch': 0.55}

 55%|█████▍    | 1499/2733 [5:36:01<4:37:28, 13.49s/it]
 55%|█████▍    | 1500/2733 [5:36:14<4:32:54, 13.28s/it]
                                                       
{'loss': 1.5956, 'learning_rate': 0.00014048613748575767, 'epoch': 0.55}

 55%|█████▍    | 1500/2733 [5:36:14<4:32:54, 13.28s/it]
 55%|█████▍    | 1501/2733 [5:36:27<4:31:28, 13.22s/it]
                                                       
{'loss': 1.7067, 'learning_rate': 0.00014037219901253322, 'epoch': 0.55}

 55%|█████▍    | 1501/2733 [5:36:27<4:31:28, 13.22s/it]
 55%|█████▍    | 1502/2733 [5:36:41<4:33:49, 13.35s/it]
                                                       
{'loss': 1.7319, 'learning_rate': 0.00014025826053930875, 'epoch': 0.55}

 55%|█████▍    | 1502/2733 [5:36:41<4:33:49, 13.35s/it]
 55%|█████▍    | 1503/2733 [5:36:54<4:32:50, 13.31s/it]
                                                       
{'loss': 1.6255, 'learning_rate': 0.0001401443220660843, 'epoch': 0.55}

 55%|█████▍    | 1503/2733 [5:36:54<4:32:50, 13.31s/it]
 55%|█████▌    | 1504/2733 [5:37:08<4:35:07, 13.43s/it]
                                                       
{'loss': 1.6505, 'learning_rate': 0.00014003038359285984, 'epoch': 0.55}

 55%|█████▌    | 1504/2733 [5:37:08<4:35:07, 13.43s/it]
 55%|█████▌    | 1505/2733 [5:37:20<4:30:51, 13.23s/it]
                                                       
{'loss': 1.5816, 'learning_rate': 0.0001399164451196354, 'epoch': 0.55}

 55%|█████▌    | 1505/2733 [5:37:20<4:30:51, 13.23s/it]
 55%|█████▌    | 1506/2733 [5:37:34<4:32:36, 13.33s/it]
                                                       
{'loss': 1.6341, 'learning_rate': 0.00013980250664641093, 'epoch': 0.55}

 55%|█████▌    | 1506/2733 [5:37:34<4:32:36, 13.33s/it]
 55%|█████▌    | 1507/2733 [5:37:47<4:32:18, 13.33s/it]
                                                       
{'loss': 1.6506, 'learning_rate': 0.00013968856817318646, 'epoch': 0.55}

 55%|█████▌    | 1507/2733 [5:37:47<4:32:18, 13.33s/it]
 55%|█████▌    | 1508/2733 [5:38:00<4:32:08, 13.33s/it]
                                                       
{'loss': 1.6201, 'learning_rate': 0.00013957462969996202, 'epoch': 0.55}

 55%|█████▌    | 1508/2733 [5:38:01<4:32:08, 13.33s/it]
 55%|█████▌    | 1509/2733 [5:38:14<4:33:37, 13.41s/it]
                                                       
{'loss': 1.6958, 'learning_rate': 0.00013946069122673755, 'epoch': 0.55}

 55%|█████▌    | 1509/2733 [5:38:14<4:33:37, 13.41s/it]
 55%|█████▌    | 1510/2733 [5:38:27<4:32:20, 13.36s/it]
                                                       
{'loss': 1.6765, 'learning_rate': 0.0001393467527535131, 'epoch': 0.55}

 55%|█████▌    | 1510/2733 [5:38:27<4:32:20, 13.36s/it]
 55%|█████▌    | 1511/2733 [5:38:41<4:33:47, 13.44s/it]
                                                       
{'loss': 1.7234, 'learning_rate': 0.00013923281428028863, 'epoch': 0.55}

 55%|█████▌    | 1511/2733 [5:38:41<4:33:47, 13.44s/it]
 55%|█████▌    | 1512/2733 [5:38:54<4:30:36, 13.30s/it]
                                                       
{'loss': 1.7258, 'learning_rate': 0.00013911887580706416, 'epoch': 0.55}

 55%|█████▌    | 1512/2733 [5:38:54<4:30:36, 13.30s/it]
 55%|█████▌    | 1513/2733 [5:39:08<4:32:45, 13.41s/it]
                                                       
{'loss': 1.6938, 'learning_rate': 0.00013900493733383972, 'epoch': 0.55}

 55%|█████▌    | 1513/2733 [5:39:08<4:32:45, 13.41s/it]
 55%|█████▌    | 1514/2733 [5:39:21<4:32:56, 13.43s/it]
                                                       
{'loss': 1.497, 'learning_rate': 0.00013889099886061525, 'epoch': 0.55}

 55%|█████▌    | 1514/2733 [5:39:21<4:32:56, 13.43s/it]
 55%|█████▌    | 1515/2733 [5:39:34<4:30:38, 13.33s/it]
                                                       
{'loss': 1.6125, 'learning_rate': 0.00013877706038739078, 'epoch': 0.55}

 55%|█████▌    | 1515/2733 [5:39:34<4:30:38, 13.33s/it]
 55%|█████▌    | 1516/2733 [5:39:48<4:33:09, 13.47s/it]
                                                       
{'loss': 1.7298, 'learning_rate': 0.00013866312191416634, 'epoch': 0.55}

 55%|█████▌    | 1516/2733 [5:39:48<4:33:09, 13.47s/it]
 56%|█████▌    | 1517/2733 [5:40:00<4:23:49, 13.02s/it]
                                                       
{'loss': 1.672, 'learning_rate': 0.00013854918344094187, 'epoch': 0.56}

 56%|█████▌    | 1517/2733 [5:40:00<4:23:49, 13.02s/it]
 56%|█████▌    | 1518/2733 [5:40:13<4:25:30, 13.11s/it]
                                                       
{'loss': 1.6497, 'learning_rate': 0.00013843524496771743, 'epoch': 0.56}

 56%|█████▌    | 1518/2733 [5:40:13<4:25:30, 13.11s/it]
 56%|█████▌    | 1519/2733 [5:40:26<4:23:22, 13.02s/it]
                                                       
{'loss': 1.634, 'learning_rate': 0.00013832130649449296, 'epoch': 0.56}

 56%|█████▌    | 1519/2733 [5:40:26<4:23:22, 13.02s/it]
 56%|█████▌    | 1520/2733 [5:40:39<4:24:17, 13.07s/it]
                                                       
{'loss': 1.6435, 'learning_rate': 0.00013820736802126852, 'epoch': 0.56}

 56%|█████▌    | 1520/2733 [5:40:39<4:24:17, 13.07s/it]
 56%|█████▌    | 1521/2733 [5:40:54<4:31:11, 13.43s/it]
                                                       
{'loss': 1.6514, 'learning_rate': 0.00013809342954804405, 'epoch': 0.56}

 56%|█████▌    | 1521/2733 [5:40:54<4:31:11, 13.43s/it]
 56%|█████▌    | 1522/2733 [5:41:07<4:30:33, 13.40s/it]
                                                       
{'loss': 1.7075, 'learning_rate': 0.00013797949107481958, 'epoch': 0.56}

 56%|█████▌    | 1522/2733 [5:41:07<4:30:33, 13.40s/it]
 56%|█████▌    | 1523/2733 [5:41:20<4:28:54, 13.33s/it]
                                                       
{'loss': 1.6169, 'learning_rate': 0.00013786555260159513, 'epoch': 0.56}

 56%|█████▌    | 1523/2733 [5:41:20<4:28:54, 13.33s/it]
 56%|█████▌    | 1524/2733 [5:41:34<4:34:09, 13.61s/it]
                                                       
{'loss': 1.6614, 'learning_rate': 0.00013775161412837066, 'epoch': 0.56}

 56%|█████▌    | 1524/2733 [5:41:34<4:34:09, 13.61s/it]
 56%|█████▌    | 1525/2733 [5:41:48<4:32:59, 13.56s/it]
                                                       
{'loss': 1.6917, 'learning_rate': 0.00013763767565514622, 'epoch': 0.56}

 56%|█████▌    | 1525/2733 [5:41:48<4:32:59, 13.56s/it]
 56%|█████▌    | 1526/2733 [5:42:01<4:33:06, 13.58s/it]
                                                       
{'loss': 1.7905, 'learning_rate': 0.00013752373718192175, 'epoch': 0.56}

 56%|█████▌    | 1526/2733 [5:42:01<4:33:06, 13.58s/it]
 56%|█████▌    | 1527/2733 [5:42:15<4:34:15, 13.64s/it]
                                                       
{'loss': 1.7389, 'learning_rate': 0.0001374097987086973, 'epoch': 0.56}

 56%|█████▌    | 1527/2733 [5:42:15<4:34:15, 13.64s/it]
 56%|█████▌    | 1528/2733 [5:42:29<4:33:39, 13.63s/it]
                                                       
{'loss': 1.7376, 'learning_rate': 0.0001372958602354728, 'epoch': 0.56}

 56%|█████▌    | 1528/2733 [5:42:29<4:33:39, 13.63s/it]
 56%|█████▌    | 1529/2733 [5:42:43<4:36:50, 13.80s/it]
                                                       
{'loss': 1.5578, 'learning_rate': 0.00013718192176224837, 'epoch': 0.56}

 56%|█████▌    | 1529/2733 [5:42:43<4:36:50, 13.80s/it]
 56%|█████▌    | 1530/2733 [5:42:57<4:36:35, 13.80s/it]
                                                       
{'loss': 1.6169, 'learning_rate': 0.0001370679832890239, 'epoch': 0.56}

 56%|█████▌    | 1530/2733 [5:42:57<4:36:35, 13.80s/it]
 56%|█████▌    | 1531/2733 [5:43:10<4:35:19, 13.74s/it]
                                                       
{'loss': 1.7192, 'learning_rate': 0.00013695404481579946, 'epoch': 0.56}

 56%|█████▌    | 1531/2733 [5:43:10<4:35:19, 13.74s/it]
 56%|█████▌    | 1532/2733 [5:43:24<4:34:03, 13.69s/it]
                                                       
{'loss': 1.6468, 'learning_rate': 0.000136840106342575, 'epoch': 0.56}

 56%|█████▌    | 1532/2733 [5:43:24<4:34:03, 13.69s/it]
 56%|█████▌    | 1533/2733 [5:43:38<4:36:50, 13.84s/it]
                                                       
{'loss': 1.6766, 'learning_rate': 0.00013672616786935055, 'epoch': 0.56}

 56%|█████▌    | 1533/2733 [5:43:38<4:36:50, 13.84s/it]
 56%|█████▌    | 1534/2733 [5:43:52<4:35:44, 13.80s/it]
                                                       
{'loss': 1.7686, 'learning_rate': 0.00013661222939612608, 'epoch': 0.56}

 56%|█████▌    | 1534/2733 [5:43:52<4:35:44, 13.80s/it]
 56%|█████▌    | 1535/2733 [5:44:05<4:31:28, 13.60s/it]
                                                       
{'loss': 1.6874, 'learning_rate': 0.0001364982909229016, 'epoch': 0.56}

 56%|█████▌    | 1535/2733 [5:44:05<4:31:28, 13.60s/it]
 56%|█████▌    | 1536/2733 [5:44:18<4:29:35, 13.51s/it]
                                                       
{'loss': 1.6522, 'learning_rate': 0.00013638435244967716, 'epoch': 0.56}

 56%|█████▌    | 1536/2733 [5:44:18<4:29:35, 13.51s/it]
 56%|█████▌    | 1537/2733 [5:44:31<4:24:20, 13.26s/it]
                                                       
{'loss': 1.5778, 'learning_rate': 0.0001362704139764527, 'epoch': 0.56}

 56%|█████▌    | 1537/2733 [5:44:31<4:24:20, 13.26s/it]
 56%|█████▋    | 1538/2733 [5:44:44<4:23:17, 13.22s/it]
                                                       
{'loss': 1.7218, 'learning_rate': 0.00013615647550322825, 'epoch': 0.56}

 56%|█████▋    | 1538/2733 [5:44:44<4:23:17, 13.22s/it]
 56%|█████▋    | 1539/2733 [5:44:57<4:21:50, 13.16s/it]
                                                       
{'loss': 1.666, 'learning_rate': 0.00013604253703000378, 'epoch': 0.56}

 56%|█████▋    | 1539/2733 [5:44:57<4:21:50, 13.16s/it]
 56%|█████▋    | 1540/2733 [5:45:10<4:19:44, 13.06s/it]
                                                       
{'loss': 1.6492, 'learning_rate': 0.00013592859855677934, 'epoch': 0.56}

 56%|█████▋    | 1540/2733 [5:45:10<4:19:44, 13.06s/it]
 56%|█████▋    | 1541/2733 [5:45:24<4:23:04, 13.24s/it]
                                                       
{'loss': 1.713, 'learning_rate': 0.00013581466008355487, 'epoch': 0.56}

 56%|█████▋    | 1541/2733 [5:45:24<4:23:04, 13.24s/it]
 56%|█████▋    | 1542/2733 [5:45:37<4:22:52, 13.24s/it]
                                                       
{'loss': 1.6553, 'learning_rate': 0.0001357007216103304, 'epoch': 0.56}

 56%|█████▋    | 1542/2733 [5:45:37<4:22:52, 13.24s/it]
 56%|█████▋    | 1543/2733 [5:45:51<4:28:39, 13.55s/it]
                                                       
{'loss': 1.7247, 'learning_rate': 0.00013558678313710596, 'epoch': 0.56}

 56%|█████▋    | 1543/2733 [5:45:51<4:28:39, 13.55s/it]
 56%|█████▋    | 1544/2733 [5:46:04<4:25:46, 13.41s/it]
                                                       
{'loss': 1.7159, 'learning_rate': 0.0001354728446638815, 'epoch': 0.56}

 56%|█████▋    | 1544/2733 [5:46:04<4:25:46, 13.41s/it]
 57%|█████▋    | 1545/2733 [5:46:17<4:22:46, 13.27s/it]
                                                       
{'loss': 1.5788, 'learning_rate': 0.00013535890619065705, 'epoch': 0.57}

 57%|█████▋    | 1545/2733 [5:46:17<4:22:46, 13.27s/it]
 57%|█████▋    | 1546/2733 [5:46:30<4:20:19, 13.16s/it]
                                                       
{'loss': 1.6541, 'learning_rate': 0.00013524496771743258, 'epoch': 0.57}

 57%|█████▋    | 1546/2733 [5:46:30<4:20:19, 13.16s/it]
 57%|█████▋    | 1547/2733 [5:46:43<4:16:44, 12.99s/it]
                                                       
{'loss': 1.6427, 'learning_rate': 0.00013513102924420813, 'epoch': 0.57}

 57%|█████▋    | 1547/2733 [5:46:43<4:16:44, 12.99s/it]
 57%|█████▋    | 1548/2733 [5:46:56<4:21:28, 13.24s/it]
                                                       
{'loss': 1.6362, 'learning_rate': 0.00013501709077098364, 'epoch': 0.57}

 57%|█████▋    | 1548/2733 [5:46:56<4:21:28, 13.24s/it]
 57%|█████▋    | 1549/2733 [5:47:10<4:23:08, 13.33s/it]
                                                       
{'loss': 1.6442, 'learning_rate': 0.0001349031522977592, 'epoch': 0.57}

 57%|█████▋    | 1549/2733 [5:47:10<4:23:08, 13.33s/it]
 57%|█████▋    | 1550/2733 [5:47:23<4:23:17, 13.35s/it]
                                                       
{'loss': 1.7238, 'learning_rate': 0.00013478921382453472, 'epoch': 0.57}

 57%|█████▋    | 1550/2733 [5:47:23<4:23:17, 13.35s/it]
 57%|█████▋    | 1551/2733 [5:47:36<4:18:19, 13.11s/it]
                                                       
{'loss': 1.6192, 'learning_rate': 0.00013467527535131028, 'epoch': 0.57}

 57%|█████▋    | 1551/2733 [5:47:36<4:18:19, 13.11s/it]
 57%|█████▋    | 1552/2733 [5:47:50<4:22:13, 13.32s/it]
                                                       
{'loss': 1.6677, 'learning_rate': 0.0001345613368780858, 'epoch': 0.57}

 57%|█████▋    | 1552/2733 [5:47:50<4:22:13, 13.32s/it]
 57%|█████▋    | 1553/2733 [5:48:03<4:20:33, 13.25s/it]
                                                       
{'loss': 1.6849, 'learning_rate': 0.00013444739840486137, 'epoch': 0.57}

 57%|█████▋    | 1553/2733 [5:48:03<4:20:33, 13.25s/it]
 57%|█████▋    | 1554/2733 [5:48:17<4:23:20, 13.40s/it]
                                                       
{'loss': 1.6131, 'learning_rate': 0.0001343334599316369, 'epoch': 0.57}

 57%|█████▋    | 1554/2733 [5:48:17<4:23:20, 13.40s/it]
 57%|█████▋    | 1555/2733 [5:48:30<4:23:39, 13.43s/it]
                                                       
{'loss': 1.6235, 'learning_rate': 0.00013421952145841243, 'epoch': 0.57}

 57%|█████▋    | 1555/2733 [5:48:30<4:23:39, 13.43s/it]
 57%|█████▋    | 1556/2733 [5:48:44<4:27:14, 13.62s/it]
                                                       
{'loss': 1.6953, 'learning_rate': 0.000134105582985188, 'epoch': 0.57}

 57%|█████▋    | 1556/2733 [5:48:44<4:27:14, 13.62s/it]
 57%|█████▋    | 1557/2733 [5:48:57<4:24:37, 13.50s/it]
                                                       
{'loss': 1.6377, 'learning_rate': 0.00013399164451196352, 'epoch': 0.57}

 57%|█████▋    | 1557/2733 [5:48:57<4:24:37, 13.50s/it]
 57%|█████▋    | 1558/2733 [5:49:11<4:27:36, 13.66s/it]
                                                       
{'loss': 1.6795, 'learning_rate': 0.00013387770603873908, 'epoch': 0.57}

 57%|█████▋    | 1558/2733 [5:49:11<4:27:36, 13.66s/it]
 57%|█████▋    | 1559/2733 [5:49:25<4:26:46, 13.63s/it]
                                                       
{'loss': 1.6537, 'learning_rate': 0.0001337637675655146, 'epoch': 0.57}

 57%|█████▋    | 1559/2733 [5:49:25<4:26:46, 13.63s/it]
 57%|█████▋    | 1560/2733 [5:49:39<4:27:29, 13.68s/it]
                                                       
{'loss': 1.5545, 'learning_rate': 0.00013364982909229016, 'epoch': 0.57}

 57%|█████▋    | 1560/2733 [5:49:39<4:27:29, 13.68s/it]
 57%|█████▋    | 1561/2733 [5:49:52<4:23:33, 13.49s/it]
                                                       
{'loss': 1.7011, 'learning_rate': 0.0001335358906190657, 'epoch': 0.57}

 57%|█████▋    | 1561/2733 [5:49:52<4:23:33, 13.49s/it]
 57%|█████▋    | 1562/2733 [5:50:05<4:23:31, 13.50s/it]
                                                       
{'loss': 1.6791, 'learning_rate': 0.00013342195214584122, 'epoch': 0.57}

 57%|█████▋    | 1562/2733 [5:50:05<4:23:31, 13.50s/it]
 57%|█████▋    | 1563/2733 [5:50:20<4:27:39, 13.73s/it]
                                                       
{'loss': 1.7079, 'learning_rate': 0.00013330801367261678, 'epoch': 0.57}

 57%|█████▋    | 1563/2733 [5:50:20<4:27:39, 13.73s/it]
 57%|█████▋    | 1564/2733 [5:50:33<4:27:54, 13.75s/it]
                                                       
{'loss': 1.6907, 'learning_rate': 0.0001331940751993923, 'epoch': 0.57}

 57%|█████▋    | 1564/2733 [5:50:33<4:27:54, 13.75s/it]
 57%|█████▋    | 1565/2733 [5:50:46<4:21:47, 13.45s/it]
                                                       
{'loss': 1.6757, 'learning_rate': 0.00013308013672616787, 'epoch': 0.57}

 57%|█████▋    | 1565/2733 [5:50:46<4:21:47, 13.45s/it]
 57%|█████▋    | 1566/2733 [5:50:58<4:14:21, 13.08s/it]
                                                       
{'loss': 1.657, 'learning_rate': 0.0001329661982529434, 'epoch': 0.57}

 57%|█████▋    | 1566/2733 [5:50:58<4:14:21, 13.08s/it]
 57%|█████▋    | 1567/2733 [5:51:11<4:11:25, 12.94s/it]
                                                       
{'loss': 1.5415, 'learning_rate': 0.00013285225977971896, 'epoch': 0.57}

 57%|█████▋    | 1567/2733 [5:51:11<4:11:25, 12.94s/it]
 57%|█████▋    | 1568/2733 [5:51:24<4:12:10, 12.99s/it]
                                                       
{'loss': 1.6834, 'learning_rate': 0.0001327383213064945, 'epoch': 0.57}

 57%|█████▋    | 1568/2733 [5:51:24<4:12:10, 12.99s/it]
 57%|█████▋    | 1569/2733 [5:51:38<4:17:44, 13.29s/it]
                                                       
{'loss': 1.7187, 'learning_rate': 0.00013262438283327002, 'epoch': 0.57}

 57%|█████▋    | 1569/2733 [5:51:38<4:17:44, 13.29s/it]
 57%|█████▋    | 1570/2733 [5:51:52<4:19:04, 13.37s/it]
                                                       
{'loss': 1.7307, 'learning_rate': 0.00013251044436004555, 'epoch': 0.57}

 57%|█████▋    | 1570/2733 [5:51:52<4:19:04, 13.37s/it]
 57%|█████▋    | 1571/2733 [5:52:06<4:23:52, 13.62s/it]
                                                       
{'loss': 1.6835, 'learning_rate': 0.0001323965058868211, 'epoch': 0.57}

 57%|█████▋    | 1571/2733 [5:52:06<4:23:52, 13.62s/it]
 58%|█████▊    | 1572/2733 [5:52:19<4:23:37, 13.62s/it]
                                                       
{'loss': 1.6768, 'learning_rate': 0.00013228256741359664, 'epoch': 0.58}

 58%|█████▊    | 1572/2733 [5:52:19<4:23:37, 13.62s/it]
 58%|█████▊    | 1573/2733 [5:52:33<4:25:28, 13.73s/it]
                                                       
{'loss': 1.6558, 'learning_rate': 0.0001321686289403722, 'epoch': 0.58}

 58%|█████▊    | 1573/2733 [5:52:33<4:25:28, 13.73s/it]
 58%|█████▊    | 1574/2733 [5:52:47<4:24:46, 13.71s/it]
                                                       
{'loss': 1.7543, 'learning_rate': 0.00013205469046714772, 'epoch': 0.58}

 58%|█████▊    | 1574/2733 [5:52:47<4:24:46, 13.71s/it]
 58%|█████▊    | 1575/2733 [5:53:01<4:27:23, 13.85s/it]
                                                       
{'loss': 1.8149, 'learning_rate': 0.00013194075199392328, 'epoch': 0.58}

 58%|█████▊    | 1575/2733 [5:53:01<4:27:23, 13.85s/it]
 58%|█████▊    | 1576/2733 [5:53:16<4:29:34, 13.98s/it]
                                                       
{'loss': 1.7282, 'learning_rate': 0.0001318268135206988, 'epoch': 0.58}

 58%|█████▊    | 1576/2733 [5:53:16<4:29:34, 13.98s/it]
 58%|█████▊    | 1577/2733 [5:53:28<4:21:00, 13.55s/it]
                                                       
{'loss': 1.5928, 'learning_rate': 0.00013171287504747434, 'epoch': 0.58}

 58%|█████▊    | 1577/2733 [5:53:28<4:21:00, 13.55s/it]
 58%|█████▊    | 1578/2733 [5:53:41<4:17:46, 13.39s/it]
                                                       
{'loss': 1.6437, 'learning_rate': 0.0001315989365742499, 'epoch': 0.58}

 58%|█████▊    | 1578/2733 [5:53:41<4:17:46, 13.39s/it]
 58%|█████▊    | 1579/2733 [5:53:55<4:17:52, 13.41s/it]
                                                       
{'loss': 1.7624, 'learning_rate': 0.00013148499810102543, 'epoch': 0.58}

 58%|█████▊    | 1579/2733 [5:53:55<4:17:52, 13.41s/it]
 58%|█████▊    | 1580/2733 [5:54:08<4:16:18, 13.34s/it]
                                                       
{'loss': 1.6416, 'learning_rate': 0.000131371059627801, 'epoch': 0.58}

 58%|█████▊    | 1580/2733 [5:54:08<4:16:18, 13.34s/it]
 58%|█████▊    | 1581/2733 [5:54:21<4:18:20, 13.46s/it]
                                                       
{'loss': 1.6503, 'learning_rate': 0.00013125712115457652, 'epoch': 0.58}

 58%|█████▊    | 1581/2733 [5:54:21<4:18:20, 13.46s/it]
 58%|█████▊    | 1582/2733 [5:54:35<4:16:02, 13.35s/it]
                                                       
{'loss': 1.7252, 'learning_rate': 0.00013114318268135208, 'epoch': 0.58}

 58%|█████▊    | 1582/2733 [5:54:35<4:16:02, 13.35s/it]
 58%|█████▊    | 1583/2733 [5:54:48<4:15:59, 13.36s/it]
                                                       
{'loss': 1.5638, 'learning_rate': 0.0001310292442081276, 'epoch': 0.58}

 58%|█████▊    | 1583/2733 [5:54:48<4:15:59, 13.36s/it]
 58%|█████▊    | 1584/2733 [5:55:01<4:12:03, 13.16s/it]
                                                       
{'loss': 1.6437, 'learning_rate': 0.00013091530573490314, 'epoch': 0.58}

 58%|█████▊    | 1584/2733 [5:55:01<4:12:03, 13.16s/it]
 58%|█████▊    | 1585/2733 [5:55:14<4:12:31, 13.20s/it]
                                                       
{'loss': 1.5625, 'learning_rate': 0.0001308013672616787, 'epoch': 0.58}

 58%|█████▊    | 1585/2733 [5:55:14<4:12:31, 13.20s/it]
 58%|█████▊    | 1586/2733 [5:55:28<4:17:14, 13.46s/it]
                                                       
{'loss': 1.6545, 'learning_rate': 0.00013068742878845422, 'epoch': 0.58}

 58%|█████▊    | 1586/2733 [5:55:28<4:17:14, 13.46s/it]
 58%|█████▊    | 1587/2733 [5:55:41<4:14:33, 13.33s/it]
                                                       
{'loss': 1.6065, 'learning_rate': 0.00013057349031522975, 'epoch': 0.58}

 58%|█████▊    | 1587/2733 [5:55:41<4:14:33, 13.33s/it]
 58%|█████▊    | 1588/2733 [5:55:54<4:12:46, 13.25s/it]
                                                       
{'loss': 1.6371, 'learning_rate': 0.0001304595518420053, 'epoch': 0.58}

 58%|█████▊    | 1588/2733 [5:55:54<4:12:46, 13.25s/it]
 58%|█████▊    | 1589/2733 [5:56:08<4:14:12, 13.33s/it]
                                                       
{'loss': 1.7623, 'learning_rate': 0.00013034561336878084, 'epoch': 0.58}

 58%|█████▊    | 1589/2733 [5:56:08<4:14:12, 13.33s/it]
 58%|█████▊    | 1590/2733 [5:56:21<4:14:56, 13.38s/it]
                                                       
{'loss': 1.5368, 'learning_rate': 0.00013023167489555637, 'epoch': 0.58}

 58%|█████▊    | 1590/2733 [5:56:21<4:14:56, 13.38s/it]
 58%|█████▊    | 1591/2733 [5:56:34<4:11:59, 13.24s/it]
                                                       
{'loss': 1.7186, 'learning_rate': 0.00013011773642233193, 'epoch': 0.58}

 58%|█████▊    | 1591/2733 [5:56:34<4:11:59, 13.24s/it]
 58%|█████▊    | 1592/2733 [5:56:48<4:14:03, 13.36s/it]
                                                       
{'loss': 1.7264, 'learning_rate': 0.00013000379794910746, 'epoch': 0.58}

 58%|█████▊    | 1592/2733 [5:56:48<4:14:03, 13.36s/it]
 58%|█████▊    | 1593/2733 [5:57:01<4:15:20, 13.44s/it]
                                                       
{'loss': 1.6828, 'learning_rate': 0.00012988985947588302, 'epoch': 0.58}

 58%|█████▊    | 1593/2733 [5:57:01<4:15:20, 13.44s/it]
 58%|█████▊    | 1594/2733 [5:57:13<4:07:51, 13.06s/it]
                                                       
{'loss': 1.6628, 'learning_rate': 0.00012977592100265855, 'epoch': 0.58}

 58%|█████▊    | 1594/2733 [5:57:13<4:07:51, 13.06s/it]
 58%|█████▊    | 1595/2733 [5:57:27<4:13:11, 13.35s/it]
                                                       
{'loss': 1.6666, 'learning_rate': 0.0001296619825294341, 'epoch': 0.58}

 58%|█████▊    | 1595/2733 [5:57:27<4:13:11, 13.35s/it]
 58%|█████▊    | 1596/2733 [5:57:42<4:18:03, 13.62s/it]
                                                       
{'loss': 1.6341, 'learning_rate': 0.00012954804405620964, 'epoch': 0.58}

 58%|█████▊    | 1596/2733 [5:57:42<4:18:03, 13.62s/it]
 58%|█████▊    | 1597/2733 [5:57:56<4:18:59, 13.68s/it]
                                                       
{'loss': 1.6468, 'learning_rate': 0.00012943410558298517, 'epoch': 0.58}

 58%|█████▊    | 1597/2733 [5:57:56<4:18:59, 13.68s/it]
 58%|█████▊    | 1598/2733 [5:58:09<4:15:47, 13.52s/it]
                                                       
{'loss': 1.5474, 'learning_rate': 0.00012932016710976072, 'epoch': 0.58}

 58%|█████▊    | 1598/2733 [5:58:09<4:15:47, 13.52s/it]
 59%|█████▊    | 1599/2733 [5:58:22<4:14:46, 13.48s/it]
                                                       
{'loss': 1.6944, 'learning_rate': 0.00012920622863653625, 'epoch': 0.59}

 59%|█████▊    | 1599/2733 [5:58:22<4:14:46, 13.48s/it]
 59%|█████▊    | 1600/2733 [5:58:35<4:12:43, 13.38s/it]
                                                       
{'loss': 1.6975, 'learning_rate': 0.0001290922901633118, 'epoch': 0.59}

 59%|█████▊    | 1600/2733 [5:58:35<4:12:43, 13.38s/it]
 59%|█████▊    | 1601/2733 [5:58:49<4:12:45, 13.40s/it]
                                                       
{'loss': 1.581, 'learning_rate': 0.00012897835169008734, 'epoch': 0.59}

 59%|█████▊    | 1601/2733 [5:58:49<4:12:45, 13.40s/it]
 59%|█████▊    | 1602/2733 [5:59:02<4:09:30, 13.24s/it]
                                                       
{'loss': 1.7586, 'learning_rate': 0.0001288644132168629, 'epoch': 0.59}

 59%|█████▊    | 1602/2733 [5:59:02<4:09:30, 13.24s/it]
 59%|█████▊    | 1603/2733 [5:59:15<4:11:40, 13.36s/it]
                                                       
{'loss': 1.6934, 'learning_rate': 0.00012875047474363843, 'epoch': 0.59}

 59%|█████▊    | 1603/2733 [5:59:15<4:11:40, 13.36s/it]
 59%|█████▊    | 1604/2733 [5:59:29<4:13:24, 13.47s/it]
                                                       
{'loss': 1.6419, 'learning_rate': 0.00012863653627041396, 'epoch': 0.59}

 59%|█████▊    | 1604/2733 [5:59:29<4:13:24, 13.47s/it]
 59%|█████▊    | 1605/2733 [5:59:43<4:14:01, 13.51s/it]
                                                       
{'loss': 1.7263, 'learning_rate': 0.0001285225977971895, 'epoch': 0.59}

 59%|█████▊    | 1605/2733 [5:59:43<4:14:01, 13.51s/it]
 59%|█████▉    | 1606/2733 [5:59:56<4:14:43, 13.56s/it]
                                                       
{'loss': 1.8033, 'learning_rate': 0.00012840865932396505, 'epoch': 0.59}

 59%|█████▉    | 1606/2733 [5:59:56<4:14:43, 13.56s/it]
 59%|█████▉    | 1607/2733 [6:00:10<4:14:40, 13.57s/it]
                                                       
{'loss': 1.7479, 'learning_rate': 0.00012829472085074058, 'epoch': 0.59}

 59%|█████▉    | 1607/2733 [6:00:10<4:14:40, 13.57s/it]
 59%|█████▉    | 1608/2733 [6:00:23<4:14:02, 13.55s/it]
                                                       
{'loss': 1.5796, 'learning_rate': 0.00012818078237751614, 'epoch': 0.59}

 59%|█████▉    | 1608/2733 [6:00:23<4:14:02, 13.55s/it]
 59%|█████▉    | 1609/2733 [6:00:37<4:14:07, 13.57s/it]
                                                       
{'loss': 1.6942, 'learning_rate': 0.00012806684390429167, 'epoch': 0.59}

 59%|█████▉    | 1609/2733 [6:00:37<4:14:07, 13.57s/it]
 59%|█████▉    | 1610/2733 [6:00:50<4:14:06, 13.58s/it]
                                                       
{'loss': 1.6015, 'learning_rate': 0.00012795290543106722, 'epoch': 0.59}

 59%|█████▉    | 1610/2733 [6:00:51<4:14:06, 13.58s/it]
 59%|█████▉    | 1611/2733 [6:01:04<4:11:06, 13.43s/it]
                                                       
{'loss': 1.6935, 'learning_rate': 0.00012783896695784275, 'epoch': 0.59}

 59%|█████▉    | 1611/2733 [6:01:04<4:11:06, 13.43s/it]
 59%|█████▉    | 1612/2733 [6:01:17<4:09:47, 13.37s/it]
                                                       
{'loss': 1.6303, 'learning_rate': 0.00012772502848461828, 'epoch': 0.59}

 59%|█████▉    | 1612/2733 [6:01:17<4:09:47, 13.37s/it]
 59%|█████▉    | 1613/2733 [6:01:30<4:08:22, 13.31s/it]
                                                       
{'loss': 1.6321, 'learning_rate': 0.00012761109001139384, 'epoch': 0.59}

 59%|█████▉    | 1613/2733 [6:01:30<4:08:22, 13.31s/it]
 59%|█████▉    | 1614/2733 [6:01:44<4:11:48, 13.50s/it]
                                                       
{'loss': 1.7883, 'learning_rate': 0.00012749715153816937, 'epoch': 0.59}

 59%|█████▉    | 1614/2733 [6:01:44<4:11:48, 13.50s/it]
 59%|█████▉    | 1615/2733 [6:01:57<4:07:25, 13.28s/it]
                                                       
{'loss': 1.6577, 'learning_rate': 0.00012738321306494493, 'epoch': 0.59}

 59%|█████▉    | 1615/2733 [6:01:57<4:07:25, 13.28s/it]
 59%|█████▉    | 1616/2733 [6:02:10<4:08:53, 13.37s/it]
                                                       
{'loss': 1.678, 'learning_rate': 0.00012726927459172046, 'epoch': 0.59}

 59%|█████▉    | 1616/2733 [6:02:10<4:08:53, 13.37s/it]
 59%|█████▉    | 1617/2733 [6:02:23<4:07:02, 13.28s/it]
                                                       
{'loss': 1.6912, 'learning_rate': 0.000127155336118496, 'epoch': 0.59}

 59%|█████▉    | 1617/2733 [6:02:23<4:07:02, 13.28s/it]
 59%|█████▉    | 1618/2733 [6:02:36<4:05:28, 13.21s/it]
                                                       
{'loss': 1.6685, 'learning_rate': 0.00012704139764527155, 'epoch': 0.59}

 59%|█████▉    | 1618/2733 [6:02:36<4:05:28, 13.21s/it]
 59%|█████▉    | 1619/2733 [6:02:50<4:06:37, 13.28s/it]
                                                       
{'loss': 1.6713, 'learning_rate': 0.00012692745917204708, 'epoch': 0.59}

 59%|█████▉    | 1619/2733 [6:02:50<4:06:37, 13.28s/it]
 59%|█████▉    | 1620/2733 [6:03:03<4:07:31, 13.34s/it]
                                                       
{'loss': 1.7186, 'learning_rate': 0.00012681352069882264, 'epoch': 0.59}

 59%|█████▉    | 1620/2733 [6:03:03<4:07:31, 13.34s/it]
 59%|█████▉    | 1621/2733 [6:03:17<4:08:48, 13.42s/it]
                                                       
{'loss': 1.7991, 'learning_rate': 0.00012669958222559817, 'epoch': 0.59}

 59%|█████▉    | 1621/2733 [6:03:17<4:08:48, 13.42s/it]
 59%|█████▉    | 1622/2733 [6:03:29<4:02:19, 13.09s/it]
                                                       
{'loss': 1.7193, 'learning_rate': 0.00012658564375237372, 'epoch': 0.59}

 59%|█████▉    | 1622/2733 [6:03:29<4:02:19, 13.09s/it]
 59%|█████▉    | 1623/2733 [6:03:42<3:58:01, 12.87s/it]
                                                       
{'loss': 1.5226, 'learning_rate': 0.00012647170527914925, 'epoch': 0.59}

 59%|█████▉    | 1623/2733 [6:03:42<3:58:01, 12.87s/it]
 59%|█████▉    | 1624/2733 [6:03:55<3:59:22, 12.95s/it]
                                                       
{'loss': 1.6949, 'learning_rate': 0.00012635776680592478, 'epoch': 0.59}

 59%|█████▉    | 1624/2733 [6:03:55<3:59:22, 12.95s/it]
 59%|█████▉    | 1625/2733 [6:04:07<3:55:39, 12.76s/it]
                                                       
{'loss': 1.6222, 'learning_rate': 0.00012624382833270031, 'epoch': 0.59}

 59%|█████▉    | 1625/2733 [6:04:07<3:55:39, 12.76s/it]
 59%|█████▉    | 1626/2733 [6:04:21<3:59:50, 13.00s/it]
                                                       
{'loss': 1.7198, 'learning_rate': 0.00012612988985947587, 'epoch': 0.59}

 59%|█████▉    | 1626/2733 [6:04:21<3:59:50, 13.00s/it]
 60%|█████▉    | 1627/2733 [6:04:34<4:03:38, 13.22s/it]
                                                       
{'loss': 1.7753, 'learning_rate': 0.0001260159513862514, 'epoch': 0.6}

 60%|█████▉    | 1627/2733 [6:04:34<4:03:38, 13.22s/it]
 60%|█████▉    | 1628/2733 [6:04:48<4:04:47, 13.29s/it]
                                                       
{'loss': 1.6726, 'learning_rate': 0.00012590201291302696, 'epoch': 0.6}

 60%|█████▉    | 1628/2733 [6:04:48<4:04:47, 13.29s/it]
 60%|█████▉    | 1629/2733 [6:05:01<4:03:28, 13.23s/it]
                                                       
{'loss': 1.6799, 'learning_rate': 0.0001257880744398025, 'epoch': 0.6}

 60%|█████▉    | 1629/2733 [6:05:01<4:03:28, 13.23s/it]
 60%|█████▉    | 1630/2733 [6:05:14<4:05:07, 13.33s/it]
                                                       
{'loss': 1.6876, 'learning_rate': 0.00012567413596657805, 'epoch': 0.6}

 60%|█████▉    | 1630/2733 [6:05:14<4:05:07, 13.33s/it]
 60%|█████▉    | 1631/2733 [6:05:28<4:03:23, 13.25s/it]
                                                       
{'loss': 1.6111, 'learning_rate': 0.00012556019749335358, 'epoch': 0.6}

 60%|█████▉    | 1631/2733 [6:05:28<4:03:23, 13.25s/it]
 60%|█████▉    | 1632/2733 [6:05:41<4:04:24, 13.32s/it]
                                                       
{'loss': 1.6043, 'learning_rate': 0.0001254462590201291, 'epoch': 0.6}

 60%|█████▉    | 1632/2733 [6:05:41<4:04:24, 13.32s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 60%|█████▉    | 1633/2733 [6:05:55<4:06:54, 13.47s/it]
                                                       
{'loss': 1.7084, 'learning_rate': 0.00012533232054690467, 'epoch': 0.6}

 60%|█████▉    | 1633/2733 [6:05:55<4:06:54, 13.47s/it]
 60%|█████▉    | 1634/2733 [6:06:08<4:07:50, 13.53s/it]
                                                       
{'loss': 1.6504, 'learning_rate': 0.0001252183820736802, 'epoch': 0.6}

 60%|█████▉    | 1634/2733 [6:06:09<4:07:50, 13.53s/it]
 60%|█████▉    | 1635/2733 [6:06:21<4:04:22, 13.35s/it]
                                                       
{'loss': 1.7427, 'learning_rate': 0.00012510444360045575, 'epoch': 0.6}

 60%|█████▉    | 1635/2733 [6:06:21<4:04:22, 13.35s/it]
 60%|█████▉    | 1636/2733 [6:06:35<4:04:53, 13.39s/it]
                                                       
{'loss': 1.6796, 'learning_rate': 0.00012499050512723128, 'epoch': 0.6}

 60%|█████▉    | 1636/2733 [6:06:35<4:04:53, 13.39s/it]
 60%|█████▉    | 1637/2733 [6:06:49<4:07:37, 13.56s/it]
                                                       
{'loss': 1.6875, 'learning_rate': 0.00012487656665400684, 'epoch': 0.6}

 60%|█████▉    | 1637/2733 [6:06:49<4:07:37, 13.56s/it]
 60%|█████▉    | 1638/2733 [6:07:03<4:08:48, 13.63s/it]
                                                       
{'loss': 1.7092, 'learning_rate': 0.00012476262818078237, 'epoch': 0.6}

 60%|█████▉    | 1638/2733 [6:07:03<4:08:48, 13.63s/it]
 60%|█████▉    | 1639/2733 [6:07:17<4:09:48, 13.70s/it]
                                                       
{'loss': 1.7735, 'learning_rate': 0.0001246486897075579, 'epoch': 0.6}

 60%|█████▉    | 1639/2733 [6:07:17<4:09:48, 13.70s/it]
 60%|██████    | 1640/2733 [6:07:30<4:06:37, 13.54s/it]
                                                       
{'loss': 1.6673, 'learning_rate': 0.00012453475123433346, 'epoch': 0.6}

 60%|██████    | 1640/2733 [6:07:30<4:06:37, 13.54s/it]
 60%|██████    | 1641/2733 [6:07:43<4:07:11, 13.58s/it]
                                                       
{'loss': 1.6683, 'learning_rate': 0.000124420812761109, 'epoch': 0.6}

 60%|██████    | 1641/2733 [6:07:43<4:07:11, 13.58s/it]
 60%|██████    | 1642/2733 [6:07:56<4:04:11, 13.43s/it]
                                                       
{'loss': 1.7075, 'learning_rate': 0.00012430687428788455, 'epoch': 0.6}

 60%|██████    | 1642/2733 [6:07:56<4:04:11, 13.43s/it]
 60%|██████    | 1643/2733 [6:08:09<4:00:17, 13.23s/it]
                                                       
{'loss': 1.6591, 'learning_rate': 0.00012419293581466008, 'epoch': 0.6}

 60%|██████    | 1643/2733 [6:08:09<4:00:17, 13.23s/it]
 60%|██████    | 1644/2733 [6:08:23<4:00:55, 13.27s/it]
                                                       
{'loss': 1.7045, 'learning_rate': 0.0001240789973414356, 'epoch': 0.6}

 60%|██████    | 1644/2733 [6:08:23<4:00:55, 13.27s/it]
 60%|██████    | 1645/2733 [6:08:36<4:01:37, 13.33s/it]
                                                       
{'loss': 1.6214, 'learning_rate': 0.00012396505886821114, 'epoch': 0.6}

 60%|██████    | 1645/2733 [6:08:36<4:01:37, 13.33s/it]
 60%|██████    | 1646/2733 [6:08:49<4:00:26, 13.27s/it]
                                                       
{'loss': 1.6912, 'learning_rate': 0.0001238511203949867, 'epoch': 0.6}

 60%|██████    | 1646/2733 [6:08:49<4:00:26, 13.27s/it]
 60%|██████    | 1647/2733 [6:09:03<4:02:25, 13.39s/it]
                                                       
{'loss': 1.6083, 'learning_rate': 0.00012373718192176223, 'epoch': 0.6}

 60%|██████    | 1647/2733 [6:09:03<4:02:25, 13.39s/it]
 60%|██████    | 1648/2733 [6:09:16<4:00:17, 13.29s/it]
                                                       
{'loss': 1.65, 'learning_rate': 0.00012362324344853778, 'epoch': 0.6}

 60%|██████    | 1648/2733 [6:09:16<4:00:17, 13.29s/it]
 60%|██████    | 1649/2733 [6:09:29<3:58:09, 13.18s/it]
                                                       
{'loss': 1.7725, 'learning_rate': 0.0001235093049753133, 'epoch': 0.6}

 60%|██████    | 1649/2733 [6:09:29<3:58:09, 13.18s/it]
 60%|██████    | 1650/2733 [6:09:43<4:03:39, 13.50s/it]
                                                       
{'loss': 1.7098, 'learning_rate': 0.00012339536650208887, 'epoch': 0.6}

 60%|██████    | 1650/2733 [6:09:43<4:03:39, 13.50s/it]
 60%|██████    | 1651/2733 [6:09:56<4:02:25, 13.44s/it]
                                                       
{'loss': 1.6592, 'learning_rate': 0.0001232814280288644, 'epoch': 0.6}

 60%|██████    | 1651/2733 [6:09:56<4:02:25, 13.44s/it]
 60%|██████    | 1652/2733 [6:10:10<4:00:45, 13.36s/it]
                                                       
{'loss': 1.6497, 'learning_rate': 0.00012316748955563993, 'epoch': 0.6}

 60%|██████    | 1652/2733 [6:10:10<4:00:45, 13.36s/it]
 60%|██████    | 1653/2733 [6:10:23<4:00:59, 13.39s/it]
                                                       
{'loss': 1.7309, 'learning_rate': 0.0001230535510824155, 'epoch': 0.6}

 60%|██████    | 1653/2733 [6:10:23<4:00:59, 13.39s/it]
 61%|██████    | 1654/2733 [6:10:37<4:01:55, 13.45s/it]
                                                       
{'loss': 1.6446, 'learning_rate': 0.00012293961260919102, 'epoch': 0.61}

 61%|██████    | 1654/2733 [6:10:37<4:01:55, 13.45s/it]
 61%|██████    | 1655/2733 [6:10:49<3:56:49, 13.18s/it]
                                                       
{'loss': 1.6023, 'learning_rate': 0.00012282567413596658, 'epoch': 0.61}

 61%|██████    | 1655/2733 [6:10:49<3:56:49, 13.18s/it]
 61%|██████    | 1656/2733 [6:11:03<3:57:45, 13.25s/it]
                                                       
{'loss': 1.6779, 'learning_rate': 0.0001227117356627421, 'epoch': 0.61}

 61%|██████    | 1656/2733 [6:11:03<3:57:45, 13.25s/it]
 61%|██████    | 1657/2733 [6:11:17<4:01:57, 13.49s/it]
                                                       
{'loss': 1.5554, 'learning_rate': 0.00012259779718951766, 'epoch': 0.61}

 61%|██████    | 1657/2733 [6:11:17<4:01:57, 13.49s/it]
 61%|██████    | 1658/2733 [6:11:30<4:00:43, 13.44s/it]
                                                       
{'loss': 1.6126, 'learning_rate': 0.0001224838587162932, 'epoch': 0.61}

 61%|██████    | 1658/2733 [6:11:30<4:00:43, 13.44s/it]
 61%|██████    | 1659/2733 [6:11:43<4:00:35, 13.44s/it]
                                                       
{'loss': 1.6891, 'learning_rate': 0.00012236992024306873, 'epoch': 0.61}

 61%|██████    | 1659/2733 [6:11:43<4:00:35, 13.44s/it]
 61%|██████    | 1660/2733 [6:11:57<4:01:18, 13.49s/it]
                                                       
{'loss': 1.6621, 'learning_rate': 0.00012225598176984426, 'epoch': 0.61}

 61%|██████    | 1660/2733 [6:11:57<4:01:18, 13.49s/it]
 61%|██████    | 1661/2733 [6:12:10<4:00:55, 13.48s/it]
                                                       
{'loss': 1.6574, 'learning_rate': 0.0001221420432966198, 'epoch': 0.61}

 61%|██████    | 1661/2733 [6:12:10<4:00:55, 13.48s/it]
 61%|██████    | 1662/2733 [6:12:24<4:02:59, 13.61s/it]
                                                       
{'loss': 1.5779, 'learning_rate': 0.00012202810482339536, 'epoch': 0.61}

 61%|██████    | 1662/2733 [6:12:24<4:02:59, 13.61s/it]
 61%|██████    | 1663/2733 [6:12:38<4:02:59, 13.63s/it]
                                                       
{'loss': 1.6831, 'learning_rate': 0.0001219141663501709, 'epoch': 0.61}

 61%|██████    | 1663/2733 [6:12:38<4:02:59, 13.63s/it]
 61%|██████    | 1664/2733 [6:12:51<4:01:03, 13.53s/it]
                                                       
{'loss': 1.7027, 'learning_rate': 0.00012180022787694644, 'epoch': 0.61}

 61%|██████    | 1664/2733 [6:12:51<4:01:03, 13.53s/it]
 61%|██████    | 1665/2733 [6:13:04<3:58:38, 13.41s/it]
                                                       
{'loss': 1.7115, 'learning_rate': 0.00012168628940372199, 'epoch': 0.61}

 61%|██████    | 1665/2733 [6:13:04<3:58:38, 13.41s/it]
 61%|██████    | 1666/2733 [6:13:18<3:57:57, 13.38s/it]
                                                       
{'loss': 1.8035, 'learning_rate': 0.00012157235093049752, 'epoch': 0.61}

 61%|██████    | 1666/2733 [6:13:18<3:57:57, 13.38s/it]
 61%|██████    | 1667/2733 [6:13:32<4:02:41, 13.66s/it]
                                                       
{'loss': 1.6653, 'learning_rate': 0.00012145841245727306, 'epoch': 0.61}

 61%|██████    | 1667/2733 [6:13:32<4:02:41, 13.66s/it]
 61%|██████    | 1668/2733 [6:13:46<4:01:22, 13.60s/it]
                                                       
{'loss': 1.7744, 'learning_rate': 0.0001213444739840486, 'epoch': 0.61}

 61%|██████    | 1668/2733 [6:13:46<4:01:22, 13.60s/it]
 61%|██████    | 1669/2733 [6:13:59<3:59:06, 13.48s/it]
                                                       
{'loss': 1.6738, 'learning_rate': 0.00012123053551082415, 'epoch': 0.61}

 61%|██████    | 1669/2733 [6:13:59<3:59:06, 13.48s/it]
 61%|██████    | 1670/2733 [6:14:12<3:57:07, 13.38s/it]
                                                       
{'loss': 1.7393, 'learning_rate': 0.0001211165970375997, 'epoch': 0.61}

 61%|██████    | 1670/2733 [6:14:12<3:57:07, 13.38s/it]
 61%|██████    | 1671/2733 [6:14:25<3:57:20, 13.41s/it]
                                                       
{'loss': 1.7168, 'learning_rate': 0.00012100265856437524, 'epoch': 0.61}

 61%|██████    | 1671/2733 [6:14:25<3:57:20, 13.41s/it]
 61%|██████    | 1672/2733 [6:14:39<3:59:54, 13.57s/it]
                                                       
{'loss': 1.7622, 'learning_rate': 0.00012088872009115078, 'epoch': 0.61}

 61%|██████    | 1672/2733 [6:14:39<3:59:54, 13.57s/it]
 61%|██████    | 1673/2733 [6:14:53<4:00:43, 13.63s/it]
                                                       
{'loss': 1.7739, 'learning_rate': 0.0001207747816179263, 'epoch': 0.61}

 61%|██████    | 1673/2733 [6:14:53<4:00:43, 13.63s/it]
 61%|██████▏   | 1674/2733 [6:15:07<4:00:59, 13.65s/it]
                                                       
{'loss': 1.7177, 'learning_rate': 0.00012066084314470184, 'epoch': 0.61}

 61%|██████▏   | 1674/2733 [6:15:07<4:00:59, 13.65s/it]
 61%|██████▏   | 1675/2733 [6:15:21<4:01:15, 13.68s/it]
                                                       
{'loss': 1.6936, 'learning_rate': 0.00012054690467147739, 'epoch': 0.61}

 61%|██████▏   | 1675/2733 [6:15:21<4:01:15, 13.68s/it]
 61%|██████▏   | 1676/2733 [6:15:33<3:57:13, 13.47s/it]
                                                       
{'loss': 1.6564, 'learning_rate': 0.00012043296619825293, 'epoch': 0.61}

 61%|██████▏   | 1676/2733 [6:15:34<3:57:13, 13.47s/it]
 61%|██████▏   | 1677/2733 [6:15:47<3:56:50, 13.46s/it]
                                                       
{'loss': 1.6922, 'learning_rate': 0.00012031902772502847, 'epoch': 0.61}

 61%|██████▏   | 1677/2733 [6:15:47<3:56:50, 13.46s/it]
 61%|██████▏   | 1678/2733 [6:16:01<3:59:11, 13.60s/it]
                                                       
{'loss': 1.6847, 'learning_rate': 0.00012020508925180402, 'epoch': 0.61}

 61%|██████▏   | 1678/2733 [6:16:01<3:59:11, 13.60s/it]
 61%|██████▏   | 1679/2733 [6:16:14<3:54:01, 13.32s/it]
                                                       
{'loss': 1.6893, 'learning_rate': 0.00012009115077857956, 'epoch': 0.61}

 61%|██████▏   | 1679/2733 [6:16:14<3:54:01, 13.32s/it]
 61%|██████▏   | 1680/2733 [6:16:27<3:54:17, 13.35s/it]
                                                       
{'loss': 1.698, 'learning_rate': 0.00011997721230535509, 'epoch': 0.61}

 61%|██████▏   | 1680/2733 [6:16:27<3:54:17, 13.35s/it]
 62%|██████▏   | 1681/2733 [6:16:40<3:52:01, 13.23s/it]
                                                       
{'loss': 1.58, 'learning_rate': 0.00011986327383213064, 'epoch': 0.62}

 62%|██████▏   | 1681/2733 [6:16:40<3:52:01, 13.23s/it]
 62%|██████▏   | 1682/2733 [6:16:54<3:54:48, 13.41s/it]
                                                       
{'loss': 1.7627, 'learning_rate': 0.00011974933535890618, 'epoch': 0.62}

 62%|██████▏   | 1682/2733 [6:16:54<3:54:48, 13.41s/it]
 62%|██████▏   | 1683/2733 [6:17:07<3:54:25, 13.40s/it]
                                                       
{'loss': 1.5909, 'learning_rate': 0.00011963539688568172, 'epoch': 0.62}

 62%|██████▏   | 1683/2733 [6:17:07<3:54:25, 13.40s/it]
 62%|██████▏   | 1684/2733 [6:17:20<3:52:32, 13.30s/it]
                                                       
{'loss': 1.6227, 'learning_rate': 0.00011952145841245727, 'epoch': 0.62}

 62%|██████▏   | 1684/2733 [6:17:20<3:52:32, 13.30s/it]
 62%|██████▏   | 1685/2733 [6:17:34<3:56:53, 13.56s/it]
                                                       
{'loss': 1.6704, 'learning_rate': 0.00011940751993923281, 'epoch': 0.62}

 62%|██████▏   | 1685/2733 [6:17:34<3:56:53, 13.56s/it]
 62%|██████▏   | 1686/2733 [6:17:48<3:56:16, 13.54s/it]
                                                       
{'loss': 1.711, 'learning_rate': 0.00011929358146600836, 'epoch': 0.62}

 62%|██████▏   | 1686/2733 [6:17:48<3:56:16, 13.54s/it]
 62%|██████▏   | 1687/2733 [6:18:02<3:56:43, 13.58s/it]
                                                       
{'loss': 1.7306, 'learning_rate': 0.00011917964299278389, 'epoch': 0.62}

 62%|██████▏   | 1687/2733 [6:18:02<3:56:43, 13.58s/it]
 62%|██████▏   | 1688/2733 [6:18:15<3:53:57, 13.43s/it]
                                                       
{'loss': 1.579, 'learning_rate': 0.00011906570451955943, 'epoch': 0.62}

 62%|██████▏   | 1688/2733 [6:18:15<3:53:57, 13.43s/it]
 62%|██████▏   | 1689/2733 [6:18:27<3:49:47, 13.21s/it]
                                                       
{'loss': 1.6456, 'learning_rate': 0.00011895176604633496, 'epoch': 0.62}

 62%|██████▏   | 1689/2733 [6:18:27<3:49:47, 13.21s/it]
 62%|██████▏   | 1690/2733 [6:18:41<3:49:43, 13.22s/it]
                                                       
{'loss': 1.7793, 'learning_rate': 0.0001188378275731105, 'epoch': 0.62}

 62%|██████▏   | 1690/2733 [6:18:41<3:49:43, 13.22s/it]
 62%|██████▏   | 1691/2733 [6:18:54<3:50:37, 13.28s/it]
                                                       
{'loss': 1.7569, 'learning_rate': 0.00011872388909988605, 'epoch': 0.62}

 62%|██████▏   | 1691/2733 [6:18:54<3:50:37, 13.28s/it]
 62%|██████▏   | 1692/2733 [6:19:08<3:54:15, 13.50s/it]
                                                       
{'loss': 1.807, 'learning_rate': 0.00011860995062666159, 'epoch': 0.62}

 62%|██████▏   | 1692/2733 [6:19:08<3:54:15, 13.50s/it]
 62%|██████▏   | 1693/2733 [6:19:22<3:55:22, 13.58s/it]
                                                       
{'loss': 1.6743, 'learning_rate': 0.00011849601215343712, 'epoch': 0.62}

 62%|██████▏   | 1693/2733 [6:19:22<3:55:22, 13.58s/it]
 62%|██████▏   | 1694/2733 [6:19:34<3:50:39, 13.32s/it]
                                                       
{'loss': 1.6575, 'learning_rate': 0.00011838207368021267, 'epoch': 0.62}

 62%|██████▏   | 1694/2733 [6:19:34<3:50:39, 13.32s/it]
 62%|██████▏   | 1695/2733 [6:19:48<3:53:09, 13.48s/it]
                                                       
{'loss': 1.7278, 'learning_rate': 0.00011826813520698821, 'epoch': 0.62}

 62%|██████▏   | 1695/2733 [6:19:48<3:53:09, 13.48s/it]
 62%|██████▏   | 1696/2733 [6:20:02<3:54:04, 13.54s/it]
                                                       
{'loss': 1.5754, 'learning_rate': 0.00011815419673376375, 'epoch': 0.62}

 62%|██████▏   | 1696/2733 [6:20:02<3:54:04, 13.54s/it]
 62%|██████▏   | 1697/2733 [6:20:16<3:55:55, 13.66s/it]
                                                       
{'loss': 1.6645, 'learning_rate': 0.0001180402582605393, 'epoch': 0.62}

 62%|██████▏   | 1697/2733 [6:20:16<3:55:55, 13.66s/it]
 62%|██████▏   | 1698/2733 [6:20:30<3:57:31, 13.77s/it]
                                                       
{'loss': 1.603, 'learning_rate': 0.00011792631978731484, 'epoch': 0.62}

 62%|██████▏   | 1698/2733 [6:20:30<3:57:31, 13.77s/it]
 62%|██████▏   | 1699/2733 [6:20:44<3:58:30, 13.84s/it]
                                                       
{'loss': 1.5831, 'learning_rate': 0.00011781238131409039, 'epoch': 0.62}

 62%|██████▏   | 1699/2733 [6:20:44<3:58:30, 13.84s/it]
 62%|██████▏   | 1700/2733 [6:20:58<3:57:10, 13.78s/it]
                                                       
{'loss': 1.754, 'learning_rate': 0.00011769844284086592, 'epoch': 0.62}

 62%|██████▏   | 1700/2733 [6:20:58<3:57:10, 13.78s/it]
 62%|██████▏   | 1701/2733 [6:21:10<3:51:18, 13.45s/it]
                                                       
{'loss': 1.6088, 'learning_rate': 0.00011758450436764146, 'epoch': 0.62}

 62%|██████▏   | 1701/2733 [6:21:10<3:51:18, 13.45s/it]
 62%|██████▏   | 1702/2733 [6:21:24<3:52:59, 13.56s/it]
                                                       
{'loss': 1.5699, 'learning_rate': 0.000117470565894417, 'epoch': 0.62}

 62%|██████▏   | 1702/2733 [6:21:24<3:52:59, 13.56s/it]
 62%|██████▏   | 1703/2733 [6:21:37<3:49:06, 13.35s/it]
                                                       
{'loss': 1.6388, 'learning_rate': 0.00011735662742119255, 'epoch': 0.62}

 62%|██████▏   | 1703/2733 [6:21:37<3:49:06, 13.35s/it]
 62%|██████▏   | 1704/2733 [6:21:51<3:53:06, 13.59s/it]
                                                       
{'loss': 1.7466, 'learning_rate': 0.00011724268894796809, 'epoch': 0.62}

 62%|██████▏   | 1704/2733 [6:21:51<3:53:06, 13.59s/it]
 62%|██████▏   | 1705/2733 [6:22:04<3:48:33, 13.34s/it]
                                                       
{'loss': 1.7193, 'learning_rate': 0.00011712875047474364, 'epoch': 0.62}

 62%|██████▏   | 1705/2733 [6:22:04<3:48:33, 13.34s/it]
 62%|██████▏   | 1706/2733 [6:22:17<3:48:01, 13.32s/it]
                                                       
{'loss': 1.6686, 'learning_rate': 0.00011701481200151918, 'epoch': 0.62}

 62%|██████▏   | 1706/2733 [6:22:17<3:48:01, 13.32s/it]
 62%|██████▏   | 1707/2733 [6:22:29<3:42:42, 13.02s/it]
                                                       
{'loss': 1.683, 'learning_rate': 0.0001169008735282947, 'epoch': 0.62}

 62%|██████▏   | 1707/2733 [6:22:29<3:42:42, 13.02s/it]
 62%|██████▏   | 1708/2733 [6:22:43<3:46:03, 13.23s/it]
                                                       
{'loss': 1.7311, 'learning_rate': 0.00011678693505507024, 'epoch': 0.62}

 62%|██████▏   | 1708/2733 [6:22:43<3:46:03, 13.23s/it]
 63%|██████▎   | 1709/2733 [6:22:57<3:46:28, 13.27s/it]
                                                       
{'loss': 1.6532, 'learning_rate': 0.00011667299658184578, 'epoch': 0.63}

 63%|██████▎   | 1709/2733 [6:22:57<3:46:28, 13.27s/it]
 63%|██████▎   | 1710/2733 [6:23:10<3:45:19, 13.22s/it]
                                                       
{'loss': 1.6509, 'learning_rate': 0.00011655905810862133, 'epoch': 0.63}

 63%|██████▎   | 1710/2733 [6:23:10<3:45:19, 13.22s/it]
 63%|██████▎   | 1711/2733 [6:23:23<3:47:35, 13.36s/it]
                                                       
{'loss': 1.72, 'learning_rate': 0.00011644511963539687, 'epoch': 0.63}

 63%|██████▎   | 1711/2733 [6:23:23<3:47:35, 13.36s/it]
 63%|██████▎   | 1712/2733 [6:23:37<3:47:16, 13.36s/it]
                                                       
{'loss': 1.6958, 'learning_rate': 0.00011633118116217242, 'epoch': 0.63}

 63%|██████▎   | 1712/2733 [6:23:37<3:47:16, 13.36s/it]
 63%|██████▎   | 1713/2733 [6:23:50<3:46:12, 13.31s/it]
                                                       
{'loss': 1.6694, 'learning_rate': 0.00011621724268894796, 'epoch': 0.63}

 63%|██████▎   | 1713/2733 [6:23:50<3:46:12, 13.31s/it]
 63%|██████▎   | 1714/2733 [6:24:04<3:47:59, 13.42s/it]
                                                       
{'loss': 1.7116, 'learning_rate': 0.00011610330421572349, 'epoch': 0.63}

 63%|██████▎   | 1714/2733 [6:24:04<3:47:59, 13.42s/it]
 63%|██████▎   | 1715/2733 [6:24:17<3:45:36, 13.30s/it]
                                                       
{'loss': 1.6534, 'learning_rate': 0.00011598936574249903, 'epoch': 0.63}

 63%|██████▎   | 1715/2733 [6:24:17<3:45:36, 13.30s/it]
 63%|██████▎   | 1716/2733 [6:24:30<3:46:21, 13.35s/it]
                                                       
{'loss': 1.6888, 'learning_rate': 0.00011587542726927458, 'epoch': 0.63}

 63%|██████▎   | 1716/2733 [6:24:30<3:46:21, 13.35s/it]
 63%|██████▎   | 1717/2733 [6:24:44<3:47:50, 13.46s/it]
                                                       
{'loss': 1.7559, 'learning_rate': 0.00011576148879605012, 'epoch': 0.63}

 63%|██████▎   | 1717/2733 [6:24:44<3:47:50, 13.46s/it]
 63%|██████▎   | 1718/2733 [6:24:57<3:46:46, 13.41s/it]
                                                       
{'loss': 1.6756, 'learning_rate': 0.00011564755032282567, 'epoch': 0.63}

 63%|██████▎   | 1718/2733 [6:24:57<3:46:46, 13.41s/it]
 63%|██████▎   | 1719/2733 [6:25:10<3:45:37, 13.35s/it]
                                                       
{'loss': 1.7489, 'learning_rate': 0.00011553361184960121, 'epoch': 0.63}

 63%|██████▎   | 1719/2733 [6:25:10<3:45:37, 13.35s/it]
 63%|██████▎   | 1720/2733 [6:25:24<3:46:41, 13.43s/it]
                                                       
{'loss': 1.7207, 'learning_rate': 0.00011541967337637675, 'epoch': 0.63}

 63%|██████▎   | 1720/2733 [6:25:24<3:46:41, 13.43s/it]
 63%|██████▎   | 1721/2733 [6:25:37<3:44:21, 13.30s/it]
                                                       
{'loss': 1.7037, 'learning_rate': 0.00011530573490315228, 'epoch': 0.63}

 63%|██████▎   | 1721/2733 [6:25:37<3:44:21, 13.30s/it]
 63%|██████▎   | 1722/2733 [6:25:50<3:42:16, 13.19s/it]
                                                       
{'loss': 1.6934, 'learning_rate': 0.00011519179642992783, 'epoch': 0.63}

 63%|██████▎   | 1722/2733 [6:25:50<3:42:16, 13.19s/it]
 63%|██████▎   | 1723/2733 [6:26:03<3:42:03, 13.19s/it]
                                                       
{'loss': 1.6824, 'learning_rate': 0.00011507785795670337, 'epoch': 0.63}

 63%|██████▎   | 1723/2733 [6:26:03<3:42:03, 13.19s/it]
 63%|██████▎   | 1724/2733 [6:26:16<3:42:43, 13.24s/it]
                                                       
{'loss': 1.5937, 'learning_rate': 0.00011496391948347892, 'epoch': 0.63}

 63%|██████▎   | 1724/2733 [6:26:16<3:42:43, 13.24s/it]
 63%|██████▎   | 1725/2733 [6:26:29<3:41:44, 13.20s/it]
                                                       
{'loss': 1.5828, 'learning_rate': 0.00011484998101025446, 'epoch': 0.63}

 63%|██████▎   | 1725/2733 [6:26:29<3:41:44, 13.20s/it]
 63%|██████▎   | 1726/2733 [6:26:43<3:43:08, 13.30s/it]
                                                       
{'loss': 1.6354, 'learning_rate': 0.00011473604253703, 'epoch': 0.63}

 63%|██████▎   | 1726/2733 [6:26:43<3:43:08, 13.30s/it]
 63%|██████▎   | 1727/2733 [6:26:56<3:41:38, 13.22s/it]
                                                       
{'loss': 1.8252, 'learning_rate': 0.00011462210406380555, 'epoch': 0.63}

 63%|██████▎   | 1727/2733 [6:26:56<3:41:38, 13.22s/it]
 63%|██████▎   | 1728/2733 [6:27:10<3:43:16, 13.33s/it]
                                                       
{'loss': 1.8325, 'learning_rate': 0.00011450816559058106, 'epoch': 0.63}

 63%|██████▎   | 1728/2733 [6:27:10<3:43:16, 13.33s/it]
 63%|██████▎   | 1729/2733 [6:27:23<3:44:00, 13.39s/it]
                                                       
{'loss': 1.7099, 'learning_rate': 0.00011439422711735661, 'epoch': 0.63}

 63%|██████▎   | 1729/2733 [6:27:23<3:44:00, 13.39s/it]
 63%|██████▎   | 1730/2733 [6:27:37<3:44:10, 13.41s/it]
                                                       
{'loss': 1.7421, 'learning_rate': 0.00011428028864413215, 'epoch': 0.63}

 63%|██████▎   | 1730/2733 [6:27:37<3:44:10, 13.41s/it]
 63%|██████▎   | 1731/2733 [6:27:50<3:44:59, 13.47s/it]
                                                       
{'loss': 1.6946, 'learning_rate': 0.0001141663501709077, 'epoch': 0.63}

 63%|██████▎   | 1731/2733 [6:27:50<3:44:59, 13.47s/it]
 63%|██████▎   | 1732/2733 [6:28:04<3:48:20, 13.69s/it]
                                                       
{'loss': 1.6767, 'learning_rate': 0.00011405241169768324, 'epoch': 0.63}

 63%|██████▎   | 1732/2733 [6:28:04<3:48:20, 13.69s/it]
 63%|██████▎   | 1733/2733 [6:28:18<3:49:01, 13.74s/it]
                                                       
{'loss': 1.7518, 'learning_rate': 0.00011393847322445878, 'epoch': 0.63}

 63%|██████▎   | 1733/2733 [6:28:18<3:49:01, 13.74s/it]
 63%|██████▎   | 1734/2733 [6:28:32<3:48:13, 13.71s/it]
                                                       
{'loss': 1.6859, 'learning_rate': 0.00011382453475123433, 'epoch': 0.63}

 63%|██████▎   | 1734/2733 [6:28:32<3:48:13, 13.71s/it]
 63%|██████▎   | 1735/2733 [6:28:46<3:48:11, 13.72s/it]
                                                       
{'loss': 1.6409, 'learning_rate': 0.00011371059627800986, 'epoch': 0.63}

 63%|██████▎   | 1735/2733 [6:28:46<3:48:11, 13.72s/it]
 64%|██████▎   | 1736/2733 [6:28:59<3:44:43, 13.52s/it]
                                                       
{'loss': 1.5505, 'learning_rate': 0.0001135966578047854, 'epoch': 0.64}

 64%|██████▎   | 1736/2733 [6:28:59<3:44:43, 13.52s/it]
 64%|██████▎   | 1737/2733 [6:29:12<3:41:15, 13.33s/it]
                                                       
{'loss': 1.6082, 'learning_rate': 0.00011348271933156095, 'epoch': 0.64}

 64%|██████▎   | 1737/2733 [6:29:12<3:41:15, 13.33s/it]
 64%|██████▎   | 1738/2733 [6:29:25<3:39:40, 13.25s/it]
                                                       
{'loss': 1.6064, 'learning_rate': 0.00011336878085833649, 'epoch': 0.64}

 64%|██████▎   | 1738/2733 [6:29:25<3:39:40, 13.25s/it]
 64%|██████▎   | 1739/2733 [6:29:38<3:41:59, 13.40s/it]
                                                       
{'loss': 1.6302, 'learning_rate': 0.00011325484238511203, 'epoch': 0.64}

 64%|██████▎   | 1739/2733 [6:29:38<3:41:59, 13.40s/it]
 64%|██████▎   | 1740/2733 [6:29:52<3:44:26, 13.56s/it]
                                                       
{'loss': 1.6391, 'learning_rate': 0.00011314090391188758, 'epoch': 0.64}

 64%|██████▎   | 1740/2733 [6:29:52<3:44:26, 13.56s/it]
 64%|██████▎   | 1741/2733 [6:30:07<3:47:32, 13.76s/it]
                                                       
{'loss': 1.7281, 'learning_rate': 0.00011302696543866312, 'epoch': 0.64}

 64%|██████▎   | 1741/2733 [6:30:07<3:47:32, 13.76s/it]
 64%|██████▎   | 1742/2733 [6:30:20<3:46:18, 13.70s/it]
                                                       
{'loss': 1.4933, 'learning_rate': 0.00011291302696543865, 'epoch': 0.64}

 64%|██████▎   | 1742/2733 [6:30:20<3:46:18, 13.70s/it]
 64%|██████▍   | 1743/2733 [6:30:34<3:45:32, 13.67s/it]
                                                       
{'loss': 1.6715, 'learning_rate': 0.0001127990884922142, 'epoch': 0.64}

 64%|██████▍   | 1743/2733 [6:30:34<3:45:32, 13.67s/it]
 64%|██████▍   | 1744/2733 [6:30:47<3:43:46, 13.58s/it]
                                                       
{'loss': 1.6655, 'learning_rate': 0.00011268515001898974, 'epoch': 0.64}

 64%|██████▍   | 1744/2733 [6:30:47<3:43:46, 13.58s/it]
 64%|██████▍   | 1745/2733 [6:31:00<3:41:28, 13.45s/it]
                                                       
{'loss': 1.7016, 'learning_rate': 0.00011257121154576528, 'epoch': 0.64}

 64%|██████▍   | 1745/2733 [6:31:00<3:41:28, 13.45s/it]
 64%|██████▍   | 1746/2733 [6:31:13<3:37:09, 13.20s/it]
                                                       
{'loss': 1.6966, 'learning_rate': 0.00011245727307254083, 'epoch': 0.64}

 64%|██████▍   | 1746/2733 [6:31:13<3:37:09, 13.20s/it]
 64%|██████▍   | 1747/2733 [6:31:26<3:36:36, 13.18s/it]
                                                       
{'loss': 1.6144, 'learning_rate': 0.00011234333459931637, 'epoch': 0.64}

 64%|██████▍   | 1747/2733 [6:31:26<3:36:36, 13.18s/it]
 64%|██████▍   | 1748/2733 [6:31:40<3:41:31, 13.49s/it]
                                                       
{'loss': 1.7261, 'learning_rate': 0.0001122293961260919, 'epoch': 0.64}

 64%|██████▍   | 1748/2733 [6:31:40<3:41:31, 13.49s/it]
 64%|██████▍   | 1749/2733 [6:31:53<3:38:09, 13.30s/it]
                                                       
{'loss': 1.5629, 'learning_rate': 0.00011211545765286743, 'epoch': 0.64}

 64%|██████▍   | 1749/2733 [6:31:53<3:38:09, 13.30s/it]
 64%|██████▍   | 1750/2733 [6:32:06<3:35:32, 13.16s/it]
                                                       
{'loss': 1.7503, 'learning_rate': 0.00011200151917964298, 'epoch': 0.64}

 64%|██████▍   | 1750/2733 [6:32:06<3:35:32, 13.16s/it]
 64%|██████▍   | 1751/2733 [6:32:20<3:38:54, 13.37s/it]
                                                       
{'loss': 1.7504, 'learning_rate': 0.00011188758070641852, 'epoch': 0.64}

 64%|██████▍   | 1751/2733 [6:32:20<3:38:54, 13.37s/it]
 64%|██████▍   | 1752/2733 [6:32:33<3:37:54, 13.33s/it]
                                                       
{'loss': 1.6033, 'learning_rate': 0.00011177364223319406, 'epoch': 0.64}

 64%|██████▍   | 1752/2733 [6:32:33<3:37:54, 13.33s/it]
 64%|██████▍   | 1753/2733 [6:32:46<3:34:23, 13.13s/it]
                                                       
{'loss': 1.6873, 'learning_rate': 0.00011165970375996961, 'epoch': 0.64}

 64%|██████▍   | 1753/2733 [6:32:46<3:34:23, 13.13s/it]
 64%|██████▍   | 1754/2733 [6:33:00<3:39:28, 13.45s/it]
                                                       
{'loss': 1.6729, 'learning_rate': 0.00011154576528674515, 'epoch': 0.64}

 64%|██████▍   | 1754/2733 [6:33:00<3:39:28, 13.45s/it]
 64%|██████▍   | 1755/2733 [6:33:13<3:39:05, 13.44s/it]
                                                       
{'loss': 1.6942, 'learning_rate': 0.0001114318268135207, 'epoch': 0.64}

 64%|██████▍   | 1755/2733 [6:33:13<3:39:05, 13.44s/it]
 64%|██████▍   | 1756/2733 [6:33:27<3:39:44, 13.49s/it]
                                                       
{'loss': 1.6342, 'learning_rate': 0.00011131788834029623, 'epoch': 0.64}

 64%|██████▍   | 1756/2733 [6:33:27<3:39:44, 13.49s/it]
 64%|██████▍   | 1757/2733 [6:33:40<3:39:17, 13.48s/it]
                                                       
{'loss': 1.6411, 'learning_rate': 0.00011120394986707177, 'epoch': 0.64}

 64%|██████▍   | 1757/2733 [6:33:40<3:39:17, 13.48s/it]
 64%|██████▍   | 1758/2733 [6:33:54<3:38:49, 13.47s/it]
                                                       
{'loss': 1.6586, 'learning_rate': 0.00011109001139384731, 'epoch': 0.64}

 64%|██████▍   | 1758/2733 [6:33:54<3:38:49, 13.47s/it]
 64%|██████▍   | 1759/2733 [6:34:08<3:40:21, 13.57s/it]
                                                       
{'loss': 1.7144, 'learning_rate': 0.00011097607292062286, 'epoch': 0.64}

 64%|██████▍   | 1759/2733 [6:34:08<3:40:21, 13.57s/it]
 64%|██████▍   | 1760/2733 [6:34:21<3:38:45, 13.49s/it]
                                                       
{'loss': 1.6487, 'learning_rate': 0.0001108621344473984, 'epoch': 0.64}

 64%|██████▍   | 1760/2733 [6:34:21<3:38:45, 13.49s/it]
 64%|██████▍   | 1761/2733 [6:34:35<3:40:50, 13.63s/it]
                                                       
{'loss': 1.8397, 'learning_rate': 0.00011074819597417395, 'epoch': 0.64}

 64%|██████▍   | 1761/2733 [6:34:35<3:40:50, 13.63s/it]
 64%|██████▍   | 1762/2733 [6:34:48<3:38:37, 13.51s/it]
                                                       
{'loss': 1.6448, 'learning_rate': 0.00011063425750094949, 'epoch': 0.64}

 64%|██████▍   | 1762/2733 [6:34:48<3:38:37, 13.51s/it]
 65%|██████▍   | 1763/2733 [6:35:01<3:36:07, 13.37s/it]
                                                       
{'loss': 1.6009, 'learning_rate': 0.000110520319027725, 'epoch': 0.65}

 65%|██████▍   | 1763/2733 [6:35:01<3:36:07, 13.37s/it]
 65%|██████▍   | 1764/2733 [6:35:14<3:34:21, 13.27s/it]
                                                       
{'loss': 1.6658, 'learning_rate': 0.00011040638055450055, 'epoch': 0.65}

 65%|██████▍   | 1764/2733 [6:35:14<3:34:21, 13.27s/it]
 65%|██████▍   | 1765/2733 [6:35:28<3:36:22, 13.41s/it]
                                                       
{'loss': 1.6889, 'learning_rate': 0.0001102924420812761, 'epoch': 0.65}

 65%|██████▍   | 1765/2733 [6:35:28<3:36:22, 13.41s/it]
 65%|██████▍   | 1766/2733 [6:35:41<3:35:43, 13.39s/it]
                                                       
{'loss': 1.6751, 'learning_rate': 0.00011017850360805164, 'epoch': 0.65}

 65%|██████▍   | 1766/2733 [6:35:41<3:35:43, 13.39s/it]
 65%|██████▍   | 1767/2733 [6:35:55<3:36:16, 13.43s/it]
                                                       
{'loss': 1.628, 'learning_rate': 0.00011006456513482718, 'epoch': 0.65}

 65%|██████▍   | 1767/2733 [6:35:55<3:36:16, 13.43s/it]
 65%|██████▍   | 1768/2733 [6:36:07<3:32:41, 13.22s/it]
                                                       
{'loss': 1.6415, 'learning_rate': 0.00010995062666160273, 'epoch': 0.65}

 65%|██████▍   | 1768/2733 [6:36:08<3:32:41, 13.22s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 65%|██████▍   | 1769/2733 [6:36:21<3:32:53, 13.25s/it]
                                                       
{'loss': 1.5596, 'learning_rate': 0.00010983668818837826, 'epoch': 0.65}

 65%|██████▍   | 1769/2733 [6:36:21<3:32:53, 13.25s/it]
 65%|██████▍   | 1770/2733 [6:36:34<3:32:54, 13.27s/it]
                                                       
{'loss': 1.66, 'learning_rate': 0.0001097227497151538, 'epoch': 0.65}

 65%|██████▍   | 1770/2733 [6:36:34<3:32:54, 13.27s/it]
 65%|██████▍   | 1771/2733 [6:36:47<3:33:04, 13.29s/it]
                                                       
{'loss': 1.6511, 'learning_rate': 0.00010960881124192934, 'epoch': 0.65}

 65%|██████▍   | 1771/2733 [6:36:47<3:33:04, 13.29s/it]
 65%|██████▍   | 1772/2733 [6:37:01<3:33:47, 13.35s/it]
                                                       
{'loss': 1.7109, 'learning_rate': 0.00010949487276870489, 'epoch': 0.65}

 65%|██████▍   | 1772/2733 [6:37:01<3:33:47, 13.35s/it]
 65%|██████▍   | 1773/2733 [6:37:14<3:32:46, 13.30s/it]
                                                       
{'loss': 1.6596, 'learning_rate': 0.00010938093429548043, 'epoch': 0.65}

 65%|██████▍   | 1773/2733 [6:37:14<3:32:46, 13.30s/it]
 65%|██████▍   | 1774/2733 [6:37:27<3:31:51, 13.26s/it]
                                                       
{'loss': 1.7466, 'learning_rate': 0.00010926699582225598, 'epoch': 0.65}

 65%|██████▍   | 1774/2733 [6:37:27<3:31:51, 13.26s/it]
 65%|██████▍   | 1775/2733 [6:37:40<3:29:13, 13.10s/it]
                                                       
{'loss': 1.6828, 'learning_rate': 0.00010915305734903152, 'epoch': 0.65}

 65%|██████▍   | 1775/2733 [6:37:40<3:29:13, 13.10s/it]
 65%|██████▍   | 1776/2733 [6:37:54<3:31:03, 13.23s/it]
                                                       
{'loss': 1.6248, 'learning_rate': 0.00010903911887580705, 'epoch': 0.65}

 65%|██████▍   | 1776/2733 [6:37:54<3:31:03, 13.23s/it]
 65%|██████▌   | 1777/2733 [6:38:06<3:29:28, 13.15s/it]
                                                       
{'loss': 1.6175, 'learning_rate': 0.0001089251804025826, 'epoch': 0.65}

 65%|██████▌   | 1777/2733 [6:38:07<3:29:28, 13.15s/it]
 65%|██████▌   | 1778/2733 [6:38:20<3:30:29, 13.22s/it]
                                                       
{'loss': 1.734, 'learning_rate': 0.00010881124192935814, 'epoch': 0.65}

 65%|██████▌   | 1778/2733 [6:38:20<3:30:29, 13.22s/it]
 65%|██████▌   | 1779/2733 [6:38:33<3:31:39, 13.31s/it]
                                                       
{'loss': 1.6441, 'learning_rate': 0.00010869730345613368, 'epoch': 0.65}

 65%|██████▌   | 1779/2733 [6:38:33<3:31:39, 13.31s/it]
 65%|██████▌   | 1780/2733 [6:38:47<3:31:06, 13.29s/it]
                                                       
{'loss': 1.6947, 'learning_rate': 0.00010858336498290923, 'epoch': 0.65}

 65%|██████▌   | 1780/2733 [6:38:47<3:31:06, 13.29s/it]
 65%|██████▌   | 1781/2733 [6:39:00<3:32:36, 13.40s/it]
                                                       
{'loss': 1.7567, 'learning_rate': 0.00010846942650968477, 'epoch': 0.65}

 65%|██████▌   | 1781/2733 [6:39:00<3:32:36, 13.40s/it]
 65%|██████▌   | 1782/2733 [6:39:14<3:34:41, 13.55s/it]
                                                       
{'loss': 1.6966, 'learning_rate': 0.00010835548803646031, 'epoch': 0.65}

 65%|██████▌   | 1782/2733 [6:39:14<3:34:41, 13.55s/it]
 65%|██████▌   | 1783/2733 [6:39:28<3:33:33, 13.49s/it]
                                                       
{'loss': 1.7301, 'learning_rate': 0.00010824154956323583, 'epoch': 0.65}

 65%|██████▌   | 1783/2733 [6:39:28<3:33:33, 13.49s/it]
 65%|██████▌   | 1784/2733 [6:39:41<3:32:08, 13.41s/it]
                                                       
{'loss': 1.6627, 'learning_rate': 0.00010812761109001137, 'epoch': 0.65}

 65%|██████▌   | 1784/2733 [6:39:41<3:32:08, 13.41s/it]
 65%|██████▌   | 1785/2733 [6:39:55<3:34:52, 13.60s/it]
                                                       
{'loss': 1.6325, 'learning_rate': 0.00010801367261678692, 'epoch': 0.65}

 65%|██████▌   | 1785/2733 [6:39:55<3:34:52, 13.60s/it]
 65%|██████▌   | 1786/2733 [6:40:09<3:36:29, 13.72s/it]
                                                       
{'loss': 1.5879, 'learning_rate': 0.00010789973414356246, 'epoch': 0.65}

 65%|██████▌   | 1786/2733 [6:40:09<3:36:29, 13.72s/it]
 65%|██████▌   | 1787/2733 [6:40:21<3:30:18, 13.34s/it]
                                                       
{'loss': 1.5735, 'learning_rate': 0.000107785795670338, 'epoch': 0.65}

 65%|██████▌   | 1787/2733 [6:40:21<3:30:18, 13.34s/it]
 65%|██████▌   | 1788/2733 [6:40:34<3:28:10, 13.22s/it]
                                                       
{'loss': 1.6421, 'learning_rate': 0.00010767185719711355, 'epoch': 0.65}

 65%|██████▌   | 1788/2733 [6:40:34<3:28:10, 13.22s/it]
 65%|██████▌   | 1789/2733 [6:40:48<3:29:35, 13.32s/it]
                                                       
{'loss': 1.6703, 'learning_rate': 0.0001075579187238891, 'epoch': 0.65}

 65%|██████▌   | 1789/2733 [6:40:48<3:29:35, 13.32s/it]
 65%|██████▌   | 1790/2733 [6:41:01<3:31:15, 13.44s/it]
                                                       
{'loss': 1.6389, 'learning_rate': 0.00010744398025066462, 'epoch': 0.65}

 65%|██████▌   | 1790/2733 [6:41:02<3:31:15, 13.44s/it]
 66%|██████▌   | 1791/2733 [6:41:15<3:31:53, 13.50s/it]
                                                       
{'loss': 1.6807, 'learning_rate': 0.00010733004177744017, 'epoch': 0.66}

 66%|██████▌   | 1791/2733 [6:41:15<3:31:53, 13.50s/it]
 66%|██████▌   | 1792/2733 [6:41:28<3:29:58, 13.39s/it]
                                                       
{'loss': 1.6669, 'learning_rate': 0.00010721610330421571, 'epoch': 0.66}

 66%|██████▌   | 1792/2733 [6:41:28<3:29:58, 13.39s/it]
 66%|██████▌   | 1793/2733 [6:41:42<3:33:24, 13.62s/it]
                                                       
{'loss': 1.6208, 'learning_rate': 0.00010710216483099126, 'epoch': 0.66}

 66%|██████▌   | 1793/2733 [6:41:42<3:33:24, 13.62s/it]
 66%|██████▌   | 1794/2733 [6:41:56<3:31:27, 13.51s/it]
                                                       
{'loss': 1.6943, 'learning_rate': 0.0001069882263577668, 'epoch': 0.66}

 66%|██████▌   | 1794/2733 [6:41:56<3:31:27, 13.51s/it]
 66%|██████▌   | 1795/2733 [6:42:10<3:34:16, 13.71s/it]
                                                       
{'loss': 1.6748, 'learning_rate': 0.00010687428788454234, 'epoch': 0.66}

 66%|██████▌   | 1795/2733 [6:42:10<3:34:16, 13.71s/it]
 66%|██████▌   | 1796/2733 [6:42:23<3:31:55, 13.57s/it]
                                                       
{'loss': 1.6836, 'learning_rate': 0.00010676034941131789, 'epoch': 0.66}

 66%|██████▌   | 1796/2733 [6:42:23<3:31:55, 13.57s/it]
 66%|██████▌   | 1797/2733 [6:42:36<3:29:10, 13.41s/it]
                                                       
{'loss': 1.6537, 'learning_rate': 0.00010664641093809342, 'epoch': 0.66}

 66%|██████▌   | 1797/2733 [6:42:36<3:29:10, 13.41s/it]
 66%|██████▌   | 1798/2733 [6:42:49<3:25:07, 13.16s/it]
                                                       
{'loss': 1.5997, 'learning_rate': 0.00010653247246486896, 'epoch': 0.66}

 66%|██████▌   | 1798/2733 [6:42:49<3:25:07, 13.16s/it]
 66%|██████▌   | 1799/2733 [6:43:02<3:24:02, 13.11s/it]
                                                       
{'loss': 1.6168, 'learning_rate': 0.0001064185339916445, 'epoch': 0.66}

 66%|██████▌   | 1799/2733 [6:43:02<3:24:02, 13.11s/it]
 66%|██████▌   | 1800/2733 [6:43:15<3:24:48, 13.17s/it]
                                                       
{'loss': 1.5318, 'learning_rate': 0.00010630459551842005, 'epoch': 0.66}

 66%|██████▌   | 1800/2733 [6:43:15<3:24:48, 13.17s/it]
 66%|██████▌   | 1801/2733 [6:43:28<3:24:51, 13.19s/it]
                                                       
{'loss': 1.6333, 'learning_rate': 0.00010619065704519559, 'epoch': 0.66}

 66%|██████▌   | 1801/2733 [6:43:28<3:24:51, 13.19s/it]
 66%|██████▌   | 1802/2733 [6:43:42<3:25:28, 13.24s/it]
                                                       
{'loss': 1.6468, 'learning_rate': 0.00010607671857197114, 'epoch': 0.66}

 66%|██████▌   | 1802/2733 [6:43:42<3:25:28, 13.24s/it]
 66%|██████▌   | 1803/2733 [6:43:54<3:22:50, 13.09s/it]
                                                       
{'loss': 1.7159, 'learning_rate': 0.00010596278009874668, 'epoch': 0.66}

 66%|██████▌   | 1803/2733 [6:43:54<3:22:50, 13.09s/it]
 66%|██████▌   | 1804/2733 [6:44:07<3:22:36, 13.09s/it]
                                                       
{'loss': 1.6649, 'learning_rate': 0.0001058488416255222, 'epoch': 0.66}

 66%|██████▌   | 1804/2733 [6:44:07<3:22:36, 13.09s/it]
 66%|██████▌   | 1805/2733 [6:44:21<3:25:08, 13.26s/it]
                                                       
{'loss': 1.7234, 'learning_rate': 0.00010573490315229774, 'epoch': 0.66}

 66%|██████▌   | 1805/2733 [6:44:21<3:25:08, 13.26s/it]
 66%|██████▌   | 1806/2733 [6:44:34<3:24:45, 13.25s/it]
                                                       
{'loss': 1.7369, 'learning_rate': 0.00010562096467907329, 'epoch': 0.66}

 66%|██████▌   | 1806/2733 [6:44:34<3:24:45, 13.25s/it]
 66%|██████▌   | 1807/2733 [6:44:48<3:26:55, 13.41s/it]
                                                       
{'loss': 1.712, 'learning_rate': 0.00010550702620584883, 'epoch': 0.66}

 66%|██████▌   | 1807/2733 [6:44:48<3:26:55, 13.41s/it]
 66%|██████▌   | 1808/2733 [6:45:01<3:23:34, 13.21s/it]
                                                       
{'loss': 1.5785, 'learning_rate': 0.00010539308773262437, 'epoch': 0.66}

 66%|██████▌   | 1808/2733 [6:45:01<3:23:34, 13.21s/it]
 66%|██████▌   | 1809/2733 [6:45:14<3:25:06, 13.32s/it]
                                                       
{'loss': 1.6345, 'learning_rate': 0.00010527914925939992, 'epoch': 0.66}

 66%|██████▌   | 1809/2733 [6:45:14<3:25:06, 13.32s/it]
 66%|██████▌   | 1810/2733 [6:45:29<3:29:04, 13.59s/it]
                                                       
{'loss': 1.7408, 'learning_rate': 0.00010516521078617546, 'epoch': 0.66}

 66%|██████▌   | 1810/2733 [6:45:29<3:29:04, 13.59s/it]
 66%|██████▋   | 1811/2733 [6:45:41<3:22:09, 13.16s/it]
                                                       
{'loss': 1.6117, 'learning_rate': 0.00010505127231295099, 'epoch': 0.66}

 66%|██████▋   | 1811/2733 [6:45:41<3:22:09, 13.16s/it]
 66%|██████▋   | 1812/2733 [6:45:54<3:21:35, 13.13s/it]
                                                       
{'loss': 1.5833, 'learning_rate': 0.00010493733383972654, 'epoch': 0.66}

 66%|██████▋   | 1812/2733 [6:45:54<3:21:35, 13.13s/it]
 66%|██████▋   | 1813/2733 [6:46:08<3:24:43, 13.35s/it]
                                                       
{'loss': 1.6113, 'learning_rate': 0.00010482339536650208, 'epoch': 0.66}

 66%|██████▋   | 1813/2733 [6:46:08<3:24:43, 13.35s/it]
 66%|██████▋   | 1814/2733 [6:46:21<3:26:06, 13.46s/it]
                                                       
{'loss': 1.6825, 'learning_rate': 0.00010470945689327762, 'epoch': 0.66}

 66%|██████▋   | 1814/2733 [6:46:21<3:26:06, 13.46s/it]
 66%|██████▋   | 1815/2733 [6:46:35<3:24:21, 13.36s/it]
                                                       
{'loss': 1.5461, 'learning_rate': 0.00010459551842005317, 'epoch': 0.66}

 66%|██████▋   | 1815/2733 [6:46:35<3:24:21, 13.36s/it]
 66%|██████▋   | 1816/2733 [6:46:48<3:24:21, 13.37s/it]
                                                       
{'loss': 1.7029, 'learning_rate': 0.00010448157994682871, 'epoch': 0.66}

 66%|██████▋   | 1816/2733 [6:46:48<3:24:21, 13.37s/it]
 66%|██████▋   | 1817/2733 [6:47:01<3:22:25, 13.26s/it]
                                                       
{'loss': 1.6266, 'learning_rate': 0.00010436764147360425, 'epoch': 0.66}

 66%|██████▋   | 1817/2733 [6:47:01<3:22:25, 13.26s/it]
 67%|██████▋   | 1818/2733 [6:47:14<3:22:03, 13.25s/it]
                                                       
{'loss': 1.6326, 'learning_rate': 0.00010425370300037979, 'epoch': 0.67}

 67%|██████▋   | 1818/2733 [6:47:14<3:22:03, 13.25s/it]
 67%|██████▋   | 1819/2733 [6:47:28<3:24:02, 13.39s/it]
                                                       
{'loss': 1.7079, 'learning_rate': 0.00010413976452715533, 'epoch': 0.67}

 67%|██████▋   | 1819/2733 [6:47:28<3:24:02, 13.39s/it]
 67%|██████▋   | 1820/2733 [6:47:42<3:26:44, 13.59s/it]
                                                       
{'loss': 1.6007, 'learning_rate': 0.00010402582605393087, 'epoch': 0.67}

 67%|██████▋   | 1820/2733 [6:47:42<3:26:44, 13.59s/it]
 67%|██████▋   | 1821/2733 [6:47:55<3:24:52, 13.48s/it]
                                                       
{'loss': 1.5965, 'learning_rate': 0.0001039118875807064, 'epoch': 0.67}

 67%|██████▋   | 1821/2733 [6:47:55<3:24:52, 13.48s/it]
 67%|██████▋   | 1822/2733 [6:48:08<3:21:12, 13.25s/it]
                                                       
{'loss': 1.5937, 'learning_rate': 0.00010379794910748195, 'epoch': 0.67}

 67%|██████▋   | 1822/2733 [6:48:08<3:21:12, 13.25s/it]
 67%|██████▋   | 1823/2733 [6:48:22<3:22:45, 13.37s/it]
                                                       
{'loss': 1.7542, 'learning_rate': 0.00010368401063425749, 'epoch': 0.67}

 67%|██████▋   | 1823/2733 [6:48:22<3:22:45, 13.37s/it]
 67%|██████▋   | 1824/2733 [6:48:35<3:23:30, 13.43s/it]
                                                       
{'loss': 1.5743, 'learning_rate': 0.00010357007216103303, 'epoch': 0.67}

 67%|██████▋   | 1824/2733 [6:48:35<3:23:30, 13.43s/it]
 67%|██████▋   | 1825/2733 [6:48:48<3:22:30, 13.38s/it]
                                                       
{'loss': 1.5589, 'learning_rate': 0.00010345613368780857, 'epoch': 0.67}

 67%|██████▋   | 1825/2733 [6:48:48<3:22:30, 13.38s/it]
 67%|██████▋   | 1826/2733 [6:49:02<3:21:24, 13.32s/it]
                                                       
{'loss': 1.6137, 'learning_rate': 0.00010334219521458411, 'epoch': 0.67}

 67%|██████▋   | 1826/2733 [6:49:02<3:21:24, 13.32s/it]
 67%|██████▋   | 1827/2733 [6:49:15<3:22:25, 13.41s/it]
                                                       
{'loss': 1.6332, 'learning_rate': 0.00010322825674135965, 'epoch': 0.67}

 67%|██████▋   | 1827/2733 [6:49:15<3:22:25, 13.41s/it]
 67%|██████▋   | 1828/2733 [6:49:29<3:24:37, 13.57s/it]
                                                       
{'loss': 1.6783, 'learning_rate': 0.0001031143182681352, 'epoch': 0.67}

 67%|██████▋   | 1828/2733 [6:49:29<3:24:37, 13.57s/it]
 67%|██████▋   | 1829/2733 [6:49:43<3:25:19, 13.63s/it]
                                                       
{'loss': 1.6953, 'learning_rate': 0.00010300037979491074, 'epoch': 0.67}

 67%|██████▋   | 1829/2733 [6:49:43<3:25:19, 13.63s/it]
 67%|██████▋   | 1830/2733 [6:49:57<3:25:42, 13.67s/it]
                                                       
{'loss': 1.656, 'learning_rate': 0.00010288644132168628, 'epoch': 0.67}

 67%|██████▋   | 1830/2733 [6:49:57<3:25:42, 13.67s/it]
 67%|██████▋   | 1831/2733 [6:50:09<3:21:30, 13.40s/it]
                                                       
{'loss': 1.6232, 'learning_rate': 0.00010277250284846183, 'epoch': 0.67}

 67%|██████▋   | 1831/2733 [6:50:09<3:21:30, 13.40s/it]
 67%|██████▋   | 1832/2733 [6:50:22<3:16:07, 13.06s/it]
                                                       
{'loss': 1.6125, 'learning_rate': 0.00010265856437523736, 'epoch': 0.67}

 67%|██████▋   | 1832/2733 [6:50:22<3:16:07, 13.06s/it]
 67%|██████▋   | 1833/2733 [6:50:35<3:15:49, 13.06s/it]
                                                       
{'loss': 1.5774, 'learning_rate': 0.0001025446259020129, 'epoch': 0.67}

 67%|██████▋   | 1833/2733 [6:50:35<3:15:49, 13.06s/it]
 67%|██████▋   | 1834/2733 [6:50:48<3:16:30, 13.12s/it]
                                                       
{'loss': 1.6838, 'learning_rate': 0.00010243068742878845, 'epoch': 0.67}

 67%|██████▋   | 1834/2733 [6:50:48<3:16:30, 13.12s/it]
 67%|██████▋   | 1835/2733 [6:51:01<3:17:28, 13.19s/it]
                                                       
{'loss': 1.6466, 'learning_rate': 0.00010231674895556399, 'epoch': 0.67}

 67%|██████▋   | 1835/2733 [6:51:01<3:17:28, 13.19s/it]
 67%|██████▋   | 1836/2733 [6:51:15<3:19:39, 13.35s/it]
                                                       
{'loss': 1.8113, 'learning_rate': 0.00010220281048233953, 'epoch': 0.67}

 67%|██████▋   | 1836/2733 [6:51:15<3:19:39, 13.35s/it]
 67%|██████▋   | 1837/2733 [6:51:28<3:19:20, 13.35s/it]
                                                       
{'loss': 1.6558, 'learning_rate': 0.00010208887200911508, 'epoch': 0.67}

 67%|██████▋   | 1837/2733 [6:51:28<3:19:20, 13.35s/it]
 67%|██████▋   | 1838/2733 [6:51:41<3:17:33, 13.24s/it]
                                                       
{'loss': 1.7014, 'learning_rate': 0.00010197493353589062, 'epoch': 0.67}

 67%|██████▋   | 1838/2733 [6:51:41<3:17:33, 13.24s/it]
 67%|██████▋   | 1839/2733 [6:51:55<3:18:04, 13.29s/it]
                                                       
{'loss': 1.7213, 'learning_rate': 0.00010186099506266614, 'epoch': 0.67}

 67%|██████▋   | 1839/2733 [6:51:55<3:18:04, 13.29s/it]
 67%|██████▋   | 1840/2733 [6:52:09<3:19:58, 13.44s/it]
                                                       
{'loss': 1.5695, 'learning_rate': 0.00010174705658944168, 'epoch': 0.67}

 67%|██████▋   | 1840/2733 [6:52:09<3:19:58, 13.44s/it]
 67%|██████▋   | 1841/2733 [6:52:22<3:18:11, 13.33s/it]
                                                       
{'loss': 1.7111, 'learning_rate': 0.00010163311811621723, 'epoch': 0.67}

 67%|██████▋   | 1841/2733 [6:52:22<3:18:11, 13.33s/it]
 67%|██████▋   | 1842/2733 [6:52:36<3:21:35, 13.58s/it]
                                                       
{'loss': 1.6183, 'learning_rate': 0.00010151917964299277, 'epoch': 0.67}

 67%|██████▋   | 1842/2733 [6:52:36<3:21:35, 13.58s/it]
 67%|██████▋   | 1843/2733 [6:52:49<3:19:50, 13.47s/it]
                                                       
{'loss': 1.7097, 'learning_rate': 0.00010140524116976831, 'epoch': 0.67}

 67%|██████▋   | 1843/2733 [6:52:49<3:19:50, 13.47s/it]
 67%|██████▋   | 1844/2733 [6:53:02<3:18:32, 13.40s/it]
                                                       
{'loss': 1.5776, 'learning_rate': 0.00010129130269654386, 'epoch': 0.67}

 67%|██████▋   | 1844/2733 [6:53:02<3:18:32, 13.40s/it]
 68%|██████▊   | 1845/2733 [6:53:15<3:17:23, 13.34s/it]
                                                       
{'loss': 1.5655, 'learning_rate': 0.00010117736422331939, 'epoch': 0.68}

 68%|██████▊   | 1845/2733 [6:53:16<3:17:23, 13.34s/it]
 68%|██████▊   | 1846/2733 [6:53:28<3:14:50, 13.18s/it]
                                                       
{'loss': 1.5984, 'learning_rate': 0.00010106342575009493, 'epoch': 0.68}

 68%|██████▊   | 1846/2733 [6:53:28<3:14:50, 13.18s/it]
 68%|██████▊   | 1847/2733 [6:53:42<3:18:55, 13.47s/it]
                                                       
{'loss': 1.696, 'learning_rate': 0.00010094948727687048, 'epoch': 0.68}

 68%|██████▊   | 1847/2733 [6:53:42<3:18:55, 13.47s/it]
 68%|██████▊   | 1848/2733 [6:53:57<3:21:41, 13.67s/it]
                                                       
{'loss': 1.6729, 'learning_rate': 0.00010083554880364602, 'epoch': 0.68}

 68%|██████▊   | 1848/2733 [6:53:57<3:21:41, 13.67s/it]
 68%|██████▊   | 1849/2733 [6:54:09<3:17:20, 13.39s/it]
                                                       
{'loss': 1.6482, 'learning_rate': 0.00010072161033042156, 'epoch': 0.68}

 68%|██████▊   | 1849/2733 [6:54:09<3:17:20, 13.39s/it]
 68%|██████▊   | 1850/2733 [6:54:23<3:18:14, 13.47s/it]
                                                       
{'loss': 1.7749, 'learning_rate': 0.00010060767185719711, 'epoch': 0.68}

 68%|██████▊   | 1850/2733 [6:54:23<3:18:14, 13.47s/it]
 68%|██████▊   | 1851/2733 [6:54:37<3:18:21, 13.49s/it]
                                                       
{'loss': 1.6635, 'learning_rate': 0.00010049373338397265, 'epoch': 0.68}

 68%|██████▊   | 1851/2733 [6:54:37<3:18:21, 13.49s/it]
 68%|██████▊   | 1852/2733 [6:54:50<3:17:50, 13.47s/it]
                                                       
{'loss': 1.7524, 'learning_rate': 0.00010037979491074818, 'epoch': 0.68}

 68%|██████▊   | 1852/2733 [6:54:50<3:17:50, 13.47s/it]
 68%|██████▊   | 1853/2733 [6:55:04<3:18:20, 13.52s/it]
                                                       
{'loss': 1.6391, 'learning_rate': 0.00010026585643752373, 'epoch': 0.68}

 68%|██████▊   | 1853/2733 [6:55:04<3:18:20, 13.52s/it]
 68%|██████▊   | 1854/2733 [6:55:17<3:16:01, 13.38s/it]
                                                       
{'loss': 1.5646, 'learning_rate': 0.00010015191796429927, 'epoch': 0.68}

 68%|██████▊   | 1854/2733 [6:55:17<3:16:01, 13.38s/it]
 68%|██████▊   | 1855/2733 [6:55:29<3:11:50, 13.11s/it]
                                                       
{'loss': 1.557, 'learning_rate': 0.00010003797949107481, 'epoch': 0.68}

 68%|██████▊   | 1855/2733 [6:55:29<3:11:50, 13.11s/it]
 68%|██████▊   | 1856/2733 [6:55:43<3:12:55, 13.20s/it]
                                                       
{'loss': 1.7336, 'learning_rate': 9.992404101785036e-05, 'epoch': 0.68}

 68%|██████▊   | 1856/2733 [6:55:43<3:12:55, 13.20s/it]
 68%|██████▊   | 1857/2733 [6:55:57<3:17:00, 13.49s/it]
                                                       
{'loss': 1.6395, 'learning_rate': 9.98101025446259e-05, 'epoch': 0.68}

 68%|██████▊   | 1857/2733 [6:55:57<3:17:00, 13.49s/it]
 68%|██████▊   | 1858/2733 [6:56:10<3:18:03, 13.58s/it]
                                                       
{'loss': 1.6747, 'learning_rate': 9.969616407140145e-05, 'epoch': 0.68}

 68%|██████▊   | 1858/2733 [6:56:11<3:18:03, 13.58s/it]
 68%|██████▊   | 1859/2733 [6:56:24<3:17:28, 13.56s/it]
                                                       
{'loss': 1.7244, 'learning_rate': 9.958222559817696e-05, 'epoch': 0.68}

 68%|██████▊   | 1859/2733 [6:56:24<3:17:28, 13.56s/it]
 68%|██████▊   | 1860/2733 [6:56:38<3:18:24, 13.64s/it]
                                                       
{'loss': 1.6343, 'learning_rate': 9.946828712495251e-05, 'epoch': 0.68}

 68%|██████▊   | 1860/2733 [6:56:38<3:18:24, 13.64s/it]
 68%|██████▊   | 1861/2733 [6:56:51<3:17:44, 13.61s/it]
                                                       
{'loss': 1.6706, 'learning_rate': 9.935434865172805e-05, 'epoch': 0.68}

 68%|██████▊   | 1861/2733 [6:56:51<3:17:44, 13.61s/it]
 68%|██████▊   | 1862/2733 [6:57:05<3:18:42, 13.69s/it]
                                                       
{'loss': 1.7522, 'learning_rate': 9.92404101785036e-05, 'epoch': 0.68}

 68%|██████▊   | 1862/2733 [6:57:05<3:18:42, 13.69s/it]
 68%|██████▊   | 1863/2733 [6:57:19<3:18:25, 13.68s/it]
                                                       
{'loss': 1.6331, 'learning_rate': 9.912647170527914e-05, 'epoch': 0.68}

 68%|██████▊   | 1863/2733 [6:57:19<3:18:25, 13.68s/it]
 68%|██████▊   | 1864/2733 [6:57:32<3:16:32, 13.57s/it]
                                                       
{'loss': 1.6684, 'learning_rate': 9.901253323205468e-05, 'epoch': 0.68}

 68%|██████▊   | 1864/2733 [6:57:32<3:16:32, 13.57s/it]
 68%|██████▊   | 1865/2733 [6:57:46<3:17:35, 13.66s/it]
                                                       
{'loss': 1.6635, 'learning_rate': 9.889859475883023e-05, 'epoch': 0.68}

 68%|██████▊   | 1865/2733 [6:57:46<3:17:35, 13.66s/it]
 68%|██████▊   | 1866/2733 [6:57:59<3:15:41, 13.54s/it]
                                                       
{'loss': 1.7288, 'learning_rate': 9.878465628560576e-05, 'epoch': 0.68}

 68%|██████▊   | 1866/2733 [6:57:59<3:15:41, 13.54s/it]
 68%|██████▊   | 1867/2733 [6:58:13<3:17:27, 13.68s/it]
                                                       
{'loss': 1.6501, 'learning_rate': 9.86707178123813e-05, 'epoch': 0.68}

 68%|██████▊   | 1867/2733 [6:58:13<3:17:27, 13.68s/it]
 68%|██████▊   | 1868/2733 [6:58:27<3:16:38, 13.64s/it]
                                                       
{'loss': 1.6219, 'learning_rate': 9.855677933915684e-05, 'epoch': 0.68}

 68%|██████▊   | 1868/2733 [6:58:27<3:16:38, 13.64s/it]
 68%|██████▊   | 1869/2733 [6:58:41<3:17:00, 13.68s/it]
                                                       
{'loss': 1.6467, 'learning_rate': 9.844284086593239e-05, 'epoch': 0.68}

 68%|██████▊   | 1869/2733 [6:58:41<3:17:00, 13.68s/it]
 68%|██████▊   | 1870/2733 [6:58:54<3:15:01, 13.56s/it]
                                                       
{'loss': 1.5528, 'learning_rate': 9.832890239270793e-05, 'epoch': 0.68}

 68%|██████▊   | 1870/2733 [6:58:54<3:15:01, 13.56s/it]
 68%|██████▊   | 1871/2733 [6:59:08<3:17:37, 13.76s/it]
                                                       
{'loss': 1.6908, 'learning_rate': 9.821496391948348e-05, 'epoch': 0.68}

 68%|██████▊   | 1871/2733 [6:59:08<3:17:37, 13.76s/it]
 68%|██████▊   | 1872/2733 [6:59:22<3:15:52, 13.65s/it]
                                                       
{'loss': 1.674, 'learning_rate': 9.810102544625902e-05, 'epoch': 0.68}

 68%|██████▊   | 1872/2733 [6:59:22<3:15:52, 13.65s/it]
 69%|██████▊   | 1873/2733 [6:59:35<3:14:07, 13.54s/it]
                                                       
{'loss': 1.7583, 'learning_rate': 9.798708697303455e-05, 'epoch': 0.69}

 69%|██████▊   | 1873/2733 [6:59:35<3:14:07, 13.54s/it]
 69%|██████▊   | 1874/2733 [6:59:48<3:14:03, 13.56s/it]
                                                       
{'loss': 1.6208, 'learning_rate': 9.78731484998101e-05, 'epoch': 0.69}

 69%|██████▊   | 1874/2733 [6:59:48<3:14:03, 13.56s/it]
 69%|██████▊   | 1875/2733 [7:00:03<3:16:27, 13.74s/it]
                                                       
{'loss': 1.6818, 'learning_rate': 9.775921002658564e-05, 'epoch': 0.69}

 69%|██████▊   | 1875/2733 [7:00:03<3:16:27, 13.74s/it]
 69%|██████▊   | 1876/2733 [7:00:15<3:10:23, 13.33s/it]
                                                       
{'loss': 1.6743, 'learning_rate': 9.764527155336118e-05, 'epoch': 0.69}

 69%|██████▊   | 1876/2733 [7:00:15<3:10:23, 13.33s/it]
 69%|██████▊   | 1877/2733 [7:00:27<3:05:47, 13.02s/it]
                                                       
{'loss': 1.6544, 'learning_rate': 9.753133308013673e-05, 'epoch': 0.69}

 69%|██████▊   | 1877/2733 [7:00:27<3:05:47, 13.02s/it]
 69%|██████▊   | 1878/2733 [7:00:41<3:09:44, 13.32s/it]
                                                       
{'loss': 1.6564, 'learning_rate': 9.741739460691227e-05, 'epoch': 0.69}

 69%|██████▊   | 1878/2733 [7:00:41<3:09:44, 13.32s/it]
 69%|██████▉   | 1879/2733 [7:00:54<3:08:11, 13.22s/it]
                                                       
{'loss': 1.6639, 'learning_rate': 9.730345613368781e-05, 'epoch': 0.69}

 69%|██████▉   | 1879/2733 [7:00:54<3:08:11, 13.22s/it]
 69%|██████▉   | 1880/2733 [7:01:07<3:07:20, 13.18s/it]
                                                       
{'loss': 1.7565, 'learning_rate': 9.718951766046333e-05, 'epoch': 0.69}

 69%|██████▉   | 1880/2733 [7:01:07<3:07:20, 13.18s/it]
 69%|██████▉   | 1881/2733 [7:01:21<3:08:38, 13.28s/it]
                                                       
{'loss': 1.7229, 'learning_rate': 9.707557918723887e-05, 'epoch': 0.69}

 69%|██████▉   | 1881/2733 [7:01:21<3:08:38, 13.28s/it]
 69%|██████▉   | 1882/2733 [7:01:35<3:09:48, 13.38s/it]
                                                       
{'loss': 1.6375, 'learning_rate': 9.696164071401442e-05, 'epoch': 0.69}

 69%|██████▉   | 1882/2733 [7:01:35<3:09:48, 13.38s/it]
 69%|██████▉   | 1883/2733 [7:01:48<3:09:06, 13.35s/it]
                                                       
{'loss': 1.6542, 'learning_rate': 9.684770224078996e-05, 'epoch': 0.69}

 69%|██████▉   | 1883/2733 [7:01:48<3:09:06, 13.35s/it]
 69%|██████▉   | 1884/2733 [7:02:01<3:08:08, 13.30s/it]
                                                       
{'loss': 1.7201, 'learning_rate': 9.67337637675655e-05, 'epoch': 0.69}

 69%|██████▉   | 1884/2733 [7:02:01<3:08:08, 13.30s/it]
 69%|██████▉   | 1885/2733 [7:02:14<3:06:51, 13.22s/it]
                                                       
{'loss': 1.7146, 'learning_rate': 9.661982529434105e-05, 'epoch': 0.69}

 69%|██████▉   | 1885/2733 [7:02:14<3:06:51, 13.22s/it]
 69%|██████▉   | 1886/2733 [7:02:28<3:08:33, 13.36s/it]
                                                       
{'loss': 1.607, 'learning_rate': 9.65058868211166e-05, 'epoch': 0.69}

 69%|██████▉   | 1886/2733 [7:02:28<3:08:33, 13.36s/it]
 69%|██████▉   | 1887/2733 [7:02:41<3:09:47, 13.46s/it]
                                                       
{'loss': 1.6907, 'learning_rate': 9.639194834789212e-05, 'epoch': 0.69}

 69%|██████▉   | 1887/2733 [7:02:41<3:09:47, 13.46s/it]
 69%|██████▉   | 1888/2733 [7:02:55<3:08:14, 13.37s/it]
                                                       
{'loss': 1.5969, 'learning_rate': 9.627800987466767e-05, 'epoch': 0.69}

 69%|██████▉   | 1888/2733 [7:02:55<3:08:14, 13.37s/it]
 69%|██████▉   | 1889/2733 [7:03:08<3:09:14, 13.45s/it]
                                                       
{'loss': 1.744, 'learning_rate': 9.616407140144321e-05, 'epoch': 0.69}

 69%|██████▉   | 1889/2733 [7:03:08<3:09:14, 13.45s/it]
 69%|██████▉   | 1890/2733 [7:03:22<3:08:47, 13.44s/it]
                                                       
{'loss': 1.661, 'learning_rate': 9.605013292821876e-05, 'epoch': 0.69}

 69%|██████▉   | 1890/2733 [7:03:22<3:08:47, 13.44s/it]
 69%|██████▉   | 1891/2733 [7:03:35<3:07:44, 13.38s/it]
                                                       
{'loss': 1.6423, 'learning_rate': 9.59361944549943e-05, 'epoch': 0.69}

 69%|██████▉   | 1891/2733 [7:03:35<3:07:44, 13.38s/it]
 69%|██████▉   | 1892/2733 [7:03:48<3:08:22, 13.44s/it]
                                                       
{'loss': 1.7697, 'learning_rate': 9.582225598176984e-05, 'epoch': 0.69}

 69%|██████▉   | 1892/2733 [7:03:48<3:08:22, 13.44s/it]
 69%|██████▉   | 1893/2733 [7:04:02<3:09:29, 13.54s/it]
                                                       
{'loss': 1.6814, 'learning_rate': 9.570831750854539e-05, 'epoch': 0.69}

 69%|██████▉   | 1893/2733 [7:04:02<3:09:29, 13.54s/it]
 69%|██████▉   | 1894/2733 [7:04:16<3:08:49, 13.50s/it]
                                                       
{'loss': 1.6615, 'learning_rate': 9.559437903532092e-05, 'epoch': 0.69}

 69%|██████▉   | 1894/2733 [7:04:16<3:08:49, 13.50s/it]
 69%|██████▉   | 1895/2733 [7:04:29<3:07:40, 13.44s/it]
                                                       
{'loss': 1.6948, 'learning_rate': 9.548044056209645e-05, 'epoch': 0.69}

 69%|██████▉   | 1895/2733 [7:04:29<3:07:40, 13.44s/it]
 69%|██████▉   | 1896/2733 [7:04:42<3:06:00, 13.33s/it]
                                                       
{'loss': 1.7279, 'learning_rate': 9.536650208887199e-05, 'epoch': 0.69}

 69%|██████▉   | 1896/2733 [7:04:42<3:06:00, 13.33s/it]
 69%|██████▉   | 1897/2733 [7:04:55<3:04:31, 13.24s/it]
                                                       
{'loss': 1.6844, 'learning_rate': 9.525256361564754e-05, 'epoch': 0.69}

 69%|██████▉   | 1897/2733 [7:04:55<3:04:31, 13.24s/it]
 69%|██████▉   | 1898/2733 [7:05:08<3:04:37, 13.27s/it]
                                                       
{'loss': 1.6976, 'learning_rate': 9.513862514242308e-05, 'epoch': 0.69}

 69%|██████▉   | 1898/2733 [7:05:08<3:04:37, 13.27s/it]
 69%|██████▉   | 1899/2733 [7:05:21<3:02:53, 13.16s/it]
                                                       
{'loss': 1.6581, 'learning_rate': 9.502468666919862e-05, 'epoch': 0.69}

 69%|██████▉   | 1899/2733 [7:05:21<3:02:53, 13.16s/it]
 70%|██████▉   | 1900/2733 [7:05:34<3:02:49, 13.17s/it]
                                                       
{'loss': 1.7219, 'learning_rate': 9.491074819597417e-05, 'epoch': 0.7}

 70%|██████▉   | 1900/2733 [7:05:34<3:02:49, 13.17s/it]
 70%|██████▉   | 1901/2733 [7:05:47<3:02:10, 13.14s/it]
                                                       
{'loss': 1.5944, 'learning_rate': 9.47968097227497e-05, 'epoch': 0.7}

 70%|██████▉   | 1901/2733 [7:05:48<3:02:10, 13.14s/it]
 70%|██████▉   | 1902/2733 [7:06:01<3:02:16, 13.16s/it]
                                                       
{'loss': 1.638, 'learning_rate': 9.468287124952524e-05, 'epoch': 0.7}

 70%|██████▉   | 1902/2733 [7:06:01<3:02:16, 13.16s/it]
 70%|██████▉   | 1903/2733 [7:06:14<3:03:54, 13.29s/it]
                                                       
{'loss': 1.7366, 'learning_rate': 9.456893277630079e-05, 'epoch': 0.7}

 70%|██████▉   | 1903/2733 [7:06:14<3:03:54, 13.29s/it]
 70%|██████▉   | 1904/2733 [7:06:28<3:03:16, 13.26s/it]
                                                       
{'loss': 1.6184, 'learning_rate': 9.445499430307633e-05, 'epoch': 0.7}

 70%|██████▉   | 1904/2733 [7:06:28<3:03:16, 13.26s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 70%|██████▉   | 1905/2733 [7:06:41<3:02:48, 13.25s/it]
                                                       
{'loss': 1.7204, 'learning_rate': 9.434105582985187e-05, 'epoch': 0.7}

 70%|██████▉   | 1905/2733 [7:06:41<3:02:48, 13.25s/it]
 70%|██████▉   | 1906/2733 [7:06:54<3:02:37, 13.25s/it]
                                                       
{'loss': 1.6774, 'learning_rate': 9.422711735662742e-05, 'epoch': 0.7}

 70%|██████▉   | 1906/2733 [7:06:54<3:02:37, 13.25s/it]
 70%|██████▉   | 1907/2733 [7:07:07<3:01:00, 13.15s/it]
                                                       
{'loss': 1.6262, 'learning_rate': 9.411317888340296e-05, 'epoch': 0.7}

 70%|██████▉   | 1907/2733 [7:07:07<3:01:00, 13.15s/it]
 70%|██████▉   | 1908/2733 [7:07:20<3:02:07, 13.25s/it]
                                                       
{'loss': 1.6431, 'learning_rate': 9.399924041017849e-05, 'epoch': 0.7}

 70%|██████▉   | 1908/2733 [7:07:20<3:02:07, 13.25s/it]
 70%|██████▉   | 1909/2733 [7:07:33<2:59:16, 13.05s/it]
                                                       
{'loss': 1.5972, 'learning_rate': 9.388530193695404e-05, 'epoch': 0.7}

 70%|██████▉   | 1909/2733 [7:07:33<2:59:16, 13.05s/it]
 70%|██████▉   | 1910/2733 [7:07:47<3:01:44, 13.25s/it]
                                                       
{'loss': 1.6412, 'learning_rate': 9.377136346372958e-05, 'epoch': 0.7}

 70%|██████▉   | 1910/2733 [7:07:47<3:01:44, 13.25s/it]
 70%|██████▉   | 1911/2733 [7:08:00<3:00:45, 13.19s/it]
                                                       
{'loss': 1.6649, 'learning_rate': 9.365742499050512e-05, 'epoch': 0.7}

 70%|██████▉   | 1911/2733 [7:08:00<3:00:45, 13.19s/it]
 70%|██████▉   | 1912/2733 [7:08:12<2:58:11, 13.02s/it]
                                                       
{'loss': 1.5877, 'learning_rate': 9.354348651728067e-05, 'epoch': 0.7}

 70%|██████▉   | 1912/2733 [7:08:12<2:58:11, 13.02s/it]
 70%|██████▉   | 1913/2733 [7:08:26<2:58:59, 13.10s/it]
                                                       
{'loss': 1.6457, 'learning_rate': 9.342954804405621e-05, 'epoch': 0.7}

 70%|██████▉   | 1913/2733 [7:08:26<2:58:59, 13.10s/it]
 70%|███████   | 1914/2733 [7:08:39<2:58:37, 13.09s/it]
                                                       
{'loss': 1.6963, 'learning_rate': 9.331560957083176e-05, 'epoch': 0.7}

 70%|███████   | 1914/2733 [7:08:39<2:58:37, 13.09s/it]
 70%|███████   | 1915/2733 [7:08:52<3:01:11, 13.29s/it]
                                                       
{'loss': 1.7004, 'learning_rate': 9.320167109760727e-05, 'epoch': 0.7}

 70%|███████   | 1915/2733 [7:08:52<3:01:11, 13.29s/it]
 70%|███████   | 1916/2733 [7:09:06<3:02:13, 13.38s/it]
                                                       
{'loss': 1.6406, 'learning_rate': 9.308773262438282e-05, 'epoch': 0.7}

 70%|███████   | 1916/2733 [7:09:06<3:02:13, 13.38s/it]
 70%|███████   | 1917/2733 [7:09:19<3:02:17, 13.40s/it]
                                                       
{'loss': 1.7148, 'learning_rate': 9.297379415115836e-05, 'epoch': 0.7}

 70%|███████   | 1917/2733 [7:09:20<3:02:17, 13.40s/it]
 70%|███████   | 1918/2733 [7:09:33<3:02:54, 13.47s/it]
                                                       
{'loss': 1.6707, 'learning_rate': 9.28598556779339e-05, 'epoch': 0.7}

 70%|███████   | 1918/2733 [7:09:33<3:02:54, 13.47s/it]
 70%|███████   | 1919/2733 [7:09:47<3:02:24, 13.45s/it]
                                                       
{'loss': 1.6893, 'learning_rate': 9.274591720470945e-05, 'epoch': 0.7}

 70%|███████   | 1919/2733 [7:09:47<3:02:24, 13.45s/it]
 70%|███████   | 1920/2733 [7:10:00<3:02:58, 13.50s/it]
                                                       
{'loss': 1.6868, 'learning_rate': 9.263197873148499e-05, 'epoch': 0.7}

 70%|███████   | 1920/2733 [7:10:00<3:02:58, 13.50s/it]
 70%|███████   | 1921/2733 [7:10:13<3:00:22, 13.33s/it]
                                                       
{'loss': 1.792, 'learning_rate': 9.251804025826052e-05, 'epoch': 0.7}

 70%|███████   | 1921/2733 [7:10:13<3:00:22, 13.33s/it]
 70%|███████   | 1922/2733 [7:10:27<3:00:36, 13.36s/it]
                                                       
{'loss': 1.7692, 'learning_rate': 9.240410178503607e-05, 'epoch': 0.7}

 70%|███████   | 1922/2733 [7:10:27<3:00:36, 13.36s/it]
 70%|███████   | 1923/2733 [7:10:40<2:59:37, 13.31s/it]
                                                       
{'loss': 1.7257, 'learning_rate': 9.229016331181161e-05, 'epoch': 0.7}

 70%|███████   | 1923/2733 [7:10:40<2:59:37, 13.31s/it]
 70%|███████   | 1924/2733 [7:10:54<3:02:19, 13.52s/it]
                                                       
{'loss': 1.6643, 'learning_rate': 9.217622483858715e-05, 'epoch': 0.7}

 70%|███████   | 1924/2733 [7:10:54<3:02:19, 13.52s/it]
 70%|███████   | 1925/2733 [7:11:07<3:01:08, 13.45s/it]
                                                       
{'loss': 1.6673, 'learning_rate': 9.20622863653627e-05, 'epoch': 0.7}

 70%|███████   | 1925/2733 [7:11:07<3:01:08, 13.45s/it]
 70%|███████   | 1926/2733 [7:11:20<3:00:09, 13.39s/it]
                                                       
{'loss': 1.7137, 'learning_rate': 9.194834789213824e-05, 'epoch': 0.7}

 70%|███████   | 1926/2733 [7:11:20<3:00:09, 13.39s/it]
 71%|███████   | 1927/2733 [7:11:34<3:02:52, 13.61s/it]
                                                       
{'loss': 1.7147, 'learning_rate': 9.183440941891379e-05, 'epoch': 0.71}

 71%|███████   | 1927/2733 [7:11:34<3:02:52, 13.61s/it]
 71%|███████   | 1928/2733 [7:11:48<3:01:58, 13.56s/it]
                                                       
{'loss': 1.6191, 'learning_rate': 9.172047094568932e-05, 'epoch': 0.71}

 71%|███████   | 1928/2733 [7:11:48<3:01:58, 13.56s/it]
 71%|███████   | 1929/2733 [7:12:01<3:02:06, 13.59s/it]
                                                       
{'loss': 1.7212, 'learning_rate': 9.160653247246486e-05, 'epoch': 0.71}

 71%|███████   | 1929/2733 [7:12:02<3:02:06, 13.59s/it]
 71%|███████   | 1930/2733 [7:12:15<3:02:19, 13.62s/it]
                                                       
{'loss': 1.7001, 'learning_rate': 9.14925939992404e-05, 'epoch': 0.71}

 71%|███████   | 1930/2733 [7:12:15<3:02:19, 13.62s/it]
 71%|███████   | 1931/2733 [7:12:29<3:01:43, 13.60s/it]
                                                       
{'loss': 1.7649, 'learning_rate': 9.137865552601595e-05, 'epoch': 0.71}

 71%|███████   | 1931/2733 [7:12:29<3:01:43, 13.60s/it]
 71%|███████   | 1932/2733 [7:12:42<3:00:00, 13.48s/it]
                                                       
{'loss': 1.7534, 'learning_rate': 9.126471705279149e-05, 'epoch': 0.71}

 71%|███████   | 1932/2733 [7:12:42<3:00:00, 13.48s/it]
 71%|███████   | 1933/2733 [7:12:55<2:59:33, 13.47s/it]
                                                       
{'loss': 1.6067, 'learning_rate': 9.115077857956704e-05, 'epoch': 0.71}

 71%|███████   | 1933/2733 [7:12:55<2:59:33, 13.47s/it]
 71%|███████   | 1934/2733 [7:13:09<3:01:23, 13.62s/it]
                                                       
{'loss': 1.6257, 'learning_rate': 9.103684010634258e-05, 'epoch': 0.71}

 71%|███████   | 1934/2733 [7:13:09<3:01:23, 13.62s/it]
 71%|███████   | 1935/2733 [7:13:23<2:59:31, 13.50s/it]
                                                       
{'loss': 1.7446, 'learning_rate': 9.09229016331181e-05, 'epoch': 0.71}

 71%|███████   | 1935/2733 [7:13:23<2:59:31, 13.50s/it]
 71%|███████   | 1936/2733 [7:13:35<2:57:04, 13.33s/it]
                                                       
{'loss': 1.6192, 'learning_rate': 9.080896315989364e-05, 'epoch': 0.71}

 71%|███████   | 1936/2733 [7:13:36<2:57:04, 13.33s/it]
 71%|███████   | 1937/2733 [7:13:49<2:58:09, 13.43s/it]
                                                       
{'loss': 1.6214, 'learning_rate': 9.069502468666918e-05, 'epoch': 0.71}

 71%|███████   | 1937/2733 [7:13:49<2:58:09, 13.43s/it]
 71%|███████   | 1938/2733 [7:14:03<2:58:10, 13.45s/it]
                                                       
{'loss': 1.734, 'learning_rate': 9.058108621344473e-05, 'epoch': 0.71}

 71%|███████   | 1938/2733 [7:14:03<2:58:10, 13.45s/it]
 71%|███████   | 1939/2733 [7:14:16<2:58:21, 13.48s/it]
                                                       
{'loss': 1.7155, 'learning_rate': 9.046714774022027e-05, 'epoch': 0.71}

 71%|███████   | 1939/2733 [7:14:16<2:58:21, 13.48s/it]
 71%|███████   | 1940/2733 [7:14:30<2:58:33, 13.51s/it]
                                                       
{'loss': 1.6331, 'learning_rate': 9.035320926699582e-05, 'epoch': 0.71}

 71%|███████   | 1940/2733 [7:14:30<2:58:33, 13.51s/it]
 71%|███████   | 1941/2733 [7:14:43<2:56:23, 13.36s/it]
                                                       
{'loss': 1.7444, 'learning_rate': 9.023927079377136e-05, 'epoch': 0.71}

 71%|███████   | 1941/2733 [7:14:43<2:56:23, 13.36s/it]
 71%|███████   | 1942/2733 [7:14:57<2:58:26, 13.54s/it]
                                                       
{'loss': 1.5637, 'learning_rate': 9.012533232054689e-05, 'epoch': 0.71}

 71%|███████   | 1942/2733 [7:14:57<2:58:26, 13.54s/it]
 71%|███████   | 1943/2733 [7:15:10<2:56:36, 13.41s/it]
                                                       
{'loss': 1.7159, 'learning_rate': 9.001139384732243e-05, 'epoch': 0.71}

 71%|███████   | 1943/2733 [7:15:10<2:56:36, 13.41s/it]
 71%|███████   | 1944/2733 [7:15:23<2:53:49, 13.22s/it]
                                                       
{'loss': 1.5952, 'learning_rate': 8.989745537409798e-05, 'epoch': 0.71}

 71%|███████   | 1944/2733 [7:15:23<2:53:49, 13.22s/it]
 71%|███████   | 1945/2733 [7:15:36<2:52:48, 13.16s/it]
                                                       
{'loss': 1.5822, 'learning_rate': 8.978351690087352e-05, 'epoch': 0.71}

 71%|███████   | 1945/2733 [7:15:36<2:52:48, 13.16s/it]
 71%|███████   | 1946/2733 [7:15:49<2:52:25, 13.15s/it]
                                                       
{'loss': 1.6871, 'learning_rate': 8.966957842764907e-05, 'epoch': 0.71}

 71%|███████   | 1946/2733 [7:15:49<2:52:25, 13.15s/it]
 71%|███████   | 1947/2733 [7:16:02<2:51:23, 13.08s/it]
                                                       
{'loss': 1.7546, 'learning_rate': 8.955563995442461e-05, 'epoch': 0.71}

 71%|███████   | 1947/2733 [7:16:02<2:51:23, 13.08s/it]
 71%|███████▏  | 1948/2733 [7:16:16<2:55:25, 13.41s/it]
                                                       
{'loss': 1.6689, 'learning_rate': 8.944170148120015e-05, 'epoch': 0.71}

 71%|███████▏  | 1948/2733 [7:16:16<2:55:25, 13.41s/it]
 71%|███████▏  | 1949/2733 [7:16:29<2:56:02, 13.47s/it]
                                                       
{'loss': 1.7241, 'learning_rate': 8.932776300797568e-05, 'epoch': 0.71}

 71%|███████▏  | 1949/2733 [7:16:30<2:56:02, 13.47s/it]
 71%|███████▏  | 1950/2733 [7:16:44<2:58:27, 13.68s/it]
                                                       
{'loss': 1.7157, 'learning_rate': 8.921382453475123e-05, 'epoch': 0.71}

 71%|███████▏  | 1950/2733 [7:16:44<2:58:27, 13.68s/it]
 71%|███████▏  | 1951/2733 [7:16:57<2:56:51, 13.57s/it]
                                                       
{'loss': 1.6314, 'learning_rate': 8.909988606152677e-05, 'epoch': 0.71}

 71%|███████▏  | 1951/2733 [7:16:57<2:56:51, 13.57s/it]
 71%|███████▏  | 1952/2733 [7:17:10<2:55:12, 13.46s/it]
                                                       
{'loss': 1.657, 'learning_rate': 8.898594758830232e-05, 'epoch': 0.71}

 71%|███████▏  | 1952/2733 [7:17:10<2:55:12, 13.46s/it]
 71%|███████▏  | 1953/2733 [7:17:24<2:55:46, 13.52s/it]
                                                       
{'loss': 1.5607, 'learning_rate': 8.887200911507786e-05, 'epoch': 0.71}

 71%|███████▏  | 1953/2733 [7:17:24<2:55:46, 13.52s/it]
 71%|███████▏  | 1954/2733 [7:17:36<2:51:28, 13.21s/it]
                                                       
{'loss': 1.684, 'learning_rate': 8.875807064185339e-05, 'epoch': 0.71}

 71%|███████▏  | 1954/2733 [7:17:36<2:51:28, 13.21s/it]
 72%|███████▏  | 1955/2733 [7:17:49<2:50:48, 13.17s/it]
                                                       
{'loss': 1.6507, 'learning_rate': 8.864413216862893e-05, 'epoch': 0.72}

 72%|███████▏  | 1955/2733 [7:17:49<2:50:48, 13.17s/it]
 72%|███████▏  | 1956/2733 [7:18:03<2:53:47, 13.42s/it]
                                                       
{'loss': 1.74, 'learning_rate': 8.853019369540446e-05, 'epoch': 0.72}

 72%|███████▏  | 1956/2733 [7:18:03<2:53:47, 13.42s/it]
 72%|███████▏  | 1957/2733 [7:18:16<2:50:47, 13.21s/it]
                                                       
{'loss': 1.6484, 'learning_rate': 8.841625522218001e-05, 'epoch': 0.72}

 72%|███████▏  | 1957/2733 [7:18:16<2:50:47, 13.21s/it]
 72%|███████▏  | 1958/2733 [7:18:29<2:50:50, 13.23s/it]
                                                       
{'loss': 1.7213, 'learning_rate': 8.830231674895555e-05, 'epoch': 0.72}

 72%|███████▏  | 1958/2733 [7:18:29<2:50:50, 13.23s/it]
 72%|███████▏  | 1959/2733 [7:18:43<2:51:14, 13.28s/it]
                                                       
{'loss': 1.7118, 'learning_rate': 8.81883782757311e-05, 'epoch': 0.72}

 72%|███████▏  | 1959/2733 [7:18:43<2:51:14, 13.28s/it]
 72%|███████▏  | 1960/2733 [7:18:56<2:50:41, 13.25s/it]
                                                       
{'loss': 1.6774, 'learning_rate': 8.807443980250664e-05, 'epoch': 0.72}

 72%|███████▏  | 1960/2733 [7:18:56<2:50:41, 13.25s/it]
 72%|███████▏  | 1961/2733 [7:19:09<2:48:07, 13.07s/it]
                                                       
{'loss': 1.6966, 'learning_rate': 8.796050132928218e-05, 'epoch': 0.72}

 72%|███████▏  | 1961/2733 [7:19:09<2:48:07, 13.07s/it]
 72%|███████▏  | 1962/2733 [7:19:22<2:50:11, 13.24s/it]
                                                       
{'loss': 1.637, 'learning_rate': 8.784656285605773e-05, 'epoch': 0.72}

 72%|███████▏  | 1962/2733 [7:19:22<2:50:11, 13.24s/it]
 72%|███████▏  | 1963/2733 [7:19:36<2:50:31, 13.29s/it]
                                                       
{'loss': 1.7062, 'learning_rate': 8.773262438283326e-05, 'epoch': 0.72}

 72%|███████▏  | 1963/2733 [7:19:36<2:50:31, 13.29s/it]
 72%|███████▏  | 1964/2733 [7:19:49<2:50:01, 13.27s/it]
                                                       
{'loss': 1.7572, 'learning_rate': 8.76186859096088e-05, 'epoch': 0.72}

 72%|███████▏  | 1964/2733 [7:19:49<2:50:01, 13.27s/it]
 72%|███████▏  | 1965/2733 [7:20:02<2:50:58, 13.36s/it]
                                                       
{'loss': 1.6249, 'learning_rate': 8.750474743638435e-05, 'epoch': 0.72}

 72%|███████▏  | 1965/2733 [7:20:02<2:50:58, 13.36s/it]
 72%|███████▏  | 1966/2733 [7:20:16<2:50:41, 13.35s/it]
                                                       
{'loss': 1.6993, 'learning_rate': 8.739080896315989e-05, 'epoch': 0.72}

 72%|███████▏  | 1966/2733 [7:20:16<2:50:41, 13.35s/it]
 72%|███████▏  | 1967/2733 [7:20:29<2:48:56, 13.23s/it]
                                                       
{'loss': 1.7624, 'learning_rate': 8.727687048993543e-05, 'epoch': 0.72}

 72%|███████▏  | 1967/2733 [7:20:29<2:48:56, 13.23s/it]
 72%|███████▏  | 1968/2733 [7:20:43<2:50:59, 13.41s/it]
                                                       
{'loss': 1.7074, 'learning_rate': 8.716293201671098e-05, 'epoch': 0.72}

 72%|███████▏  | 1968/2733 [7:20:43<2:50:59, 13.41s/it]
 72%|███████▏  | 1969/2733 [7:20:56<2:50:30, 13.39s/it]
                                                       
{'loss': 1.6836, 'learning_rate': 8.704899354348652e-05, 'epoch': 0.72}

 72%|███████▏  | 1969/2733 [7:20:56<2:50:30, 13.39s/it]
 72%|███████▏  | 1970/2733 [7:21:10<2:51:27, 13.48s/it]
                                                       
{'loss': 1.7606, 'learning_rate': 8.693505507026204e-05, 'epoch': 0.72}

 72%|███████▏  | 1970/2733 [7:21:10<2:51:27, 13.48s/it]
 72%|███████▏  | 1971/2733 [7:21:23<2:50:38, 13.44s/it]
                                                       
{'loss': 1.6865, 'learning_rate': 8.682111659703758e-05, 'epoch': 0.72}

 72%|███████▏  | 1971/2733 [7:21:23<2:50:38, 13.44s/it]
 72%|███████▏  | 1972/2733 [7:21:37<2:51:12, 13.50s/it]
                                                       
{'loss': 1.5645, 'learning_rate': 8.670717812381313e-05, 'epoch': 0.72}

 72%|███████▏  | 1972/2733 [7:21:37<2:51:12, 13.50s/it]
 72%|███████▏  | 1973/2733 [7:21:50<2:49:11, 13.36s/it]
                                                       
{'loss': 1.678, 'learning_rate': 8.659323965058867e-05, 'epoch': 0.72}

 72%|███████▏  | 1973/2733 [7:21:50<2:49:11, 13.36s/it]
 72%|███████▏  | 1974/2733 [7:22:03<2:48:07, 13.29s/it]
                                                       
{'loss': 1.7614, 'learning_rate': 8.647930117736421e-05, 'epoch': 0.72}

 72%|███████▏  | 1974/2733 [7:22:03<2:48:07, 13.29s/it]
 72%|███████▏  | 1975/2733 [7:22:16<2:49:27, 13.41s/it]
                                                       
{'loss': 1.6278, 'learning_rate': 8.636536270413976e-05, 'epoch': 0.72}

 72%|███████▏  | 1975/2733 [7:22:16<2:49:27, 13.41s/it]
 72%|███████▏  | 1976/2733 [7:22:30<2:49:45, 13.46s/it]
                                                       
{'loss': 1.6893, 'learning_rate': 8.62514242309153e-05, 'epoch': 0.72}

 72%|███████▏  | 1976/2733 [7:22:30<2:49:45, 13.46s/it]
 72%|███████▏  | 1977/2733 [7:22:43<2:47:46, 13.32s/it]
                                                       
{'loss': 1.5428, 'learning_rate': 8.613748575769083e-05, 'epoch': 0.72}

 72%|███████▏  | 1977/2733 [7:22:43<2:47:46, 13.32s/it]
 72%|███████▏  | 1978/2733 [7:22:56<2:46:28, 13.23s/it]
                                                       
{'loss': 1.6083, 'learning_rate': 8.602354728446638e-05, 'epoch': 0.72}

 72%|███████▏  | 1978/2733 [7:22:56<2:46:28, 13.23s/it]
 72%|███████▏  | 1979/2733 [7:23:09<2:47:15, 13.31s/it]
                                                       
{'loss': 1.7069, 'learning_rate': 8.590960881124192e-05, 'epoch': 0.72}

 72%|███████▏  | 1979/2733 [7:23:10<2:47:15, 13.31s/it]
 72%|███████▏  | 1980/2733 [7:23:23<2:47:42, 13.36s/it]
                                                       
{'loss': 1.683, 'learning_rate': 8.579567033801746e-05, 'epoch': 0.72}

 72%|███████▏  | 1980/2733 [7:23:23<2:47:42, 13.36s/it]
 72%|███████▏  | 1981/2733 [7:23:36<2:45:04, 13.17s/it]
                                                       
{'loss': 1.653, 'learning_rate': 8.568173186479301e-05, 'epoch': 0.72}

 72%|███████▏  | 1981/2733 [7:23:36<2:45:04, 13.17s/it]
 73%|███████▎  | 1982/2733 [7:23:50<2:47:27, 13.38s/it]
                                                       
{'loss': 1.6055, 'learning_rate': 8.556779339156855e-05, 'epoch': 0.73}

 73%|███████▎  | 1982/2733 [7:23:50<2:47:27, 13.38s/it]
 73%|███████▎  | 1983/2733 [7:24:03<2:45:41, 13.26s/it]
                                                       
{'loss': 1.6226, 'learning_rate': 8.54538549183441e-05, 'epoch': 0.73}

 73%|███████▎  | 1983/2733 [7:24:03<2:45:41, 13.26s/it]
 73%|███████▎  | 1984/2733 [7:24:16<2:45:37, 13.27s/it]
                                                       
{'loss': 1.5954, 'learning_rate': 8.533991644511963e-05, 'epoch': 0.73}

 73%|███████▎  | 1984/2733 [7:24:16<2:45:37, 13.27s/it]
 73%|███████▎  | 1985/2733 [7:24:30<2:49:06, 13.56s/it]
                                                       
{'loss': 1.7108, 'learning_rate': 8.522597797189517e-05, 'epoch': 0.73}

 73%|███████▎  | 1985/2733 [7:24:30<2:49:06, 13.56s/it]
 73%|███████▎  | 1986/2733 [7:24:44<2:49:00, 13.57s/it]
                                                       
{'loss': 1.5454, 'learning_rate': 8.511203949867071e-05, 'epoch': 0.73}

 73%|███████▎  | 1986/2733 [7:24:44<2:49:00, 13.57s/it]
 73%|███████▎  | 1987/2733 [7:24:56<2:43:58, 13.19s/it]
                                                       
{'loss': 1.5803, 'learning_rate': 8.499810102544626e-05, 'epoch': 0.73}

 73%|███████▎  | 1987/2733 [7:24:56<2:43:58, 13.19s/it]
 73%|███████▎  | 1988/2733 [7:25:09<2:42:44, 13.11s/it]
                                                       
{'loss': 1.5911, 'learning_rate': 8.48841625522218e-05, 'epoch': 0.73}

 73%|███████▎  | 1988/2733 [7:25:09<2:42:44, 13.11s/it]
 73%|███████▎  | 1989/2733 [7:25:22<2:42:21, 13.09s/it]
                                                       
{'loss': 1.6098, 'learning_rate': 8.477022407899734e-05, 'epoch': 0.73}

 73%|███████▎  | 1989/2733 [7:25:22<2:42:21, 13.09s/it]
 73%|███████▎  | 1990/2733 [7:25:36<2:44:56, 13.32s/it]
                                                       
{'loss': 1.6837, 'learning_rate': 8.465628560577289e-05, 'epoch': 0.73}

 73%|███████▎  | 1990/2733 [7:25:36<2:44:56, 13.32s/it]
 73%|███████▎  | 1991/2733 [7:25:49<2:43:53, 13.25s/it]
                                                       
{'loss': 1.6517, 'learning_rate': 8.45423471325484e-05, 'epoch': 0.73}

 73%|███████▎  | 1991/2733 [7:25:49<2:43:53, 13.25s/it]
 73%|███████▎  | 1992/2733 [7:26:02<2:44:41, 13.33s/it]
                                                       
{'loss': 1.672, 'learning_rate': 8.442840865932395e-05, 'epoch': 0.73}

 73%|███████▎  | 1992/2733 [7:26:02<2:44:41, 13.33s/it]
 73%|███████▎  | 1993/2733 [7:26:16<2:43:40, 13.27s/it]
                                                       
{'loss': 1.6361, 'learning_rate': 8.431447018609949e-05, 'epoch': 0.73}

 73%|███████▎  | 1993/2733 [7:26:16<2:43:40, 13.27s/it]
 73%|███████▎  | 1994/2733 [7:26:30<2:46:44, 13.54s/it]
                                                       
{'loss': 1.7116, 'learning_rate': 8.420053171287504e-05, 'epoch': 0.73}

 73%|███████▎  | 1994/2733 [7:26:30<2:46:44, 13.54s/it]
 73%|███████▎  | 1995/2733 [7:26:43<2:45:29, 13.45s/it]
                                                       
{'loss': 1.6293, 'learning_rate': 8.408659323965058e-05, 'epoch': 0.73}

 73%|███████▎  | 1995/2733 [7:26:43<2:45:29, 13.45s/it]
 73%|███████▎  | 1996/2733 [7:26:56<2:44:28, 13.39s/it]
                                                       
{'loss': 1.668, 'learning_rate': 8.397265476642612e-05, 'epoch': 0.73}

 73%|███████▎  | 1996/2733 [7:26:56<2:44:28, 13.39s/it]
 73%|███████▎  | 1997/2733 [7:27:10<2:45:38, 13.50s/it]
                                                       
{'loss': 1.6233, 'learning_rate': 8.385871629320167e-05, 'epoch': 0.73}

 73%|███████▎  | 1997/2733 [7:27:10<2:45:38, 13.50s/it]
 73%|███████▎  | 1998/2733 [7:27:24<2:47:46, 13.70s/it]
                                                       
{'loss': 1.6643, 'learning_rate': 8.37447778199772e-05, 'epoch': 0.73}

 73%|███████▎  | 1998/2733 [7:27:24<2:47:46, 13.70s/it]
 73%|███████▎  | 1999/2733 [7:27:38<2:47:26, 13.69s/it]
                                                       
{'loss': 1.6021, 'learning_rate': 8.363083934675274e-05, 'epoch': 0.73}

 73%|███████▎  | 1999/2733 [7:27:38<2:47:26, 13.69s/it]
 73%|███████▎  | 2000/2733 [7:27:51<2:45:49, 13.57s/it]
                                                       
{'loss': 1.6481, 'learning_rate': 8.351690087352829e-05, 'epoch': 0.73}

 73%|███████▎  | 2000/2733 [7:27:51<2:45:49, 13.57s/it]
 73%|███████▎  | 2001/2733 [7:28:05<2:48:04, 13.78s/it]
                                                       
{'loss': 1.7124, 'learning_rate': 8.340296240030383e-05, 'epoch': 0.73}

 73%|███████▎  | 2001/2733 [7:28:05<2:48:04, 13.78s/it]
 73%|███████▎  | 2002/2733 [7:28:19<2:47:10, 13.72s/it]
                                                       
{'loss': 1.7028, 'learning_rate': 8.328902392707937e-05, 'epoch': 0.73}

 73%|███████▎  | 2002/2733 [7:28:19<2:47:10, 13.72s/it]
 73%|███████▎  | 2003/2733 [7:28:33<2:47:05, 13.73s/it]
                                                       
{'loss': 1.6309, 'learning_rate': 8.317508545385492e-05, 'epoch': 0.73}

 73%|███████▎  | 2003/2733 [7:28:33<2:47:05, 13.73s/it]
 73%|███████▎  | 2004/2733 [7:28:47<2:47:32, 13.79s/it]
                                                       
{'loss': 1.6606, 'learning_rate': 8.306114698063045e-05, 'epoch': 0.73}

 73%|███████▎  | 2004/2733 [7:28:47<2:47:32, 13.79s/it]
 73%|███████▎  | 2005/2733 [7:29:00<2:45:25, 13.63s/it]
                                                       
{'loss': 1.6106, 'learning_rate': 8.294720850740599e-05, 'epoch': 0.73}

 73%|███████▎  | 2005/2733 [7:29:00<2:45:25, 13.63s/it]
 73%|███████▎  | 2006/2733 [7:29:13<2:43:18, 13.48s/it]
                                                       
{'loss': 1.6098, 'learning_rate': 8.283327003418154e-05, 'epoch': 0.73}

 73%|███████▎  | 2006/2733 [7:29:13<2:43:18, 13.48s/it]
 73%|███████▎  | 2007/2733 [7:29:27<2:44:00, 13.55s/it]
                                                       
{'loss': 1.6814, 'learning_rate': 8.271933156095708e-05, 'epoch': 0.73}

 73%|███████▎  | 2007/2733 [7:29:27<2:44:00, 13.55s/it]
 73%|███████▎  | 2008/2733 [7:29:40<2:42:25, 13.44s/it]
                                                       
{'loss': 1.568, 'learning_rate': 8.260539308773262e-05, 'epoch': 0.73}

 73%|███████▎  | 2008/2733 [7:29:40<2:42:25, 13.44s/it]
 74%|███████▎  | 2009/2733 [7:29:52<2:38:57, 13.17s/it]
                                                       
{'loss': 1.7045, 'learning_rate': 8.249145461450817e-05, 'epoch': 0.74}

 74%|███████▎  | 2009/2733 [7:29:52<2:38:57, 13.17s/it]
 74%|███████▎  | 2010/2733 [7:30:06<2:39:12, 13.21s/it]
                                                       
{'loss': 1.6275, 'learning_rate': 8.237751614128371e-05, 'epoch': 0.74}

 74%|███████▎  | 2010/2733 [7:30:06<2:39:12, 13.21s/it]
 74%|███████▎  | 2011/2733 [7:30:17<2:33:33, 12.76s/it]
                                                       
{'loss': 1.5173, 'learning_rate': 8.226357766805923e-05, 'epoch': 0.74}

 74%|███████▎  | 2011/2733 [7:30:18<2:33:33, 12.76s/it]
 74%|███████▎  | 2012/2733 [7:30:32<2:38:30, 13.19s/it]
                                                       
{'loss': 1.6985, 'learning_rate': 8.214963919483477e-05, 'epoch': 0.74}

 74%|███████▎  | 2012/2733 [7:30:32<2:38:30, 13.19s/it]
 74%|███████▎  | 2013/2733 [7:30:45<2:40:11, 13.35s/it]
                                                       
{'loss': 1.6629, 'learning_rate': 8.203570072161032e-05, 'epoch': 0.74}

 74%|███████▎  | 2013/2733 [7:30:45<2:40:11, 13.35s/it]
 74%|███████▎  | 2014/2733 [7:30:58<2:35:43, 12.99s/it]
                                                       
{'loss': 1.7194, 'learning_rate': 8.192176224838586e-05, 'epoch': 0.74}

 74%|███████▎  | 2014/2733 [7:30:58<2:35:43, 12.99s/it]
 74%|███████▎  | 2015/2733 [7:31:11<2:38:38, 13.26s/it]
                                                       
{'loss': 1.6006, 'learning_rate': 8.18078237751614e-05, 'epoch': 0.74}

 74%|███████▎  | 2015/2733 [7:31:11<2:38:38, 13.26s/it]
 74%|███████▍  | 2016/2733 [7:31:25<2:39:08, 13.32s/it]
                                                       
{'loss': 1.7426, 'learning_rate': 8.169388530193695e-05, 'epoch': 0.74}

 74%|███████▍  | 2016/2733 [7:31:25<2:39:08, 13.32s/it]
 74%|███████▍  | 2017/2733 [7:31:38<2:38:39, 13.30s/it]
                                                       
{'loss': 1.6561, 'learning_rate': 8.157994682871249e-05, 'epoch': 0.74}

 74%|███████▍  | 2017/2733 [7:31:38<2:38:39, 13.30s/it]
 74%|███████▍  | 2018/2733 [7:31:52<2:40:22, 13.46s/it]
                                                       
{'loss': 1.6346, 'learning_rate': 8.146600835548802e-05, 'epoch': 0.74}

 74%|███████▍  | 2018/2733 [7:31:52<2:40:22, 13.46s/it]
 74%|███████▍  | 2019/2733 [7:32:06<2:42:16, 13.64s/it]
                                                       
{'loss': 1.7639, 'learning_rate': 8.135206988226357e-05, 'epoch': 0.74}

 74%|███████▍  | 2019/2733 [7:32:06<2:42:16, 13.64s/it]
 74%|███████▍  | 2020/2733 [7:32:20<2:41:48, 13.62s/it]
                                                       
{'loss': 1.6711, 'learning_rate': 8.123813140903911e-05, 'epoch': 0.74}

 74%|███████▍  | 2020/2733 [7:32:20<2:41:48, 13.62s/it]
 74%|███████▍  | 2021/2733 [7:32:33<2:40:09, 13.50s/it]
                                                       
{'loss': 1.6785, 'learning_rate': 8.112419293581465e-05, 'epoch': 0.74}

 74%|███████▍  | 2021/2733 [7:32:33<2:40:09, 13.50s/it]
 74%|███████▍  | 2022/2733 [7:32:47<2:42:05, 13.68s/it]
                                                       
{'loss': 1.6707, 'learning_rate': 8.10102544625902e-05, 'epoch': 0.74}

 74%|███████▍  | 2022/2733 [7:32:47<2:42:05, 13.68s/it]
 74%|███████▍  | 2023/2733 [7:33:01<2:41:59, 13.69s/it]
                                                       
{'loss': 1.6977, 'learning_rate': 8.089631598936574e-05, 'epoch': 0.74}

 74%|███████▍  | 2023/2733 [7:33:01<2:41:59, 13.69s/it]
 74%|███████▍  | 2024/2733 [7:33:15<2:43:31, 13.84s/it]
                                                       
{'loss': 1.8343, 'learning_rate': 8.078237751614129e-05, 'epoch': 0.74}

 74%|███████▍  | 2024/2733 [7:33:15<2:43:31, 13.84s/it]
 74%|███████▍  | 2025/2733 [7:33:28<2:40:48, 13.63s/it]
                                                       
{'loss': 1.6014, 'learning_rate': 8.066843904291682e-05, 'epoch': 0.74}

 74%|███████▍  | 2025/2733 [7:33:28<2:40:48, 13.63s/it]
 74%|███████▍  | 2026/2733 [7:33:42<2:40:35, 13.63s/it]
                                                       
{'loss': 1.6415, 'learning_rate': 8.055450056969236e-05, 'epoch': 0.74}

 74%|███████▍  | 2026/2733 [7:33:42<2:40:35, 13.63s/it]
 74%|███████▍  | 2027/2733 [7:33:55<2:38:37, 13.48s/it]
                                                       
{'loss': 1.709, 'learning_rate': 8.04405620964679e-05, 'epoch': 0.74}

 74%|███████▍  | 2027/2733 [7:33:55<2:38:37, 13.48s/it]
 74%|███████▍  | 2028/2733 [7:34:08<2:38:51, 13.52s/it]
                                                       
{'loss': 1.5595, 'learning_rate': 8.032662362324343e-05, 'epoch': 0.74}

 74%|███████▍  | 2028/2733 [7:34:08<2:38:51, 13.52s/it]
 74%|███████▍  | 2029/2733 [7:34:22<2:39:14, 13.57s/it]
                                                       
{'loss': 1.7371, 'learning_rate': 8.021268515001898e-05, 'epoch': 0.74}

 74%|███████▍  | 2029/2733 [7:34:22<2:39:14, 13.57s/it]
 74%|███████▍  | 2030/2733 [7:34:35<2:37:06, 13.41s/it]
                                                       
{'loss': 1.6385, 'learning_rate': 8.009874667679452e-05, 'epoch': 0.74}

 74%|███████▍  | 2030/2733 [7:34:35<2:37:06, 13.41s/it]
 74%|███████▍  | 2031/2733 [7:34:49<2:37:45, 13.48s/it]
                                                       
{'loss': 1.8452, 'learning_rate': 7.998480820357007e-05, 'epoch': 0.74}

 74%|███████▍  | 2031/2733 [7:34:49<2:37:45, 13.48s/it]
 74%|███████▍  | 2032/2733 [7:35:02<2:37:11, 13.45s/it]
                                                       
{'loss': 1.6239, 'learning_rate': 7.98708697303456e-05, 'epoch': 0.74}

 74%|███████▍  | 2032/2733 [7:35:02<2:37:11, 13.45s/it]
 74%|███████▍  | 2033/2733 [7:35:15<2:36:20, 13.40s/it]
                                                       
{'loss': 1.6095, 'learning_rate': 7.975693125712114e-05, 'epoch': 0.74}

 74%|███████▍  | 2033/2733 [7:35:15<2:36:20, 13.40s/it]
 74%|███████▍  | 2034/2733 [7:35:29<2:36:24, 13.42s/it]
                                                       
{'loss': 1.713, 'learning_rate': 7.964299278389668e-05, 'epoch': 0.74}

 74%|███████▍  | 2034/2733 [7:35:29<2:36:24, 13.42s/it]
 74%|███████▍  | 2035/2733 [7:35:42<2:36:41, 13.47s/it]
                                                       
{'loss': 1.7444, 'learning_rate': 7.952905431067223e-05, 'epoch': 0.74}

 74%|███████▍  | 2035/2733 [7:35:42<2:36:41, 13.47s/it]
 74%|███████▍  | 2036/2733 [7:35:55<2:33:23, 13.20s/it]
                                                       
{'loss': 1.7134, 'learning_rate': 7.941511583744777e-05, 'epoch': 0.74}

 74%|███████▍  | 2036/2733 [7:35:55<2:33:23, 13.20s/it]
 75%|███████▍  | 2037/2733 [7:36:08<2:33:01, 13.19s/it]
                                                       
{'loss': 1.6544, 'learning_rate': 7.930117736422332e-05, 'epoch': 0.75}

 75%|███████▍  | 2037/2733 [7:36:08<2:33:01, 13.19s/it]
 75%|███████▍  | 2038/2733 [7:36:21<2:32:21, 13.15s/it]
                                                       
{'loss': 1.5828, 'learning_rate': 7.918723889099886e-05, 'epoch': 0.75}

 75%|███████▍  | 2038/2733 [7:36:21<2:32:21, 13.15s/it]
 75%|███████▍  | 2039/2733 [7:36:35<2:34:35, 13.37s/it]
                                                       
{'loss': 1.5968, 'learning_rate': 7.907330041777439e-05, 'epoch': 0.75}

 75%|███████▍  | 2039/2733 [7:36:35<2:34:35, 13.37s/it]
 75%|███████▍  | 2040/2733 [7:36:49<2:34:51, 13.41s/it]
                                                       
{'loss': 1.6658, 'learning_rate': 7.895936194454993e-05, 'epoch': 0.75}

 75%|███████▍  | 2040/2733 [7:36:49<2:34:51, 13.41s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 75%|███████▍  | 2041/2733 [7:37:02<2:34:12, 13.37s/it]
                                                       
{'loss': 1.6562, 'learning_rate': 7.884542347132548e-05, 'epoch': 0.75}

 75%|███████▍  | 2041/2733 [7:37:02<2:34:12, 13.37s/it]
 75%|███████▍  | 2042/2733 [7:37:15<2:33:23, 13.32s/it]
                                                       
{'loss': 1.6281, 'learning_rate': 7.873148499810102e-05, 'epoch': 0.75}

 75%|███████▍  | 2042/2733 [7:37:15<2:33:23, 13.32s/it]
 75%|███████▍  | 2043/2733 [7:37:28<2:32:42, 13.28s/it]
                                                       
{'loss': 1.6556, 'learning_rate': 7.861754652487657e-05, 'epoch': 0.75}

 75%|███████▍  | 2043/2733 [7:37:28<2:32:42, 13.28s/it]
 75%|███████▍  | 2044/2733 [7:37:42<2:33:47, 13.39s/it]
                                                       
{'loss': 1.7722, 'learning_rate': 7.850360805165211e-05, 'epoch': 0.75}

 75%|███████▍  | 2044/2733 [7:37:42<2:33:47, 13.39s/it]
 75%|███████▍  | 2045/2733 [7:37:55<2:31:47, 13.24s/it]
                                                       
{'loss': 1.5978, 'learning_rate': 7.838966957842765e-05, 'epoch': 0.75}

 75%|███████▍  | 2045/2733 [7:37:55<2:31:47, 13.24s/it]
 75%|███████▍  | 2046/2733 [7:38:07<2:29:19, 13.04s/it]
                                                       
{'loss': 1.6144, 'learning_rate': 7.827573110520317e-05, 'epoch': 0.75}

 75%|███████▍  | 2046/2733 [7:38:07<2:29:19, 13.04s/it]
 75%|███████▍  | 2047/2733 [7:38:20<2:28:51, 13.02s/it]
                                                       
{'loss': 1.5694, 'learning_rate': 7.816179263197871e-05, 'epoch': 0.75}

 75%|███████▍  | 2047/2733 [7:38:20<2:28:51, 13.02s/it]
 75%|███████▍  | 2048/2733 [7:38:33<2:28:42, 13.03s/it]
                                                       
{'loss': 1.7445, 'learning_rate': 7.804785415875426e-05, 'epoch': 0.75}

 75%|███████▍  | 2048/2733 [7:38:33<2:28:42, 13.03s/it]
 75%|███████▍  | 2049/2733 [7:38:47<2:29:36, 13.12s/it]
                                                       
{'loss': 1.6561, 'learning_rate': 7.79339156855298e-05, 'epoch': 0.75}

 75%|███████▍  | 2049/2733 [7:38:47<2:29:36, 13.12s/it]
 75%|███████▌  | 2050/2733 [7:39:00<2:31:02, 13.27s/it]
                                                       
{'loss': 1.7775, 'learning_rate': 7.781997721230535e-05, 'epoch': 0.75}

 75%|███████▌  | 2050/2733 [7:39:00<2:31:02, 13.27s/it]
 75%|███████▌  | 2051/2733 [7:39:13<2:28:55, 13.10s/it]
                                                       
{'loss': 1.7088, 'learning_rate': 7.770603873908089e-05, 'epoch': 0.75}

 75%|███████▌  | 2051/2733 [7:39:13<2:28:55, 13.10s/it]
 75%|███████▌  | 2052/2733 [7:39:26<2:26:49, 12.94s/it]
                                                       
{'loss': 1.6905, 'learning_rate': 7.759210026585643e-05, 'epoch': 0.75}

 75%|███████▌  | 2052/2733 [7:39:26<2:26:49, 12.94s/it]
 75%|███████▌  | 2053/2733 [7:39:38<2:23:22, 12.65s/it]
                                                       
{'loss': 1.5744, 'learning_rate': 7.747816179263196e-05, 'epoch': 0.75}

 75%|███████▌  | 2053/2733 [7:39:38<2:23:22, 12.65s/it]
 75%|███████▌  | 2054/2733 [7:39:51<2:26:35, 12.95s/it]
                                                       
{'loss': 1.7622, 'learning_rate': 7.736422331940751e-05, 'epoch': 0.75}

 75%|███████▌  | 2054/2733 [7:39:51<2:26:35, 12.95s/it]
 75%|███████▌  | 2055/2733 [7:40:05<2:29:51, 13.26s/it]
                                                       
{'loss': 1.7169, 'learning_rate': 7.725028484618305e-05, 'epoch': 0.75}

 75%|███████▌  | 2055/2733 [7:40:05<2:29:51, 13.26s/it]
 75%|███████▌  | 2056/2733 [7:40:18<2:29:35, 13.26s/it]
                                                       
{'loss': 1.7979, 'learning_rate': 7.71363463729586e-05, 'epoch': 0.75}

 75%|███████▌  | 2056/2733 [7:40:19<2:29:35, 13.26s/it]
 75%|███████▌  | 2057/2733 [7:40:32<2:30:34, 13.37s/it]
                                                       
{'loss': 1.6961, 'learning_rate': 7.702240789973414e-05, 'epoch': 0.75}

 75%|███████▌  | 2057/2733 [7:40:32<2:30:34, 13.37s/it]
 75%|███████▌  | 2058/2733 [7:40:46<2:31:31, 13.47s/it]
                                                       
{'loss': 1.6476, 'learning_rate': 7.690846942650968e-05, 'epoch': 0.75}

 75%|███████▌  | 2058/2733 [7:40:46<2:31:31, 13.47s/it]
 75%|███████▌  | 2059/2733 [7:40:59<2:31:19, 13.47s/it]
                                                       
{'loss': 1.5966, 'learning_rate': 7.679453095328523e-05, 'epoch': 0.75}

 75%|███████▌  | 2059/2733 [7:40:59<2:31:19, 13.47s/it]
 75%|███████▌  | 2060/2733 [7:41:14<2:33:53, 13.72s/it]
                                                       
{'loss': 1.6308, 'learning_rate': 7.668059248006076e-05, 'epoch': 0.75}

 75%|███████▌  | 2060/2733 [7:41:14<2:33:53, 13.72s/it]
 75%|███████▌  | 2061/2733 [7:41:27<2:33:35, 13.71s/it]
                                                       
{'loss': 1.7093, 'learning_rate': 7.65666540068363e-05, 'epoch': 0.75}

 75%|███████▌  | 2061/2733 [7:41:27<2:33:35, 13.71s/it]
 75%|███████▌  | 2062/2733 [7:41:41<2:31:43, 13.57s/it]
                                                       
{'loss': 1.6498, 'learning_rate': 7.645271553361185e-05, 'epoch': 0.75}

 75%|███████▌  | 2062/2733 [7:41:41<2:31:43, 13.57s/it]
 75%|███████▌  | 2063/2733 [7:41:54<2:32:09, 13.63s/it]
                                                       
{'loss': 1.6773, 'learning_rate': 7.633877706038739e-05, 'epoch': 0.75}

 75%|███████▌  | 2063/2733 [7:41:54<2:32:09, 13.63s/it]
 76%|███████▌  | 2064/2733 [7:42:08<2:31:11, 13.56s/it]
                                                       
{'loss': 1.7076, 'learning_rate': 7.622483858716293e-05, 'epoch': 0.76}

 76%|███████▌  | 2064/2733 [7:42:08<2:31:11, 13.56s/it]
 76%|███████▌  | 2065/2733 [7:42:21<2:30:34, 13.52s/it]
                                                       
{'loss': 1.7144, 'learning_rate': 7.611090011393848e-05, 'epoch': 0.76}

 76%|███████▌  | 2065/2733 [7:42:21<2:30:34, 13.52s/it]
 76%|███████▌  | 2066/2733 [7:42:35<2:30:54, 13.58s/it]
                                                       
{'loss': 1.68, 'learning_rate': 7.599696164071402e-05, 'epoch': 0.76}

 76%|███████▌  | 2066/2733 [7:42:35<2:30:54, 13.58s/it]
 76%|███████▌  | 2067/2733 [7:42:48<2:29:55, 13.51s/it]
                                                       
{'loss': 1.6178, 'learning_rate': 7.588302316748954e-05, 'epoch': 0.76}

 76%|███████▌  | 2067/2733 [7:42:48<2:29:55, 13.51s/it]
 76%|███████▌  | 2068/2733 [7:43:02<2:29:57, 13.53s/it]
                                                       
{'loss': 1.6441, 'learning_rate': 7.576908469426508e-05, 'epoch': 0.76}

 76%|███████▌  | 2068/2733 [7:43:02<2:29:57, 13.53s/it]
 76%|███████▌  | 2069/2733 [7:43:15<2:28:23, 13.41s/it]
                                                       
{'loss': 1.6523, 'learning_rate': 7.565514622104063e-05, 'epoch': 0.76}

 76%|███████▌  | 2069/2733 [7:43:15<2:28:23, 13.41s/it]
 76%|███████▌  | 2070/2733 [7:43:28<2:26:54, 13.29s/it]
                                                       
{'loss': 1.652, 'learning_rate': 7.554120774781617e-05, 'epoch': 0.76}

 76%|███████▌  | 2070/2733 [7:43:28<2:26:54, 13.29s/it]
 76%|███████▌  | 2071/2733 [7:43:42<2:29:19, 13.53s/it]
                                                       
{'loss': 1.6519, 'learning_rate': 7.542726927459171e-05, 'epoch': 0.76}

 76%|███████▌  | 2071/2733 [7:43:42<2:29:19, 13.53s/it]
 76%|███████▌  | 2072/2733 [7:43:55<2:28:53, 13.51s/it]
                                                       
{'loss': 1.5388, 'learning_rate': 7.531333080136726e-05, 'epoch': 0.76}

 76%|███████▌  | 2072/2733 [7:43:55<2:28:53, 13.51s/it]
 76%|███████▌  | 2073/2733 [7:44:08<2:25:34, 13.23s/it]
                                                       
{'loss': 1.584, 'learning_rate': 7.51993923281428e-05, 'epoch': 0.76}

 76%|███████▌  | 2073/2733 [7:44:08<2:25:34, 13.23s/it]
 76%|███████▌  | 2074/2733 [7:44:22<2:26:24, 13.33s/it]
                                                       
{'loss': 1.6198, 'learning_rate': 7.508545385491833e-05, 'epoch': 0.76}

 76%|███████▌  | 2074/2733 [7:44:22<2:26:24, 13.33s/it]
 76%|███████▌  | 2075/2733 [7:44:35<2:27:03, 13.41s/it]
                                                       
{'loss': 1.5851, 'learning_rate': 7.497151538169388e-05, 'epoch': 0.76}

 76%|███████▌  | 2075/2733 [7:44:35<2:27:03, 13.41s/it]
 76%|███████▌  | 2076/2733 [7:44:49<2:27:17, 13.45s/it]
                                                       
{'loss': 1.6697, 'learning_rate': 7.485757690846942e-05, 'epoch': 0.76}

 76%|███████▌  | 2076/2733 [7:44:49<2:27:17, 13.45s/it]
 76%|███████▌  | 2077/2733 [7:45:03<2:28:59, 13.63s/it]
                                                       
{'loss': 1.6319, 'learning_rate': 7.474363843524496e-05, 'epoch': 0.76}

 76%|███████▌  | 2077/2733 [7:45:03<2:28:59, 13.63s/it]
 76%|███████▌  | 2078/2733 [7:45:16<2:27:10, 13.48s/it]
                                                       
{'loss': 1.6933, 'learning_rate': 7.462969996202051e-05, 'epoch': 0.76}

 76%|███████▌  | 2078/2733 [7:45:16<2:27:10, 13.48s/it]
 76%|███████▌  | 2079/2733 [7:45:29<2:26:05, 13.40s/it]
                                                       
{'loss': 1.6793, 'learning_rate': 7.451576148879604e-05, 'epoch': 0.76}

 76%|███████▌  | 2079/2733 [7:45:29<2:26:05, 13.40s/it]
 76%|███████▌  | 2080/2733 [7:45:43<2:26:54, 13.50s/it]
                                                       
{'loss': 1.5483, 'learning_rate': 7.440182301557158e-05, 'epoch': 0.76}

 76%|███████▌  | 2080/2733 [7:45:43<2:26:54, 13.50s/it]
 76%|███████▌  | 2081/2733 [7:45:57<2:27:39, 13.59s/it]
                                                       
{'loss': 1.7815, 'learning_rate': 7.428788454234713e-05, 'epoch': 0.76}

 76%|███████▌  | 2081/2733 [7:45:57<2:27:39, 13.59s/it]
 76%|███████▌  | 2082/2733 [7:46:10<2:27:39, 13.61s/it]
                                                       
{'loss': 1.5958, 'learning_rate': 7.417394606912267e-05, 'epoch': 0.76}

 76%|███████▌  | 2082/2733 [7:46:10<2:27:39, 13.61s/it]
 76%|███████▌  | 2083/2733 [7:46:24<2:26:21, 13.51s/it]
                                                       
{'loss': 1.7414, 'learning_rate': 7.406000759589821e-05, 'epoch': 0.76}

 76%|███████▌  | 2083/2733 [7:46:24<2:26:21, 13.51s/it]
 76%|███████▋  | 2084/2733 [7:46:37<2:25:17, 13.43s/it]
                                                       
{'loss': 1.6675, 'learning_rate': 7.394606912267376e-05, 'epoch': 0.76}

 76%|███████▋  | 2084/2733 [7:46:37<2:25:17, 13.43s/it]
 76%|███████▋  | 2085/2733 [7:46:49<2:22:29, 13.19s/it]
                                                       
{'loss': 1.6723, 'learning_rate': 7.38321306494493e-05, 'epoch': 0.76}

 76%|███████▋  | 2085/2733 [7:46:50<2:22:29, 13.19s/it]
 76%|███████▋  | 2086/2733 [7:47:03<2:23:28, 13.31s/it]
                                                       
{'loss': 1.6255, 'learning_rate': 7.371819217622483e-05, 'epoch': 0.76}

 76%|███████▋  | 2086/2733 [7:47:03<2:23:28, 13.31s/it]
 76%|███████▋  | 2087/2733 [7:47:17<2:25:23, 13.50s/it]
                                                       
{'loss': 1.7454, 'learning_rate': 7.360425370300038e-05, 'epoch': 0.76}

 76%|███████▋  | 2087/2733 [7:47:17<2:25:23, 13.50s/it]
 76%|███████▋  | 2088/2733 [7:47:30<2:24:16, 13.42s/it]
                                                       
{'loss': 1.6825, 'learning_rate': 7.349031522977592e-05, 'epoch': 0.76}

 76%|███████▋  | 2088/2733 [7:47:30<2:24:16, 13.42s/it]
 76%|███████▋  | 2089/2733 [7:47:43<2:22:58, 13.32s/it]
                                                       
{'loss': 1.6817, 'learning_rate': 7.337637675655145e-05, 'epoch': 0.76}

 76%|███████▋  | 2089/2733 [7:47:43<2:22:58, 13.32s/it]
 76%|███████▋  | 2090/2733 [7:47:57<2:24:19, 13.47s/it]
                                                       
{'loss': 1.6603, 'learning_rate': 7.3262438283327e-05, 'epoch': 0.76}

 76%|███████▋  | 2090/2733 [7:47:57<2:24:19, 13.47s/it]
 77%|███████▋  | 2091/2733 [7:48:10<2:23:08, 13.38s/it]
                                                       
{'loss': 1.6522, 'learning_rate': 7.314849981010254e-05, 'epoch': 0.77}

 77%|███████▋  | 2091/2733 [7:48:10<2:23:08, 13.38s/it]
 77%|███████▋  | 2092/2733 [7:48:23<2:21:13, 13.22s/it]
                                                       
{'loss': 1.7047, 'learning_rate': 7.303456133687808e-05, 'epoch': 0.77}

 77%|███████▋  | 2092/2733 [7:48:23<2:21:13, 13.22s/it]
 77%|███████▋  | 2093/2733 [7:48:37<2:22:16, 13.34s/it]
                                                       
{'loss': 1.6418, 'learning_rate': 7.292062286365363e-05, 'epoch': 0.77}

 77%|███████▋  | 2093/2733 [7:48:37<2:22:16, 13.34s/it]
 77%|███████▋  | 2094/2733 [7:48:50<2:22:29, 13.38s/it]
                                                       
{'loss': 1.6104, 'learning_rate': 7.280668439042916e-05, 'epoch': 0.77}

 77%|███████▋  | 2094/2733 [7:48:50<2:22:29, 13.38s/it]
 77%|███████▋  | 2095/2733 [7:49:03<2:20:04, 13.17s/it]
                                                       
{'loss': 1.61, 'learning_rate': 7.26927459172047e-05, 'epoch': 0.77}

 77%|███████▋  | 2095/2733 [7:49:03<2:20:04, 13.17s/it]
 77%|███████▋  | 2096/2733 [7:49:15<2:17:00, 12.90s/it]
                                                       
{'loss': 1.7424, 'learning_rate': 7.257880744398024e-05, 'epoch': 0.77}

 77%|███████▋  | 2096/2733 [7:49:15<2:17:00, 12.90s/it]
 77%|███████▋  | 2097/2733 [7:49:29<2:19:21, 13.15s/it]
                                                       
{'loss': 1.6196, 'learning_rate': 7.246486897075579e-05, 'epoch': 0.77}

 77%|███████▋  | 2097/2733 [7:49:29<2:19:21, 13.15s/it]
 77%|███████▋  | 2098/2733 [7:49:43<2:21:43, 13.39s/it]
                                                       
{'loss': 1.7219, 'learning_rate': 7.235093049753133e-05, 'epoch': 0.77}

 77%|███████▋  | 2098/2733 [7:49:43<2:21:43, 13.39s/it]
 77%|███████▋  | 2099/2733 [7:49:56<2:20:36, 13.31s/it]
                                                       
{'loss': 1.6237, 'learning_rate': 7.223699202430686e-05, 'epoch': 0.77}

 77%|███████▋  | 2099/2733 [7:49:56<2:20:36, 13.31s/it]
 77%|███████▋  | 2100/2733 [7:50:10<2:22:41, 13.53s/it]
                                                       
{'loss': 1.6058, 'learning_rate': 7.21230535510824e-05, 'epoch': 0.77}

 77%|███████▋  | 2100/2733 [7:50:10<2:22:41, 13.53s/it]
 77%|███████▋  | 2101/2733 [7:50:23<2:20:52, 13.37s/it]
                                                       
{'loss': 1.5889, 'learning_rate': 7.200911507785795e-05, 'epoch': 0.77}

 77%|███████▋  | 2101/2733 [7:50:23<2:20:52, 13.37s/it]
 77%|███████▋  | 2102/2733 [7:50:36<2:18:59, 13.22s/it]
                                                       
{'loss': 1.6777, 'learning_rate': 7.18951766046335e-05, 'epoch': 0.77}

 77%|███████▋  | 2102/2733 [7:50:36<2:18:59, 13.22s/it]
 77%|███████▋  | 2103/2733 [7:50:49<2:17:20, 13.08s/it]
                                                       
{'loss': 1.6078, 'learning_rate': 7.178123813140902e-05, 'epoch': 0.77}

 77%|███████▋  | 2103/2733 [7:50:49<2:17:20, 13.08s/it]
 77%|███████▋  | 2104/2733 [7:51:03<2:19:37, 13.32s/it]
                                                       
{'loss': 1.6062, 'learning_rate': 7.166729965818457e-05, 'epoch': 0.77}

 77%|███████▋  | 2104/2733 [7:51:03<2:19:37, 13.32s/it]
 77%|███████▋  | 2105/2733 [7:51:16<2:18:17, 13.21s/it]
                                                       
{'loss': 1.6563, 'learning_rate': 7.155336118496011e-05, 'epoch': 0.77}

 77%|███████▋  | 2105/2733 [7:51:16<2:18:17, 13.21s/it]
 77%|███████▋  | 2106/2733 [7:51:28<2:17:19, 13.14s/it]
                                                       
{'loss': 1.7406, 'learning_rate': 7.143942271173566e-05, 'epoch': 0.77}

 77%|███████▋  | 2106/2733 [7:51:29<2:17:19, 13.14s/it]
 77%|███████▋  | 2107/2733 [7:51:42<2:17:52, 13.21s/it]
                                                       
{'loss': 1.6474, 'learning_rate': 7.13254842385112e-05, 'epoch': 0.77}

 77%|███████▋  | 2107/2733 [7:51:42<2:17:52, 13.21s/it]
 77%|███████▋  | 2108/2733 [7:51:55<2:18:31, 13.30s/it]
                                                       
{'loss': 1.6218, 'learning_rate': 7.121154576528674e-05, 'epoch': 0.77}

 77%|███████▋  | 2108/2733 [7:51:55<2:18:31, 13.30s/it]
 77%|███████▋  | 2109/2733 [7:52:08<2:17:16, 13.20s/it]
                                                       
{'loss': 1.6833, 'learning_rate': 7.109760729206229e-05, 'epoch': 0.77}

 77%|███████▋  | 2109/2733 [7:52:08<2:17:16, 13.20s/it]
 77%|███████▋  | 2110/2733 [7:52:22<2:17:42, 13.26s/it]
                                                       
{'loss': 1.7175, 'learning_rate': 7.098366881883782e-05, 'epoch': 0.77}

 77%|███████▋  | 2110/2733 [7:52:22<2:17:42, 13.26s/it]
 77%|███████▋  | 2111/2733 [7:52:34<2:14:41, 12.99s/it]
                                                       
{'loss': 1.6518, 'learning_rate': 7.086973034561336e-05, 'epoch': 0.77}

 77%|███████▋  | 2111/2733 [7:52:34<2:14:41, 12.99s/it]
 77%|███████▋  | 2112/2733 [7:52:47<2:14:58, 13.04s/it]
                                                       
{'loss': 1.6266, 'learning_rate': 7.07557918723889e-05, 'epoch': 0.77}

 77%|███████▋  | 2112/2733 [7:52:47<2:14:58, 13.04s/it]
 77%|███████▋  | 2113/2733 [7:53:01<2:15:45, 13.14s/it]
                                                       
{'loss': 1.5972, 'learning_rate': 7.064185339916444e-05, 'epoch': 0.77}

 77%|███████▋  | 2113/2733 [7:53:01<2:15:45, 13.14s/it]
 77%|███████▋  | 2114/2733 [7:53:14<2:15:30, 13.13s/it]
                                                       
{'loss': 1.6475, 'learning_rate': 7.052791492593998e-05, 'epoch': 0.77}

 77%|███████▋  | 2114/2733 [7:53:14<2:15:30, 13.13s/it]
 77%|███████▋  | 2115/2733 [7:53:26<2:13:18, 12.94s/it]
                                                       
{'loss': 1.5457, 'learning_rate': 7.041397645271552e-05, 'epoch': 0.77}

 77%|███████▋  | 2115/2733 [7:53:26<2:13:18, 12.94s/it]
 77%|███████▋  | 2116/2733 [7:53:39<2:14:00, 13.03s/it]
                                                       
{'loss': 1.7815, 'learning_rate': 7.030003797949107e-05, 'epoch': 0.77}

 77%|███████▋  | 2116/2733 [7:53:40<2:14:00, 13.03s/it]
 77%|███████▋  | 2117/2733 [7:53:53<2:15:48, 13.23s/it]
                                                       
{'loss': 1.7606, 'learning_rate': 7.018609950626661e-05, 'epoch': 0.77}

 77%|███████▋  | 2117/2733 [7:53:53<2:15:48, 13.23s/it]
 77%|███████▋  | 2118/2733 [7:54:07<2:16:44, 13.34s/it]
                                                       
{'loss': 1.7399, 'learning_rate': 7.007216103304216e-05, 'epoch': 0.77}

 77%|███████▋  | 2118/2733 [7:54:07<2:16:44, 13.34s/it]
 78%|███████▊  | 2119/2733 [7:54:19<2:14:03, 13.10s/it]
                                                       
{'loss': 1.6996, 'learning_rate': 6.99582225598177e-05, 'epoch': 0.78}

 78%|███████▊  | 2119/2733 [7:54:19<2:14:03, 13.10s/it]
 78%|███████▊  | 2120/2733 [7:54:33<2:14:24, 13.16s/it]
                                                       
{'loss': 1.6481, 'learning_rate': 6.984428408659323e-05, 'epoch': 0.78}

 78%|███████▊  | 2120/2733 [7:54:33<2:14:24, 13.16s/it]
 78%|███████▊  | 2121/2733 [7:54:47<2:17:27, 13.48s/it]
                                                       
{'loss': 1.705, 'learning_rate': 6.973034561336877e-05, 'epoch': 0.78}

 78%|███████▊  | 2121/2733 [7:54:47<2:17:27, 13.48s/it]
 78%|███████▊  | 2122/2733 [7:55:00<2:16:11, 13.37s/it]
                                                       
{'loss': 1.718, 'learning_rate': 6.961640714014432e-05, 'epoch': 0.78}

 78%|███████▊  | 2122/2733 [7:55:00<2:16:11, 13.37s/it]
 78%|███████▊  | 2123/2733 [7:55:13<2:14:24, 13.22s/it]
                                                       
{'loss': 1.645, 'learning_rate': 6.950246866691986e-05, 'epoch': 0.78}

 78%|███████▊  | 2123/2733 [7:55:13<2:14:24, 13.22s/it]
 78%|███████▊  | 2124/2733 [7:55:27<2:16:22, 13.44s/it]
                                                       
{'loss': 1.7297, 'learning_rate': 6.938853019369539e-05, 'epoch': 0.78}

 78%|███████▊  | 2124/2733 [7:55:27<2:16:22, 13.44s/it]
 78%|███████▊  | 2125/2733 [7:55:40<2:16:49, 13.50s/it]
                                                       
{'loss': 1.6452, 'learning_rate': 6.927459172047094e-05, 'epoch': 0.78}

 78%|███████▊  | 2125/2733 [7:55:40<2:16:49, 13.50s/it]
 78%|███████▊  | 2126/2733 [7:55:54<2:17:14, 13.57s/it]
                                                       
{'loss': 1.69, 'learning_rate': 6.916065324724648e-05, 'epoch': 0.78}

 78%|███████▊  | 2126/2733 [7:55:54<2:17:14, 13.57s/it]
 78%|███████▊  | 2127/2733 [7:56:08<2:17:30, 13.61s/it]
                                                       
{'loss': 1.7607, 'learning_rate': 6.904671477402202e-05, 'epoch': 0.78}

 78%|███████▊  | 2127/2733 [7:56:08<2:17:30, 13.61s/it]
 78%|███████▊  | 2128/2733 [7:56:21<2:16:08, 13.50s/it]
                                                       
{'loss': 1.6333, 'learning_rate': 6.893277630079757e-05, 'epoch': 0.78}

 78%|███████▊  | 2128/2733 [7:56:21<2:16:08, 13.50s/it]
 78%|███████▊  | 2129/2733 [7:56:35<2:16:01, 13.51s/it]
                                                       
{'loss': 1.6879, 'learning_rate': 6.881883782757311e-05, 'epoch': 0.78}

 78%|███████▊  | 2129/2733 [7:56:35<2:16:01, 13.51s/it]
 78%|███████▊  | 2130/2733 [7:56:49<2:17:03, 13.64s/it]
                                                       
{'loss': 1.556, 'learning_rate': 6.870489935434865e-05, 'epoch': 0.78}

 78%|███████▊  | 2130/2733 [7:56:49<2:17:03, 13.64s/it]
 78%|███████▊  | 2131/2733 [7:57:01<2:14:27, 13.40s/it]
                                                       
{'loss': 1.667, 'learning_rate': 6.859096088112419e-05, 'epoch': 0.78}

 78%|███████▊  | 2131/2733 [7:57:01<2:14:27, 13.40s/it]
 78%|███████▊  | 2132/2733 [7:57:15<2:15:08, 13.49s/it]
                                                       
{'loss': 1.6309, 'learning_rate': 6.847702240789973e-05, 'epoch': 0.78}

 78%|███████▊  | 2132/2733 [7:57:15<2:15:08, 13.49s/it]
 78%|███████▊  | 2133/2733 [7:57:28<2:14:00, 13.40s/it]
                                                       
{'loss': 1.71, 'learning_rate': 6.836308393467527e-05, 'epoch': 0.78}

 78%|███████▊  | 2133/2733 [7:57:28<2:14:00, 13.40s/it]
 78%|███████▊  | 2134/2733 [7:57:42<2:14:21, 13.46s/it]
                                                       
{'loss': 1.5633, 'learning_rate': 6.82491454614508e-05, 'epoch': 0.78}

 78%|███████▊  | 2134/2733 [7:57:42<2:14:21, 13.46s/it]
 78%|███████▊  | 2135/2733 [7:57:55<2:13:05, 13.35s/it]
                                                       
{'loss': 1.6514, 'learning_rate': 6.813520698822635e-05, 'epoch': 0.78}

 78%|███████▊  | 2135/2733 [7:57:55<2:13:05, 13.35s/it]
 78%|███████▊  | 2136/2733 [7:58:08<2:12:40, 13.33s/it]
                                                       
{'loss': 1.7256, 'learning_rate': 6.802126851500189e-05, 'epoch': 0.78}

 78%|███████▊  | 2136/2733 [7:58:08<2:12:40, 13.33s/it]
 78%|███████▊  | 2137/2733 [7:58:22<2:13:40, 13.46s/it]
                                                       
{'loss': 1.6544, 'learning_rate': 6.790733004177744e-05, 'epoch': 0.78}

 78%|███████▊  | 2137/2733 [7:58:22<2:13:40, 13.46s/it]
 78%|███████▊  | 2138/2733 [7:58:35<2:13:05, 13.42s/it]
                                                       
{'loss': 1.6244, 'learning_rate': 6.779339156855298e-05, 'epoch': 0.78}

 78%|███████▊  | 2138/2733 [7:58:35<2:13:05, 13.42s/it]
 78%|███████▊  | 2139/2733 [7:58:50<2:15:10, 13.65s/it]
                                                       
{'loss': 1.7547, 'learning_rate': 6.767945309532852e-05, 'epoch': 0.78}

 78%|███████▊  | 2139/2733 [7:58:50<2:15:10, 13.65s/it]
 78%|███████▊  | 2140/2733 [7:59:03<2:14:41, 13.63s/it]
                                                       
{'loss': 1.7086, 'learning_rate': 6.756551462210407e-05, 'epoch': 0.78}

 78%|███████▊  | 2140/2733 [7:59:03<2:14:41, 13.63s/it]
 78%|███████▊  | 2141/2733 [7:59:16<2:13:17, 13.51s/it]
                                                       
{'loss': 1.7139, 'learning_rate': 6.74515761488796e-05, 'epoch': 0.78}

 78%|███████▊  | 2141/2733 [7:59:16<2:13:17, 13.51s/it]
 78%|███████▊  | 2142/2733 [7:59:30<2:13:26, 13.55s/it]
                                                       
{'loss': 1.7723, 'learning_rate': 6.733763767565514e-05, 'epoch': 0.78}

 78%|███████▊  | 2142/2733 [7:59:30<2:13:26, 13.55s/it]
 78%|███████▊  | 2143/2733 [7:59:43<2:12:04, 13.43s/it]
                                                       
{'loss': 1.6294, 'learning_rate': 6.722369920243068e-05, 'epoch': 0.78}

 78%|███████▊  | 2143/2733 [7:59:43<2:12:04, 13.43s/it]
 78%|███████▊  | 2144/2733 [7:59:56<2:09:59, 13.24s/it]
                                                       
{'loss': 1.6215, 'learning_rate': 6.710976072920622e-05, 'epoch': 0.78}

 78%|███████▊  | 2144/2733 [7:59:56<2:09:59, 13.24s/it]
 78%|███████▊  | 2145/2733 [8:00:09<2:09:13, 13.19s/it]
                                                       
{'loss': 1.7036, 'learning_rate': 6.699582225598176e-05, 'epoch': 0.78}

 78%|███████▊  | 2145/2733 [8:00:09<2:09:13, 13.19s/it]
 79%|███████▊  | 2146/2733 [8:00:22<2:08:30, 13.14s/it]
                                                       
{'loss': 1.6141, 'learning_rate': 6.68818837827573e-05, 'epoch': 0.79}

 79%|███████▊  | 2146/2733 [8:00:22<2:08:30, 13.14s/it]
 79%|███████▊  | 2147/2733 [8:00:35<2:07:42, 13.08s/it]
                                                       
{'loss': 1.7076, 'learning_rate': 6.676794530953285e-05, 'epoch': 0.79}

 79%|███████▊  | 2147/2733 [8:00:35<2:07:42, 13.08s/it]
 79%|███████▊  | 2148/2733 [8:00:48<2:08:16, 13.16s/it]
                                                       
{'loss': 1.5848, 'learning_rate': 6.665400683630839e-05, 'epoch': 0.79}

 79%|███████▊  | 2148/2733 [8:00:48<2:08:16, 13.16s/it]
 79%|███████▊  | 2149/2733 [8:01:02<2:08:07, 13.16s/it]
                                                       
{'loss': 1.6357, 'learning_rate': 6.654006836308393e-05, 'epoch': 0.79}

 79%|███████▊  | 2149/2733 [8:01:02<2:08:07, 13.16s/it]
 79%|███████▊  | 2150/2733 [8:01:14<2:06:27, 13.01s/it]
                                                       
{'loss': 1.6839, 'learning_rate': 6.642612988985948e-05, 'epoch': 0.79}

 79%|███████▊  | 2150/2733 [8:01:14<2:06:27, 13.01s/it]
 79%|███████▊  | 2151/2733 [8:01:27<2:06:39, 13.06s/it]
                                                       
{'loss': 1.6958, 'learning_rate': 6.631219141663501e-05, 'epoch': 0.79}

 79%|███████▊  | 2151/2733 [8:01:27<2:06:39, 13.06s/it]
 79%|███████▊  | 2152/2733 [8:01:41<2:08:52, 13.31s/it]
                                                       
{'loss': 1.6651, 'learning_rate': 6.619825294341055e-05, 'epoch': 0.79}

 79%|███████▊  | 2152/2733 [8:01:41<2:08:52, 13.31s/it]
 79%|███████▉  | 2153/2733 [8:01:55<2:09:08, 13.36s/it]
                                                       
{'loss': 1.6938, 'learning_rate': 6.60843144701861e-05, 'epoch': 0.79}

 79%|███████▉  | 2153/2733 [8:01:55<2:09:08, 13.36s/it]
 79%|███████▉  | 2154/2733 [8:02:08<2:08:08, 13.28s/it]
                                                       
{'loss': 1.5078, 'learning_rate': 6.597037599696164e-05, 'epoch': 0.79}

 79%|███████▉  | 2154/2733 [8:02:08<2:08:08, 13.28s/it]
 79%|███████▉  | 2155/2733 [8:02:21<2:07:34, 13.24s/it]
                                                       
{'loss': 1.7483, 'learning_rate': 6.585643752373717e-05, 'epoch': 0.79}

 79%|███████▉  | 2155/2733 [8:02:21<2:07:34, 13.24s/it]
 79%|███████▉  | 2156/2733 [8:02:35<2:08:33, 13.37s/it]
                                                       
{'loss': 1.6115, 'learning_rate': 6.574249905051272e-05, 'epoch': 0.79}

 79%|███████▉  | 2156/2733 [8:02:35<2:08:33, 13.37s/it]
 79%|███████▉  | 2157/2733 [8:02:47<2:06:32, 13.18s/it]
                                                       
{'loss': 1.6536, 'learning_rate': 6.562856057728826e-05, 'epoch': 0.79}

 79%|███████▉  | 2157/2733 [8:02:47<2:06:32, 13.18s/it]
 79%|███████▉  | 2158/2733 [8:03:00<2:06:07, 13.16s/it]
                                                       
{'loss': 1.6456, 'learning_rate': 6.55146221040638e-05, 'epoch': 0.79}

 79%|███████▉  | 2158/2733 [8:03:00<2:06:07, 13.16s/it]
 79%|███████▉  | 2159/2733 [8:03:13<2:04:58, 13.06s/it]
                                                       
{'loss': 1.5846, 'learning_rate': 6.540068363083935e-05, 'epoch': 0.79}

 79%|███████▉  | 2159/2733 [8:03:13<2:04:58, 13.06s/it]
 79%|███████▉  | 2160/2733 [8:03:27<2:05:47, 13.17s/it]
                                                       
{'loss': 1.748, 'learning_rate': 6.528674515761488e-05, 'epoch': 0.79}

 79%|███████▉  | 2160/2733 [8:03:27<2:05:47, 13.17s/it]
 79%|███████▉  | 2161/2733 [8:03:39<2:03:57, 13.00s/it]
                                                       
{'loss': 1.6624, 'learning_rate': 6.517280668439042e-05, 'epoch': 0.79}

 79%|███████▉  | 2161/2733 [8:03:39<2:03:57, 13.00s/it]
 79%|███████▉  | 2162/2733 [8:03:52<2:02:32, 12.88s/it]
                                                       
{'loss': 1.652, 'learning_rate': 6.505886821116596e-05, 'epoch': 0.79}

 79%|███████▉  | 2162/2733 [8:03:52<2:02:32, 12.88s/it]
 79%|███████▉  | 2163/2733 [8:04:05<2:02:29, 12.89s/it]
                                                       
{'loss': 1.561, 'learning_rate': 6.494492973794151e-05, 'epoch': 0.79}

 79%|███████▉  | 2163/2733 [8:04:05<2:02:29, 12.89s/it]
 79%|███████▉  | 2164/2733 [8:04:18<2:03:27, 13.02s/it]
                                                       
{'loss': 1.6971, 'learning_rate': 6.483099126471705e-05, 'epoch': 0.79}

 79%|███████▉  | 2164/2733 [8:04:18<2:03:27, 13.02s/it]
 79%|███████▉  | 2165/2733 [8:04:32<2:06:34, 13.37s/it]
                                                       
{'loss': 1.7339, 'learning_rate': 6.471705279149258e-05, 'epoch': 0.79}

 79%|███████▉  | 2165/2733 [8:04:32<2:06:34, 13.37s/it]
 79%|███████▉  | 2166/2733 [8:04:47<2:08:51, 13.64s/it]
                                                       
{'loss': 1.7225, 'learning_rate': 6.460311431826813e-05, 'epoch': 0.79}

 79%|███████▉  | 2166/2733 [8:04:47<2:08:51, 13.64s/it]
 79%|███████▉  | 2167/2733 [8:05:00<2:09:08, 13.69s/it]
                                                       
{'loss': 1.6552, 'learning_rate': 6.448917584504367e-05, 'epoch': 0.79}

 79%|███████▉  | 2167/2733 [8:05:00<2:09:08, 13.69s/it]
 79%|███████▉  | 2168/2733 [8:05:15<2:10:32, 13.86s/it]
                                                       
{'loss': 1.5871, 'learning_rate': 6.437523737181921e-05, 'epoch': 0.79}

 79%|███████▉  | 2168/2733 [8:05:15<2:10:32, 13.86s/it]
 79%|███████▉  | 2169/2733 [8:05:28<2:09:22, 13.76s/it]
                                                       
{'loss': 1.6569, 'learning_rate': 6.426129889859475e-05, 'epoch': 0.79}

 79%|███████▉  | 2169/2733 [8:05:28<2:09:22, 13.76s/it]
 79%|███████▉  | 2170/2733 [8:05:42<2:08:42, 13.72s/it]
                                                       
{'loss': 1.643, 'learning_rate': 6.414736042537029e-05, 'epoch': 0.79}

 79%|███████▉  | 2170/2733 [8:05:42<2:08:42, 13.72s/it]
 79%|███████▉  | 2171/2733 [8:05:55<2:07:55, 13.66s/it]
                                                       
{'loss': 1.6504, 'learning_rate': 6.403342195214583e-05, 'epoch': 0.79}

 79%|███████▉  | 2171/2733 [8:05:55<2:07:55, 13.66s/it]
 79%|███████▉  | 2172/2733 [8:06:08<2:05:53, 13.46s/it]
                                                       
{'loss': 1.6354, 'learning_rate': 6.391948347892138e-05, 'epoch': 0.79}

 79%|███████▉  | 2172/2733 [8:06:08<2:05:53, 13.46s/it]
 80%|███████▉  | 2173/2733 [8:06:21<2:03:35, 13.24s/it]
                                                       
{'loss': 1.5033, 'learning_rate': 6.380554500569692e-05, 'epoch': 0.8}

 80%|███████▉  | 2173/2733 [8:06:21<2:03:35, 13.24s/it]
 80%|███████▉  | 2174/2733 [8:06:34<2:03:46, 13.29s/it]
                                                       
{'loss': 1.6299, 'learning_rate': 6.369160653247246e-05, 'epoch': 0.8}

 80%|███████▉  | 2174/2733 [8:06:35<2:03:46, 13.29s/it]
 80%|███████▉  | 2175/2733 [8:06:48<2:02:56, 13.22s/it]
                                                       
{'loss': 1.6406, 'learning_rate': 6.3577668059248e-05, 'epoch': 0.8}

 80%|███████▉  | 2175/2733 [8:06:48<2:02:56, 13.22s/it]
 80%|███████▉  | 2176/2733 [8:07:01<2:04:31, 13.41s/it]
                                                       
{'loss': 1.7937, 'learning_rate': 6.346372958602354e-05, 'epoch': 0.8}

 80%|███████▉  | 2176/2733 [8:07:01<2:04:31, 13.41s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 80%|███████▉  | 2177/2733 [8:07:15<2:04:41, 13.46s/it]
                                                       
{'loss': 1.6412, 'learning_rate': 6.334979111279908e-05, 'epoch': 0.8}

 80%|███████▉  | 2177/2733 [8:07:15<2:04:41, 13.46s/it]
 80%|███████▉  | 2178/2733 [8:07:27<2:01:46, 13.16s/it]
                                                       
{'loss': 1.65, 'learning_rate': 6.323585263957463e-05, 'epoch': 0.8}

 80%|███████▉  | 2178/2733 [8:07:27<2:01:46, 13.16s/it]
 80%|███████▉  | 2179/2733 [8:07:41<2:03:27, 13.37s/it]
                                                       
{'loss': 1.623, 'learning_rate': 6.312191416635016e-05, 'epoch': 0.8}

 80%|███████▉  | 2179/2733 [8:07:41<2:03:27, 13.37s/it]
 80%|███████▉  | 2180/2733 [8:07:55<2:04:00, 13.45s/it]
                                                       
{'loss': 1.7343, 'learning_rate': 6.30079756931257e-05, 'epoch': 0.8}

 80%|███████▉  | 2180/2733 [8:07:55<2:04:00, 13.45s/it]
 80%|███████▉  | 2181/2733 [8:08:09<2:04:07, 13.49s/it]
                                                       
{'loss': 1.5681, 'learning_rate': 6.289403721990124e-05, 'epoch': 0.8}

 80%|███████▉  | 2181/2733 [8:08:09<2:04:07, 13.49s/it]
 80%|███████▉  | 2182/2733 [8:08:22<2:04:13, 13.53s/it]
                                                       
{'loss': 1.6873, 'learning_rate': 6.278009874667679e-05, 'epoch': 0.8}

 80%|███████▉  | 2182/2733 [8:08:22<2:04:13, 13.53s/it]
 80%|███████▉  | 2183/2733 [8:08:35<2:02:13, 13.33s/it]
                                                       
{'loss': 1.5668, 'learning_rate': 6.266616027345233e-05, 'epoch': 0.8}

 80%|███████▉  | 2183/2733 [8:08:35<2:02:13, 13.33s/it]
 80%|███████▉  | 2184/2733 [8:08:48<2:01:56, 13.33s/it]
                                                       
{'loss': 1.6961, 'learning_rate': 6.255222180022788e-05, 'epoch': 0.8}

 80%|███████▉  | 2184/2733 [8:08:48<2:01:56, 13.33s/it]
 80%|███████▉  | 2185/2733 [8:09:02<2:01:34, 13.31s/it]
                                                       
{'loss': 1.6436, 'learning_rate': 6.243828332700342e-05, 'epoch': 0.8}

 80%|███████▉  | 2185/2733 [8:09:02<2:01:34, 13.31s/it]
 80%|███████▉  | 2186/2733 [8:09:15<2:02:29, 13.44s/it]
                                                       
{'loss': 1.6722, 'learning_rate': 6.243828332700342e-05, 'epoch': 0.8}

 80%|███████▉  | 2186/2733 [8:09:15<2:02:29, 13.44s/it]
 80%|████████  | 2187/2733 [8:09:28<2:01:28, 13.35s/it]
                                                       
{'loss': 1.6147, 'learning_rate': 6.232434485377895e-05, 'epoch': 0.8}

 80%|████████  | 2187/2733 [8:09:29<2:01:28, 13.35s/it]
 80%|████████  | 2188/2733 [8:09:42<2:00:37, 13.28s/it]
                                                       
{'loss': 1.6697, 'learning_rate': 6.22104063805545e-05, 'epoch': 0.8}

 80%|████████  | 2188/2733 [8:09:42<2:00:37, 13.28s/it]
 80%|████████  | 2189/2733 [8:09:55<2:00:02, 13.24s/it]
                                                       
{'loss': 1.6337, 'learning_rate': 6.209646790733004e-05, 'epoch': 0.8}

 80%|████████  | 2189/2733 [8:09:55<2:00:02, 13.24s/it]
 80%|████████  | 2190/2733 [8:10:08<2:01:09, 13.39s/it]
                                                       
{'loss': 1.7665, 'learning_rate': 6.198252943410557e-05, 'epoch': 0.8}

 80%|████████  | 2190/2733 [8:10:09<2:01:09, 13.39s/it]
 80%|████████  | 2191/2733 [8:10:22<2:00:34, 13.35s/it]
                                                       
{'loss': 1.6776, 'learning_rate': 6.186859096088111e-05, 'epoch': 0.8}

 80%|████████  | 2191/2733 [8:10:22<2:00:34, 13.35s/it]
 80%|████████  | 2192/2733 [8:10:36<2:02:38, 13.60s/it]
                                                       
{'loss': 1.6557, 'learning_rate': 6.175465248765666e-05, 'epoch': 0.8}

 80%|████████  | 2192/2733 [8:10:36<2:02:38, 13.60s/it]
 80%|████████  | 2193/2733 [8:10:48<1:59:32, 13.28s/it]
                                                       
{'loss': 1.7178, 'learning_rate': 6.16407140144322e-05, 'epoch': 0.8}

 80%|████████  | 2193/2733 [8:10:48<1:59:32, 13.28s/it]
 80%|████████  | 2194/2733 [8:11:02<2:01:07, 13.48s/it]
                                                       
{'loss': 1.7165, 'learning_rate': 6.152677554120774e-05, 'epoch': 0.8}

 80%|████████  | 2194/2733 [8:11:02<2:01:07, 13.48s/it]
 80%|████████  | 2195/2733 [8:11:15<1:59:44, 13.35s/it]
                                                       
{'loss': 1.6814, 'learning_rate': 6.141283706798329e-05, 'epoch': 0.8}

 80%|████████  | 2195/2733 [8:11:15<1:59:44, 13.35s/it]
 80%|████████  | 2196/2733 [8:11:29<2:00:03, 13.41s/it]
                                                       
{'loss': 1.7261, 'learning_rate': 6.129889859475883e-05, 'epoch': 0.8}

 80%|████████  | 2196/2733 [8:11:29<2:00:03, 13.41s/it]
 80%|████████  | 2197/2733 [8:11:43<2:00:53, 13.53s/it]
                                                       
{'loss': 1.5998, 'learning_rate': 6.118496012153436e-05, 'epoch': 0.8}

 80%|████████  | 2197/2733 [8:11:43<2:00:53, 13.53s/it]
 80%|████████  | 2198/2733 [8:11:56<1:59:11, 13.37s/it]
                                                       
{'loss': 1.8143, 'learning_rate': 6.10710216483099e-05, 'epoch': 0.8}

 80%|████████  | 2198/2733 [8:11:56<1:59:11, 13.37s/it]
 80%|████████  | 2199/2733 [8:12:09<1:58:40, 13.33s/it]
                                                       
{'loss': 1.6412, 'learning_rate': 6.095708317508545e-05, 'epoch': 0.8}

 80%|████████  | 2199/2733 [8:12:09<1:58:40, 13.33s/it]
 80%|████████  | 2200/2733 [8:12:23<2:00:42, 13.59s/it]
                                                       
{'loss': 1.7349, 'learning_rate': 6.0843144701860994e-05, 'epoch': 0.8}

 80%|████████  | 2200/2733 [8:12:23<2:00:42, 13.59s/it]
 81%|████████  | 2201/2733 [8:12:36<1:59:05, 13.43s/it]
                                                       
{'loss': 1.6658, 'learning_rate': 6.072920622863653e-05, 'epoch': 0.81}

 81%|████████  | 2201/2733 [8:12:36<1:59:05, 13.43s/it]
 81%|████████  | 2202/2733 [8:12:50<1:59:04, 13.46s/it]
                                                       
{'loss': 1.6765, 'learning_rate': 6.0615267755412075e-05, 'epoch': 0.81}

 81%|████████  | 2202/2733 [8:12:50<1:59:04, 13.46s/it]
 81%|████████  | 2203/2733 [8:13:04<2:00:22, 13.63s/it]
                                                       
{'loss': 1.6651, 'learning_rate': 6.050132928218762e-05, 'epoch': 0.81}

 81%|████████  | 2203/2733 [8:13:04<2:00:22, 13.63s/it]
 81%|████████  | 2204/2733 [8:13:17<2:00:01, 13.61s/it]
                                                       
{'loss': 1.6779, 'learning_rate': 6.038739080896315e-05, 'epoch': 0.81}

 81%|████████  | 2204/2733 [8:13:17<2:00:01, 13.61s/it]
 81%|████████  | 2205/2733 [8:13:31<1:59:13, 13.55s/it]
                                                       
{'loss': 1.69, 'learning_rate': 6.0273452335738693e-05, 'epoch': 0.81}

 81%|████████  | 2205/2733 [8:13:31<1:59:13, 13.55s/it]
 81%|████████  | 2206/2733 [8:13:45<1:59:56, 13.66s/it]
                                                       
{'loss': 1.6948, 'learning_rate': 6.015951386251424e-05, 'epoch': 0.81}

 81%|████████  | 2206/2733 [8:13:45<1:59:56, 13.66s/it]
 81%|████████  | 2207/2733 [8:13:58<1:59:19, 13.61s/it]
                                                       
{'loss': 1.6674, 'learning_rate': 6.004557538928978e-05, 'epoch': 0.81}

 81%|████████  | 2207/2733 [8:13:58<1:59:19, 13.61s/it]
 81%|████████  | 2208/2733 [8:14:11<1:56:00, 13.26s/it]
                                                       
{'loss': 1.6916, 'learning_rate': 5.993163691606532e-05, 'epoch': 0.81}

 81%|████████  | 2208/2733 [8:14:11<1:56:00, 13.26s/it]
 81%|████████  | 2209/2733 [8:14:24<1:55:26, 13.22s/it]
                                                       
{'loss': 1.6143, 'learning_rate': 5.981769844284086e-05, 'epoch': 0.81}

 81%|████████  | 2209/2733 [8:14:24<1:55:26, 13.22s/it]
 81%|████████  | 2210/2733 [8:14:37<1:55:11, 13.21s/it]
                                                       
{'loss': 1.6571, 'learning_rate': 5.9703759969616406e-05, 'epoch': 0.81}

 81%|████████  | 2210/2733 [8:14:37<1:55:11, 13.21s/it]
 81%|████████  | 2211/2733 [8:14:49<1:52:44, 12.96s/it]
                                                       
{'loss': 1.5983, 'learning_rate': 5.958982149639194e-05, 'epoch': 0.81}

 81%|████████  | 2211/2733 [8:14:49<1:52:44, 12.96s/it]
 81%|████████  | 2212/2733 [8:15:02<1:52:33, 12.96s/it]
                                                       
{'loss': 1.586, 'learning_rate': 5.947588302316748e-05, 'epoch': 0.81}

 81%|████████  | 2212/2733 [8:15:02<1:52:33, 12.96s/it]
 81%|████████  | 2213/2733 [8:15:16<1:53:25, 13.09s/it]
                                                       
{'loss': 1.5893, 'learning_rate': 5.9361944549943024e-05, 'epoch': 0.81}

 81%|████████  | 2213/2733 [8:15:16<1:53:25, 13.09s/it]
 81%|████████  | 2214/2733 [8:15:29<1:54:55, 13.29s/it]
                                                       
{'loss': 1.6825, 'learning_rate': 5.924800607671856e-05, 'epoch': 0.81}

 81%|████████  | 2214/2733 [8:15:30<1:54:55, 13.29s/it]
 81%|████████  | 2215/2733 [8:15:43<1:55:48, 13.41s/it]
                                                       
{'loss': 1.7365, 'learning_rate': 5.9134067603494105e-05, 'epoch': 0.81}

 81%|████████  | 2215/2733 [8:15:43<1:55:48, 13.41s/it]
 81%|████████  | 2216/2733 [8:15:56<1:54:33, 13.29s/it]
                                                       
{'loss': 1.6513, 'learning_rate': 5.902012913026965e-05, 'epoch': 0.81}

 81%|████████  | 2216/2733 [8:15:56<1:54:33, 13.29s/it]
 81%|████████  | 2217/2733 [8:16:10<1:55:28, 13.43s/it]
                                                       
{'loss': 1.6828, 'learning_rate': 5.890619065704519e-05, 'epoch': 0.81}

 81%|████████  | 2217/2733 [8:16:10<1:55:28, 13.43s/it]
 81%|████████  | 2218/2733 [8:16:23<1:54:27, 13.33s/it]
                                                       
{'loss': 1.6661, 'learning_rate': 5.879225218382073e-05, 'epoch': 0.81}

 81%|████████  | 2218/2733 [8:16:23<1:54:27, 13.33s/it]
 81%|████████  | 2219/2733 [8:16:36<1:53:19, 13.23s/it]
                                                       
{'loss': 1.6597, 'learning_rate': 5.8678313710596274e-05, 'epoch': 0.81}

 81%|████████  | 2219/2733 [8:16:36<1:53:19, 13.23s/it]
 81%|████████  | 2220/2733 [8:16:50<1:53:57, 13.33s/it]
                                                       
{'loss': 1.6756, 'learning_rate': 5.856437523737182e-05, 'epoch': 0.81}

 81%|████████  | 2220/2733 [8:16:50<1:53:57, 13.33s/it]
 81%|████████▏ | 2221/2733 [8:17:03<1:53:37, 13.32s/it]
                                                       
{'loss': 1.6347, 'learning_rate': 5.845043676414735e-05, 'epoch': 0.81}

 81%|████████▏ | 2221/2733 [8:17:03<1:53:37, 13.32s/it]
 81%|████████▏ | 2222/2733 [8:17:16<1:52:32, 13.21s/it]
                                                       
{'loss': 1.6425, 'learning_rate': 5.833649829092289e-05, 'epoch': 0.81}

 81%|████████▏ | 2222/2733 [8:17:16<1:52:32, 13.21s/it]
 81%|████████▏ | 2223/2733 [8:17:29<1:52:19, 13.21s/it]
                                                       
{'loss': 1.6427, 'learning_rate': 5.8222559817698436e-05, 'epoch': 0.81}

 81%|████████▏ | 2223/2733 [8:17:29<1:52:19, 13.21s/it]
 81%|████████▏ | 2224/2733 [8:17:42<1:52:14, 13.23s/it]
                                                       
{'loss': 1.6628, 'learning_rate': 5.810862134447398e-05, 'epoch': 0.81}

 81%|████████▏ | 2224/2733 [8:17:42<1:52:14, 13.23s/it]
 81%|████████▏ | 2225/2733 [8:17:55<1:51:46, 13.20s/it]
                                                       
{'loss': 1.683, 'learning_rate': 5.799468287124952e-05, 'epoch': 0.81}

 81%|████████▏ | 2225/2733 [8:17:56<1:51:46, 13.20s/it]
 81%|████████▏ | 2226/2733 [8:18:09<1:53:12, 13.40s/it]
                                                       
{'loss': 1.6444, 'learning_rate': 5.788074439802506e-05, 'epoch': 0.81}

 81%|████████▏ | 2226/2733 [8:18:09<1:53:12, 13.40s/it]
 81%|████████▏ | 2227/2733 [8:18:24<1:55:00, 13.64s/it]
                                                       
{'loss': 1.6406, 'learning_rate': 5.7766805924800605e-05, 'epoch': 0.81}

 81%|████████▏ | 2227/2733 [8:18:24<1:55:00, 13.64s/it]
 82%|████████▏ | 2228/2733 [8:18:37<1:54:44, 13.63s/it]
                                                       
{'loss': 1.6772, 'learning_rate': 5.765286745157614e-05, 'epoch': 0.82}

 82%|████████▏ | 2228/2733 [8:18:37<1:54:44, 13.63s/it]
 82%|████████▏ | 2229/2733 [8:18:51<1:54:24, 13.62s/it]
                                                       
{'loss': 1.6114, 'learning_rate': 5.7538928978351686e-05, 'epoch': 0.82}

 82%|████████▏ | 2229/2733 [8:18:51<1:54:24, 13.62s/it]
 82%|████████▏ | 2230/2733 [8:19:04<1:54:02, 13.60s/it]
                                                       
{'loss': 1.6358, 'learning_rate': 5.742499050512723e-05, 'epoch': 0.82}

 82%|████████▏ | 2230/2733 [8:19:04<1:54:02, 13.60s/it]
 82%|████████▏ | 2231/2733 [8:19:17<1:52:37, 13.46s/it]
                                                       
{'loss': 1.7329, 'learning_rate': 5.7311052031902774e-05, 'epoch': 0.82}

 82%|████████▏ | 2231/2733 [8:19:17<1:52:37, 13.46s/it]
 82%|████████▏ | 2232/2733 [8:19:31<1:53:02, 13.54s/it]
                                                       
{'loss': 1.6086, 'learning_rate': 5.7197113558678304e-05, 'epoch': 0.82}

 82%|████████▏ | 2232/2733 [8:19:31<1:53:02, 13.54s/it]
 82%|████████▏ | 2233/2733 [8:19:45<1:52:50, 13.54s/it]
                                                       
{'loss': 1.6711, 'learning_rate': 5.708317508545385e-05, 'epoch': 0.82}

 82%|████████▏ | 2233/2733 [8:19:45<1:52:50, 13.54s/it]
 82%|████████▏ | 2234/2733 [8:19:58<1:51:14, 13.38s/it]
                                                       
{'loss': 1.6525, 'learning_rate': 5.696923661222939e-05, 'epoch': 0.82}

 82%|████████▏ | 2234/2733 [8:19:58<1:51:14, 13.38s/it]
 82%|████████▏ | 2235/2733 [8:20:11<1:50:17, 13.29s/it]
                                                       
{'loss': 1.6465, 'learning_rate': 5.685529813900493e-05, 'epoch': 0.82}

 82%|████████▏ | 2235/2733 [8:20:11<1:50:17, 13.29s/it]
 82%|████████▏ | 2236/2733 [8:20:24<1:50:44, 13.37s/it]
                                                       
{'loss': 1.6917, 'learning_rate': 5.674135966578047e-05, 'epoch': 0.82}

 82%|████████▏ | 2236/2733 [8:20:24<1:50:44, 13.37s/it]
 82%|████████▏ | 2237/2733 [8:20:38<1:50:32, 13.37s/it]
                                                       
{'loss': 1.6069, 'learning_rate': 5.662742119255602e-05, 'epoch': 0.82}

 82%|████████▏ | 2237/2733 [8:20:38<1:50:32, 13.37s/it]
 82%|████████▏ | 2238/2733 [8:20:51<1:50:14, 13.36s/it]
                                                       
{'loss': 1.6738, 'learning_rate': 5.651348271933156e-05, 'epoch': 0.82}

 82%|████████▏ | 2238/2733 [8:20:51<1:50:14, 13.36s/it]
 82%|████████▏ | 2239/2733 [8:21:05<1:50:34, 13.43s/it]
                                                       
{'loss': 1.7131, 'learning_rate': 5.63995442461071e-05, 'epoch': 0.82}

 82%|████████▏ | 2239/2733 [8:21:05<1:50:34, 13.43s/it]
 82%|████████▏ | 2240/2733 [8:21:19<1:52:10, 13.65s/it]
                                                       
{'loss': 1.8409, 'learning_rate': 5.628560577288264e-05, 'epoch': 0.82}

 82%|████████▏ | 2240/2733 [8:21:19<1:52:10, 13.65s/it]
 82%|████████▏ | 2241/2733 [8:21:32<1:51:09, 13.55s/it]
                                                       
{'loss': 1.6401, 'learning_rate': 5.6171667299658186e-05, 'epoch': 0.82}

 82%|████████▏ | 2241/2733 [8:21:32<1:51:09, 13.55s/it]
 82%|████████▏ | 2242/2733 [8:21:46<1:50:59, 13.56s/it]
                                                       
{'loss': 1.6131, 'learning_rate': 5.6057728826433716e-05, 'epoch': 0.82}

 82%|████████▏ | 2242/2733 [8:21:46<1:50:59, 13.56s/it]
 82%|████████▏ | 2243/2733 [8:21:59<1:49:08, 13.37s/it]
                                                       
{'loss': 1.6294, 'learning_rate': 5.594379035320926e-05, 'epoch': 0.82}

 82%|████████▏ | 2243/2733 [8:21:59<1:49:08, 13.37s/it]
 82%|████████▏ | 2244/2733 [8:22:12<1:49:16, 13.41s/it]
                                                       
{'loss': 1.6921, 'learning_rate': 5.5829851879984804e-05, 'epoch': 0.82}

 82%|████████▏ | 2244/2733 [8:22:12<1:49:16, 13.41s/it]
 82%|████████▏ | 2245/2733 [8:22:25<1:48:19, 13.32s/it]
                                                       
{'loss': 1.6234, 'learning_rate': 5.571591340676035e-05, 'epoch': 0.82}

 82%|████████▏ | 2245/2733 [8:22:25<1:48:19, 13.32s/it]
 82%|████████▏ | 2246/2733 [8:22:38<1:47:30, 13.25s/it]
                                                       
{'loss': 1.632, 'learning_rate': 5.5601974933535885e-05, 'epoch': 0.82}

 82%|████████▏ | 2246/2733 [8:22:38<1:47:30, 13.25s/it]
 82%|████████▏ | 2247/2733 [8:22:51<1:46:39, 13.17s/it]
                                                       
{'loss': 1.5475, 'learning_rate': 5.548803646031143e-05, 'epoch': 0.82}

 82%|████████▏ | 2247/2733 [8:22:51<1:46:39, 13.17s/it]
 82%|████████▏ | 2248/2733 [8:23:04<1:45:55, 13.10s/it]
                                                       
{'loss': 1.6334, 'learning_rate': 5.537409798708697e-05, 'epoch': 0.82}

 82%|████████▏ | 2248/2733 [8:23:04<1:45:55, 13.10s/it]
 82%|████████▏ | 2249/2733 [8:23:18<1:46:30, 13.20s/it]
                                                       
{'loss': 1.7276, 'learning_rate': 5.52601595138625e-05, 'epoch': 0.82}

 82%|████████▏ | 2249/2733 [8:23:18<1:46:30, 13.20s/it]
 82%|████████▏ | 2250/2733 [8:23:32<1:48:13, 13.44s/it]
                                                       
{'loss': 1.6483, 'learning_rate': 5.514622104063805e-05, 'epoch': 0.82}

 82%|████████▏ | 2250/2733 [8:23:32<1:48:13, 13.44s/it]
 82%|████████▏ | 2251/2733 [8:23:45<1:47:51, 13.43s/it]
                                                       
{'loss': 1.5924, 'learning_rate': 5.503228256741359e-05, 'epoch': 0.82}

 82%|████████▏ | 2251/2733 [8:23:45<1:47:51, 13.43s/it]
 82%|████████▏ | 2252/2733 [8:23:59<1:49:37, 13.68s/it]
                                                       
{'loss': 1.7424, 'learning_rate': 5.491834409418913e-05, 'epoch': 0.82}

 82%|████████▏ | 2252/2733 [8:23:59<1:49:37, 13.68s/it]
 82%|████████▏ | 2253/2733 [8:24:12<1:48:07, 13.52s/it]
                                                       
{'loss': 1.5774, 'learning_rate': 5.480440562096467e-05, 'epoch': 0.82}

 82%|████████▏ | 2253/2733 [8:24:13<1:48:07, 13.52s/it]
 82%|████████▏ | 2254/2733 [8:24:26<1:48:24, 13.58s/it]
                                                       
{'loss': 1.6218, 'learning_rate': 5.4690467147740216e-05, 'epoch': 0.82}

 82%|████████▏ | 2254/2733 [8:24:26<1:48:24, 13.58s/it]
 83%|████████▎ | 2255/2733 [8:24:40<1:49:00, 13.68s/it]
                                                       
{'loss': 1.6087, 'learning_rate': 5.457652867451576e-05, 'epoch': 0.83}

 83%|████████▎ | 2255/2733 [8:24:40<1:49:00, 13.68s/it]
 83%|████████▎ | 2256/2733 [8:24:53<1:45:59, 13.33s/it]
                                                       
{'loss': 1.6153, 'learning_rate': 5.44625902012913e-05, 'epoch': 0.83}

 83%|████████▎ | 2256/2733 [8:24:53<1:45:59, 13.33s/it]
 83%|████████▎ | 2257/2733 [8:25:07<1:47:01, 13.49s/it]
                                                       
{'loss': 1.7246, 'learning_rate': 5.434865172806684e-05, 'epoch': 0.83}

 83%|████████▎ | 2257/2733 [8:25:07<1:47:01, 13.49s/it]
 83%|████████▎ | 2258/2733 [8:25:21<1:48:20, 13.69s/it]
                                                       
{'loss': 1.687, 'learning_rate': 5.4234713254842385e-05, 'epoch': 0.83}

 83%|████████▎ | 2258/2733 [8:25:21<1:48:20, 13.69s/it]
 83%|████████▎ | 2259/2733 [8:25:35<1:49:22, 13.84s/it]
                                                       
{'loss': 1.6732, 'learning_rate': 5.4120774781617915e-05, 'epoch': 0.83}

 83%|████████▎ | 2259/2733 [8:25:35<1:49:22, 13.84s/it]
 83%|████████▎ | 2260/2733 [8:25:48<1:48:22, 13.75s/it]
                                                       
{'loss': 1.7649, 'learning_rate': 5.400683630839346e-05, 'epoch': 0.83}

 83%|████████▎ | 2260/2733 [8:25:48<1:48:22, 13.75s/it]
 83%|████████▎ | 2261/2733 [8:26:01<1:45:35, 13.42s/it]
                                                       
{'loss': 1.5678, 'learning_rate': 5.3892897835169e-05, 'epoch': 0.83}

 83%|████████▎ | 2261/2733 [8:26:01<1:45:35, 13.42s/it]
 83%|████████▎ | 2262/2733 [8:26:15<1:46:45, 13.60s/it]
                                                       
{'loss': 1.5912, 'learning_rate': 5.377895936194455e-05, 'epoch': 0.83}

 83%|████████▎ | 2262/2733 [8:26:15<1:46:45, 13.60s/it]
 83%|████████▎ | 2263/2733 [8:26:29<1:46:11, 13.56s/it]
                                                       
{'loss': 1.6679, 'learning_rate': 5.3665020888720084e-05, 'epoch': 0.83}

 83%|████████▎ | 2263/2733 [8:26:29<1:46:11, 13.56s/it]
 83%|████████▎ | 2264/2733 [8:26:43<1:47:32, 13.76s/it]
                                                       
{'loss': 1.8396, 'learning_rate': 5.355108241549563e-05, 'epoch': 0.83}

 83%|████████▎ | 2264/2733 [8:26:43<1:47:32, 13.76s/it]
 83%|████████▎ | 2265/2733 [8:26:56<1:46:06, 13.60s/it]
                                                       
{'loss': 1.6872, 'learning_rate': 5.343714394227117e-05, 'epoch': 0.83}

 83%|████████▎ | 2265/2733 [8:26:56<1:46:06, 13.60s/it]
 83%|████████▎ | 2266/2733 [8:27:10<1:46:02, 13.62s/it]
                                                       
{'loss': 1.6989, 'learning_rate': 5.332320546904671e-05, 'epoch': 0.83}

 83%|████████▎ | 2266/2733 [8:27:10<1:46:02, 13.62s/it]
 83%|████████▎ | 2267/2733 [8:27:23<1:45:52, 13.63s/it]
                                                       
{'loss': 1.6147, 'learning_rate': 5.320926699582225e-05, 'epoch': 0.83}

 83%|████████▎ | 2267/2733 [8:27:23<1:45:52, 13.63s/it]
 83%|████████▎ | 2268/2733 [8:27:37<1:45:20, 13.59s/it]
                                                       
{'loss': 1.6892, 'learning_rate': 5.3095328522597796e-05, 'epoch': 0.83}

 83%|████████▎ | 2268/2733 [8:27:37<1:45:20, 13.59s/it]
 83%|████████▎ | 2269/2733 [8:27:50<1:43:53, 13.43s/it]
                                                       
{'loss': 1.5936, 'learning_rate': 5.298139004937334e-05, 'epoch': 0.83}

 83%|████████▎ | 2269/2733 [8:27:50<1:43:53, 13.43s/it]
 83%|████████▎ | 2270/2733 [8:28:03<1:44:01, 13.48s/it]
                                                       
{'loss': 1.7087, 'learning_rate': 5.286745157614887e-05, 'epoch': 0.83}

 83%|████████▎ | 2270/2733 [8:28:03<1:44:01, 13.48s/it]
 83%|████████▎ | 2271/2733 [8:28:17<1:44:31, 13.58s/it]
                                                       
{'loss': 1.7635, 'learning_rate': 5.2753513102924415e-05, 'epoch': 0.83}

 83%|████████▎ | 2271/2733 [8:28:17<1:44:31, 13.58s/it]
 83%|████████▎ | 2272/2733 [8:28:30<1:42:50, 13.39s/it]
                                                       
{'loss': 1.6472, 'learning_rate': 5.263957462969996e-05, 'epoch': 0.83}

 83%|████████▎ | 2272/2733 [8:28:30<1:42:50, 13.39s/it]
 83%|████████▎ | 2273/2733 [8:28:43<1:42:00, 13.30s/it]
                                                       
{'loss': 1.6711, 'learning_rate': 5.2525636156475496e-05, 'epoch': 0.83}

 83%|████████▎ | 2273/2733 [8:28:43<1:42:00, 13.30s/it]
 83%|████████▎ | 2274/2733 [8:28:57<1:42:01, 13.34s/it]
                                                       
{'loss': 1.72, 'learning_rate': 5.241169768325104e-05, 'epoch': 0.83}

 83%|████████▎ | 2274/2733 [8:28:57<1:42:01, 13.34s/it]
 83%|████████▎ | 2275/2733 [8:29:10<1:41:59, 13.36s/it]
                                                       
{'loss': 1.6615, 'learning_rate': 5.2297759210026583e-05, 'epoch': 0.83}

 83%|████████▎ | 2275/2733 [8:29:10<1:41:59, 13.36s/it]
 83%|████████▎ | 2276/2733 [8:29:23<1:40:21, 13.18s/it]
                                                       
{'loss': 1.6282, 'learning_rate': 5.218382073680213e-05, 'epoch': 0.83}

 83%|████████▎ | 2276/2733 [8:29:23<1:40:21, 13.18s/it]
 83%|████████▎ | 2277/2733 [8:29:35<1:38:45, 12.99s/it]
                                                       
{'loss': 1.6036, 'learning_rate': 5.2069882263577664e-05, 'epoch': 0.83}

 83%|████████▎ | 2277/2733 [8:29:35<1:38:45, 12.99s/it]
 83%|████████▎ | 2278/2733 [8:29:49<1:39:23, 13.11s/it]
                                                       
{'loss': 1.6915, 'learning_rate': 5.19559437903532e-05, 'epoch': 0.83}

 83%|████████▎ | 2278/2733 [8:29:49<1:39:23, 13.11s/it]
 83%|████████▎ | 2279/2733 [8:30:02<1:40:07, 13.23s/it]
                                                       
{'loss': 1.718, 'learning_rate': 5.1842005317128745e-05, 'epoch': 0.83}

 83%|████████▎ | 2279/2733 [8:30:02<1:40:07, 13.23s/it]
 83%|████████▎ | 2280/2733 [8:30:16<1:39:55, 13.24s/it]
                                                       
{'loss': 1.6057, 'learning_rate': 5.172806684390428e-05, 'epoch': 0.83}

 83%|████████▎ | 2280/2733 [8:30:16<1:39:55, 13.24s/it]
 83%|████████▎ | 2281/2733 [8:30:29<1:40:03, 13.28s/it]
                                                       
{'loss': 1.6636, 'learning_rate': 5.1614128370679827e-05, 'epoch': 0.83}

 83%|████████▎ | 2281/2733 [8:30:29<1:40:03, 13.28s/it]
 83%|████████▎ | 2282/2733 [8:30:42<1:38:28, 13.10s/it]
                                                       
{'loss': 1.5497, 'learning_rate': 5.150018989745537e-05, 'epoch': 0.83}

 83%|████████▎ | 2282/2733 [8:30:42<1:38:28, 13.10s/it]
 84%|████████▎ | 2283/2733 [8:30:56<1:40:39, 13.42s/it]
                                                       
{'loss': 1.5998, 'learning_rate': 5.1386251424230914e-05, 'epoch': 0.84}

 84%|████████▎ | 2283/2733 [8:30:56<1:40:39, 13.42s/it]
 84%|████████▎ | 2284/2733 [8:31:09<1:38:55, 13.22s/it]
                                                       
{'loss': 1.7104, 'learning_rate': 5.127231295100645e-05, 'epoch': 0.84}

 84%|████████▎ | 2284/2733 [8:31:09<1:38:55, 13.22s/it]
 84%|████████▎ | 2285/2733 [8:31:22<1:38:22, 13.18s/it]
                                                       
{'loss': 1.7574, 'learning_rate': 5.1158374477781995e-05, 'epoch': 0.84}

 84%|████████▎ | 2285/2733 [8:31:22<1:38:22, 13.18s/it]
 84%|████████▎ | 2286/2733 [8:31:36<1:40:23, 13.47s/it]
                                                       
{'loss': 1.7178, 'learning_rate': 5.104443600455754e-05, 'epoch': 0.84}

 84%|████████▎ | 2286/2733 [8:31:36<1:40:23, 13.47s/it]
 84%|████████▎ | 2287/2733 [8:31:49<1:40:17, 13.49s/it]
                                                       
{'loss': 1.7049, 'learning_rate': 5.093049753133307e-05, 'epoch': 0.84}

 84%|████████▎ | 2287/2733 [8:31:49<1:40:17, 13.49s/it]
 84%|████████▎ | 2288/2733 [8:32:02<1:37:58, 13.21s/it]
                                                       
{'loss': 1.6636, 'learning_rate': 5.0816559058108613e-05, 'epoch': 0.84}

 84%|████████▎ | 2288/2733 [8:32:02<1:37:58, 13.21s/it]
 84%|████████▍ | 2289/2733 [8:32:15<1:36:49, 13.09s/it]
                                                       
{'loss': 1.5622, 'learning_rate': 5.070262058488416e-05, 'epoch': 0.84}

 84%|████████▍ | 2289/2733 [8:32:15<1:36:49, 13.09s/it]
 84%|████████▍ | 2290/2733 [8:32:28<1:35:58, 13.00s/it]
                                                       
{'loss': 1.6089, 'learning_rate': 5.0588682111659695e-05, 'epoch': 0.84}

 84%|████████▍ | 2290/2733 [8:32:28<1:35:58, 13.00s/it]
 84%|████████▍ | 2291/2733 [8:32:41<1:35:49, 13.01s/it]
                                                       
{'loss': 1.7055, 'learning_rate': 5.047474363843524e-05, 'epoch': 0.84}

 84%|████████▍ | 2291/2733 [8:32:41<1:35:49, 13.01s/it]
 84%|████████▍ | 2292/2733 [8:32:53<1:34:47, 12.90s/it]
                                                       
{'loss': 1.6028, 'learning_rate': 5.036080516521078e-05, 'epoch': 0.84}

 84%|████████▍ | 2292/2733 [8:32:53<1:34:47, 12.90s/it]
 84%|████████▍ | 2293/2733 [8:33:07<1:37:24, 13.28s/it]
                                                       
{'loss': 1.7427, 'learning_rate': 5.0246866691986326e-05, 'epoch': 0.84}

 84%|████████▍ | 2293/2733 [8:33:07<1:37:24, 13.28s/it]
 84%|████████▍ | 2294/2733 [8:33:22<1:39:21, 13.58s/it]
                                                       
{'loss': 1.6796, 'learning_rate': 5.013292821876186e-05, 'epoch': 0.84}

 84%|████████▍ | 2294/2733 [8:33:22<1:39:21, 13.58s/it]
 84%|████████▍ | 2295/2733 [8:33:34<1:36:18, 13.19s/it]
                                                       
{'loss': 1.5775, 'learning_rate': 5.001898974553741e-05, 'epoch': 0.84}

 84%|████████▍ | 2295/2733 [8:33:34<1:36:18, 13.19s/it]
 84%|████████▍ | 2296/2733 [8:33:47<1:35:44, 13.14s/it]
                                                       
{'loss': 1.7259, 'learning_rate': 4.990505127231295e-05, 'epoch': 0.84}

 84%|████████▍ | 2296/2733 [8:33:47<1:35:44, 13.14s/it]
 84%|████████▍ | 2297/2733 [8:34:01<1:36:22, 13.26s/it]
                                                       
{'loss': 1.6757, 'learning_rate': 4.979111279908848e-05, 'epoch': 0.84}

 84%|████████▍ | 2297/2733 [8:34:01<1:36:22, 13.26s/it]
 84%|████████▍ | 2298/2733 [8:34:13<1:35:11, 13.13s/it]
                                                       
{'loss': 1.716, 'learning_rate': 4.9677174325864025e-05, 'epoch': 0.84}

 84%|████████▍ | 2298/2733 [8:34:13<1:35:11, 13.13s/it]
 84%|████████▍ | 2299/2733 [8:34:28<1:37:18, 13.45s/it]
                                                       
{'loss': 1.7499, 'learning_rate': 4.956323585263957e-05, 'epoch': 0.84}

 84%|████████▍ | 2299/2733 [8:34:28<1:37:18, 13.45s/it]
 84%|████████▍ | 2300/2733 [8:34:41<1:37:19, 13.49s/it]
                                                       
{'loss': 1.7558, 'learning_rate': 4.944929737941511e-05, 'epoch': 0.84}

 84%|████████▍ | 2300/2733 [8:34:41<1:37:19, 13.49s/it]
 84%|████████▍ | 2301/2733 [8:34:55<1:38:35, 13.69s/it]
                                                       
{'loss': 1.662, 'learning_rate': 4.933535890619065e-05, 'epoch': 0.84}

 84%|████████▍ | 2301/2733 [8:34:55<1:38:35, 13.69s/it]
 84%|████████▍ | 2302/2733 [8:35:08<1:37:08, 13.52s/it]
                                                       
{'loss': 1.6955, 'learning_rate': 4.9221420432966194e-05, 'epoch': 0.84}

 84%|████████▍ | 2302/2733 [8:35:08<1:37:08, 13.52s/it]
 84%|████████▍ | 2303/2733 [8:35:22<1:36:57, 13.53s/it]
                                                       
{'loss': 1.537, 'learning_rate': 4.910748195974174e-05, 'epoch': 0.84}

 84%|████████▍ | 2303/2733 [8:35:22<1:36:57, 13.53s/it]
 84%|████████▍ | 2304/2733 [8:35:35<1:36:46, 13.53s/it]
                                                       
{'loss': 1.7262, 'learning_rate': 4.8993543486517275e-05, 'epoch': 0.84}

 84%|████████▍ | 2304/2733 [8:35:36<1:36:46, 13.53s/it]
 84%|████████▍ | 2305/2733 [8:35:49<1:36:13, 13.49s/it]
                                                       
{'loss': 1.6558, 'learning_rate': 4.887960501329282e-05, 'epoch': 0.84}

 84%|████████▍ | 2305/2733 [8:35:49<1:36:13, 13.49s/it]
 84%|████████▍ | 2306/2733 [8:36:02<1:35:03, 13.36s/it]
                                                       
{'loss': 1.6108, 'learning_rate': 4.876566654006836e-05, 'epoch': 0.84}

 84%|████████▍ | 2306/2733 [8:36:02<1:35:03, 13.36s/it]
 84%|████████▍ | 2307/2733 [8:36:15<1:35:12, 13.41s/it]
                                                       
{'loss': 1.6906, 'learning_rate': 4.865172806684391e-05, 'epoch': 0.84}

 84%|████████▍ | 2307/2733 [8:36:15<1:35:12, 13.41s/it]
 84%|████████▍ | 2308/2733 [8:36:29<1:35:24, 13.47s/it]
                                                       
{'loss': 1.6778, 'learning_rate': 4.853778959361944e-05, 'epoch': 0.84}

 84%|████████▍ | 2308/2733 [8:36:29<1:35:24, 13.47s/it]
 84%|████████▍ | 2309/2733 [8:36:42<1:34:23, 13.36s/it]
                                                       
{'loss': 1.67, 'learning_rate': 4.842385112039498e-05, 'epoch': 0.84}

 84%|████████▍ | 2309/2733 [8:36:42<1:34:23, 13.36s/it]
 85%|████████▍ | 2310/2733 [8:36:56<1:34:45, 13.44s/it]
                                                       
{'loss': 1.6327, 'learning_rate': 4.8309912647170525e-05, 'epoch': 0.85}

 85%|████████▍ | 2310/2733 [8:36:56<1:34:45, 13.44s/it]
 85%|████████▍ | 2311/2733 [8:37:09<1:34:29, 13.44s/it]
                                                       
{'loss': 1.6338, 'learning_rate': 4.819597417394606e-05, 'epoch': 0.85}

 85%|████████▍ | 2311/2733 [8:37:09<1:34:29, 13.44s/it]
 85%|████████▍ | 2312/2733 [8:37:23<1:34:52, 13.52s/it]
                                                       
{'loss': 1.6868, 'learning_rate': 4.8082035700721606e-05, 'epoch': 0.85}

 85%|████████▍ | 2312/2733 [8:37:23<1:34:52, 13.52s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 85%|████████▍ | 2313/2733 [8:37:37<1:36:19, 13.76s/it]
                                                       
{'loss': 1.6952, 'learning_rate': 4.796809722749715e-05, 'epoch': 0.85}

 85%|████████▍ | 2313/2733 [8:37:37<1:36:19, 13.76s/it]
 85%|████████▍ | 2314/2733 [8:37:51<1:36:17, 13.79s/it]
                                                       
{'loss': 1.6302, 'learning_rate': 4.7854158754272694e-05, 'epoch': 0.85}

 85%|████████▍ | 2314/2733 [8:37:51<1:36:17, 13.79s/it]
 85%|████████▍ | 2315/2733 [8:38:05<1:36:56, 13.92s/it]
                                                       
{'loss': 1.7133, 'learning_rate': 4.7740220281048224e-05, 'epoch': 0.85}

 85%|████████▍ | 2315/2733 [8:38:05<1:36:56, 13.92s/it]
 85%|████████▍ | 2316/2733 [8:38:19<1:36:30, 13.89s/it]
                                                       
{'loss': 1.7284, 'learning_rate': 4.762628180782377e-05, 'epoch': 0.85}

 85%|████████▍ | 2316/2733 [8:38:19<1:36:30, 13.89s/it]
 85%|████████▍ | 2317/2733 [8:38:32<1:34:50, 13.68s/it]
                                                       
{'loss': 1.6733, 'learning_rate': 4.751234333459931e-05, 'epoch': 0.85}

 85%|████████▍ | 2317/2733 [8:38:32<1:34:50, 13.68s/it]
 85%|████████▍ | 2318/2733 [8:38:45<1:32:28, 13.37s/it]
                                                       
{'loss': 1.5461, 'learning_rate': 4.739840486137485e-05, 'epoch': 0.85}

 85%|████████▍ | 2318/2733 [8:38:45<1:32:28, 13.37s/it]
 85%|████████▍ | 2319/2733 [8:38:58<1:31:02, 13.19s/it]
                                                       
{'loss': 1.6646, 'learning_rate': 4.728446638815039e-05, 'epoch': 0.85}

 85%|████████▍ | 2319/2733 [8:38:58<1:31:02, 13.19s/it]
 85%|████████▍ | 2320/2733 [8:39:11<1:31:28, 13.29s/it]
                                                       
{'loss': 1.7298, 'learning_rate': 4.717052791492594e-05, 'epoch': 0.85}

 85%|████████▍ | 2320/2733 [8:39:11<1:31:28, 13.29s/it]
 85%|████████▍ | 2321/2733 [8:39:25<1:32:25, 13.46s/it]
                                                       
{'loss': 1.6785, 'learning_rate': 4.705658944170148e-05, 'epoch': 0.85}

 85%|████████▍ | 2321/2733 [8:39:25<1:32:25, 13.46s/it]
 85%|████████▍ | 2322/2733 [8:39:38<1:31:41, 13.39s/it]
                                                       
{'loss': 1.7031, 'learning_rate': 4.694265096847702e-05, 'epoch': 0.85}

 85%|████████▍ | 2322/2733 [8:39:38<1:31:41, 13.39s/it]
 85%|████████▍ | 2323/2733 [8:39:52<1:32:42, 13.57s/it]
                                                       
{'loss': 1.6537, 'learning_rate': 4.682871249525256e-05, 'epoch': 0.85}

 85%|████████▍ | 2323/2733 [8:39:52<1:32:42, 13.57s/it]
 85%|████████▌ | 2324/2733 [8:40:07<1:33:56, 13.78s/it]
                                                       
{'loss': 1.7564, 'learning_rate': 4.6714774022028106e-05, 'epoch': 0.85}

 85%|████████▌ | 2324/2733 [8:40:07<1:33:56, 13.78s/it]
 85%|████████▌ | 2325/2733 [8:40:20<1:33:23, 13.73s/it]
                                                       
{'loss': 1.7224, 'learning_rate': 4.6600835548803636e-05, 'epoch': 0.85}

 85%|████████▌ | 2325/2733 [8:40:20<1:33:23, 13.73s/it]
 85%|████████▌ | 2326/2733 [8:40:34<1:32:56, 13.70s/it]
                                                       
{'loss': 1.7246, 'learning_rate': 4.648689707557918e-05, 'epoch': 0.85}

 85%|████████▌ | 2326/2733 [8:40:34<1:32:56, 13.70s/it]
 85%|████████▌ | 2327/2733 [8:40:47<1:31:50, 13.57s/it]
                                                       
{'loss': 1.5967, 'learning_rate': 4.6372958602354724e-05, 'epoch': 0.85}

 85%|████████▌ | 2327/2733 [8:40:47<1:31:50, 13.57s/it]
 85%|████████▌ | 2328/2733 [8:41:01<1:31:51, 13.61s/it]
                                                       
{'loss': 1.7682, 'learning_rate': 4.625902012913026e-05, 'epoch': 0.85}

 85%|████████▌ | 2328/2733 [8:41:01<1:31:51, 13.61s/it]
 85%|████████▌ | 2329/2733 [8:41:14<1:29:57, 13.36s/it]
                                                       
{'loss': 1.549, 'learning_rate': 4.6145081655905805e-05, 'epoch': 0.85}

 85%|████████▌ | 2329/2733 [8:41:14<1:29:57, 13.36s/it]
 85%|████████▌ | 2330/2733 [8:41:28<1:30:59, 13.55s/it]
                                                       
{'loss': 1.664, 'learning_rate': 4.603114318268135e-05, 'epoch': 0.85}

 85%|████████▌ | 2330/2733 [8:41:28<1:30:59, 13.55s/it]
 85%|████████▌ | 2331/2733 [8:41:40<1:28:50, 13.26s/it]
                                                       
{'loss': 1.6751, 'learning_rate': 4.591720470945689e-05, 'epoch': 0.85}

 85%|████████▌ | 2331/2733 [8:41:40<1:28:50, 13.26s/it]
 85%|████████▌ | 2332/2733 [8:41:54<1:29:51, 13.44s/it]
                                                       
{'loss': 1.7369, 'learning_rate': 4.580326623623243e-05, 'epoch': 0.85}

 85%|████████▌ | 2332/2733 [8:41:54<1:29:51, 13.44s/it]
 85%|████████▌ | 2333/2733 [8:42:07<1:28:01, 13.20s/it]
                                                       
{'loss': 1.6476, 'learning_rate': 4.5689327763007974e-05, 'epoch': 0.85}

 85%|████████▌ | 2333/2733 [8:42:07<1:28:01, 13.20s/it]
 85%|████████▌ | 2334/2733 [8:42:20<1:27:19, 13.13s/it]
                                                       
{'loss': 1.5663, 'learning_rate': 4.557538928978352e-05, 'epoch': 0.85}

 85%|████████▌ | 2334/2733 [8:42:20<1:27:19, 13.13s/it]
 85%|████████▌ | 2335/2733 [8:42:32<1:25:55, 12.95s/it]
                                                       
{'loss': 1.6495, 'learning_rate': 4.546145081655905e-05, 'epoch': 0.85}

 85%|████████▌ | 2335/2733 [8:42:32<1:25:55, 12.95s/it]
 85%|████████▌ | 2336/2733 [8:42:46<1:26:53, 13.13s/it]
                                                       
{'loss': 1.6026, 'learning_rate': 4.534751234333459e-05, 'epoch': 0.85}

 85%|████████▌ | 2336/2733 [8:42:46<1:26:53, 13.13s/it]
 86%|████████▌ | 2337/2733 [8:42:59<1:26:24, 13.09s/it]
                                                       
{'loss': 1.6832, 'learning_rate': 4.5233573870110136e-05, 'epoch': 0.86}

 86%|████████▌ | 2337/2733 [8:42:59<1:26:24, 13.09s/it]
 86%|████████▌ | 2338/2733 [8:43:12<1:26:33, 13.15s/it]
                                                       
{'loss': 1.6331, 'learning_rate': 4.511963539688568e-05, 'epoch': 0.86}

 86%|████████▌ | 2338/2733 [8:43:12<1:26:33, 13.15s/it]
 86%|████████▌ | 2339/2733 [8:43:26<1:27:18, 13.30s/it]
                                                       
{'loss': 1.5725, 'learning_rate': 4.500569692366122e-05, 'epoch': 0.86}

 86%|████████▌ | 2339/2733 [8:43:26<1:27:18, 13.30s/it]
 86%|████████▌ | 2340/2733 [8:43:39<1:26:46, 13.25s/it]
                                                       
{'loss': 1.6543, 'learning_rate': 4.489175845043676e-05, 'epoch': 0.86}

 86%|████████▌ | 2340/2733 [8:43:39<1:26:46, 13.25s/it]
 86%|████████▌ | 2341/2733 [8:43:52<1:26:10, 13.19s/it]
                                                       
{'loss': 1.6399, 'learning_rate': 4.4777819977212305e-05, 'epoch': 0.86}

 86%|████████▌ | 2341/2733 [8:43:52<1:26:10, 13.19s/it]
 86%|████████▌ | 2342/2733 [8:44:05<1:25:02, 13.05s/it]
                                                       
{'loss': 1.6402, 'learning_rate': 4.466388150398784e-05, 'epoch': 0.86}

 86%|████████▌ | 2342/2733 [8:44:05<1:25:02, 13.05s/it]
 86%|████████▌ | 2343/2733 [8:44:18<1:25:28, 13.15s/it]
                                                       
{'loss': 1.694, 'learning_rate': 4.4549943030763386e-05, 'epoch': 0.86}

 86%|████████▌ | 2343/2733 [8:44:18<1:25:28, 13.15s/it]
 86%|████████▌ | 2344/2733 [8:44:31<1:25:50, 13.24s/it]
                                                       
{'loss': 1.7158, 'learning_rate': 4.443600455753893e-05, 'epoch': 0.86}

 86%|████████▌ | 2344/2733 [8:44:31<1:25:50, 13.24s/it]
 86%|████████▌ | 2345/2733 [8:44:44<1:25:16, 13.19s/it]
                                                       
{'loss': 1.6835, 'learning_rate': 4.432206608431447e-05, 'epoch': 0.86}

 86%|████████▌ | 2345/2733 [8:44:44<1:25:16, 13.19s/it]
 86%|████████▌ | 2346/2733 [8:44:58<1:24:55, 13.17s/it]
                                                       
{'loss': 1.6079, 'learning_rate': 4.4208127611090004e-05, 'epoch': 0.86}

 86%|████████▌ | 2346/2733 [8:44:58<1:24:55, 13.17s/it]
 86%|████████▌ | 2347/2733 [8:45:11<1:25:40, 13.32s/it]
                                                       
{'loss': 1.7308, 'learning_rate': 4.409418913786555e-05, 'epoch': 0.86}

 86%|████████▌ | 2347/2733 [8:45:11<1:25:40, 13.32s/it]
 86%|████████▌ | 2348/2733 [8:45:23<1:23:04, 12.95s/it]
                                                       
{'loss': 1.6731, 'learning_rate': 4.398025066464109e-05, 'epoch': 0.86}

 86%|████████▌ | 2348/2733 [8:45:23<1:23:04, 12.95s/it]
 86%|████████▌ | 2349/2733 [8:45:37<1:24:33, 13.21s/it]
                                                       
{'loss': 1.719, 'learning_rate': 4.386631219141663e-05, 'epoch': 0.86}

 86%|████████▌ | 2349/2733 [8:45:37<1:24:33, 13.21s/it]
 86%|████████▌ | 2350/2733 [8:45:51<1:24:53, 13.30s/it]
                                                       
{'loss': 1.7265, 'learning_rate': 4.375237371819217e-05, 'epoch': 0.86}

 86%|████████▌ | 2350/2733 [8:45:51<1:24:53, 13.30s/it]
 86%|████████▌ | 2351/2733 [8:46:03<1:23:20, 13.09s/it]
                                                       
{'loss': 1.6883, 'learning_rate': 4.3638435244967717e-05, 'epoch': 0.86}

 86%|████████▌ | 2351/2733 [8:46:03<1:23:20, 13.09s/it]
 86%|████████▌ | 2352/2733 [8:46:17<1:23:33, 13.16s/it]
                                                       
{'loss': 1.6625, 'learning_rate': 4.352449677174326e-05, 'epoch': 0.86}

 86%|████████▌ | 2352/2733 [8:46:17<1:23:33, 13.16s/it]
 86%|████████▌ | 2353/2733 [8:46:31<1:25:03, 13.43s/it]
                                                       
{'loss': 1.6354, 'learning_rate': 4.341055829851879e-05, 'epoch': 0.86}

 86%|████████▌ | 2353/2733 [8:46:31<1:25:03, 13.43s/it]
 86%|████████▌ | 2354/2733 [8:46:45<1:25:40, 13.56s/it]
                                                       
{'loss': 1.7732, 'learning_rate': 4.3296619825294335e-05, 'epoch': 0.86}

 86%|████████▌ | 2354/2733 [8:46:45<1:25:40, 13.56s/it]
 86%|████████▌ | 2355/2733 [8:46:57<1:23:46, 13.30s/it]
                                                       
{'loss': 1.5429, 'learning_rate': 4.318268135206988e-05, 'epoch': 0.86}

 86%|████████▌ | 2355/2733 [8:46:57<1:23:46, 13.30s/it]
 86%|████████▌ | 2356/2733 [8:47:11<1:23:33, 13.30s/it]
                                                       
{'loss': 1.6597, 'learning_rate': 4.3068742878845416e-05, 'epoch': 0.86}

 86%|████████▌ | 2356/2733 [8:47:11<1:23:33, 13.30s/it]
 86%|████████▌ | 2357/2733 [8:47:24<1:22:49, 13.22s/it]
                                                       
{'loss': 1.667, 'learning_rate': 4.295480440562096e-05, 'epoch': 0.86}

 86%|████████▌ | 2357/2733 [8:47:24<1:22:49, 13.22s/it]
 86%|████████▋ | 2358/2733 [8:47:36<1:21:59, 13.12s/it]
                                                       
{'loss': 1.6114, 'learning_rate': 4.2840865932396503e-05, 'epoch': 0.86}

 86%|████████▋ | 2358/2733 [8:47:36<1:21:59, 13.12s/it]
 86%|████████▋ | 2359/2733 [8:47:50<1:22:00, 13.16s/it]
                                                       
{'loss': 1.7078, 'learning_rate': 4.272692745917205e-05, 'epoch': 0.86}

 86%|████████▋ | 2359/2733 [8:47:50<1:22:00, 13.16s/it]
 86%|████████▋ | 2360/2733 [8:48:04<1:23:38, 13.45s/it]
                                                       
{'loss': 1.6352, 'learning_rate': 4.2612988985947585e-05, 'epoch': 0.86}

 86%|████████▋ | 2360/2733 [8:48:04<1:23:38, 13.45s/it]
 86%|████████▋ | 2361/2733 [8:48:16<1:21:38, 13.17s/it]
                                                       
{'loss': 1.543, 'learning_rate': 4.249905051272313e-05, 'epoch': 0.86}

 86%|████████▋ | 2361/2733 [8:48:16<1:21:38, 13.17s/it]
 86%|████████▋ | 2362/2733 [8:48:30<1:23:00, 13.42s/it]
                                                       
{'loss': 1.6349, 'learning_rate': 4.238511203949867e-05, 'epoch': 0.86}

 86%|████████▋ | 2362/2733 [8:48:30<1:23:00, 13.42s/it]
 86%|████████▋ | 2363/2733 [8:48:44<1:22:40, 13.41s/it]
                                                       
{'loss': 1.736, 'learning_rate': 4.22711735662742e-05, 'epoch': 0.86}

 86%|████████▋ | 2363/2733 [8:48:44<1:22:40, 13.41s/it]
 86%|████████▋ | 2364/2733 [8:48:58<1:23:44, 13.62s/it]
                                                       
{'loss': 1.5786, 'learning_rate': 4.2157235093049747e-05, 'epoch': 0.86}

 86%|████████▋ | 2364/2733 [8:48:58<1:23:44, 13.62s/it]
 87%|████████▋ | 2365/2733 [8:49:11<1:23:08, 13.56s/it]
                                                       
{'loss': 1.655, 'learning_rate': 4.204329661982529e-05, 'epoch': 0.87}

 87%|████████▋ | 2365/2733 [8:49:11<1:23:08, 13.56s/it]
 87%|████████▋ | 2366/2733 [8:49:25<1:24:12, 13.77s/it]
                                                       
{'loss': 1.6676, 'learning_rate': 4.1929358146600834e-05, 'epoch': 0.87}

 87%|████████▋ | 2366/2733 [8:49:26<1:24:12, 13.77s/it]
 87%|████████▋ | 2367/2733 [8:49:38<1:21:42, 13.40s/it]
                                                       
{'loss': 1.6152, 'learning_rate': 4.181541967337637e-05, 'epoch': 0.87}

 87%|████████▋ | 2367/2733 [8:49:38<1:21:42, 13.40s/it]
 87%|████████▋ | 2368/2733 [8:49:52<1:21:42, 13.43s/it]
                                                       
{'loss': 1.6555, 'learning_rate': 4.1701481200151915e-05, 'epoch': 0.87}

 87%|████████▋ | 2368/2733 [8:49:52<1:21:42, 13.43s/it]
 87%|████████▋ | 2369/2733 [8:50:05<1:21:30, 13.44s/it]
                                                       
{'loss': 1.6511, 'learning_rate': 4.158754272692746e-05, 'epoch': 0.87}

 87%|████████▋ | 2369/2733 [8:50:05<1:21:30, 13.44s/it]
 87%|████████▋ | 2370/2733 [8:50:19<1:21:44, 13.51s/it]
                                                       
{'loss': 1.6923, 'learning_rate': 4.1473604253702996e-05, 'epoch': 0.87}

 87%|████████▋ | 2370/2733 [8:50:19<1:21:44, 13.51s/it]
 87%|████████▋ | 2371/2733 [8:50:31<1:19:34, 13.19s/it]
                                                       
{'loss': 1.6034, 'learning_rate': 4.135966578047854e-05, 'epoch': 0.87}

 87%|████████▋ | 2371/2733 [8:50:31<1:19:34, 13.19s/it]
 87%|████████▋ | 2372/2733 [8:50:44<1:17:59, 12.96s/it]
                                                       
{'loss': 1.7535, 'learning_rate': 4.1245727307254084e-05, 'epoch': 0.87}

 87%|████████▋ | 2372/2733 [8:50:44<1:17:59, 12.96s/it]
 87%|████████▋ | 2373/2733 [8:50:57<1:18:54, 13.15s/it]
                                                       
{'loss': 1.6732, 'learning_rate': 4.1131788834029615e-05, 'epoch': 0.87}

 87%|████████▋ | 2373/2733 [8:50:57<1:18:54, 13.15s/it]
 87%|████████▋ | 2374/2733 [8:51:11<1:20:24, 13.44s/it]
                                                       
{'loss': 1.6815, 'learning_rate': 4.101785036080516e-05, 'epoch': 0.87}

 87%|████████▋ | 2374/2733 [8:51:11<1:20:24, 13.44s/it]
 87%|████████▋ | 2375/2733 [8:51:25<1:21:28, 13.66s/it]
                                                       
{'loss': 1.6775, 'learning_rate': 4.09039118875807e-05, 'epoch': 0.87}

 87%|████████▋ | 2375/2733 [8:51:25<1:21:28, 13.66s/it]
 87%|████████▋ | 2376/2733 [8:51:39<1:21:26, 13.69s/it]
                                                       
{'loss': 1.6864, 'learning_rate': 4.0789973414356246e-05, 'epoch': 0.87}

 87%|████████▋ | 2376/2733 [8:51:39<1:21:26, 13.69s/it]
 87%|████████▋ | 2377/2733 [8:51:53<1:21:19, 13.71s/it]
                                                       
{'loss': 1.7547, 'learning_rate': 4.067603494113178e-05, 'epoch': 0.87}

 87%|████████▋ | 2377/2733 [8:51:53<1:21:19, 13.71s/it]
 87%|████████▋ | 2378/2733 [8:52:06<1:20:29, 13.60s/it]
                                                       
{'loss': 1.6544, 'learning_rate': 4.056209646790733e-05, 'epoch': 0.87}

 87%|████████▋ | 2378/2733 [8:52:06<1:20:29, 13.60s/it]
 87%|████████▋ | 2379/2733 [8:52:20<1:20:45, 13.69s/it]
                                                       
{'loss': 1.5581, 'learning_rate': 4.044815799468287e-05, 'epoch': 0.87}

 87%|████████▋ | 2379/2733 [8:52:20<1:20:45, 13.69s/it]
 87%|████████▋ | 2380/2733 [8:52:34<1:20:03, 13.61s/it]
                                                       
{'loss': 1.6347, 'learning_rate': 4.033421952145841e-05, 'epoch': 0.87}

 87%|████████▋ | 2380/2733 [8:52:34<1:20:03, 13.61s/it]
 87%|████████▋ | 2381/2733 [8:52:47<1:19:38, 13.57s/it]
                                                       
{'loss': 1.6432, 'learning_rate': 4.022028104823395e-05, 'epoch': 0.87}

 87%|████████▋ | 2381/2733 [8:52:47<1:19:38, 13.57s/it]
 87%|████████▋ | 2382/2733 [8:53:01<1:19:32, 13.60s/it]
                                                       
{'loss': 1.6068, 'learning_rate': 4.010634257500949e-05, 'epoch': 0.87}

 87%|████████▋ | 2382/2733 [8:53:01<1:19:32, 13.60s/it]
 87%|████████▋ | 2383/2733 [8:53:14<1:19:14, 13.58s/it]
                                                       
{'loss': 1.6086, 'learning_rate': 3.999240410178503e-05, 'epoch': 0.87}

 87%|████████▋ | 2383/2733 [8:53:14<1:19:14, 13.58s/it]
 87%|████████▋ | 2384/2733 [8:53:28<1:19:16, 13.63s/it]
                                                       
{'loss': 1.5741, 'learning_rate': 3.987846562856057e-05, 'epoch': 0.87}

 87%|████████▋ | 2384/2733 [8:53:28<1:19:16, 13.63s/it]
 87%|████████▋ | 2385/2733 [8:53:41<1:17:50, 13.42s/it]
                                                       
{'loss': 1.6484, 'learning_rate': 3.9764527155336114e-05, 'epoch': 0.87}

 87%|████████▋ | 2385/2733 [8:53:41<1:17:50, 13.42s/it]
 87%|████████▋ | 2386/2733 [8:53:53<1:15:49, 13.11s/it]
                                                       
{'loss': 1.6482, 'learning_rate': 3.965058868211166e-05, 'epoch': 0.87}

 87%|████████▋ | 2386/2733 [8:53:53<1:15:49, 13.11s/it]
 87%|████████▋ | 2387/2733 [8:54:07<1:16:29, 13.27s/it]
                                                       
{'loss': 1.6831, 'learning_rate': 3.9536650208887195e-05, 'epoch': 0.87}

 87%|████████▋ | 2387/2733 [8:54:07<1:16:29, 13.27s/it]
 87%|████████▋ | 2388/2733 [8:54:20<1:15:25, 13.12s/it]
                                                       
{'loss': 1.5907, 'learning_rate': 3.942271173566274e-05, 'epoch': 0.87}

 87%|████████▋ | 2388/2733 [8:54:20<1:15:25, 13.12s/it]
 87%|████████▋ | 2389/2733 [8:54:34<1:17:01, 13.43s/it]
                                                       
{'loss': 1.7527, 'learning_rate': 3.930877326243828e-05, 'epoch': 0.87}

 87%|████████▋ | 2389/2733 [8:54:34<1:17:01, 13.43s/it]
 87%|████████▋ | 2390/2733 [8:54:48<1:17:25, 13.54s/it]
                                                       
{'loss': 1.642, 'learning_rate': 3.919483478921383e-05, 'epoch': 0.87}

 87%|████████▋ | 2390/2733 [8:54:48<1:17:25, 13.54s/it]
 87%|████████▋ | 2391/2733 [8:55:01<1:16:58, 13.51s/it]
                                                       
{'loss': 1.6729, 'learning_rate': 3.908089631598936e-05, 'epoch': 0.87}

 87%|████████▋ | 2391/2733 [8:55:01<1:16:58, 13.51s/it]
 88%|████████▊ | 2392/2733 [8:55:15<1:17:31, 13.64s/it]
                                                       
{'loss': 1.7092, 'learning_rate': 3.89669578427649e-05, 'epoch': 0.88}

 88%|████████▊ | 2392/2733 [8:55:15<1:17:31, 13.64s/it]
 88%|████████▊ | 2393/2733 [8:55:29<1:18:06, 13.78s/it]
                                                       
{'loss': 1.6717, 'learning_rate': 3.8853019369540445e-05, 'epoch': 0.88}

 88%|████████▊ | 2393/2733 [8:55:29<1:18:06, 13.78s/it]
 88%|████████▊ | 2394/2733 [8:55:43<1:18:31, 13.90s/it]
                                                       
{'loss': 1.5763, 'learning_rate': 3.873908089631598e-05, 'epoch': 0.88}

 88%|████████▊ | 2394/2733 [8:55:43<1:18:31, 13.90s/it]
 88%|████████▊ | 2395/2733 [8:55:57<1:17:15, 13.72s/it]
                                                       
{'loss': 1.6279, 'learning_rate': 3.8625142423091526e-05, 'epoch': 0.88}

 88%|████████▊ | 2395/2733 [8:55:57<1:17:15, 13.72s/it]
 88%|████████▊ | 2396/2733 [8:56:10<1:15:35, 13.46s/it]
                                                       
{'loss': 1.5689, 'learning_rate': 3.851120394986707e-05, 'epoch': 0.88}

 88%|████████▊ | 2396/2733 [8:56:10<1:15:35, 13.46s/it]
 88%|████████▊ | 2397/2733 [8:56:22<1:13:31, 13.13s/it]
                                                       
{'loss': 1.6466, 'learning_rate': 3.8397265476642614e-05, 'epoch': 0.88}

 88%|████████▊ | 2397/2733 [8:56:22<1:13:31, 13.13s/it]
 88%|████████▊ | 2398/2733 [8:56:35<1:13:58, 13.25s/it]
                                                       
{'loss': 1.6861, 'learning_rate': 3.828332700341815e-05, 'epoch': 0.88}

 88%|████████▊ | 2398/2733 [8:56:35<1:13:58, 13.25s/it]
 88%|████████▊ | 2399/2733 [8:56:50<1:15:15, 13.52s/it]
                                                       
{'loss': 1.6765, 'learning_rate': 3.8169388530193695e-05, 'epoch': 0.88}

 88%|████████▊ | 2399/2733 [8:56:50<1:15:15, 13.52s/it]
 88%|████████▊ | 2400/2733 [8:57:02<1:13:08, 13.18s/it]
                                                       
{'loss': 1.6148, 'learning_rate': 3.805545005696924e-05, 'epoch': 0.88}

 88%|████████▊ | 2400/2733 [8:57:02<1:13:08, 13.18s/it]
 88%|████████▊ | 2401/2733 [8:57:15<1:13:07, 13.21s/it]
                                                       
{'loss': 1.7509, 'learning_rate': 3.794151158374477e-05, 'epoch': 0.88}

 88%|████████▊ | 2401/2733 [8:57:15<1:13:07, 13.21s/it]
 88%|████████▊ | 2402/2733 [8:57:29<1:13:53, 13.39s/it]
                                                       
{'loss': 1.6609, 'learning_rate': 3.782757311052031e-05, 'epoch': 0.88}

 88%|████████▊ | 2402/2733 [8:57:29<1:13:53, 13.39s/it]
 88%|████████▊ | 2403/2733 [8:57:42<1:13:01, 13.28s/it]
                                                       
{'loss': 1.6959, 'learning_rate': 3.771363463729586e-05, 'epoch': 0.88}

 88%|████████▊ | 2403/2733 [8:57:42<1:13:01, 13.28s/it]
 88%|████████▊ | 2404/2733 [8:57:55<1:11:57, 13.12s/it]
                                                       
{'loss': 1.6369, 'learning_rate': 3.75996961640714e-05, 'epoch': 0.88}

 88%|████████▊ | 2404/2733 [8:57:55<1:11:57, 13.12s/it]
 88%|████████▊ | 2405/2733 [8:58:08<1:12:00, 13.17s/it]
                                                       
{'loss': 1.5548, 'learning_rate': 3.748575769084694e-05, 'epoch': 0.88}

 88%|████████▊ | 2405/2733 [8:58:08<1:12:00, 13.17s/it]
 88%|████████▊ | 2406/2733 [8:58:22<1:12:31, 13.31s/it]
                                                       
{'loss': 1.6957, 'learning_rate': 3.737181921762248e-05, 'epoch': 0.88}

 88%|████████▊ | 2406/2733 [8:58:22<1:12:31, 13.31s/it]
 88%|████████▊ | 2407/2733 [8:58:36<1:13:26, 13.52s/it]
                                                       
{'loss': 1.6056, 'learning_rate': 3.725788074439802e-05, 'epoch': 0.88}

 88%|████████▊ | 2407/2733 [8:58:36<1:13:26, 13.52s/it]
 88%|████████▊ | 2408/2733 [8:58:49<1:12:37, 13.41s/it]
                                                       
{'loss': 1.6014, 'learning_rate': 3.714394227117356e-05, 'epoch': 0.88}

 88%|████████▊ | 2408/2733 [8:58:49<1:12:37, 13.41s/it]
 88%|████████▊ | 2409/2733 [8:59:02<1:11:48, 13.30s/it]
                                                       
{'loss': 1.8126, 'learning_rate': 3.703000379794911e-05, 'epoch': 0.88}

 88%|████████▊ | 2409/2733 [8:59:02<1:11:48, 13.30s/it]
 88%|████████▊ | 2410/2733 [8:59:15<1:11:35, 13.30s/it]
                                                       
{'loss': 1.6096, 'learning_rate': 3.691606532472465e-05, 'epoch': 0.88}

 88%|████████▊ | 2410/2733 [8:59:15<1:11:35, 13.30s/it]
 88%|████████▊ | 2411/2733 [8:59:29<1:11:23, 13.30s/it]
                                                       
{'loss': 1.6385, 'learning_rate': 3.680212685150019e-05, 'epoch': 0.88}

 88%|████████▊ | 2411/2733 [8:59:29<1:11:23, 13.30s/it]
 88%|████████▊ | 2412/2733 [8:59:42<1:11:56, 13.45s/it]
                                                       
{'loss': 1.7042, 'learning_rate': 3.6688188378275725e-05, 'epoch': 0.88}

 88%|████████▊ | 2412/2733 [8:59:42<1:11:56, 13.45s/it]
 88%|████████▊ | 2413/2733 [8:59:56<1:11:57, 13.49s/it]
                                                       
{'loss': 1.6454, 'learning_rate': 3.657424990505127e-05, 'epoch': 0.88}

 88%|████████▊ | 2413/2733 [8:59:56<1:11:57, 13.49s/it]
 88%|████████▊ | 2414/2733 [9:00:10<1:12:14, 13.59s/it]
                                                       
{'loss': 1.6814, 'learning_rate': 3.646031143182681e-05, 'epoch': 0.88}

 88%|████████▊ | 2414/2733 [9:00:10<1:12:14, 13.59s/it]
 88%|████████▊ | 2415/2733 [9:00:23<1:11:38, 13.52s/it]
                                                       
{'loss': 1.675, 'learning_rate': 3.634637295860235e-05, 'epoch': 0.88}

 88%|████████▊ | 2415/2733 [9:00:23<1:11:38, 13.52s/it]
 88%|████████▊ | 2416/2733 [9:00:36<1:10:55, 13.42s/it]
                                                       
{'loss': 1.654, 'learning_rate': 3.6232434485377894e-05, 'epoch': 0.88}

 88%|████████▊ | 2416/2733 [9:00:36<1:10:55, 13.42s/it]
 88%|████████▊ | 2417/2733 [9:00:50<1:10:25, 13.37s/it]
                                                       
{'loss': 1.7041, 'learning_rate': 3.611849601215343e-05, 'epoch': 0.88}

 88%|████████▊ | 2417/2733 [9:00:50<1:10:25, 13.37s/it]
 88%|████████▊ | 2418/2733 [9:01:03<1:10:00, 13.33s/it]
                                                       
{'loss': 1.686, 'learning_rate': 3.6004557538928975e-05, 'epoch': 0.88}

 88%|████████▊ | 2418/2733 [9:01:03<1:10:00, 13.33s/it]
 89%|████████▊ | 2419/2733 [9:01:16<1:10:13, 13.42s/it]
                                                       
{'loss': 1.6041, 'learning_rate': 3.589061906570451e-05, 'epoch': 0.89}

 89%|████████▊ | 2419/2733 [9:01:16<1:10:13, 13.42s/it]
 89%|████████▊ | 2420/2733 [9:01:30<1:09:36, 13.34s/it]
                                                       
{'loss': 1.7742, 'learning_rate': 3.5776680592480056e-05, 'epoch': 0.89}

 89%|████████▊ | 2420/2733 [9:01:30<1:09:36, 13.34s/it]
 89%|████████▊ | 2421/2733 [9:01:44<1:10:32, 13.56s/it]
                                                       
{'loss': 1.7126, 'learning_rate': 3.56627421192556e-05, 'epoch': 0.89}

 89%|████████▊ | 2421/2733 [9:01:44<1:10:32, 13.56s/it]
 89%|████████▊ | 2422/2733 [9:01:58<1:10:51, 13.67s/it]
                                                       
{'loss': 1.6136, 'learning_rate': 3.5548803646031144e-05, 'epoch': 0.89}

 89%|████████▊ | 2422/2733 [9:01:58<1:10:51, 13.67s/it]
 89%|████████▊ | 2423/2733 [9:02:10<1:09:17, 13.41s/it]
                                                       
{'loss': 1.7119, 'learning_rate': 3.543486517280668e-05, 'epoch': 0.89}

 89%|████████▊ | 2423/2733 [9:02:10<1:09:17, 13.41s/it]
 89%|████████▊ | 2424/2733 [9:02:24<1:09:31, 13.50s/it]
                                                       
{'loss': 1.5966, 'learning_rate': 3.532092669958222e-05, 'epoch': 0.89}

 89%|████████▊ | 2424/2733 [9:02:24<1:09:31, 13.50s/it]
 89%|████████▊ | 2425/2733 [9:02:37<1:07:45, 13.20s/it]
                                                       
{'loss': 1.5912, 'learning_rate': 3.520698822635776e-05, 'epoch': 0.89}

 89%|████████▊ | 2425/2733 [9:02:37<1:07:45, 13.20s/it]
 89%|████████▉ | 2426/2733 [9:02:49<1:05:37, 12.83s/it]
                                                       
{'loss': 1.5853, 'learning_rate': 3.5093049753133306e-05, 'epoch': 0.89}

 89%|████████▉ | 2426/2733 [9:02:49<1:05:37, 12.83s/it]
 89%|████████▉ | 2427/2733 [9:03:03<1:07:15, 13.19s/it]
                                                       
{'loss': 1.6187, 'learning_rate': 3.497911127990885e-05, 'epoch': 0.89}

 89%|████████▉ | 2427/2733 [9:03:03<1:07:15, 13.19s/it]
 89%|████████▉ | 2428/2733 [9:03:16<1:08:01, 13.38s/it]
                                                       
{'loss': 1.5616, 'learning_rate': 3.486517280668439e-05, 'epoch': 0.89}

 89%|████████▉ | 2428/2733 [9:03:16<1:08:01, 13.38s/it]
 89%|████████▉ | 2429/2733 [9:03:30<1:08:36, 13.54s/it]
                                                       
{'loss': 1.7491, 'learning_rate': 3.475123433345993e-05, 'epoch': 0.89}

 89%|████████▉ | 2429/2733 [9:03:30<1:08:36, 13.54s/it]
 89%|████████▉ | 2430/2733 [9:03:43<1:07:39, 13.40s/it]
                                                       
{'loss': 1.5904, 'learning_rate': 3.463729586023547e-05, 'epoch': 0.89}

 89%|████████▉ | 2430/2733 [9:03:43<1:07:39, 13.40s/it]
 89%|████████▉ | 2431/2733 [9:03:56<1:06:47, 13.27s/it]
                                                       
{'loss': 1.6524, 'learning_rate': 3.452335738701101e-05, 'epoch': 0.89}

 89%|████████▉ | 2431/2733 [9:03:56<1:06:47, 13.27s/it]
 89%|████████▉ | 2432/2733 [9:04:10<1:07:16, 13.41s/it]
                                                       
{'loss': 1.7731, 'learning_rate': 3.4409418913786556e-05, 'epoch': 0.89}

 89%|████████▉ | 2432/2733 [9:04:10<1:07:16, 13.41s/it]
 89%|████████▉ | 2433/2733 [9:04:23<1:06:29, 13.30s/it]
                                                       
{'loss': 1.7131, 'learning_rate': 3.429548044056209e-05, 'epoch': 0.89}

 89%|████████▉ | 2433/2733 [9:04:23<1:06:29, 13.30s/it]
 89%|████████▉ | 2434/2733 [9:04:37<1:07:34, 13.56s/it]
                                                       
{'loss': 1.6424, 'learning_rate': 3.4181541967337637e-05, 'epoch': 0.89}

 89%|████████▉ | 2434/2733 [9:04:37<1:07:34, 13.56s/it]
 89%|████████▉ | 2435/2733 [9:04:50<1:06:40, 13.42s/it]
                                                       
{'loss': 1.6404, 'learning_rate': 3.4067603494113174e-05, 'epoch': 0.89}

 89%|████████▉ | 2435/2733 [9:04:50<1:06:40, 13.42s/it]
 89%|████████▉ | 2436/2733 [9:05:04<1:05:57, 13.33s/it]
                                                       
{'loss': 1.6725, 'learning_rate': 3.395366502088872e-05, 'epoch': 0.89}

 89%|████████▉ | 2436/2733 [9:05:04<1:05:57, 13.33s/it]
 89%|████████▉ | 2437/2733 [9:05:17<1:05:18, 13.24s/it]
                                                       
{'loss': 1.6316, 'learning_rate': 3.383972654766426e-05, 'epoch': 0.89}

 89%|████████▉ | 2437/2733 [9:05:17<1:05:18, 13.24s/it]
 89%|████████▉ | 2438/2733 [9:05:30<1:05:58, 13.42s/it]
                                                       
{'loss': 1.7672, 'learning_rate': 3.37257880744398e-05, 'epoch': 0.89}

 89%|████████▉ | 2438/2733 [9:05:30<1:05:58, 13.42s/it]
 89%|████████▉ | 2439/2733 [9:05:44<1:05:27, 13.36s/it]
                                                       
{'loss': 1.7088, 'learning_rate': 3.361184960121534e-05, 'epoch': 0.89}

 89%|████████▉ | 2439/2733 [9:05:44<1:05:27, 13.36s/it]
 89%|████████▉ | 2440/2733 [9:05:57<1:05:07, 13.34s/it]
                                                       
{'loss': 1.6665, 'learning_rate': 3.349791112799088e-05, 'epoch': 0.89}

 89%|████████▉ | 2440/2733 [9:05:57<1:05:07, 13.34s/it]
 89%|████████▉ | 2441/2733 [9:06:10<1:04:40, 13.29s/it]
                                                       
{'loss': 1.6648, 'learning_rate': 3.3383972654766424e-05, 'epoch': 0.89}

 89%|████████▉ | 2441/2733 [9:06:10<1:04:40, 13.29s/it]
 89%|████████▉ | 2442/2733 [9:06:22<1:02:50, 12.96s/it]
                                                       
{'loss': 1.5619, 'learning_rate': 3.327003418154197e-05, 'epoch': 0.89}

 89%|████████▉ | 2442/2733 [9:06:22<1:02:50, 12.96s/it]
 89%|████████▉ | 2443/2733 [9:06:35<1:02:11, 12.87s/it]
                                                       
{'loss': 1.7211, 'learning_rate': 3.3156095708317505e-05, 'epoch': 0.89}

 89%|████████▉ | 2443/2733 [9:06:35<1:02:11, 12.87s/it]
 89%|████████▉ | 2444/2733 [9:06:48<1:03:01, 13.09s/it]
                                                       
{'loss': 1.6297, 'learning_rate': 3.304215723509305e-05, 'epoch': 0.89}

 89%|████████▉ | 2444/2733 [9:06:49<1:03:01, 13.09s/it]
 89%|████████▉ | 2445/2733 [9:07:01<1:02:40, 13.06s/it]
                                                       
{'loss': 1.6473, 'learning_rate': 3.2928218761868586e-05, 'epoch': 0.89}

 89%|████████▉ | 2445/2733 [9:07:02<1:02:40, 13.06s/it]
 89%|████████▉ | 2446/2733 [9:07:14<1:01:33, 12.87s/it]
                                                       
{'loss': 1.6981, 'learning_rate': 3.281428028864413e-05, 'epoch': 0.89}

 89%|████████▉ | 2446/2733 [9:07:14<1:01:33, 12.87s/it]
 90%|████████▉ | 2447/2733 [9:07:28<1:02:22, 13.08s/it]
                                                       
{'loss': 1.6523, 'learning_rate': 3.270034181541967e-05, 'epoch': 0.9}

 90%|████████▉ | 2447/2733 [9:07:28<1:02:22, 13.08s/it]
 90%|████████▉ | 2448/2733 [9:07:41<1:02:15, 13.11s/it]
                                                       
{'loss': 1.5906, 'learning_rate': 3.258640334219521e-05, 'epoch': 0.9}

 90%|████████▉ | 2448/2733 [9:07:41<1:02:15, 13.11s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 90%|████████▉ | 2449/2733 [9:07:55<1:03:19, 13.38s/it]
                                                       
{'loss': 1.6882, 'learning_rate': 3.2472464868970754e-05, 'epoch': 0.9}

 90%|████████▉ | 2449/2733 [9:07:55<1:03:19, 13.38s/it]
 90%|████████▉ | 2450/2733 [9:08:08<1:03:18, 13.42s/it]
                                                       
{'loss': 1.6583, 'learning_rate': 3.235852639574629e-05, 'epoch': 0.9}

 90%|████████▉ | 2450/2733 [9:08:08<1:03:18, 13.42s/it]
 90%|████████▉ | 2451/2733 [9:08:21<1:02:30, 13.30s/it]
                                                       
{'loss': 1.7113, 'learning_rate': 3.2244587922521835e-05, 'epoch': 0.9}

 90%|████████▉ | 2451/2733 [9:08:21<1:02:30, 13.30s/it]
 90%|████████▉ | 2452/2733 [9:08:35<1:02:53, 13.43s/it]
                                                       
{'loss': 1.6483, 'learning_rate': 3.213064944929737e-05, 'epoch': 0.9}

 90%|████████▉ | 2452/2733 [9:08:35<1:02:53, 13.43s/it]
 90%|████████▉ | 2453/2733 [9:08:48<1:01:32, 13.19s/it]
                                                       
{'loss': 1.598, 'learning_rate': 3.2016710976072916e-05, 'epoch': 0.9}

 90%|████████▉ | 2453/2733 [9:08:48<1:01:32, 13.19s/it]
 90%|████████▉ | 2454/2733 [9:09:01<1:02:03, 13.35s/it]
                                                       
{'loss': 1.5929, 'learning_rate': 3.190277250284846e-05, 'epoch': 0.9}

 90%|████████▉ | 2454/2733 [9:09:01<1:02:03, 13.35s/it]
 90%|████████▉ | 2455/2733 [9:09:15<1:02:58, 13.59s/it]
                                                       
{'loss': 1.7697, 'learning_rate': 3.1788834029624e-05, 'epoch': 0.9}

 90%|████████▉ | 2455/2733 [9:09:15<1:02:58, 13.59s/it]
 90%|████████▉ | 2456/2733 [9:09:29<1:02:44, 13.59s/it]
                                                       
{'loss': 1.6567, 'learning_rate': 3.167489555639954e-05, 'epoch': 0.9}

 90%|████████▉ | 2456/2733 [9:09:29<1:02:44, 13.59s/it]
 90%|████████▉ | 2457/2733 [9:09:43<1:03:08, 13.73s/it]
                                                       
{'loss': 1.6434, 'learning_rate': 3.156095708317508e-05, 'epoch': 0.9}

 90%|████████▉ | 2457/2733 [9:09:43<1:03:08, 13.73s/it]
 90%|████████▉ | 2458/2733 [9:09:56<1:02:20, 13.60s/it]
                                                       
{'loss': 1.616, 'learning_rate': 3.144701860995062e-05, 'epoch': 0.9}

 90%|████████▉ | 2458/2733 [9:09:56<1:02:20, 13.60s/it]
 90%|████████▉ | 2459/2733 [9:10:10<1:02:23, 13.66s/it]
                                                       
{'loss': 1.5857, 'learning_rate': 3.1333080136726166e-05, 'epoch': 0.9}

 90%|████████▉ | 2459/2733 [9:10:10<1:02:23, 13.66s/it]
 90%|█████████ | 2460/2733 [9:10:24<1:02:37, 13.76s/it]
                                                       
{'loss': 1.6724, 'learning_rate': 3.121914166350171e-05, 'epoch': 0.9}

 90%|█████████ | 2460/2733 [9:10:24<1:02:37, 13.76s/it]
 90%|█████████ | 2461/2733 [9:10:38<1:01:50, 13.64s/it]
                                                       
{'loss': 1.7749, 'learning_rate': 3.110520319027725e-05, 'epoch': 0.9}

 90%|█████████ | 2461/2733 [9:10:38<1:01:50, 13.64s/it]
 90%|█████████ | 2462/2733 [9:10:50<1:00:40, 13.43s/it]
                                                       
{'loss': 1.6266, 'learning_rate': 3.0991264717052784e-05, 'epoch': 0.9}

 90%|█████████ | 2462/2733 [9:10:51<1:00:40, 13.43s/it]
 90%|█████████ | 2463/2733 [9:11:04<1:00:22, 13.42s/it]
                                                       
{'loss': 1.6422, 'learning_rate': 3.087732624382833e-05, 'epoch': 0.9}

 90%|█████████ | 2463/2733 [9:11:04<1:00:22, 13.42s/it]
 90%|█████████ | 2464/2733 [9:11:17<59:33, 13.28s/it]  
                                                     
{'loss': 1.6731, 'learning_rate': 3.076338777060387e-05, 'epoch': 0.9}

 90%|█████████ | 2464/2733 [9:11:17<59:33, 13.28s/it]
 90%|█████████ | 2465/2733 [9:11:30<59:27, 13.31s/it]
                                                     
{'loss': 1.607, 'learning_rate': 3.0649449297379416e-05, 'epoch': 0.9}

 90%|█████████ | 2465/2733 [9:11:30<59:27, 13.31s/it]
 90%|█████████ | 2466/2733 [9:11:43<58:32, 13.15s/it]
                                                     
{'loss': 1.6435, 'learning_rate': 3.053551082415495e-05, 'epoch': 0.9}

 90%|█████████ | 2466/2733 [9:11:43<58:32, 13.15s/it]
 90%|█████████ | 2467/2733 [9:11:57<58:54, 13.29s/it]
                                                     
{'loss': 1.7552, 'learning_rate': 3.0421572350930497e-05, 'epoch': 0.9}

 90%|█████████ | 2467/2733 [9:11:57<58:54, 13.29s/it]
 90%|█████████ | 2468/2733 [9:12:10<58:24, 13.22s/it]
                                                     
{'loss': 1.6221, 'learning_rate': 3.0307633877706038e-05, 'epoch': 0.9}

 90%|█████████ | 2468/2733 [9:12:10<58:24, 13.22s/it]
 90%|█████████ | 2469/2733 [9:12:23<58:51, 13.38s/it]
                                                     
{'loss': 1.6609, 'learning_rate': 3.0193695404481575e-05, 'epoch': 0.9}

 90%|█████████ | 2469/2733 [9:12:23<58:51, 13.38s/it]
 90%|█████████ | 2470/2733 [9:12:36<58:05, 13.25s/it]
                                                     
{'loss': 1.7422, 'learning_rate': 3.007975693125712e-05, 'epoch': 0.9}

 90%|█████████ | 2470/2733 [9:12:36<58:05, 13.25s/it]
 90%|█████████ | 2471/2733 [9:12:50<58:13, 13.34s/it]
                                                     
{'loss': 1.704, 'learning_rate': 2.996581845803266e-05, 'epoch': 0.9}

 90%|█████████ | 2471/2733 [9:12:50<58:13, 13.34s/it]
 90%|█████████ | 2472/2733 [9:13:03<58:15, 13.39s/it]
                                                     
{'loss': 1.7374, 'learning_rate': 2.9851879984808203e-05, 'epoch': 0.9}

 90%|█████████ | 2472/2733 [9:13:03<58:15, 13.39s/it]
 90%|█████████ | 2473/2733 [9:13:17<58:12, 13.43s/it]
                                                     
{'loss': 1.5853, 'learning_rate': 2.973794151158374e-05, 'epoch': 0.9}

 90%|█████████ | 2473/2733 [9:13:17<58:12, 13.43s/it]
 91%|█████████ | 2474/2733 [9:13:30<58:03, 13.45s/it]
                                                     
{'loss': 1.6706, 'learning_rate': 2.962400303835928e-05, 'epoch': 0.91}

 91%|█████████ | 2474/2733 [9:13:30<58:03, 13.45s/it]
 91%|█████████ | 2475/2733 [9:13:45<58:43, 13.66s/it]
                                                     
{'loss': 1.709, 'learning_rate': 2.9510064565134825e-05, 'epoch': 0.91}

 91%|█████████ | 2475/2733 [9:13:45<58:43, 13.66s/it]
 91%|█████████ | 2476/2733 [9:13:58<58:17, 13.61s/it]
                                                     
{'loss': 1.6458, 'learning_rate': 2.9396126091910365e-05, 'epoch': 0.91}

 91%|█████████ | 2476/2733 [9:13:58<58:17, 13.61s/it]
 91%|█████████ | 2477/2733 [9:14:10<56:09, 13.16s/it]
                                                     
{'loss': 1.5973, 'learning_rate': 2.928218761868591e-05, 'epoch': 0.91}

 91%|█████████ | 2477/2733 [9:14:10<56:09, 13.16s/it]
 91%|█████████ | 2478/2733 [9:14:24<57:00, 13.41s/it]
                                                     
{'loss': 1.6854, 'learning_rate': 2.9168249145461446e-05, 'epoch': 0.91}

 91%|█████████ | 2478/2733 [9:14:24<57:00, 13.41s/it]
 91%|█████████ | 2479/2733 [9:14:38<57:01, 13.47s/it]
                                                     
{'loss': 1.6157, 'learning_rate': 2.905431067223699e-05, 'epoch': 0.91}

 91%|█████████ | 2479/2733 [9:14:38<57:01, 13.47s/it]
 91%|█████████ | 2480/2733 [9:14:52<57:42, 13.69s/it]
                                                     
{'loss': 1.6437, 'learning_rate': 2.894037219901253e-05, 'epoch': 0.91}

 91%|█████████ | 2480/2733 [9:14:52<57:42, 13.69s/it]
 91%|█████████ | 2481/2733 [9:15:05<56:57, 13.56s/it]
                                                     
{'loss': 1.6872, 'learning_rate': 2.882643372578807e-05, 'epoch': 0.91}

 91%|█████████ | 2481/2733 [9:15:05<56:57, 13.56s/it]
 91%|█████████ | 2482/2733 [9:15:18<55:43, 13.32s/it]
                                                     
{'loss': 1.7262, 'learning_rate': 2.8712495252563615e-05, 'epoch': 0.91}

 91%|█████████ | 2482/2733 [9:15:18<55:43, 13.32s/it]
 91%|█████████ | 2483/2733 [9:15:31<55:09, 13.24s/it]
                                                     
{'loss': 1.6734, 'learning_rate': 2.8598556779339152e-05, 'epoch': 0.91}

 91%|█████████ | 2483/2733 [9:15:31<55:09, 13.24s/it]
 91%|█████████ | 2484/2733 [9:15:44<54:23, 13.11s/it]
                                                     
{'loss': 1.5778, 'learning_rate': 2.8484618306114696e-05, 'epoch': 0.91}

 91%|█████████ | 2484/2733 [9:15:44<54:23, 13.11s/it]
 91%|█████████ | 2485/2733 [9:15:57<54:07, 13.10s/it]
                                                     
{'loss': 1.5944, 'learning_rate': 2.8370679832890237e-05, 'epoch': 0.91}

 91%|█████████ | 2485/2733 [9:15:57<54:07, 13.10s/it]
 91%|█████████ | 2486/2733 [9:16:11<54:35, 13.26s/it]
                                                     
{'loss': 1.553, 'learning_rate': 2.825674135966578e-05, 'epoch': 0.91}

 91%|█████████ | 2486/2733 [9:16:11<54:35, 13.26s/it]
 91%|█████████ | 2487/2733 [9:16:24<54:00, 13.17s/it]
                                                     
{'loss': 1.5963, 'learning_rate': 2.814280288644132e-05, 'epoch': 0.91}

 91%|█████████ | 2487/2733 [9:16:24<54:00, 13.17s/it]
 91%|█████████ | 2488/2733 [9:16:38<54:58, 13.46s/it]
                                                     
{'loss': 1.5403, 'learning_rate': 2.8028864413216858e-05, 'epoch': 0.91}

 91%|█████████ | 2488/2733 [9:16:38<54:58, 13.46s/it]
 91%|█████████ | 2489/2733 [9:16:51<53:58, 13.27s/it]
                                                     
{'loss': 1.5742, 'learning_rate': 2.7914925939992402e-05, 'epoch': 0.91}

 91%|█████████ | 2489/2733 [9:16:51<53:58, 13.27s/it]
 91%|█████████ | 2490/2733 [9:17:04<54:08, 13.37s/it]
                                                     
{'loss': 1.7018, 'learning_rate': 2.7800987466767942e-05, 'epoch': 0.91}

 91%|█████████ | 2490/2733 [9:17:04<54:08, 13.37s/it]
 91%|█████████ | 2491/2733 [9:17:18<54:20, 13.48s/it]
                                                     
{'loss': 1.6268, 'learning_rate': 2.7687048993543486e-05, 'epoch': 0.91}

 91%|█████████ | 2491/2733 [9:17:18<54:20, 13.48s/it]
 91%|█████████ | 2492/2733 [9:17:31<54:17, 13.52s/it]
                                                     
{'loss': 1.751, 'learning_rate': 2.7573110520319023e-05, 'epoch': 0.91}

 91%|█████████ | 2492/2733 [9:17:31<54:17, 13.52s/it]
 91%|█████████ | 2493/2733 [9:17:45<54:02, 13.51s/it]
                                                     
{'loss': 1.6433, 'learning_rate': 2.7459172047094564e-05, 'epoch': 0.91}

 91%|█████████ | 2493/2733 [9:17:45<54:02, 13.51s/it]
 91%|█████████▏| 2494/2733 [9:17:58<53:10, 13.35s/it]
                                                     
{'loss': 1.7155, 'learning_rate': 2.7345233573870108e-05, 'epoch': 0.91}

 91%|█████████▏| 2494/2733 [9:17:58<53:10, 13.35s/it]
 91%|█████████▏| 2495/2733 [9:18:10<51:54, 13.08s/it]
                                                     
{'loss': 1.6442, 'learning_rate': 2.723129510064565e-05, 'epoch': 0.91}

 91%|█████████▏| 2495/2733 [9:18:10<51:54, 13.08s/it]
 91%|█████████▏| 2496/2733 [9:18:24<51:55, 13.14s/it]
                                                     
{'loss': 1.6799, 'learning_rate': 2.7117356627421192e-05, 'epoch': 0.91}

 91%|█████████▏| 2496/2733 [9:18:24<51:55, 13.14s/it]
 91%|█████████▏| 2497/2733 [9:18:37<51:32, 13.10s/it]
                                                     
{'loss': 1.5733, 'learning_rate': 2.700341815419673e-05, 'epoch': 0.91}

 91%|█████████▏| 2497/2733 [9:18:37<51:32, 13.10s/it]
 91%|█████████▏| 2498/2733 [9:18:51<52:34, 13.42s/it]
                                                     
{'loss': 1.6967, 'learning_rate': 2.6889479680972273e-05, 'epoch': 0.91}

 91%|█████████▏| 2498/2733 [9:18:51<52:34, 13.42s/it]
 91%|█████████▏| 2499/2733 [9:19:04<52:03, 13.35s/it]
                                                     
{'loss': 1.7102, 'learning_rate': 2.6775541207747814e-05, 'epoch': 0.91}

 91%|█████████▏| 2499/2733 [9:19:04<52:03, 13.35s/it]
 91%|█████████▏| 2500/2733 [9:19:18<52:09, 13.43s/it]
                                                     
{'loss': 1.5621, 'learning_rate': 2.6661602734523354e-05, 'epoch': 0.91}

 91%|█████████▏| 2500/2733 [9:19:18<52:09, 13.43s/it]
 92%|█████████▏| 2501/2733 [9:19:31<51:44, 13.38s/it]
                                                     
{'loss': 1.6015, 'learning_rate': 2.6547664261298898e-05, 'epoch': 0.92}

 92%|█████████▏| 2501/2733 [9:19:31<51:44, 13.38s/it]
 92%|█████████▏| 2502/2733 [9:19:43<50:28, 13.11s/it]
                                                     
{'loss': 1.6156, 'learning_rate': 2.6433725788074435e-05, 'epoch': 0.92}

 92%|█████████▏| 2502/2733 [9:19:43<50:28, 13.11s/it]
 92%|█████████▏| 2503/2733 [9:19:57<51:18, 13.39s/it]
                                                     
{'loss': 1.7472, 'learning_rate': 2.631978731484998e-05, 'epoch': 0.92}

 92%|█████████▏| 2503/2733 [9:19:57<51:18, 13.39s/it]
 92%|█████████▏| 2504/2733 [9:20:11<51:14, 13.42s/it]
                                                     
{'loss': 1.6424, 'learning_rate': 2.620584884162552e-05, 'epoch': 0.92}

 92%|█████████▏| 2504/2733 [9:20:11<51:14, 13.42s/it]
 92%|█████████▏| 2505/2733 [9:20:24<50:46, 13.36s/it]
                                                     
{'loss': 1.6817, 'learning_rate': 2.6091910368401064e-05, 'epoch': 0.92}

 92%|█████████▏| 2505/2733 [9:20:24<50:46, 13.36s/it]
 92%|█████████▏| 2506/2733 [9:20:37<50:10, 13.26s/it]
                                                     
{'loss': 1.6463, 'learning_rate': 2.59779718951766e-05, 'epoch': 0.92}

 92%|█████████▏| 2506/2733 [9:20:37<50:10, 13.26s/it]
 92%|█████████▏| 2507/2733 [9:20:51<50:01, 13.28s/it]
                                                     
{'loss': 1.6193, 'learning_rate': 2.586403342195214e-05, 'epoch': 0.92}

 92%|█████████▏| 2507/2733 [9:20:51<50:01, 13.28s/it]
 92%|█████████▏| 2508/2733 [9:21:04<50:07, 13.37s/it]
                                                     
{'loss': 1.6464, 'learning_rate': 2.5750094948727685e-05, 'epoch': 0.92}

 92%|█████████▏| 2508/2733 [9:21:04<50:07, 13.37s/it]
 92%|█████████▏| 2509/2733 [9:21:17<49:27, 13.25s/it]
                                                     
{'loss': 1.5832, 'learning_rate': 2.5636156475503226e-05, 'epoch': 0.92}

 92%|█████████▏| 2509/2733 [9:21:17<49:27, 13.25s/it]
 92%|█████████▏| 2510/2733 [9:21:31<50:17, 13.53s/it]
                                                     
{'loss': 1.6876, 'learning_rate': 2.552221800227877e-05, 'epoch': 0.92}

 92%|█████████▏| 2510/2733 [9:21:31<50:17, 13.53s/it]
 92%|█████████▏| 2511/2733 [9:21:44<49:10, 13.29s/it]
                                                     
{'loss': 1.5718, 'learning_rate': 2.5408279529054307e-05, 'epoch': 0.92}

 92%|█████████▏| 2511/2733 [9:21:44<49:10, 13.29s/it]
 92%|█████████▏| 2512/2733 [9:21:57<49:04, 13.33s/it]
                                                     
{'loss': 1.677, 'learning_rate': 2.5294341055829847e-05, 'epoch': 0.92}

 92%|█████████▏| 2512/2733 [9:21:57<49:04, 13.33s/it]
 92%|█████████▏| 2513/2733 [9:22:11<49:31, 13.51s/it]
                                                     
{'loss': 1.7509, 'learning_rate': 2.518040258260539e-05, 'epoch': 0.92}

 92%|█████████▏| 2513/2733 [9:22:11<49:31, 13.51s/it]
 92%|█████████▏| 2514/2733 [9:22:25<50:03, 13.71s/it]
                                                     
{'loss': 1.642, 'learning_rate': 2.506646410938093e-05, 'epoch': 0.92}

 92%|█████████▏| 2514/2733 [9:22:26<50:03, 13.71s/it]
 92%|█████████▏| 2515/2733 [9:22:39<49:45, 13.70s/it]
                                                     
{'loss': 1.6377, 'learning_rate': 2.4952525636156476e-05, 'epoch': 0.92}

 92%|█████████▏| 2515/2733 [9:22:39<49:45, 13.70s/it]
 92%|█████████▏| 2516/2733 [9:22:52<48:52, 13.51s/it]
                                                     
{'loss': 1.7585, 'learning_rate': 2.4838587162932013e-05, 'epoch': 0.92}

 92%|█████████▏| 2516/2733 [9:22:52<48:52, 13.51s/it]
 92%|█████████▏| 2517/2733 [9:23:06<48:44, 13.54s/it]
                                                     
{'loss': 1.5449, 'learning_rate': 2.4724648689707557e-05, 'epoch': 0.92}

 92%|█████████▏| 2517/2733 [9:23:06<48:44, 13.54s/it]
 92%|█████████▏| 2518/2733 [9:23:20<49:05, 13.70s/it]
                                                     
{'loss': 1.6408, 'learning_rate': 2.4610710216483097e-05, 'epoch': 0.92}

 92%|█████████▏| 2518/2733 [9:23:20<49:05, 13.70s/it]
 92%|█████████▏| 2519/2733 [9:23:34<49:04, 13.76s/it]
                                                     
{'loss': 1.6899, 'learning_rate': 2.4496771743258638e-05, 'epoch': 0.92}

 92%|█████████▏| 2519/2733 [9:23:34<49:04, 13.76s/it]
 92%|█████████▏| 2520/2733 [9:23:47<48:40, 13.71s/it]
                                                     
{'loss': 1.6299, 'learning_rate': 2.438283327003418e-05, 'epoch': 0.92}

 92%|█████████▏| 2520/2733 [9:23:47<48:40, 13.71s/it]
 92%|█████████▏| 2521/2733 [9:24:00<47:32, 13.45s/it]
                                                     
{'loss': 1.6469, 'learning_rate': 2.426889479680972e-05, 'epoch': 0.92}

 92%|█████████▏| 2521/2733 [9:24:00<47:32, 13.45s/it]
 92%|█████████▏| 2522/2733 [9:24:14<47:29, 13.50s/it]
                                                     
{'loss': 1.6649, 'learning_rate': 2.4154956323585263e-05, 'epoch': 0.92}

 92%|█████████▏| 2522/2733 [9:24:14<47:29, 13.50s/it]
 92%|█████████▏| 2523/2733 [9:24:27<46:59, 13.43s/it]
                                                     
{'loss': 1.6627, 'learning_rate': 2.4041017850360803e-05, 'epoch': 0.92}

 92%|█████████▏| 2523/2733 [9:24:27<46:59, 13.43s/it]
 92%|█████████▏| 2524/2733 [9:24:41<47:14, 13.56s/it]
                                                     
{'loss': 1.6614, 'learning_rate': 2.3927079377136347e-05, 'epoch': 0.92}

 92%|█████████▏| 2524/2733 [9:24:41<47:14, 13.56s/it]
 92%|█████████▏| 2525/2733 [9:24:54<46:54, 13.53s/it]
                                                     
{'loss': 1.7276, 'learning_rate': 2.3813140903911884e-05, 'epoch': 0.92}

 92%|█████████▏| 2525/2733 [9:24:54<46:54, 13.53s/it]
 92%|█████████▏| 2526/2733 [9:25:08<46:29, 13.48s/it]
                                                     
{'loss': 1.6362, 'learning_rate': 2.3699202430687425e-05, 'epoch': 0.92}

 92%|█████████▏| 2526/2733 [9:25:08<46:29, 13.48s/it]
 92%|█████████▏| 2527/2733 [9:25:21<46:21, 13.50s/it]
                                                     
{'loss': 1.6998, 'learning_rate': 2.358526395746297e-05, 'epoch': 0.92}

 92%|█████████▏| 2527/2733 [9:25:21<46:21, 13.50s/it]
 92%|█████████▏| 2528/2733 [9:25:35<46:07, 13.50s/it]
                                                     
{'loss': 1.7499, 'learning_rate': 2.347132548423851e-05, 'epoch': 0.92}

 92%|█████████▏| 2528/2733 [9:25:35<46:07, 13.50s/it]
 93%|█████████▎| 2529/2733 [9:25:48<45:54, 13.50s/it]
                                                     
{'loss': 1.7584, 'learning_rate': 2.3357387011014053e-05, 'epoch': 0.93}

 93%|█████████▎| 2529/2733 [9:25:48<45:54, 13.50s/it]
 93%|█████████▎| 2530/2733 [9:26:02<45:54, 13.57s/it]
                                                     
{'loss': 1.6683, 'learning_rate': 2.324344853778959e-05, 'epoch': 0.93}

 93%|█████████▎| 2530/2733 [9:26:02<45:54, 13.57s/it]
 93%|█████████▎| 2531/2733 [9:26:16<46:02, 13.67s/it]
                                                     
{'loss': 1.6777, 'learning_rate': 2.312951006456513e-05, 'epoch': 0.93}

 93%|█████████▎| 2531/2733 [9:26:16<46:02, 13.67s/it]
 93%|█████████▎| 2532/2733 [9:26:29<45:34, 13.61s/it]
                                                     
{'loss': 1.683, 'learning_rate': 2.3015571591340674e-05, 'epoch': 0.93}

 93%|█████████▎| 2532/2733 [9:26:29<45:34, 13.61s/it]
 93%|█████████▎| 2533/2733 [9:26:43<45:26, 13.63s/it]
                                                     
{'loss': 1.7195, 'learning_rate': 2.2901633118116215e-05, 'epoch': 0.93}

 93%|█████████▎| 2533/2733 [9:26:43<45:26, 13.63s/it]
 93%|█████████▎| 2534/2733 [9:26:57<45:28, 13.71s/it]
                                                     
{'loss': 1.7466, 'learning_rate': 2.278769464489176e-05, 'epoch': 0.93}

 93%|█████████▎| 2534/2733 [9:26:57<45:28, 13.71s/it]
 93%|█████████▎| 2535/2733 [9:27:10<44:25, 13.46s/it]
                                                     
{'loss': 1.5949, 'learning_rate': 2.2673756171667296e-05, 'epoch': 0.93}

 93%|█████████▎| 2535/2733 [9:27:10<44:25, 13.46s/it]
 93%|█████████▎| 2536/2733 [9:27:24<44:17, 13.49s/it]
                                                     
{'loss': 1.6642, 'learning_rate': 2.255981769844284e-05, 'epoch': 0.93}

 93%|█████████▎| 2536/2733 [9:27:24<44:17, 13.49s/it]
 93%|█████████▎| 2537/2733 [9:27:38<44:48, 13.72s/it]
                                                     
{'loss': 1.7416, 'learning_rate': 2.244587922521838e-05, 'epoch': 0.93}

 93%|█████████▎| 2537/2733 [9:27:38<44:48, 13.72s/it]
 93%|█████████▎| 2538/2733 [9:27:51<43:50, 13.49s/it]
                                                     
{'loss': 1.5754, 'learning_rate': 2.233194075199392e-05, 'epoch': 0.93}

 93%|█████████▎| 2538/2733 [9:27:51<43:50, 13.49s/it]
 93%|█████████▎| 2539/2733 [9:28:05<43:56, 13.59s/it]
                                                     
{'loss': 1.6177, 'learning_rate': 2.2218002278769465e-05, 'epoch': 0.93}

 93%|█████████▎| 2539/2733 [9:28:05<43:56, 13.59s/it]
 93%|█████████▎| 2540/2733 [9:28:18<43:44, 13.60s/it]
                                                     
{'loss': 1.6611, 'learning_rate': 2.2104063805545002e-05, 'epoch': 0.93}

 93%|█████████▎| 2540/2733 [9:28:18<43:44, 13.60s/it]
 93%|█████████▎| 2541/2733 [9:28:31<42:20, 13.23s/it]
                                                     
{'loss': 1.6395, 'learning_rate': 2.1990125332320546e-05, 'epoch': 0.93}

 93%|█████████▎| 2541/2733 [9:28:31<42:20, 13.23s/it]
 93%|█████████▎| 2542/2733 [9:28:45<42:51, 13.46s/it]
                                                     
{'loss': 1.7795, 'learning_rate': 2.1876186859096086e-05, 'epoch': 0.93}

 93%|█████████▎| 2542/2733 [9:28:45<42:51, 13.46s/it]
 93%|█████████▎| 2543/2733 [9:28:57<42:05, 13.29s/it]
                                                     
{'loss': 1.5719, 'learning_rate': 2.176224838587163e-05, 'epoch': 0.93}

 93%|█████████▎| 2543/2733 [9:28:57<42:05, 13.29s/it]
 93%|█████████▎| 2544/2733 [9:29:10<41:33, 13.19s/it]
                                                     
{'loss': 1.6181, 'learning_rate': 2.1648309912647167e-05, 'epoch': 0.93}

 93%|█████████▎| 2544/2733 [9:29:10<41:33, 13.19s/it]
 93%|█████████▎| 2545/2733 [9:29:24<41:50, 13.35s/it]
                                                     
{'loss': 1.7455, 'learning_rate': 2.1534371439422708e-05, 'epoch': 0.93}

 93%|█████████▎| 2545/2733 [9:29:24<41:50, 13.35s/it]
 93%|█████████▎| 2546/2733 [9:29:37<41:37, 13.35s/it]
                                                     
{'loss': 1.6542, 'learning_rate': 2.1420432966198252e-05, 'epoch': 0.93}

 93%|█████████▎| 2546/2733 [9:29:37<41:37, 13.35s/it]
 93%|█████████▎| 2547/2733 [9:29:51<41:34, 13.41s/it]
                                                     
{'loss': 1.7807, 'learning_rate': 2.1306494492973792e-05, 'epoch': 0.93}

 93%|█████████▎| 2547/2733 [9:29:51<41:34, 13.41s/it]
 93%|█████████▎| 2548/2733 [9:30:05<41:29, 13.45s/it]
                                                     
{'loss': 1.5951, 'learning_rate': 2.1192556019749336e-05, 'epoch': 0.93}

 93%|█████████▎| 2548/2733 [9:30:05<41:29, 13.45s/it]
 93%|█████████▎| 2549/2733 [9:30:18<41:39, 13.58s/it]
                                                     
{'loss': 1.5638, 'learning_rate': 2.1078617546524873e-05, 'epoch': 0.93}

 93%|█████████▎| 2549/2733 [9:30:18<41:39, 13.58s/it]
 93%|█████████▎| 2550/2733 [9:30:32<41:19, 13.55s/it]
                                                     
{'loss': 1.7151, 'learning_rate': 2.0964679073300417e-05, 'epoch': 0.93}

 93%|█████████▎| 2550/2733 [9:30:32<41:19, 13.55s/it]
 93%|█████████▎| 2551/2733 [9:30:45<40:51, 13.47s/it]
                                                     
{'loss': 1.6009, 'learning_rate': 2.0850740600075958e-05, 'epoch': 0.93}

 93%|█████████▎| 2551/2733 [9:30:45<40:51, 13.47s/it]
 93%|█████████▎| 2552/2733 [9:30:59<40:36, 13.46s/it]
                                                     
{'loss': 1.6501, 'learning_rate': 2.0736802126851498e-05, 'epoch': 0.93}

 93%|█████████▎| 2552/2733 [9:30:59<40:36, 13.46s/it]
 93%|█████████▎| 2553/2733 [9:31:13<40:45, 13.58s/it]
                                                     
{'loss': 1.6481, 'learning_rate': 2.0622863653627042e-05, 'epoch': 0.93}

 93%|█████████▎| 2553/2733 [9:31:13<40:45, 13.58s/it]
 93%|█████████▎| 2554/2733 [9:31:26<40:06, 13.45s/it]
                                                     
{'loss': 1.7433, 'learning_rate': 2.050892518040258e-05, 'epoch': 0.93}

 93%|█████████▎| 2554/2733 [9:31:26<40:06, 13.45s/it]
 93%|█████████▎| 2555/2733 [9:31:39<40:04, 13.51s/it]
                                                     
{'loss': 1.6443, 'learning_rate': 2.0394986707178123e-05, 'epoch': 0.93}

 93%|█████████▎| 2555/2733 [9:31:39<40:04, 13.51s/it]
 94%|█████████▎| 2556/2733 [9:31:54<40:28, 13.72s/it]
                                                     
{'loss': 1.7023, 'learning_rate': 2.0281048233953664e-05, 'epoch': 0.94}

 94%|█████████▎| 2556/2733 [9:31:54<40:28, 13.72s/it]
 94%|█████████▎| 2557/2733 [9:32:07<40:19, 13.75s/it]
                                                     
{'loss': 1.695, 'learning_rate': 2.0167109760729204e-05, 'epoch': 0.94}

 94%|█████████▎| 2557/2733 [9:32:07<40:19, 13.75s/it]
 94%|█████████▎| 2558/2733 [9:32:21<39:37, 13.59s/it]
                                                     
{'loss': 1.6405, 'learning_rate': 2.0053171287504745e-05, 'epoch': 0.94}

 94%|█████████▎| 2558/2733 [9:32:21<39:37, 13.59s/it]
 94%|█████████▎| 2559/2733 [9:32:34<39:06, 13.48s/it]
                                                     
{'loss': 1.6834, 'learning_rate': 1.9939232814280285e-05, 'epoch': 0.94}

 94%|█████████▎| 2559/2733 [9:32:34<39:06, 13.48s/it]
 94%|█████████▎| 2560/2733 [9:32:46<38:07, 13.23s/it]
                                                     
{'loss': 1.5512, 'learning_rate': 1.982529434105583e-05, 'epoch': 0.94}

 94%|█████████▎| 2560/2733 [9:32:46<38:07, 13.23s/it]
 94%|█████████▎| 2561/2733 [9:33:00<38:12, 13.33s/it]
                                                     
{'loss': 1.5895, 'learning_rate': 1.971135586783137e-05, 'epoch': 0.94}

 94%|█████████▎| 2561/2733 [9:33:00<38:12, 13.33s/it]
 94%|█████████▎| 2562/2733 [9:33:14<38:11, 13.40s/it]
                                                     
{'loss': 1.5717, 'learning_rate': 1.9597417394606913e-05, 'epoch': 0.94}

 94%|█████████▎| 2562/2733 [9:33:14<38:11, 13.40s/it]
 94%|█████████▍| 2563/2733 [9:33:27<38:21, 13.54s/it]
                                                     
{'loss': 1.6462, 'learning_rate': 1.948347892138245e-05, 'epoch': 0.94}

 94%|█████████▍| 2563/2733 [9:33:27<38:21, 13.54s/it]
 94%|█████████▍| 2564/2733 [9:33:40<37:24, 13.28s/it]
                                                     
{'loss': 1.6857, 'learning_rate': 1.936954044815799e-05, 'epoch': 0.94}

 94%|█████████▍| 2564/2733 [9:33:40<37:24, 13.28s/it]
 94%|█████████▍| 2565/2733 [9:33:53<37:09, 13.27s/it]
                                                     
{'loss': 1.6927, 'learning_rate': 1.9255601974933535e-05, 'epoch': 0.94}

 94%|█████████▍| 2565/2733 [9:33:53<37:09, 13.27s/it]
 94%|█████████▍| 2566/2733 [9:34:07<37:12, 13.37s/it]
                                                     
{'loss': 1.6948, 'learning_rate': 1.9141663501709076e-05, 'epoch': 0.94}

 94%|█████████▍| 2566/2733 [9:34:07<37:12, 13.37s/it]
 94%|█████████▍| 2567/2733 [9:34:20<36:45, 13.29s/it]
                                                     
{'loss': 1.7093, 'learning_rate': 1.902772502848462e-05, 'epoch': 0.94}

 94%|█████████▍| 2567/2733 [9:34:20<36:45, 13.29s/it]
 94%|█████████▍| 2568/2733 [9:34:33<36:14, 13.18s/it]
                                                     
{'loss': 1.6417, 'learning_rate': 1.8913786555260157e-05, 'epoch': 0.94}

 94%|█████████▍| 2568/2733 [9:34:33<36:14, 13.18s/it]
 94%|█████████▍| 2569/2733 [9:34:46<35:48, 13.10s/it]
                                                     
{'loss': 1.6536, 'learning_rate': 1.87998480820357e-05, 'epoch': 0.94}

 94%|█████████▍| 2569/2733 [9:34:46<35:48, 13.10s/it]
 94%|█████████▍| 2570/2733 [9:35:00<36:06, 13.29s/it]
                                                     
{'loss': 1.6538, 'learning_rate': 1.868590960881124e-05, 'epoch': 0.94}

 94%|█████████▍| 2570/2733 [9:35:00<36:06, 13.29s/it]
 94%|█████████▍| 2571/2733 [9:35:13<35:48, 13.26s/it]
                                                     
{'loss': 1.752, 'learning_rate': 1.857197113558678e-05, 'epoch': 0.94}

 94%|█████████▍| 2571/2733 [9:35:13<35:48, 13.26s/it]
 94%|█████████▍| 2572/2733 [9:35:26<35:21, 13.18s/it]
                                                     
{'loss': 1.6737, 'learning_rate': 1.8458032662362325e-05, 'epoch': 0.94}

 94%|█████████▍| 2572/2733 [9:35:26<35:21, 13.18s/it]
 94%|█████████▍| 2573/2733 [9:35:40<35:47, 13.42s/it]
                                                     
{'loss': 1.6892, 'learning_rate': 1.8344094189137862e-05, 'epoch': 0.94}

 94%|█████████▍| 2573/2733 [9:35:40<35:47, 13.42s/it]
 94%|█████████▍| 2574/2733 [9:35:54<36:13, 13.67s/it]
                                                     
{'loss': 1.6819, 'learning_rate': 1.8230155715913406e-05, 'epoch': 0.94}

 94%|█████████▍| 2574/2733 [9:35:54<36:13, 13.67s/it]
 94%|█████████▍| 2575/2733 [9:36:08<35:55, 13.64s/it]
                                                     
{'loss': 1.6157, 'learning_rate': 1.8116217242688947e-05, 'epoch': 0.94}

 94%|█████████▍| 2575/2733 [9:36:08<35:55, 13.64s/it]
 94%|█████████▍| 2576/2733 [9:36:21<35:22, 13.52s/it]
                                                     
{'loss': 1.6605, 'learning_rate': 1.8002278769464487e-05, 'epoch': 0.94}

 94%|█████████▍| 2576/2733 [9:36:21<35:22, 13.52s/it]
 94%|█████████▍| 2577/2733 [9:36:34<35:07, 13.51s/it]
                                                     
{'loss': 1.752, 'learning_rate': 1.7888340296240028e-05, 'epoch': 0.94}

 94%|█████████▍| 2577/2733 [9:36:34<35:07, 13.51s/it]
 94%|█████████▍| 2578/2733 [9:36:47<34:13, 13.25s/it]
                                                     
{'loss': 1.6537, 'learning_rate': 1.7774401823015572e-05, 'epoch': 0.94}

 94%|█████████▍| 2578/2733 [9:36:47<34:13, 13.25s/it]
 94%|█████████▍| 2579/2733 [9:37:00<33:29, 13.05s/it]
                                                     
{'loss': 1.5911, 'learning_rate': 1.766046334979111e-05, 'epoch': 0.94}

 94%|█████████▍| 2579/2733 [9:37:00<33:29, 13.05s/it]
 94%|█████████▍| 2580/2733 [9:37:13<33:41, 13.21s/it]
                                                     
{'loss': 1.6864, 'learning_rate': 1.7546524876566653e-05, 'epoch': 0.94}

 94%|█████████▍| 2580/2733 [9:37:13<33:41, 13.21s/it]
 94%|█████████▍| 2581/2733 [9:37:27<33:51, 13.36s/it]
                                                     
{'loss': 1.65, 'learning_rate': 1.7432586403342193e-05, 'epoch': 0.94}

 94%|█████████▍| 2581/2733 [9:37:27<33:51, 13.36s/it]
 94%|█████████▍| 2582/2733 [9:37:39<32:47, 13.03s/it]
                                                     
{'loss': 1.5981, 'learning_rate': 1.7318647930117734e-05, 'epoch': 0.94}

 94%|█████████▍| 2582/2733 [9:37:39<32:47, 13.03s/it]
 95%|█████████▍| 2583/2733 [9:37:53<32:53, 13.16s/it]
                                                     
{'loss': 1.6529, 'learning_rate': 1.7204709456893278e-05, 'epoch': 0.95}

 95%|█████████▍| 2583/2733 [9:37:53<32:53, 13.16s/it]
 95%|█████████▍| 2584/2733 [9:38:06<33:02, 13.30s/it]
                                                     
{'loss': 1.696, 'learning_rate': 1.7090770983668818e-05, 'epoch': 0.95}

 95%|█████████▍| 2584/2733 [9:38:06<33:02, 13.30s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

 95%|█████████▍| 2585/2733 [9:38:20<32:57, 13.36s/it]
                                                     
{'loss': 1.7666, 'learning_rate': 1.697683251044436e-05, 'epoch': 0.95}

 95%|█████████▍| 2585/2733 [9:38:20<32:57, 13.36s/it]
 95%|█████████▍| 2586/2733 [9:38:33<32:54, 13.43s/it]
                                                     
{'loss': 1.6568, 'learning_rate': 1.68628940372199e-05, 'epoch': 0.95}

 95%|█████████▍| 2586/2733 [9:38:33<32:54, 13.43s/it]
 95%|█████████▍| 2587/2733 [9:38:45<31:43, 13.04s/it]
                                                     
{'loss': 1.5918, 'learning_rate': 1.674895556399544e-05, 'epoch': 0.95}

 95%|█████████▍| 2587/2733 [9:38:45<31:43, 13.04s/it]
 95%|█████████▍| 2588/2733 [9:38:58<31:14, 12.93s/it]
                                                     
{'loss': 1.6472, 'learning_rate': 1.6635017090770984e-05, 'epoch': 0.95}

 95%|█████████▍| 2588/2733 [9:38:58<31:14, 12.93s/it]
 95%|█████████▍| 2589/2733 [9:39:11<31:09, 12.98s/it]
                                                     
{'loss': 1.6379, 'learning_rate': 1.6521078617546524e-05, 'epoch': 0.95}

 95%|█████████▍| 2589/2733 [9:39:11<31:09, 12.98s/it]
 95%|█████████▍| 2590/2733 [9:39:25<31:42, 13.31s/it]
                                                     
{'loss': 1.6458, 'learning_rate': 1.6407140144322065e-05, 'epoch': 0.95}

 95%|█████████▍| 2590/2733 [9:39:25<31:42, 13.31s/it]
 95%|█████████▍| 2591/2733 [9:39:39<31:40, 13.38s/it]
                                                     
{'loss': 1.6103, 'learning_rate': 1.6293201671097605e-05, 'epoch': 0.95}

 95%|█████████▍| 2591/2733 [9:39:39<31:40, 13.38s/it]
 95%|█████████▍| 2592/2733 [9:39:52<31:32, 13.42s/it]
                                                     
{'loss': 1.5953, 'learning_rate': 1.6179263197873146e-05, 'epoch': 0.95}

 95%|█████████▍| 2592/2733 [9:39:52<31:32, 13.42s/it]
 95%|█████████▍| 2593/2733 [9:40:06<31:09, 13.35s/it]
                                                     
{'loss': 1.6861, 'learning_rate': 1.6065324724648686e-05, 'epoch': 0.95}

 95%|█████████▍| 2593/2733 [9:40:06<31:09, 13.35s/it]
 95%|█████████▍| 2594/2733 [9:40:19<31:05, 13.42s/it]
                                                     
{'loss': 1.6635, 'learning_rate': 1.595138625142423e-05, 'epoch': 0.95}

 95%|█████████▍| 2594/2733 [9:40:19<31:05, 13.42s/it]
 95%|█████████▍| 2595/2733 [9:40:33<30:51, 13.42s/it]
                                                     
{'loss': 1.5735, 'learning_rate': 1.583744777819977e-05, 'epoch': 0.95}

 95%|█████████▍| 2595/2733 [9:40:33<30:51, 13.42s/it]
 95%|█████████▍| 2596/2733 [9:40:46<30:24, 13.32s/it]
                                                     
{'loss': 1.7377, 'learning_rate': 1.572350930497531e-05, 'epoch': 0.95}

 95%|█████████▍| 2596/2733 [9:40:46<30:24, 13.32s/it]
 95%|█████████▌| 2597/2733 [9:40:59<30:21, 13.40s/it]
                                                     
{'loss': 1.6803, 'learning_rate': 1.5609570831750855e-05, 'epoch': 0.95}

 95%|█████████▌| 2597/2733 [9:40:59<30:21, 13.40s/it]
 95%|█████████▌| 2598/2733 [9:41:13<30:10, 13.41s/it]
                                                     
{'loss': 1.483, 'learning_rate': 1.5495632358526392e-05, 'epoch': 0.95}

 95%|█████████▌| 2598/2733 [9:41:13<30:10, 13.41s/it]
 95%|█████████▌| 2599/2733 [9:41:26<30:08, 13.49s/it]
                                                     
{'loss': 1.6873, 'learning_rate': 1.5381693885301936e-05, 'epoch': 0.95}

 95%|█████████▌| 2599/2733 [9:41:26<30:08, 13.49s/it]
 95%|█████████▌| 2600/2733 [9:41:40<30:02, 13.55s/it]
                                                     
{'loss': 1.6645, 'learning_rate': 1.5267755412077477e-05, 'epoch': 0.95}

 95%|█████████▌| 2600/2733 [9:41:40<30:02, 13.55s/it]
 95%|█████████▌| 2601/2733 [9:41:54<29:47, 13.54s/it]
                                                     
{'loss': 1.6839, 'learning_rate': 1.5153816938853019e-05, 'epoch': 0.95}

 95%|█████████▌| 2601/2733 [9:41:54<29:47, 13.54s/it]
 95%|█████████▌| 2602/2733 [9:42:07<29:37, 13.57s/it]
                                                     
{'loss': 1.7431, 'learning_rate': 1.503987846562856e-05, 'epoch': 0.95}

 95%|█████████▌| 2602/2733 [9:42:07<29:37, 13.57s/it]
 95%|█████████▌| 2603/2733 [9:42:21<29:17, 13.52s/it]
                                                     
{'loss': 1.7333, 'learning_rate': 1.4925939992404102e-05, 'epoch': 0.95}

 95%|█████████▌| 2603/2733 [9:42:21<29:17, 13.52s/it]
 95%|█████████▌| 2604/2733 [9:42:33<28:27, 13.24s/it]
                                                     
{'loss': 1.6801, 'learning_rate': 1.481200151917964e-05, 'epoch': 0.95}

 95%|█████████▌| 2604/2733 [9:42:33<28:27, 13.24s/it]
 95%|█████████▌| 2605/2733 [9:42:47<28:36, 13.41s/it]
                                                     
{'loss': 1.7684, 'learning_rate': 1.4698063045955183e-05, 'epoch': 0.95}

 95%|█████████▌| 2605/2733 [9:42:47<28:36, 13.41s/it]
 95%|█████████▌| 2606/2733 [9:43:00<28:02, 13.25s/it]
                                                     
{'loss': 1.6161, 'learning_rate': 1.4584124572730723e-05, 'epoch': 0.95}

 95%|█████████▌| 2606/2733 [9:43:00<28:02, 13.25s/it]
 95%|█████████▌| 2607/2733 [9:43:14<28:10, 13.42s/it]
                                                     
{'loss': 1.7375, 'learning_rate': 1.4470186099506265e-05, 'epoch': 0.95}

 95%|█████████▌| 2607/2733 [9:43:14<28:10, 13.42s/it]
 95%|█████████▌| 2608/2733 [9:43:27<27:50, 13.36s/it]
                                                     
{'loss': 1.755, 'learning_rate': 1.4356247626281807e-05, 'epoch': 0.95}

 95%|█████████▌| 2608/2733 [9:43:27<27:50, 13.36s/it]
 95%|█████████▌| 2609/2733 [9:43:40<27:46, 13.44s/it]
                                                     
{'loss': 1.6151, 'learning_rate': 1.4242309153057348e-05, 'epoch': 0.95}

 95%|█████████▌| 2609/2733 [9:43:41<27:46, 13.44s/it]
 95%|█████████▌| 2610/2733 [9:43:54<27:46, 13.55s/it]
                                                     
{'loss': 1.6821, 'learning_rate': 1.412837067983289e-05, 'epoch': 0.95}

 95%|█████████▌| 2610/2733 [9:43:54<27:46, 13.55s/it]
 96%|█████████▌| 2611/2733 [9:44:08<27:21, 13.45s/it]
                                                     
{'loss': 1.648, 'learning_rate': 1.4014432206608429e-05, 'epoch': 0.96}

 96%|█████████▌| 2611/2733 [9:44:08<27:21, 13.45s/it]
 96%|█████████▌| 2612/2733 [9:44:20<26:35, 13.19s/it]
                                                     
{'loss': 1.5727, 'learning_rate': 1.3900493733383971e-05, 'epoch': 0.96}

 96%|█████████▌| 2612/2733 [9:44:20<26:35, 13.19s/it]
 96%|█████████▌| 2613/2733 [9:44:34<26:40, 13.34s/it]
                                                     
{'loss': 1.6789, 'learning_rate': 1.3786555260159512e-05, 'epoch': 0.96}

 96%|█████████▌| 2613/2733 [9:44:34<26:40, 13.34s/it]
 96%|█████████▌| 2614/2733 [9:44:47<26:17, 13.26s/it]
                                                     
{'loss': 1.5593, 'learning_rate': 1.3672616786935054e-05, 'epoch': 0.96}

 96%|█████████▌| 2614/2733 [9:44:47<26:17, 13.26s/it]
 96%|█████████▌| 2615/2733 [9:45:00<25:46, 13.10s/it]
                                                     
{'loss': 1.5362, 'learning_rate': 1.3558678313710596e-05, 'epoch': 0.96}

 96%|█████████▌| 2615/2733 [9:45:00<25:46, 13.10s/it]
 96%|█████████▌| 2616/2733 [9:45:13<25:53, 13.28s/it]
                                                     
{'loss': 1.6701, 'learning_rate': 1.3444739840486137e-05, 'epoch': 0.96}

 96%|█████████▌| 2616/2733 [9:45:13<25:53, 13.28s/it]
 96%|█████████▌| 2617/2733 [9:45:27<25:54, 13.40s/it]
                                                     
{'loss': 1.7917, 'learning_rate': 1.3330801367261677e-05, 'epoch': 0.96}

 96%|█████████▌| 2617/2733 [9:45:27<25:54, 13.40s/it]
 96%|█████████▌| 2618/2733 [9:45:40<25:34, 13.34s/it]
                                                     
{'loss': 1.6597, 'learning_rate': 1.3216862894037218e-05, 'epoch': 0.96}

 96%|█████████▌| 2618/2733 [9:45:40<25:34, 13.34s/it]
 96%|█████████▌| 2619/2733 [9:45:54<25:23, 13.36s/it]
                                                     
{'loss': 1.6401, 'learning_rate': 1.310292442081276e-05, 'epoch': 0.96}

 96%|█████████▌| 2619/2733 [9:45:54<25:23, 13.36s/it]
 96%|█████████▌| 2620/2733 [9:46:07<25:16, 13.42s/it]
                                                     
{'loss': 1.6328, 'learning_rate': 1.29889859475883e-05, 'epoch': 0.96}

 96%|█████████▌| 2620/2733 [9:46:07<25:16, 13.42s/it]
 96%|█████████▌| 2621/2733 [9:46:21<25:15, 13.54s/it]
                                                     
{'loss': 1.7546, 'learning_rate': 1.2875047474363843e-05, 'epoch': 0.96}

 96%|█████████▌| 2621/2733 [9:46:21<25:15, 13.54s/it]
 96%|█████████▌| 2622/2733 [9:46:34<24:58, 13.50s/it]
                                                     
{'loss': 1.719, 'learning_rate': 1.2761109001139385e-05, 'epoch': 0.96}

 96%|█████████▌| 2622/2733 [9:46:34<24:58, 13.50s/it]
 96%|█████████▌| 2623/2733 [9:46:48<24:41, 13.47s/it]
                                                     
{'loss': 1.6211, 'learning_rate': 1.2647170527914924e-05, 'epoch': 0.96}

 96%|█████████▌| 2623/2733 [9:46:48<24:41, 13.47s/it]
 96%|█████████▌| 2624/2733 [9:47:01<24:24, 13.44s/it]
                                                     
{'loss': 1.6467, 'learning_rate': 1.2533232054690466e-05, 'epoch': 0.96}

 96%|█████████▌| 2624/2733 [9:47:01<24:24, 13.44s/it]
 96%|█████████▌| 2625/2733 [9:47:15<24:21, 13.53s/it]
                                                     
{'loss': 1.6314, 'learning_rate': 1.2419293581466006e-05, 'epoch': 0.96}

 96%|█████████▌| 2625/2733 [9:47:15<24:21, 13.53s/it]
 96%|█████████▌| 2626/2733 [9:47:28<24:03, 13.49s/it]
                                                     
{'loss': 1.6667, 'learning_rate': 1.2305355108241549e-05, 'epoch': 0.96}

 96%|█████████▌| 2626/2733 [9:47:28<24:03, 13.49s/it]
 96%|█████████▌| 2627/2733 [9:47:42<23:44, 13.43s/it]
                                                     
{'loss': 1.6545, 'learning_rate': 1.219141663501709e-05, 'epoch': 0.96}

 96%|█████████▌| 2627/2733 [9:47:42<23:44, 13.43s/it]
 96%|█████████▌| 2628/2733 [9:47:55<23:38, 13.51s/it]
                                                     
{'loss': 1.7282, 'learning_rate': 1.2077478161792631e-05, 'epoch': 0.96}

 96%|█████████▌| 2628/2733 [9:47:55<23:38, 13.51s/it]
 96%|█████████▌| 2629/2733 [9:48:09<23:28, 13.55s/it]
                                                     
{'loss': 1.6078, 'learning_rate': 1.1963539688568173e-05, 'epoch': 0.96}

 96%|█████████▌| 2629/2733 [9:48:09<23:28, 13.55s/it]
 96%|█████████▌| 2630/2733 [9:48:22<23:09, 13.49s/it]
                                                     
{'loss': 1.6409, 'learning_rate': 1.1849601215343712e-05, 'epoch': 0.96}

 96%|█████████▌| 2630/2733 [9:48:22<23:09, 13.49s/it]
 96%|█████████▋| 2631/2733 [9:48:36<23:03, 13.56s/it]
                                                     
{'loss': 1.7323, 'learning_rate': 1.1735662742119254e-05, 'epoch': 0.96}

 96%|█████████▋| 2631/2733 [9:48:36<23:03, 13.56s/it]
 96%|█████████▋| 2632/2733 [9:48:50<23:10, 13.76s/it]
                                                     
{'loss': 1.7316, 'learning_rate': 1.1621724268894795e-05, 'epoch': 0.96}

 96%|█████████▋| 2632/2733 [9:48:50<23:10, 13.76s/it]
 96%|█████████▋| 2633/2733 [9:49:04<23:09, 13.89s/it]
                                                     
{'loss': 1.6528, 'learning_rate': 1.1507785795670337e-05, 'epoch': 0.96}

 96%|█████████▋| 2633/2733 [9:49:04<23:09, 13.89s/it]
 96%|█████████▋| 2634/2733 [9:49:18<22:33, 13.67s/it]
                                                     
{'loss': 1.6963, 'learning_rate': 1.139384732244588e-05, 'epoch': 0.96}

 96%|█████████▋| 2634/2733 [9:49:18<22:33, 13.67s/it]
 96%|█████████▋| 2635/2733 [9:49:31<22:15, 13.63s/it]
                                                     
{'loss': 1.7232, 'learning_rate': 1.127990884922142e-05, 'epoch': 0.96}

 96%|█████████▋| 2635/2733 [9:49:31<22:15, 13.63s/it]
 96%|█████████▋| 2636/2733 [9:49:44<21:40, 13.41s/it]
                                                     
{'loss': 1.6806, 'learning_rate': 1.116597037599696e-05, 'epoch': 0.96}

 96%|█████████▋| 2636/2733 [9:49:44<21:40, 13.41s/it]
 96%|█████████▋| 2637/2733 [9:49:56<20:48, 13.01s/it]
                                                     
{'loss': 1.6165, 'learning_rate': 1.1052031902772501e-05, 'epoch': 0.96}

 96%|█████████▋| 2637/2733 [9:49:56<20:48, 13.01s/it]
 97%|█████████▋| 2638/2733 [9:50:09<20:21, 12.86s/it]
                                                     
{'loss': 1.6036, 'learning_rate': 1.0938093429548043e-05, 'epoch': 0.97}

 97%|█████████▋| 2638/2733 [9:50:09<20:21, 12.86s/it]
 97%|█████████▋| 2639/2733 [9:50:22<20:26, 13.05s/it]
                                                     
{'loss': 1.6223, 'learning_rate': 1.0824154956323584e-05, 'epoch': 0.97}

 97%|█████████▋| 2639/2733 [9:50:22<20:26, 13.05s/it]
 97%|█████████▋| 2640/2733 [9:50:35<20:14, 13.05s/it]
                                                     
{'loss': 1.6544, 'learning_rate': 1.0710216483099126e-05, 'epoch': 0.97}

 97%|█████████▋| 2640/2733 [9:50:35<20:14, 13.05s/it]
 97%|█████████▋| 2641/2733 [9:50:49<20:16, 13.22s/it]
                                                     
{'loss': 1.7284, 'learning_rate': 1.0596278009874668e-05, 'epoch': 0.97}

 97%|█████████▋| 2641/2733 [9:50:49<20:16, 13.22s/it]
 97%|█████████▋| 2642/2733 [9:51:02<19:54, 13.12s/it]
                                                     
{'loss': 1.5345, 'learning_rate': 1.0482339536650209e-05, 'epoch': 0.97}

 97%|█████████▋| 2642/2733 [9:51:02<19:54, 13.12s/it]
 97%|█████████▋| 2643/2733 [9:51:15<19:43, 13.16s/it]
                                                     
{'loss': 1.6603, 'learning_rate': 1.0368401063425749e-05, 'epoch': 0.97}

 97%|█████████▋| 2643/2733 [9:51:15<19:43, 13.16s/it]
 97%|█████████▋| 2644/2733 [9:51:29<20:00, 13.49s/it]
                                                     
{'loss': 1.6724, 'learning_rate': 1.025446259020129e-05, 'epoch': 0.97}

 97%|█████████▋| 2644/2733 [9:51:29<20:00, 13.49s/it]
 97%|█████████▋| 2645/2733 [9:51:42<19:28, 13.27s/it]
                                                     
{'loss': 1.6785, 'learning_rate': 1.0140524116976832e-05, 'epoch': 0.97}

 97%|█████████▋| 2645/2733 [9:51:42<19:28, 13.27s/it]
 97%|█████████▋| 2646/2733 [9:51:55<19:22, 13.37s/it]
                                                     
{'loss': 1.6018, 'learning_rate': 1.0026585643752372e-05, 'epoch': 0.97}

 97%|█████████▋| 2646/2733 [9:51:56<19:22, 13.37s/it]
 97%|█████████▋| 2647/2733 [9:52:09<19:09, 13.37s/it]
                                                     
{'loss': 1.591, 'learning_rate': 9.912647170527915e-06, 'epoch': 0.97}

 97%|█████████▋| 2647/2733 [9:52:09<19:09, 13.37s/it]
 97%|█████████▋| 2648/2733 [9:52:22<18:47, 13.26s/it]
                                                     
{'loss': 1.6005, 'learning_rate': 9.798708697303457e-06, 'epoch': 0.97}

 97%|█████████▋| 2648/2733 [9:52:22<18:47, 13.26s/it]
 97%|█████████▋| 2649/2733 [9:52:35<18:39, 13.33s/it]
                                                     
{'loss': 1.793, 'learning_rate': 9.684770224078996e-06, 'epoch': 0.97}

 97%|█████████▋| 2649/2733 [9:52:35<18:39, 13.33s/it]
 97%|█████████▋| 2650/2733 [9:52:48<18:16, 13.21s/it]
                                                     
{'loss': 1.497, 'learning_rate': 9.570831750854538e-06, 'epoch': 0.97}

 97%|█████████▋| 2650/2733 [9:52:48<18:16, 13.21s/it]
 97%|█████████▋| 2651/2733 [9:53:01<18:02, 13.20s/it]
                                                     
{'loss': 1.604, 'learning_rate': 9.456893277630078e-06, 'epoch': 0.97}

 97%|█████████▋| 2651/2733 [9:53:01<18:02, 13.20s/it]
 97%|█████████▋| 2652/2733 [9:53:15<17:50, 13.22s/it]
                                                     
{'loss': 1.6982, 'learning_rate': 9.34295480440562e-06, 'epoch': 0.97}

 97%|█████████▋| 2652/2733 [9:53:15<17:50, 13.22s/it]
 97%|█████████▋| 2653/2733 [9:53:28<17:35, 13.19s/it]
                                                     
{'loss': 1.5688, 'learning_rate': 9.229016331181163e-06, 'epoch': 0.97}

 97%|█████████▋| 2653/2733 [9:53:28<17:35, 13.19s/it]
 97%|█████████▋| 2654/2733 [9:53:42<17:40, 13.42s/it]
                                                     
{'loss': 1.5855, 'learning_rate': 9.115077857956703e-06, 'epoch': 0.97}

 97%|█████████▋| 2654/2733 [9:53:42<17:40, 13.42s/it]
 97%|█████████▋| 2655/2733 [9:53:55<17:27, 13.43s/it]
                                                     
{'loss': 1.7108, 'learning_rate': 9.001139384732244e-06, 'epoch': 0.97}

 97%|█████████▋| 2655/2733 [9:53:55<17:27, 13.43s/it]
 97%|█████████▋| 2656/2733 [9:54:08<17:08, 13.35s/it]
                                                     
{'loss': 1.5386, 'learning_rate': 8.887200911507786e-06, 'epoch': 0.97}

 97%|█████████▋| 2656/2733 [9:54:08<17:08, 13.35s/it]
 97%|█████████▋| 2657/2733 [9:54:23<17:14, 13.62s/it]
                                                     
{'loss': 1.5806, 'learning_rate': 8.773262438283326e-06, 'epoch': 0.97}

 97%|█████████▋| 2657/2733 [9:54:23<17:14, 13.62s/it]
 97%|█████████▋| 2658/2733 [9:54:36<17:03, 13.65s/it]
                                                     
{'loss': 1.6922, 'learning_rate': 8.659323965058867e-06, 'epoch': 0.97}

 97%|█████████▋| 2658/2733 [9:54:36<17:03, 13.65s/it]
 97%|█████████▋| 2659/2733 [9:54:50<16:45, 13.59s/it]
                                                     
{'loss': 1.6171, 'learning_rate': 8.545385491834409e-06, 'epoch': 0.97}

 97%|█████████▋| 2659/2733 [9:54:50<16:45, 13.59s/it]
 97%|█████████▋| 2660/2733 [9:55:03<16:30, 13.57s/it]
                                                     
{'loss': 1.626, 'learning_rate': 8.43144701860995e-06, 'epoch': 0.97}

 97%|█████████▋| 2660/2733 [9:55:03<16:30, 13.57s/it]
 97%|█████████▋| 2661/2733 [9:55:17<16:08, 13.45s/it]
                                                     
{'loss': 1.6977, 'learning_rate': 8.317508545385492e-06, 'epoch': 0.97}

 97%|█████████▋| 2661/2733 [9:55:17<16:08, 13.45s/it]
 97%|█████████▋| 2662/2733 [9:55:30<15:47, 13.34s/it]
                                                     
{'loss': 1.6573, 'learning_rate': 8.203570072161032e-06, 'epoch': 0.97}

 97%|█████████▋| 2662/2733 [9:55:30<15:47, 13.34s/it]
 97%|█████████▋| 2663/2733 [9:55:43<15:34, 13.36s/it]
                                                     
{'loss': 1.6436, 'learning_rate': 8.089631598936573e-06, 'epoch': 0.97}

 97%|█████████▋| 2663/2733 [9:55:43<15:34, 13.36s/it]
 97%|█████████▋| 2664/2733 [9:55:57<15:26, 13.43s/it]
                                                     
{'loss': 1.6073, 'learning_rate': 7.975693125712115e-06, 'epoch': 0.97}

 97%|█████████▋| 2664/2733 [9:55:57<15:26, 13.43s/it]
 98%|█████████▊| 2665/2733 [9:56:10<15:12, 13.42s/it]
                                                     
{'loss': 1.6178, 'learning_rate': 7.861754652487656e-06, 'epoch': 0.98}

 98%|█████████▊| 2665/2733 [9:56:10<15:12, 13.42s/it]
 98%|█████████▊| 2666/2733 [9:56:23<14:58, 13.41s/it]
                                                     
{'loss': 1.5393, 'learning_rate': 7.747816179263196e-06, 'epoch': 0.98}

 98%|█████████▊| 2666/2733 [9:56:23<14:58, 13.41s/it]
 98%|█████████▊| 2667/2733 [9:56:36<14:37, 13.30s/it]
                                                     
{'loss': 1.7306, 'learning_rate': 7.633877706038738e-06, 'epoch': 0.98}

 98%|█████████▊| 2667/2733 [9:56:36<14:37, 13.30s/it]
 98%|█████████▊| 2668/2733 [9:56:50<14:19, 13.23s/it]
                                                     
{'loss': 1.71, 'learning_rate': 7.51993923281428e-06, 'epoch': 0.98}

 98%|█████████▊| 2668/2733 [9:56:50<14:19, 13.23s/it]
 98%|█████████▊| 2669/2733 [9:57:04<14:24, 13.51s/it]
                                                     
{'loss': 1.698, 'learning_rate': 7.40600075958982e-06, 'epoch': 0.98}

 98%|█████████▊| 2669/2733 [9:57:04<14:24, 13.51s/it]
 98%|█████████▊| 2670/2733 [9:57:17<14:12, 13.54s/it]
                                                     
{'loss': 1.5357, 'learning_rate': 7.2920622863653615e-06, 'epoch': 0.98}

 98%|█████████▊| 2670/2733 [9:57:17<14:12, 13.54s/it]
 98%|█████████▊| 2671/2733 [9:57:31<14:00, 13.56s/it]
                                                     
{'loss': 1.714, 'learning_rate': 7.178123813140904e-06, 'epoch': 0.98}

 98%|█████████▊| 2671/2733 [9:57:31<14:00, 13.56s/it]
 98%|█████████▊| 2672/2733 [9:57:44<13:43, 13.50s/it]
                                                     
{'loss': 1.7271, 'learning_rate': 7.064185339916445e-06, 'epoch': 0.98}

 98%|█████████▊| 2672/2733 [9:57:44<13:43, 13.50s/it]
 98%|█████████▊| 2673/2733 [9:57:58<13:30, 13.51s/it]
                                                     
{'loss': 1.7241, 'learning_rate': 6.950246866691986e-06, 'epoch': 0.98}

 98%|█████████▊| 2673/2733 [9:57:58<13:30, 13.51s/it]
 98%|█████████▊| 2674/2733 [9:58:11<13:20, 13.58s/it]
                                                     
{'loss': 1.7016, 'learning_rate': 6.836308393467527e-06, 'epoch': 0.98}

 98%|█████████▊| 2674/2733 [9:58:12<13:20, 13.58s/it]
 98%|█████████▊| 2675/2733 [9:58:25<13:11, 13.65s/it]
                                                     
{'loss': 1.647, 'learning_rate': 6.722369920243068e-06, 'epoch': 0.98}

 98%|█████████▊| 2675/2733 [9:58:25<13:11, 13.65s/it]
 98%|█████████▊| 2676/2733 [9:58:38<12:44, 13.42s/it]
                                                     
{'loss': 1.7031, 'learning_rate': 6.608431447018609e-06, 'epoch': 0.98}

 98%|█████████▊| 2676/2733 [9:58:38<12:44, 13.42s/it]
 98%|█████████▊| 2677/2733 [9:58:52<12:35, 13.50s/it]
                                                     
{'loss': 1.6586, 'learning_rate': 6.49449297379415e-06, 'epoch': 0.98}

 98%|█████████▊| 2677/2733 [9:58:52<12:35, 13.50s/it]
 98%|█████████▊| 2678/2733 [9:59:05<12:22, 13.50s/it]
                                                     
{'loss': 1.661, 'learning_rate': 6.380554500569692e-06, 'epoch': 0.98}

 98%|█████████▊| 2678/2733 [9:59:05<12:22, 13.50s/it]
 98%|█████████▊| 2679/2733 [9:59:19<12:12, 13.56s/it]
                                                     
{'loss': 1.669, 'learning_rate': 6.266616027345233e-06, 'epoch': 0.98}

 98%|█████████▊| 2679/2733 [9:59:19<12:12, 13.56s/it]
 98%|█████████▊| 2680/2733 [9:59:32<11:43, 13.28s/it]
                                                     
{'loss': 1.5963, 'learning_rate': 6.152677554120774e-06, 'epoch': 0.98}

 98%|█████████▊| 2680/2733 [9:59:32<11:43, 13.28s/it]
 98%|█████████▊| 2681/2733 [9:59:46<11:39, 13.44s/it]
                                                     
{'loss': 1.7852, 'learning_rate': 6.038739080896316e-06, 'epoch': 0.98}

 98%|█████████▊| 2681/2733 [9:59:46<11:39, 13.44s/it]
 98%|█████████▊| 2682/2733 [9:59:59<11:23, 13.40s/it]
                                                     
{'loss': 1.6114, 'learning_rate': 5.924800607671856e-06, 'epoch': 0.98}

 98%|█████████▊| 2682/2733 [9:59:59<11:23, 13.40s/it]
 98%|█████████▊| 2683/2733 [10:00:12<11:00, 13.22s/it]
                                                      
{'loss': 1.5931, 'learning_rate': 5.8108621344473975e-06, 'epoch': 0.98}

 98%|█████████▊| 2683/2733 [10:00:12<11:00, 13.22s/it]
 98%|█████████▊| 2684/2733 [10:00:25<10:54, 13.36s/it]
                                                      
{'loss': 1.6423, 'learning_rate': 5.69692366122294e-06, 'epoch': 0.98}

 98%|█████████▊| 2684/2733 [10:00:25<10:54, 13.36s/it]
 98%|█████████▊| 2685/2733 [10:00:39<10:45, 13.46s/it]
                                                      
{'loss': 1.7931, 'learning_rate': 5.58298518799848e-06, 'epoch': 0.98}

 98%|█████████▊| 2685/2733 [10:00:39<10:45, 13.46s/it]
 98%|█████████▊| 2686/2733 [10:00:52<10:24, 13.29s/it]
                                                      
{'loss': 1.6302, 'learning_rate': 5.469046714774022e-06, 'epoch': 0.98}

 98%|█████████▊| 2686/2733 [10:00:52<10:24, 13.29s/it]
 98%|█████████▊| 2687/2733 [10:01:06<10:23, 13.56s/it]
                                                      
{'loss': 1.6179, 'learning_rate': 5.355108241549563e-06, 'epoch': 0.98}

 98%|█████████▊| 2687/2733 [10:01:06<10:23, 13.56s/it]
 98%|█████████▊| 2688/2733 [10:01:20<10:14, 13.65s/it]
                                                      
{'loss': 1.7335, 'learning_rate': 5.241169768325104e-06, 'epoch': 0.98}

 98%|█████████▊| 2688/2733 [10:01:20<10:14, 13.65s/it]
 98%|█████████▊| 2689/2733 [10:01:33<09:47, 13.35s/it]
                                                      
{'loss': 1.5353, 'learning_rate': 5.127231295100645e-06, 'epoch': 0.98}

 98%|█████████▊| 2689/2733 [10:01:33<09:47, 13.35s/it]
 98%|█████████▊| 2690/2733 [10:01:46<09:34, 13.37s/it]
                                                      
{'loss': 1.6898, 'learning_rate': 5.013292821876186e-06, 'epoch': 0.98}

 98%|█████████▊| 2690/2733 [10:01:46<09:34, 13.37s/it]
 98%|█████████▊| 2691/2733 [10:02:00<09:25, 13.46s/it]
                                                      
{'loss': 1.7206, 'learning_rate': 4.899354348651728e-06, 'epoch': 0.98}

 98%|█████████▊| 2691/2733 [10:02:00<09:25, 13.46s/it]
 98%|█████████▊| 2692/2733 [10:02:13<09:13, 13.50s/it]
                                                      
{'loss': 1.6672, 'learning_rate': 4.785415875427269e-06, 'epoch': 0.98}

 98%|█████████▊| 2692/2733 [10:02:13<09:13, 13.50s/it]
 99%|█████████▊| 2693/2733 [10:02:26<08:52, 13.31s/it]
                                                      
{'loss': 1.7551, 'learning_rate': 4.67147740220281e-06, 'epoch': 0.99}

 99%|█████████▊| 2693/2733 [10:02:26<08:52, 13.31s/it]
 99%|█████████▊| 2694/2733 [10:02:40<08:43, 13.41s/it]
                                                      
{'loss': 1.6417, 'learning_rate': 4.557538928978352e-06, 'epoch': 0.99}

 99%|█████████▊| 2694/2733 [10:02:40<08:43, 13.41s/it]
 99%|█████████▊| 2695/2733 [10:02:52<08:20, 13.18s/it]
                                                      
{'loss': 1.6387, 'learning_rate': 4.443600455753893e-06, 'epoch': 0.99}

 99%|█████████▊| 2695/2733 [10:02:52<08:20, 13.18s/it]
 99%|█████████▊| 2696/2733 [10:03:06<08:14, 13.36s/it]
                                                      
{'loss': 1.6675, 'learning_rate': 4.3296619825294335e-06, 'epoch': 0.99}

 99%|█████████▊| 2696/2733 [10:03:06<08:14, 13.36s/it]
 99%|█████████▊| 2697/2733 [10:03:19<07:54, 13.18s/it]
                                                      
{'loss': 1.7116, 'learning_rate': 4.215723509304975e-06, 'epoch': 0.99}

 99%|█████████▊| 2697/2733 [10:03:19<07:54, 13.18s/it]
 99%|█████████▊| 2698/2733 [10:03:33<07:47, 13.34s/it]
                                                      
{'loss': 1.7358, 'learning_rate': 4.101785036080516e-06, 'epoch': 0.99}

 99%|█████████▊| 2698/2733 [10:03:33<07:47, 13.34s/it]
 99%|█████████▉| 2699/2733 [10:03:46<07:37, 13.46s/it]
                                                      
{'loss': 1.6945, 'learning_rate': 3.9878465628560575e-06, 'epoch': 0.99}

 99%|█████████▉| 2699/2733 [10:03:46<07:37, 13.46s/it]
 99%|█████████▉| 2700/2733 [10:03:59<07:17, 13.25s/it]
                                                      
{'loss': 1.5948, 'learning_rate': 3.873908089631598e-06, 'epoch': 0.99}

 99%|█████████▉| 2700/2733 [10:03:59<07:17, 13.25s/it]
 99%|█████████▉| 2701/2733 [10:04:12<06:56, 13.01s/it]
                                                      
{'loss': 1.5816, 'learning_rate': 3.75996961640714e-06, 'epoch': 0.99}

 99%|█████████▉| 2701/2733 [10:04:12<06:56, 13.01s/it]
 99%|█████████▉| 2702/2733 [10:04:26<06:54, 13.36s/it]
                                                      
{'loss': 1.6065, 'learning_rate': 3.6460311431826808e-06, 'epoch': 0.99}

 99%|█████████▉| 2702/2733 [10:04:26<06:54, 13.36s/it]
 99%|█████████▉| 2703/2733 [10:04:39<06:38, 13.29s/it]
                                                      
{'loss': 1.7004, 'learning_rate': 3.5320926699582226e-06, 'epoch': 0.99}

 99%|█████████▉| 2703/2733 [10:04:39<06:38, 13.29s/it]
 99%|█████████▉| 2704/2733 [10:04:52<06:22, 13.20s/it]
                                                      
{'loss': 1.6871, 'learning_rate': 3.4181541967337635e-06, 'epoch': 0.99}

 99%|█████████▉| 2704/2733 [10:04:52<06:22, 13.20s/it]
 99%|█████████▉| 2705/2733 [10:05:05<06:07, 13.14s/it]
                                                      
{'loss': 1.6591, 'learning_rate': 3.3042157235093044e-06, 'epoch': 0.99}

 99%|█████████▉| 2705/2733 [10:05:05<06:07, 13.14s/it]
 99%|█████████▉| 2706/2733 [10:05:18<05:55, 13.15s/it]
                                                      
{'loss': 1.696, 'learning_rate': 3.190277250284846e-06, 'epoch': 0.99}

 99%|█████████▉| 2706/2733 [10:05:18<05:55, 13.15s/it]
 99%|█████████▉| 2707/2733 [10:05:32<05:48, 13.39s/it]
                                                      
{'loss': 1.5805, 'learning_rate': 3.076338777060387e-06, 'epoch': 0.99}

 99%|█████████▉| 2707/2733 [10:05:32<05:48, 13.39s/it]
 99%|█████████▉| 2708/2733 [10:05:46<05:41, 13.64s/it]
                                                      
{'loss': 1.6319, 'learning_rate': 2.962400303835928e-06, 'epoch': 0.99}

 99%|█████████▉| 2708/2733 [10:05:46<05:41, 13.64s/it]
 99%|█████████▉| 2709/2733 [10:06:00<05:24, 13.53s/it]
                                                      
{'loss': 1.7152, 'learning_rate': 2.84846183061147e-06, 'epoch': 0.99}

 99%|█████████▉| 2709/2733 [10:06:00<05:24, 13.53s/it]
 99%|█████████▉| 2710/2733 [10:06:13<05:09, 13.46s/it]
                                                      
{'loss': 1.6776, 'learning_rate': 2.734523357387011e-06, 'epoch': 0.99}

 99%|█████████▉| 2710/2733 [10:06:13<05:09, 13.46s/it]
 99%|█████████▉| 2711/2733 [10:06:26<04:53, 13.32s/it]
                                                      
{'loss': 1.6353, 'learning_rate': 2.620584884162552e-06, 'epoch': 0.99}

 99%|█████████▉| 2711/2733 [10:06:26<04:53, 13.32s/it]
 99%|█████████▉| 2712/2733 [10:06:39<04:39, 13.31s/it]
                                                      
{'loss': 1.6697, 'learning_rate': 2.506646410938093e-06, 'epoch': 0.99}

 99%|█████████▉| 2712/2733 [10:06:39<04:39, 13.31s/it]
 99%|█████████▉| 2713/2733 [10:06:52<04:22, 13.14s/it]
                                                      
{'loss': 1.5108, 'learning_rate': 2.3927079377136344e-06, 'epoch': 0.99}

 99%|█████████▉| 2713/2733 [10:06:52<04:22, 13.14s/it]
 99%|█████████▉| 2714/2733 [10:07:06<04:12, 13.28s/it]
                                                      
{'loss': 1.6593, 'learning_rate': 2.278769464489176e-06, 'epoch': 0.99}

 99%|█████████▉| 2714/2733 [10:07:06<04:12, 13.28s/it]
 99%|█████████▉| 2715/2733 [10:07:18<03:57, 13.19s/it]
                                                      
{'loss': 1.6599, 'learning_rate': 2.1648309912647167e-06, 'epoch': 0.99}

 99%|█████████▉| 2715/2733 [10:07:19<03:57, 13.19s/it]
 99%|█████████▉| 2716/2733 [10:07:31<03:42, 13.11s/it]
                                                      
{'loss': 1.6134, 'learning_rate': 2.050892518040258e-06, 'epoch': 0.99}

 99%|█████████▉| 2716/2733 [10:07:31<03:42, 13.11s/it]
 99%|█████████▉| 2717/2733 [10:07:45<03:31, 13.25s/it]
                                                      
{'loss': 1.6773, 'learning_rate': 1.936954044815799e-06, 'epoch': 0.99}

 99%|█████████▉| 2717/2733 [10:07:45<03:31, 13.25s/it]
 99%|█████████▉| 2718/2733 [10:07:58<03:17, 13.15s/it]
                                                      
{'loss': 1.6763, 'learning_rate': 1.8230155715913404e-06, 'epoch': 0.99}

 99%|█████████▉| 2718/2733 [10:07:58<03:17, 13.15s/it]
 99%|█████████▉| 2719/2733 [10:08:11<03:04, 13.16s/it]
                                                      
{'loss': 1.608, 'learning_rate': 1.7090770983668817e-06, 'epoch': 0.99}

 99%|█████████▉| 2719/2733 [10:08:11<03:04, 13.16s/it]
100%|█████████▉| 2720/2733 [10:08:24<02:49, 13.06s/it]
                                                      
{'loss': 1.6651, 'learning_rate': 1.595138625142423e-06, 'epoch': 1.0}

100%|█████████▉| 2720/2733 [10:08:24<02:49, 13.06s/it]/nfs4/llm/h2o-llm/env/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:318: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization
  warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")

100%|█████████▉| 2721/2733 [10:08:37<02:36, 13.07s/it]
                                                      
{'loss': 1.7718, 'learning_rate': 1.481200151917964e-06, 'epoch': 1.0}

100%|█████████▉| 2721/2733 [10:08:37<02:36, 13.07s/it]
100%|█████████▉| 2722/2733 [10:08:50<02:24, 13.15s/it]
                                                      
{'loss': 1.6481, 'learning_rate': 1.3672616786935054e-06, 'epoch': 1.0}

100%|█████████▉| 2722/2733 [10:08:50<02:24, 13.15s/it]
100%|█████████▉| 2723/2733 [10:09:03<02:10, 13.06s/it]
                                                      
{'loss': 1.6142, 'learning_rate': 1.2533232054690465e-06, 'epoch': 1.0}

100%|█████████▉| 2723/2733 [10:09:03<02:10, 13.06s/it]
100%|█████████▉| 2724/2733 [10:09:16<01:56, 12.96s/it]
                                                      
{'loss': 1.6483, 'learning_rate': 1.139384732244588e-06, 'epoch': 1.0}

100%|█████████▉| 2724/2733 [10:09:16<01:56, 12.96s/it]
100%|█████████▉| 2725/2733 [10:09:29<01:44, 13.09s/it]
                                                      
{'loss': 1.6411, 'learning_rate': 1.025446259020129e-06, 'epoch': 1.0}

100%|█████████▉| 2725/2733 [10:09:29<01:44, 13.09s/it]
100%|█████████▉| 2726/2733 [10:09:42<01:30, 12.89s/it]
                                                      
{'loss': 1.6259, 'learning_rate': 9.115077857956702e-07, 'epoch': 1.0}

100%|█████████▉| 2726/2733 [10:09:42<01:30, 12.89s/it]
100%|█████████▉| 2727/2733 [10:09:56<01:19, 13.27s/it]
                                                      
{'loss': 1.6417, 'learning_rate': 7.975693125712116e-07, 'epoch': 1.0}

100%|█████████▉| 2727/2733 [10:09:56<01:19, 13.27s/it]
100%|█████████▉| 2728/2733 [10:10:09<01:05, 13.10s/it]
                                                      
{'loss': 1.5317, 'learning_rate': 6.836308393467527e-07, 'epoch': 1.0}

100%|█████████▉| 2728/2733 [10:10:09<01:05, 13.10s/it]
100%|█████████▉| 2729/2733 [10:10:22<00:52, 13.12s/it]
                                                      
{'loss': 1.6016, 'learning_rate': 5.69692366122294e-07, 'epoch': 1.0}

100%|█████████▉| 2729/2733 [10:10:22<00:52, 13.12s/it]
100%|█████████▉| 2730/2733 [10:10:35<00:39, 13.24s/it]
                                                      
{'loss': 1.6943, 'learning_rate': 4.557538928978351e-07, 'epoch': 1.0}

100%|█████████▉| 2730/2733 [10:10:35<00:39, 13.24s/it]
100%|█████████▉| 2731/2733 [10:10:49<00:26, 13.34s/it]
                                                      
{'loss': 1.6896, 'learning_rate': 3.4181541967337635e-07, 'epoch': 1.0}

100%|█████████▉| 2731/2733 [10:10:49<00:26, 13.34s/it]
100%|█████████▉| 2732/2733 [10:11:02<00:13, 13.29s/it]
                                                      
{'loss': 1.6741, 'learning_rate': 2.2787694644891755e-07, 'epoch': 1.0}

100%|█████████▉| 2732/2733 [10:11:02<00:13, 13.29s/it]
100%|██████████| 2733/2733 [10:11:15<00:00, 13.09s/it]
                                                      
{'loss': 1.6662, 'learning_rate': 1.1393847322445877e-07, 'epoch': 1.0}

100%|██████████| 2733/2733 [10:11:15<00:00, 13.09s/it]
                                                      
{'train_runtime': 36693.5282, 'train_samples_per_second': 9.534, 'train_steps_per_second': 0.074, 'train_loss': 1.705727982119767, 'epoch': 1.0}

100%|██████████| 2733/2733 [10:11:15<00:00, 13.09s/it]
100%|██████████| 2733/2733 [10:11:15<00:00, 13.42s/it]

 If there's a warning about missing keys above, please disregard :)
