setting up the environment by loading in conda environment at Fri Sep  5 13:42:49 CDT 2025
running the batched olmo categorization job at Fri Sep  5 13:42:50 CDT 2025
[nltk_data] Downloading package punkt_tab to
[nltk_data]     /home/nws8519/nltk_data...
[nltk_data]   Package punkt_tab is already up-to-date!
cuda
NVIDIA A100-SXM4-80GB
_CudaDeviceProperties(name='NVIDIA A100-SXM4-80GB', major=8, minor=0, total_memory=81153MB, multi_processor_count=108, uuid=83b2afae-0102-1408-8043-b11b77d85fc8, L2_cache_size=40MB)

Loading checkpoint shards:   0%|          | 0/12 [00:00<?, ?it/s]
Loading checkpoint shards:   8%|▊         | 1/12 [00:00<00:05,  2.02it/s]
Loading checkpoint shards:  17%|█▋        | 2/12 [00:01<00:05,  1.91it/s]
Loading checkpoint shards:  25%|██▌       | 3/12 [00:01<00:04,  1.82it/s]
Loading checkpoint shards:  33%|███▎      | 4/12 [00:02<00:04,  1.84it/s]
Loading checkpoint shards:  42%|████▏     | 5/12 [00:02<00:04,  1.71it/s]
Loading checkpoint shards:  50%|█████     | 6/12 [00:03<00:03,  1.60it/s]
Loading checkpoint shards:  58%|█████▊    | 7/12 [00:03<00:02,  1.77it/s]
Loading checkpoint shards:  67%|██████▋   | 8/12 [00:04<00:02,  1.78it/s]
Loading checkpoint shards:  75%|███████▌  | 9/12 [00:05<00:01,  1.80it/s]
Loading checkpoint shards:  83%|████████▎ | 10/12 [00:05<00:01,  1.80it/s]
Loading checkpoint shards:  92%|█████████▏| 11/12 [00:05<00:00,  1.98it/s]
Loading checkpoint shards: 100%|██████████| 12/12 [00:06<00:00,  1.99it/s]
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (4096). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
unsupervised batched olmo categorization pau at Sun Sep  7 05:04:39 CDT 2025