17 lines
2.2 KiB
Plaintext
17 lines
2.2 KiB
Plaintext
setting up the environment by loading in conda environment at Sat Oct 11 07:52:03 CDT 2025
|
|
running the batched olmo categorization job at Sat Oct 11 07:52:03 CDT 2025
|
|
[nltk_data] Downloading package punkt_tab to
|
|
[nltk_data] /home/nws8519/nltk_data...
|
|
[nltk_data] Package punkt_tab is already up-to-date!
|
|
cuda
|
|
NVIDIA A100-SXM4-80GB
|
|
_CudaDeviceProperties(name='NVIDIA A100-SXM4-80GB', major=8, minor=0, total_memory=81153MB, multi_processor_count=108, uuid=ee0bd2a7-af54-5f2e-c2d3-fcd3f57270c9, L2_cache_size=40MB)
|
|
|
|
Loading checkpoint shards: 0%| | 0/12 [00:00<?, ?it/s]
|
|
Loading checkpoint shards: 8%|▊ | 1/12 [00:00<00:04, 2.72it/s]
|
|
Loading checkpoint shards: 17%|█▋ | 2/12 [00:00<00:04, 2.27it/s]
|
|
Loading checkpoint shards: 25%|██▌ | 3/12 [00:01<00:04, 2.12it/s]
|
|
Loading checkpoint shards: 33%|███▎ | 4/12 [00:01<00:03, 2.12it/s]
|
|
Loading checkpoint shards: 42%|████▏ | 5/12 [00:02<00:03, 1.96it/s]
|
|
Loading checkpoint shards: 50%|█████ | 6/12 [00:02<00:03, 1.98it/s]
|
|
Loading checkpoint shards: 58%|█████▊ | 7/12 [00:03<00:02, 1.87it/s]
|
|
Loading checkpoint shards: 67%|██████▋ | 8/12 [00:03<00:02, 1.94it/s]
|
|
Loading checkpoint shards: 75%|███████▌ | 9/12 [00:04<00:01, 1.91it/s]
|
|
Loading checkpoint shards: 83%|████████▎ | 10/12 [00:05<00:01, 1.87it/s]
|
|
Loading checkpoint shards: 92%|█████████▏| 11/12 [00:05<00:00, 2.05it/s]
|
|
Loading checkpoint shards: 100%|██████████| 12/12 [00:05<00:00, 2.18it/s]
|
|
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
|
|
This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (4096). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
|
|
Traceback (most recent call last):
|
|
File "/home/nws8519/git/mw-lifecycle-analysis/p2/quest/python_scripts/090425_batched_olmo_cat.py", line 66, in <module>
|
|
for row in reader:
|
|
_csv.Error: field larger than field limit (131072)
|
|
unsupervised batched olmo categorization pau at Sun Oct 12 14:11:43 CDT 2025
|