mgaughan/mw-lifecycle-analysis
1
0
Fork 0
Code

olmo batched for getting the title in there too, i think

Browse Source
This commit is contained in:
mgaughan 2025-09-22 19:18:11 -05:00
parent e2413ed955
commit bcfa688e11
2 changed files with 151690 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

151688
all_091625_olmo_batched_categorized.csv Normal file
View File

File diff suppressed because one or more lines are too long

2
batched-mw-olmo-info-cat.log
View File

@ -8,3 +8,5 @@ NVIDIA A100-SXM4-80GB
_CudaDeviceProperties(name='NVIDIA A100-SXM4-80GB', major=8, minor=0, total_memory=81153MB, multi_processor_count=108, uuid=b6c5753c-65f3-91cd-dd90-e56a02d2cf99, L2_cache_size=40MB)
Loading checkpoint shards: 0%| | 0/12 [00:00<?, ?it/s] Loading checkpoint shards: 8%|▊ | 1/12 [00:00<00:05, 2.11it/s] Loading checkpoint shards: 17%|█▋ | 2/12 [00:00<00:05, 1.99it/s] Loading checkpoint shards: 25%|██▌ | 3/12 [00:01<00:05, 1.79it/s] Loading checkpoint shards: 33%|███▎ | 4/12 [00:02<00:05, 1.54it/s] Loading checkpoint shards: 42%|████▏ | 5/12 [00:02<00:04, 1.63it/s] Loading checkpoint shards: 50%|█████ | 6/12 [00:03<00:04, 1.49it/s] Loading checkpoint shards: 58%|█████▊ | 7/12 [00:04<00:03, 1.50it/s] Loading checkpoint shards: 67%|██████▋ | 8/12 [00:05<00:02, 1.54it/s] Loading checkpoint shards: 75%|███████▌ | 9/12 [00:05<00:01, 1.63it/s] Loading checkpoint shards: 83%|████████▎ | 10/12 [00:05<00:01, 1.78it/s] Loading checkpoint shards: 92%|█████████▏| 11/12 [00:06<00:00, 1.79it/s] Loading checkpoint shards: 100%|██████████| 12/12 [00:06<00:00, 1.82it/s]
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (4096). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
unsupervised batched olmo categorization pau at Thu Sep 18 03:22:25 CDT 2025