LocalEntryNotFoundError when loading downloaded pretrained model using timm.create_model (side load offline weights, e.g. on Kaggle)

[eesoymilk]

Describe the bug

I'm trying to load swin transformer in a kaggle's offline environment. While it ran without issue yesterday, it now fails to load with a LocalEntryNotFoundError. I believe this is because the recent update of kaggle's docker image, which uses a newer version of timm and therefore breaks the code.

To Reproduce

1. Download the pretrain model from SwinTransformer/storage and upload it to kaggle's dataset. In my case, I use swinv2_large_patch4_window12to16_192to256_22kto1k_ft.pth, but I don't think any of them work as well.

2. Minimal code to reproduce the bug

   import os
   from timm import create_model
   
   if not os.path.exists('/root/.cache/torch/hub/checkpoints/'):
       os.makedirs('/root/.cache/torch/hub/checkpoints/')
   !cp '../input/swin-transformer-classification-models/swinv2_large_patch4_window12to16_192to256_22kto1k_ft.pth' '/root/.cache/torch/hub/checkpoints/swinv2_large_patch4_window12to16_192to256_22kto1k_ft.pth'
   
   model = create_model(
       'swinv2_large_window12to16_192to256_22kft1k', 
       pretrained=True
   )

Expected behavior
This should load swinv2_large_patch4_window12to16_192to256_22kto1k_ft.pth from /root/.cache/torch/hub/checkpoints/.

Error message

---------------------------------------------------------------------------
LocalEntryNotFoundError                   Traceback (most recent call last)
Cell In[9], line 1
----> 1 model = create_model(
      2     'swinv2_large_window12to16_192to256_22kft1k', pretrained=True)

File /opt/conda/lib/python3.10/site-packages/timm/models/_factory.py:114, in create_model(model_name, pretrained, pretrained_cfg, pretrained_cfg_overlay, checkpoint_path, scriptable, exportable, no_jit, **kwargs)
    112 create_fn = model_entrypoint(model_name)
    113 with set_layer_config(scriptable=scriptable, exportable=exportable, no_jit=no_jit):
--> 114     model = create_fn(
    115         pretrained=pretrained,
    116         pretrained_cfg=pretrained_cfg,
    117         pretrained_cfg_overlay=pretrained_cfg_overlay,
    118         **kwargs,
    119     )
    121 if checkpoint_path:
    122     load_checkpoint(model, checkpoint_path)

File /opt/conda/lib/python3.10/site-packages/timm/models/_registry.py:139, in _deprecated_model_shim.<locals>._fn(pretrained, **kwargs)
    137 warnings.warn(f'Mapping deprecated model name {deprecated_name} to current {current_name}.', stacklevel=2)
    138 pretrained_cfg = kwargs.pop('pretrained_cfg', None)
--> 139 return current_fn(pretrained=pretrained, pretrained_cfg=pretrained_cfg or current_tag, **kwargs)

File /opt/conda/lib/python3.10/site-packages/timm/models/swin_transformer_v2.py:814, in swinv2_large_window12to16_192to256(pretrained, **kwargs)
    809 """
    810 """
    811 model_args = dict(
    812     window_size=16, embed_dim=192, depths=(2, 2, 18, 2), num_heads=(6, 12, 24, 48),
    813     pretrained_window_sizes=(12, 12, 12, 6))
--> 814 return _create_swin_transformer_v2(
    815     'swinv2_large_window12to16_192to256', pretrained=pretrained, **dict(model_args, **kwargs))

File /opt/conda/lib/python3.10/site-packages/timm/models/swin_transformer_v2.py:636, in _create_swin_transformer_v2(variant, pretrained, **kwargs)
    633 default_out_indices = tuple(i for i, _ in enumerate(kwargs.get('depths', (1, 1, 1, 1))))
    634 out_indices = kwargs.pop('out_indices', default_out_indices)
--> 636 model = build_model_with_cfg(
    637     SwinTransformerV2, variant, pretrained,
    638     pretrained_filter_fn=checkpoint_filter_fn,
    639     feature_cfg=dict(flatten_sequential=True, out_indices=out_indices),
    640     **kwargs)
    641 return model

File /opt/conda/lib/python3.10/site-packages/timm/models/_builder.py:393, in build_model_with_cfg(model_cls, variant, pretrained, pretrained_cfg, pretrained_cfg_overlay, model_cfg, feature_cfg, pretrained_strict, pretrained_filter_fn, kwargs_filter, **kwargs)
    391 num_classes_pretrained = 0 if features else getattr(model, 'num_classes', kwargs.get('num_classes', 1000))
    392 if pretrained:
--> 393     load_pretrained(
    394         model,
    395         pretrained_cfg=pretrained_cfg,
    396         num_classes=num_classes_pretrained,
    397         in_chans=kwargs.get('in_chans', 3),
    398         filter_fn=pretrained_filter_fn,
    399         strict=pretrained_strict,
    400     )
    402 # Wrap the model in a feature extraction module if enabled
    403 if features:

File /opt/conda/lib/python3.10/site-packages/timm/models/_builder.py:186, in load_pretrained(model, pretrained_cfg, num_classes, in_chans, filter_fn, strict)
    184         state_dict = load_state_dict_from_hf(*pretrained_loc)
    185     else:
--> 186         state_dict = load_state_dict_from_hf(pretrained_loc)
    187 else:
    188     model_name = pretrained_cfg.get('architecture', 'this model')

File /opt/conda/lib/python3.10/site-packages/timm/models/_hub.py:188, in load_state_dict_from_hf(model_id, filename)
    185             pass
    187 # Otherwise, load using pytorch.load
--> 188 cached_file = hf_hub_download(hf_model_id, filename=filename, revision=hf_revision)
    189 _logger.debug(f"[{model_id}] Safe alternative not found for '{filename}'. Loading weights using default pytorch.")
    190 return torch.load(cached_file, map_location='cpu')

File /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:120, in validate_hf_hub_args.<locals>._inner_fn(*args, **kwargs)
    117 if check_use_auth_token:
    118     kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__name__, has_token=has_token, kwargs=kwargs)
--> 120 return fn(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1291, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, local_dir_use_symlinks, user_agent, force_download, force_filename, proxies, etag_timeout, resume_download, token, local_files_only, legacy_cache_layout)
   1285         raise LocalEntryNotFoundError(
   1286             "Cannot find the requested files in the disk cache and"
   1287             " outgoing traffic has been disabled. To enable hf.co look-ups"
   1288             " and downloads online, set 'local_files_only' to False."
   1289         )
   1290     else:
-> 1291         raise LocalEntryNotFoundError(
   1292             "Connection error, and we cannot find the requested files in"
   1293             " the disk cache. Please try again or make sure your Internet"
   1294             " connection is on."
   1295         )
   1297 # From now on, etag and commit_hash are not None.
   1298 assert etag is not None, "etag must have been retrieved from server"

LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the disk cache. Please try again or make sure your Internet connection is on.

[rwightman]

@eesoymilk latest timm versions (0.9.x) use the huggingface hub for weights by default now, that takes priority over the torch hub cache, many weights have been remapped for model changes so best download via HF hub. Kaggle really should support passing through HF hub or at least caching it properly but they seem to have no interest in making things simpler for people so the manual caching of weights in datasets madness continues...

To override pretrained location, you download the weights file manually from the hub and try the following.

timm.create_model(
  'swinv2_large_window12to16_192to256',
  pretrained=True,
  pretrained_cfg_overlay=dict(file='path/to/checkpoint'),
)

where checkpoint is a .safetensors/.bin/.pth/.pt etc file...

[eesoymilk]

Thanks for the reply. The code is working right now!
One thing to add is that I think this can be added somewhere in the docs of timm since this is really confusing a ton of people.

[rwightman]

yes, this is definitely something I should document better, noted

[Tianyahechu]

KeyError: 'embedding/kernel is not a file in the archive'

I encountered this issue when I was loading the pretrained weights downloaded from huggingface. timm==1.0.11
code like below:
timm.create_model(model_name='vit_large_patch16_384',
pretrained=True,
pretrained_cfg_overlay=dict(file='vit_large_patch16_384.augreg_in21k_ft_in1k/pytorch_model.bin'),
# num_classes=2
)
how to fix it? Thanks

[rwightman]

@Tianyahechu hmm, not the most obvious, but the overlay needs to also have custom_load=False in the overlay as the original weights for that model were .npz and used the custom load mechanism.

[Tianyahechu]

@rwightman Thanks for your response. It worked successfully after I added custom_load=False to pretrained_cfg_overlay

[rwightman]

moving to discussions for future reference

[kostiantynbrandbrigade]

@rwightman can you pls point me what is the default path where loaded models are stored in 0.9.x versions?

[K-D-Gallagher]

@kostiantynbrandbrigade I'm using timm 0.9 and found them here:
~/.cache/huggingface/hub