Keeping several AI models on one SSD can mean storing the same component more than once. Hugging Face’s October 26 tooling recap describes a change to that arithmetic: its Hub cache can now share identical Xet files across repositories, so a second model that needs an already-cached file gets a link instead of another download.
The company’s example is the 9.5 GB text encoder shared by FLUX.1-dev, FLUX.1-schnell and FLUX.1-Krea-dev. That is a useful illustration of the opportunity, not a promise that every collection of models will shrink by the same amount. The saving depends on repositories requesting the same file, and on the local cache being able to use the shared layout.
This extends a distinction already familiar inside the Hub cache. Different revisions of one repository can point to the same stored blob when a file has not changed. The newer shared store takes reuse across repository boundaries for Xet files. According to the current cache guide, the physical file lives in a top-level blobs directory, while each repository’s blob entry links to it. The snapshot layout used to locate a model’s files stays the same.
For someone comparing models, that means a shared component need not occupy another full copy merely because the surrounding model has a different repository name. It also means that adding up the sizes displayed beside individual repositories can give the wrong answer to a different question: how much space the whole cache physically occupies.
One file can appear in several repository totals
Hugging Face’s documentation distinguishes logical repository sizes from cache-wide physical storage. Each repository’s logical total counts the shared file it uses. The cache-wide size_on_disk counts that physical file once, including shared files no repository currently uses. Those figures describe different things, so a discrepancy between their totals is not, by itself, evidence that the cache is broken.
Cleanup has to respect the same relationship. Removing one model must not remove a shared component another model still needs. The implementation keeps a references manifest beside each shared file, but the guide says that manifest is only a hint: entries are checked against the filesystem, and files with missing or unreadable metadata are retained. The design favors leaving reclaimable data behind over breaking another valid cache entry.
The recap also introduces removing a single cached file, such as one GGUF quantization, rather than deleting its entire repository. Separately, hf cache prune removes shared files that no cached repository uses. These serve different purposes. A user can discard a particular model variant while preserving other variants, then reclaim shared data that has become unreferenced. SekaiTech has not run those operations or measured the resulting disk savings.
There are compatibility boundaries behind the announcement’s automatic behavior. The shared store requires the symlink-based cache layout and is disabled when HF_HUB_DISABLE_XET=1. Unsupported filesystems, permission errors or a pre-existing unmarked blobs directory can trigger a fallback to ordinary repository-local storage. On Windows, the cache guide separately explains that symlink support can require Developer Mode or an appropriately privileged Python process. The headline feature therefore does not establish that a particular Windows installation is already deduplicating across models.
Older readers can continue using the familiar repository folders, but deletion tools deserve more care. The guide says huggingface_hub<=1.31 can leave a deleted repository’s shared files behind; other clients’ deletion tools may instead delete shared data another repository still uses, causing a later download. It recommends an up-to-date huggingface_hub for cache removal, pruning and programmatic deletion. This is a maintenance compatibility issue, distinct from whether an older application can load an existing snapshot.
The change complements the local-model choices in SekaiTech’s coverage of Ollama speech checkpoints, but it belongs to the Hugging Face Hub cache, not every application’s model directory. A model’s checkpoint license and its storage mechanism also remain separate questions. Sharing bytes locally does not change permission to use the model.
For a model collection, the useful inventory now has two views: which files each repository needs, and how much physical storage the cache consumes. Hugging Face’s recap supplies the release context; its cache guide explains why those views can differ. The practical benefit is less duplicated storage when identical Xet files overlap, without requiring the model loader to abandon its existing snapshot paths.