# Error on fsdp2 trainer with fsdp\_cpu\_ram\_efficient\_loading = True

**URL:** <https://discuss.huggingface.co/t/error-on-fsdp2-trainer-with-fsdp-cpu-ram-efficient-loading-true/171753>\
**Category:** 🤗Accelerate\
**Created:** [December 19, 2025, 4:29am UTC](https://discuss.huggingface.co/t/error-on-fsdp2-trainer-with-fsdp-cpu-ram-efficient-loading-true/171753 "2025-12-19T04:29:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![enmixkesi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/enmixkesi/32/55516_2.png) [@enmixkesi](https://discuss.huggingface.co/u/enmixkesi)\
**Post date:** [December 19, 2025, 4:29am UTC](https://discuss.huggingface.co/t/error-on-fsdp2-trainer-with-fsdp-cpu-ram-efficient-loading-true/171753/1 "2025-12-19T04:29:06Z")

</div>

Hi all, I had a quick question. There’s an error when I try to train my model using fsdp2 with the setting: fsdp\_cpu\_ram\_efficient\_loading = True. It will raise error:

[rank0]: Traceback (most recent call last):  
[rank0]: File “trainpy”, line 259, in  
[rank0]: invoke\_main()  
[rank0]: File “train.py”, line 256, in invoke\_main  
[rank0]: main(config)  
[rank0]: File “train.py”, line 99, in main  
[rank0]: model, optimizer, train\_loader, valid\_loader, scheduler = accelerator.prepare(  
[rank0]: ^^^^^^^^^^^^^^^^^^^^  
[rank0]: File “/data0/miniconda3/envs/esat/lib/python3.12/site-packages/accelerate/accelerator.py”, line 1555, in prepare  
[rank0]: result = self.\_prepare\_fsdp2(\*args)  
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^  
[rank0]: File “/data0/miniconda3/envs/esat/lib/python3.12/site-packages/accelerate/accelerator.py”, line 1687, in \_prepare\_fsdp2  
[rank0]: model = fsdp2\_prepare\_model(self, model)  
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
[rank0]: File “/data0/miniconda3/envs/esat/lib/python3.12/site-packages/accelerate/utils/fsdp\_utils.py”, line 678, in fsdp2\_prepare\_model  
[rank0]: fsdp2\_load\_full\_state\_dict(accelerator, model, original\_sd)  
[rank0]: File “/data0/miniconda3/envs/esat/lib/python3.12/site-packages/accelerate/utils/fsdp\_utils.py”, line 507, in fsdp2\_load\_full\_state\_dict  
[rank0]: device\_mesh = sharded\_param.device\_mesh  
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^  
[rank0]: AttributeError: ‘Tensor’ object has no attribute ‘device\_mesh’

and my accelerate configs:

distributed\_type: FSDP  
downcast\_bf16: ‘no’  
dynamo\_backend: ‘INDUCTOR’ # ‘INDUCTOR’ ‘NO’  
fsdp\_config:  
fsdp\_version: 2  
fsdp\_reshard\_after\_forward: True  
fsdp\_mixed\_precision\_policy:  
param\_dtype: “bf16”  
reduce\_dtype: “bf16”  
buffer\_dtype: “fp32”  
fsdp\_auto\_wrap\_policy: TRANSFORMER\_BASED\_WRAP  
fsdp\_transformer\_layer\_cls\_to\_wrap: Block  
fsdp\_offload\_params: False  
fsdp\_state\_dict\_type: SHARDED\_STATE\_DICT  
fsdp\_cpu\_ram\_efficient\_loading: False  
fsdp\_activation\_checkpointing: False  
mixed\_precision: bf16  
num\_processes: 2  
gpu\_ids: 5,6  
use\_cpu: False

I noticed that fsdp\_cpu\_ram\_efficient\_loading can prevent CPU memory overflow (OOM), which is helpful when i want to scale up my model, what’s more, fsdp\_cpu\_ram\_efficient\_loading=True is the default setting. However, this feature appears unavailable. How can I resolve this issue?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [December 19, 2025, 6:00am UTC](https://discuss.huggingface.co/t/error-on-fsdp2-trainer-with-fsdp-cpu-ram-efficient-loading-true/171753/2 "2025-12-19T06:00:07Z")

</div>

Seems known bug?

* * *

This failure is coming from the same root class of bugs that others hit in **Accelerate FSDP2 “CPU RAM efficient loading”** : Accelerate’s FSDP2 loading path assumes it can read **DTensor metadata (DeviceMesh)** from “sharded params”, but in some configurations it instead sees a plain **Tensor** , so accessing `.device_mesh` throws exactly what you see.

This is not you mis-typing YAML. It is a real integration edge.

* * *

## What `fsdp_cpu_ram_efficient_loading` actually does

Accelerate documents this flag very narrowly:

- It is **only applicable to Transformers models**.
- It is meant for **loading via `from_pretrained`**.
- When enabled, **only rank 0 loads the pretrained checkpoint** , other ranks start with “empty” weights, then **weights are broadcast/synced** to other ranks. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))
- Accelerate explicitly says to **set it to False if you get errors** while loading pretrained Transformers weights. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))

So this flag is not a general FSDP2 “reduce CPU RAM always” switch. It is specifically a Transformers `from_pretrained` loading optimization.

* * *

## Why you get `Tensor has no attribute device_mesh`

### Background: FSDP2 is DTensor-based

In PyTorch FSDP2 (`fully_shard`), parameters are converted **in-place** from plain tensors to **DTensors** , and DTensors carry a **DeviceMesh** that describes the topology of ranks and how tensors are sharded/replicated. ([PyTorch Documentation](https://docs.pytorch.org/docs/stable/distributed.fsdp.fully_shard.html "torch.distributed.fsdp.fully\_shard"))

That is why “DeviceMesh” exists at all: DTensor layouts are defined by `(DeviceMesh, Placement)`. ([PyTorch Documentation](https://docs.pytorch.org/docs/stable/distributed.tensor.html "torch.distributed.tensor"))

### What Accelerate is doing in the failing path

Accelerate’s FSDP2 utility `fsdp2_load_full_state_dict` is documented as:

- “Loads the full state dict (could be only on rank 0) into the sharded model”
- “done by broadcasting the parameters from rank 0”
- expects the model to be on **meta** device to avoid VRAM spikes. ([Hugging Face](https://huggingface.co/docs/accelerate/main/en/package_reference/fsdp "Fully Sharded Data Parallel utilities"))

To broadcast correctly, it tries to infer sharding metadata from the sharded parameters, and that’s where `.device_mesh` is accessed. Your crash means the “sharded param” it looked at was not a DTensor (no `.device_mesh`), it was a plain Tensor.

### This is a known, recurring incompatibility

There are recent Accelerate issues showing the same family of breakage when `cpu_ram_efficient_loading=True` triggers broadcast-based loading under FSDP2, especially when DTensor/DeviceMesh context is not what Accelerate expects. Example: an Oct 2025 issue shows `cpu_ram_efficient_loading=True` leading into `fsdp2_load_full_state_dict` and then failing inside `dist.broadcast` because DTensor dispatch cannot find a DeviceMesh context. ([GitHub](https://github.com/huggingface/accelerate/issues/3820 "AssertionError: found no DeviceMesh from dtensor args for c10d.broadcast\_.default · Issue #3820 · huggingface/accelerate · GitHub"))

Another related issue (Aug 2025) notes the error is tied to `cpu_ram_efficient_loading` and that setting it to **False** makes training run, with maintainers stating they were working on making it compatible (in that report, with TP). ([GitHub](https://github.com/huggingface/accelerate/issues/3729 "FSDP V2 + TP give DeviceMesh Error · Issue #3729"))

* * *

## Practical resolution options

### Option A (most reliable): disable it for now

Set:

```yaml
fsdp_cpu_ram_efficient_loading: False

```

This is explicitly recommended in the docs when it causes loading errors. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))  
And it matches real-world maintainers’ guidance in similar reports. ([GitHub](https://github.com/huggingface/accelerate/issues/3729 "FSDP V2 + TP give DeviceMesh Error · Issue #3729"))

If you only need a working run today, this is the fastest path.

* * *

### Option B: keep it, but satisfy the real prerequisites

If you want to keep `True`, make sure you are actually using it in the narrow supported way:

1. **You must be loading a Transformers model via `from_pretrained`** (not constructing weights yourself, not loading custom state dicts first). ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))

2. **Distributed must be initialized before `from_pretrained`.**  
Accelerate’s doc explicitly calls this out. In Trainer, this usually happens when `TrainingArguments` is created; in a custom loop, it means create the `Accelerator` (or otherwise init the process group) before calling `from_pretrained`. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))

3. Ensure `fsdp_sync_module_states=True` (or let Accelerate force it).  
Docs: it “must be True” when CPU RAM efficient loading is True. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))

If you do all three and still hit the crash, you are in the “known bug” territory, not the “misuse” territory.

* * *

### Option C: get the CPU-RAM benefit without this brittle path

If your core goal is “avoid CPU OOM when scaling model size,” you can often avoid the broadcast-based full-state-dict load entirely:

#### 1) Prefer SHARDED checkpoint formats for resume and large-scale

You already set:

```yaml
fsdp_state_dict_type: SHARDED_STATE_DICT

```

That is good. The remaining pain point is usually the **initial pretrained load** , which is still a full checkpoint.

A common workaround is a one-time **conversion step** : load the pretrained model on a machine with enough CPU RAM once, then save out a sharded checkpoint, and from then on resume from shards.

Accelerate even provides utilities around sharded weights (and warns merging is CPU-bound). ([Hugging Face](https://huggingface.co/docs/accelerate/main/en/package_reference/fsdp "Fully Sharded Data Parallel utilities"))

#### 2) Use PyTorch Distributed Checkpointing (DCP) when possible

PyTorch DCP is designed to load/save in a distributed way and requires sharding info before loading. That is the same scaling problem you are fighting (avoid “gather everything on one rank” patterns). ([PyTorch Tutorials KR](https://tutorials.pytorch.kr/recipes/distributed_checkpoint_recipe.html "Getting Started with Distributed Checkpoint (DCP)"))

* * *

## Strong “do this next” checklist

1. Print versions:

```bash
python -c "import torch,accelerate,transformers; print(torch. __version__ , accelerate. __version__ , transformers. __version__ )"

```

FSDP2 is fast-moving and Accelerate has had multiple FSDP2 fixes recently. ([GitHub](https://github.com/huggingface/accelerate/releases "Releases · huggingface/accelerate · GitHub"))

1. If you need it to work now: set `fsdp_cpu_ram_efficient_loading=False`. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))

2. If you need the memory win: plan to **convert pretrained weights once** into a sharded checkpoint or move to **DCP-based checkpointing** to avoid full-state broadcast style loading. ([PyTorch Tutorials KR](https://tutorials.pytorch.kr/recipes/distributed_checkpoint_recipe.html "Getting Started with Distributed Checkpoint (DCP)"))

* * *

## Key references and “similar cases”

- Accelerate doc for `fsdp_cpu_ram_efficient_loading` requirements and “disable if errors.” ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))
- Accelerate FSDP2 loading utility description (broadcast from rank 0; expects meta-device model). ([Hugging Face](https://huggingface.co/docs/accelerate/main/en/package_reference/fsdp "Fully Sharded Data Parallel utilities"))
- Similar recent failures: DeviceMesh/DTensor broadcast path errors with `cpu_ram_efficient_loading=True`. ([GitHub](https://github.com/huggingface/accelerate/issues/3820 "AssertionError: found no DeviceMesh from dtensor args for c10d.broadcast\_.default · Issue #3820 · huggingface/accelerate · GitHub"))
- Another similar issue: setting `cpu_ram_efficient_loading=False` as workaround. ([GitHub](https://github.com/huggingface/accelerate/issues/3729 "FSDP V2 + TP give DeviceMesh Error · Issue #3729"))
- PyTorch FSDP2 `fully_shard` converts params to DTensor (why DeviceMesh is expected). ([PyTorch Documentation](https://docs.pytorch.org/docs/stable/distributed.fsdp.fully_shard.html "torch.distributed.fsdp.fully\_shard"))

* * *

### Summary

- Your crash is a DTensor/DeviceMesh mismatch in Accelerate’s FSDP2 “RAM efficient loading” path. ([Hugging Face](https://huggingface.co/docs/accelerate/main/en/package_reference/fsdp "Fully Sharded Data Parallel utilities"))
- The flag is Transformers `from_pretrained`–specific, not a universal knob. ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))
- Most reliable fix: set it to **False** (docs and similar issues endorse this). ([Hugging Face](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp "Fully Sharded Data Parallel"))
- To still avoid CPU OOM: move to **sharded checkpoints** or **PyTorch DCP** flows. ([PyTorch Tutorials KR](https://tutorials.pytorch.kr/recipes/distributed_checkpoint_recipe.html "Getting Started with Distributed Checkpoint (DCP)"))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [December 19, 2025, 6:00pm UTC](https://discuss.huggingface.co/t/error-on-fsdp2-trainer-with-fsdp-cpu-ram-efficient-loading-true/171753/3 "2025-12-19T18:00:28Z")

</div>

Closed on December 19, 2025, 6:00pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.