# Huggingface Seq2SeqTrainer uses accelerate so it cannot be run with DDP?

**URL:** <https://discuss.huggingface.co/t/huggingface-seq2seqtrainer-uses-accelerate-so-it-cannot-be-run-with-ddp/68172>\
**Category:** 🤗Accelerate\
**Created:** [January 6, 2024, 3:54pm UTC](https://discuss.huggingface.co/t/huggingface-seq2seqtrainer-uses-accelerate-so-it-cannot-be-run-with-ddp/68172 "2024-01-06T15:54:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![liveseongho](https://avatars.discourse-cdn.com/v4/letter/l/2bfe46/32.png) [@liveseongho](https://discuss.huggingface.co/u/liveseongho)\
**Post date:** [January 6, 2024, 3:54pm UTC](https://discuss.huggingface.co/t/huggingface-seq2seqtrainer-uses-accelerate-so-it-cannot-be-run-with-ddp/68172/1 "2024-01-06T15:54:50Z")

</div>

Literally. Huggingface Seq2SeqTrainer uses accelerate so it cannot be run with DDP?  
As I know in [here](https://github.com/huggingface/accelerate/issues/1368), Accelerate is not compatible with DDP. DDP shows n\_gpu=1 for every process (even if I use multiple GPUs), but accelerate requires multiple devices…

I got following error message.

```auto
ValueError: DistributedDataParallel device_ids and output_device arguments only work with single-device/multiple-device GPU modules or CPU modules, but got device_ids [0], output_device 0, and module parameters {device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)}.

```

How can I avoid to use accelerate in Huggingface Seq2SeqTrainer, as I must use DDP.

---

<div class="post-metadata">

**Author:** ![marcsun13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcsun13/32/19059_2.png) [@marcsun13](https://discuss.huggingface.co/u/marcsun13)\
**Post date:** [January 24, 2024, 4:30pm UTC](https://discuss.huggingface.co/t/huggingface-seq2seqtrainer-uses-accelerate-so-it-cannot-be-run-with-ddp/68172/2 "2024-01-24T16:30:00Z")

</div>

Accelerate is totally compatible with DDP. Can you provide a reproducer ? You are probably using it wrong. See more info [here](https://huggingface.co/docs/transformers/trainer#accelerate-and-trainer).