# Set\_seed is not working **URL:** \ **Category:** Beginners\ **Created:** [September 8, 2024, 11:36am UTC](https://discuss.huggingface.co/t/set-seed-is-not-working/105846 "2024-09-08T11:36:23Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![nasos10](https://avatars.discourse-cdn.com/v4/letter/n/f0a364/32.png) [@nasos10](https://discuss.huggingface.co/u/nasos10)\ **Post date:** [September 8, 2024, 11:36am UTC](https://discuss.huggingface.co/t/set-seed-is-not-working/105846/1 "2024-09-08T11:36:23Z")
Okay, it’s my first post here and I am not a really experienced programmer so forgive me if I am asking something trivial :)) . I am working on finetuning LLM’s on a downstream task, and I use Accelerate library to work on 2 GPUs (DDP setting). I can’t share the code rn, but my setup looks like the tutorial in [run\_native\_loop](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_mlm_no_trainer.py#L311) I am trying to set\_seed with the transformers method, to have reproducible experiments, and I am altering the seed to have 3 different runs. The problem is that, the data are loaded in batches with the same way (meaning that the first batch is always the same, in both GPUs. After shuffling, in epoch 2 i get shuffled training samples but still I get the same ones, even when altering the seed. This is something i cannot explain. How is this even possible? The paradox is that setting the seed works fine for things like adding new randomly initialized rows in the embedding layer (different seed=different initialization), but the data keeps loading in the same order. Is this the way it is supposed to be? It seems really strange to me. Maybe accelerator creates the problem? or is it the way Dataloaders are wrapped for the DDP scenario. - To clarify, data is indeed shuffled before a new epoch. What drives me crazy is that it is still shuffled in the same way, I alter the seed and i get the same samples order in epoch 2. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [September 8, 2024, 12:11pm UTC](https://discuss.huggingface.co/t/set-seed-is-not-working/105846/2 "2024-09-08T12:11:04Z")
I am not familiar with LLM, but maybe this is it? > > > \### System Info > > \- \`transformers\` version: 4.28.0.dev0 > \- Platform: Linux-5.19.0…-41-generic-x86\_64-with-glibc2.35 > \- Python version: 3.10.10 > \- Huggingface\_hub version: 0.13.3 > \- Safetensors version: not installed > \- PyTorch version (GPU?): 2.0.0 (True) > \- Tensorflow version (GPU?): not installed (NA) > \- Flax version (CPU?/GPU?/TPU?): not installed (NA) > \- Jax version: not installed > \- JaxLib version: not installed > \- Using GPU in script?: Yes > \- Using distributed or parallel set-up in script?: No > > \### Who can help? > > @gante, @younesbelkada > > \### Information > > \- \[\] The official example scripts > \- \[X\] My own modified scripts > > \### Tasks > > \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...) > \- \[X\] My own task or dataset (give details below) > > \### Reproduction > > 1. install transformes with support for alpaca model > 2. run this code with one seed > 3. run this code with any other seed > 4. see that the results are the same > \`\`\` > from transformers import GenerationConfig, LlamaTokenizer, LlamaForCausalLM, set\_seed > from torch import float16, compile, no\_grad > > > set\_seed(621) > > \# Enhances prompt > def enhance\_prompt(prompt, input=None): > if input: > return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. > > \### Instruction: > {prompt} > > \### Input: > {input} > > \### Response:""" > else: > return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request. > > \### Instruction: > {prompt} > > \### Response:""" > > \# Gets response from Alpaca > def get\_response(prompt): > with no\_grad(): > outputs = alpaca.generate(input\_ids=tokenizer(prompt, return\_tensors="pt").input\_ids.to("cuda"), generation\_config=generation\_config, return\_dict\_in\_generate=True, output\_scores=True) > outputs = tokenizer.decode(outputs.sequences\[0\], skip\_special\_tokens=True) > return outputs.split("### Response:")\[1\] > > > \# Sets up Alpaca > tokenizer = LlamaTokenizer.from\_pretrained("chainyo/alpaca-lora-7b") > alpaca = LlamaForCausalLM.from\_pretrained("chainyo/alpaca-lora-7b", load\_in\_8bit=True, torch\_dtype=float16, device\_map="auto") > generation\_config = GenerationConfig(temperature=0.2, top\_p=0.75, top\_k=40, num\_beams=4, max\_new\_tokens=64) > alpaca.eval() > compile(alpaca) > > \# Gets output from Alpaca > prompt = enhance\_prompt("Write a simple poem about flowers.") > out = get\_response(prompt) > > \# Prints Alpaca's output > print(out) > \`\`\` > > \### Expected behavior > > Model will output two different answers but now it gives the same every seed I try. > Two notes: > > 1. You should pass `do_sample=True` in your generation config or in your `.generate()` call. Most models have it off by default, causing the generation to be deterministic (and ignoring parameters like `temperature`, `top_k`, etc). > 2. With `temperature=0.2`, the relative weight of the most likely logits is massively increased, making generation almost deterministic. Even if there are no bugs in your script, it’s far from guaranteed that two different seeds produce different outputs with such low temperature 🙂 ---
**Author:** ![nasos10](https://avatars.discourse-cdn.com/v4/letter/n/f0a364/32.png) [@nasos10](https://discuss.huggingface.co/u/nasos10)\ **Post date:** [September 8, 2024, 12:22pm UTC](https://discuss.huggingface.co/t/set-seed-is-not-working/105846/3 "2024-09-08T12:22:51Z")
Well, thank you for your response but I don’t think this is the case. My code is only for training, while the abovementioned post was about the generate function (used at inference). The dataloader is the problem with my code, but i can’t see why.