# Applying \`.map\` results in getting \`List\` type on \`input\_values\`

**URL:** <https://discuss.huggingface.co/t/applying-map-results-in-getting-list-type-on-input-values/61600>\
**Category:** 🤗Datasets\
**Created:** [November 9, 2023, 5:19am UTC](https://discuss.huggingface.co/t/applying-map-results-in-getting-list-type-on-input-values/61600 "2023-11-09T05:19:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ifedorov](https://avatars.discourse-cdn.com/v4/letter/i/ac8455/32.png) [@ifedorov](https://discuss.huggingface.co/u/ifedorov)\
**Post date:** [November 9, 2023, 5:19am UTC](https://discuss.huggingface.co/t/applying-map-results-in-getting-list-type-on-input-values/61600/1 "2023-11-09T05:19:34Z")

</div>

Hi, I have audio dataset. Using `.map` method, I apply a function that reads the audios from the disk, resamples them and applies `Wav2Vec2FeatureExtractor`, which normalizes the audio and converts it to torch tensor.

```auto
def preprocess_function(samples):
    speech_list = [speech_file_to_array_fn(path) for path in samples[input_column]]
    target_list = [label_to_id(label, label_list) for label in samples[output_column]]

    result = processor(speech_list, sampling_rate=target_sampling_rate, return_tensors='pt')
    result['labels'] = list(target_list)
    return result

eval_dataset = eval_dataset.map(
    preprocess_function,
    num_proc=1,
    batched=True,
    batch_size=1
)

```

The `result` variable in the preprocess function contains a dict with pytorch tensors as values. But when I index the dataset after the transformation, I get `List` type of `input_values`. Is it possible to not convert the values to `List` and keep them as `torch.tensor`?

---

<div class="post-metadata">

**Author:** ![panigrah](https://avatars.discourse-cdn.com/v4/letter/p/bb73d2/32.png) [@panigrah](https://discuss.huggingface.co/u/panigrah)\
**Post date:** [November 9, 2023, 6:02am UTC](https://discuss.huggingface.co/t/applying-map-results-in-getting-list-type-on-input-values/61600/2 "2023-11-09T06:02:42Z")

</div>

see here… dataset returns pure python objects.

> <https://github.com/huggingface/datasets/issues/1046>
>
> I apply \`Dataset.map()\` to a function that returns a dict of torch tensors (like… a tokenizer from the repo transformers). However, in the mapped dataset, these tensors have turned to lists!
> 
> \`\`\`import datasets
> import torch  
> from datasets import load\_dataset                                                                                                                 
> print("version datasets", datasets.\_\_version\_\_)
> 
> dataset = load\_dataset("snli", split='train\[0:50\]')  
> 
> def tokenizer\_fn(example):
> # actually uses a tokenizer which does something like:
> return {'input\_ids': torch.tensor(\[\[0, 1, 2\]\])}
> 
> print("First item in dataset:\\n", dataset\[0\])
> tokenized = tokenizer\_fn(dataset\[0\])
> print("Tokenized hyp:\\n", tokenized)
> dataset\_tok = dataset.map(tokenizer\_fn, batched=False,
> remove\_columns=\['label', 'premise', 'hypothesis'\])
> print("Tokenized using map:\\n", dataset\_tok\[0\])
> print(type(tokenized\['input\_ids'\]), type(dataset\_tok\[0\]\['input\_ids'\]))
> dataset\_tok = dataset.map(tokenizer\_fn, batched=False,
> remove\_columns=\['label', 'premise', 'hypothesis'\])
> print("Tokenized using map:\\n", dataset\_tok\[0\])
> print(type(tokenized\['input\_ids'\]), type(dataset\_tok\[0\]\['input\_ids'\]))
> \`\`\`
> 
> The output is:
> 
> \`\`\`
> version datasets 1.1.3
> Reusing dataset snli (/home/tom/.cache/huggingface/datasets/snli/plain\_text/1.0.0/bb1102591c6230bd78813e229d5dd4c7fbf4fc478cec28f298761eb69e5b537c)
> First item in dataset:
> {'premise': 'A person on a horse jumps over a broken down airplane.', 'hypothesis': 'A person is training his horse for a competition.', 'label': 1}
> Tokenized hyp:
> {'input\_ids': tensor(\[\[0, 1, 2\]\])}
> Loading cached processed dataset at /home/tom/.cache/huggingface/datasets/snli/plain\_text/1.0.0/bb1102591c6230bd78813e229d5dd4c7fbf4fc478cec28f298761eb69e5b537c/cache-fe38f449fe9ac46f.arrow
> Tokenized using map:
> {'input\_ids': \[\[0, 1, 2\]\]}
> \<class 'torch.Tensor'\> \<class 'list'\>
> \`\`\`
> 
> Or am I doing something wrong?

here is one possible approach but it has other side effects.

`eval_dataset = eval_dataset.with_format('tf')`