# ExitCode 1 ErrorMessage "KeyError: 'Image' when using entry\_point script in Huggingface Estimator **URL:** \ **Category:** Amazon SageMaker\ **Created:** [September 30, 2022, 9:20am UTC](https://discuss.huggingface.co/t/exitcode-1-errormessage-keyerror-image-when-using-entry-point-script-in-huggingface-estimator/23821 "2022-09-30T09:20:10Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![ShuaHousetable](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@ShuaHousetable](https://discuss.huggingface.co/u/ShuaHousetable)\ **Post date:** [September 30, 2022, 9:20am UTC](https://discuss.huggingface.co/t/exitcode-1-errormessage-keyerror-image-when-using-entry-point-script-in-huggingface-estimator/23821/1 "2022-09-30T09:20:10Z")
Trying to train and deploy a Vision Transformer model in Sagemaker using huggingface. I have used tried the Huggingface example `[notebook on the matter](https://github.com/huggingface/notebooks/tree/main/sagemaker/09_image_classification_vision_transformer)` and still getting the same error: KeyError ‘Image’: The output is saying that the error is in the load\_from\_disk function, but it doesn’t make sense to me: With the same path, when performing load\_from\_disk outside of the training script (meaning, in the notebook I am trying to invoke the training script from), the dataset loads fine. This is the full output: 2022-09-30 09:08:29 Training - Downloading the training image…bash: cannot set terminal process group (-1): Inappropriate ioctl for device bash: no job control in this shell 2022-09-30 09:11:42,795 sagemaker-training-toolkit INFO Imported framework sagemaker\_pytorch\_container.training 2022-09-30 09:11:42,830 sagemaker\_pytorch\_container.training INFO Block until all host DNS lookups succeed. 2022-09-30 09:11:42,837 sagemaker\_pytorch\_container.training INFO Invoking user training script. 2022-09-30 09:11:43,425 sagemaker-training-toolkit INFO Invoking user script Training Env: { “additional\_framework\_parameters”: {}, “channel\_input\_dirs”: { “test”: “/opt/ml/input/data/test”, “train”: “/opt/ml/input/data/train” }, “current\_host”: “algo-1”, “framework\_module”: “sagemaker\_pytorch\_container.training:main”, “hosts”: [ “algo-1” ], “hyperparameters”: { “model\_name”: “google/vit-base-patch16-224-in21k”, “num\_train\_epochs”: 1, “per\_device\_train\_batch\_size”: 16 }, “input\_config\_dir”: “/opt/ml/input/config”, “input\_data\_config”: { “test”: { “TrainingInputMode”: “File”, “S3DistributionType”: “FullyReplicated”, “RecordWrapperType”: “None” }, “train”: { “TrainingInputMode”: “File”, “S3DistributionType”: “FullyReplicated”, “RecordWrapperType”: “None” } }, “input\_dir”: “/opt/ml/input”, “is\_master”: true, “job\_name”: “huggingface-pytorch-training-2022-09-30-09-04-04-385”, “log\_level”: 20, “master\_hostname”: “algo-1”, “model\_dir”: “/opt/ml/model”, “module\_dir”: “s3://sagemaker-eu-central-1-232977032562/huggingface-pytorch-training-2022-09-30-09-04-04-385/source/sourcedir.tar.gz”, “module\_name”: “train”, “network\_interface\_name”: “eth0”, “num\_cpus”: 8, “num\_gpus”: 1, “output\_data\_dir”: “/opt/ml/output/data”, “output\_dir”: “/opt/ml/output”, “output\_intermediate\_dir”: “/opt/ml/output/intermediate”, “resource\_config”: { “current\_host”: “algo-1”, “current\_instance\_type”: “ml.p3.2xlarge”, “current\_group\_name”: “homogeneousCluster”, “hosts”: [ “algo-1” ], “instance\_groups”: [ { “instance\_group\_name”: “homogeneousCluster”, “instance\_type”: “ml.p3.2xlarge”, “hosts”: [ “algo-1” ] } ], “network\_interface\_name”: “eth0” }, “user\_entry\_point”: “train.py” } Environment variables: SM\_HOSTS=[“algo-1”] SM\_NETWORK\_INTERFACE\_NAME=eth0 SM\_HPS={“model\_name”:“google/vit-base-patch16-224-in21k”,“num\_train\_epochs”:1,“per\_device\_train\_batch\_size”:16} SM\_USER\_ENTRY\_POINT=train.py SM\_FRAMEWORK\_PARAMS={} SM\_RESOURCE\_CONFIG={“current\_group\_name”:“homogeneousCluster”,“current\_host”:“algo-1”,“current\_instance\_type”:“ml.p3.2xlarge”,“hosts”:[“algo-1”],“instance\_groups”:[{“hosts”:[“algo-1”],“instance\_group\_name”:“homogeneousCluster”,“instance\_type”:“ml.p3.2xlarge”}],“network\_interface\_name”:“eth0”} SM\_INPUT\_DATA\_CONFIG={“test”:{“RecordWrapperType”:“None”,“S3DistributionType”:“FullyReplicated”,“TrainingInputMode”:“File”},“train”:{“RecordWrapperType”:“None”,“S3DistributionType”:“FullyReplicated”,“TrainingInputMode”:“File”}} SM\_OUTPUT\_DATA\_DIR=/opt/ml/output/data SM\_CHANNELS=[“test”,“train”] SM\_CURRENT\_HOST=algo-1 SM\_MODULE\_NAME=train SM\_LOG\_LEVEL=20 SM\_FRAMEWORK\_MODULE=sagemaker\_pytorch\_container.training:main SM\_INPUT\_DIR=/opt/ml/input SM\_INPUT\_CONFIG\_DIR=/opt/ml/input/config SM\_OUTPUT\_DIR=/opt/ml/output SM\_NUM\_CPUS=8 SM\_NUM\_GPUS=1 SM\_MODEL\_DIR=/opt/ml/model SM\_MODULE\_DIR=s3://sagemaker-eu-central-1-232977032562/huggingface-pytorch-training-2022-09-30-09-04-04-385/source/sourcedir.tar.gz SM\_TRAINING\_ENV={“additional\_framework\_parameters”:{},“channel\_input\_dirs”:{“test”:“/opt/ml/input/data/test”,“train”:“/opt/ml/input/data/train”},“current\_host”:“algo-1”,“framework\_module”:“sagemaker\_pytorch\_container.training:main”,“hosts”:[“algo-1”],“hyperparameters”:{“model\_name”:“google/vit-base-patch16-224-in21k”,“num\_train\_epochs”:1,“per\_device\_train\_batch\_size”:16},“input\_config\_dir”:“/opt/ml/input/config”,“input\_data\_config”:{“test”:{“RecordWrapperType”:“None”,“S3DistributionType”:“FullyReplicated”,“TrainingInputMode”:“File”},“train”:{“RecordWrapperType”:“None”,“S3DistributionType”:“FullyReplicated”,“TrainingInputMode”:“File”}},“input\_dir”:“/opt/ml/input”,“is\_master”:true,“job\_name”:“huggingface-pytorch-training-2022-09-30-09-04-04-385”,“log\_level”:20,“master\_hostname”:“algo-1”,“model\_dir”:“/opt/ml/model”,“module\_dir”:“s3://sagemaker-eu-central-1-232977032562/huggingface-pytorch-training-2022-09-30-09-04-04-385/source/sourcedir.tar.gz”,“module\_name”:“train”,“network\_interface\_name”:“eth0”,“num\_cpus”:8,“num\_gpus”:1,“output\_data\_dir”:“/opt/ml/output/data”,“output\_dir”:“/opt/ml/output”,“output\_intermediate\_dir”:“/opt/ml/output/intermediate”,“resource\_config”:{“current\_group\_name”:“homogeneousCluster”,“current\_host”:“algo-1”,“current\_instance\_type”:“ml.p3.2xlarge”,“hosts”:[“algo-1”],“instance\_groups”:[{“hosts”:[“algo-1”],“instance\_group\_name”:“homogeneousCluster”,“instance\_type”:“ml.p3.2xlarge”}],“network\_interface\_name”:“eth0”},“user\_entry\_point”:“train.py”} SM\_USER\_ARGS=[“–model\_name”,“google/vit-base-patch16-224-in21k”,“–num\_train\_epochs”,“1”,“–per\_device\_train\_batch\_size”,“16”] SM\_OUTPUT\_INTERMEDIATE\_DIR=/opt/ml/output/intermediate SM\_CHANNEL\_TEST=/opt/ml/input/data/test SM\_CHANNEL\_TRAIN=/opt/ml/input/data/train SM\_HP\_MODEL\_NAME=google/vit-base-patch16-224-in21k SM\_HP\_NUM\_TRAIN\_EPOCHS=1 SM\_HP\_PER\_DEVICE\_TRAIN\_BATCH\_SIZE=16 PYTHONPATH=/opt/ml/code:/opt/conda/bin:/opt/conda/lib/python38.zip:/opt/conda/lib/python3.8:/opt/conda/lib/python3.8/lib-dynload:/opt/conda/lib/python3.8/site-packages Invoking script with the following command: /opt/conda/bin/python train.py --model\_name google/vit-base-patch16-224-in21k --num\_train\_epochs 1 --per\_device\_train\_batch\_size 16 Traceback (most recent call last): File “train.py”, line 61, in train\_dataset = load\_from\_disk(args.training\_dir) File “/opt/conda/lib/python3.8/site-packages/datasets/load.py”, line 1685, in load\_from\_disk return Dataset.load\_from\_disk(dataset\_path, fs, keep\_in\_memory=keep\_in\_memory) File “/opt/conda/lib/python3.8/site-packages/datasets/arrow\_dataset.py”, line 1057, in load\_from\_disk dataset\_info = DatasetInfo.from\_dict(json.load(dataset\_info\_file)) File “/opt/conda/lib/python3.8/site-packages/datasets/info.py”, line 261, in from\_dict return cls(\*\*{k: v for k, v in dataset\_info\_dict.items() if k in field\_names}) File “”, line 20, in **init** File “/opt/conda/lib/python3.8/site-packages/datasets/info.py”, line 144, in **post\_init** self.features = Features.from\_dict(self.features) File “/opt/conda/lib/python3.8/site-packages/datasets/features/features.py”, line 1019, in from\_dict obj = generate\_from\_dict(dic) File “/opt/conda/lib/python3.8/site-packages/datasets/features/features.py”, line 876, in generate\_from\_dict return {key: generate\_from\_dict(value) for key, value in obj.items()} File “/opt/conda/lib/python3.8/site-packages/datasets/features/features.py”, line 876, in return {key: generate\_from\_dict(value) for key, value in obj.items()} File “/opt/conda/lib/python3.8/site-packages/datasets/features/features.py”, line 877, in generate\_from\_dict class\_type = globals()[obj.pop(“\_type”)] KeyError: ‘Image’ 2022-09-30 09:11:48,904 sagemaker-training-toolkit INFO Waiting for the process to finish and give a return code. 2022-09-30 09:11:48,904 sagemaker-training-toolkit INFO Done waiting for a return code. Received 1 from exiting process. 2022-09-30 09:11:48,904 sagemaker-training-toolkit ERROR Reporting training FAILURE 2022-09-30 09:11:48,904 sagemaker-training-toolkit ERROR ExecuteUserScriptError: ExitCode 1 ErrorMessage "KeyError: ‘Image’ " Command “/opt/conda/bin/python train.py --model\_name google/vit-base-patch16-224-in21k --num\_train\_epochs 1 --per\_device\_train\_batch\_size 16” 2022-09-30 09:11:48,904 sagemaker-training-toolkit ERROR Encountered exit\_code 1 ---
**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\ **Post date:** [October 8, 2022, 11:35am UTC](https://discuss.huggingface.co/t/exitcode-1-errormessage-keyerror-image-when-using-entry-point-script-in-huggingface-estimator/23821/2 "2022-10-08T11:35:50Z")
> [@ShuaHousetable](#): > > With the same path, when performing load\_from\_disk outside of the training script (meaning, in the notebook I am trying to invoke the training script from), the dataset loads fine. Hi there It’s worth noting that being able to load the dataset in the notebook instance does NOT mean you can successfully load it with the training script. The reason for that is that the training script runs in a seperate EC2 instance that has no knowledge of your notebook instance. This is by design: You want a small, cheap notebook instance to orchestrate the data prep and training setup but you (potentially) want a powerful, expensive instance to run the actual training on. To learn more about training HF models on SageMaker, have a look at this example: [https://github.com/huggingface/notebooks/tree/main/sagemaker/01\_getting\_started\_pytorch](https://github.com/huggingface/notebooks/tree/main/sagemaker/01_getting_started_pytorch) What does that mean for your particular case? Without seeing the notebook where you orchestrate the setup I can only guess, but it looks like you have either (a) not stored the dataset in the correct S3 bucket or (b) have not told the training job the correct S3 path for the training job. Again, check out @philschmid’s example notebook, that should give you an idea how to pass on dataset paths to the training job. Cheers Heiko