How to pass a Public Docker Image to the Estimater

[kaushikb11]

Motivation

I would like to pass a public Docker Hub Image through the image_uri parameter to the Estimator(Pytorch)

[ChoiByungWook]

Hey @kaushikb11,

I believe SageMaker doesn't support pubic docker images currently and only through ECR. Most likely what you will have to do is pull the public docker image and push it to a ECR repository in your AWS account.

For more information see: https://sagemaker-workshop.com/custom/containers.html

[kaushikb11]

Got it! @ChoiByungWook

Is there a way to run smdistributed data-parallel code without the Estimator for debugging?
As it is only available when you run it with the Estimator.

[akinolawilson]

@kaushikb11

I am facing the same problem.

There is little to no documentation on how exactly this module smdistributed is handled by the Sagemaker training API.

I am currently attempting to use the model parallelism feature of the API and now, following the creation of a custom container based on the Sagemaker PyTorch training toolkit, I receive an error informing that the library; smdistributed is not found.

If there was a way to debug a training script, prior to submitting it to the estimator, it would be very helpful.