How to Deploy DeepSeek R1 on Amazon SageMaker

Question

"Hi, I’m Jurgen. I’ve been hearing great things about DeepSeek-R1, and I want to deploy it on AWS SageMaker. I’m not sure how to set it up or get it running efficiently. Can you guide me through the process?"

Greeting

Hello, Jurgen! It’s fantastic that you’re diving into the world of DeepSeek-R1 and exploring its potential. Let’s walk you through deploying this groundbreaking AI model on AWS SageMaker so you can get it up and running without any hiccups.

Clarifying the Issue

Jurgen’s question highlights a common challenge for developers exploring large language models (LLMs) like DeepSeek-R1. Deploying such models on AWS SageMaker involves navigating prerequisites, configurations, and deployment processes, which can feel overwhelming. Our goal is to simplify this journey, ensuring you can confidently deploy and use DeepSeek-R1 on SageMaker for your projects.

Clarifying the Issue

Jurgen’s question highlights a common challenge for developers exploring large language models (LLMs) like DeepSeek-R1. Deploying such models on AWS SageMaker involves navigating prerequisites, configurations, and deployment processes, which can feel overwhelming. Our goal is to simplify this journey, ensuring you can confidently deploy and use DeepSeek-R1 on SageMaker for your projects.

Why It Matters

DeepSeek-R1 is a game-changer in the LLM landscape, offering OpenAI-level performance at a fraction of the cost. By deploying it on AWS SageMaker, you can scale its capabilities to meet your project’s demands, whether you’re working in research, business intelligence, or development.

Key Terms

• DeepSeek-R1: An open-source LLM optimized for reasoning and generative tasks.
• Amazon SageMaker: A managed service for building, training, and deploying machine learning models.
• Endpoint: A resource in SageMaker enabling real-time model interactions.
• Hugging Face Model: A pre-trained model architecture integrated with SageMaker.

Steps at a Glance

1. Set up your SageMaker domain and user profile.
2. Launch SageMaker Studio and configure JupyterLab.
3. Deploy the DeepSeek-R1 model to a GPU-optimized instance using Hugging Face integration.
4. Test the model with a sample inference request.

Detailed Steps

1. Set Up SageMaker Domain and User Profile

   Start by setting up your SageMaker domain in the AWS Management Console. Navigate to the SageMaker section, create a domain, and configure a user profile. Once complete, navigate to User Profiles and launch SageMaker Studio.

2. Launch SageMaker Studio

   Inside SageMaker Studio, open JupyterLab by clicking the + button in the Launcher tab. Create a new Python 3 notebook to prepare your deployment environment.

3. Deploy DeepSeek-R1

   Copy and paste the following code into your notebook to initialize a SageMaker session, configure the Hugging Face model, and deploy it as an endpoint:

   Python

   from sagemaker.huggingface import HuggingFaceModel
   import sagemaker
   
   # Initialize session
   session = sagemaker.Session()
   
   # Model configuration
   model_config = {
       "HF_MODEL_ID": "deepseek-ai/DeepSeek-R1",
       "HF_TASK": "text-generation",
       "SM_NUM_GPUS": "8"
   }
   
   # Create Hugging Face Model
   huggingface_model = HuggingFaceModel(
       env=model_config,
       role=sagemaker.get_execution_role(),
       transformers_version="4.37",
       pytorch_version="2.1",
       py_version="py310",
       name="deepseek-r1-sagemaker"
   )
   
   # Deploy the model
   predictor = huggingface_model.deploy(
       initial_instance_count=1,
       instance_type="ml.g4dn.xlarge",
       endpoint_name="deepseek-r1-endpoint"
   )
   

4. Test the Model

   After deployment, test your endpoint with the following inference request:

   Python

   # Define generation parameters
   generation_params = {
       "do_sample": True,
       "top_p": 0.9,
       "temperature": 0.7,
       "max_new_tokens": 512
   }
   
   # Make a prediction
   response = predictor.predict({
       "inputs": "Explain quantum computing in simple terms:",
       "parameters": generation_params
   })
   
   print(response[0]['generated_text'])
   

Special Thanks

We’d like to extend a heartfelt thanks to the AWS engineers, Germaine Ong and Jarrett Yeo, for their detailed two-part blog series that served as the foundation for this guidance. Their expertise has been invaluable in making cloud-based AI accessible to developers worldwide.

Need AWS Expertise?

If you're looking for guidance on AWS challenges or want to collaborate, feel free to reach out! We'd love to help you tackle your cloud projects. 🚀

Email us at: info@pacificw.com

Image: Gemini

Sources:

Deploying DeepSeek-R1 on Amazon SageMaker Part 1

Deploying DeepSeek-R1 on Amazon SageMaker Part 2