How Do I Install DeepSeek R1 on Amazon Bedrock?

Question

Hi team, How do I install DeepSeek R1 on Amazon Bedrock? Thanks!  —Jurgen

Greeting

Hi Jurgen, Great question! Let’s walk through the full process of deploying the DeepSeek R1 model on Amazon Bedrock, starting from setting up your environment to running inference. This guide will ensure a smooth installation and deployment. 😊

Clarifying the Issue

You want to install DeepSeek R1 on Amazon Bedrock, ensuring all steps—such as environment setup, model preparation, and integration—are handled correctly. This will allow you to take advantage of Bedrock's powerful serverless infrastructure. 🚀

Why It Matters

Amazon Bedrock simplifies deploying AI models, providing a serverless, scalable, and easy-to-use framework for inference. Successfully deploying DeepSeek R1 ensures you can use it effectively for your AI applications without infrastructure management overhead.

Key Terms

• DeepSeek R1: A fine-tuned AI model compatible with architectures like Llama 2.
• Amazon Bedrock: A serverless service for deploying and running foundational models.
• S3 Bucket: An AWS storage solution where model files are uploaded for deployment.
• Custom Model Import: A Bedrock feature enabling external models to be imported into its infrastructure.

Steps at a Glance

• Set up your Python environment.
• Download DeepSeek R1 from Hugging Face.
• Upload the model to Amazon S3.
• Import the model into Bedrock.
• Run inference using the Bedrock Runtime API.

Detailed Steps

1. Set Up Your Python Environment

Install the required dependencies to interact with Hugging Face and AWS services:

pip install huggingface_hub boto3

This ensures you can download the model and manage S3 uploads seamlessly.

2. Download the DeepSeek R1 Model

Use the Hugging Face Hub to download the desired variant of the DeepSeek R1 model. Here’s an example to download the 8B Distilled Model:

Python

from huggingface_hub import snapshot_download

model_id = "deepseek-ai/DeepSeek-R1-Distill-Llama-8B"
local_dir = snapshot_download(
    repo_id=model_id, 
    local_dir="DeepSeek-R1-Distill-Llama-8B"
)

This will download the model weights (.safetensors), configuration files, and tokenizers into a local directory.

3. Upload the Model to Amazon S3

Transfer the downloaded files to an S3 bucket in a Bedrock-supported region (e.g., us-east-1):

Python

import boto3
import os

s3_client = boto3.client('s3', region_name='us-east-1')
bucket_name = 'your-s3-bucket-name'
local_directory = 'DeepSeek-R1-Distill-Llama-8B'

# Upload all model files to S3
for root, dirs, files in os.walk(local_directory):
    for file in files:
        local_path = os.path.join(root, file)
        s3_key = os.path.relpath(local_path, local_directory)
        s3_client.upload_file(local_path, bucket_name, s3_key)

Ensure your S3 bucket is in the same region as your Bedrock deployment.

4. Import the Model into Bedrock

Now, use the Bedrock console to import your model:

• Navigate to Custom Models in the Bedrock dashboard.
• Select Import Model and enter the S3 URI for the model (e.g., s3://your-s3-bucket-name/DeepSeek-R1-Distill-Llama-8B/).
• Follow the prompts to complete the import process.

After importing, Bedrock will generate a Model ARN, which you’ll need for inference.

5. Run Inference with the Bedrock Runtime API

Once the model is deployed, use the Bedrock Runtime API to invoke it. Here’s an example:

Python

import boto3
import json

# Initialize the Bedrock runtime client
client = boto3.client('bedrock-runtime', region_name='us-east-1')

# Your model's ARN
model_id = 'arn:aws:bedrock:us-east-1:your-account-id:imported-model/your-model-id'

# Example inference call
def invoke_model(prompt):
    response = client.invoke_model(
        modelId=model_id,
        body=json.dumps({'prompt': prompt}),
        accept='application/json',
        contentType='application/json'
    )

    return json.loads(response['body'].read().decode('utf-8'))

# Example usage
result = invoke_model("Explain quantum computing in simple terms.")
print(result)

This code initializes the Bedrock runtime client, invokes the model using your prompt, and returns the result.

Closing Thoughts

By following these steps, you’ll have successfully deployed DeepSeek R1 on Amazon Bedrock and be ready to run inference seamlessly. For more details on related tools and processes, check out these resources:

• Hugging Face Hub: DeepSeek R1 Models
• Amazon Bedrock: Custom Model Import Documentation
• Boto3 Documentation for S3
• Boto3 Documentation for Bedrock Runtime

If you encounter any challenges, feel free to reach out. Good luck with your deployment, Jurgen! 🚀😊

Need AWS Expertise?

If you're looking for guidance on AWS challenges or want to collaborate, feel free to reach out! We'd love to help you tackle your cloud projects. 🚀

Email us at: info@pacificw.com

Source: DEV - Deploying DeepSeek R1 Model on Amazon Bedrock: A Comprehensive Guide

Image: Gemini