AWS Bedrock

AWS Bedrock is Amazon's managed AI service, providing foundation models from multiple providers.

Supported Models

Anthropic Claude

• anthropic.claude-3-opus-20240229-v1:0 - Claude 3 Opus
• anthropic.claude-3-sonnet-20240229-v1:0 - Claude 3 Sonnet
• anthropic.claude-3-haiku-20240307-v1:0 - Claude 3 Haiku

Meta Llama

• meta.llama3-8b-instruct-v1:0 - LLaMA 3 8B
• meta.llama3-70b-instruct-v1:0 - LLaMA 3 70B
• meta.llama2-13b-chat-v1 - LLaMA 2 13B
• meta.llama2-70b-chat-v1 - LLaMA 2 70B

Mistral AI

• mistral.mistral-7b-instruct-v0:2 - Mistral 7B
• mistral.mixtral-8x7b-instruct-v0:1 - Mixtral 8x7B

AI21 Labs

• ai21.j2-ultra-v1 - Jurassic-2 Ultra
• ai21.j2-mid-v1 - Jurassic-2 Mid

Cohere

• cohere.command-text-v14 - Command Text
• cohere.command-light-text-v14 - Command Light

Configuration

Basic Configuration

Configure in config.yaml or ~/.bytebuddy/config.yaml:

models:
  - name: "bedrock-claude"
    provider: "bedrock"
    model: "anthropic.claude-3-sonnet-20240229-v1:0"
    roles: ["chat", "edit"]
    env:
      region: "us-east-1"
      accessKeyId: "${AWS_ACCESS_KEY_ID}"
      secretAccessKey: "${AWS_SECRET_ACCESS_KEY}"
    defaultCompletionOptions:
      temperature: 0.7
      maxTokens: 4000

Using IAM Role

Use IAM role in EC2 or Lambda (no credentials needed):

models:
  - name: "bedrock-iam"
    provider: "bedrock"
    model: "anthropic.claude-3-sonnet-20240229-v1:0"
    roles: ["chat"]
    env:
      region: "us-east-1"

Using AWS Profile

models:
  - name: "bedrock-profile"
    provider: "bedrock"
    model: "anthropic.claude-3-sonnet-20240229-v1:0"
    roles: ["chat"]
    env:
      region: "us-east-1"
      profile: "my-aws-profile"

Multi-Model Configuration

models:
  - name: "claude-sonnet"
    provider: "bedrock"
    model: "anthropic.claude-3-sonnet-20240229-v1:0"
    roles: ["chat", "edit"]
    env:
      region: "us-west-2"
      accessKeyId: "${AWS_ACCESS_KEY_ID}"
      secretAccessKey: "${AWS_SECRET_ACCESS_KEY}"
    defaultCompletionOptions:
      temperature: 0.7
      maxTokens: 4000

  - name: "llama3-70b"
    provider: "bedrock"
    model: "meta.llama3-70b-instruct-v1:0"
    roles: ["chat"]
    env:
      region: "us-west-2"
      profile: "bedrock-profile"
    defaultCompletionOptions:
      temperature: 0.5
      maxTokens: 4000

Configuration Fields

Required Fields

• name: Unique identifier for the model configuration
• provider: Set to "bedrock"
• model: Bedrock model ID

Environment Configuration (env)

• region: AWS region (required)
• accessKeyId: AWS access key ID (optional, not needed with IAM role)
• secretAccessKey: AWS secret access key (optional)
• profile: AWS profile name (optional)
• sessionToken: AWS session token (optional, for temporary credentials)

Optional Fields

• roles: Model roles [chat, edit, apply]
• defaultCompletionOptions:
  • temperature: Control randomness (0-1)
  • maxTokens: Maximum tokens
  • topP: Nucleus sampling parameter

Environment Variables

# ~/.bashrc or ~/.zshrc
export AWS_ACCESS_KEY_ID="your-access-key-id"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"
export AWS_SESSION_TOKEN="your-session-token"  # Optional

Setup Steps

1. Enable Bedrock Access

1. Log in to AWS Management Console
2. Navigate to Bedrock service
3. Request access to required models

2. Configure IAM Permissions

Ensure your AWS identity has the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "arn:aws:bedrock:*:*:model/*"
    }
  ]
}

3. Get Credentials

• Option A: Create access keys (for local development)
• Option B: Configure IAM role (for EC2/Lambda)
• Option C: Use AWS profile

Use Case Configurations

Code Generation

models:
  - name: "code-assistant"
    provider: "bedrock"
    model: "anthropic.claude-3-sonnet-20240229-v1:0"
    roles: ["chat", "edit"]
    env:
      region: "us-east-1"
      accessKeyId: "${AWS_ACCESS_KEY_ID}"
      secretAccessKey: "${AWS_SECRET_ACCESS_KEY}"
    defaultCompletionOptions:
      temperature: 0.3
      maxTokens: 2000

General Chat

models:
  - name: "chat-bot"
    provider: "bedrock"
    model: "anthropic.claude-3-haiku-20240307-v1:0"
    roles: ["chat"]
    env:
      region: "us-east-1"
      profile: "default"
    defaultCompletionOptions:
      temperature: 0.7
      maxTokens: 1000

Troubleshooting

Common Errors

1. AccessDeniedException: Check IAM permissions and model access
2. ValidationException: Verify model ID and region are correct
3. ThrottlingException: Rate limit reached, wait and retry
4. ModelTimeoutException: Model response timeout, try reducing input length
5. ResourceNotFoundException: Model not available in that region

Debugging Steps

1. Verify AWS credentials and permissions
2. Confirm model is enabled in target region
3. Check network connection and firewall settings
4. View CloudTrail logs for detailed error information
5. Confirm quota limits

Best Practices

1. Security

• Use environment variables to store credentials
• Use IAM roles instead of long-term credentials
• Rotate access keys regularly
• Enable CloudTrail audit logging

2. Performance Optimization

• Choose nearest region to reduce latency
• Use appropriate model size
• Enable streaming responses
• Implement request caching

3. Cost Control

• Monitor usage and costs
• Configure different IAM roles for different environments
• Set quota alerts
• Use smaller models for simple tasks

4. Error Handling

• Implement appropriate retry logic
• Handle rate limiting
• Log errors
• Set request timeouts

Region Availability

Region	Code	Claude 3	Llama 3	Mistral
US East (Virginia)	us-east-1	✅	✅	✅
US West (Oregon)	us-west-2	✅	✅	✅
Europe (Frankfurt)	eu-central-1	✅	✅	❌
Asia Pacific (Tokyo)	ap-northeast-1	✅	❌	❌