ByteBuddy

English

简体中文

English

简体中文

深色模式

Model Roles Introduction ​

ByteBuddy's model role system allows you to select the most suitable model configuration for different task types, optimizing performance and cost-effectiveness.

What are Model Roles? ​

Model roles are pre-configured model settings for specific task types, including:

  • Model selection
  • Parameter tuning
  • Context management
  • Performance optimization

Supported Role Types ​

💬 Chat ​

For conversation and interactive tasks:

  • Natural language understanding
  • Contextual dialogue
  • Multi-turn conversations
  • User support

🔧 Autocomplete ​

For code and text completion:

  • Real-time code suggestions
  • Smart text completion
  • Context-aware completion
  • Fast response

✏️ Edit ​

For text and code editing:

  • Content refactoring
  • Format adjustment
  • Syntax correction
  • Style optimization

🎯 Apply ​

For specific application scenarios:

  • Code generation
  • Document creation
  • Data processing
  • Automated tasks

🔍 Embeddings ​

For vectorization and semantic search:

  • Text vectorization
  • Semantic similarity
  • Retrieval augmentation
  • Clustering analysis

📊 Reranking ​

For result sorting and filtering:

  • Relevance ranking
  • Quality assessment
  • Priority adjustment
  • Precise matching

Role Configuration Structure ​

Basic Configuration Example ​

Configure model roles in config.yaml:

yaml
models:
  - name: "chat-assistant"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"
    roles: ["chat"]
    defaultCompletionOptions:
      temperature: 0.7
      maxTokens: 2000
      topP: 0.9

  - name: "code-editor"
    provider: "anthropic"
    model: "claude-3-sonnet"
    apiKey: "${ANTHROPIC_API_KEY}"
    roles: ["edit", "apply"]
    defaultCompletionOptions:
      temperature: 0.3
      maxTokens: 4000

  - name: "autocomplete-engine"
    provider: "together"
    model: "codellama/CodeLlama-13b-Instruct-hf"
    apiKey: "${TOGETHER_API_KEY}"
    roles: ["autocomplete"]
    defaultCompletionOptions:
      temperature: 0.1
      maxTokens: 256

Multi-Role Configuration ​

A single model can serve multiple roles:

yaml
models:
  - name: "versatile-model"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"
    roles: ["chat", "edit", "apply"]
    defaultCompletionOptions:
      temperature: 0.5
      maxTokens: 4096

Role-Specific Configuration ​

Chat Role Configuration ​

yaml
models:
  - name: "chat-model"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"
    roles: ["chat"]
    defaultCompletionOptions:
      temperature: 0.7
      maxTokens: 2000
      topP: 0.9

Edit Role Configuration ​

yaml
models:
  - name: "edit-model"
    provider: "anthropic"
    model: "claude-3-sonnet"
    apiKey: "${ANTHROPIC_API_KEY}"
    roles: ["edit"]
    defaultCompletionOptions:
      temperature: 0.2
      maxTokens: 4000

Autocomplete Role Configuration ​

yaml
models:
  - name: "autocomplete-model"
    provider: "together"
    model: "codellama/CodeLlama-13b-Instruct-hf"
    apiKey: "${TOGETHER_API_KEY}"
    roles: ["autocomplete"]
    defaultCompletionOptions:
      temperature: 0.1
      maxTokens: 256

Apply Role Configuration ​

yaml
models:
  - name: "apply-model"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"
    roles: ["apply"]
    defaultCompletionOptions:
      temperature: 0.3
      maxTokens: 4096

Embed Role Configuration ​

yaml
models:
  - name: "embed-model"
    provider: "openai"
    model: "text-embedding-3-large"
    apiKey: "${OPENAI_API_KEY}"
    roles: ["embed"]

Rerank Role Configuration ​

yaml
models:
  - name: "rerank-model"
    provider: "cohere"
    model: "rerank-english-v3.0"
    apiKey: "${COHERE_API_KEY}"
    roles: ["rerank"]

Best Practices ​

1. Role Design Principles ​

  • Single Responsibility: Each role focuses on specific tasks
  • Clear Boundaries: Define clear applicable scenarios
  • Performance Optimization: Balance quality and efficiency

2. Configuration Management ​

  • Version Control: Track role configuration changes
  • Test Validation: Ensure role behavior meets expectations
  • Documentation: Keep configuration documentation updated

3. Usage Guidelines ​

  • Start Small: Begin with predefined roles
  • Gradual Customization: Adjust based on actual needs
  • Continuous Optimization: Improve roles based on feedback

Next Steps ​

  1. Learn About Roles: Review specific role type documentation
  2. Configure Models: Set up model roles in config.yaml
  3. Test and Adjust: Continuously improve based on usage
  4. Share Experience: Share quality configurations with team

Copyright © 2025 ByteBuddy