Amazon Bedrock

This documentation describes the integration of MindsDB with Amazon Bedrock, a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies. The integration allows for the deployment of models offered by Amazon Bedrock within MindsDB, providing the models with access to data from various data sources.

​

Prerequisites

Before proceeding, ensure the following prerequisites are met:

1. Install MindsDB locally via Docker or Docker Desktop.
2. To use Amaon Bedrock within MindsDB, install the required dependencies following this instruction.
3. Obtain the AWS credentials for a user with access to the Amazon Bedrock service.

​

Setup

Create an AI engine from the Amazon Bedrock handler.

CREATE ML_ENGINE bedrock_engine
FROM bedrock
USING
    aws_access_key_id = 'AQAXEQK89OX07YS34OP',
    aws_secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
    aws_session_token = 'FwoGZXIvYXdzEJr...',
    aws_region = 'us-east-1',

Required parameters for creating an engine include the following:

• aws_access_key_id: The AWS access key ID for the user.
• aws_secret_access_key: The AWS secret access key for the user.
• aws_region: The AWS region to use.

Optional parameters include the following:

• aws_session_token: The AWS session token for the user. This is required when using temporary security credentials.

Create a model using bedrock_engine as an engine.

CREATE MODEL bedrock_model
PREDICT answer
USING
    engine = 'bedrock_engine',
    question_column = 'question',
    max_tokens = 100,
    temperature = 0.3,

Required parameters for creating a model include the following:

• engine: The name of the engine created via CREATE ML_ENGINE.

Optional parameters include the following:

• mode: The mode to run inference in. The default mode is default and the other supported mode is conversational.
• model_id: The model ID to use for inference. The default model ID is amazon.titan-text-premier-v1:0 and a list of other supported models can be found https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html.
• question_column: The column that stores the user input.
• context_column: The column that stores context to the user input.
• prompt_template: A template for the prompt with placeholders to be replaced by the user input.
• max_tokens: The maximum number of tokens to be generated in the model’s responses.
• temperature: The likelihood of the model selecting higher-probability options while generating a response.
• top_p: The percentage of most-likely candidates that the model considers for the next token.
• stop: A list of tokens that the model should stop generating at.

​

Usage

​

Default Mode

In the default mode, the model will generate a separate response for each input provided. No context is maintained between the inputs.

CREATE MODEL bedrock_default_model
PREDICT answer
USING
    engine = 'bedrock_engine',
    prompt_template = 'Answer the users input in a helpful way: {{question}}';

To generate a response for a single input, the following query can be used:

SELECT *
FROM bedrock_default_model
WHERE question = 'What is the capital of Sweden?';

The response will look like the following:

To generate responses for multiple inputs, the following query can be used:

SELECT *
FROM files.unrelated_questions AS d
JOIN bedrock_default_model AS m

The response will look like the following:

​

Conversational Mode

In the conversational mode, the model will maintain context between the inputs and generate a single response. This response will be placed in the last row of the result set.

CREATE MODEL bedrock_conversational_model
PREDICT answer
USING
    engine = 'bedrock_engine',
    mode = 'conversational',
    question_column = 'question';

The syntax for generating responses in the conversational mode is the same as in the default mode.

However, when generating responses for multiple inputs, the difference between the two modes becomes apparent. As mentioned above, the conversational mode maintains context between the inputs and generates a single response, which is placed in the last row of the result set:

SELECT *
FROM files.related_questions AS d
JOIN bedrock_default_model AS m

This is what the response will look like: