Monitor AWS Bedrock APIs

Splunk AppDynamics Python Agent helps you monitor the AWS Bedrock usage and related metrics. It monitors the converse and converse_stream requests created using the Boto3 library.

The following metrics are reported at the application, tier, and node levels:

  • Input Tokens
  • Completion Tokens
  • Tokens
  • Latency
  • Calls per minute

Prerequisites

Python agent supports the Boto3 for AWS Bedrock in AppDynamics. For more information, see Boto3 documentation.

Ensure that you meet the following prerequisites to monitor the OpenAI API exit calls with AppDynamics:

Create Custom Dashboard

  1. Download Custom_dashboard_script.zip. It contains:
    Directory Description
    templates Contains json templates to create the dashboard files.
    dashboard Dashboard json files are created here. It contains the sample json files.
    main.py Python script to create the dashboards.
    readme.md Description of the solution.
  2. Run the following command:
    CODE 
    cd Custom_dashboard_script
python3 main.py

    The preceding command prompts you to specify the dashboard type (application or tier) that you want to create.

  3. Based on your selected dashboard type, you will be prompted to specify the following dashboard details.
    Application Level Dashboard Detals
    Variable Description Default Possible Values Required
    Dashboard Name Dashboard Title that is displayed in Dashboard and Reports. bedrock_app_dashboard bedrock_app_dashboard Optional
    Application Name Name of the Application with bedrock backend calls. bedrock_app None Yes
    Model Level Dashboard Details
    Variable Description Default Possible Values Required
    Model Name Name of the Model which you want to monitor. amazon-titan-dashbaord None Yes
    Dashboard Name Dashboard Title that is displayed in Dashboard and Reports. bedrock_app_dashboard bedrock_app_dashboard Optional
    Application Name Name of the Application with bedrock backend calls. bedrock_app None Yes
    Note: If Model Name has special characters, you must replace them with hyphens ( - ). For example, if the model name is "amazon-titan-lite:1", change the name to "amazon-titan-lite-1".
  4. Import the dashboard in the Controller UI. See Create and Manage Custom Dashboards and Templates
    Tip: Navigate to Tiers & Nodes > My Dashboards and create a dashboard for the selected application by copying and pasting widgets from the existing dashboard for easier navigation.