Performance

Database Monitoring relies on data (metrics and logs) that you send to Splunk Observability Cloud through an OpenTelemetry Collector (“collector”). You set up the collector to monitor your database platforms, and you use a different configuration for the collector for each different database platform type that you monitor. This topic explains:

• Performance testing results: The performance impact of the collector’s actions, in other words the data collection overhead, on the database servers it monitors, and how we measured this impact.
• Best practices: Our suggestions for optimal configuration for the collector. An optimal configuration is one that minimizes the impact on both the monitored database servers and on the collector’s host machine.

Microsoft SQL Server

Test setup

Setting	Value
Collector host machine	AWS m5.large container (used for running collector monitoring multiple databases)
Database host machine	AWS RDS instance, db.m5.large (MS SQL Server 2016 Standard Edition)
Database type	MS SQL Server 2016 Standard Edition
Collector version	Splunk 0.131.1
Collection interval	10 seconds
TopX count	200
TopX collection interval	60 seconds
Sample size	1000
Lookback	20 seconds
Load generation tool	HammerDB (TPC-C workload)
Number of target databases	Up to 30 databases monitored concurrently by one collector instance
Collector memory allocation	Minimum 2048 MiB to avoid forced garbage collection
VM tested	5a.large

Performance testing results

• The collector's CPU impact on the db.m5.large database host machine is less than 1% on idle systems. On systems under heavy load, the collector’s impact remains small and generally indistinguishable from normal CPU usage fluctuations.
• The collector can monitor up to about 30 databases per instance on a properly sized VM with at least 2048 MiB memory to avoid performance issues.
• CPU usage on the collector host remains low (around 5.21% average) even when monitoring 30 databases, allowing multiple collector instances (up to 7 or 8) to run on a single VM depending on CPU capacity.
• The collector runs queries on the database, but the overhead from these queries is low enough to be effectively invisible in typical operational environments.

Best practices

• Use a single collector instance to monitor a maximum of 30 database servers concurrently. This limit also reduces the single collector configuration file to a maximum of 30 sets of parameters, which helps you to manage its complexity.
• Run each collector on a host machine with memory and CPU that meets or exceeds an AWS m5.large instance (2 vCPUs, 8GBB RAM) to avoid forced garbage collection and data loss. Proper memory allocation (at least 2048 MiB) is critical to avoid performance issues on the collector host.
• Based on the observation that a single collector can monitor up to about 30 databases per instance, it is plausable that you can run multiple instances of a collector on a single host machine depending on the machine's CPU capacity.
• Recommended collection settings for the collector:
  • 10-second collection interval
  • TopX count of 200
  • TopX collection interval of 60 seconds
  • Sample size of 1000
  • Lookback window of 20 seconds

Oracle Database

Test setup

Setting	Value
Collector host machine	AWS m5.large container (used for running collector monitoring multiple databases)
Database host machine	AWS RDS Oracle instance, db.m5.large (Oracle 19c Enterprise Edition)
Database type	Oracle 19c Enterprise Edition
Collector version	Splunk 0.131.1
Collection interval	10 seconds
TopX count	200
TopX collection interval	60 seconds
Sample size	1000
Lookback	20 seconds
Load generation tool	HammerDB (TPC-C workload)
Number of target databases	Up to 30 databases monitored concurrently by one collector instance
Collector memory allocation	Minimum 512 MiB to avoid forced garbage collection

Performance testing results

• The collector's CPU impact on the db.m5.large database host machine is less than 0.63% on idle systems. On systems under heavy load, the collector’s impact remains small and generally indistinguishable from normal CPU usage fluctuations.
• The collector can monitor up to about 30 databases per instance on a properly sized VM with at least 512 MiB memory to avoid performance issues.
• CPU usage on the collector host remains low (around 5.25% average) even when monitoring 30 databases, allowing multiple collector instances (up to 7 or 8) to run on a single VM depending on CPU capacity.
• The collector runs queries on the database, but the overhead from these queries is low enough to be effectively invisible in typical operational environments.

Best practices

• Use a single collector instance to monitor a maximum of 30 database servers concurrently. This limit also reduces the single collector configuration file to a maximum of 30 sets of parameters, which helps you to manage its complexity.
• Run each collector on a host machine with memory and CPU that meets or exceeds an AWS m5.large instance (2 vCPUs, 8GB RAM) to avoid forced garbage collection and data loss. Proper memory allocation (at least 512 MiB) is critical to avoid performance issues on the collector host.
• Based on the observation that a single collector can monitor up to about 30 databases per instance, it is plausable that you can run multiple instances of a collector on a single host machine depending on the machine's CPU capacity.
• Recommended collection settings for the collector:
  • 10-second collection interval
  • TopX count of 200
  • TopX collection interval of 60 seconds
  • Sample size of 1000
  • Lookback window of 20 seconds