Skip to main content

Monitor PgBouncer

PgBouncer acts as a connection pool mechanism for PostgreSQL. PgBouncer creates a connection pool and enables the application to use the existing connections in the pool to interact with the database.

The number of PostgreSQL backend processes can be reduced by this mechanism. PgBouncer reduces the resource (memory, backend, fork, etc..) consumption of the PostgreSQL. It also decreases the response time between the client application and the database.

By integrating PgBouncer with SnappyFlow, you enable SnappyFlow to collect the connection metrics from PgBouncer and visualize them in SnappyFlow's dashboard.

note

The configurations given below apply only to the PgBouncer running on an instance.

PgBouncer Monitoring

PgBouncer running on Linux can be monitored using Snappyflow sfagent. Refer to the link for installation of sfagent https://docs.snappyflow.io/docs/Integrations/os/linux/sfagent_linux.

note

PgBouncer version 1.12 and above can be monitored.

PgBouncer monitoring is tested on version 1.12.

Pre-Requisites

PgBouncer admin and stats user configuration.

Configure the pgbouncer user as admin and stats user by adding the below lines in the pgbouncer.ini config file.

The default location of the pgbouncer configuration file is /etc/pgbouncer/pgbouncer.ini.

Execute the following command in the Pgbouncer configuration file.

admin_users = <username>

stats_users = <username>

Add ignore startup parameters

By default, PgBouncer allows only the parameters it can keep track of in the startup packets: client_encoding, datestyle, timezone, and standard_conforming_strings. All other parameters will raise an error. To allow other parameters, ignore startup parameters command is used.

Example: ignore_startup_parameters = extra_float_digits

Configuration 

Add the following configuration to the config.yaml file:

key: <profile key>
tags:
Name: <unique instance name or will be generated from IP>
appName: <add application name>
projectName: <add project name>
metrics:
plugins:
- name: pgbouncer
enabled: true
interval: 30
config:
port: <port>
host: <host/IP>
user: <username>
password: <password>
logging:
  plugins:
- name: pgbouncer
enabled: true
config:
log_path: "/var/log/postgresql/pgbouncer.log"
   

View Metrics and Logs

  1. Login into SnappyFlow.

  2. Navigate to the Application tab > Project > Application and click the Dashboard icon.

  3. Click the tab menu ... icon on the Metric tab.

  4. Select the Import from template option.

  5. In the Import to Metrics Panes window, select Filter By: All or Standard, Template Name: PgBouncer.

  6. Click the Save button.

General logs

To view general logs:

  1. Navigate to your Application's dashboard > Log management > Primary storage.
  2. In the Primary Storage window, set the Log Type to a general log as per the requirement.

OR

In the Browse Data window, set the following filters:

  • Index: Log
  • Instance: Select your instance
  • Plugin: pgbouncer
  • Document type: pgbouncerLogs

Metrics

To view database metrics:

  1. Navigate to Application's dashboard> Browse Data.
  2. In the Browse Data window, set the following filters:
  • Index: Metrics
  • Instance: Select your instance
  • Plugin: pgbouncer
  • Document type: pgbouncerDetails, databaseStats, databasePools

Metrics list

PgbouncerDetails
MetricDescription
versionCurrent Version of the PgBouncer
numServerActiveTotal server connections that are linked to a client.
numServerUsedTotal server connections that have been idle for more than server_check_delay, so they need server_check_query to run on them before they can be used again.
numServerIdleServer connections that are unused and immediately usable for client queries.
numServerLoginServer connections currently in the process of logging in.
numServerTestedServer connections that are currently running either server_reset_query or server_check_query
numClientsActiveServer connections that are linked to a client.
numClientsWaitClient connections that have sent queries but have not yet got a server connection.
maximumClientConnMaximum number of clients that can connect to PgBouncer
totalTransactionCountTotal number of SQL transactions pooled by pgbouncer.
totalQueryCountTotal number of SQL queries pooled by pgbouncer.
totalRecievedBytesTotal volume in bytes of network traffic received by pgbouncer.
totalSentBytesTotal volume in bytes of network traffic sent by pgbouncer.
averageTransactionCountPerSecondAverage transactions per second in last stat period.
averageQueryCountPerSecondAverage queries per second in last stat period.
averageSentBytesPerSecondAverage sent (to clients) bytes per second.
averageRecievedBytesPerSecondAverage received (from clients) bytes per second.
Database Stats
MetricDescription
databaseNameStatistics are presented per database.
totalTransactionTimeMicrosecondsTotal number of microseconds spent by pgbouncer when connected to PostgreSQL in a transaction, either idle in transaction or executing queries.
averageTransactionTimeMicrosecondsAverage transactions per second in last stat period.
totalRecievedBytesTotal volume in bytes of network traffic received by pgbouncer.
averageTransactionCountPerSecondAverage transactions per second in last stat period.
averageRecievedBytesPerSecondAverage received (from clients) bytes per second.
totalTransactionCountTotal number of SQL transactions pooled by pgbouncer.
averageWaitTimeMicrosecondsAverage time spent by clients waiting for a server that were assigned a backend connection within the current stats_period, in microseconds (averaged per second within that period).
totalWaitTimeMicrosecondsTime spent by clients waiting for a server, in microseconds. Updated when a client connection is assigned a backend connection.
averageQueryTimeMicrosecondsAverage query duration, in microseconds.
totalQueryCountTotal number of SQL queries pooled by pgbouncer.
averageQueryCountPerSecondAverage queries per second in last stat period.
averageSentBytesPerSecondAverage sent (to clients) bytes per second.
totalSentBytesTotal volume in bytes of network traffic sent by pgbouncer.
totalQueryTimeMicrosecondsTotal number of microseconds spent by pgbouncer when actively connected to PostgreSQL, executing queries.
Database Pools
MetricDescription
userUser name
databaseNameDatabase name
numServerActiveServer connections that are linked to a client.
numServerUsedServer connections that have been idle for more than server_check_delay, so they need server_check_query to run on them before they can be used again.
numServerIdleServer connections that are unused and immediately usable for client queries.
numServerLoginServer connections currently in the process of logging in.
numServerTestedServer connections that are currently running either server_reset_query or server_check_query
numClientsActiveServer connections that are linked to a client.
numClientsWaitClient connections that have sent queries but have not yet got a server connection.
maximumWaitTimeinSecondsMaximum wait time of the first (oldest) client in the queue has waited, in seconds
maximumWaitMicrosecondsMicrosecond part of the maximum waiting time.
poolModeThe pooling mode in use.