Skip to main content

Monitoring MongoDB

Overview

MongoDB plugin is an sfAgent plugin used to collect mongodb metrics. It uses serverStatus, dbstats command to retrieve the statistics. If replication is enabled and in cluster mode(replicaset) then collects the replication information.

Support monitoring on:

  1. Standalone mode
  2. Cluster (replicaset) mode

MongoDB Monitoring in Standalone mode

Prerequisites

  1. sfAgent requires access to MongoDB system tables to collect metrics. This will require adding user credentials to sfAgent plugin configuration. User role and credentials can be created with the procedure mentioned below

    • Open /etc/mongod.conf file and comment security config if it exists. This is needed to make changes to config file and will be uncommented in subsequent step

       #security: 
      #authorization: enabled
    • Start mongodb service with command service mongod start

    • Check mongodb status is active using command service mongod status

    • Create mongostatRole and listDatabases roles

      • Use admin

      • Execute the command below

        db.createRole(
        {
        role: "mongostatRole",
        privileges: [
        { resource: { cluster: true },
        actions: [ "serverStatus" ] }],
        roles: []
        })

        db.createRole(
        {
        role: "listDatabases",
        privileges: [
        { resource: { cluster: true },
        actions: [ "listDatabases" ] }],
        roles: []
        })
  1. Create new user with clusterMonitor , mongostatRole and listDatabases roles

    db.createUser( { user:"Newuser", pwd: "pass",
    roles : [
    { role : "mongostatRole", db : "admin" },
    { role : "listDatabases", db :"admin" },
    { role : "clusterMonitor", db : "admin" }
    ] } )
  2. Exit mongo using exit command

  3. Open /etc/mongod.conf file and uncomment security config if exists else add the config

    security:
    authorization: enabled
  4. Restart mongodb using command service mongod restart

  5. Use the created mongo user credentials in sfagent config and start sfagent

Configuration

sfAgent section provides steps to install and automatically generate plugin configurations. User can also manually add the configuration shown below to config.yaml under /opt/sfagent/ directory.

key: <Profile_key>
tags:
Name: <instance_name>
appName: <app_name>
projectName: <project_name>
metrics:
plugins:
- name: mongodb
enabled: true
interval: 300
config:
port: 27017
host: localhost
username: Newuser
password: pass

Metrics list

documentType : dbStats
NameDescription
CollectionsContains a count of the number of collections in that database
DataSizeTotal size of the data held in this database including the padding factor.Shown as byte
Dbdatabase name
IndexSizeTotal size of all indexes created on this database.Shown as byte
StorageSizeTotal amount of space allocated to collections in this database for document storage.Shown as byte
TotalSizeTotal size of the data held in this database including the padding factor
documentType : systemInfo
NameDescription
Versionversion of mongodb
UptimeMillisNumber of seconds that the mongos or mongod process has been active
availabeConnectionNumber of unused available incoming connections the database can provide
currentConnectionNumber of connections to the database server from clients
activeConnectionsNumber of active connections to the database server from clients
aggDataSizeTotal size of the data held in this database including the padding factor
FsTotalSizeTotal size of all disk capacity on the filesystem where MongoDB stores data
aggFsUsedSizeTotal size of all disk space in use on the filesystem where MongoDB stores data
aggStorageSizeSum of the space allocated to all collections in the database for document storage, including free space
aggTotalSizeSum of the space allocated for both documents and indexes in all collections in the database. Includes used and free storage space. This is the sum of storageSize and indexSize
TotalDbtotal number of databases
documentType : operationalMetrics
NameDescription
insertQueryNumber of times insert executed
updateQueryNumber of times update executed
deleteQueryNumber of times delete executed
dirtyBytesInCacheSize of the dirty data in the cache
CacheSizeMaximum cache size
CurrentQueueTotalTotal number of operations queued waiting for the lock
CurrentQueueReadersNumber of operations that are currently queued and waiting for the read lock
CurrentQueueWritersNumber of operations that are currently queued and waiting for the write lock
ActiveClientsTotalTotal number of active client connections to the database
ActiveClientsReadersCount of the active client connections performing read operations
ActiveClientswritersCount of active client connections performing write operations
readLatencyTotal latency statistics for read requests per second
writeLatencyTotal latency statistics for write operations per second
PhysicalBytesInThe number of bytes that reflects the amount of network traffic received by this database
PhysicalBytesOutThe number of bytes that reflects the amount of network traffic sent from this database
CurrentCacheSize of the data currently in cache

Viewing data and dashboards

Data collected by the plugin can be viewed in SnappyFlow’s browse data section.

  • plugin: mongodb
  • documentType: dbStats, systemInfo, operationalMetrics
  • Dashboard template: MongoDB
  • Alerts template: MongoDB

MongoDB Monitoring in cluster(replicaset) mode

MongoDB cluster is a set of nodes that are primary and secondary nodes. The nodes are synced and maintain same data in all the nodes in the replicaset type. The synchronization and the node information are collected about the cluster.

Install the agent on all the nodes in the cluster, both the primary nodes and the secondary nodes.

There are four documentTypes related to the replication:

  1. nodeDetails

  2. replicationMemberDetails

  3. replicationSyncDetails

  4. oplogDetails

    Pre-requisites

Note: The replication should be the replicaset based replication.

  1. sfAgent requires access to MongoDB system tables to collect metrics. This will require adding user credentials to sfAgent plugin configuration. User role and credentials can be created with the procedure mentioned below
  • Open /etc/mongod.conf file and comment security config if it exists. This is needed to make changes to config file and will be uncommented in subsequent step

     #security: 
    #authorization: enabled
  • Start mongodb service with command service mongod start

  • Check mongodb status is active using command service mongod status

  • Create mongostatRole and listDatabases roles

    • Use admin

    • Execute the command below

      db.createRole(
      {
      role: "mongostatRole",
      privileges: [
      { resource: { cluster: true },
      actions: [ "serverStatus" ] }],
      roles: []
      })

      db.createRole(
      {
      role: "listDatabases",
      privileges: [
      { resource: { cluster: true },
      actions: [ "listDatabases" ] }],
      roles: []
      })
  1. Create new user with clusterMonitor , mongostatRole and listDatabases roles

    db.createUser( { user:"Newuser", pwd: "pass",
    roles : [
    { role : "mongostatRole", db : "admin" },
    { role : "listDatabases", db :"admin" },
    { role : "clusterMonitor", db : "admin" }
    ] } )
  2. Exit mongo using exit command

  3. Open /etc/mongod.conf file and uncomment security config if exists else add the config

    security:
    authorization: enabled
  4. Restart mongodb using command service mongod restart

  5. Use the created mongo user credentials in sfagent config and start sfagent

Configuration

sfAgent section provides steps to install and automatically generate plugin configurations. User can also manually add the configuration shown below to config.yaml under /opt/sfagent/ directory.

key: <Profile_key>
tags:
Name: <instance_name>
appName: <app_name>
projectName: <project_name>
metrics:
plugins:
- name: mongodb
enabled: true
interval: 300
config:
port: 27017
host: localhost
username: Newuser
password: pass

Metrics list

documentType : ReplicationSyncDetails
NameDescription
sourcesecondary node name(ip)
syncedTosyncronised time
repllagDelay between a write operation on the primary and its copy to a secondary. Computed on each node and tagged by 'host', but may not be representative of cluster health. Negative values do not indicate that the secondary is ahead of the primary.
SyncStatewhen instance down it shows the info of the state
ErrMessageit shows the error message when the instance is down
documentType : ReplicationMemberdetails
NameDescription
nameThe name of the member
Idid of the member
SyncSourceIDThe syncSourceId field holds the [replSetGetStatus.membersn]._id of the member from which this instance syncs
healthA number that indicates if the member is up (i.e. 1) or down (i.e. 0)
stateAn integer between 0 and 10 that represents the replica state of the member
stateStrA string that describes state
uptimeThe uptime field holds a value that reflects the number of seconds that this member has been online
lastHeartbeatAn ISODate formatted date and time that reflects the last time the server that processed the replSetGetStatus command received a response from a heartbeat that it sent to this member (members[n])
lastHeartbeatRecvAn ISODate formatted date and time that reflects the last time the server that processed the replSetGetStatus command received a heartbeat request from this member (members[n])
syncSourceHostThe syncSourceHost field holds the hostname of the member from which this instance syncs
lastHeartbeatMessageWhen the last heartbeat included an extra message, the lastHeartbeatMessage contains a string representation of that message
lastCommittedOpTimeInformation, from the viewpoint of this member, regarding the most recent operation that has been written to a majority of replica set members.
appliedOpTimeInformation, from the viewpoint of this member, regarding the most recent operation that has been applied to this member of the replica set
durableOpTimeInformation, from the viewpoint of this member, regarding the most recent operation that has been written to the journal of this member of the replica set
InfoMessagegives information on the member state If the instance is down
documentType : nodeDetails
NameDescription
isWritablePrimaryis the node writable
namethe node name(ip)
primaryis a primary member of a replica set
readOnlyis node running in read-only mode
secondaryis a secondary member of a replica set
setNamereplicaset name
documentType : oplogDetails
NameDescription
logSizeMBTotal size of the oplog
usedMBTotal amount of space used by the oplog
timeDiff(in secs)difference between the first and last operation in the oplog
tFirstReturns a time stamp for the first (i.e. earliest) operation in the oplog. Compare this value to the last write operation issued against the server.Only present if there are entries in the oplog
tLastReturns a time stamp for the last (i.e. latest) operation in the oplog. Compare this value to the last write operation issued against the server.Only present if there are entries in the oplog
nowReturns a timestamp that reflects the current time. The shell process generates this value, and the datum may differ slightly from the server time if you're connecting from a remote host as a result. Equivalent to Date().Only present if there are entries in the oplog

 Viewing data and dashboards

Data collected by the plugin can be viewed in SnappyFlow’s browse data section    

  • Plugin = mongodb
  • documentType(in primary)= dbStats, operationalMetrics, systemInfo, nodeDetails, replicationMemberDetails, replicationSyncDetails, oplogDetails
  • documentType(in secondary)= dbStats, operationalMetrics, systemInfo, nodeDetails, oplogDetails
  • Dashboard template: MongoDBReplication
  • Alerts template: MongoDBReplication

See Also

MySQL

PostgresDB