Overview

This check monitors IBM MQ versions 9.1 and above.

Setup

Installation

The IBM MQ check is included in the Datadog Agent package.

To use the IBM MQ check, ensure that an IBM MQ Client version 9.1+ is installed (unless a compatible version of IBM MQ server is already installed on the Agent host). For example the 9.3 Redistributable client. Currently, the IBM MQ check does not support connecting to an IBM MQ server on z/OS.

On Linux

Update your LD_LIBRARY_PATH to include the location of the libraries. Create this environment variable if it doesn’t exist yet. For example, if you installed the client under /opt:

export LD_LIBRARY_PATH=/opt/mqm/lib64:/opt/mqm/lib:$LD_LIBRARY_PATH

Note: Agent v6+ uses upstart, systemd or launchd to orchestrate the datadog-agent service. Environment variables may need to be added to the service configuration files at the default locations of:

• Upstart (Linux): /etc/init/datadog-agent.conf
• Systemd (Linux): /lib/systemd/system/datadog-agent.service
• Launchd (MacOS): ~/Library/LaunchAgents/com.datadoghq.agent.plist
  • This only works if MacOS SIP is disabled (might not be recommended depending on your security policy). This is due to SIP purging LD_LIBRARY_PATH environ variable.

Example of the configuration for systemd:

[Unit]
Description="Datadog Agent"
After=network.target
Wants=datadog-agent-trace.service datadog-agent-process.service
StartLimitIntervalSec=10
StartLimitBurst=5

[Service]
Type=simple
PIDFile=/opt/datadog-agent/run/agent.pid
Environment="LD_LIBRARY_PATH=/opt/mqm/lib64:/opt/mqm/lib:$LD_LIBRARY_PATH"
User=dd-agent
Restart=on-failure
ExecStart=/opt/datadog-agent/bin/agent/agent run -p /opt/datadog-agent/run/agent.pid

[Install]
WantedBy=multi-user.target

Example of the configuration for upstart:

description "Datadog Agent"

start on started networking
stop on runlevel [!2345]

respawn
respawn limit 10 5
normal exit 0

console log
env DD_LOG_TO_CONSOLE=false
env LD_LIBRARY_PATH=/opt/mqm/lib64:/opt/mqm/lib:$LD_LIBRARY_PATH

setuid dd-agent

script
  exec /opt/datadog-agent/bin/agent/agent start -p /opt/datadog-agent/run/agent.pid
end script

post-stop script
  rm -f /opt/datadog-agent/run/agent.pid
end script

Example of the configuration for launchd:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>KeepAlive</key>
        <dict>
            <key>SuccessfulExit</key>
            <false/>
        </dict>
        <key>Label</key>
        <string>com.datadoghq.agent</string>
        <key>EnvironmentVariables</key>
        <dict>
            <key>DD_LOG_TO_CONSOLE</key>
            <string>false</string>
            <key>LD_LIBRARY_PATH</key>
            <string>/opt/mqm/lib64:/opt/mqm/lib</string>
        </dict>
        <key>ProgramArguments</key>
        <array>
            <string>/opt/datadog-agent/bin/agent/agent</string>
            <string>run</string>
        </array>
        <key>StandardOutPath</key>
        <string>/var/log/datadog/launchd.log</string>
        <key>StandardErrorPath</key>
        <string>/var/log/datadog/launchd.log</string>
        <key>ExitTimeOut</key>
        <integer>10</integer>
    </dict>
</plist>

Each time there is an Agent update, these files are wiped and need to be updated again.

Alternatively, if you are using Linux, after the MQ Client is installed ensure the runtime linker can find the libraries. For example, using ldconfig:

Put the library location in an ld configuration file.

sudo sh -c "echo /opt/mqm/lib64 > /etc/ld.so.conf.d/mqm64.conf"
sudo sh -c "echo /opt/mqm/lib > /etc/ld.so.conf.d/mqm.conf"

Update the bindings:

sudo ldconfig

On Windows

There is a file called mqclient.ini in the IBM MQ data directory. It is normally C:\ProgramData\IBM\MQ. Configure the environment variable MQ_FILE_PATH, to point at the data directory.

Permissions and authentication

There are many ways to set up permissions in IBM MQ. Depending on how your setup works, create a datadog user within MQ with read only permissions and, optionally, +chg permissions. +chg permissions are required to collect metrics for reset queue statistics (MQCMD_RESET_Q_STATS). If you do not wish to collect these metrics you can disable collect_reset_queue_metrics on the configuration. Collecting reset queue statistics performance data will also reset the performance data.

The example below sets the required permissions on the queue manager QM1 for the mqclient group, the group the datadog user is using to execute commands. You can use wildcards to grant permissions to many queues at once.

setmqaut -m QM1 -n SYSTEM.ADMIN.COMMAND.QUEUE -t queue -g mqclient +dsp +inq +get +put
setmqaut -m QM1 -n SYSTEM.MQEXPLORER.REPLY.MODEL -t queue -g mqclient +dsp +inq +get +put

Note: “Queue Monitoring” must be enabled on the MQ server and set to at least “Medium”. This can be done using the MQ UI or with an mqsc command in the server’s host:

> /opt/mqm/bin/runmqsc
5724-H72 (C) Copyright IBM Corp. 1994, 2018.
Starting MQSC for queue manager datadog.


ALTER QMGR MONQ(MEDIUM) MONCHL(MEDIUM)
     1 : ALTER QMGR MONQ(MEDIUM) MONCHL(MEDIUM)
AMQ8005I: IBM MQ queue manager changed.

       :
One MQSC command read.
No commands have a syntax error.
All valid MQSC commands were processed.

Configuration

Validation

Run the Agent’s status subcommand and look for ibm_mq under the Checks section.

Data Collected

Metrics

Events

IBM MQ does not include any events.

Service Checks

ibm_mq.can_connect

Returns CRITICAL if the Agent cannot connect to the MQ server for any reason or UNKNOWN if the configured queue manager is not matched by the queue_manager_process option. Returns OK otherwise.

Statuses: ok, critical, unknown

ibm_mq.queue_manager

Returns CRITICAL if the Agent cannot retrieve stats from the queue manager or UNKNOWN if the configured queue manager is not matched by the queue_manager_process option. Returns OK otherwise.

Statuses: ok, critical, unknown

ibm_mq.queue

Returns CRITICAL if the Agent cannot retrieve queue stats. Returns OK otherwise.

Statuses: ok, critical

ibm_mq.channel

Returns CRITICAL if the Agent cannot retrieve channel stats. Returns OK otherwise.

Statuses: ok, critical

ibm_mq.channel.status

Return CRITICAL if the status is INACTIVE/STOPPED/STOPPING. Returns OK if the status is RUNNING. Returns WARNING if the status might lead to running.

Statuses: ok, critical, warning, unknown

Troubleshooting

Reset queue statistics MQRC_NOT_AUTHORIZED permission warning

If you are getting the following warning:

Warning: Error getting pcf queue reset metrics for SAMPLE.QUEUE.1: MQI Error. Comp: 2, Reason 2035: FAILED: MQRC_NOT_AUTHORIZED

This is due to the datadog user not having the +chg permission to collect reset queue metrics. To fix this, you can either give +chg permissions to the datadog user using setmqaut and collect queue reset metrics, or you can disable the collect_reset_queue_metrics:

collect_reset_queue_metrics: false

High resource utilization

The IBM MQ check performs queries on the server, sometimes these queries can be expensive and cause a degradation on the check.

If you observe that the check is taking a long time to execute or that is consuming many resources on your host, you can potentially reduce the scope of the check by trying the following:

• If you are using auto_discover_queues, try using queue_patterns or queue_regex instead to only discover certain queues. This is particularly relevant if your system creates dynamic queues.
• If you are autodiscovering queues with queue_patterns or queue_regex, try tightening the pattern or regex so it matches less queues.
• Disable auto_discover_channels if you have too many channels.
• Disable collect_statistics_metrics.

Errors in the logs

• Unpack for type ((67108864,)) not implemented: If you’re seeing errors like this, and your MQ server is running on a IBM OS, enable convert_endianness and restart your Agent.

Warnings in the logs

• Error getting [...]: MQI Error. Comp: 2, Reason 2085: FAILED: MQRC_UNKNOWN_OBJECT_NAME: If you’re seeing messages like this, it is because the integration is trying to collect metrics from a queue that doesn’t exist. This can be either due to misconfiguration or, if you’re using auto_discover_queues, the integration can discover a dynamic queue and then, when it tries to gather its metrics, the queue no longer exists. In this case you can mitigate the issue by providing a stricter queue_patterns or queue_regex, or just ignore the warning.

Other

Need help? Contact Datadog support.

Further Reading

Additional helpful documentation, links, and articles:

• Monitor IBM MQ metrics and logs with Datadog