Recently I needed to change the Nagios plugin that monitors my DNSSEC signed domains. The old plugin sends queries to PowerDNS that it will no longer reply to. I don’t want to patch the plugin, because there are some replacements that I can use: nagval and CHECK_ZONE_RRSIG_EXPIRATION.
The first plugin install worked without problems, but the CHECK_ZONE_RRSIG_EXPIRATION plugin did not work. I just got a Nagios critical without reason. Executing the plugin from the command line worked. Then I needed to learn how to debug Nagios.
First check the debug settings in /etc/nagios3/nagios.cfg and make sure that the debug level is high enough:
debug_file=/var/log/nagios3/nagios.debug
debug_level=24
I forced the next plugin run and found this error message in the debug file:
[1435325317.107438] [016.1] [pid=7464] HOST: zone.example.com, SERVICE: DNSRRSIG, CHECK TYPE: Active, OPTIONS: 1, SCHEDULED: Yes, RESCHEDULE: Yes, EXITED OK: No, RETURN CODE: 3, OUTPUT: **ePN failed to compile /usr/local/bin/check_zone_rrsig_expiration: "Variable "$data" will not stay shared at (eval 12) line 149," at /usr/lib/nagios3/p1.pl line 161.\n
First I did not understand why this error message occured. Then I learned about the embedded Perl interpreter in Nagios. The plugin does not work when run with the embedded Perl interpreter. As a solution, you can disable the embedded Perl interpreter for this plugin by adding a single line at the start of the plugin after the shebang:
#!/usr/bin/perl
# nagios: -epn