Getting mod_watch running with Apache 2.2

mod_watch is a now unsupported package, originally created by Anthony C. Howe. I first learned of this software, from this Cacti post, when looking to graph statistics from my various Apache virtual hosts. I want to incorporate this data into my existing cacti installations. For those suggesting I use something other than cacti: thanks, but switching tools is not in the scope of this problem.

mod_watch is available on github where it was imported by Ryan McKern.

Downloading and installing mod_watch is pretty straight forward. This command will configure and install.

make -f Makefile.dso install

I recommend using the github version. I didn’t find it immediately and started encountering errors.

For the record

This section can be ignored if you are using the Ryan McKern (i.e. github) version of mod_watch.

There are some patches which you should apply if you use the version found on the cacti forum. I think these are because of API changes, but I have not confirmed this.

If you get this error:

httpd: Syntax error on line 109 of /usr/local/etc/apache22/httpd.conf: Cannot load /usr/local/libexec/apache22/mod_watch.so into server: /usr/local/libexec/apache22/mod_watch.so: Undefined symbol "APR_STATUS_IS_SUCCESS">

Then you need to apply this patch.

After doing that, you should then see this error:

httpd: Syntax error on line 109 of /usr/local/etc/apache22/httpd.conf: Cannot load /usr/local/libexec/apache22/mod_watch.so into server: /usr/local/libexec/apache22/mod_watch.so: Undefined symbol "shPrintFormat"

It was at this point that I found, and started using, the Ryan McKern version.

Current status

I have been able to get mod_watch running and collecting statistics. However, there is nagging problem which I have been unable to solve, and which others have also encountered. I have tried their solutions and failed.

This error arrives in /var/log/httpd-error.log on a regular basis:

$ grep crit /var/log/httpd-error.log | tail -5
[Wed Aug 01 12:17:34 2012] [crit] (20014)Internal error: apr_global_mutex_child_init(28ebbdc0, /var/lib/mod_watch/SharedHash.lock, 28fea018) failed in shChildInit()
[Wed Aug 01 12:20:19 2012] [crit] (20014)Internal error: apr_global_mutex_child_init(28ebbdc0, /var/lib/mod_watch/SharedHash.lock, 28fea018) failed in shChildInit()
[Wed Aug 01 12:20:20 2012] [crit] (20014)Internal error: apr_global_mutex_child_init(28ebbdc0, /var/lib/mod_watch/SharedHash.lock, 28fea018) failed in shChildInit()
[Wed Aug 01 12:20:20 2012] [crit] (20014)Internal error: apr_global_mutex_child_init(28ebbdc0, /var/lib/mod_watch/SharedHash.lock, 28fea018) failed in shChildInit()
[Wed Aug 01 12:22:16 2012] [crit] (20014)Internal error: apr_global_mutex_child_init(28ebbdc0, /var/lib/mod_watch/SharedHash.lock, 28fea018) failed in shChildInit()
$

I am working with patches applied to the master version of mod_watch on github.

References for these patches:

• http://forums.cacti.net/about15475.html – look for: Regarding the error log / watchLogHash I found a fix here
• http://lists.pld-linux.org/mailman/pipermail/pld-cvs-commit/Week-of-Mon-20110328/307470.html
• http://wiki.kartbuilding.net/index.php/Munin_Statistics#Apache_Mod_Watch_.26_Munin_Graphs_showing_vhost_accesses

My current diff is shown below (but that URL may be more useful to you):

diff -ruN mckern-mod_watch-42134b9/SharedHash.c MyWorkingCopy/SharedHash.c
--- mckern-mod_watch-42134b9/SharedHash.c	2011-05-04 19:06:34.000000000 +0000
+++ MyWorkingCopy/SharedHash.c	2012-07-31 20:07:55.000000000 +0000
@@ -95,11 +95,14 @@
 const char shScanFormat[] = SH_SCAN_FORMAT;
 const char shPrintFormat[] = SH_PRINT_FORMAT;
 
+int shMutexMech = APR_LOCK_DEFAULT;
+
 #ifdef BOUNDARY_CHECKING
 char *
 shVerifyString(struct shTable *tp, char *str)
 {
-	if ((char *) tp->shared < = str && str < (char *) tp->eshared) {
+	/*if ((char *) tp->shared < = str && str < (char *) tp->eshared) { */
+	if ((char *) apr_shm_baseaddr_get(tp->shared) < = str && str < (char *) tp->eshared) {
 		if (str + strlen(str) < (char *) tp->eshared)
 			return str;
 	}
@@ -509,7 +512,7 @@
 
 	rc = apr_global_mutex_create(
 		(apr_global_mutex_t **) &tp->mutex,
-		tp->lockfile, APR_LOCK_DEFAULT, p
+		tp->lockfile, shMutexMech, p
 	);
 	if (rc != APR_SUCCESS) {
 		ap_log_error(
diff -ruN mckern-mod_watch-42134b9/SharedHash.h MyWorkingCopy/SharedHash.h
--- mckern-mod_watch-42134b9/SharedHash.h	2011-05-04 19:06:34.000000000 +0000
+++ MyWorkingCopy/SharedHash.h	2012-07-31 20:08:22.000000000 +0000
@@ -110,6 +110,8 @@
 extern "C" {
 #endif
 
+extern int shMutexMech;
+
 extern const char shLockFile[];
 extern const char shScanFormat[];
 extern const char shPrintFormat[];

mod_watch is producing output

I have this entry in httpd.conf:

LoadModule watch_module       libexec/apache22/mod_watch.so

And these entries in the main virtual host for this Apache installation:

 <IfModule mod_watch.c>

    #WatchMutexMech file
    WatchDynamicVirtualHost on
    WatchLogAll off

    # Allows the URL used to query virtual host data:
    #
    #   http://www.snert.com/watch-info
    #
    <Location /watch-info>
    SetHandler watch-info
    </Location>

    # Allows the URL used to query file owner and web
    # server data:
    #
    #   http://www.snert.com/~achowe/watch-info
    #   http://www.snert.com/~SERVER/watch-info
    #
    <Location /~*/watch-info>
    SetHandler watch-info
    </Location>
  </IfModule>

I am getting output from this module.

$ ls -l /var/lib/mod_watch
total 26
-rw-r--r--  1 www  www   57 Aug  1 00:00 SERVER
-rw-------  1 www  www    0 Aug  1 00:00 SharedHash.lock
-rw-r--r--  1 www  www   43 Aug  1 00:00 ads.unixathome.org
-rw-r--r--  1 www  www   52 Aug  1 00:00 beta.freshports.org
-rw-r--r--  1 www  www   52 Aug  1 00:00 dan.langille.org
drwxr-xr-x  2 www  www  512 Jul 31 19:13 ip
-rw-r--r--  1 www  www   42 Aug  1 00:00 lists.bsdcan.org
-rw-r--r--  1 www  www   36 Aug  1 00:00 lists.freebsddiary.org
-rw-r--r--  1 www  www   35 Aug  1 00:00 lists.freshports.org
-rw-r--r--  1 www  www   39 Aug  1 00:00 lists.pgcon.org
-rw-r--r--  1 www  www   36 Aug  1 00:00 lists.phorum.org
-rw-r--r--  1 www  www   35 Aug  1 00:00 lists.racingsystem.com
-rw-r--r--  1 www  www   50 Aug  1 00:00 news.freshports.org
-rw-r--r--  1 www  www   52 Aug  1 00:00 nyi.unixathome.org

And the files contain what looks like great stats:

$ cat /var/lib/mod_watch/nyi.unixathome.org
834921 1863358 4998 4984 0 133.336 33744 1343778963

I just wish that error didn’t keep happening. I have not tried to use the statistics yet.

Any ideas? Suggestions? Comments?

I’d like to see if we can resurrect mod_watch and get it working error free. If that happens, I’ll push the patches to github.