NEWS:

Resource containers for FreeBSD jails

July 14th, 2010

The FreeBSD Foundation says:

“We are pleased to announce that Edward Tomasz Napierala has been awarded a grant to implement resource containers and a simple per-jail resource limits mechanism.”

http://lists.freebsd.org/pipermail/freebsd-announce/2010-July/001335.html

A simple nested loop

July 11th, 2010

In a previous post, I wrote about a suspected LIMIT problem. It turns out that suspicion proved correct. The solution was to move to a nested query which limits the underlying data and then allows the outer query to grab all the associated fluff that surrounds it.

Here is the fix to the original code:

$ cvs di -ub commits_by_committer.php
Index: commits_by_committer.php
===================================================================
RCS file: /home/repositories/freshports-1/classes/commits_by_committer.php,v
retrieving revision 1.3
diff -u -b -r1.3 commits_by_committer.php
--- commits_by_committer.php    10 Feb 2008 19:17:22 -0000      1.3
+++ commits_by_committer.php    11 Jul 2010 17:36:53 -0000
@@ -89,11 +89,12 @@
                }

                $sql .= "
-         WHERE commit_log.committer = '" . AddSlashes($this->Committer) . "'
-           AND commit_log_elements.commit_log_id = commit_log.id
-           AND commit_log_elements.element_id    = element.id
-   ORDER BY 1 desc,
-                       commit_log_id";
+         WHERE commit_log.id IN (SELECT tmp.id FROM (SELECT DISTINCT CL.id, CL.commit_date
+  FROM element_pathname EP, commit_log_elements CLE, commit_log CL
+ WHERE CL.committer  = '" . AddSlashes($this->Committer) . "'
+   AND EP.element_id = CLE.element_ID
+   AND CL.id         = CLE.commit_log_id
+ORDER BY CL.commit_date DESC ";

                if ($this->Limit) {
                        $sql .= "\nLIMIT " . $this->Limit;
@@ -105,6 +106,13 @@

+
+               $sql .= ")as tmp)
+           AND commit_log_elements.commit_log_id = commit_log.id
+           AND commit_log_elements.element_id    = element.id
+   ORDER BY 1 desc,
+                       commit_log_id";
+
                if ($this->Debug) echo '
' . $sql . '

‘;

$this->LocalResult = pg_exec($this->dbh, $sql);
$

The query (greatly simplified), before, the fix was:

SELECT DISTINCT
	commit_log.commit_date - SystemTimeAdjust() AS commit_date_raw,
	commit_log.id                       AS commit_log_id,
	commit_log.encoding_losses          AS encoding_losses,
	commit_log.message_id               AS message_id,
	commit_log.committer                AS committer,
	commit_log.description              AS commit_description
   FROM commit_log, commit_log_elements, element
  WHERE commit_log.committer = 'jb'
    AND commit_log_elements.commit_log_id = commit_log.id
    AND commit_log_elements.element_id    = element.id
ORDER BY 1 desc, commit_log_id
LIMIT 100

With the fix, the query becomes:

SELECT DISTINCT
	commit_log.commit_date - SystemTimeAdjust()        AS commit_date_raw,
	commit_log.id                                      AS commit_log_id,
	commit_log.encoding_losses                         AS encoding_losses,
	commit_log.message_id                              AS message_id,
	commit_log.committer                               AS committer,
	commit_log.description                             AS commit_description,
   FROM commit_log, commit_log_elements, element
	  WHERE commit_log.id IN (
   SELECT tmp.id FROM (SELECT DISTINCT CL.id, CL.commit_date
     FROM commit_log CL
    WHERE CL.committer  = 'jb'
    ORDER BY CL.commit_date DESC
    LIMIT 100
    )as tmp)
    AND commit_log_elements.commit_log_id = commit_log.id
    AND commit_log_elements.element_id    = element.id
ORDER BY 1 desc, commit_log_id

How many FreshPorts accounts are enough?

April 28th, 2010

I found this interesting. The usual FreshPorts logging indicated an unusual number of account creations in a short amount of time. So I went looking. I found one IP address had created 20 logins.

freshports.org=#   SELECT ip_address,
freshports.org-#          count(ip_address)
freshports.org-#     FROM users
freshports.org-# GROUP BY ip_address
freshports.org-# ORDER BY 2 DESC
freshports.org-#    LIMIT 2;
  ip_address   | count
---------------+-------
 0.0.0.0       |  1203
 95.211.27.210 |    38
(2 rows)

freshports.org=#

Ignore that 0.0.0.0. That indicates a user which was created before the ip_address field was added to the table.

So…. what’s going on here?

freshports.org=#   SELECT email,
freshports.org-#          firstlogin
freshports.org-#     FROM users
freshports.org-#    WHERE ip_address = '95.211.27.210'
freshports.org-# ORDER BY 2 DESC;
            email            |          firstlogin
-----------------------------+-------------------------------
 pamelathorn@mail15.com      | 2010-04-29 17:15:14.23305+01
 barbarajackson@mail15.com   | 2010-04-29 12:52:33.786686+01
 barbarajackson@mail15.com   | 2010-04-29 12:08:25.597785+01
 elizabethmartin@mail15.com  | 2010-04-29 11:04:21.886639+01
 elizabethmartin@mail15.com  | 2010-04-29 10:56:06.860984+01
 elizabethmartin@mail15.com  | 2010-04-29 10:20:51.615789+01
 pamelathorn@mail15.com      | 2010-04-29 10:00:03.366337+01
 elizabethmartin@mail15.com  | 2010-04-29 09:38:11.397616+01
 elizabethmartin@mail15.com  | 2010-04-29 09:26:49.760226+01
 elizabethmartin@mail15.com  | 2010-04-29 08:07:33.958235+01
 pamelathorn@mail15.com      | 2010-04-29 05:10:11.033442+01
 pamelathorn@mail15.com      | 2010-04-29 04:43:53.755426+01
 jennifermartinez@mail15.com | 2010-04-29 04:26:49.43041+01
 jennifermartinez@mail15.com | 2010-04-29 04:24:43.374981+01
 jennifermartinez@mail15.com | 2010-04-29 01:47:24.950648+01
 jennifermartinez@mail15.com | 2010-04-29 01:23:12.473853+01
 jennifermartinez@mail15.com | 2010-04-29 01:14:24.086015+01
 barbarajackson@mail15.com   | 2010-04-29 01:10:49.826533+01
 barbarajackson@mail15.com   | 2010-04-29 00:47:02.95436+01
 barbarajackson@mail15.com   | 2010-04-28 22:52:17.371778+01
 barbarajackson@mail15.com   | 2010-04-28 22:02:52.512775+01
 pamelathorn@mail15.com      | 2010-04-28 21:20:53.972371+01
 elizabethmartin@mail15.com  | 2010-04-28 18:43:28.266215+01
 pamelathorn@mail15.com      | 2010-04-28 14:48:55.668841+01
 barbarajackson@mail15.com   | 2010-04-28 13:33:13.06128+01
 barbarajackson@mail15.com   | 2010-04-28 13:33:12.524171+01
 barbarajackson@mail15.com   | 2010-04-28 13:24:23.736056+01
 elizabethmartin@mail15.com  | 2010-04-28 11:03:50.452646+01
 elizabethmartin@mail15.com  | 2010-04-28 11:00:55.18447+01
 barbarajackson@mail15.com   | 2010-04-28 10:56:58.46371+01
 pamelathorn@mail15.com      | 2010-04-28 07:42:18.979039+01
 barbarajackson@mail15.com   | 2010-04-28 04:56:03.343899+01
 pamelathorn@mail15.com      | 2010-04-28 03:30:26.976301+01
 jennifermartinez@mail15.com | 2010-04-28 02:46:50.949887+01
 jennifermartinez@mail15.com | 2010-04-27 10:58:40.681218+01
 jennifermartinez@mail15.com | 2010-04-27 10:45:52.53576+01
 barbarajackson@mail15.com   | 2010-04-27 06:54:22.046983+01
 elizabethmartin@mail15.com  | 2010-04-27 06:31:13.662795+01
(38 rows)

freshports.org=#

OK, so he’s doing this persistently. I sent him an email earlier. No reply. Shocking.

So his next attempt will redirect him to this page.

Bacula - errors with 5.0.0 backup catalog

March 23rd, 2010

While creating the FreeBSD package for 5.0.0, I noticed a few new things to be aware of.

With 5.0.0 comes a new backup catalog script. It parses the bacula-dir.conf file and extracts what it needs from that. This means you only need to specify the database connection information in one place and it is not passed to a script via the command line.

If you see this error while running that script:

25-Jan 16:37 ducky.unixathome.org-dir JobId 32500: shell command: run BeforeJob “/usr/local/share/bacula/make_catalog_backup.pl MyCatalog”
25-Jan 16:37 ducky.unixathome.org-dir JobId 32500: BeforeJob: Can’t get catalog information at /usr/local/share/bacula/make_catalog_backup.pl line 129.
25-Jan 16:37 ducky.unixathome.org-dir JobId 32500: Error: Runscript: BeforeJob returned non-zero status=13. ERR=Child exited with code 13
25-Jan 16:37 ducky.unixathome.org-dir JobId 32500: Error: Bacula ducky.unixathome.org-dir 5.0.0 (26Jan10):
25-Jan-2010 16:37:10

Then I suspect the permissions on make_catalog_backup.pl are insufficient for Bacula to run it. I made mine chown bacula:bacula and chmod 744:

-rwxr-xr-- 1 bacula bacula 4120 Jan 25 17:10 make_catalog_backup.pl

If you see this error:

25-Jan 16:44 ducky.unixathome.org-dir JobId 32502: shell command: run BeforeJob “/usr/local/share/bacula/make_catalog_backup.pl MyCatalog”
25-Jan 16:44 ducky.unixathome.org-dir JobId 32502: BeforeJob: Can’t find your catalog (MyCatalog) in director configuration
25-Jan 16:44 ducky.unixathome.org-dir JobId 32502: Error: Runscript: BeforeJob returned non-zero status=1. ERR=Child exited with code 1

Then chances are the permissions on bacula-dir.conf or dbcheck are insufficient. Make sure this command works:

# su -m bacula -c '/usr/local/sbin/dbcheck -B -c /usr/local/etc/bacula-dir.conf'
catalog=MyCatalog
db_name=bacula
db_driver=
db_user=bacula
db_password=
db_address=
db_port=0
db_socket=
db_type=PostgreSQL
working_dir=/var/db/bacula

If you get: /usr/local/sbin/dbcheck: Permission denied.

Make sure bacula is allowed to run dbcheck. I use this:

-rwxr-xr-- 1 root bacula 100225 Jan 25 17:12 dbcheck

If you get this, check the permissions on bacula-dir.conf:

26-Jan 01:37 dbcheck: ERROR TERMINATION at parse_conf.c:898 Config error: Cannot open config file “/usr/local/etc/bacula-dir.conf”: Permission denied

I use this:

-rw-r----- 1 root bacula 9095 Jan 25 17:16 bacula-dir.conf

Finally, to be sure, run this command:

su -m bacula -c '/usr/local/share/bacula/make_catalog_backup.pl MyCatalog'

and if that runs fine, Bacula should be able to run it.

Also, if during the backup, you see permission errors on pathhierarchy or pathvisibility then the database upgrade wasn’t run as the correct user. You may have to adjust the owner of the database to match that of
the other tables.

NOTE: This was originally posted to the Bacula users mailing list on 1 January 2010.

power failure and gmirror

March 15th, 2010

From this morning’s log after a weekend power failure:

Mar 15 09:15:45 subie kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad8 finished.
Mar 15 09:20:58 subie fsck: /dev/mirror/gm0s1f: UNREF FILE I=35  OWNER=root MODE=100400
Mar 15 09:20:58 subie fsck: /dev/mirror/gm0s1f: SIZE=151568652480 MTIME=Mar 13 16:03 2010  (CLEARED)
Mar 15 09:20:58 subie fsck: /dev/mirror/gm0s1f: UNREF FILE I=15356429  OWNER=dan MODE=100644
Mar 15 09:20:58 subie fsck: /dev/mirror/gm0s1f: SIZE=216159 MTIME=Mar 13 15:35 2010  (CLEARED)
Mar 15 09:20:58 subie fsck: /dev/mirror/gm0s1f: Reclaimed: 0 directories, 3 files, 97 fragments
Mar 15 09:20:58 subie fsck: /dev/mirror/gm0s1f: 796585 files, 14968261 used, 56710199 free (218983 frags, 7061402 blocks, 0.3% fragmentation)
Mar 15 09:21:27 subie fsck: /dev/mirror/gm0s1d: 27569 files, 188950 used, 1307441 free (11913 frags, 161941 blocks, 0.8% fragmentation)

No ZFS there…

Pharma GCP standards

March 13th, 2010

A friend is looking for a job. She specializes in testing of software systems for compliance with Pharma GCP standards. Please share this post.

NRPE: Unable to read output

March 10th, 2010

After rebooting kraken to take a photo, I found nagios was displaying an error for my smartmon checks: NRPE: Unable to read output.

Running the command by hand on the nagios server, I found:

$ /usr/local/libexec/nagios/check_nrpe2 -H kraken -c check_smartmon_ad24
NRPE: Unable to read output

But from the remote server I got:

# /usr/local/libexec/nagios/check_smartmon -d /dev/ad24
OK: device is functional and stable (temperature: 29)

I restarted npre and the problem went away… not sure what this is yet….

smartmontools - patch

March 10th, 2010

The smartmon bug I mentioned a few days ago has a patch. It was submitted 24 Jan 2010 but was closed after the maintainer said he would release a new distfile soon.

It hasn’t arrived. I sent him and email tonight as a follow up the PR.

In the meantime, I’ve applied the patch to my own syst

NewEgg - RMA

March 10th, 2010

NewEgg has received and processed my returned purchases. I’m impressed. I mailed them on Thursday (I think) and here it is, Wednesday. I should expect to see the refunds on my credit card within 2-3 days.

Interesting hardware site

March 8th, 2010

I do like CoolDrives.com.

I was shown this quick swap drive tray and started hunting around. This spare parts page is interesting and reminds me that I want to buy different cables for the file server.

Also neat are these hard drive cases for when your HDD is not in use.