Bacula is a backup system with three different retention periods:

• Volume
• Job
• File

I set all three to the same value.

It is important to note that these values affect only the Catalog. It determines how long a given record is held in the database. For example, if you set File Retention to be 45 days, it means that there are no records in the database for any files backed up in any jobs more than 45 days ago.

The result of this means you can’t restore a given file after the File retention passed. If the Job retention has not passed, you can still restore the whole job.

Volumes are eligible for recycling once ALL retention periods have passed.

Symptoms of File Retention < Job Retention look like this (given that you are trying to restore after File Retention has passed, but before Job Retention has passed):

Hi there. Using 2.4.4 under Debian with PostgreSQL backend and file volumes. I want to restore all files from a client, but not all files are found, however the last full backup has not been purged from the database, and saved 224000 files for 50 GB, and only 18000 files can be marked with the console in restore mode. What could be wrong ?

The problem is there are no file Retention records for the FULL backup, only for the differential and incremental backups.

Be aware.

I have about 14,000 data points collected from backups. I started in Oct 2007, but have not done anything with the data. I’d like your help.

I have put the data into a PostgreSQL database and dumped it. The dump is here: http://langille.org/tmp/dlt_stats.sql.gz

The table looks like:

$ psql dan
psql (8.4.2)
Type "help" for help.

dan=# \d dlt_stats
                                      Table "public.dlt_stats"
   Column    |            Type             |                       Modifiers
-------------+-----------------------------+--------------------------------------------------------
 id          | integer                     | not null default nextval('dlt_stats_id_seq'::regclass)
 backup_date | timestamp without time zone | not null
 dlt_id      | text                        | not null
 errors      | integer                     | not null

dan=#

The dlt_id is the label on the tape. The errors field presents the number of correctable errors per GB for that backup job. I run a script after each Bacula backup job which grabs the data from the SCSI device. I then toss that data into a flat file, which I have since imported into PostgreSQL

What I’d like is graphs on a webpage. Use whatever tools you want. I’d like to be able to select a label, and see the results. I think a line graph is best for this, but you do what you prefer.

Bonus points for pretty, simple, and elegant. Your results will be published unless otherwise requested by you. I’ll use the results of the graphs to decide what tapes to toss, what tapes to use for full backups etc.

Thank you.

I’m thinking about creating a storage box. One with a lot of disk space. It will be used for backups and stuff. I will continue to backup to tape as well.

I’m considering this case, which has 8xSATA hot-swap bays:

• http://www.newegg.com/Product/Product.aspx?Item=N82E16811192058

If I was buying a RAID card, I’d buy this one:

• http://www.newegg.com/Product/Product.aspx?Item=N82E16816131002&Tpk=areca%201120

But I may go with ZFS instead of hardware RAID.

Both BSDCan and PGCon use Pentabarf for accepting and reviewing proposals for talks. An issue raised in 2009 highlighted the ability to see the speaker name when review the list of submissions. It is a bit of very useful information.

This year, I’ve found out that we can see that vital data. Provided the speaker role has been set to confirmed. I think Pentabarf design assumes that you have created a talk, and then try to find speakers. The default value of speaker role is ‘offer’. It is not until the role is set to ‘confirmed’ that the speaker name starts to turn up in lists. I can see the logic behind this choice. However, it does not suit the way we work.

Fortunately, with SQL, there is a practical solution.

UPDATE event_person
SET event_role_state = ‘confirmed’
WHERE event_id IN (SELECT event_id FROM event WHERE conference_id = XXXX
AND event_role = ’speaker’;

where XXXX is the conference you wish to adjust.

I recommend wrapping the above in a transaction and reviewing your chance with the following query before you commit:

SELECT EP.*
FROM event_person EP, event E
WHERE EP.event_id = E.event_id
AND E.conference_id=XXXX
AND event_role = ’speaker’;

Hope that helps.

Seen this?

/libexec/ld-elf.so.1: Shared object "libcroco-0.6.so.3" not found, required by "   msgfmt"

Rebuild gettext.

portupgrade -f gettext

And try again

Port details, not relevant:

# less /var/db/ports/postgresql73/options
# This file is auto-generated by 'make config'.
# No user-servicable parts inside!
# Options for postgresql-server-7.3.21_1
_OPTIONS_READ=postgresql-server-7.3.21_1
WITH_NLS=true
WITHOUT_PAM=true
WITHOUT_MIT_KRB5=true
WITHOUT_HEIMDAL_KRB5=true
WITHOUT_OPTIMIZED_CFLAGS=true
WITHOUT_LIBC_R=true
WITHOUT_TESTS=true
WITHOUT_DEBUG=true

The building errors:

[root@pg73 /usr/ports/databases/postgresql73-server]# ldd `which msgfmt `
/usr/local/bin/msgfmt:
        libgettextsrc-0.17.so => /usr/local/lib/libgettextsrc-0.17.so (0x800655000)
        libgettextlib-0.17.so => /usr/local/lib/libgettextlib-0.17.so (0x80078f000)
        libcroco-0.6.so.3 => not found (0x0)
        libxml2.so.5 => not found (0x0)
        libz.so.4 => /lib/libz.so.4 (0x800979000)
        libm.so.5 => /lib/libm.so.5 (0x800a8d000)
        libglib-2.0.so.0 => not found (0x0)
        libintl.so.8 => /usr/local/lib/libintl.so.8 (0x800bac000)
        libpcre.so.0 => not found (0x0)
        libncurses.so.7 => /lib/libncurses.so.7 (0x800cb5000)
        libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x800e01000)
        libc.so.7 => /lib/libc.so.7 (0x800ffa000)
        libcroco-0.6.so.3 => not found (0x0)
        libxml2.so.5 => not found (0x0)
        libglib-2.0.so.0 => not found (0x0)
        libpcre.so.0 => not found (0x0)
[root@pg73 /usr/ports/databases/postgresql73-server]#

NOTE: this problem may not be a problem. After rebooting the system while having the tape library powered up, the problem went away. This may or may not be coincident.

Note to self: mtx-changer needs more customization than I thought.

Today showed great progress in completing the integration of my tape library into my Bacula system. I have been writing the testing and configuration of supporting tools. I had moved onto a new article outlining the integration phase (still under construction). That’s when I found a pretty big block in the mtx-changer script. It is this function:

#
# The purpose of this function to wait a maximum
#   time for the drive. It will
#   return as soon as the drive is ready, or after
#   waiting a maximum of 300 seconds.
# Note, this is very system dependent, so if you are
#   not running on Linux, you will probably need to
#   re-write it, or at least change the grep target.
#   We've attempted to get the appropriate OS grep targets
#   in the code at the top of this script.
#
wait_for_drive() {
  i=0
  while [ $i -le 300 ]; do  # Wait max 300 seconds
    if mt -f $1 status 2>&1 | grep "${ready}" >/dev/null 2>&1; then
      break
    fi
    debug "Device $1 - not ready, retrying..."
    sleep 1
    i=`expr $i + 1`
  done
}

The problem is, when bacula-sd is running, you cannot get the status of the tape drive. At least with my system. It is *always* busy. This, this function always times out.

For example:

$ sudo mtx -f /dev/nsa1 status
cannot open SCSI device '/dev/nsa1' - Device busy

My solution, yet to be tested:

wait_for_drive() {
  debug "waiting for Device $1"
  sleep 25
}

We’ll see.

These are the places we’re looking at to hire a mountain bike for the upcoming trip:

• http://www.bicyclerentals.co.nz/
• http://www.cyclehire.co.nz/cycle_hire.htm
• http://www.mountainbikerentals.com/

Pretty sure I will be in New Zealand for Easter. Probably Wellington. Going to rent the bike there, rather than take mine with me.

I’ll be asking the boss for the time off tomorrow.

I noticed a difference in the regression testing output between two systems today.

cmake-2.6.4

Stopping the Bacula File daemon
Stopping the Bacula Storage daemon
Stopping the Bacula Director daemon
  ===== pretest OK 00:00:39 =====
  1/110 Testing disk:acl-xattr-test              Passed
  2/110 Testing disk:action-on-purge-test        Passed
  3/110 Testing disk:accurate-test               Passed
  4/110 Testing disk:allowcompress-test          Passed
  5/110 Testing disk:auto-label-test             Passed
  6/110 Testing disk:backup-bacula-test          Passed
  7/110 Testing disk:backup-to-null              Passed
  8/110 Testing disk:base-job-test               Passed
  9/110 Testing disk:bextract-test               Passed
 10/110 Testing disk:big-fileset-test            Passed
 11/110 Testing disk:big-vol-test                Passed
 12/110 Testing disk:bscan-test                  Passed
 13/110 Testing disk:bsr-opt-test                Passed
 14/110 Testing disk:compressed-test             Passed
 15/110 Testing disk:compress-encrypt-test       Passed
 16/110 Testing disk:concurrent-jobs-test        Passed
 17/110 Testing disk:copy-jobspan-test

cmake-2.8.0

Stopping the Bacula File daemon
Stopping the Bacula Storage daemon
Stopping the Bacula Director daemon
  ===== pretest OK 16:05:36 =====
      Start  1: disk:acl-xattr-test
 1/81 Test  #1: disk:acl-xattr-test ..............   Passed   21.00 sec
      Start  2: disk:action-on-purge-test
 2/81 Test  #2: disk:action-on-purge-test ........   Passed   34.00 sec
      Start  3: disk:accurate-test
 3/81 Test  #3: disk:accurate-test ...............   Passed  155.00 sec
      Start  4: disk:allowcompress-test
 4/81 Test  #4: disk:allowcompress-test ..........   Passed   37.00 sec
      Start  5: disk:auto-label-test
 5/81 Test  #5: disk:auto-label-test .............   Passed   25.00 sec
      Start  6: disk:backup-bacula-test
 6/81 Test  #6: disk:backup-bacula-test ..........   Passed   35.00 sec
      Start  7: disk:backup-to-null
 7/81 Test  #7: disk:backup-to-null ..............   Passed   26.00 sec
      Start  8: disk:base-job-test
 8/81 Test  #8: disk:base-job-test ...............   Passed   35.00 sec
      Start  9: disk:bextract-test
 9/81 Test  #9: disk:bextract-test ...............   Passed   13.00 sec
      Start 10: disk:big-fileset-test
10/81 Test #10: disk:big-fileset-test ............   Passed   19.00 sec
      Start 11: disk:big-vol-test
11/81 Test #11: disk:big-vol-test ................   Passed   31.00 sec
      Start 12: disk:bscan-test
12/81 Test #12: disk:bscan-test ..................   Passed   25.00 sec
      Start 13: disk:bsr-opt-test
13/81 Test #13: disk:bsr-opt-test ................   Passed   21.00 sec
      Start 14: disk:compressed-test
14/81 Test #14: disk:compressed-test .............   Passed   23.00 sec
      Start 15: disk:compress-encrypt-test
15/81 Test #15: disk:compress-encrypt-test .......   Passed   25.00 sec
      Start 16: disk:concurrent-jobs-test
16/81 Test #16: disk:concurrent-jobs-test ........   Passed   32.00 sec
      Start 17: disk:copy-jobspan-test
17/81 Test #17: disk:copy-jobspan-test ...........   Passed   29.00 sec
      Start 18: disk:copy-job-test
18/81 Test #18: disk:copy-job-test ...............   Passed   28.00 sec
      Start 19: disk:copy-uncopied-test

I must say: I like the change.

I lived in New Zealand from 1985-2001. I haven’t been back since. It’s time.

The plan: March-April 2010, two weeks. Not nearly enough time for a proper holiday or visit, but we’ll hit the hottest trails. This should be a blast.

We plan to do 8-10 rides in that time. We’ll see. I hope to document as much as I can. I just ordered the mountain biking book to help with some of the planning.

We’ll rent a car, bike, and go from there.