Dell TL4000 tape library: running btape and configuring Bacula’s mtx-changer

In this post, I am using FreeBSD 10.2 and Bacula 7.4.0.

I am now the proud second-hand owner of a Dell TL4000. The first thing I did was add an air filter. Why? Because I can. I got one made, with custom dimensions, by DEMCiflex. You can see that filter here in this photo.

My next task is configuring the tape library to work with Bacula. The first part of that is mtx-changer and btape.

btape

btape is a Bacula utility which performs some basic tests on your tape drive. In short, it tests the drive according to the given configuration file you pass it. You should run btape on every drive before you start using it with Bacula.

mtx-changer

mtx-changer is a shell script provided with Bacula which provides the glue between the Bacula binaries and the mtx binary. This script also permits you to customize it to your installation.

I like to keep my own copy of mtx-changer (and its configuration file) separate from the Bacula installation. I started with this:

# cp /usr/local/share/bacula/mtx-changer      /usr/local/sbin/
# cp /usr/local/share/bacula/mtx-changer.conf /usr/local/etc/bacula

Along with that copy, I need to first alter mtx-changer to pull in mtx-changer.conf from the new location. I also supplied a full path to mtx.

The diff for my patches are here.

Simple queries

Let’s do basic queries on the devices:

$ sudo camcontrol devlist
<IBM ULT3580-HH4 C7Q1>             at scbus0 target 0 lun 0 (pass0,sa0)
<IBM 3573-TL B.60>                 at scbus0 target 0 lun 1 (pass1,ch0)
<IBM ULT3580-HH4 C7Q1>             at scbus0 target 1 lun 0 (pass2,sa1)
<IBM 3573-TL B.60>                 at scbus0 target 1 lun 1 (pass3,ch1)
<CT480BX200SSD1 MU01.4>            at scbus1 target 0 lun 0 (ada0,pass4)
<CT480BX200SSD1 MU01.4>            at scbus3 target 0 lun 0 (ada1,pass5)

Query on the first tape drive and changer:

$ sudo mtx -f /dev/pass0 inquiry
Product Type: Tape Drive
Vendor ID: 'IBM     '
Product ID: 'ULT3580-HH4     '
Revision: 'C7Q1'
Attached Changer API: No

$ sudo tapeinfo -f /dev/pass0
Product Type: Tape Drive
Vendor ID: 'IBM     '
Product ID: 'ULT3580-HH4     '
Revision: 'C7Q1'
Attached Changer API: No
SerialNumber: '1K10012996'
MinBlock: 1
MaxBlock: 16777215
Ready: yes
BufferedMode: yes
Medium Type: 0x48
Density Code: 0x46
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0

$ sudo tapeinfo -f /dev/pass1
Product Type: Medium Changer
Vendor ID: 'IBM     '
Product ID: '3573-TL         '
Revision: 'B.60'
Attached Changer API: No
SerialNumber: '00X4U78G4272_LL0'
Ready: yes

On the second drive (NOTE: this drive did not contain a tape):

$ sudo tapeinfo -f /dev/pass2
Product Type: Tape Drive
Vendor ID: 'IBM     '
Product ID: 'ULT3580-HH4     '
Revision: 'C7Q1'
Attached Changer API: No
SerialNumber: '9K11801866'
MinBlock: 1
MaxBlock: 16777215
Ready: no

$ sudo tapeinfo -f /dev/pass3
Product Type: Medium Changer
Vendor ID: 'IBM     '
Product ID: '3573-TL         '
Revision: 'B.60'
Attached Changer API: No
SerialNumber: '00X4U78G4272_LL0'
Ready: yes

Notice that the /dev/pass1 and /dev/pass3 both have the same serial number. I suspect two changers are reported but only one exists.

Start by walking

I always start with mt(1) and mtx(1), and then proceed to mtx-changer, which uses those two commands.

Here is what is loaded in the tape library:

$ sudo mtx -f /dev/pass1 status
  Storage Changer /dev/pass1:2 Drives, 47 Slots ( 0 Import/Export )
Data Transfer Element 0:Full (Storage Element 2 Loaded):VolumeTag = 000002L4                       
Data Transfer Element 1:Empty
      Storage Element 1:Full :VolumeTag=000001L4                       
      Storage Element 2:Empty
      Storage Element 3:Full :VolumeTag=000003L4                       
      Storage Element 4:Full :VolumeTag=000004L4                       
      Storage Element 5:Full :VolumeTag=000005L4                       
      Storage Element 6:Full :VolumeTag=000006L4                       
      Storage Element 7:Full :VolumeTag=000007L4                       
      Storage Element 8:Full :VolumeTag=000008L4                       
      Storage Element 9:Full :VolumeTag=000009L4                       
      Storage Element 10:Full :VolumeTag=000010L4                       
      Storage Element 11:Full :VolumeTag=000011L4                       
      Storage Element 12:Full :VolumeTag=000012L4                       
      Storage Element 13:Full :VolumeTag=000013L4                       
      Storage Element 14:Full :VolumeTag=000014L4                       
      Storage Element 15:Full :VolumeTag=000015L4                       
      Storage Element 16:Full :VolumeTag=000016L4                       
      Storage Element 17:Full :VolumeTag=000017L4                       
      Storage Element 18:Full :VolumeTag=000018L4                       
      Storage Element 19:Full :VolumeTag=000019L4                       
      Storage Element 20:Full :VolumeTag=000020L4                       
      Storage Element 21:Full :VolumeTag=000021L4                       
      Storage Element 22:Full :VolumeTag=000022L4                       
      Storage Element 23:Full :VolumeTag=000023L4                       
      Storage Element 24:Full :VolumeTag=000024L4                       
      Storage Element 25:Full :VolumeTag=000025L4                       
      Storage Element 26:Full :VolumeTag=000026L4                       
      Storage Element 27:Full :VolumeTag=000027L4                       
      Storage Element 28:Full :VolumeTag=000028L4                       
      Storage Element 29:Full :VolumeTag=000029L4                       
      Storage Element 30:Full :VolumeTag=000030L4                       
      Storage Element 31:Full :VolumeTag=000031L4                       
      Storage Element 32:Full :VolumeTag=000032L4                       
      Storage Element 33:Full :VolumeTag=000033L4                       
      Storage Element 34:Full :VolumeTag=000034L4                       
      Storage Element 35:Full :VolumeTag=000035L4                       
      Storage Element 36:Full :VolumeTag=000036L4                       
      Storage Element 37:Full :VolumeTag=000037L4                       
      Storage Element 38:Full :VolumeTag=000038L4                       
      Storage Element 39:Full :VolumeTag=000039L4                       
      Storage Element 40:Empty
      Storage Element 41:Full :VolumeTag=000041L4                       
      Storage Element 42:Full :VolumeTag=000042L4                       
      Storage Element 43:Full :VolumeTag=000043L4                       
      Storage Element 44:Full :VolumeTag=000044L4                       
      Storage Element 45:Full :VolumeTag=000045L4                       
      Storage Element 46:Full :VolumeTag=000046L4                       
      Storage Element 47:Full :VolumeTag=000048L4    

Now I load from slot 1 into drive 1:

$ sudo mtx -f /dev/pass3 load 1 1
Loading media from Storage Element 1 into drive 1...done

I am not see any Drive needs offline before move messages like I did with the Dell PERC 6 card.

After a bit of settling time, about 30s I think, I was able to run this command:

$ sudo mt -f /dev/sa0 status
Drive: sa0: <IBM ULT3580-HH4 C7Q1> Serial Number: 1K10012996
---------------------------------
Mode      Density              Blocksize      bpi      Compression
Current:  0x46:LTO-4           variable       323215   enabled (0x1)
---------------------------------
Current Driver State: at rest.
---------------------------------
Partition:   0      Calc File Number:   0     Calc Record Number: 0
Residual:    0  Reported File Number:   0 Reported Record Number: 0
Flags: BOP

Now try mtx-changer

Let’s start off by asking what is loaded in drive 0:

$ sudo /usr/local/sbin/mtx-changer /dev/pass1 loaded x x 0
2

The x x are just place holders… the last parameter must be drive-index, and the parameters slot and archive-device are ignored for the loaded command.

Let’s try unloading that into slot 40:

$ sudo /usr/local/sbin/mtx-changer /dev/pass1 unload 40 /dev/sa0 0
Unloading drive 0 into Storage Element 40...done

That offline statement is related to the configuration setting I mentioned above (offline=1). I changed it to 0, and that message disappeared from the unload process. However, it continues to appear in the load command.

bacula-sd.conf

This is the configuration file for this bacula-sd instance. You will notice that I am using nsa0 and nsa1, and not sa0 and sa1. The ‘n’ means No rewind on close. See sa(4) (definitely do not read sa(8)).

NOTE: Always use the non-automatic-rewind device. Always specify the ‘n’.

Storage {
  Name                      = "tape01-sd"
  WorkingDirectory          = "/usr/local/bacula/working"
  Pid Directory             = "/var/run"
  Maximum Concurrent Jobs   = 20
}

Director {
  Name     = "bacula-dir"
  Password = "password from Storage resource in bacula-dir.conf"
}

Messages {
  Name     = Standard
  director = bacula-dir = all
}

Autochanger {
  Name            = "tape01"
  Device          = LTO_0, LTO_1
  Description     = "Dell PowerVault TL4000 Tape Library"
  Changer Device  = /dev/pass1
  Changer Command = "/usr/local/sbin/mtx-changer %c %o %S %a %d"
}

Device {
  Name                    = "LTO_0"
  Description             = "LTO first drive"
  Media Type              = LTO4
  Archive Device          = /dev/nsa0
        
  Autochanger             = yes
  Drive Index             = 0
            
  Offline On Unmount      = no
  Hardware End of Medium  = no
  BSF at EOM              = yes
  Fast Forward Space File = yes
  TWO EOF                 = yes

  AutomaticMount          = yes
  AlwaysOpen              = yes
  RemovableMedia          = yes
  RandomAccess            = no

  Spool Directory         = /usr/local/bacula/spooling
  Maximum Spool Size      = 375809638400
  Maximum Job Spool Size  = 375809638400
}

Device {
  Name                    = "LTO_1"
  Description             = "LTO second drive"
  Media Type              = LTO4
  Archive Device          = /dev/nsa1
        
  Autochanger             = yes
  Drive Index             = 0
            
  Offline On Unmount      = no
  Hardware End of Medium  = no
  BSF at EOM              = yes
  Backward Space Record   = no
  Fast Forward Space File = yes
  TWO EOF                 = yes

  AutomaticMount          = yes
  AlwaysOpen              = yes
  RemovableMedia          = yes
  RandomAccess            = no

  Spool Directory         = /usr/local/bacula/spooling
  Maximum Spool Size      = 375809638400
  Maximum Job Spool Size  = 375809638400
}

Notes on the above:

  1. Yes, that is not that password I use, but I hope it helps you understand that Bacula passwords are really shared secrets and they need to be recorded in at least two places: with the user of the password, and with the authenticator of the connection.
  2. I use a spooling directory because this is an LTO-4 tape drive and my disk drives are not fast enough to keep it fed. As I type this, I am wondering: do copy/migration jobs honour spooling? I suspect they do.

btape

I’m going to run btape again. As described earlier in this post, mtx-changer has been installed and configured.

The following is the btape testing.

$ sudo btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa0
Tape block granularity is 1024 bytes.
btape: butil.c:291-0 Using device: "/dev/nsa0" for writing.
btape: btape.c:471-0 open device "LTO_0" (/dev/nsa0): OK
*test

=== Write, rewind, and re-read test ===

I'm going to write 10000 records and an EOF
then write 10000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.

This is an *essential* feature ...

btape: btape.c:1154-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1170-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1212-0 Rewind OK.
10000 blocks re-read correctly.
Got EOF on tape.
10000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===

btape: btape.c:1279-0 Block position test
btape: btape.c:1291-0 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:9999
Block 10000 re-read correctly.
Reposition to file:block 1:0
Block 10001 re-read correctly.
Reposition to file:block 1:600
Block 10601 re-read correctly.
Reposition to file:block 1:9999
Block 20000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2

btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:471-0 open device "LTO_0" (/dev/nsa0): OK
btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:1422-0 Now moving to end of medium.
btape: btape.c:627-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 4. This is correct!

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to 
the tape.


=== Write, backup, and re-read test ===

I'm going to write three records and an EOF
then backup over the EOF and re-read the last record.
Bacula does this after writing the last block on the
tape to verify that the block was written correctly.

This is not an *essential* feature ...

btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:813-0 Wrote first record of 64412 bytes.
btape: btape.c:824-0 Wrote second record of 64412 bytes.
btape: btape.c:835-0 Wrote third record of 64412 bytes.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:851-0 Backspaced over EOF OK.
btape: btape.c:856-0 Backspace record OK.
btape: btape.c:873-0 
Block re-read correct. Test succeeded!
=== End Write, backup, and re-read test ===



=== Forward space files test ===

This test is essential to Bacula.

I'm going to write five files then test forward spacing

btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:1909-0 Wrote one record of 64412 bytes.
btape: btape.c:1911-0 Wrote block to device.
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:606-0 Wrote 1 EOF to "LTO_0" (/dev/nsa0)
btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:1636-0 Now forward spacing 1 file.
We should be in file 1. I am at file 1. This is correct!
btape: btape.c:1648-0 Now forward spacing 2 files.
We should be in file 3. I am at file 3. This is correct!
btape: btape.c:576-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:1661-0 Now forward spacing 4 files.
We should be in file 4. I am at file 4. This is correct!

btape: btape.c:1679-0 Now forward spacing 1 more file.
We should be in file 5. I am at file 5. This is correct!

=== End Forward space files test ===


Ah, I see you have an autochanger configured.
To test the autochanger you must have a blank tape
 that I can write on in Slot 1.

Do you wish to continue with the Autochanger test? (y/n): y


=== Autochanger test ===

3301 Issuing autochanger "loaded" command.
Slot 2 loaded. I am going to unload it.
3302 Issuing autochanger "unload 2 0" command.
unload status=OK 0
3303 Issuing autochanger "load 1 0" command.
3303 Autochanger "load 1 0" status is OK.
btape: btape.c:471-0 open device "LTO_0" (/dev/nsa0): OK
btape: btape.c:1566-0 Rewound "LTO_0" (/dev/nsa0)
btape: btape.c:1573-0 Wrote EOF to "LTO_0" (/dev/nsa0)

The test autochanger worked!!

*quit
[dan@tape01:~] $ 

Look for The above Bacula scan should have output identical in the above output. Make sure the results are indeed identical in your output.

After the above test, I ran the same command on nsa1. You need to test every drive.

When I first ran this, I did not have a tape in slot 1. This was the failure text:

...
3301 Issuing autochanger "loaded" command.
Slot 40 loaded. I am going to unload it.
3302 Issuing autochanger "unload 40 0" command.
unload status=OK 0
3303 Issuing autochanger "load 1 0" command.
3993 Bad autochanger command: /usr/local/sbin/mtx-changer /dev/pass1 load 1 /dev/nsa0 0
3993 result="Loading media from Storage Element 1 into drive 0...Source Element Address 4096 is Empty
": ERR=Child died from signal 15: Termination
You must correct this error or the Autochanger will not work.
*

The next blog post will cover getting bacula-sd running for this tape library.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Leave a Comment

Scroll to Top