Bacula uses flat text files for configuration. From time to time, people, usually new to the project, ask: why isn’t the configuration in the database?
Bacula uses the database to store information about what files were backed up, from what client, and when. This information is useful when you need to know how if you have a copy of a file from last Tuesday before you wrote crap into it but after you made those important changes. [I tend to refer to Bacula directives with a capital letter; you’ll see this in the next sentence.] If you use the Catalog, you don’t have to go searching through tarballs, or logs to find what you need. Bacula will find it for you.
So, newbies figure that it makes perfect sense to put all this into the Catalog. Yes, on the face of it, that makes sense. Drive everything off the database. Sorry, no, it’s not going to happen.
Flat text files are much better for configuration. They are easily human readable, and the last thing you want to lose if you database crashes is your configuration.
There are claims that writing a GUI interface to Bacula would be much easier if only we had the configuration in the database. Sorry. BAT seems to be a great GUI tool and it works off the existing configuration files.
There has been a call for a configuration tool. I point out Lilac (formerly Fruity) which is a configuration tool for Nagios. It produces the configuration files, then signals Nagios. I think something like that for Bacula would be warmly greeted.
While I agree text files are very useful and easy to deal with in small systems, they do not scale or work well with larger, more dynamic systems – we are building a Bacula system for 250-1000+ hosts where we add/delete/change hosts, filesets, etc. daily, or even hourly using by staff globally – not having a DB-based system will eventually kill us, so we are looking at how to build it ourselves in 2011; given our size we have to standardize everything so the actual changes we allow are modest and templated so this should not be that hard to automate and cross-check, but would be nice as a real included toolset.
In addition, given our rate of config change, we constantly have to validate the backups vs. the host configs, file locations (for example adding vhosts on servers and making sure their content gets backed up), so again having the config in a DB helps cross-check all that, validating against our monitoring and host config / provisioning systems, etc. Text files spread across many servers makes this painful (1 director and backup, but eventually dozens of global storage directors).
Probably use a PHP front-end and the DB, then Puppet or related to spread config files in real time or at least hourly to all the systems.
Not to mention all the fun of PKI management for all this at large scale, both for TLS and data encryption. Plus our automated recovery testing system that recovers each key host each month to a test system to validate accuracy and that we got consistent non-corrupt DBs, etc. All globally, on 1000+ hosts. Fun, fun.
Wow! Steve, did you notice that first paragraph of yours is all one sentence? It took me a while to follow it.
The bacula text files are no different than other text files you have on the systems in question. Anyone that has hundreds/thousands of systems already has this issue and presumably a way to deal with it. Bacula is there for backups. That is what it is designed for.
The issue of managing configuration files is best left to other solutions, such as, in your example, puppet.
Well, one sentence with a few dashes and semi-colons, but agreed, a bit long.
For me, most large-scale systems are run from databases, for monitoring and most other backup systems, even lighter weight like BackupExec, and presumably Galaxy and Veritas. In this case, all these main files are on the Bacula director, and still seems a tad absurd we can store all the results in a DB, but not the configuration/control. Seems backwards, as many systems are the other way around; DB with complex configs but poor result management, logs, etc.
Regarding your example of local configs, yes, somewhat similar, but everyone works hard to avoid that, drive from templates, puppet, etc. and validate with databases if they can. To purposely build a centralized system with no DB just strikes me as very challenging, very prone to error, hard to verify/crosscheck, update, etc.
But, that’s the way it is – do you or anyone know of anyone who worked on generating these files from a master DB for central Bacula control, verification, etc. ?