Bacula Open Source Project Bacula Systems (professional support)

The Enterprise Ready Open Source Network Backup Soluon

Kern Sibbald

Bacula Systems SA Introduction – What is Bacula?

• A network backup soluon designed for *BSD, Linux, Mac OS X, Unix and Windows systems • Project goals are to:  Backup any client from a Palm to a mainframe computer  Provide «Enterprise» features similar to the largest commercial applicaons in an Open Source soluon  Assure data compability for 30 years (providing that appropriate hardware is available)  Use a Free and Open Source (GPLv2) license

Bacula Systems SA Page 2 Introduction – Project History

• «Bacula» = Backup + Dracula  It comes by night and sucks the vital essence from your computers. • Time line  January 2000 – Project started  14 April 2002 – First release to Source Forge (version 1.16)  29 June 2006 – Release 1.38.11  January 2007 – Release 2.0.0  August 2007 – Release 2.2.0  June 2008 – Release 2.4.0  July 2009 – Current release 3.0.2 • Downloads to date: (Sep 2009)  1,027,109 all versions  6.9 TB

Bacula Systems SA Page 3 Bacula Download History

Release 3.0.2

Bacula Systems SA Page 4 Why Backup?

• Hardware Failure Protecon • Unauthorized Intrusion (Hackers) • Virus Protecon • User Error Protecon • Service Level Agreements • Legal and/or Regulatory Requirements • Business Cost of Time/Data Loss • Disaster Recovery

Bacula Systems SA Page 5 What is «Backup»?

• Data protecon  All files  Databases (MySQL, PostgreSQL, MSSQL, Oracle, DB2, …)  Applicaons (E‐mail: MS Exchange, Lotus, ...)  Groupware applicaons (Lotus Notes, etc.)  Enterprise Resource Planning (ERP)  CRM and sales order entry systems  Others…

Bacula Systems SA Page 6 What is «Backup»?

• Off‐site backup  Storing copies off‐site • Disaster recovery plan (DRP)  Bare Metal recovery  Server configuraon  Bacula Catalog database  Bacula binaries and configuraon files  Key people to be contacted  Locaon of recovery media

Bacula Systems SA Page 7 Module 1 What is «Backup»?

• Archiving  Compliance with regulatory requirements  Business requirements  Long retenon (7‐30 years)  Guarantees data are not changed or deleted  Audit requirements

Bacula Systems SA Page 8 Reflection – Key Questions

• Do you do backups?  No  Yes, I did one last month  Yes, tar files every week  Somemes I rsync ...  Yes, CDs every week  I use custom scripts • Have you encountered the typical problems?  Retrieving and restoring: • How to find the files you need to restore? • How to restore to a point in me? • Which files are on which medium?  Scalability: • Could the soluon handle 2000+ machines?  Compliance: • Can the soluon assure that all government regulaons are met?

Bacula Systems SA Page 9 Introduction – the Bacula Solution

• Open Source (GPLv2) • Centrally managed • Network backup/restore • Mul‐plaorm support (*BSD, Linux, Mac OS X, Unix Win32, ...) • Different media (Tape, disk, USB, CD/DVD) • Reliable • Tracks comprehensively what was backed up when and where • Allows restoraon of specific files required (Catalog + GUI) • Restores to a point in me • Scales to handle 10,000 machines

Bacula Systems SA Page 10 What is Bacula?

• Enterprise backup soluon  Network based  Centralized Administraon  Automac operaon  Performs bookkeeping  Mul‐plaorm  Many high‐end features • Robust design built for high performance  Modular  Mulple Backup Media  Customizable  Reliable  Rapid Restores  Advanced Reporng, Noficaon, Monitoring

Bacula Systems SA Page 11 Module 1 What is Bacula?

• Open Source  Zero‐cost license  Large community of users and developers  Ability to customize to your specific requirements • Extensive configuraon • Contribute to Bacula Project  Security of soluon guaranteed • Users have the source code • No license audits

Bacula Systems SA Page 12 Module 1 Architecture – 5 Bacula Components

Bacula Systems SA Page 13 Architecture – 5 Bacula Components

• Director (DIR)  Centralized control and administraon for all funcons  Basic unit is a Job (one client, one set of files, ...)  Schedules, iniates and supervises all Jobs  Maintains the catalog (SQL database) • Typical installaon  One Director per Bacula installaon  Excepon – very large environments that require addional coordinaon

Bacula Systems SA Page 14 Architecture – 5 Bacula Components

• File Daemon or Client (FD)  Performs file backup, restore and verificaon requested by Director  Installed on each machine as a service (daemon)  Communicates over network with Director and Storage daemon  Needs access to all files to be backed up (root, SYSTEM) • Typical installaon  Mulple File daemons per Director • One for each machine

6.3 Bacula Systems SA Page 15 Architecture – 5 Bacula Components

• Storage daemon (SD)  Reads and writes data to the physical medium  Disk, Tape, CD/DVD, USB, ...  Accepts orders and authorizaon from the Director  Accepts and returns data to/from File daemons (FD)  Sends file storage locaon to Director ‐> Catalog  Supports mulple devices per SD • Typical installaon  One SD per Director

Bacula Systems SA Page 16 Architecture – 5 Bacula Components

• Console  Allows user or administrator to control Bacula  Communicates with Director via network  Start jobs, review Job output, query/modify catalog • Consoles available  TTY (bconsole)  bat a Qt 4 (GUI) – most comprehensive  wxWidgets (GUI) – Linux, Unix, Win32  Mulple web interfaces • bweb = most comprehensive  Restricted consoles • Users can restore their own files

Bacula Systems SA Page 17 Architecture – 5 Bacula Components

• Catalog database  Only component not wrien by Bacula team  SQL database (MySQL, PostgreSQL, or SQLite) – unique  Tracks Jobs run, Volumes used, File locaons, ...  Permits rapid restores  Allows inquiry of when and where files were backed up  Old data automacally pruned by Director  Supports mulple databases for scaling

Bacula Systems SA Page 18 Bacula Release Files

• bacula‐3.0.2.tar.gz • bacula‐rescue‐3.0.2.tar.gz  Primary Bacula source code  Bare Metal Recovery release  CD‐ROM depreciated • bacula‐docs‐3.0.2.tar.bz2  Replaced by USB key  Prebuilt docs: HTML directory, single HTML file, and PDF file.. • winbacula‐3.0.2.exe  32‐bit installer for • bacula‐gui‐3.0.2.tar.gz Windows® client (File  Non‐core GUI programs Daemon)  bweb • win64bacula‐3.0.2.exe  bacula‐web  64‐bit installer for  Bimagemgr Windows® client (File • program for burning Daemon) CDROM images with Bacula Volumes

Bacula Systems SA Page 19 Bacula Administration Tool (BAT)

• GUI form of bconsole – with many addional features • Key features  Graphical representaon of Jobs run by Bacula  Ability to create reports more easily and quickly  Sorng of run/failed Jobs by clicking on the column header  Context sensive menu with ready access to key features

Bacula Systems SA Page 20 The BAT Interface

Bacula Systems SA Page 21 The BAT Interface – Job List

Bacula Systems SA Page 22 The BAT Interface – Job Log

Bacula Systems SA Page 23 The BAT Interface – Volume Actions

Bacula Systems SA Page 24 Module 4 The BAT Interface – Running a Job

Bacula Systems SA Page 25 Module 4 The BAT Interface – Restoring Files

Bacula Systems SA Page 26 Bweb Management Interface

• Web based management tool • Key features  Graphical representaon of Jobs run by Bacula  Ability drill into problem areas  Restart failed jobs  Point and click autochanger control  Volume locaon tracking  Job grouping

Bacula Systems SA Page 27 Main Bweb Window

Bacula Systems SA Page 28 Job Log for Failed Job

Bacula Systems SA Page 29 Job Log for Job with Warnings

Bacula Systems SA Page 30 Media View (in Catalog)

Bacula Systems SA Page 31 Autochanger View (in changer)

Bacula Systems SA Page 32 Bacula Reliability

• Reliability designed in from the beginning • Low level memory clobber and buffer loss checking • Own code for vprin – avoids exploits • 10 – 20% of code cleaned up (refactored) each release • Extensive regression tesng • User regression tesng posted on dashboard nightly from many machines

Bacula Systems SA Page 33 Bacula Test Dashboard

Bacula Systems SA Page 34 Failed Job

Bacula Systems SA Page 35 Failed Job Details

Bacula Systems SA Page 36 Open Bugs Database

• Bugs database open to everyone • Can find fixes to unusual problems • hp://bugs.bacula.org

Bacula Systems SA Page 37 Bacula Bugs Database

Bacula Systems SA Page 38 Bacula Bugs Database

Bacula Systems SA Page 39 Development Roadmap

• Version 3.0.3 bug fix to be released by mid‐October • Version 3.2.0 to be released aer 1 January 2010 • New 3.2.0 features  Maximum Concurrent Jobs for Devices  Enhancements to Accurate backup  Restore from mulple Storage daemons  File deduplicaon with Base Jobs  Bacula virtual catalog filesystem API  Enhanced tape drive tesng funconality

Bacula Systems SA Page 40 Development Roadmap (cont)

• New 3.2.0 features  Disable block checksums (performance)  New Bat funconality  Restart failed Jobs recovering what was already save  File daemon drops root permission but can backup all files

Bacula Systems SA Page 41 Future Projects

• Deduplicaon  Base Job deduplicaon  “rsync” to server tree  Rdiff using precomputed hash codes  Laptop backup (similar to BackupPC) • Archive  Jobs kept forever  Paroned catalog  Move catalog data to another database • Management Console (perhaps bweb)  Control of mulple Directors • Write to mulple Storage daemons

Bacula Systems SA Page 42 Resources – Email lists

• bacula‐users  General forum to ask or respond to quesons about Bacula from other users. • All users are encouraged to subscribe and parcipate  Sign up at: hp://lists.sourceforge.net/lists/lisnfo/bacula‐users  Once subscribed, send email to the Bacula Users List at: • bacula‐[email protected] • bacula‐devel  For communicaon directly with developers  Sign up at: hp://lists.sourceforge.net/lists/lisnfo/bacula‐devel  Email address: bacula‐[email protected]

Bacula Systems SA Page 43 Typical Open Source Resources

• Bugs database (hp://bugs.bacula.org) • Bacula Wiki (hp://wiki.bacula.org) • Bacula Email Lists  hp://www.bacula.org ➯ Email Lists • Recommended: bacula‐announce  Sign up at: hp://lists.sourceforge.net/lists/lisnfo/bacula‐announce  Read‐only low volume list for users who wish to be nofied of important events (bugs, new releases, ...) • For email noficaon of new package release  Sign up at: hp://sourceforge.net/projects/bacula • Click on Download, then Monitor next to desired package

Bacula Systems SA Page 44 Resources – Email lists

• Recommended: bacula‐bugs (Noficaon about new bugs)  hp://bugs.bacula.org  Sign up at: hp://lists.sourceforge.net/lists/lisnfo/bacula‐bugs • bacula‐beta (Noficaon of new beta releases)  Sign up at: hp://lists.sourceforge.net/lists/lisnfo/bacula‐beta • Bacula‐commits (diff of each commit to the SVN)  Normally used only by developers • Sign up at: hp://lists.sourceforge.net/lists/lisnfo/bacula‐commits • Archives (Available on Source Forge, and Gmane)  hp://news.gmane.org/search.php?match=bacula

Bacula Systems SA Page 45 Resources

• Support opons  Professional support – from Bacula Systems or authorized partners  Other professional support  Free support – from the Bacula community • Bacula Systems Support  Professional support on a fee‐paid basis: • hp://www.baculasystems.com • Bacula Community Support • On an as‐available basis (users), by email:  bacula‐[email protected]

Bacula Systems SA Page 46 Bacula Systems Support

• If the code is free why do I need a support contract?  Insurance policy in case of problems  Ticket system  Guaranteed response to quesons and problems  Priority bug fixing  Funded development  Configuraon verificaon  Advanced informaon on development projects  Input to development project priories  Direct contact with the Bacula developers  Bacula Enterprise Edion binaries

Bacula Systems SA Page 47 Thank You

Bacula Systems SA Rue Galilée 5

CH-1400 Yverdon-les-Bains

Switzerland T: +41 24 420 7890 F: +41 24 420 7891 E: [email protected] W: www.baculasystems.com German Branch Office French Branch Office Bacula Systems S.A. Bacula Systems S.A. Sandstrasse 6 18 rue de l'Eglise DE-49080 Osnabrück FR-14440 Douvres La Délivrande Germany France T: +49 (0) 541 75 04 24 01 T: +33 (0) 231 97 33 78

Bacula Systems SA Page 48