Bacula® Developer’s Guide Kern Sibbald May 29, 2021 This manual documents Bacula Community Edition 11.0.4 (29 May 2021) Copyright © 1999-2021, Kern Sibbald Bacula® is a registered trademark of Kern Sibbald. This Bacula documentation by Kern Sibbald with contributions from many others, a complete list can be found in the License chapter. Creative Commons Attribution-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-sa/4.0/ Developer’s Guide 2/146 Bacula Community Edition v.11.0.4 (29 May 2021) All trademarks are the property of their respective owners CONTENTS Developer’s Guide Contents 1 Bacula Developer Notes1 1.1 Contributions.......................................1 1.2 Patches...........................................1 1.3 Copyrights.........................................2 1.4 Copyright Assignment – Fiduciary License Agreement................2 1.5 The Development Cycle.................................3 1.5.1 Feature Requests.................................3 1.6 Bacula Code Submissions and Projects.........................5 1.7 Patches for Released Versions..............................5 1.8 Developing Bacula.....................................6 1.9 Debugging.........................................7 1.10 Using a Debugger.....................................7 1.11 Memory Leaks.......................................7 1.12 Special Files........................................8 1.13 When Implementing Incomplete Code.........................8 1.13.1 Bacula Source File Structure..........................8 1.14 Header Files........................................9 1.15 Programming Standards.................................9 1.16 Do Not Use........................................ 10 1.17 Avoid if Possible...................................... 10 1.18 Do Use Whenever Possible................................ 10 1.19 Indenting Standards.................................... 11 1.20 Naming Convention.................................... 12 1.21 Locks and Threads.................................... 13 1.22 Tabbing........................................... 14 Bacula Community Edition v.11.0.4 (29 May 2021) i/146 All trademarks are the property of their respective owners Developer’s Guide CONTENTS 1.23 Don’ts........................................... 14 1.24 Message Classes...................................... 15 1.25 Debug Messages...................................... 15 1.25.1 Debug Tags.................................... 16 1.26 Error Messages...................................... 16 1.27 Job Messages....................................... 17 1.28 Queued Job Messages.................................. 17 1.29 Memory Messages..................................... 17 1.30 Bugs Database....................................... 18 2 Bacula Git Usage 19 2.1 Bacula Git repositories.................................. 19 2.2 Git Usage.......................................... 19 2.2.1 Learning Git.................................... 20 2.3 Step by Step Modifying Bacula Code.......................... 21 2.3.1 More Details................................... 23 2.4 Forcing Changes...................................... 24 3 Bacula FD Plugin API 27 3.1 Normal vs Command vs Options Plugins........................ 27 3.2 Loading Plugins...................................... 28 3.3 loadPlugin......................................... 29 3.4 Plugin Entry Points.................................... 31 3.4.1 newPlugin(bpContext *ctx)........................... 31 3.4.2 freePlugin(bpContext *ctx)........................... 31 3.4.3 getPluginValue(bpContext *ctx, pVariable var, void *value)........ 32 3.4.4 setPluginValue(bpContext *ctx, pVariable var, void *value)......... 32 3.4.5 handlePluginEvent(bpContext *ctx, bEvent *event, void *value)...... 32 3.4.6 startBackupFile(bpContext *ctx, struct save_pkt *sp)............ 34 3.4.7 endBackupFile(bpContext *ctx)........................ 35 3.4.8 startRestoreFile(bpContext *ctx, const char *cmd)............. 35 3.4.9 createFile(bpContext *ctx, struct restore_pkt *rp).............. 35 3.4.10 setFileAttributes(bpContext *ctx, struct restore_pkt *rp).......... 36 3.4.11 endRestoreFile(bpContext *ctx)........................ 37 ii/146 Bacula Community Edition v.11.0.4 (29 May 2021) All trademarks are the property of their respective owners CONTENTS Developer’s Guide 3.4.12 pluginIO(bpContext *ctx, struct io_pkt *io)................. 37 3.4.13 bool checkFile(bpContext *ctx, char *fname)................. 38 3.5 Bacula Plugin Entrypoints................................ 38 3.5.1 bRC registerBaculaEvents(bpContext *ctx, ...)................ 38 3.5.2 bRC getBaculaValue(bpContext *ctx, bVariable var, void *value)..... 38 3.5.3 bRC setBaculaValue(bpContext *ctx, bVariable var, void *value)..... 39 3.5.4 bRC JobMessage(bpContext *ctx, const char *file, int line, int type, utime_t mtime, const char *fmt, ...)...................... 39 3.5.5 bRC DebugMessage(bpContext *ctx, const char *file, int line, int level, const char *fmt, ...)............................... 39 3.5.6 void baculaMalloc(bpContext *ctx, const char *file, int line, size_t size). 39 3.5.7 void baculaFree(bpContext *ctx, const char *file, int line, void *mem).. 39 3.6 Building Bacula Plugins................................. 39 3.7 Advanced Restore Options................................ 40 3.8 Bacula Auth Plugin Documentation........................... 41 3.8.1 Overview...................................... 41 3.8.2 Dictionary..................................... 41 3.8.3 Bacula DIR Plugin API............................. 41 3.8.4 Bacula Pluggable Authentication Modules API Framework BPAM].... 46 4 Platform Support 53 4.1 General........................................... 53 4.2 Requirements to become a Supported Platform.................... 53 5 Daemon Protocol 55 5.1 General........................................... 55 5.2 Low Level Network Protocol............................... 55 5.3 General Daemon Protocol................................ 55 5.4 The Protocol Used Between the Director and the Storage Daemon......... 56 5.5 The Protocol Used Between the Director and the File Daemon........... 56 5.6 The Save Protocol Between the File Daemon and the Storage Daemon...... 57 5.6.1 Command and Control Information....................... 57 5.6.2 Data Information................................. 57 6 Director Services Daemon 59 Bacula Community Edition v.11.0.4 (29 May 2021) iii/146 All trademarks are the property of their respective owners Developer’s Guide CONTENTS 7 File Services Daemon 61 7.1 Commands Received from the Director for a Backup................. 61 7.2 Commands Received from the Director for a Restore................. 62 8 Storage Daemon Design 63 8.1 SD Design Introduction.................................. 63 8.2 SD Development Outline................................. 63 8.3 SD Connections and Sessions.............................. 64 8.3.1 SD Append Requests............................... 64 8.3.2 SD Read Requests................................ 65 8.4 SD Data Structures.................................... 65 9 Catalog Services 67 9.1 General........................................... 67 9.1.1 Filenames and Maximum Filename Length.................. 67 9.1.2 Installing and Configuring MySQL ........................ 68 9.1.3 Installing and Configuring PostgreSQL .................... 68 9.1.4 Internal Bacula Catalog............................. 68 9.1.5 Database Table Design............................. 68 9.2 Sequence of Creation of Records for a Save Job.................... 68 9.3 Database Tables...................................... 69 10 Storage Media Output Format 81 10.1 General........................................... 81 10.2 Definitions......................................... 81 10.3 Storage Daemon File Output Format.......................... 83 10.4 Overall Format....................................... 83 10.5 Serialization........................................ 83 10.6 Block Header....................................... 83 10.7 Record Header....................................... 84 10.8 Version BB02 Block Header............................... 85 10.9 Version 2 Record Header................................. 85 10.10Volume Label Format................................... 85 10.11Session Label....................................... 86 iv/146 Bacula Community Edition v.11.0.4 (29 May 2021) All trademarks are the property of their respective owners CONTENTS Developer’s Guide 10.12Overall Storage Format.................................. 87 10.13Unix File Attributes.................................... 90 10.14Old Depreciated Tape Format.............................. 91 11 Bacula Porting Notes 97 11.1 Porting Requirements................................... 97 11.2 Steps to Take for Porting................................. 98 11.1 General........................................... 101 11.1.1 Minimal Code in Console Program....................... 101 11.1.2 GUI Interface is Difficult............................. 101 11.2 Bvfs API.......................................... 102 12 TLS 107 12.1 Introduction to TLS.................................... 107 12.2 New Configuration Directives.............................. 107 12.3 TLS API Implementation................................. 108 12.3.1 Library Initialization and Cleanup........................ 108 12.3.2 Manipulating TLS Contexts........................... 108 12.3.3 Performing Post-Connection Verification................... 109 12.3.4 Manipulating TLS Connections........................