Using GNU Make for Neuroimaging Workflow University of Washington Using GNU make for Neuroimaging Workflow Mary K. Askren∗, Trevor K. McAllister-Day, Natalie Koh, Zoé Mestre, Jennifer N. Dines, Benjamin A. Korman, Susan J. Melhorn, Daniel J. Peterson, Matthew Peverill, Swati D. Rane, Melissa A. Reilly, Maya A. Reiter, Kelly A. Sambrook, Karl A. Woelfer, Xiaoyan Qin, Thomas J. Grabowski, and Tara M. Madhyastha† University of Washington, Seattle May 26, 2016 ∗[email protected] †[email protected] i Contents Contents ii List of Figures vi I Manual1 1 Introduction to make 2 Conventions Used in This Manual ...................... 3 Quick Start Example.............................. 3 A More Realistic Example........................... 5 What is the Difference Between a Makefile and a Shell Script?...... 7 Make Built-In Functions............................ 8 The shell and wildcard Functions .................. 8 Text and Filename Manipulation.................... 8 Logging Messages from make ...................... 10 Additional Resources For Learning About make . 11 2 Running make in Context 13 File Naming Conventions ........................... 13 Subject Identifiers ............................ 13 Filenames................................. 15 Directory Structure............................... 16 Setting up an Analysis Directory....................... 18 Defining the basic directory structure ................. 18 Creating the session-level makefiles................... 19 Creating the common subject-level makefile for each session . 20 Creating links to the session-level makefile............... 21 Creating links to subject-level makefile................. 21 Running analyses............................. 21 Setting Important Variables.......................... 22 ii Contents Variables that control make’s Behavior................. 22 Other important variables........................ 23 Variable overrides............................. 24 Suggested targets............................. 25 3 Running make in Parallel 27 Guidelines for Writing Parallelizable Makefiles ............... 27 Each line in a recipe is, by default, executed in its own shell . 27 Filenames must be unique........................ 28 Separate time-consuming tasks ..................... 29 Executing in Parallel.............................. 30 Using multiple cores ........................... 30 Using the grid engine.............................. 30 Setting FSLPARALLEL for FSL jobs ................... 31 Using qmake ................................ 31 How long will everything take? ..................... 32 Consideration of memory ........................ 32 Troubleshooting make ............................. 33 Find out what make is trying to do................... 33 Use the trace option in make 4.1 .................... 33 Check for line continuation ....................... 34 No rule to make target! ......................... 34 Suspicious continuation in line # .................... 35 make keeps rebuilding targets...................... 35 make doesn’t rebuild things that it should............... 35 4 Recipes 36 Obtaining a List of Subjects.......................... 36 Setting a variable that contains the subject id ............ 37 Using Conditional Statements......................... 37 Setting a conditional flag ........................ 37 Using a conditional flag ......................... 38 Conditional execution based on the environment ........... 38 Conditional execution based on the Linux version........... 39 II Practicals 41 1 Overview of make 43 Lecture: The Conceptual Example of Making Breakfast.......... 43 Practical Example: Skull-stripping...................... 45 iii Contents Manipulating a single subject...................... 45 Pattern rules and multiple subjects................... 47 Phony targets............................... 48 Secondary targets............................. 49 make clean ................................ 50 2 make in Context 52 Lecture: Organizing Subject Directories................... 52 Practical Example: A More Realistic Longitudinal Directory Structure . 53 Organizing Longitudinal Data...................... 53 Recursive make .............................. 55 Running make over multiple subject directories ............ 56 Running FreeSurfer............................ 57 3 Getting Down and Dirty with make 58 Practical Example: Running QMON..................... 58 Practical Example: A Test Subject...................... 59 Estimated total intracranial volume .................. 61 Hippocampal volumes .......................... 63 Implementing a Resting State Pipeline: Converting a Shell Script into a Makefile.................................. 64 4 Advanced Topics & Quality Assurance with R Markdown 68 Creating a make help system ......................... 68 Makefile Directory Structure ......................... 69 The clean target................................ 70 Creating New Makefile Rules On The Fly.................. 71 Incorporating R Markdown into make .................... 71 III Examples 77 1 Downloading Data From XNAT 79 2 Running FreeSurfer 82 3 DTI Distortion Correction with Conditionals 85 4 Quantifying Arterial Spin Labeling Data 90 5 Processing Scan Data for a Single Test Subject 95 Testsubject Main Makefile........................... 95 iv Contents Testsubject FreeSurfer............................. 99 Testsubject Transformations.......................... 101 Testsubject QA Makefile............................ 103 6 Preprocessing Resting State Data 108 7 Generating A Methods Section 114 8 Seed-based Functional Connectivity Analysis I 117 9 Seed-based Functional Connectivity Analysis II 119 Group Level Makefile - tapping/Makefile . 119 Subject Level Makefile - tapping/lib/resting/subject.mk . 120 Preprocessing - tapping/lib/resting/Preprocess.mk . 121 Generation of Nuisance Regressors - tapping/lib/resting/Regressors.mk123 ROI Timeseries Extraction - lib/resting/timeseries.mk . 125 First Level Analysis - lib/resting/fsl.mk . 126 Quality Assurance Reports - lib/resting/qa.mk . 126 10 Using ANTs Registration with FEAT 128 Group Level Makefile.............................. 128 Subject Level Makefile............................. 129 Preparatory Registrations - Prep.mk . 130 Running Feat and Applying ANTs Registrations - Feat.mk . 131 11 Creating Result Tables Automatically Using Make 135 Simple Result Tables.............................. 135 Multiple Group Analyses ........................... 136 12 Plotting Group FEAT Results Against Behavioral Measures 138 Bibliography 146 Other Resources ................................ 147 v List of Figures 1.1 Basic make syntax. ............................. 4 1.2 A very basic make recipe. .......................... 4 1.3 Automatic make variables.......................... 5 1.4 A more realistic example........................... 5 1.5 An expansion of Figure 1.4 ......................... 6 1.6 A makefile expressed in bash ........................ 7 1.7 make filename manipulation functions ................... 9 1.8 How to use subst .............................. 9 1.9 How to use word ............................... 10 1.10 Using strip .................................. 10 1.11 make filename manipulation functions ................... 11 1.12 make filename manipulation functions ................... 11 2.1 Example of good file naming conventions ................. 15 2.2 An example of a project directory. ..................... 17 2.3 A longitudinal analysis directory....................... 19 2.4 Session-level Makefile ............................ 20 2.5 Subject-level makefile ............................ 20 2.6 Creating symbolic links to the session-level makefile ........... 21 2.7 Making test ................................. 21 2.8 Output of recursive make .......................... 22 2.9 Setting $(SHELL) ............................... 22 2.10 Running DTIPrep in make ......................... 24 2.11 Controlling software version......................... 24 2.12 Specifying BET flags in make........................ 25 3.1 A non-functional multi-line recipe ..................... 28 3.2 A now-functional multi-line recipe ..................... 28 3.3 A multi-line recipe using “&&\”....................... 28 3.4 How not to name files ............................ 28 vi List of Figures 3.5 The wrong way to run two long tasks ................... 29 3.6 The right way to run two long tasks .................... 29 3.7 Pattern-matching error handling ...................... 35 4.1 Obtaining a list of subjects from a file.................... 36 4.2 Obtaining a list of subjects using a wildcard. ............... 37 4.3 Determining the subject name from the current working directory and a pattern..................................... 37 4.4 Setting a variable to determine whether a FLAIR image has been acquired. 38 4.5 Setting a variable to indicate the study site................. 38 4.6 Testing the site variable to determine the name of the T1 image. 39 4.7 Testing to see whether an environment variable is undefined . 39 4.8 /etc/os-release .............................. 40 4.9 Testing the Linux version to determine whether to proceed . 40 1.1 Creating a waffwich ............................. 44 1.2 How to make a waffwich........................... 45 1.3 Pattern-matching as a sieve......................... 48 2.1 Script to create symbolic links within a longitudinal directory structure 54 3.1 resting-script file............................. 66 4.1 make’s Help System ............................. 69 4.2 Example of a R Markdown File......................