Introduction to IDL

Mark Fardal

Programming Languages

● Lower­level: C, C++, Fortran – Closer to “machine level” – Strong and static variable typing ● Higher­level: Matlab, Python, IDL – More powerful basic syntax – Interactive – Slower number­crunching, not as structured – not as good for big projects – IDL, Matlab not free ● Non­languages: ds9, GAIA, gimp, imagemagick... ● Symbolic mathematics: Mathematica, Maple, Matlab ● IDL advantages – Commonly used in astronomy – Lots of astronomy­oriented routines – Visualization, documentation neatly packaged

IDL features

● Interactive tool, but also full programming language on its own: – structured programming (functions and procedures) – array­oriented: operations on arrays faster and easier than doing a loop – many numeric types; strings – flow control constructs – semi­automatically quasi­compiled – garbage collection to reclaim memory – program structure workarounds: system variables, common blocks – structures, objects ● Three graphic models in IDL 8.x – direct graphics: 2­d, 3­d plots (contours, images, surface plots, points, volumes), cursor interaction, lots of ways to customize IDL> plot, x, y – direct graphics plot objects: IDL> p = plot(x,y) – object graphics (more advanced, not really for individual use)

IDL modes of operation

● Lots of ways to use it: – Vanilla: bash> idl – Integrated development environment: bash> idlde – Within emacs: idlwave mode (idlde replacement) ● Running IDL code – type it in (or cut+paste) – batch file: idl labscript (can omit the .pro) – main program: IDL> .run labmain – calling procedures and subroutines ● Main program format idl_statement idl_statement idl_statement end

Getting started with IDL

● Basic setup: terminal + text editor – alternative setups: idlde, or else idl shell within emacs ● IDL setup: (depends on your shell, type printenv $shell to find out which one you're using); can replace “cp” step with copy and paste U:~> cd U:~> cp ­r ~fardal/a335 . U:~> cd a335 U:~> source setupidl.bash

● Start IDL for /bin/tcsh or /bin/csh: U:~> idl Replace last step with U:~> source setupidl.tcsh ● Now type:

IDL> print, 2+2

IDL as a calculator

● Some examples:

IDL> print, 13 * 15 IDL> print, 23. / 2 IDL> print, 6.626e­27 * 3.29e15 IDL> print, exp(10.) IDL> print, alog10(30.0)

● A couple ways to do a sum:

IDL> print, 1 + 2 + 3 + 4 + 5 IDL> print, total( [ 1, 2, 3, 4, 5 ] )

What IDL does when starting

● Checks for license ● Runs commands in IDL startup file ● Sets up “path” – In this case we added to path in the startup file – To see the current path, type – IDL> print, !path – Or now that your path IS set up: – IDL> printpath ● Recommend organizing your own code in 2­3 locations, use path to make sure it's found

IDL as a calculator

● Some examples:

IDL> print, 13 * 15 IDL> print, 23. / 2 IDL> print, 6.626e­27 * 3.29e15 IDL> print, exp(10.) IDL> print, alog10(30.0)

● A couple ways to do a sum:

IDL> print, 1 + 2 + 3 + 4 + 5 IDL> print, total( [ 1, 2, 3, 4, 5 ] )

Some basic features

● Case insensitive ● Comment character is ; (semi­colon) ● Statements ended by return character; for longer lines use continuation character $ ● Language help: ? (but web help might be faster) ● Session help: help ● Quitting: exit or Ctrl­D ● Operating system access: $ at beginning of line – single line like $ls *.pro, or just typing $ starts up an “internal” shell ● Try copy and paste from lab_start.pro

Constants, variables, math

● Number types – a = 1 integer: 16 bits, ­32767 to +32767 – a = 1L long: 32 bits, ­2147483647L to 2147483647L – a = 1.0 float: 32 bits, sig decimal digits, magnitude 1e­38 to 1e38 – a = 1.d double: 64 bits, ~16 decimal digits, magnitude 1d­308 ● Math operators + ­ * / ^ mod < > – Watch out for integer arithmetic: 2 / 3 is zero ● Math functions exp alog alog10 ... ● IDL's variables are dynamic – IDL> a = 30 – IDL> b = a – IDL> a = “Al Capone”

Some basic features

● Interact by IDL by series of “statements” – IDL> a = 10^2 – IDL> b = exp(30.) – IDL> print, a, b ● Not case sensitive ● Get help on language by typing ? ● Get help on current session by typing help ● Quitting: exit or Ctrl­D ● To interrupt a rogue program type Ctrl­C ● Operating system access: $ at beginning of line – single line, or just $ starts up an “internal” shell

IDL Routine Library

● 3D Visualization ● Mapping ● Array Creation ● Mathematics ● Array Manipulation ● Operating System Access ● Color Table Manipulation ● Plotting ● Debugging ● Programming and IDL Control ● Dialog Routines ● Query Routines ● Direct Graphics ● Saving/Restoring a Session ● Error Handling ● Scientific Data Formats ● Executive Commands ● String Processing ● Help Routines ● Structures ● Image Processing ● Type Conversion ● Input/Output ● and many add­on libraries

Arrays

● Array = bunch of numbers treated as single expression ● To make arrays: – x = [0,3,7,2] – intarr or indgen for integer Matlab to IDL: – fltarr or findgen for float a = [1 2 3] a=[1,2,3] a[1] a[0] – intarr form gives array filled with zeros; zeros(1,5) fltarr(5) indgen form gives [0,1,2,3...] 0:6 indgen(7) [1 2 3 ; 4 5 6] [[1,2,3],[4,5,6]] ● Can use operators on arrays – a = [1,3,5] b = [4,6,8] c = a + b ● ImportantImportant: first element is a[0] not a[1] ● Many useful array­related routines: min max where sort ● Multidimensional arrays too

More on arrays

● Basic subscripting – a[0:2] first three elements – a[3:*] fourth to end – a[*,0] all elements of a's first row – a[1,*] all elements of a's second column – can always subscript with single index ­ use reverse­odometer order ● Subscripting with variables – a = findgen(10)+10 – set = [3,4,5] – print, a[set] ● The all­important “where” function – set = where(a gt 4.5) ● Try lab_arrays

2­d plotting ● Basic plot: given arrays x, y plot, x, y ● Many aspects of plots customized by keywords – plot, sin(d/10), psym=4, ytitle='Response (keV)' ● Shorthand for log/linear axes: plot_io, x, y Matlab to IDL: plot_oi, x, y plot(x, y,'d') plot, x, y, psym=4 plot_oo, x, y ● Shorthand for overplotting oplot, x, y or oplot, x, y, psym=5, col=!blue ● Plots written to “devices” – X for screen plot – PS for postscript file – help, /device tells which device we're in – many device details customizable: color options, paper size... ● See lab_plot (histograms, fancy axes, etc)

Coyote Graphics system

● Included in Coyote library in the $HOME/REU/idl_lib directory ● Already used the PS_START and PS_END routines ● Basic plot: given arrays x, y cgplot, x, y, /window ● Then you can expand, shrink, save as postscript, image, data + commands... ● Recommend using this for colors, images, surface plots, contour plots...

Functions and Procedures

● Many functions and procedures built in but you can add your own ● Using these promotes code re­use, helps debugging, saves sanity of yourself and others ● Functions: – Declare this way: function circumference, radius – Invoke this way: c = circumference(radius) – Must return a value return, 2.*!dpi*radius ● Procedures: – Declare this way: pro dostuff, a – Invoke this way: dostuff, a – Don't return a value (but can return things through argument list) ● Either one can modify any variables you send as arguments ● See lab_functions

Hey IDL: what's going on?

Print, a Help, a Help, /trace Help, /source Retall Close, /all .reset_session

● Controlling IDL execution: .run (.c), stop, retall, .c(ontinue)

FITS files / image display

● FITS files are astronomy­specific format ● Multiple routines to read and write, see link on help page ● Here is one way to read in and display:

IDL> im = readfits('gopal.fits')

Plotting it:

IDL> implot = bytscl( alog(im>100<1000) ) IDL> cgloadct, 0 IDL> cgimage, implot

● Also other data formats such as HDF files ● For large files: use binary I/O with arrays of numbers/structures ● See lab_fits.pro

Structures

● Special variables that package unlike items together – Good for grouping related variables – Makes passing arguments much simpler ● Syntax: – IDL> tuning = {note, pitch:'A', freq:'440.'} – IDL> print, tuning.freq ● See lab_structures

Reading and writing files

● Try lab_read – uses intrinsic routine to read table of numbers ● generic input/output steps:

get_lun openr / openw readf / printf close free_lun

● But for reading tables, it's easier to use a canned routine, for example readcol in the astron library ● journal is simple way to record both your code and the output

Beyond the command line

● Code lives in files, usually with suffix .pro ● IDL compiles code once before using ● Compilation normally automatic ● Usual strategy to force compilation: put each function or subroutine in its own file ● If you change the code, IDL will notnot go recompile on its own! ● Can force recompilation with “.compile file” or “.run file” ● Comment character is semi­colon ; ● Statements ended by return character; for longer lines use continuation character $

Logic

● Relational operators: eq lt le gt ge ● Logical operators: && || ~ ● Bitwise operators: and or not xor ● If statements – One line: if (a gt 0) then print, 'positive' else print, 'negative' – Block form: note the begin and endif and endelse

if (a gt 0) then begin print, 'positive' b = a*2 endif else if (a eq 0) then begin print, 'zero' b = a+1 endif else begin print, 'negative' b = ­a*3 endelse

● See lab_logic

Loops

● “For” loop – One line form: for k = 0L, 7 do print, k^2 – And block form: note endfor

for i=0L,10 do begin x[i] = a c = a+b a = b b = c endfor

– Run over range 0, n_elements(x)­1 for array indexing – Initialize to 0L to use longs, avoid wraparound at 32k

● Other flow control constructs: “While”, “until”, “case” ● Try lab_forloops

Other IDL syntax you may see Structures: tuning = {note, pitch:'A', freq: '440.'}

Objects: c = obj_new('sedcalculator') c­>set_temp, 45. flux = c­>getset(wavelength)

System variables: print, sin(!dpi) plot, !x.crange, !y.crange

Pointers: x = ptr_new([1,2,3])

help, x, *x IDL Gotchas

● Integer division: avoid it ● Integers are too short! use longs instead, especially in loops ● Confusion between main programs and procedures ● Confusion between procedures and functions ● IDL's confusion between functions and arrays (soln: compile in order) ● Path problems ● Auto­compilation problems (partial soln: put program unit myprog in a file myprog.pro, stick any internal helper routines at the top) ● See also online tutorials, David Fanning's www.idlcoyote.com, www.astro.umass.edu/~fardal/idlsummer and resources there

Generic programming advice

● Make your code document itself – Meaningful routine and variable names – Named constants – Write pseudocode program, then turn it into comments as you go – Finally be careful to document inputs and outputs ● Defensive programming ● Building programs: stubs / scaffolds ● Optimize last, only in parts, with a timer ● Despite appearances, programming is a social activity; can people read/use/rely on your code?