Perforce Training, 2010.1 Scripting Perforce Introduction • Introductions • Class Schedule • About the Exercises 1 Perforce Training, 2010.1 Intended course audience • Experienced Perforce users who need to write scripts • Configuration management, build, and release engineers, and others Prerequisite experience • PfPerforce user and ddiitt administrator • Writing scripts using Perl 2 Perforce Training, 2010.1 Objectives • DlDevelop scr ittfipts to perform common tktasks • Follow best practices Uses of scripts • Web browser interface to Perforce • Plug-in for an IDE • Back-up and recovery routines • Wrappers for Perforce commands • Customized d iff a nd me rge too l s • Conversion from other SCM tools 3 Perforce Training, 2010.1 Script examples in this course • Automating processes • Generating reports • Sending email • Process support • Auditing metadata Course Contents • Preliminary Decisions • Perforce Commands for Scripting • Command Line Program (p4) • Application Program Interface (API) • Daemons • TiTriggers • Wrappers and Administrative Scripts 4 Perforce Training, 2010.1 Preliminary Decisions • Setting Environment Variables • User Authentication • Client Program Interface Preliminary Decisions Setting Environment Variables 5 Perforce Training, 2010.1 Use global options • Set environment variables in script p4 –p serverport –u username info • Advantage: Overrides all other settings • Disadvantage: Edit each script to change Use configuration files • Set P4CONFIG to a file name p4 set P4CONFIG=config.txt • Place config.txt files in script folders • List environment variables in config.txt P4PORT=lemon:1909 P4CLIENT=script_user_reporter P4USER=script_user 6 Perforce Training, 2010.1 P4CONFIG example Environment Variable: P4CONFIG=config.txt C:\scripts P4PORT=krypton:1666 P4CLIENT=scriptuser-prod config.txt P4USER=scriptuser \Krypton P4PORT=xenon:1666 P4CLIENT=report-writer config.txt P4USER=reporter \Xenon Using P4CONFIG • Advantages • Configures many scripts • Active in specific directories • Disadvantage • If P4CONFIG unset, script environment may change 7 Perforce Training, 2010.1 Use environment variables • Set environment variables (Windows) p4 set P4PORT=p4demo.com:1666 • List environment variables p4 set P4CLIENT=b ru no o_w s (set) P4PORT=p4demo.com:1666 (set) P4USER=bruno (set) Using environment variables • Advantaggye: Easy to set • Disadvantages • May be accidentally changed • Depends upon shell being used 8 Perforce Training, 2010.1 Configuration order of precedence • Command-line flags • P4CONFIG files • Environment variable settings • Registry variable settings (on Windows) • Default value for environment variables Preliminary Decisions User Authentication 9 Perforce Training, 2010.1 Use login • Works for all server security levels • Usage: p4 login Enter password: UitldiUser script_user logged in. Use a group to extend session p4 group scriptsonly Group: scriptsonly MaxResults: 100000 Maxscanrows: 500000 MaxLockTime: 30000 Timeout: unlimited Subgroups: Owners: bruno Users: script_user 10 Perforce Training, 2010.1 Use a password • Security levels 0-2 p4 -P Script_pass users • Security level 3 p4 login -p Enter password: 9CC3EACDEFCC8C7020C134D6D333F46F p4 -P 9CC3EACDEFCC8C7020C134D6D333F46F users Preliminary Decisions Client Program Interface 11 Perforce Training, 2010.1 Comparing client program interfaces Command Line Program Apppplication Prog ram Interface p4 Perforce C/C++ API One connection per User controls connection command Returns lines of text Tagged data available Some language support Language derivatives available Java-native API (P4Java) Objective-C API (P4ObjC) Tagging output: Command line and API • Command line returns lines of text • Format output by using -ztag p4 -ztag clients ... client bruno_ws ... Update 1104271684 ... Access 1104340062 ... etc. • Perforce API supports tagged data output 12 Perforce Training, 2010.1 Command line program: bypass an editor • Redirect to standard output p4 change -o • Read from standard input p4 submit -i Command line program: Using Python • Create Python marshalled dictionary object p4 -G job -o 13 Perforce Training, 2010.1 Sample Python Script and p4 -G Output #!/usr/local/bin/python #Example script named readmarshal.py import marshal, sys true = 1 try: while true: vars = marshal.load(sys.stdin) print vars except EOFError: '' #note that these are two single-quotes p4 -G user -o raj |dhl| readmarshal.py {'Email': '[email protected]', 'Update': '2005/12/06 11:16:30', 'Reviews1': '//depot/dev/main/...', 'Reviews0': '//depot/www/...', 'FullName': 'Raj Bai', 'User': 'raj', 'code': 'stat', 'Access': '2006/07/06 15:16:30'} Command line program: Using Ruby • Create Ruby marshalled dictionary object p4 -R job –o 14 Perforce Training, 2010.1 Sample Ruby script and p4 -R output # Example script named readruby.rb f = IO.popen("p4 -R user -o " + ARGV[0]) user_info = Marshal.load(f) p user_info readruby.rb bruno {"code"=>"stat", "User"=>"bruno", "Email"=>"[email protected]", "FullName"=>"Bruno Batswan"} Perforce’s derived APIs • P4Per l • P4Python • P4Ruby 15 Perforce Training, 2010.1 Scripting environment for the exercises • Use P4CONFIG to Set Environment Variables • User Authentication p4 login • Client Program Interface • p4 and API (using P4Perl) examples Scripting Perforce Exercise Setting the Perforce Environment 16 Perforce Training, 2010.1 Scripting Perforce Perforce Commands for Scripting Perforce Commands for Scripting • General • Review Daemons • Data Mining 17 Perforce Training, 2010.1 Perforce Commands General Capture errors, warnings and messages p4 -s opened -c default error: File(s) not opened on this client. exit: 0 18 Perforce Training, 2010.1 Submit without invoking an editor p4 submit -d "Fixed results of sim scans." Submitting change 1825. Locking 2 files ... edit //Sim/Prod/MAIN/src/sim.c#36 edit //Sim/Prod/MAIN/src/scan.c#2 Change 1825 submitted. Script efficiently • Limit number of lines of data returned • Narrow scope of Perforce commands • Refer to known workspace spec • Access small ggproups of dep ot files 19 Perforce Training, 2010.1 List a single changelist UiUsing a c lient wor kspace name p4 changes -m1 –s submitted –c raj-spruce Change 421 on 2001/01/07 by raj@raj-spruce 'Update sim to work reasonably on' List several changelists Using a depot path p4 changes –m10 //depot/www/... Change 1769 on 2006/12/11 by quinn@quinn-azalea 'Copy jamgraph example to web' Change 1763 on 2006/11/23 by hera@hera-amphora 'Jamgraph 1.0 is live.' ...etc. 20 Perforce Training, 2010.1 Perforce Commands Review Daemons Use a counter • Set personal counter p4 counter notifyd 746 Counter notifyd set. • Poll a counter p4 counter notifyd 746 21 Perforce Training, 2010.1 List all counter values p4 counters change = 750 job = 4 journal = 15 notifyd = 746 ... etc. Review daemon tools • View changelists past counter value p4 review -t notifydy Change 747 bruno [email protected](Bruno Batswan) Change 749 brad [email protected](Brad Manners) Change 750 sam [email protected](Sam Maxwell) • View subscribers for email notification p4 reviews -c 747 bruno <[email protected]> (Bruno Batswan) laura <[email protected]> (Laura Germain) 22 Perforce Training, 2010.1 Change your user profile p4 user User: earl Email: [email protected] Update: 2007/08/09 13:45:59 Access: 2008/03/07 16:45:05 FullName: Earl Ashby Reviews: Add “Reviews:” field if //Sim/Prod/MAIN/... running review daemon //depot/www/... List recently edited/created jobs p4 jobs -e "date>=2007/06/21" job002127 on 2007/11/19 by earl *open* ‘Fix parse error’ job002097 on 2007/10/21 by raj *closed* ‘Exit off’ job001931 on 2007/10/16 by gail *punted* ‘Web style’ job001762 on 2007/07/08 by raj *closed* ‘Scan func’ 23 Perforce Training, 2010.1 List fixed jobs p4 fixes //Sim/Prod/MAIN/src/... job000001 fixed by change 55 on 1999/01/19 by earl@earl-dev job000002 fixed by change 114 on 1999/02/28 by earl@earl-dev job000003 fixed by change 114 on 1999/02/28 by earl@earl-dev job000004 fixed by change 148 on 1999/07/19 by earl@earl-dev job000004 fixed by change 132 on 1999/04/30 by earl@earl-dev ... etc. Perforce Commands Data Mining 24 Perforce Training, 2010.1 Report depot data • List directories p4 dirs //depot/* //depot/Jam //depot/Jamgraph ...etc. • List files p4 files //depot/Jam/MAIN/src/... //depot/Jam/MAIN/src/Build.com#9 - edit change 839 //depot/Jam/MAIN/src/Build.mpw#3 - edit change 839 //depot/Jam/MAIN/src/command.c#12 - edit change 839 ...etc. Gather file data as tagged output p4 fstat execcmd.h ... depotFile //depot/Jam/MAIN/src/execcmd. h ... clientFile c:\raj\Jam\MAIN\src\execcmd.h ... isMapped ... headAction edit ... headType text ... headTime 1106848927 ... headRev 2 ... headChange 30 ... headModTime 1106847314 ... haveRev 2 25 Perforce Training, 2010.1 Report on a single file p4 annotate -ac jam. c //depot/Jam/MAIN/src/jam.c#37 - edit change 93 (text) 1-93: /* 1-1: * Copyright 1993 Christopher Seiwald. 30-93: * Copyright 1993, 1995 Christopher Seiwald. ... etc. Search for patterns in files p4 grep -a -n -F -e "/* debug */" jam.c //depot/Jam/MAIN/src/jam.c#14:114: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#13:114: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#12:114: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#11:106: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#10:106: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#9:96: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#8:96: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#6:93: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#5:93: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#4:93: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#3:93: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#2:91: 1, /* debug */ //depot/Jam/MAIN/src/jam.c#1:86: 1, /* debug */ 26 Perforce Training, 2010.1 Report Perforce metadata • Lis t a ll c lien t wor kspace specs p4 clients • List client workspace specs by owner p4 clients –u bruno Use -n flag to report status p4 integrate -n -b SimMAINtoFLAT //Sim/Prod/FLAT/src/scan.c#3 sync/integrate from //Sim/Prod/MAIN/src/scan.c#12 //Sim/Prod/FLAT/src/scan.h#2 sync/integrate from //Sim/Prod/MAIN/src/scan.h#14 ...etc.