The (Active) State of Tcl

15th Annual Tcl Conference (2008) Jeff Hobbs Director, Languages ActiveState Corporation Agenda

 Introduction  What has happened…  Developments in the Tcl community  The Active State  Developments of the Tcl/Tk core  Future directions About ActiveState

 ActiveState Software Inc.  Over 10 years focusing on open source dynamic languages  Independent again since February, 2006  ActiveState provides multi-language, cross-platform software & services  Tcl, Perl, PHP, Python, Ruby  AIX, HP-UX, Linux, Mac OS X, Solaris, Windows, …  Comprehensive Software Stacks for Dynamic Languages  Make programming easier using dynamic languages  Active distributions  Software development tools (IDEs, Dev Kits)  Programmer resources (ASPN)  Enterprise support & OEM redistribution History of Tcl

1. Tcl created as general-purpose 4. Scriptics formed: command/scripting language by 7. ActiveState introduces • Evolve and extend Tcl platform Tcl support and services John Ousterhout • Create development tools

6.0 Sept 7.0 Sept 7.4 July 7.6 Oct 8.0 Aug 8.1 Apr 8.3 Feb 8.4 Sept 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 … 8.2 Aug

5. Tcl Core Team 8.3.5 EOL Oct formed (August) 2. Open source distributions from U.C. Berkeley: 6. Scriptics/Ajuba Acquired • Easy GUIs under Unix by Interwoven (Nov) • Extensible applications

3. Tcl enhanced at Sun Microsystems: 8. ActiveState acquired by • Windows, Macintosh ports Sophos (Sept) • Web/Internet support • Java support Recent History of Tcl

Sophos Acquires ActiveState (Sept) ActiveState Publishes Dynamic Languages ActiveState Tcl Dev ActiveState Tcl Dev Whitepaper (July) Kit 2.6 (May) Kit 3.0 (Dec) 8.5a1 Mar 8.5a3 June 8.5.0 ?? 2003 2004 2005 … 9.0 ?? 10th Tcl Conference 11th Tcl Conference 12th Tcl Conference (July) Ann Arbor (Oct) New Orleans (Oct) Portland

8.4.7 July 8.4.11 June

Tcl’Europe 2003 (May) Recent History of Tcl

ActiveState Becomes Independent From Sophos (Feb)

8.5b1 Sep 8.5.0 Dec 8.6a3 Oct 8.6.0 !! 2006 2007 2008 … 9.0 ?? 13th Tcl Conference 14th Tcl Conference 13th Tcl Conference (Oct) Chicago (Sep) New Orleans (Oct) Manassas

8.4.14 Oct 8.4.16 Sep 8.4.19 Apr 8.5.5 Oct

You are here Status as of Tcl’2007

 September 24-28 New Orleans  Mmm mmm good  Tcl/Tk 8.4.16 was the standard stable version  Further stabilization  8.4 now base standard in all distros  5 years since 8.4.0  Numerous enhancements and additions  Getting long in the tooth ...  8.5b1 released  Focus on putting 8.5 to bed  8.6 vs. 9.0  Primary topics …  8.5 finalization  Food Tcl Core Team

 Formed in August 2000 with 14 charter members  Elected by community vote  Collectively manage development of the core  Now at 16 members

Mo DeJong Andreas Kupries Donal Fellows Karl Lehenbauer Mark Harrison Joe English (2005) D. Richard Hipp Jan Nijtmans Jeffrey Hobbs John Ousterhout George Howlett Don Porter Jim Ingham Brent Welch Kevin Kenny (2001) Miguel Sofer (2004) TCT: TIP Initiatives

 TIP page for Tcl http://tip.tcl.tk/  TIPs are intended to guide and document development on the core  They focus on new or changing features, not bugs  Voted on by the TCT following community discussion using the TYANNOTT process  Currently 335 TIPs  Divided into process, informational & project TIPs  TCT discussion is open on the public mailing list: [email protected] Tcl/Tk Maintainers

 Tcl/Tk maintainers are separate from the TCT  Maintainers oversee a specific area of the core, as defined in TIP #16 (Tcl) and TIP #23 (Tk)  They assist, but are not solely responsible for, fixing bugs and adding docs in their area  They are responsible for reviewing code and approving code changes to their area  Open to anyone willing to learn the core  New volunteers always welcome The Maintainers…

Tcl (TIP #24): Daniel Steffen Jim Ingham Kevin Kenny Jeff Hobbs Miguel Sofer Andreas Kupries Rolf Schroedter Vince Darley

Don Porter Jan Nijtmans Donal Fellows Mo DeJong Pat Thoyts Joe Mistachkin Pavel Goran Tk (TIP #30): Allen Flick Peter Spjuth Todd Helfter Jeff Hobbs George Smith Frédéric Bonnet Kevin Griffin Vince Darley

Chengye Mao Jan Nijtmans Donal Fellows Mo DeJong Joe English Daniel Steffen Ben Riefenstahl Miguel Bañón Always open for new recruits … Tcl @ ActiveState

 ActiveState established 1997  “Dynamic Tools for Dynamic Languages”  Used to be Perl specific . Well known ActivePerl distribution  Added Python and XML/XSLT expertise in 2000  Jeff Hobbs & Andreas Kupries hired in 2001  Other knowledgeable Tcl’ers on staff  Wealth of scripting knowledge at ActiveState  Acquired by Sophos in September 2003  Freed from Sophos in February 2006  ActiveState remains dedicated to dynamic languages ActiveState and Tcl

 ActiveState provides the Tcl community with…  Improvements to open source Tcl core  Host of the Tcl Developer Xchange  Enterprise support infrastructure  Professional services for Tcl  High quality development tools . ActiveTcl • Batteries-Included free distribution . Komodo IDE • Professional IDE with extensive Tcl support . Tcl Dev Kit • Essential tools for Tcl programmers . ActiveTcl Pro Studio • Tcl Dev Kit, Komodo IDE, Safari Online ActiveState 2008 Tcl Cookbook on new ASPN ActiveTcl 8.4.19.0

 Based on the 8.4.19 Tcl core  Released September 2008  Enhanced with many popular extensions  XOTcl, [incr Tcl], TclX, expect, tcllib, tklib, Bwidgets, iwidgets, tktable, tkcon, tile, SQLite, …  300+ packages in all  A standard batteries-included distribution for Tcl users  Available for AIX, HP-UX, Linux, Mac OS X (universal), Solaris (x86 and sparc), Windows . Windows and OS X builds are fully threaded  Other platforms supported in Enterprise Edition  TEApot code repository via teacup

ActiveTcl aes asn autoproxy autoscroll Banking base64 bee bibtex blowfish BWidget calendar cgi2dom cksum cmdline comm compiler control counter crc16 crc32 critcl csv ctext cursor datefield dde des dns docstrip doctools dom dommap domtext domtree exif expat Expect fileutil ftp ftp::geturl ftpd getstring history html htmlparse http ico ident Img inifile ip ipentry irc Itcl Itk Iwidgets javascript jpeg jpegtcl ldap log logger math md4 md5 md5crypt Memchan mime Mk4tcl mk4vfs msgcat multiplexer ncgi nntp opt Oratcl parser Plotchart pluginmgr png pngtcl pop3 pop3d pop3d::dbox pop3d::udb profiler puretclparser rc4 rcs registry report resolv ripemd128 ripemd160 rpcvar SASL sgml sgmlparser sha1 sha256 smtp smtpd snack snackogg snacksphere snit SOAP sound soundex spf sqlite3 starkit stooop struct style sum swaplist switched Tablelist tar tbcload Tcl tclDES tclDESjr tcllibc tclparser TclScript tcltest Tclx tcom tdom tdomhtml textutil Thread tie tifftcl tile time tipstack Tk Tkhtml tkpiechart Tktable tnc tooltip treectrl treeql Trf trofs units uri uri::urn uuencode uuid vfs vfslib vu widget widget::all widget::dialog widget::panelframe widget::ruler widget::screenruler widget::scrolledwindow widget::superframe win32 xml xmldefs XMLRPC xmlswitch XOTcl xpath xslt xslt::cache yencode zipvfs zlibtcl ActiveTcl 8.5.5.0

 Based on the 8.5.5 Tcl core  Released October 2008 – on your USB key  Distributed with fewer extensions  Can utilise the 8.4 extension set  90+ packages - the rest available via teacup  Considered the default download  An enhancement to the stable 8.4 distribution  Available for AIX, HP-UX, Linux, Mac OS X, Solaris, Windows . All builds are fully threaded (including known pthread/fork issues)  What are you missing? ActiveTcl 8.6.0.0 Beta 1

 Based on 8.6a3 Tcl core  Not yet released - on your USB key  Similar to ActiveTcl 8.5 extension set

 More discussion on 8.6 to follow The TEApot

 Binary code repository in the style of ActivePerl's PPM  Fully populated for ActiveTcl platforms  Makes use of Tcl module system  Allows zip archives with special teapot.txt  Repository is web visible, but rough

 Downloads index – local searches  Integrated with TclApp package searching (TDK 5) ActiveTcl Pro Studio

 Formerly ASPN Tcl  Introduced December 2001  Professional Tools for Tcl Developers  Tcl Dev Kit 5.0  Komodo Pro IDE 4.4 (5.0 now in beta)  Safari Bookshelf online books Tcl Dev Kit 5

 5.0 released October 2008  Added 8.6 support  Enhanced static syntax analysis  Supports multi-architecture starkits, x-platform starpacks

 Includes the traditional TclPro tools, revamped:  Debugger: source-level debugger with GUI  Checker: find errors without running application  TclApp (Wrapper): create self-contained applications for distribution  Compiler: protect your source code  Enhanced features and UIs for original tools  Version independent from ActiveTcl Tcl Dev Kit 5

 Additional tools  Coverage: coverage and hot-spot analysis built into the Debugger  TclSvc: create and manage Tcl scripts as Windows NT services  TclApp: rewrite of the wrapper based on VFS and starkit architecture, cross-platform wrapping  Inspector: runtime Tcl application browser to complement the Debugger  X-ref: examine dependencies in your code  VFS Explorer: examine starkits/starpacks Komodo 5

 Professional IDE for dynamic languages  Professional IDE features  Integration with Tcl Dev Kit  Unique Tcl editing features  Cross-platform (Windows, Linux, & OS X)  Also supports Perl, Python, PHP, Ruby and more  New in 5.x ...  Added support for git, hg, bazaar (4.x had cvs, p4, svn)  Code formatting tool hooks  Multiple toplevel windows  Faster and slicker UI  Performance improvements and bug fixes Komodo 5 In the Community…

 The Tcl Developer Xchange:  http://www.tcl.tk/ aka http://tcl.ActiveState.com/  Official home page for Tcl and Tk  Formal front-end, downloads, advocacy, history, …  The Tcl’ers Wiki has ~22,000 pages now:  http://wiki.tcl.tk/  Wiki chat has constant traffic . HTTP-based with IRC bridge  Primary discussion forum  Community contributed and maintained  Tcl-URL! continues to provide weekly news:  Posted weekly on news:comp.lang.tcl  Or subscribe: [email protected] Tcl/Tk Dev Online

 SourceForge is for developers:  http://tcl.sourceforge.net/  Bug and patch database  Mailing lists  CVS repositories  File server . Has Tcl sources back to Tcl 2.1  Developer web pages  Managed by TCT and Tcl/Tk maintainers  Used by many other Tcl/Tk extensions and applications Tcl Developer Xchange Tcl'ers Wiki Tcl @ SourceForge Tcl/Tk 8.4

 8.4.19 released April 2008  At end of development life  8.5 is now considered the stable line  Several core feature enhancements  Virtual File System (VFS)  lset, lsearch, trace command, …  64-bit integer and file-system support  Improved locale support in Tk  Significant work on performance  20+% performance improvement over 8.3  Near or better than 8.0, but now Unicode aware and thread- safe  New Tk core widgets and enhancements Tcl/Tk 8.5

 Finished after almost 4 years in development  8.5a1 released March 2004  8.5.0 final December 2007  8.5.5 released October 2008  8.6a1 development initiated April 2008  8.5 is 99.99% backwards compatible with 8.4 at the Tcl level  Exceptions are in error cases  Bytecodes have changed, watch version compatibility  Over 100 TIP features included in 8.5  Coming soon ... Tcl & Tk Toolkit, 2nd ed. Tcl 8.5 expand and dict

 New {*}$aList expansion operator  Replacement for oft-misused eval eval exec [auto_execok start] [list $url] => exec {*}[auto_execok start] $url

 New dict data structure  Order-preserving key/value pairs kept as Tcl_Objs  More efficient and nestable  Close pairing with [array get] and lists  Can replace any use of TclX keyed lists  Numerous subcommands for filtering, updating data  Some operations byte-coded  8.4 emulated version in ActiveTcl Tcl 8.5 expr

 New expr operators  ** (exponentation), min, max, isqrt, bool, entier  in (in list) expr {“b” in {a b c}} => 1  ni (not in list), replaces use of lsearch != -1  New ::tcl::mathfunc namespace for functions  Define expr functions at the Tcl level  New ::tcl::mathop namespace for operations  Allows importing as procs for prefix math  All functions are byte-compiled  No more integer overflow because of bignums Tcl 8.5 bignums

 expr 2 ** 1024  1797693134862315907729305190789024733617976978942 30657273430081157732675805500963132708477322407536 02112011387987139335765878976881441662249284743063 94741243777678934248654852763022196012460941194530 82952085005768838150682342462881473913110540827237 1633505106845862982399472459384797163048353563296 24224137216  Every single bit hand-verified by Kevin Kenny  Work continuing on performance of bignums  Bignums available for all expressions, but most indexing is still limited to 32-bit values (ie: string index, lindex)  Floating point handling improved to reduce rounding errors Tcl 8.5 list ops

 Added lrepeat, lassign, lreverse  lassign is as taken from TclX  lrepeat number element1 ?element2 element3 ...?  Extended lsort, lsearch  lsearch -index -nocase -subindices  lsort -nocase -indices  List indices now support int[+-]int and end[+-]int syntax Tcl 8.5 namespaces

 namespace ensemble (ensemble commands)  Adopted from [incr Tcl], useful for subcommand-style procedures  [::carrot::foo] == [::carrot foo]  namespace unknown (per-namespace unknown control)  namespace path (name resolution control)  namespace upvar (namespace var efficiency) Tcl 8.5 clock

 Completely revamped clock command  64-bit, new parser, ensemble-based  No more Y2038 issues, even on 32-bit systems  Full localization support  Several new subcommands  clock add . Simplifies date math operations  clock microseconds|milliseconds . Improved resolution  Clock scan and format updated  User-specifiable scan formats  Full set of Olsen timezone data Tcl 8.5 interp

 Resource limiting with interp limit command  Time and command count based  Improved background error handling with interp bgerror  Enhanced debugging with info frame  Returns current command location and information  Understands origin (source, proc, eval, precompiled)  New chan command to consolidate channel operations  Subsumes most f* file id manipulation commands  Channel reflection exposed at the Tcl level (chan create)  chan truncate added Tcl 8.5 packaging

 Enhanced package versioning  Understand a, b designation in version number (8.5a5)  Allow for bounded version ranges  Control preference for stable or latest version available  Tcl Modules (.tm) package handling  Intended to improve packaging long term  tclsh and source -encoding option  encoding dirs  Tcl level control of where to find encodings Tcl 8.5 extras

 Enhanced catch and return  Make pure Tcl control structures more powerful  New apply operator for anonymous procedures (lambda) apply {x {expr {$x*$x}}} 5 => 25  Several new binary format codes and improved unsigned value support  %t (tiny/short), %n (normal/int), %m (wide), %r/R (single precision real), %q/Q (double precision real), %u (unsigned)  scan and format have %llx modifiers  New tcl_platform(pointerSize) element Tcl 8.5 switch and string

 switch -regexp can provide submatch information switch -matchvar foo -indexvar bar -regexp -- $string { {\w*(e)\w*} { puts "matched [lindex $foo 0] with 'e' at \ [lindex $bar 1 0]" } ...  switch added a -nocase option  New string is list|wideinteger classifications  New string reverse subcommand Tcl 8.5 gotchas

 global varName(elemName) raises an error now  Also for upvar 0 x(1) y(1)  Was previously an impotent operation that created inaccessible variables  Don’t rely on integer overflow - shouldn’t happen  list properly quotes the # char  tcl_precision now defaults to 0  Auto-initialize for incr  Previously an undefined var would cause an error  Gains semantic parity with append, lappend Tk 8.5

 Themed Widgets!  Based on tile 0.7.8 work by Joe English . Identifies as tile 0.8.0  Complements of most classic Tk widgets  Some new widgets . notebook, combobox, progressbar, separator, treeview  Enhancements to existing widgets and dialogs  Peering and smooth-scrolling text widgets  Tri-state checkbutton  panedwindow -stretch and -hide options  Standardized font names  Xft support in X11 (anti-alias fonts)  Requires recent Linux distro Tk 8.5 extras

 Enhanced wm commands  More toplevel attribute control  Photo images for icons  Dockable windows (reparenting of frames)  Cubic Bezier curves on the canvas  Improved grid geometry management  row|columnconfigure by widget name  Overall grid anchoring  Virtual event user data (%d)  Control over idle (activity) timer  Enhanced demos Tk 8.5: The Future is Now! Tk 8.5: Ttk

 Improved standard Tk look and feel  New look for widgets on X  Improved bindings behavior  Restorable with tk::classic::restore  Themed Tk widgets (Ttk)  Non-default alternative in 8.5  Default in 9.0  Includes new widgets  Classic Tk in tk::* . tk::button …  Ttk in ttk::* . ttk::button …  Tk and Ttk are complementary, not mutually exclusive Tk Revitalization at Work

 ActiveState Perl Dev Kit 6.0  First commercial use of tile  First use of Perl Tcl::Tk  First use of DLL tclkit

 Pushing the envelope for Tk into revitalization Tcl/Tk 8.6

 Currently in (very) active development  8.6a3 released October, 2008  Release schedule towards feature complete finalized  8.6b1 (feature freeze) for early December 2008  8.6.0 targeted for March 2009  Numerous TIPs already targeting 8.6  21 TIPs final  71 TIPs still targeting 8.6 (in draft state) . Most unlikely to make 8.6 final  TIP review to follow  Community input is important Tcl 8.6 features

 Core OO! (TIP#257, TIP#320)  Major effort by Donal Fellows  Foundational OO system inspired by XOTcl and snit  oo::class, oo::define, oo::copy, oo::object, ...  Class-based object system with dynamic redefinition allowed  Per-object customization with filters, mixins and more  New non-recursive bytecode engine by Miguel Sofer  Enables coroutines (TIP#328), aka generators  Proper tailcalls (TIP#327)  New binary encode|decode with base64 (TIP#317)  Simplified Tcl and C option parsing (TIP#195, TIP#265)  Ability to cancel script evaluation (TIP#285) Tcl 8.6 features

 Enhanced chan command  Anonymous pipes (TIP#304) as chan pipe  Tcl level channel transformations (TIP#230) chan push|pop  Find insertion point in sorted list (TIP#313)  Add tcl_platform(pathSeparator) key  Unicode aware string trim* (TIP#318)  Grouped sort with lsort -stride (TIP#326)  No more interp->result (TIP#330)  Use Tcl_GetStringResult(interp) instead  Allow lset to extend lists (TIP#331)  lset myList end+1 foo equivalent to lappend myList foo Tk 8.6 features

 New tk busy adapted from BLT’s busy (TIP#321)  Absolute position of canvas items (TIP#236)  Lots of open TIPs targeting Tk ...  Any volunteers? Tcl/Tk 9.0

 Still in the wishlist stage  Rationalize C APIs . Fewer string APIs? . Only bytecode eval paths?  Possible restructuring of Tcl level commands and global data  More OO / encapsulation features in the core  Remove octal support  Tcl_Obj improvements (hydra vs. stork, immutability?)  Tcl 9 experience must not be like Perl 6 (or Tcl 8.5?)  Once started, we must dedicate effort to Tcl 9  We cannot rush it though … Ideas in the Pipeline

 Refactored bytecode engine  Drag & Drop  Core .zip support (234)  Printing support  Process control / bgexec (240)  Rotate canvas items (164)  Smaller, modular core  Megawidget framework (180)  Try/Catch/Finally (329)  More core widgets  IPv6 sockets (162)  treectrl, table, opengl, …  Enhance load (239)  Shaped windows  Improve process control  Fine-tuned font control  TDBC  Core tooltip support  …  Enhanced photo manipulation  … User Polls

 Versions in use  Response  Pre-8.0  8.0-8.3  8.4  8.5  8.6 (alpha)  Develop / Deploy:  Develop / Deploy:  Windows  Macintosh OS X  Linux  Solaris  HP-UX  Other … Future Directions

 The core is guided by community input  Anyone can write a TIP  Anyone can be a core maintainer  8.4 nearing end of “development” life  8.5 current stable version  8.6 near escape velocity . Looking for assistance at all levels  9.0 on the drawing board  What issues are most pressing?  ActiveState remains committed to Tcl  Working with the Tcl community  Focus on Tk revitalization Q & A ? ? ?

? ? ? ? ? ? Next ... the TEApot TEAPOT

 Binary code repository in the style of ActivePerl's PPM  http://teapot.activestate.com/  Fully populated for ActiveTcl platforms  AIX, HP-UX, Linux, Mac OS X, Solaris, Windows  Makes use of Tcl module system  Repository is web visible, but rough  Downloads index – local searches TEAPOT Applications

 All are starpacks  No external dependencies  Command line only, no GUI (yet)  Builds on Tcl package generation  Requires TIP #268 (package versioning)  General  Server  2 Clients, User & Admin  SQLite database used as meta data store TEAPOT User Client

 Query “opaque” and “transparent” repositories  Install packages in “transparent” repository  From queried repositories  Package files directly  Manage repositories to query, install to  Package operations  install, remove, get, describe, list, search, keys, profiles  Archive operations  default; archive add|remove|list; setup TEAPOT Communication

 HTTP is basic client - server protocol  Payload:  HTML, for browsers  Tcl lists embedded in HTML comments, for the client  No upload through web protocol  Might change, with careful safeguards TEAPOT Future

 Store other types of data  Sources  Documentation  Applications . Example: Basekits for TclApp  X-Ref Information . I.e. Code Intelligence Data  Checker Packages . I.e. Syntax definitions for static checker  Simple UI front-end  Possible distribution changes  Make ActiveTcl have preloaded repositories  Update of binaries from a repository? . Quick deployment of bug fixes