TSL Language Aos Engine Family SqLite TSL Integration Year-2 David Simmons, Smallscript Corp Tcl 2017 Conference SqLite Track SqLite: SqLite APIs for Cmd Extensions • “af” procedures • af: normal • afd: deterministic • afEach: normal tables • afEach: deterministic tables • Deterministic means • cacheable idempotent • SqLite Api Integration • build flags • a few other APIs used as hooks 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 2 SqLite Track SqLite: UUID - Universally unique identifier • UUID 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 3 SqLite Track SqLite: Time conversions • Time Conversion • %J: Julian days as float • The julianday() function returns the Julian day - the number of days since noon in Greenwich on November 24, 4714 B.C. (Proleptic Gregorian calendar). • • time-fmt({now +1 day -2 hours time-fmt: tsl-command -7 min, {%J}); • custom time parsing and formatting library • time-fmt(connection.time-created, • parses most text forms {%Y/%m/%d %H:%M:%S.%f UTC}); • time-fmt(uuid-create-sequential(), • parses numbers as either epoch seconds in epoch, uuid or {%Y/%m/%d %H:%M:%S.%f UTC}); FileTime, time 10-7 64-bit time 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 4 SqLite Track SqLite: JSON Interchange & Ad-Hoc Records • JSON1 Library • Json data in records • jFieldName • Json as tables in sql expressions • CRUD triggers on columns and indexes • JSON1 (flaws) • Empty and Nullable • Caching 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 5 SqLite Track SqLite: sql triggers 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 6 SqLite Track SqLite: sql query using .af command 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 7 DEMO 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 8 SqLite Track SqLite: Integrating TSL • sqlite “lib” • built w/JSON1 • convinced a number of the poplar open source tools to add JSON1 to their nightlies • afm built as a single “lib” • built from a single “cpp” file • Linked as “afm.exe” console subsystem coffbox • Renaming “afm.exe” a .SO or .DLL • Exporting standard DLL API interfaces • AOS/S# COFF binary ffi-thunk bBuilder 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 9 SqLite Track SqLite: Packages “DBs as vfs Pkg” • SqLite Header • SiteId, AppId • Common DB extension forms • .afx, .afr, .dpk, .zdb [FsPath VFS model] 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 10 SqLite Track SqLite: Schema concepts - sqlite_master • sqlite_master • VersionSchema table concepts • VersionSchemaTrigger table • db-resident scripts • StoredProcedures • Where to place stored procedure code depends whether a vfs-repo-fs table model is available • Undefined sql API hook (NFH) • Allows lazy NFH lookup into existing environment, or lazy load from db stored procedure tables 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 11 End of Slides 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 12 TSL Language Aos Engine Family Devops and the CoffBox Model Year-2 David Simmons, Smallscript Corp Tcl 2017 Conference Deployment Track CoffBox: DevOps Concepts • Executable Components • Registering script types • afm.exe • Windows • afm-symlinked-name.exe • PATHEXT • executable-db-vfs-pkg.afx • ftype, assoc • usually a “.afr” that’s been • registry twiddling for overloading symlinked • nix* • shebang-#!path or pathext-ext • shebang - #!path header registered. • chmod • script-name.afts 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 14 Deployment Track CoffBox: Private Cloud • My home office environment (by way of example) • Networks • 10GB Fiber • Mesh WiFi • ZWave, Ethernet-over-power • over • 70TB NAS storage • 20 computers • 200 devices • OS • Linux • OSX • Windows • Others 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 15 Deployment Track CoffBox: Explore Desktop • Show Files and Locations • Demo App 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 16 Deployment Track CoffBox: DevOps Concepts • .af directory patterns • Packaging Model • .conf customization model • search paths • .af directory pattern • SymLinks • .afws • client system relative • .afr, .afx • symlinks, reparse-points • host system relative • .afts scripts • junctions • absolute and relative paths • repo-treatment • ignored, as-file • xattr archive-flag • Virtual Drives • .dmg, .vhdx, linux sparse file images 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 17 Deployment Track CoffBox: Build a script • script w/command line arg processing 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 18 Build an afx w/.conf checked into it. Showcase demo webserver 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 19 Deployment Track CoffBox: DevOps Concepts Customizing a System Reflecting on a System • ChildProcess • ::system::info • pipes • elevated, admin, etc • http(s), message-queues • reg- commands • sockets • http(s) • EventLog mechanisms • telemetry reporting 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 20 Deployment Track CoffBox: DevOps Concepts • Binary build model • TSL scripts exported as native • cpp, hxx, hpp, h COFF library binaries (.so, .dll, .exe) • Single Binary for Exec & DLL • your-code-thunk.dll - thunk- (.so) bridges • Coff Format • Subsystems • Sections • Resources • Read-only Memory Repos 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 21 End of Slides 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 22 TSL Language Aos Engine Family afm Year-2 David Simmons, Smallscript Corp Tcl 2017 Conference iOT Track iOT: Purpose Built Constrained Environments Constraints Typical metrics • Maybe single app • Memory • Security • Cpu • Storage types • • Hw permission and firmware Storage controls • Power Management • Memory • Os Services • Power Management Lifecycle • CPU Cycle Cost • RTOS Constraints and Models • Embedded toolset contraints 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 24 iOT Track iOT: engine requirements for iOT deployment • Os Platforms • Cpu Targets • Windows, OSX, iOS, Android, • Arm (32, 64), Intel x86/x64, Linux, Linux Embedded, MIPs FreeRTOS, nuttx, nucleus rtos • see https://www.osrtos.com/ 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 25 iOT Track iOT: Device Use Case • Networking • UX • Wifi, Bluetooth, Wired- • custom displays Ethernet, ZWave, Zigbee • custom input/sensors • VFS • Web Server/Service • Packaging, Versioning • Telemetry • Resources, executable components • Updates • Power Management Lifecycle • Device Interop (peer, slave, • Device On/Off master) • Device Firmware Update Models • Rollback, Upgrade • Build, Package, Sign requirements • Flashing Lifecycle Constraints • OTA, Wired 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 26 iOT Track iOT: Afm’s iOT ready architecture • Pal: Host Abstraction • Coff binary build model • Memory – cpu/kernel direct • cpp, hxx, hpp, h • TLS – CPU direct (mirror OS model) • Interrupts, Signals, Exception • embedded r/o “.afr” • Debugger Aware Channels • compression • Pal: Cpu Abstraction • FPU, Vector • Codecs • MP-Sync Instructions • Bit Operations • compression • Special Forms • Pal: FsPath Pluggable Abstraction • Engine Design • Native File System, (tags, versioning) Fs built on SqLite blobs, • Booting Model Registry, Fuse/Dokan, HTTP(S), BuiltIns • Communications • Thread Model • Transports • engine-affinity, fiber-co-operative, • Ethernet, Bluetooth, Serial rpc message streams • Sockets • DNS, DHCP, HTTP (1.0, 1.1, 2.0 ALPN) • pipes, tcp, ssl, http • SSL, Certs (pal and host integrated) • json, html, … 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 27 End of Slides 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 28 TSL Language Aos Engine Family Language Concepts and Usage Year-2 David Simmons, Smallscript Corp Tcl 2017 Conference Language Track Statements: JS & TCL 12 Rules Disambiguation 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 30 Language Track Engine: Model Procs and Threads Code Locations • Process Model • FsPath Concepts • multiple engines allowed per process • SymLinks • Engines • Archive XAttr-Flag • engines have thread affinity • co-operative threading within an • Process Loading Model engine • Namespaces • Script Binding Model • ::super • ::system • Web Site and Page Model • ::afm • ::shell • ::app • ::script 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 31 Language Track Statements: Statement Tokenization and Substitution • Phase 0: Text Command Statement • Comment Forms • Statement Delimiting • // EOL • /* nestable */ • [command must defer-level1] else rules • # - special command, NFH form • ; eos • Allows #command if NO whitespace • {} eos • Word Grouping Rules • # eos • () first following command • ‘ws’ eos [tcl-mode only] • allows whitespace • () {} … no other tokens • Phase 1: Command • partitions into discrete words whether • Phase 1a: Expr Args whitespace or not • Phase 1b: semicolon args • Deferral Levels 1,2,3 • =, ? ... : • Phase 2: Binary Operators • for() … • do while {} • Phase 3: TCL Parameters • return 11/17/2017 TSL Talk by David Simmons, Smallscript Corp (c) 2017 32 Language Track Commands: Forms • Types • Modes • default • proc • ^ uplevel & native proc^ • ::super namespace • @ modules • protected from re-definition • Prototypes • func (fn) • Tson Declaration/Merge &:{} • context bound • Declaration Features