DAQRI ARToolKit 6/Open Source Introduction to ARToolKit Phil Oxford Brookes University 2017-01 DAQRI by the numbers DAQRI Smart Helmet Wearable Human-Machine Interface Hazloc-certified PPE helmet Sensor package: Logic board: • Intel R200 stereo colour/depth cameras Stereo holographic- Intel Core M7 CPU @ 2.1GHZ • Lepton 3.0 Thermal Camera waveguide full colour 8GB RAM • DAQRI VisionCube™ integrated see-through display Wireless radio IMU/wide FOV optical tracking system DAQRI’s approach to AR The DAQRI Ecosystem 4D Studio Services Analytics MDM, Remote Expert Operating System Software Core apps Advanced computer vision External framework plugins } Hardware Smart Helmet, In-vehicle display Physics Realtime holographics ARToolKit: A collection of software tools to help solve some of the fundamental problems in augmented reality, including geometric and photometric registration • geometric registration: aligning the position of the virtual environment with the actual • photometric registration: matching the appearance of objects in the virtual environment to the actual ARToolKit - History • 2004 - v.2 released open source (Sourceforge) • 1999 - first demonstrated publicly • 2008/2012 - open source innovation • 2001- v.1 released open source (Washington) • 2012/2014 - extending platform support • 2001 - ARToolworks incorporated with dual licensing model • 2015 - acquired by DAQRI - pro versions open sourced 1999 2009 2015 SIGGRAPH 1999 ARToolKit Shared Space demo VR2009: Kato award ARToolworks becomes part of DAQRI (Lamb, Vaughan, Furness, Billinghurst, Kato) 6 Goals • Open-source a high-quality texture-tracker. • Modernise target platforms and reduce need for legacy support. • Create SDK suitable for full spectrum of expertise • high-level low-complexity API for novices • expose underlying CV APIs for experts while simplifying ancillary tasks ARToolKit 6: Computer vision ⁕ On-device image recogniser ⁕ Multiple textured-surface tracker ⁕ Planar surface tracker .. ⁕ High performance pose estimator ⁕ Open lens model database ARToolKit 6 tracking features • On-the-fly tracking data generation (10s of millseconds) • Feature-based initialisation (FREAK features) • Template matching • Optical flow fallback • New pose estimation via P3P offering much improved stability ARToolKit 6 image recognition features • On-device image recognition (up to 500 images) • Tool supplied to allow pre-generation of datasets from large number of images • Dynamic saving/loading of image sets, or individual images • Image recognition also provides tracking initialisation • Wednesday ARToolKit lesson 5 by Dan Bell: Details of ARTVision ARToolKit 6 other features • On-device camera intrinsics calibration database. • Self-hostable camera intrinsics calibration database server. • Tools provided to calibrate your own cameras/devices • Thursday ARToolKit lesson 6 by Thorsten Bux: Camera calibration system ARToolKit license terms BSD-like Apache LGPLv3 GPLv3 Use of code in As linked All All None closed-source library Acknowledge No No Yes, plus link Yes, plus link ment required Retention of copyright/ If redistributing Disclaimer only Yes Yes license complete file(s) headers Requires Modifications publication of No No Yes to the library users source License for License for ARToolKit ARToolKit example code libraries & utilities Structure of ARToolKit 6 Key: C++ class External library Internal library Hybrid system Binding Objective-C API Java API C# API JNI P/Invoke C API ARController ARTracker ARTrackable ARTrackableAppearance ARTrackableSquare ARTrackerSqua ARTrackerInstant ARTrackableInstant ARVideoSource ARTracker2D ARTrackable2D ARTrackableMultiSqua ARView On re re On libARVideo libARTVision libAR libInstantOn libARG Android Camera glog CoreMotion OpenGL/GLES Video AVFoundatio calibration boost Push n Video4Linux2 database sqlite3 curl SDK structure • Resources required to modify SDK are separate from resources required to use SDK • Human-readable top-level SDK structure: • SDK source and build system (machine readable) in “Source”. AR6 • Examples in “Examples”, stand-alone source with AR6J IDE project files CMakeLists.t (iOS, macOS: Xcode, Android: Android Studio, Linux: CMake, future Windows: Visual Studio) xt Utilities build.sh Build system • CMake build-system builder • bash script-driven configuration and build system driver • One-line SDK build for all platforms: ~/artoolkit6$ cd Source; ./build.sh (macos|ios|android|linux) [examples] [utilities] • For package-based platforms (e.g. Linux Debian/Fedora), build.sh checks required dependencies, gives hints for installation • Install phase of build installs to ‘SDK’ directory Packaging script pulls from this location. • Demo Packaging and deployment • Packaging goal: SDK with built binaries of all components, plus source for examples (but not source for SDK or SDK build system) • Continuous integration and deployment via bitrise.io SAAS. • Packaging automatically performed for HEAD of master branch, output pushed to AWS bucket (demo) Packaging and deployment - 2 • CI status incorporated into approval workflow of all pull requests to master on GitHub API overview • Primary high-level API is C++ <AR6/ARController.h> • Simplified C API <AR6/AR6_c.h> • Java API in AR6J project. (Implemented via JNI interface to libAR6) • Main abstractions: • controller • videosource • tracker • trackable • view API - example #include <AR6/ARController.h> ARController *c = new ARController(); c->initialiseBase(); ARTrackable *t = findTrackable(c->addTrackable(“2d;gibraltar.jpg;160.9")); c->startRunning(); while (!done) { if (c->capture()) { c->update(); if (t->visible) { draw(t->transformationMatrix()); } } } } c->stopRunning(); c->shutdown(); Native examples • “Native” in the sense of typical development language and IDE for target platform • iOS: Objective-C and Xcode • macOS: C++ and Xcode • Android: Java and Android Studio 2.1+ • Linux: C++ and CMake • Target set of native examples: • Texture tracking example - complete (demos) • InstantOn planar tracking example - in work • Extended tracking example - in work Demo.