DOCSLIB.ORG

What If What I Need Is Not in Powerai (Yet)? What You Need to Know to Build from Scratch?

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
What If What I Need Is Not in Powerai (Yet)? What You Need to Know to Build from Scratch? IBM Systems What if what I need is not in PowerAI (yet)? What you need to know to build from scratch? Jean-Armand Broyelle June 2018 IBM Systems – Cognitive Era Things to consider when you have to rebuild a framework © 2017 International Business Machines Corporation 2 IBM Systems – Cognitive Era CUDA Downloads © 2017 International Business Machines Corporation 3 IBM Systems – Cognitive Era CUDA 8 – under Legacy Releases © 2017 International Business Machines Corporation 4 IBM Systems – Cognitive Era CUDA 8 Install Steps © 2017 International Business Machines Corporation 5 IBM Systems – Cognitive Era cuDNN and NVIDIA drivers © 2017 International Business Machines Corporation 6 IBM Systems – Cognitive Era cuDNN v6.0 for CUDA 8.0 © 2017 International Business Machines Corporation 7 IBM Systems – Cognitive Era cuDNN and NVIDIA drivers © 2017 International Business Machines Corporation 8 IBM Systems – Cognitive Era © 2017 International Business Machines Corporation 9 IBM Systems – Cognitive Era © 2017 International Business Machines Corporation 10 IBM Systems – Cognitive Era cuDNN and NVIDIA drivers © 2017 International Business Machines Corporation 11 IBM Systems – Cognitive Era Prepare your environment • When something goes wrong it’s better to Remove local anaconda installation $ cd ~; rm –rf anaconda2 .conda • Reinstall anaconda $ cd /tmp; wget https://repo.anaconda.com/archive/Anaconda2-5.1.0-Linux- ppc64le.sh $ bash /tmp/Anaconda2-5.1.0-Linux-ppc64le.sh • Activate PowerAI $ source /opt/DL/tensorflow/bin/tensorflow-activate • When you need to install a python package – Use pip install – Use pip install xx==<version> to force the installation of a given © 2017 International Business Machines Corporation 12 IBM Systems – Cognitive Era Keras installation & Tensorflow 1.8.0 compilation © 2017 International Business Machines Corporation 13 IBM Systems – Cognitive Era Install Keras • Set your anaconda in the $PATH $ export PATH=/opt/anaconda2/bin:$PATH • Activate powerAI tensorflow $ source /opt/DL/tensorflow/bin/tensorflow-activate • Install Keras via pip $ pip install keras • Test the stack $ python >>> import keras Using TensorFlow backend. © 2017 International Business Machines Corporation 14 IBM Systems – Cognitive Era Recompile Tensorflow 1.8.0 Download and Build bazel (minimum version for TF 1.8.0 is bazel 0.10) Bazel is build tool that is required to compile tensorflow. $ mkdir bazel $ cd bazel $ wget https://github.com/bazelbuild/bazel/releases/download/0.10.0/bazel-0.10.0-dist.zip $ unzip bazel-0.10.0-dist.zip $./compile.sh Building Bazel from scratch.. … … Build successful! Binary is here: /root/bazel/output/bazel Add bazel binary to the PATH export PATH=/root/bazel/output/:$PATH © 2017 International Business Machines Corporation 15 IBM Systems – Cognitive Era Download and Compile tensorflow (latest release) git clone --recurse-submodules https://github.com/tensorflow/tensorflow.git cd tensorflow/ ./configure Please specify the location of python. [Default is /opt/anaconda2/bin/python]: <leave blank> Please input the desired Python library path to use. Default is [/opt/DL/tensorflow/lib/python2.7/site- packages]: <leave blank> Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]: Y Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: Y Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: Y Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: N Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]: N Do you wish to build TensorFlow with XLA JIT support? [y/N]: N Do you wish to build TensorFlow with GDR support? [y/N]: N Do you wish to build TensorFlow with VERBS support? [y/N]: N Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: N Do you wish to build TensorFlow with CUDA support? [y/N]: Y Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: <leave blank> Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: <leave blank> Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: <leave blank> Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: <leave blank> © 2017 International Business Machines Corporation 16 IBM Systems – Cognitive Era Do you wish to build TensorFlow with TensorRT support? [y/N]: N Please specify the NCCL version you want to use. [Leave empty to default to NCCL 1.3]: <leave blank> Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,5.2]6.0 Do you want to use clang as CUDA compiler? [y/N]: N Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: <leave blank> Do you wish to build TensorFlow with MPI support? [y/N]: N Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -mcpu=native]: -mcpu=power8 Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:N Build pip package bazel build --copt='-std=gnu99' --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg Install the package pip install /tmp/tensorflow_pkg/tensorflow-1.8.0-cp27-cp27mu-linux_ppc64le.whl © 2017 International Business Machines Corporation 17 IBM Systems – Cognitive Era Exit tensorflow build directory and verify installation [root@289961934b8a ~]# python Python 2.7.13 |Anaconda custom (64-bit)| (default, Sep 15 2017, 20:54:27) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org >>> import tensorflow as tf >>> tf.__version__ '1.8.0' >>> © 2017 International Business Machines Corporation 18 IBM Systems – Cognitive Era Get access to TensorFlow's latest build we now have two options for you to access the latest TensorFlow version for Power Systems : 1. TensorFlow version 1.8 for Power has been built and is available through our Linux Technology Center partnership with IBM Unicamp process. You can get TF 1.8 for Power (along with readme info, etc.) right by contacting IBM : – call the Montpellier team for example ! 2. You can also access a Jenkins instance at the Open Source software hub with Oregon State, which builds TensorFlow images as part of a community build. Google includes us in their community distribution, and you can see the build status here: https://github.com/tensorflow/ © 2017 International Business Machines Corporation 19 IBM Systems – Cognitive Era Caffe rebuilt and installation with Large Model Support © 2017 International Business Machines Corporation 20 IBM Systems – Cognitive Era Pre-requisite • Install CUDA Toolkit and driver, CUDNN 7 • Install Openblas lib and devel packages $ sudo yum install openblas-* • Build NCCL $ git clone https://github.com/NVIDIA/nccl.git $ make • Install lmdb packages $ sudo yum install lmdb-* liblmdb-dev • Install protobuf packages $ yum install protobuf-* libprotobuf-dev protobuf-compiler • Install hdf5 packages $ yum install hdf5-* libhdf5-dev • Install boost packages $ sudo yum install boost-* libboost-* • Install leveldb and snappy $ sudo yum install libleveldb-dev libsnappy-dev $ sudo yum install libleveldb-dev libsnappy-dev • Other packages © 2017 International$ sudo Businessyum Machinesinstall Corporationlibgflags-* libgoogle-glog-dev 21 IBM Systems – Cognitive Era Rebuild caffe with LMS • The build and install steps for Caffe LMS follows the standard steps for BVLC Caffe. • Public git hub link for Caffe LMS code: https://github.com/ibmsoe/caffe/tree/master-lms $ git clone https://github.com/ibmsoe/caffe.git $ git checkout master-lms commit d04b5c56edc0a5a046ae5f0ea0edd310aed99d4d • Update Makefile.config to point to dependent library and include file paths • Compile the source code $ make all • Executable files are generated under the folder "builds/tools". © 2017 International Business Machines Corporation 22 IBM Systems – Cognitive Era Validation and test • Environment variable settings: • NOTE: In the below settings, the path must be changed to specify the installation paths on the • system used. • User specific aliases and functions : $ export LD_LIBRARY_PATH=/home/sarithav/DL/nccl/lib:$LD_LIBRARY_PATH • Command used for training using Caffe LMS: $ ./build/tools/caffe train -solver solver.prototxt -gpu 0,1,2,3 -lms 1000 © 2017 International Business Machines Corporation 23 IBM Systems – Cognitive Era torch7 compilation © 2017 International Business Machines Corporation 24 IBM Systems – Cognitive Era Install torch7 Torch7 requires LuaJIT (interpreter) and Luarocks (LUA package manager) to work. The official git repository (https://github.com/torch/luajit-rocks) does not support ppc64le architecture but ported code exists in https://github.com/PPC64/ • Install prerequisites $ yum install –y readline-devel • Clone git repository $ git clone --recurse-submodules https://github.com/regiscely/torch7 • Compile $ cd torch7/ $ export TORCH_NVCC_FLAGS="-D__CUDA_NO_HALF_OPERATORS__" $ ./install.sh ©
Load more

Recommended publications
  • Open Source Copyrights
    Kuri App - Open Source Copyrights: 001_talker_listener-master_2015-03-02 ===================================== Source Code can be found at: https://github.com/awesomebytes/python_profiling_tutorial_with_ros 001_talker_listener-master_2016-03-22 ===================================== Source Code can be found at: https://github.com/ashfaqfarooqui/ROSTutorials acl_2.2.52-1_amd64.deb ====================== Licensed under GPL 2.0 License terms can be found at: http://savannah.nongnu.org/projects/acl/ acl_2.2.52-1_i386.deb ===================== Licensed under LGPL 2.1 License terms can be found at: http://metadata.ftp- master.debian.org/changelogs/main/a/acl/acl_2.2.51-8_copyright actionlib-1.11.2 ================ Licensed under BSD Source Code can be found at: https://github.com/ros/actionlib License terms can be found at: http://wiki.ros.org/actionlib actionlib-common-1.5.4 ====================== Licensed under BSD Source Code can be found at: https://github.com/ros-windows/actionlib License terms can be found at: http://wiki.ros.org/actionlib adduser_3.113+nmu3ubuntu3_all.deb ================================= Licensed under GPL 2.0 License terms can be found at: http://mirrors.kernel.org/ubuntu/pool/main/a/adduser/adduser_3.113+nmu3ubuntu3_all. deb alsa-base_1.0.25+dfsg-0ubuntu4_all.deb ====================================== Licensed under GPL 2.0 License terms can be found at: http://mirrors.kernel.org/ubuntu/pool/main/a/alsa- driver/alsa-base_1.0.25+dfsg-0ubuntu4_all.deb alsa-utils_1.0.27.2-1ubuntu2_amd64.deb ======================================
    [Show full text]
  • Mochi-JCST-01-20.Pdf
    Ross R, Amvrosiadis G, Carns P et al. Mochi: Composing data services for high-performance computing environments. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 35(1): 121–144 Jan. 2020. DOI 10.1007/s11390-020-9802-0 Mochi: Composing Data Services for High-Performance Computing Environments Robert B. Ross1, George Amvrosiadis2, Philip Carns1, Charles D. Cranor2, Matthieu Dorier1, Kevin Harms1 Greg Ganger2, Garth Gibson3, Samuel K. Gutierrez4, Robert Latham1, Bob Robey4, Dana Robinson5 Bradley Settlemyer4, Galen Shipman4, Shane Snyder1, Jerome Soumagne5, and Qing Zheng2 1Argonne National Laboratory, Lemont, IL 60439, U.S.A. 2Parallel Data Laboratory, Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A. 3Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada 4Los Alamos National Laboratory, Los Alamos NM, U.S.A. 5The HDF Group, Champaign IL, U.S.A. E-mail: [email protected]; [email protected]; [email protected]; [email protected]; [email protected] E-mail: [email protected]; [email protected]; [email protected]; [email protected] E-mail: [email protected]; [email protected]; [email protected]; {bws, gshipman}@lanl.gov E-mail: [email protected]; [email protected]; [email protected] Received July 1, 2019; revised November 2, 2019. Abstract Technology enhancements and the growing breadth of application workflows running on high-performance computing (HPC) platforms drive the development of new data services that provide high performance on these new platforms, provide capable and productive interfaces and abstractions for a variety of applications, and are readily adapted when new technologies are deployed. The Mochi framework enables composition of specialized distributed data services from a collection of connectable modules and subservices.
    [Show full text]
  • Luarocks Fostering an Ecosystem of Lua Modules
    LuaRocks fostering an ecosystem of Lua modules Hisham Muhammad [email protected] @hisham_hm FOSDEM !"# $russels% $elgium &hat is LuaRocks 'ackage manager... like d(kg (a(t*get+% RPM% etc. like Ruby-ems, Python eggs, npm% C'/0% etc. ...for Lua e1tension modules written in Lua (.lua files) or ,inary modules (.so4.dll files) Fostering an ecosystem of Lua modules Some uni5ue challenges ...related to Lua as a language ...related to Lua as a community Lessons learned ...on building an ecosystem / one*slide introduction to Lua function process(filename, fn, ...) local f = io.open(filename) local rets = {} for line in f:lines() do rets[#rets+1] = { fn(line, ...) } end f:close() return rets end matches = process("file.txt", string.find, "foo") for i, match in ipairs(matches) do print(i, table.concat(match, ", ")) end / one*slide introduction to Lua function process(filename, fn, ...) local f = io.open(filename) local rets = {} o,6ects for line in f:lines() do rets[#rets+1] = { fn(line, ...) } end f:close() return rets end matches = process("file.txt", string.find, "foo") for i, match in ipairs(matches) do print(i, table.concat(match, ", ")) end / one*slide introduction to Lua function process(filename, fn, ...) local f = io.open(filename) local rets = {} for line in f:lines() do rets[#rets+1] = { fn(line, ...) } end f:close() return rets 3rst*class functions end matches = process("file.txt", string.find, "foo") for i, match in ipairs(matches) do print(i, table.concat(match, ", ")) end / one*slide introduction to Lua function process(filename, fn, ...)
    [Show full text]
  • Tarantool Выпуск 1.7.2
    Tarantool Выпуск 1.7.2 мар. 14, 2019 Оглавление 1 Что нового? 1 1.1 Что нового в Tarantool 1.7?....................................1 2 Общие сведения 2 2.1 Сервер приложений + СУБД...................................2 2.2 Возможности СУБД........................................3 3 Практикум 5 3.1 Практические задания на Lua..................................5 3.2 Практическое задание на C.................................... 17 4 Руководство пользователя 21 4.1 Предисловие............................................. 21 4.2 Начало работы........................................... 22 4.3 Функционал СУБД......................................... 25 4.4 Сервер приложений........................................ 58 4.5 Администрирование серверной части.............................. 72 4.6 Коннекторы............................................. 108 4.7 Вопросы и ответы......................................... 120 5 Справочники 121 5.1 Справочник по встроенной библиотеке............................. 121 5.2 Справочник по сторонним библиотекам............................. 228 5.3 Справочник по настройке..................................... 247 6 Руководство участника проекта 257 6.1 Справочник по C API....................................... 257 6.2 Детали реализации......................................... 280 6.3 Сборка и участие в проекте.................................... 293 6.4 Соглашения по разработке.................................... 298 Lua Module Index 329 i ГЛАВА 1 Что нового? Здесь собрана информация о существенных изменениях, которые
    [Show full text]
  • Enforcing Abstract Immutability
    Enforcing Abstract Immutability by Jonathan Eyolfson A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philosophy in Electrical and Computer Engineering Waterloo, Ontario, Canada, 2018 © Jonathan Eyolfson 2018 Examining Committee Membership The following served on the Examining Committee for this thesis. The decision of the Examining Committee is by majority vote. External Examiner Ana Milanova Associate Professor Rensselaer Polytechnic Institute Supervisor Patrick Lam Associate Professor University of Waterloo Internal Member Lin Tan Associate Professor University of Waterloo Internal Member Werner Dietl Assistant Professor University of Waterloo Internal-external Member Gregor Richards Assistant Professor University of Waterloo ii I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. iii Abstract Researchers have recently proposed a number of systems for expressing, verifying, and inferring immutability declarations. These systems are often rigid, and do not support “abstract immutability”. An abstractly immutable object is an object o which is immutable from the point of view of any external methods. The C++ programming language is not rigid—it allows developers to express intent by adding immutability declarations to methods. Abstract immutability allows for performance improvements such as caching, even in the presence of writes to object fields. This dissertation presents a system to enforce abstract immutability. First, we explore abstract immutability in real-world systems. We found that developers often incorrectly use abstract immutability, perhaps because no programming language helps developers correctly implement abstract immutability.
    [Show full text]
  • A Dataset for Github Repository Deduplication
    A Dataset for GitHub Repository Deduplication Diomidis Spinellis Audris Mockus Zoe Kotti [email protected] {dds,zoekotti}@aueb.gr University of Tennessee Athens University of Economics and Business ABSTRACT select distinct p1, p2 from( select project_commits.project_id as p2, GitHub projects can be easily replicated through the site’s fork first_value(project_commits.project_id) over( process or through a Git clone-push sequence. This is a problem for partition by commit_id empirical software engineering, because it can lead to skewed re- order by mean_metric desc) as p1 sults or mistrained machine learning models. We provide a dataset from project_commits of 10.6 million GitHub projects that are copies of others, and link inner join forkproj.all_project_mean_metric each record with the project’s ultimate parent. The ultimate par- on all_project_mean_metric.project_id = ents were derived from a ranking along six metrics. The related project_commits.project_id) as shared_commits projects were calculated as the connected components of an 18.2 where p1 != p2; million node and 12 million edge denoised graph created by direct- Listing 1: Identification of projects with common commits ing edges to ultimate parents. The graph was created by filtering out more than 30 hand-picked and 2.3 million pattern-matched GitHub contains many millions of copied projects. This is a prob- clumping projects. Projects that introduced unwanted clumping lem for empirical software engineering. First, when data contain- were identified by repeatedly visualizing shortest path distances ing multiple copies of a repository are analyzed, the results can between unrelated important projects. Our dataset identified 30 end up skewed [27]. Second, when such data are used to train thousand duplicate projects in an existing popular reference dataset machine learning models, the corresponding models can behave of 1.8 million projects.
    [Show full text]
  • Conda-Build Documentation Release 3.21.5+15.G174ed200.Dirty
    conda-build Documentation Release 3.21.5+15.g174ed200.dirty Anaconda, Inc. Sep 27, 2021 CONTENTS 1 Installing and updating conda-build3 2 Concepts 5 3 User guide 17 4 Resources 49 5 Release notes 115 Index 127 i ii conda-build Documentation, Release 3.21.5+15.g174ed200.dirty Conda-build contains commands and tools to use conda to build your own packages. It also provides helpful tools to constrain or pin versions in recipes. Building a conda package requires installing conda-build and creating a conda recipe. You then use the conda build command to build the conda package from the conda recipe. You can build conda packages from a variety of source code projects, most notably Python. For help packing a Python project, see the Setuptools documentation. OPTIONAL: If you are planning to upload your packages to Anaconda Cloud, you will need an Anaconda Cloud account and client. CONTENTS 1 conda-build Documentation, Release 3.21.5+15.g174ed200.dirty 2 CONTENTS CHAPTER ONE INSTALLING AND UPDATING CONDA-BUILD To enable building conda packages: • install conda • install conda-build • update conda and conda-build 1.1 Installing conda-build To install conda-build, in your terminal window or an Anaconda Prompt, run: conda install conda-build 1.2 Updating conda and conda-build Keep your versions of conda and conda-build up to date to take advantage of bug fixes and new features. To update conda and conda-build, in your terminal window or an Anaconda Prompt, run: conda update conda conda update conda-build For release notes, see the conda-build GitHub page.
    [Show full text]
  • Luarocks Past, Present and Future
    LuaRocks past, present and future Hisham Muhammad Prologue: what is LuaRocks? ● A package manager for Lua modules ● Modules written in Lua (.lua), binary Lua modules (.so/.dll) and Lua scripts ● Usual features you would expect from a package manager ○ luarocks install <module> ○ luarocks remove <module> ○ ... Rocks and rockspecs ● Rock: a LuaRocks package ○ archive files (actually .zip files) ○ *.src.rock - contains source code ○ *.win32-x86.rock - “binary rock”, contains compiled binaries for a given platform ● Rockspec: a package specification file ○ A declarative Lua script, with rules on how to build and package rocks ○ *.rockspec - a Lua file containing some tables A rockspec Part I The past: a short history of LuaRocks Origins ● Kepler Project: research project to develop a platform for web development using Lua ○ combining modules that already existed (LuaSocket, CGILua) and adding the missing pieces ○ For more of the story, read Yuri Takhteyev’s book, “Coding Places” :) ● I started (re)writing Unix makefiles to automate the packaging/install process ● Common patterns emerged LuaRocks 0.x-1.x: a bumpy start ● 0.x was a gradual evolution ○ the goal for 1.0 was for it to be able to build all Kepler modules ● The rockspec format is unchanged since 1.0 ○ We really care about compatibility ○ Learning the format and writing a rockspec are not disposable efforts ● We got many things right, but we also got some of them wrong... Annoyances in LuaRocks 1.x ● LuaRocks 1.0 did not use the standard Lua layout for modules ○ It wasn’t clear that
    [Show full text]
  • Mlsm: Making Authenticated Storage Faster in Ethereum
    mLSM: Making Authenticated Storage Faster in Ethereum Pandian Raju1 Soujanya Ponnapalli1 Evan Kaminsky1 Gilad Oved1 Zachary Keener1 Vijay Chidambaram1;2 Ittai Abraham2 1University of Texas at Austin 2VMware Research Abstract the LevelDB [15] key-value store. We show that reading a single key (e.g., the amount of ether in a given account) Ethereum provides authenticated storage: each read can result in 64 LevelDB reads, while writing a single returns a value and a proof that allows the client to verify key can lead to a similar number of LevelDB writes. In- the value returned is correct. We experimentally show ternally, LevelDB induces extra write amplification [23], that such authentication leads to high read and write am- further increasing overall amplification. Such write and × plification (64 in the worst case). We present a novel read amplification reduces throughput (storage band- data structure, Merkelized LSM (mLSM), that signifi- width is wasted by the amplification), and write ampli- cantly reduces the read and write amplification while still fication in particular significantly reduces the lifetime of allowing client verification of reads. mLSM significantly devices such as Solid State Drives (SSDs) which wear increases the performance of the storage subsystem in out after a limited number of write cycles [1, 16, 20]. Ethereum, thereby increasing the performance of a wide Thus, reducing the read and write amplification can both range of Ethereum applications. increase Ethereum throughput and reduce hardware re- 1 Introduction placement costs. Modern crypto-currencies such as Bitcoin [21] and We trace the read and write amplification in Ethereum Ethereum [26] seek to provide a decentralized, to the fact that it provides authenticated storage.
    [Show full text]
  • Tarantool Выпуск 1.7.4
    Tarantool Выпуск 1.7.4 мар. 14, 2019 Оглавление 1 Что нового в Tarantool 1.7?2 2 What’s new in Tarantool 1.6.9 after February 15, 2017?4 3 What’s new in Tarantool 1.6?5 4 Общие сведения 6 4.1 Сервер приложений + СУБД...................................6 4.2 Возможности СУБД........................................7 5 Руководство пользователя8 5.1 Предисловие.............................................8 5.2 Начало работы...........................................9 5.3 Функционал СУБД......................................... 13 5.4 Сервер приложений........................................ 30 5.5 Администрирование серверной части.............................. 58 5.6 Репликация............................................. 76 5.7 Коннекторы............................................. 96 5.8 Вопросы и ответы......................................... 107 6 Справочники 109 6.1 Built-in modules reference..................................... 109 6.2 Справочник по сторонним библиотекам............................. 244 6.3 Справочник по настройке..................................... 263 6.4 Utility tarantoolctl .......................................... 280 6.5 Tips on Lua syntax......................................... 280 7 Практикум 282 7.1 Практические задания на Lua.................................. 282 7.2 Практическое задание на C.................................... 294 7.3 libslave tutorial........................................... 299 8 Contributor’s Guide 303 8.1 Справочник по C API....................................... 303 8.2 Internals..............................................
    [Show full text]
  • An Efficient Algorithm for Type-Directed Structural Diffing
    An Efficient Algorithm for Type-Directed Structural Diffing VICTOR CACCIARI MIRALDO, Utrecht University, The Netherlands WOUTER SWIERSTRA, Utrecht University, The Netherlands Effectively computing the difference between two version of a source file has become an indispensable part of software development. The de facto standard tool used by most version control systems is the UNIX diff utility, that compares two files on a line-by-line basis without any regard forthe structure of the data stored in these files. This paper presents an alternative datatype generic algorithm for computing the difference between two values of any algebraic datatype. This algorithm maximizes sharing between the source and target trees, while still running in linear time. Finally, this paper demonstrates that by instantiating this algorithm to the Lua abstract syntax tree and mining the commit history of repositories found on GitHub, the resulting patches can often be merged automatically, even when existing technology has failed. Additional Key Words and Phrases: Generic Programming, diff, Version Control, Haskell 1 INTRODUCTION The UNIX diff [13] is an essential tool in modern software development. It has seen a number of use cases ever since it was created and lies at the heart of today’s Software Version Control Systems. Tools such as git, mercurial and darcs, that enable multiple developers to collaborate effectively, are all built around the UNIX diff utility, which is used to compute a patch between two versions of a file. It compares files on a line-by-line basis attempting to share as many lines as possible between the source and the destination files.
    [Show full text]
  • Enrico Tassi Lua Workshop 2013 [email protected]
    LuaLua and Enrico Tassi Lua workshop 2013 [email protected] Toulouse – 23 Nov Roadmap ● Debian: an opportunity for Lua ● Packaging Lua ● Packaging Lua batteries ● It's not all a bed of roses Who am I? ● Daily job: researcher in CS @ Inria ● completely unrelated topics (type theory, interactive provers, Coq, …) ● I love developing software! ● and I used Lua for some of my projects ● I support free software ● Debian developer since 2006 ● I read lua-l, but I don't post very often What is Debian? ● A project that builds a free OS ● A long tradition in assembling components ● QA: written policies, automatic checks, ... ● An opportunity for Lua: ● Debian is popular with many derivatives ● It supports “exotic” architectures ● It is huge (30K packages, 17K match “^lib”) ● It is tested (compilation & installation time) The beginning lua40 (4.0-1) unstable; urgency=low * Initial Release. * Added support for shared libraries -- Daniel Silverstone <[email protected]> Mon, 11 Feb 2002 Where are we now? Which packages depend on Lua 5.1 aegisub geany-plugin-lua libtolua++5.1-dev pax-britannica apache2-bin genometools libwreport2 pdns-backend-lua aqualung gimp-gluas lighttpd-mod-cml pepper asc gnuplot-x11 love postgresql-9.3-pllua asterisk-modules grafx2 lsyncd premake awesome gringo lua-augeas prosody bam hedgewars luakit python-lua blobby highlight megaglest qcontrol boswars httest monotone radare-common btanks ibus-pinyin mpv ruby-luabridge celestia-gnome instead mudlet simgrid conky-std kamailio-lua-modules mysql-proxy syncmaildir crawl
    [Show full text]