Torch Internals
Ronan Collobert [email protected] torch.ch Introduction Machine Learning Library
• Around since 2000 • 5 versions, implemented in C, C++, Objective C…
• Implements main machine learning algorithms: • SVMs, Neural Networks, HMMs, GMMs…
• Goal: research on large-scale machine learning It is for researchers… • It has to be modular! organized as packages • It must be easy to extend! • It must be simple to understand! a high-level language is essential
• It must be fast (large-scale machine learning) JIT-compile or efficient C interface for core routines Overview Overview
optim
torch (lua) nn nngraph
TH (C) svm
sndfile
image Overview
core
optim
torch (lua) nn nngraph
TH (C) svm
sndfile
image Overview
packages
core
optim
torch (lua) nn nngraph
TH (C) svm
sndfile
image Overview
packages
core
optim
torch (lua) nn nngraph
TH (C) svm
sndfile
image Overview
packages
core
optim
torch (lua) nn nngraph
TH (C) svm
sndfile
image Overview
packages
core
optim
torch (lua) nn nngraph
TH (C) svm
sndfile
image Overview
packages
glue core
optim
cwrap torch (lua) nn nngraph
luaT (C) TH (C) svm
sndfile
image Which Programming Language?
• LISP? (Lush http://lush.sourceforge.net) • A subset could be compiled • Could inline C code • JIT-compiled Languages? (e.g. LuaJIT) • Only a subset is actually compiled • Python? • err… Lua? • Looks like pseudo-code (yet very powerful) • Simple C API Which Programming Language?
• C++? (Torch3, PLearn, EBLearn…) • Objective C? (Torch4)
• Avoid too many abstractions • Avoid complicated syntax
• We chose C for Torch7 The Core TH and torch The core library (TH+torch)
• ML algorithms manipulate all kind of data • Represent data as a n-dimensional Tensor • 1D: a bunch of features • 2D: a gray image, some audio features • 3D: RGB images • 4D: Videos… • The core lib provides many tensor operations • Available from C (TH lib) and Lua (torch package) The Core Library (TH+torch) • Avoid memory copies • A Tensor is a view of a Storage (memory chunk) • Storages might have different views (Tensors) • Storages can be in-memory, or on-disk (mmap!) The Core Library (TH+torch) • Avoid memory copies • A Tensor is a view of a Storage (memory chunk) • Storages might have different views (Tensors) • Storages can be in-memory, or on-disk (mmap!) • Easy to get different views from another tensor x: • x:narrow(dim, idx, size) • x:select(dim, idx) • x[idx] • x:unfold(dim, kw, dw) • x:transpose(dim1, dim2) The Core Library (TH+torch) • Avoid memory copies • A Tensor is a view of a Storage (memory chunk) • Storages might have different views (Tensors) • Storages can be in-memory, or on-disk (mmap!) • Easy to get different views from another tensor x: • x:narrow(dim, idx, size) • x:select(dim, idx) new tensors share • x[idx] same storage • x:unfold(dim, kw, dw) • x:transpose(dim1, dim2) The Core Library (TH+torch)
• Avoid memory copies • All C functions are of the form • THTensor_foobar(THTensor *dst, THTensor *src1, THTensor *src2) • The Lua interface supports the destination as an optional argument: torch.foobar([dst], src1, src2) or dst:foobar(src1, src2) The Glue luaT and cwrap C/Lua Glue • Lua is a stack-based language • In Lua: a = f("how", t.x, 14) • In C:
• How to extend it? (luaT or FFI) • Lua provides the “userdata” type (a GCed C pointer) • Need to do proper type checking both in C and Lua
• How to limit the pain? (cwrap or FFI) C/Lua Glue (luaT) C/Lua Glue (luaT)
check arguments on the stack C/Lua Glue (luaT)
check arguments on the stack
push the result C/Lua Glue (luaT)
check arguments on the stack
push the result call TH C/Lua Glue (cwrap) C/Lua Glue (cwrap)
for all tensor types C/Lua Glue (cwrap)
torch.cmul(dst, src) for all tensor types or dst:cmul(src) C/Lua Glue (cwrap) C/Lua Glue (cwrap)
another default value C/Lua Glue (cwrap)
another default value two different C functions
C/Lua Glue (FFI)
• Cut-and-paste the C header • Caveats: • Need to do proper argument checking (sub-classes…) • Need to overload methods/functions properly • Less robust across different systems (#define…) • Need also to handle different types (templates) Tensor Types Dynamic Typing (lua) • Write generic code with torch.Tensor() • Specify alias with (e.g.) torch.setdefaulttensortype(‘torch.FloatTensor’) Dynamic Typing (lua) • Write generic code with torch.Tensor() • Specify alias with (e.g.) torch.setdefaulttensortype(‘torch.FloatTensor’)
return a torch.Tensor() CUDA
• cutorch package defines CudaTensor • Most CPU Tensor methods are available • Relies on few custom-made iterators and thrust • x:cuda() or x:float() for GPU <-> CPU copies
• cunn package provides a GPU backend to nn • Most popular nn layers have a GPU backend Deep Learning nn nn package nn package nn package
• Two main classes: Module and Criterion • Three main methods: • updateOutput(input) • updateGradInput(input, gradOutput) • accGradParameters(input, gradOutput) nn package Tutorial
• Torch7 • https://github.com/torch/torch7 • nn package • https://github.com/torch/nn • itorch • https://github.com/facebook/iTorch • Installation instructions • http://torch.ch/docs/getting-started.html
• Get the tutorial • http://ronan.collobert.com/torch