Google Prediction API and Distributed Machine Learning Max Lin Software Engineer | Google HPC and Cloud Computing Workshop | LBNL & CITRIS UC Berkeley | June 22nd, 2011 Wednesday, June 22, 2011 Outline • What is Machine Learning? • Why Machine Learning on the Cloud? • What is Google Prediction API? • How to scale up machine learning algorithms? Wednesday, June 22, 2011 “Machine Learning is a study of computer algorithms that improve automatically through experience.” Wednesday, June 22, 2011 Wednesday, June 22, 2011 Wednesday, June 22, 2011 Wednesday, June 22, 2011 Wednesday, June 22, 2011 Wednesday, June 22, 2011 The quick brown fox jumped over the lazy dog. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you need a computer. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien no venga. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that is the question Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that ? is the question Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that ? is the question La fe mueve montañas. Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that ? is the question La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training need a computer. No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that ? is the question La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training needInput a computer. X No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that ? is the question La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training needInput a computer. X Output Y No hay mal que por bien Spanish no venga. La tercera es la vencida. Spanish To be or not to be -- that ? is the question La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training needInput a computer. X Output Y No hay mal que por bien Spanish no venga. Model f(x) La tercera es la vencida. Spanish To be or not to be -- that ? is the question La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training needInput a computer. X Output Y No hay mal que por bien Spanish no venga. Model f(x) La tercera es la vencida. Spanish To be or not to be -- that ? Testing is the question La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training needInput a computer. X Output Y No hay mal que por bien Spanish no venga. Model f(x) La tercera es la vencida. Spanish To be or not to be -- that ? Testing is the questionf(x’) La fe mueve montañas. ? Wednesday, June 22, 2011 The quick brown fox English jumped over the lazy dog. To err is human, but to really foul things up you English Training needInput a computer. X Output Y No hay mal que por bien Spanish no venga. Model f(x) La tercera es la vencida. Spanish To be or not to be -- that ? Testing is the questionf(x’) = y’ La fe mueve montañas. ? Wednesday, June 22, 2011 Linear Classifier Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... 1, Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... 1, ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... 1, ... 1, Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... 1, ... 1, ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... 1, ... 1, ... 0, Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... 0, ... 0, ... 1, ... 1, ... 0, ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... [ 0, ... 0, ... 1, ... 1, ... 0, ... Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... [ 0, ... 0, ... 1, ... 1, ... 0, ... ] Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... x [ 0, ... 0, ... 1, ... 1, ... 0, ... ] Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... x [ 0, ... 0, ... 1, ... 1, ... 0, ... ] [0.1, ... 132, ... 150, ... 200, ... -153, ... ] Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... x [ 0, ... 0, ... 1, ... 1, ... 0, ... ] w [0.1, ... 132, ... 150, ... 200, ... -153, ... ] Wednesday, June 22, 2011 Linear Classifier The quick brown fox jumped over the lazy dog. ‘a’ ... ‘aardvark’ ... ‘dog’ ... ‘the’ ... ‘montañas’ ... x [ 0, ... 0, ..