NOSQL WITH CACHING, SEARCH AND REAL-TIME ANALYTICS James Gorlick Basho

bit.ly/bdp-on-the-under-ln @paegun About Me

Application Developer with Lead and Project Management experience in a wide variety of business applications. Particularly interested in straight-through processing, removing data duplication, and increasing information availability throughout the organization in a secure manner. Previously with Dimensional Fund Advisors performed data science with much Fortran and many data sources while helping DFA "buy the market". Developed a number of in-house solutions for Myspace to serve purposes similar to what BDP does and will provide. @paegun Word Count

Just as developer activities extend beyond writing and testing software into what were typically operational concerns, data solutions are extending beyond storing and retrieving data from a single database management system into the deployment and management of heterogeneous services.

A walk through the development and operational support of a reference application utilizing the Basho Data Platform, gains within the development and operations cycles will be demonstrated.

Excellent Data

NLC Origin TLC Station Name Government Office Region (GOR)County or Unitary AuthorityDistrict or Unitary AuthorityNUTS2 Spatial_Unit CodeNUTS2 Spatial Unit Station Facility OwnerStation Group 5131 ABW Abbey Wood London Greater London Greenwich UKI2 Outer London Southeastern Caerphilly - West Wales and the 3813 ABE Aber Wales - Cymru Caerphilly - Caerffili Caerffili UKL1 Valleys Arriva Trains Wales Rhondda Cynon Rhondda Cynon Taff - Taff - Rhondda West Wales and the 3801 ACY Abercynon Wales - Cymru Rhondda Cynon Taf Cynon Taf UKL1 Valleys Arriva Trains Wales Period Emirates and Reporting Days in Period Period Bus journeysRhondda Underground Cynon DLR Journeys Tram Journeys Overground Airline Financial Period period beginningRhondda Cynonending Taff - (m)Taff - Rhonddajourneys (m) (m) West Wales(m) and the Journeys (m) Journeys (m) 3982 ABA Aberdare year Wales - Cymru Rhondda Cynon Taf Cynon Taf UKL1 Valleys Arriva Trains Wales 8976 ABD Aberdeen Scotland Aberdeen City Aberdeen City UKM1 North Eastern Scotland First ScotRail 9090 AUR Aberdour 01_10/11 Scotland1 31 1-Apr-10Fife 1-May-10 189.1Fife 90.5 UKM2 6.3 Eastern Scotland2.3 First ScotRail 02_10/11 2 28 2-May-10 29-May-10 181.6Gwynedd - 84.5 5.8 West Wales2.2 and the 4435 AVY Aberdovey 03_10/11 Wales3 - Cymru28 30-May-10Gwynedd -26-Jun-10 Gwynedd 175.9Gwynedd 84.3 UKL1 5.8 Valleys 2.1 Arriva Trains Wales 04_10/11 4 28 27-Jun-10 24-Jul-10 183.4Gwynedd - 86.5 6.1 West Wales2.1 and the 4440 ABH Abererch Wales - Cymru Gwynedd - Gwynedd Gwynedd UKL1 Valleys Arriva Trains Wales 05_10/11 5 28 25-Jul-10 21-Aug-10 160.4 82.9 5.8 2.0 Monmouthshire - 6 28 22-Aug-10 18-Sep-10 175.8 80.9 5.5 2.0 3602 AGV Abergavenny 06_10/11 Wales - Cymru Monmouthshire - Sir Fynwy Sir Fynwy UKL2 East Wales Arriva Trains Wales 7 28 19-Sep-10 16-Oct-10 189.8 88.7 6.3 2.3 07_10/11 West Wales and the 2448 AGL Abergele & Pensarn08_10/11 Wales8 - Cymru28 17-Oct-10Conwy - Conwy13-Nov-10 179.9Conwy - Conwy90.3 UKL1 6.7 Valleys 2.2 5.6Arriva Trains Wales 09_10/11 9 28 14-Nov-10 11-Dec-10 178.8Ceredigion -90.6 Sir 6.4 West Wales2.3 and the 5.4 4303 AYW Aberystwyth 10_10/11 Wales10 - Cymru28 12-Dec-10Ceredigion8-Jan-11 - Sir Ceredigion140.1Ceredigion 72.5 UKL1 4.8 Valleys 1.8 3.5Arriva Trains Wales 2552 ACR Accrington 11_10/11 North11 West 28 9-Jan-11Lancashire5-Feb-11 183.0Hyndburn 84.4 UKD4 6.3 Lancashire2.1 5.2Northern Rail 8600 AAT Achanalt 12_10/11 Scotland12 28 6-Feb-11Highland 5-Mar-11 177.2Highland 87.8 UKM4 6.5 Highlands2.2 and Islands 5.2First ScotRail 8601 ACN Achnasheen 13_10/11 Scotland13 26 6-Mar-11HighlandPeriod31-Mar-11 and Reporting173.9 DaysHighland in Period83.5 UKM4 6.0 NumberHighlands of 2.1 andOperated Islands Kms5.8 First% ScotRail of Excess 8602 ACH Achnashellach Scotland1 30 1-Apr-11HighlandFinancial30-Apr-11 Period 183.8periodHighland ending91.2 UKM4Month6.4LostHighlands Customer2.1 and(Peak Islands and Off 6.3FirstScheduled ScotRail Journey Time 01_11/12 year Hours Peak) Operated (mins) 02_11/12 2 28 1-May-11 28-May-11 186.1 87.8 6.3 Northumberland2.2 and Tyne6.4 7662 ACK Acklington 03_11/12 North3 East 28 29-May-11Northumberland25-Jun-11 181.7Alnwick 88.9 UKC2 6.1 and Wear2.2 6.9Northern Rail 02_11/12 2 28 28-May-11 May-11 2,322,089 5,444,365 97.3% 5.90 04_11/12 4 28 26-Jun-11 23-Jul-11 186.7 92.5 6.4 2.3 7.6Abellio Greater 7286 ACL Acle East Norfolk 3 Broadland28 25-Jun-11 UKH1 East Anglia Anglia 05_11/12 5 28 24-Jul-1103_11/1220-Aug-11 161.1 85.5 Jun-116.2 1,929,4451.9 5,463,5807.7 97.6% 5.79 London Midland 4 28 23-Jul-11 4529 ACG Acocks Green 06_11/12 West6 Midlands28 21-Aug-11West 04_11/12Midlands17-Sep-11 173.9Birmingham 85.3 UKG3Jul-116.4 West2,300,769 Midlands2.1 5,465,1807.7Trains 96.8% 6.22 07_11/12 7 28 18-Sep-11 15-Oct-11 193.4 93.1 7.5 2.4 8.7 05_11/12 5 28 20-Aug-11 Aug-11 2,059,198 5,595,332 London Midland97.1% 5.32 2287 ACB Acton Bridge 08_11/12 North8 West 28 16-Oct-11Cheshire 12-Nov-11 185.2Vale Royal 95.8 UKD2 7.3 Cheshire2.3 8.8Trains 6 28 17-Sep-11 09_11/12 9 28 13-Nov-1106_11/1210-Dec-11 189.4 97.1 Sep-117.1 2,462,0852.4 5,516,5029.0 96.8% 5.58 10_11/12 10 28 11-Dec-1107_11/127-Jan-12 7 151.228 15-Oct-1179.3 Oct-115.2 2,155,1021.9 5,512,2646.9 97.5% 6.29 11_11/12 11 28 8-Jan-12 4-Feb-12 181.4 89.8 6.9 2.2 8.7 08_11/12 8 28 12-Nov-11 Nov-11 2,054,499 5,657,701 97.5% 6.08 12_11/12 12 28 5-Feb-12 3-Mar-12 179.5 91.5 7.0 2.0 8.8 9 28 10-Dec-11 Dec-11 1,989,066 5,694,703 97.4% 5.69 13_11/12 13 28 4-Mar-1209_11/1231-Mar-12 191.2 92.7 7.3 2.3 9.1 10 28 7-Jan-12 01_12/13 1 28 1-Apr-1210_11/1228-Apr-12 171.8 88.7 Jan-127.0 2,043,9012.1 5,275,1588.5 96.4% 5.07 02_12/13 2 28 29-Apr-1211_11/1226-May-12 11 189.228 4-Feb-1292.6 Feb-127.4 2,073,8842.3 5,730,9488.9 97.1% 5.59 03_12/13 3 28 27-May-12 23-Jun-12 176.5 89.7 7.0 2.2 8.6 12_11/12 12 28 3-Mar-12 Mar-12 2,604,341 5,559,547 95.9% 6.59 04_12/13 4 28 24-Jun-12 21-Jul-12 189.9 96.9 7.9 2.5 8.9 0.24 05_12/13 5 28 22-Jul-1213_11/1218-Aug-12 13 167.128 31-Mar-12103.0 Mar-1210.6 1,997,4312.3 5,682,74010.5 97.4%0.53 5.81 06_12/13 6 28 19-Aug-1201_12/1315-Sep-12 1 177.028 28-Apr-1294.0 8.7 2.4 9.8 0.38 E_TOOMANYOPTIONS Prescriptive, Open

Storage

Consistent Hash (Route) Ring Around The Rosie Multi Data Center CACHE Read-Through Write-Around

Excuse Me Miss Search Ridealong Multi Service Analysis Ship The Compute? Spread The Load No Pets

Graph Reality Window Seat

QUEUE HOST For More Information

• London Datastore - http://data.london.gov.uk/

• Basho Data Platform - http://docs.basho.com/dataplatform/latest/

• Spark - http://spark.apache.org/

• Redis - http://redis.io

• Kafka - http://kafka.apache.org/

• Solr - http://lucene.apache.org/solr/

• Riak - http://docs.basho.com/riak/latest/ REHASH?