A Bit of Algebra Massive Amounts of In-memory Key/Value Storage + In-Memory Search + Java == NoSQL Killer? A bit of Algebra Massive Amounts of In-memory Key/Value Storage + In-Memory Search + Java == NoSQL Killer? Kunal Bhasin, Deputy CTO, Terracotta 2 What is NoSQL? . NoSQL = “Not only SQL” . Structured Data not stored in traditional RDBMS . E.g. Key-Value stores, Graph Databases, Document Databases . It really is “Not only RDB” = NoRDB . Key-Value stores – BigTable (disk) – Cassandra – Dynamo – BigTable 3 Why NoSQL? . “One Size Fits All” is .. umm .. a little restrictive . Use the right tool for the job . Or the right strategy depending on business data – Not all data is equal – creation and consumption • Data Volume • Data access patterns • Consistency • Latency, Throughput • Scalability • Availability . Not meant to be “anti”-RDBMS Image Courtesy – Google Images Image URL - http://farm3.static.flickr.com/2523/4193330368_b22b644ddd.jpg http://farm4.static.flickr.com/3620/3402670280_5e8be9f09c.jpg 4 What are we looking for? . Lots of data – > 1 TB to PBs . Performance – Low latency, high throughput access . Scalability and Availability . Flexibility in CAP tradeoffs – Consistency – eventual, strong, ACID – Availability – > 99.99 up time, Durability to failures – Automatic recovery on failures, real time alerts . Flexibility in data consumption – Analytics, Compute 5 Algebra Lots of data + Performance + Scalability and Availability + Flexible CAP tradeoffs + Flexible data consumption = NoSQL or NoRDB 6 What is Ehcache? . Simple API honed by 100,000’s of production deployments Cache cache = manager.getCache("sampleCache1");! Element element = new Element("key1", "value1");! cache.put(element);! . Default cache for popular frameworks • Hibernate, MyBatis, Open JPA • Spring (Annotations), Google Annotations • Grails • JRuby • Liferay • Cold Fusion 7 Simple Get/Put API Sample Code: public Object testCache(String key) throws Exception { CacheManager cacheManager = new CacheManager( “<path to my ehcache.xml>”); Cache myCache = cacheManager.getCache("MyCache"); Object value; Element element = myCache.get(key); if (element == null) { value = "go get it from somewhere like DB or service, etc"; myCache.put(new Element(key, value)); } else { value = (Object) element.getValue(); } return value; } 8 Simple and flexible configuration <ehcache> <defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120” memoryStoreEvictionPolicy="LFU”/> <cache name=”WheelsCache" maxElementsInMemory="10000" timeToIdleSeconds="300” memoryStoreEvictionPolicy="LFU”/> <cache name=”CarCache" maxElementsInMemory="10000" timeToIdleSeconds="300” memoryStoreEvictionPolicy="LFU"/> </ehcache> 9 Efficient implementation – Highly concurrent and scalable – Complements multi-threaded app servers – Max utilization of hardware, scales to multi-core CPUs K V Key 1 Value 1 Key 2 Value 2 Key 3 Value 3 10 Some more features . Pluggable eviction policy . Large caches, GC free . Async write-behind . Bulk loader API’s . JTA support . Management console . Third-party monitoring . WAN replication integration 11 Lots of Data + Performance Ehcache BigMemory 12 Why BigMemory? Java has not kept up with Hardware (because of GC) Base Case Big Heap Stacked 4 GB GC Pause 4 GB Times 4 GB 32 GB Dev / Ops 4 GB Complexity 4 GB 4 GB 4 GB 4 GB 13 BigMemory: Scale Up GC Free . Dramatically increased usable memory per JVM . >64GB/JVM Available Today With BigMemory Memory . 10x JVM density 64GB . Predictable latency . Easier SLAs . No GC pauses . No tuning Max Usable . Pure Java Memory 2GB 14 BigMemory: Scale Up GC Free GC BigMemory • Complex, dynamic reference based • Transparent to Ehcache users, object store • Simple <Key,Value> store with no • Costly (walk the entire object graph) cross-references, to find “unused/unreachable” objects • Uses RAM directly and reclaim memory • Clean interfaces (get, put, remove) for CRUD operations Striped Memory Manager Young Tenured Chunk 1 Chunk 2 Chunk 3 Chunk 4 Buf fer Manager Direct Byte Buffers 15 BigMemory: Scale Up GC Free GC BigMemory New objects created in Young New objects are stored on Generation of heap RAM, away from java heap Striped Memory Manager Young Tenured Chunk 1 Chunk 2 Chunk 3 Chunk 4 Buf fer Manager Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V Direct Byte Buffers 16 BigMemory: Scale Up GC Free GC BigMemory Young generation full causes Hot objects are kept in BigMemory Young GC, costly but not as bad based on access pattern Striped Memory Manager Young Tenured Key,Value Key,Value Chunk 1 Chunk 2 Chunk 3 Chunk 4 Buf fer Manager Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V Direct Byte Buffers 17 BigMemory: Scale Up GC Free GC BigMemory Parallel Collector: Medium to long Objects removed on remove(key), lived objects end up in Tenured TimeToLive, TimeToIdle, frequency of Space access; no need to walk the graph Striped Memory Manager Young Tenured Chunk 1 Chunk 2 Chunk 3 Chunk 4 Buf fer Manager Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V Direct Byte Buffers 18 BigMemory: Scale Up GC Free GC BigMemory Parallel Collector: Long (stop the Highly concurrent and Intelligent world) pauses proportional to size of algorithms to seek “best fit” free the heap and amount of “collectable” memory chunks: No pauses objects Striped Memory Manager Young Tenured Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Chunk 1 Chunk 2 Chunk 3 Chunk 4 Buf fer Manager Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V K,V Key,Value Key,Value Key,Value Key,Value Key,Value K,V Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V K,V Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V Direct Byte Buffers 19 BigMemory: Scale Up GC Free GC BigMemory CMS Fragmentation: Not enough Striped Compaction = contiguous space to copy from No Fragmentation + Good young to tenured, long pauses (stop Performance the world) to run compaction cycles Striped Memory Manager Young Tenured Key,Value Key,Value Key,Value Key,Value Key,Value Chunk 1 Chunk 2 Chunk 3 Chunk 4 Buf Key,Value Key,Value fer Manager Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value Key,Value K,V Direct Byte Buffers Not enough contiguous space = Fragmentation = Full GC 20 BigMemory - Tiered Storage 21 Ehcache with BigMemory App Server JVM JVM Up to 350 GB tested JVM JVM • JVM JVM • < 1 second GC pauses EHCACHE EHCACHE EHCACHE EHCACHE • Standalone or Distributed EHCACHE EHCACHE • > 1 TB with Terracotta Server Array BigMemory 22 Sample ehcache.xml for standalone Flexibility – add BigMem Selectively <ehcache> <defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120” memoryStoreEvictionPolicy="LFU”/> <cache name=”WheelsCache" maxElementsInMemory="10000" timeToIdleSeconds="300” memoryStoreEvictionPolicy="LFU” ! ! overflowToOffHeap="true”! maxMemoryOffHeap=”30G"/> <cache name=”CarCache" maxElementsInMemory="10000" timeToIdleSeconds="300” memoryStoreEvictionPolicy="LFU"/> </ehcache> 23 Scalability & Availability Terracotta Server Array 24 What is Terracotta? . Enterprise class data management – Clustering, Distributed Caching – Highly Available (99.999) – Linear Scale Out – BigMemory - More scalability with less Hardware – ACID, Persistent to Disk (& SSD) – Ease of Operations – Flexibility with CAP tradeoffs 25 Snap In <ehcache>! !<terracottaConfigurl="someserver:9510"/>! !<defaultCache ! ! ! !maxElementsInMemory="10000” ! ! ! !eternal="false” ! ! ! !timeToLiveSeconds="120”/>! !<cache name="com.company.domain.Pets"! ! ! !maxElementsInMemory="10000"! ! ! !timeToLiveSeconds="3000”>! ! ! <terracotta clustered="true” />! !</cache>! </ehcache>! 26 Scale up or Scale out? . Do both .. App Server App Server App Server App Server >64G >1TB 27 BigData? Do you need PBs in BigMemory? Image Courtesy – Google Images Image URL - http://blog.hubspot.com/Portals/249/images//ltail-long-tail.jpg 28 High Availability 2.3.1. Start Terracottathe application Server instances -– Host1Host2 App Server <terracottaConfigurl=”host1:9510,host2:9510"/><serverApp Server host=”host1" name=”host1">! ! ! <dso-port>9510</dso-port>! !<cache name="com.company.domain.Pets"! <server<mirror-group host=”host1" group-name="stripe1"> name=”host1">!! <server!!! host=”host2"maxElementsInMemory="10000" name=”host2">!! <members><dso-port>9510</dso-port>! ! !!! <dso-port>9510</dso-port>timeToLiveSeconds="3000”>! ! !! <terracotta />! <member>host1</member>! ! <jmx-port>9520</jmx-port>! !</cache> <member>host2</member>H ! <member>host1</member>! Heartbeats<data>%(user.home)/local/usg/data</data> detect </members>! T <member>host2</member>GC Tolerance = 2 seconds! and repair failure TCP TCP Network Tolerance = 5 seconds </server</mirror-group>automatically! ! T Fetch cluster topology ! P Cluster topology Active The entire system EachXML stripe is is restartable mirrored and Mirror Group / Stripe without data loss disk backed and has no single x point of failure Terracotta Server Hot Standby start-tc-server.[sh|bat] Stripe1 HotStripe2 mirror servers Stripe3