Couchbase Server
Chris Anderson Chief Architect @jchris
1 Couchbase Server
• Simple • Fast = • Elas c • NoSQL Database
Formerly known as Membase Server
2 Couchbase Server Features
• Built-in clustering – All nodes equal
• Data replica on with auto-failover
• Zero--down me maintenance
• Clone to grow and scale horizontally
3 Couchbase Server Basic Opera on
APP SERVER 1 APP SERVER 2 § Docs distributed evenly across
servers in the cluster COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY
§ Each server stores both ac ve CLUSTER MAP CLUSTER MAP & replica docs § Only one server ac ve at a me § Client library provides app with
Read/Write/Update Read/Write/Update simple interface to database § Cluster map provides map to which server doc is on § App never needs to know SERVER 1 SERVER 2 SERVER 3 § App reads, writes, updates Ac ve Docs Ac ve Docs Ac ve Docs docs Doc 5 DOC Doc 4 DOC Doc 1 DOC § Mul ple App Servers can Doc 2 DOC Doc 7 DOC Doc 3 DOC access same document at
same me Doc 9 DOC Doc 8 DOC Doc 6 DOC
Replica Docs Replica Docs Replica Docs
Doc 4 DOC Doc 6 DOC Doc 7 DOC
Doc 1 DOC Doc 3 DOC Doc 9 DOC
Doc 8 DOC Doc 2 DOC Doc 5 DOC
COUCHBASE SERVER CLUSTER 4 Add Nodes
APP SERVER 1 APP SERVER 2
§ Two servers added to COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY cluster
§ One-click opera on CLUSTER MAP CLUSTER MAP § Docs automa cally rebalanced across cluster § Even distribu on of docs Read/Write/Update Read/Write/Update § Minimum doc movement § Cluster map updated § App database calls now distributed over larger # SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 of servers Ac ve Docs Ac ve Docs Ac ve Docs Ac ve Docs Ac ve Docs Ac ve Docs Doc 5 DOC Doc 4 DOC Doc 1 DOC Doc 3 Doc 2 DOC Doc 7 DOC Doc 3 DOC Doc 6 Doc 9 DOC Doc 8 DOC Doc 6 DOC
Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 7
Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 9
Doc 8 DOC Doc 2 DOC Doc 5 DOC
COUCHBASE SERVER CLUSTER 5 Fail Over Node
§ App servers happily accessing docs on Server 3
COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY § Server fails
§ App server requests to server 3 fail
CLUSTER MAP CLUSTER MAP § Cluster detects server has failed
§ Promotes replicas of docs to ac ve § Updates cluster map § App server requests for docs now go to appropriate server § Typically rebalance would follow
SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 Ac ve Docs Ac ve Docs Ac ve Docs Ac ve Docs Ac ve Docs Ac ve Docs Doc 5 DOC Doc 4 DOC Doc 1 DOC Doc 9 DOC Doc 6 DOC Doc 3 Doc 2 DOC Doc 7 DOC Doc 3 DOC Doc 8 DOC Doc 6
Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 5 DOC Doc 8 DOC Doc 7
Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 2 DOC Doc 9
COUCHBASE SERVER CLUSTER 6 Couchbase Server Features
• Memcached compa ble (built-in caching)
• Monitoring and administra on APIs and GUI
• Reliable storage architecture
7 Couchbase Server 1.8 Architecture
11211 11210 Memcapable 1.0 Memcapable 2.0
Moxi
Memcached
Couchbase EP Engine
Data Manager Cluster Manager Heartbeat
storage interface Process monitor Node health monitor Configura on manager Rebalance orchestrator Global singleton supervisor REST management API/Web UI vBucket state and replica on manager
Persistence Layer h p on each node one per cluster
Erlang/OTP
HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 8 Couchbase Server 1.8 Architecture
11211 11210 Memcapable 1.0 Memcapable 2.0
Moxi
Memcached
Couchbase EP Engine Heartbeat
storage interface Process monitor Node health monitor Configura on manager Rebalance orchestrator Global singleton supervisor REST management API/Web UI vBucket state and replica on manager
Persistence Layer h p on each node one per cluster
Erlang/OTP
HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 9 Couchbase Server 1.8 Architecture
11211 11210 Memcapable 1.0 Memcapable 2.0
Moxi
Memcached WebUI
Couchbase EP Engine Heartbeat
storage interface Process monitor Node health monitor Configura on Manager Rebalance orchestrator Global singleton supervisor REST Management / vBucket state and replica on manager
Persistence Layer h p on each node one per cluster
Erlang/OTP
HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 10 Couchbase SDKs
Java SDK User Code
.Net SDK CouchbaseClient cb = new CouchbaseClient(listURIs, Java client API "aBucket", "letmein");
cb.set("hello", 0, "world"); cb.get("hello"); Couchbase Java Library PHP SDK (spymemcached)
Ruby SDK Couchbase Server …and many more h p://www.couchbase.com/develop 11 DEMO TIME
12 Couchbase Server 2.0
• Next major release of Couchbase Server • Currently in Developer Preview
What’s new: • Indexing and Querying • Incremental Map Reduce • Cross Data Center Replica on
• Fully backwards compa ble with exis ng Couchbase Server
13 Couchbase Server 2.0 Architecture
8092 11211 11210 Query API Memcapable 1.0 Memcapable 2.0
Moxi
Memcached Query Engine Couchbase EP Engine Heartbeat
storage interface Process monitor Node health monitor Configura on manager Rebalance orchestrator Global singleton supervisor REST management API/Web UI vBucket state and replica on manager
New Persistence Layer h p on each node one per cluster
Erlang/OTP
HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 14 Indexing and Querying
APP SERVER 1 APP SERVER 2 APP SERVER 1 APP SERVER 2
COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY § Indexing work is distributed
CLUSTER MAP CLUSTER MAP CLUSTER MAP CLUSTER MAP amongst nodes
§ Large data set possible
§ Parallelize the effort § Each node has index for data ResponseQuery stored on it § Queries combine the results from required nodes SERVER 1 SERVER 2 SERVER 3 Ac ve Docs Ac ve Docs Ac ve Docs
Doc 5 DOC Doc 4 DOC Doc 1 DOC
Doc 2 DOC Doc 7 DOC Doc 3 DOC
Doc 9 DOC Doc 8 DOC Doc 6 DOC
Replica Docs Replica Docs Replica Docs
Doc 4 DOC Doc 6 DOC Doc 7 DOC
Doc 1 DOC Doc 3 DOC Doc 9 DOC
Doc 8 DOC Doc 2 DOC Doc 5 DOC
15 Cross Data Center Replica on
US DATA EUROPE DATA ASIA DATA CENTER CENTER CENTER Replica on Replica on
Replica on
§ Data close to users § Mul ple loca ons for disaster recovery § Independently managed clusters serving local data
16 DEMO TIME
17 Demo: The next big social game
3 Objects (documents) within game: • Players • Monsters • Items
Gameplay: • Players fight monsters • Monsters drop items • Players own items
18 Player Document
{ "_id": "Keith4540", "jsonType": "player", "uuid": "35767d02-a958-4b83-8179-616816692de1", "name": "Keith4540", "hitpoints": 75, "experience": 663, "level": 4, "loggedIn": false }
19 Item Document
{ "_id": "Katana_e5890c94-11c6-48-65746ce6c560", "jsonType": "item", "name": "Katana_e5890c94-11c6-65746ce6c560", "uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560", "ownerId": "Dale9887" } Player “_id”
20 Monster Document
{ "_id": "Bauchan9932”, "jsonType": "monster", "name": "Bauchan9932", "uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec", "hitpoints": 370, "experienceWhenKilled": 52, "itemProbability": 0.5050581341872865 }
21 GAME ON!
22 THANK YOU!
Get Couchbase Server 2.0 at h p://www.couchbase.com/downloads
Give us feedback at: h p://www.couchbase.com/forums
23 QUESTIONS
24