Pro Couchbase Server David Ostrovsky Yaniv Rodenski Apress Contents J About the Authors xvii About the Technical Reviewers xix Acknowledgments xxi Introduction xxiii Part I: Getting Started 1 Chapter 1: Getting Started with Couchbase Server 3 From SQL to NoSQL 3 The CAP Theorem 3 NoSQL and Couchbase Server 4 Couchbase as Key-Value Store vs. Document Database 5 Couchbase Server Architecture 5 Data Storage 6 Installing Couchbase Server 7 Selecting a Couchbase Server Edition 7 Installing Couchbase on Different Operating Systems 8 Configuring Couchbase Server 9 Creating a Bucket 14 Summary 16 vii CONTENTS Chapter 2: Designing Document-Oriented Databases with Couchbase 17 RanteR: The Anti-Social Network 17 Mapping Application Entities 18 Using a Relational Database 20 Using a Document-Oriented Approach with Couchbase 21 Designing Keys 22 Natural Keys 22 Creating Your First Document 23 Counters 25 Universally Unique Identifiers (UUID) 26 Metadata 26 Document Versioning 27 Versioning Considerations when Using the Couchbase SDKs 27 Versioning Considerations in Couchbase Views 28 Summary 28 Part II: Development 29 Chapter 3: The Couchbase Client Libraries 31 The Java SDK 32 Adding the Java SDK Manually 32 Using Maven 32 Using the CouchbaseClient Class 32 The .NET SDK 35 Manually Referencing the Assemblies 35 Using NuGet Package Manager 35 Using the CouchbaseClient Class 35 The C SDK 38 Linux 38 Windows 39 Mac OS X 39 Using libcouchbase 39 viii m CONTENTS The Node.js SDK 43 The PHP SDK 44 Linux 44 Windows 45 Mac OS X 45 Using the PHP SDK 46 The Python SDK 48 Linux 48 Windows 48 Using the Python SDK 48 The Ruby SDK 50 A Simple Ruby Script 50 Building a Rails Application 51 Inside the Couchbase Client Libraries 53 Thread Safety 54 Couchbase Client Initialization 54 Performing Operations Using the Couchbase Client 55 Reconfiguration 56 Summary 56 Chapter 4: CRUD and Key-Based Operations 57 Persistence and Replication 57 Concurrency 57 Document Expiration 59 Database Operations 60 Storing Data 60 Retrieving Data 63 Retrieving and Updating Data Concurrently 66 Pessimistic Concurrency through Locking 69 Appending and Prepending Binary Data 71 Numeric Data 74 ix ^ CONTENTS Deleting Data 75 Observing Data 76 General SDK-Related Notes 77 Summary 77 Chapter 5: Working with Views 79 MapReduce 101 79 Creating a View in Couchbase 80 The Map Function 82 Creating Views from Code 84 Querying Views 86 The Reduce Function 89 Views and Persistence 94 Updating Indexes 94 Views and Expiration 95 Geospatial Views 96 Creating Geospatial Views 96 Summary 98 Chapter 6: The N1QL Query Language 99 Executing N1QL Queries 100 The N1QL Language 103 The FROM Clause 107 Query Conditions and Expressions 113 Sorting and Grouping 115 Indexing Properties 116 Exploring the Query Execution Details 118 Summary 120 Chapter 7: Advanced Couchbase Techniques 121 Software Error Handling 121 Reading Replica Documents 122 Handling Temporary Out-of-Memory Errors 123 X a CONTENTS Implementing Transactions with Two-Phase Commit 124 Maintaining Sets of Items 134 Using CAS as an ETag 137 Using Couchbase as an Out-of-Process Session Store 139 Summary 141 Chapter 8: ElasticSearch Integration 143 Setting Up ElasticSearch Clusters 143 Forming a Cluster 145 Setting Up Replication between Couchbase and ElasticSearch 145 Querying with ElasticSearch 148 The ElasticSearch Query DSL 150 Summary 162 Part III: Couchbase at Scale 163 Chapter 9: Sizing and Deployment Considerations 165 Planning the Couchbase Cluster 165 RAM 166 Storage 168 Storage Performance 169 CPU 170 Nodes 170 Network Topology 171 Replication 174 Swap Space 175 Summary 175 Chapter 10: Basic Administration 177 Building the Cluster 177 Adding a Node to the Couchbase Cluster 178 Removing a Node from the Cluster 180 Rebalancing 181 xi •;» CONTENTS Swap Rebalancing 183 Changing the Data and Index Path 184 Backing Up and Restoring 184 Using cbbackup to Back Up Data 184 Using cbrestore to Restore Backed-Up Data 187 Using cbtransfer to Transfer Data 189 Backing Up and Restoring Using Data Files 189 Database and View Compaction 190 Dealing with Compaction 190 Auto-Compaction 193 Triggering Compaction Externally 196 Compaction in Action 197 Faiiover 200 Automatic Faiiover 201 Manual or Monitored Faiiover 202 Dealing with Failed Nodes 204 Summary 205 Chapter 11: Monitoring and Best Practices 207 Less Obvious Couchbase Mechanisms 207 Replication and Queues 207 Working-Set Management and Ejection 209 Controlling the Purge Interval 210 Monitoring 210 Monitoring Couchbase at the System Level 210 Detailed Per-Bucket Monitoring 214 Server Resources 216 Summary (the Category) 218 vBucket Resources 222 Disk Queues 224 TAP Queues 226 r, CONTENTS Views 228 XDCR 228 Memcached Buckets 228 Statistics and Cluster Health 229 Using cbstats 230 Retrieving Statistics from the HTTP API 231 Generating a Cluster Health Report with cbhealthchecker 232 Server Warmup 234 Logs 235 Alerts 236 Optimizing Disk Throughput 238 Summary 239 Chapter 12: Couchbase Server in the Cloud 241 Couchbase Server on Amazon Web Services 241 Considerations for Deploying Couchbase Server on AWS 242 Setting up Couchbase Server on AWS 244 Couchbase Server on Microsoft Azure 256 Considerations for Deploying Couchbase on Microsoft Azure 257 Setting Up Couchbase Server on Microsoft Azure 260 Best Practices for Running Couchbase in Microsoft Azure 265 Summary 266 Chapter 13: Cross-Datacenter Replication (XDCR) 267 Prerequisites 267 Setting Up Unidirectional XDCR 268 Advanced XDCR Settings 271 Conflict Resolution 272 Bi-Directional Replication and Advanced Topologies 273 xiii r? CONTENTS Monitoring XDCR 274 Outbound XDCR Operations 274 Incoming XDCR Operations 276 Recovering Data from a Remote Cluster 277 Summary 279 Part IV: Mobile Development with Couchbase 281 Chapter 14: Couchbase Lite on Android 283 Getting Started with Couchbase Lite 284 Adding Couchbase Lite to an Android Project 284 Creating a Couchbase Lite Manager 285 Creating a Couchbase Lite Database 287 Using Couchbase Lite 287 CRUD Operations 288 Attachments 290 Views 291 Defining Views 291 Querying Views 292 Summary 292 Chapter 15: Couchbase Lite on iOS 293 Getting Started with Couchbase Lite on iOS 293 Adding Couchbase Lite to an iOS Project 293 Creating a Couchbase Lite Manager 295 Creating a Couchbase Lite Database 295 Using Couchbase Lite 296 CRUD Operations 296 Attachments 298 xiv CONTENTS Views 299 Defining Views 299 Querying Views 300 Summary 300 Chapter 16: Synchronizing Data with the Couchbase Sync Gateway 301 Couchbase Sync Gateway 301 Installing the Sync Gateway 301 Running the Sync Gateway 302 Working with the Sync Gateway 304 Channels 306 The Sync Function 306 Sync Gateway Administration 309 Authenticating Users 309 Access Control 310 Shadowing Couchbase Server Buckets 311 Deploying the Sync Gateway 311 Summary 312 Index 313 XV.