Dell EMC Ready Architecture for Red Hat Ceph Storage 3.2 Object Storage Architecture Guide Dell EMC Service Provider Solutions ii | Contents Contents List of Figures....................................................................................................................iv List of Tables..................................................................................................................... vi Trademarks.......................................................................................................................viii Notes, Cautions, and Warnings.........................................................................................ix Chapter 1: Introduction..................................................................................................... 10 Introduction.........................................................................................................................................11 Dell PowerEdge R740xd....................................................................................................................11 Dell EMC PowerSwitch S5248F-ON................................................................................................. 12 Chapter 2: Overview of Red Hat Ceph Object Storage................................................... 13 Overview of Red Hat Ceph Storage................................................................................................. 14 Introduction to Ceph storage pools................................................................................................... 15 Selecting storage access method..................................................................................................... 18 Selecting storage protection method.................................................................................................19 BlueStore............................................................................................................................................20 Selecting a hardware configuration................................................................................................... 20 Chapter 3: Architecture components................................................................................21 Architecture overview.........................................................................................................................22 R740xd Storage Node....................................................................................................................... 22 Storage devices................................................................................................................................. 22 Disk controller.................................................................................................................................... 23 CPU and memory sizing....................................................................................................................23 Networking..........................................................................................................................................24 Storage node Ceph NICs.................................................................................................................. 25 Storage node PCIe/NUMA considerations........................................................................................ 25 Storage node hardware configuration............................................................................................... 26 R640 Admin Node............................................................................................................................. 27 Network switches............................................................................................................................... 28 Number of storage nodes..................................................................................................................28 Rack component view........................................................................................................................29 Software............................................................................................................................................. 30 Architecture summary........................................................................................................................ 30 Chapter 4: Test setup....................................................................................................... 32 Physical setup....................................................................................................................................33 Configuring Dell PowerEdge servers.................................................................................................34 Deploying Red Hat Enterprise Linux................................................................................................. 34 Deploying Red Hat Ceph Storage.....................................................................................................35 Metrics collection................................................................................................................................36 Test and production environments compared................................................................................... 36 Chapter 5: Test methodology........................................................................................... 37 Dell EMC Ready Architecture for Red Hat Ceph Storage 3.2 | Object Storage Architecture Guide | Contents | iii Overview.............................................................................................................................................38 Workload generation.......................................................................................................................... 38 COSBench..........................................................................................................................................39 Iterative tuning....................................................................................................................................40 Testing approach................................................................................................................................41 Chapter 6: Hardware baseline testing..............................................................................43 Baseline testing overview.................................................................................................................. 44 CPU baseline testing......................................................................................................................... 44 Network baseline testing....................................................................................................................44 Storage device baseline testing........................................................................................................ 44 Chapter 7: Benchmark test results...................................................................................46 Bottleneck analysis............................................................................................................................ 47 64KB read.......................................................................................................................................... 47 64KB write..........................................................................................................................................48 1MB read............................................................................................................................................49 1MB write........................................................................................................................................... 50 4MB read............................................................................................................................................51 4MB write........................................................................................................................................... 53 Analysis summary..............................................................................................................................54 Chapter 8: Conclusions.................................................................................................... 56 Intel® Optane® P4800X guidance.....................................................................................................57 Conclusions........................................................................................................................................ 57 Appendix A: References................................................................................................... 59 Bill of Materials (BOM)...................................................................................................................... 60 Tested BIOS and firmware................................................................................................................ 61 Configuration details.......................................................................................................................... 61 Benchmark details..............................................................................................................................64 To learn more.....................................................................................................................................64