Red Hat Ceph Storage 3 Installation Guide for Ubuntu Installing Red Hat Ceph Storage on Ubuntu Last Updated: 2021-07-23 Red Hat Ceph Storage 3 Installation Guide for Ubuntu Installing Red Hat Ceph Storage on Ubuntu Legal Notice Copyright © 2021 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/ . In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. The OpenStack ® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community. All other trademarks are the property of their respective owners. Abstract This document provides instructions on installing Red Hat Ceph Storage on Ubuntu 16.04 running on AMD64 and Intel 64 architectures. Table of Contents Table of Contents .C . H. .A . P. .T .E . R. 1.. .W . .H . .A .T . .I S. R. .E . D. H. .A . T. C. .E .P . H. S. .T . O. .R . A. .G . .E .? . .4 . .C . H. .A . P. .T .E . R. 2. R. .E . Q. .U . .I R. .E .M . .E . N. .T . S. .F . O. R. .I .N . S. .T .A . L. .L . I.N . G. R. .E . D. H. .A . T. C. .E . P. .H . .S .T . O. R. .A . G. .E . 6. 2.1. PREREQUISITES 6 2.2. REQUIREMENTS CHECKLIST FOR INSTALLING RED HAT CEPH STORAGE 6 2.3. OPERATING SYSTEM REQUIREMENTS FOR RED HAT CEPH STORAGE 7 2.4. ENABLING THE RED HAT CEPH STORAGE REPOSITORIES 7 Prerequisites 8 Procedure 8 Additional Resources 8 2.5. CONSIDERATIONS FOR USING A RAID CONTROLLER WITH OSD NODES (OPTIONAL) 8 2.6. CONSIDERATIONS FOR USING NVME WITH OBJECT GATEWAY (OPTIONAL) 9 2.7. VERIFYING THE NETWORK CONFIGURATION FOR RED HAT CEPH STORAGE 9 Additional Resources 9 2.8. CONFIGURING A FIREWALL FOR RED HAT CEPH STORAGE 10 2.9. CREATING AN ANSIBLE USER WITH SUDO ACCESS 13 2.10. ENABLING PASSWORD-LESS SSH FOR ANSIBLE 15 Prerequisites 15 Procedure 15 Additional Resources 16 .C . H. .A . P. .T .E . R. 3. D. E. P. .L . O. .Y . I.N . .G . .R . E. .D . .H . A. .T . .C . E. .P .H . S. T. .O . R. .A . G. E. 1. 7. 3.1. PREREQUISITES 17 3.2. INSTALLING A RED HAT CEPH STORAGE CLUSTER 17 Prerequisites 18 Procedure 18 3.3. CONFIGURING OSD ANSIBLE SETTINGS FOR ALL NVME STORAGE 29 3.4. INSTALLING METADATA SERVERS 30 3.5. INSTALLING THE CEPH CLIENT ROLE 31 Prerequisites 31 Procedure 31 Additional Resources 33 3.6. INSTALLING THE CEPH OBJECT GATEWAY 33 Prerequisites 33 Procedure 33 Additional Resources 34 3.6.1. Configuring a multisite Ceph Object Gateway 35 3.7. INSTALLING THE NFS-GANESHA GATEWAY 37 Prerequisites 37 Procedure 37 Additional Resources 38 3.8. UNDERSTANDING THE LIMIT OPTION 38 3.9. ADDITIONAL RESOURCES 39 .C . H. .A . P. .T .E . R. 4. .U . P. .G . R. .A . D. .I N. .G . A. R. .E .D . H. .A . T. .C . E. .P . H. S. .T . O. .R . A. .G . E. C. .L . U. .S . T. E. .R . .4 .0 . Prerequisites 41 4.1. UPGRADING THE STORAGE CLUSTER 42 Procedure 42 .C . H. .A . P. .T .E . R. 5. W. H. A. .T . T. .O . .D . .O . .N . E. .X . T. .? . .4 .6 . .A .P . P. .E . N. .D . I. X. A . .T .R . O. U. .B . L. E. .S . H. .O . O. T. .I N. .G . .4 . 7. 1 Red Hat Ceph Storage 3 Installation Guide for Ubuntu A.1. ANSIBLE STOPS INSTALLATION BECAUSE IT DETECTS LESS DEVICES THAN IT EXPECTED 47 .A .P . P. .E . N. .D . I. X. B . M. A. .N . .U . A. .L .L . Y. I.N . S. .T . A. .L .L . I.N . G. .R . E. D. .H . A. .T . C. .E . P. .H . .S . T. .O . R. .A . G. .E . .4 .8 . B.1. PREREQUISITES 48 Configuring the Network Time Protocol for Red Hat Ceph Storage 48 Prerequisites 48 Procedure: Configuring the Network Time Protocol for RHCS 48 Additional Resources 49 Monitor Bootstrapping 49 B.2. MANUALLY INSTALLING CEPH MANAGER 55 OSD Bootstrapping 56 .A .P . P. .E . N. .D . I. X. C . .I N. .S . T. A. .L . L. .I N. .G . T. H. .E . .C . E. .P . H. C. .O . .M . M. A. .N . D. L. .I N. E. .I N. T. E. .R . F. .A . C. .E . .6 . 2. Prerequisites 62 Procedure 62 .A .P . P. .E . N. .D . I. X. D . .M . A. .N . .U . A. .L .L . Y. I.N . S. .T . A. .L .L . I.N . .G . .C . E. .P . H. B. .L .O . .C . K. D. .E . V. .I C. .E . .6 . 3. Prerequisites 63 Procedure 63 .A .P . P. .E . N. .D . I. X. E . .. M. A. .N . U. .A . L. .L . Y. I.N . S. .T .A . L. .L . I.N . G. .C . E. P. .H . .O . .B . J. .E .C . T. G. .A . T. .E . W. A. .Y . .6 .6 . Prerequisites 66 Procedure 66 Additional Details 68 .A .P . P. .E . N. .D . I. X. F . .. O. V. .E .R . R. .I D. .I .N . G. C. .E . P. .H . .D . E. .F . A. .U . L. T. S. .E . T. T. .I N. .G . .S . .6 .9 . ..