Open Source Solutions for Building Your Own Storage Area Network and Network Attached Storage Author, Balakrishnan Subramanian A Data Science Foundation White Paper May 2020 --------------------------------------------------- Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 www.datascience.foundation Copyright 2016 - 2017 Data Science Foundation 1. INTRODUCTION Generally, Storage solutions can be grouped into following four categories: SoHo NAS systems, Cloud-based/object solutions, Microsoft Storage Server solutions, Enterprise NAS (Networked Attached Storage) and Storage Area Network (SAN) solutions. Enterprise NAS and SAN solutions are generally closed systems offered by traditional vendors like EMC and NetApp with a very large price tag, so many businesses are looking at Open Source solutions to meet their needs. This is a collection of links and brief descriptions of Open Source storage solutions currently available. Open Source of course means it’s free to use and modify, however some projects have do commercially supported versions as well for enterprise customers who require it. 1. Factors for choosing Storage Solutions When you are in the need of Storage solutions for managing your own infrastructure in your private Data Center, there are many offerings but selecting the right one depends upon your requirement. Factors involved in selecting a Storage type Budget Type of Data that you want to store Scaling concerns and Usage pattern In this article, we will be discussing two different methods (i.e. NAS and SAN), these two methods define the Structure of the storage, it is important to choose the right one based upon your use case and type of data. 2. NETWORK ATTACHED STORAGE A Network Attached Source (NAS) is a device/server attached to the TCP/IP network, the Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 data stored in this device can be shared. When a file read/write is requested in the form of an Network File system (NFS) or Common Internet File System (CIFS), it is sent to the NAS attached to a TCP/IP server, which on receiving the NFS or CIFS request, converts the request into local storage IO command set. 1. Features of NAS NAS uses File Level Data access. The main I/O protocol of NAS is NFS or CIFS. NAS may appear like a “shared folder” but it is connected by means of Ethernet, this can in some cases exacerbate the execution if the system is being utilized for some different things (which are the typical case) or the network slacks. Figure 1: Network Attached Storage with LAN Network (Source: WikiMedia) 3. STORAGE AREA NETWORK Storage Area Network (SAN) is a rapid system of storage device and switches associated with the computer systems. SAN works as the result of sharing the pool with multiple servers. Each server can have direct access to the storage. SAN provides centralized Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 storage management. SAN can move information starting with one storage device then onto the next and can share information between different servers, which makes the restoration and backup of information quickly. A productively and legitimately designed SAN encourages both calamity recuperation and high accessibility. 1. Features of SAN SAN uses block level data access. The main I/O protocol of SAN is Small Computer System Interface (SCSI). SAN appears to the computer as its own storage. It is connected by means of Fiber channel which results in excellent speeds and performance. Figure 2: Storage Area Network with Network (Source: www.slideshare.net) Nowadays many organizations and sectors have switched to Software-defined infrastructure which means they have to store large amount of data safe and secure. Many of the organizations were using open source software to meet their requirement in the section of storing data. E.g.: Projects like Hadoop, Ceph, Gluster. Home users and small businesses can also use open source storage software to Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 meet their needs of storing data, this software allows them to set up their own NAS or SAN device, which uses industry standard hardware without paying the high prices for buying a dedicated storage equipment. It is also possible to set up a cloud storage which is safe, secured and private, it is affordable too when looking for backup and recovery. 4. NAS/SAN OPEN SOURCE SOFTWARE 1. Ceph Red Hat's Ceph offers multiprotocols (i.e. unified) objects and block storage capabilities. It is a distributed storage solution that boasts scalability, reliability and excellent performance. Well-known users include Cisco, DreamHost, CERN, Bloomberg, and Deutsche Telekom. Operating System: Linux 2. CryptoNAS This project aims to simplify the process of setting up an encrypted fileserver. It comes in a live-CD package or a server package that adds a web front-end. Operating System: Linux 3. ESOS Short for Enterprise Storage OS, ESOS is a Linux distribution for setting up a storage array on your own hardware. Commercial support is available. Operating System: Linux 4. FreeNAS FreeNAS claims to be the "the World's No.1 storage OS with over 10+ million downloads." It counts the United Nations, the University of Florida, and the Salvation Army, Reuters, Michigan State University, the Department of Homeland Security and many other organizations among its users. It can be installed on nearly any hardware to turn it into a network attached storage (NAS) device. Paid, supported enterprise solutions based on the same technology are available under the TrueNAS brand name. Operating System: FreeBSD 5. NAS4Free Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 NAS4Free claims to be “the simplest and fastest way to create a centralized and easily accessible server for all kinds of data”. Its features are Zettabyte File System (ZFS), software RAID (levels 0, 1 or 5) and disk encryption. Operating System: FreeBSD 6. Openfiler As a unified storage solution, Openfiler combines the best features of NAS storage and SAN storage. Key features include high availability/failover, block replication and Web-based management. London Metropolitan Police, Bill Me Later, Pratt & Whitney and Motorola are some of the users. A paid commercial version is likewise accessible. Operating System: Linux 7. OpenMediaVault OpenMediaVault describes itself as a ”next-generation network attached storage (NAS) solution." Its goal is to give NAS shape home clients or small workplaces. It offers Web based organization support and incorporates programming RAID abilities. Operating System: Linux 8. Turnkey Linux File Server The Turnkey Linux Project gives pictures to setting up Linux-based servers for various purposes. It supports SFTP, SMB, NFS, rsync and WebDAV file transfer protocols. Operating System: Linux 9. BUILDING OF OUR OWN SAN AND NAS STORAGE USING FREENAS OPEN SOURCE SOFTWARE Boot the machine with the CD/DVD: Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 Figure 3: FreeNAS Installer Select the option 1 Install/Upgrade: Figure 4: FreeNAS Stable Console Setup Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 Select the drive where the Operating System will be installed. We recommend utilizing a Flash drive (8GB) yet you can utilize a Hard Disk as well: Figure 5: Choosing of Destination Media Then root password is specified: The installation is very quick, now you can discharge the DVD/CD and reboot: Data Science Foundation Data Science Foundation, Atlantic Business Centre, Atlantic Street, Altrincham, WA14 5NQ Tel: 0161 926 3641 Email: [email protected] Web: www.datascience.foundation Registered in England and Wales 4th June 2015, Registered Number 9624670 Figure 6: Installation Success Presently FreeNAS is fully operational. The shell educates us of the relegated IP. In our lab network we have an Active Directory Domain Services so we need to design the domain controller as the DNS server. To do as such we should choose choice 6: Data Science Foundation Data Science Foundation, Atlantic Business Centre,