Will my applications run slower with cloud storage? A common question when cloud storage is discussed is “will my applications run slower with cloud storage?”. It’s a very valid question, and the answer to that question requires examination of many key elements of the application behavior, the network, any intermediary devices (including cloud storage gateway devices or on-ramp devices like StorSimple), and of course, the cloud storage provider. Looking at typical application deployment scenarios, people tend to deploy storage to support those applications in the data center using one of three methods: – Directly-attached storage (DAS), where the storage is captive to the server (either within the server chassis, or a drive tray that is dedicated to the server). This is generally a SATA, SAS, or other SCSI connection (FC-AL, etc) – Network-attached storage (NAS), where the server accesses a shared pool of storage using file system protocols (such as CIFS or NFS) over the network – Storage-area networks (SAN), where the server accessed volumes that appear as DAS, even though they are accessed over a network and generally provisioned from a shared storage system Generally speaking, these storage attachment methods exhibit high levels of bandwidth, low levels of latency, and low levels of packet loss (one exception being NAS over the WAN, but we won’t get into that here). In the case of cloud storage, your server accesses its storage in one of four (primary) ways: – Using file protocols directly, which is identical to NAS as described above (CIFS or NFS) – Using RESTful HTTP-based APIs, which requires that the application or operating system be prepared to handle any intermediary translation between HTTP and one of the protocols mentioned above – Using SOAP APIs, which again may require translation – Using a cloud storage gateway or on-ramp device In all of these cases, accessing cloud storage occurs over a network, and that network has some measurable amount of latency, packet loss, and some limitation on bandwidth capacity. In the data center, latency, loss, and bandwidth may not be much of a concern, aside from environments using applications that are either high-throughput or transactional in nature. Interestingly enough, in many cases people find that their application servers present storage-related performance challenges of their own as well. In any case, however, decreases in bandwidth, increases in latency, or increases in packet loss can have a severely adverse affect on performance – and this is not just true for storage. With cloud storage, the effects of latency, packet loss, and bandwidth can be amplified, as you are effectively inserting the Internet in between your server and your storage for every IO (assuming you are using a public cloud storage service). Whether or not applications will run more slowly is a function of the BDP (bandwidth delay product) of the network, along with the number of IOPS, average block sizes, and response time requirements that your application would drive or require in an optimal environment. If the BDP of the network cannot provide sustained performance under those metrics, it is highly likely that your application will indeed run more slowly with cloud storage. So what is a person to do? Cloud storage has obvious benefits – pay-on-use, lower cost per GB, elasticity, simpler management, and so on. How can you overcome the possibility of using cloud storage slowing down your data center application? Performance is one of the many benefits of a cloud storage gateway or on-ramp device. Using such a system is not a guarantee that cloud storage will work for your application, as some applications may have storage I/O or access patterns that will not work well even in environments where a gateway is present. However, the majority of applications today that power corporations exhibit two characteristics that make cloud storage gateway devices like StorSimple attractive: – They exhibit high locality of reference in the data they access – They work with data that is typically compressible or able to be deduplicated For example, email data contains text (highly compressible) and attachments (able to be deduplicated), and most people tend to work with email that is the most recent, meaning locality of reference is high (i.e. a ‘working set’ of data can be identified). Document management systems (like SharePoint), file servers, content archives and libraries, and many others, also exhibit these characteristics. The reason cloud storage gateway devices are attractive for these applications is that: – They provide internal storage capacity, which can be used as a high performance on-premises tier to support your working set of data. This means that the majority of IOs are serviced locally, mitigating the need to go to the cloud – They automatically detect shifts in the working set (insertion of new data, older data becoming older and less frequently used), and transparently move that data between faster (integrated storage) and slower (cloud storage) tiers – They compress and deduplicate data, meaning that data is stored efficiently, and fewer bytes are transferred into and out of the cloud. Writes and reads are performed faster, since the amount of data being moved is smaller, meaning less dependence on WAN bandwidth, and less impact in terms of the effect of loss or latency As a side effect, what our customers have noticed is that: – Their performance is consistent with what they had experienced with using traditional storage for the applications we are targeting – The cloud storage capacity costs are reduced, due to deduplication and compression – The cloud storage transfer costs are reduced, due to deduplication, compression, and the benefits of automated data tiering (we have a patented algorithm called Weighted Storage Layout, or WSL, which performs this task) – They can leverage the myriad benefits of cloud storage without the performance worry that typically accompanies use of cloud storage We have a number of customers that are beta testing our system in primary storage use cases for applications that are considered critical to the function of their business, and are finding that performance is consistent with what is provided by traditional storage for those particular applications. If you’d like more information, I encourage you to drop me an email! Cheers, Joel Addressing Performance, Availability, Data Protection, and Security Concerns in Cloud Storage Environments Hi all, We would love to get your feedback on an article we just published on ITBusinessEdge.com, focusing on addressing performance, availability, data protection, and security concerns in cloud storage environments. Please take a read and provide feedback on other issues and concerns you see impacting and slowing broader adoption of cloud storage in your environment. Cloud Storage 101 Episode 3 – How can cloud gateway devices, or ‘on-ramp’ devices help? In the past two episodes of cloud storage 101 (ep1, ep2), we discussed what cloud storage is, what value it provides, and an overview of some of the challenges typically associated with cloud storage. In this episode, we’re going to look at how cloud gateway devices, also known as cloud on-ramp devices, can help make cloud storage a reality. In the spirit of intellectual honesty, we can’t make the bold generalization that cloud storage is applicable to any and every application. Quite the opposite, actually. Cloud storage is a wonderful fit for some applications, and for others, it simply isn’t. I tend to be an optimist, so I’m going to focus on how these devices can make cloud storage very useful, rather than on where it can’t. So, what is a ‘cloud gateway device’, or a ‘cloud on-ramp device’? Put simply, a cloud gateway device is a device deployed behind your firewall that acts as a useful intermediary between you (your application servers) and a cloud service. Typically, cloud gateway devices perform some function that improves performance, security, usability, or other aspects of the service. Several classes of cloud gateway products exist, and we’re going to focus on those that sit between your application servers and cloud storage services, i.e. cloud storage gateway devices. Over the last year there are a handful of companies that have emerged with cloud storage gateway devices, StorSimple included. So what do these devices do? How do they make the cloud storage services useful? The following are general characteristics and functions of these devices, and why the functions matter. 1) They give your application servers familiar protocols to use for accessing storage, and natively speak the language of your cloud storage service so your applications don’t have to. This simplifies integration, because many in-house applications (for instance, Exchange, SharePoint), do not natively speak using RESTful APIs when accessing storage. They expect SCSI in its many forms (iSCSI, FC, FCoE). 2) They optimize transmission of data to and from the cloud to improve performance. Typically, this is accomplished through data deduplication, TCP optimization, compression, and other techniques, which A) minimize the amount of data traversing the wire and B) allow the WAN pipe to be utilized in the most efficient manner. Moving redundancy-eliminated blocks of data moves a tremendous amount of actual data. 3) They secure data being transferred over the network (data in motion), and the data while it is being stored on the cloud storage service (data at rest), generally using keys you supply, which are not shared with your cloud storage service provider. This means that your data is rendered practically useless to third parties that sniff your WAN, compromise your cloud storage service provider’s network or systems, steal your service provider’s hardware, or anyone that becomes the recipient of your data from your service provider through litigation.