For Scalyr, It's About Speed, Scale and Simplicity in Log Management
Total Page:16
File Type:pdf, Size:1020Kb
REPORT REPRINT For Scalyr, it’s about speed, scale and simplicity in log management NANCY GOHRING 26 JAN 2017 Scalyr is among the new breed of log management vendors out to challenge Splunk. It developed its own data store so it could deliver very fast search returns for its log management service. THIS REPORT, LICENSED EXCLUSIVELY TO SCALYR, DEVELOPED AND AS PROVIDED BY 451 RESEARCH, LLC, SHALL BE OWNED IN ITS ENTIRETY BY 451 RESEARCH, LLC. THIS REPORT IS SOLELY INTENDED FOR USE BY THE RECIPIENT AND MAY NOT BE REPRODUCED OR REPOSTED, IN WHOLE OR IN PART, BY THE RECIPIENT, WITHOUT EXPRESS PERMISSION FROM 451 RESEARCH. ©2017 451 Research, LLC | WWW.451RESEARCH.COM 451 RESEARCH REPRINT Scalyr is among the new breed of log management vendors out to challenge Splunk. However, log man- agement is just the start for Scalyr, which has a broader vision of being the one tool that DevOps profes- sionals need to track system performance. THE 451 TAKE Scalyr’s log management service stands out for two capabilities: speed and ease of use. The former in par- ticular may help set it apart from competitors since slow response times can be frustrating for users. We like Scalyr’s user interface and think customers will appreciate how easy it is to try out, although many vendors in this sector are similarly focused on their UIs and customers will choose the one that works best for them. Scalyr has a broader vision beyond log management around a centralized IT operations data store, a con- cept we’re increasingly hearing about. The company should seek additional funding to ramp up development around this concept since larger competitors have already laid the groundwork to execute on a similar vision. CONTEXT Scalyr founder Steve Newman built Writely, the online document service that was acquired and became Google Docs, and went on to develop Scalyr with the goal of making log search as painless as a Google search. The service was first launched in 2013, although Scalyr didn’t ramp up in earnest until 2015 after raising $2.1m in seed funding. Total funding is $4.1m from investors that include Susa Ventures, Bloomberg Beta, Google Ventures, Sherpalo Ventures and Heroic Ventures. The company is currently considering whether to seek additional funding in order to drive growth or continue on its current path, which could result in profitability at the end of this year. Scalyr has 13 employees and 80 paying customers including Tivo, Giphy and Codecademy. Pricing for the service is based on average daily log volume and log retention period. PRODUCTS Rather than use open source tools like the Elasticstack or Hadoop, Scalyr built a custom data store for its log management service in order to try to set itself apart from the pack. Its primary differentiator is speed: Scalyr claims that 95% of queries on the system return in under a second and that it searches at a lightning-fast 750GB per second. This home-built back end also supports large volumes of data, with one e-commerce customer regularly collecting 2TB a day and spiking to 10TB on the busiest day of the year. That volume compares favorably with other new entrants in log management that are targeting high-end use cases. While building its own back end has advantages, including allowing Scalyr to develop differentiators like speed, it will have disadvantages, too. Scalyr can’t rely on a broad community of developers refining its technology like its competitors that use open source software can. Among those using open source, Loggly and Logz.io use pieces of the Elasticstack, and Rocana built its technology on open source projects including Kafka and Impala. We think Scalyr’s decision may ultimately slow down its pace of development compared with many competitors. That said, it’s not alone; market leader Splunk has also developed its own back end. Scalyr has taken a unique approach to simplifying usage. Like other log management vendors, Scalyr has discovered that while some end users will learn and use its query language to perform sophisticated searches, others won’t. We’ve seen a number of approaches to simplifying querying, including offering innumerable drop-down menus and displaying many preconfigured graphs. Scalyr’s UI takes some cues from those designed by infrastructure or application monitoring tools, where users can, for instance, select a section of a graph to zoom in on more detail. It also features a left-hand column that lists fields that Scalyr has identified related to a query. Clicking on one of the fields displays a graph showing trends for that field. The approach allows users to examine log data by clicking around on fields that are presented to them, rather than perform complex queries looking for an answer. 451 RESEARCH REPRINT We like the concept and think that users will find the UI easy to navigate. Some customers appear to agree – Scalyr reports that it has a couple of customers with large numbers of users, including one with 1,000 users, the bulk of whom log in once a week or more frequently. Users can set alerts for anything they graph or search, sending alerts to Hipchat, Slack, email or PagerDuty. STRATEGIC VISION AND BUSINESS DRIVERS For now, Scalyr is a log management tool, but the company’s vision – which it stressed is years away – is to serve as a centralized repository for all operational data. Users could potentially tap into the data via third-party tools or Scalyr might end up developing its own analytics and visualization front ends for monitoring application perfor- mance, for example. We’re hearing more and more vendors envision a similar concept of a centralized IT operations data store. It emu- lates a model we’ve seen at some of the largest webscale businesses that have built their own IT operations data repositories that they allow groups within the organization to access in their preferred ways. We think it’s a smart approach that avoids duplicating siloed data stores for each monitoring tool and opens the door to correlation of data collected across systems. Like Scalyr, other vendors are similarly deciding whether to invest in both the front and back end, or focus on unique platform capabilities and leave UI development to others. For vendors like Scalyr with relatively limited resources, we think that a focus on the back end makes sense and also gives customers the flexibility to stick with a front end they like. We see demand for such centralized data stores that can reduce com- plexity for customers, but it’s early, in terms of both the development and end-user adoption. COMPETITION Scalyr is primarily targeting enterprises and when closing deals with large customers it reports that it competes against Splunk. Splunk is the dominant log management vendor and faces a growing number of competitors seeking to deliver a lower-cost alternative and perhaps a targeted differentiator. In Scalyr’s case, customers may choose it for its lower price, speed and because they don’t need the broad feature set that Splunk offers. Sumo Logic, with its service that collects and analyzes both metrics and logs, is also a competitor to Scalyr for enterprise customers, as is the open source Elasticstack. While Scalyr is focused on enterprises, about half of its customers are SMBs and in that customer base, the com- pany competes with a handful of other log management vendors that have also emerged to challenge Splunk. They include Loggly, Papertrail (now owned by SolarWinds) and Logentries (owned by Rapid7). Logz.io, with a capability that offers recommendations for how to remediate a problem, also represents a challenger. In terms of its broader vision around offering a centralized IT operations data repository, Scalyr will likely have some catching up to do because vendors including Rocana, Sumo Logic and Wavefront are already working on executing a similar vision. Rocana similarly targets users that want to collect very large volumes of data, although it collects event data and metrics and allows users to search data from a week ago as easily as from a year ago. Scalyr currently doesn’t offer data retention beyond 30 days. Sumo Logic already collects both logs and metrics, and Wavefront, while currently focused on metrics, has plans to add more capabilities around ingesting logs. 451 RESEARCH REPRINT SWOT ANALYSIS STRENGTHS WEAKNESSES Scalyr’s focus on speed will appeal to many Scalyr currently doesn’t have a long- potential users that have been frustrated by term data retention offering or a package slow search returns via competitive products. aimed at a security use case, both missed opportunities. OPPORTUNITIES THREATS Having built its own back-end data store, Sca- A number of vendors have already begun ex- lyr may be able to develop capabilities that ecuting on a similar vision as Scalyr’s around competitors, many of which rely in part on a centralized IT data repository, meaning established open source technologies, can’t. Scalyr will likely have to play catch-up..