eBook
ESB and SOA Infrastructure
The Enterprise Service Bus (ESB) is a form of plumbing that enables effective SOA implementation. But understanding the ESB remains an industry-wide quest. Open source consultant Jeff Genender lists a message bus first on his list of the basic building blocks of a successful SOA infrastructure. And yet, some recent Forrester survey results show that first-time SOA infrastructure purchases maybe shifting from ESBs to other SOA technologies.
Read this eBook to learn more about ESBs including:
• The role of ESBs in application integration • Tips on open source and SOA infrastructure from Jeff Genender • Forrester's take on where the ESB fits in today's enterprise architecture
Sponsored By:
SearchSOA.com eBook ESB and SOA Infrastructure
eBook ESB and SOA Infrastructure Table of Contents
The ESB and its role in application integration architecture
Working with ActiveMQ – Tips from TSSJS presenter Jeff Genender part one
On SOA infrastructure – Tips from TSSJS presenter Jeff Genender part two
Forrester analysts: SOA still strong
Resources from FuseSource
Sponsored By: Page 2 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
The ESB and its role in application integration architecture
By Alan Earls
Although enterprise service busses (ESBs) are not new, they can continue to be a nexus for confusion. After many years and many implementations, what they do, how they do it and whether specific products can help create a SOA are all still matters of contention.
Chris Harding, a forum director for SOA and client computing at The Open Group, argues that there isn’t always clarity regarding the nature of ESBs. On the Internet, he points out, there is TCP, which is standardized and central to everything. There is nothing similar on the enterprise software stack for ESBs. ―All you have is a good common understanding of what an ESB does, but you don’t have the tightly defined interfaces. It is a much looser thing,‖ he says.
According to Harding, when people talk about ESBs, the key thing they want is the ability to exchange messages using the ESB as a messaging bus. ―To that basic vision can be added many other things in terms of transforming the messages, in terms of your ability to combine services and even as a composition engine, but the core is the transmission of messages.‖
Historically, Harding says a lot of discussion once focused on whether an ESB is essential for SOA. ―In a theoretical sense, the answer is no,‖ he says. Instead, SOA requires a service- oriented approach to your architecture, the definition of components as services, the identification of service definitions and the loose coupling of those services, he explains. However, notes Harding, when organizations implement loose coupling for the exchange of messages between services, they often find that an ESB as a means of message exchange is the crucial part of their service-oriented architecture.
Ken Volmer, an analyst at Forrester research, says ESBs handle the interactions between different components, creating and routing services, and even providing orchestration.
Sponsored By: Page 3 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
―ESB is the plumbing that enables someone to implement SOA effectively,‖ he adds.
Implementing that broad architectural vision is another matter. Volmer says a good starting point is an architectural reference model. ―There has been broad agreement since the first talk about ESBs that they should include an architecture layer, a connection layer, a mediation layer and an orchestration layer,‖ he says.
Citing a recent Forrester report, ―The ESB Reference Architecture Model,‖ Volmer says that several ESB components must work together to provide the basic ESB framework to deliver availability (typically, through clustering), federation (to support interoperation of ESBs), a topology (such as a hub-and-spoke structure) and extensibility (so that customers can add capabilities on their own).
ESBs are an illustration of service orientation, which is still ―a different way of doing things‖ for many organizations. ―People used to write point integration but with ESB, you should be able to create and then reuse code in different ways,‖ he says. Getting over that idea of having to develop everything from scratch is crucial – and then using an ESB to help implement SOA can make life easier. ―But developers still like to write code, and that can be a problem,‖ he adds.
Volmer says one of the main challenges in implementing an ESB is making sure you have the right product before you buy it. If throughput is a key requirement, test the product on your site before you buy it. ―Don’t buy a pig in a poke,‖ he says.
According to Volmer, implementing an ESB isn’t usually that difficult. He says the degree of challenge depends significantly on the skill level of the people involved with an ESB. ―It is not simple to implement, but we are way beyond the bleeding edge stage; ESBs have been around for 8-10 years so vendors have made them much easier to work with,‖ he says. ―Therefore,‖ he adds, ―implementing an ESB would not significantly challenge a well- organized IT department.‖
Volmer says Forrester has interviewed 20 companies that recently implemented ESBs and no areas emerged as particularly burdensome. ―More than anything, it is a different way of doing integration,‖ he adds. However, he notes, when problems did surface during initial
Sponsored By: Page 4 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
implementations, about 80 percent of them seemed to be architecture related and the balance – only about 20 percent – were linked to tuning issues.
―From my theoretical perspective, the thing that interests me the most about ESBs is how easy it is to port your services from one ESB to another,‖ adds Hardy.
Sponsored By: Page 5 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
Working with ActiveMQ – Tips from TSSJS presenter Jeff Genender part one
For those who aren’t that familiar with ActiveMQ, what might be surprising?
Jeff Genender: Its capabilities. Its ability to scale both horizontally and vertically; [its ability to] work with network clusters. That it’s one of the fastest JMS containers out there. It’s one of most popular JMS containers out there, and one reason is it’s free. We run into a lot of companies migrating from some of the bigger [commercial] JMS implementations, not only because of its capabilities, but because it has many things that are different [and go] above and beyond what a normal JMS implementation can do, and with speed. The fact that it’s open source and better than many commercial implementations people will find fascinating.
What are the most typical problems with ActiveMQ that you’ll be addressing in your presentation at TheServerSide Java Symposium?
Genender: The most common is the container appears to seize up and you need to restart. People try to tweak the memory and JVM parameters, but that usually won’t fix it. That’s one of the biggest areas – people don’t think ActiveMQ is stable and they’ll have to reboot it. The same thing with queues, why do you get a stuck queue and you can’t write to it anymore? We’ll talk about how to fix that. Seizing has to do with tuning parameters. We’ll talk about how to set up ActiveMQ and memory configurations within the MQ configuration files.
The other common problem is architecture, so they’ll be some points on how to properly utilize ActiveMQ. You don’t’ want to use it as a message store, when it’s really an event engine, so [we’ll address] how to properly set up producers and consumers so you’ve got a proper flow going through systems. There are some architectural things people need to think about in going through the development process.
Sponsored By: Page 6 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
How many of the problems are architectural vs. tuning problems?
Genender: About 25% are architectural problems. That seems to be a bigger issue because it has to do with how fast producers are producing and consumers are consuming, and whether you’re using pooling or not. You run into those a lot, usually after the architecture is done, and tuning doesn’t fix it if it’s an architecture problem. These are hot points for folks. They’ve built this system, produced this design and are in production, and now they’re being told they have to change the architecture. We’ll talk about running into these items and what they can do to mitigate this.
The problem is not knowing the container completely or understanding what JMS is all about. A lot of folks will use it as a message store or like a database as opposed to an event engine. That’s one of the biggest areas. You don’t want to park your message here. We’ll point to an example architecture to mitigate that issue.
Do you have an implementation tip you will be discussing, particularly for high load systems?
Genender: For scalability, utilize network brokers - and master-slave failover for high availability. Most of our clients in production will use both a network of brokers for high load, and almost always use master-slave for high availability.
What will the takeaway of your talk be?
Genender: To walk out and be able to implement ActiveMQ with a high degree of confidence that it will scale and perform well, and be able to debug and figure out what’s going on with their own stacks without calling in the experts.
Sponsored By: Page 7 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
On SOA infrastructure – Tips from TSSJS presenter Jeff Genender part two
Any discussion of SOA needs to start with a solid foundation. What are the basic building blocks of a successful SOA infrastructure?
Genender: A message bus/JMS container; some service end points, typically Web services; routing – you need to route your messages and have guaranteed messages; orchestration; and data transformation – some kind of code that transforms data from one component or one state to another.
What are the aspects of SOA infrastructure that organizations tend to struggle with?
Genender: They tend to think everything should be based purely on Web services instead of looking at enterprise integration patterns. You can plug in any type of component, but they will use an ESB as a glorified container. Sometimes the SOA initiative is solely based on Web services.
They’re missing the key aspects, such as lack of the need to be point-to-point. When we show them how to use Web services and routing data through messages—removing the point-to-point—the lights tend to turn on for them.
Another struggle, which goes hand in hand with that, is not understanding enterprise integration patterns, SOA patterns, and how to apply them to their own business processes. Understanding patterns like a message filter or a routing pattern – the lower levels of enterprise integration patterns – make their migration to SOA easier. [SOA] won’t do what they want it to do until they understand all the patterns they can apply to the architecture.
You’d be surprised at the large companies we go into that are putting the entire infrastructure on an ESB and using it as a glorified Web services container. It’s shocking.
Sponsored By: Page 8 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
Forrester analysts: SOA still strong
SOA is still going strong. Businesses that already use SOA are expanding their SOA initiatives and new businesses are starting to adopt SOA and to implement SOA technologies. The first-time SOA infrastructure purchase is shifting from ESBs to other technologies.
According to a recent survey from Forrester analytics, SOA ―still has strong penetration and high satisfaction rates.‖ About 75% of enterprise respondents and 80% of small to midsized businesses that responded are planning on expanding their use of SOA. Back in 2009, the Global 200 enterprises had shown a drop in satisfaction. Only 18% responded that SOA was meeting all or most of their expectations. In 2010, this number rose to 33%, which is more in line with earlier years.
The utilities/telecom sector and the financial/insurance sector are still leading other verticals in terms of SOA adoption and satisfaction. Healthcare and public sectors still trail behind the rest of the pack. Geographically, North America and Europe show similar penetration patterns for SOA, but North America shows higher satisfaction rates.
Probably the most interesting finding of the survey is that interest in ESBs – which have been seen as the starting point for SOA architecture – is actually declining. While in 2009, 48% of respondents who had only made a single SOA purchase had an ESB. In 2010, that number dropped to 39%. According to Forrester analyst Randy Hefner, the shift of focus away from ESBs stems from the increasing similarity of SOA specialty products (like SOA management tools and SOA appliances) as well as a desire to make SOA simpler and stronger.
Sponsored By: Page 9 of 10 SearchSOA.com eBook ESB and SOA Infrastructure
Resources from FuseSource
Free Sneak Peak Preview Chapter of "ActiveMQ in Action"
Enterprise Integration: Patterns and Deployments with Apache ActiveMQ
Apache ActiveMQ: Deploying ActiveMQ in the Enterprise
About FuseSource
FUSE, the first family of open source SOA tools, is a set of standards-based enterprise integration products based on popular Apache SOA projects. The FUSE products are supported by people who are active leaders and committers on the projects. The FUSE team provides professional consulting, training, and enterprise support for the FUSE open source products. FUSE includes four open source components. FUSE ESB is based on Apache ServiceMix. FUSE Message Broker is based on Apache ActiveMQ. FUSE Mediation Router is based on Apache Camel. FUSE Services Framework is based on Apache CXF. The four components are combined with development management tools to provide a comprehensive enterprise SOA infrastructure, and are deployed in many large, mission-critical applications worldwide. The FUSE community can be found at fusesource.com and twitter.com/fusenews
Sponsored By: Page 10 of 10