DZone recently sat down with Miko Matsumura, vice president and deputy CTO, SOA products at Software AG webMethods. In this interview, Matsumura discusses SOA leadership witin large organizations, the challenges of aligning IT services with business processes and the difference between open source and commercial SOA offerings. He also talks about the relationship between SOA and agility, and paints his vision for the potential future of SOA adoption: "... a dynamic assemble-to-order economy that will be almost unrecognizably well regulated, multidimensionally optimized and will drive customization down to the individual consumer level."
Aslam Khan: SOA is all about business designing business services and technicians building technologies to execute these services. Yet, we often find the IT department motivating for budget to start a SOA project. In your experience, where should the thought leadership, motivation and sponsorship for a SOA project come from in an organization?
Miko Matsumura: SOA implementation depends on IT and the committment to redesign the Project lifecycle around a governed methodology for service delivery. In many cases, this can be challenging. Now when people talk about "IT", they are talking about Central IT, not Business unit IT. We hope that there is enough excitement and buy-in at the business unit level that they will be willing to coordinate with Central IT and cooperate with an SOA program. In short, we hope to see thought leadership from Central IT, sponsorship from the C-level or as high in the organization as possible, and motivation from the business. Leadership can come from anywhere...
Khan: A while back the software development industry moved from procedural to object thinking. One of the motivators for OO was reusability and loose coupling. Now we are moving towards thinking in services and the SOA promise is the holy grail of loose coupling and reusability. Are we just shifting the problem around or are we really making great strides to achieving these valued features of a good architecture?
Matsumura: Comparing OO to SO is like comparing organelles in a cell to organs in the body. Sure, at a metaphorical and pattern level, there are similarities but the granularity is on a different order of magnitude. The other huge difference is that reuse of source code is dramatically different from "reuse" of live services. From an enterprise perspective it is as different as using a jet airplane vs reusing the CAD Design model for an airplane. You want to fly, not retest, provision, version, support and maintain new systems.
Khan: Good software development projects have good unit tests and integration tests. And having a decent test suite gives us confidence to take the solution into production. Testing of entire business processes is an entirely different matter. Do you think that vendors have given sufficient time to the development of good testing tools for SOA solutions?
Matsumura: The key here is what we call "change time". This is the biggest change since we are reusing, orchestrating and assembling pre-existing services. We know each service already works, but how will the whole system be impacted by a new process, new service consumer or new useage pattern? Reconfiguring an XML file is speedy and provides much more dynamism on a running system than the whole lifecycle of design, build, test, provision, deploy, support, maintain, version, deprecate. But this agility comes with requirements for validation above the unit level. So "testing" isn't so traditional... But the good news is that if you have this problem, you are already on your way.
Khan: Governance can easily become a bureaucratic exercise in a SOA solution. How can we manage governance in teams or organizations with simpler structures and/or those that support agile methodologies?
Matsumura: Governance almost always constrains behavior on behalf of the needs of others. People, especially developers are resistant to being constrained in any fashion. So several key principles apply. First of all "externalization" of policy enables constraints to be lifted out of the code. Pulling security and quality concerns out of the code layer helps make managing services less onerous on the developer. Of course you can only externalize some of the variables and every developer knows that there are application level issues about both security AND quality. So you externalize what you can. Secondly, you automate whatever you can. The more "architecture board" scrutiny and human "approval" steps, the more irritated people tend to get. Thirdly, the rationale for any governance policy needs to be both pre-agreed upon and the rationale needs to be understood by the developer. Nobody wants to comply with policies they either don't agree with or understand. That's just stupid.
Khan: What are the key differences that you have observed between open source SOA offerings and commercial SOA offerings?
Matsumura: Open source has been great at supporting the developer. Aspects of SOA that touch development are well taken care of in SOA including ESB, App Server, Eclipse IDE, SCA container among other things. Frankly, we have yet to see open source addressing "Enterprise" concerns such as Management, Governance, Federation, Application Modernization, B3, Enterprise Applications and B2B concerns. If you just take governance reg/rep as an example, we have put many centuries of engineering person/years into the best commercial products whereas the supposedly similar products in open source are just getting started. I use and love Open Source products every day, so I'm hopeful that things like VC funding of OSS can help push it into a more "Enterprisey" mode.
Khan: In some respects, SOA projects stand the same risk of the ERP project wave of a few years ago. Lots of promises, but lots of budget over-runs, lots of failed projects as well. Briefly, what do you consider are the main risks and how can we mitigate those risks?
Matsumura: Good analogy. The main drivers of failed ERP were cookie-cutter business processes "re-engineered" on top of real-world processes that just didn't fit or make sense. SOA is in danger of "re-engineering" IT processes without understanding or being respectful of the existing ways and their wisdom. On top of this there is added risk in that the drivers for ERP come from the CFO who controls the moneybags. SOA is even more at risk because the CIO (if they are even involved) is a much less powerful figure with respect to commanding budget, particularly in this atmosphere of cost reduction. So we need to first organically redesign IT process, not rip and replace those processes. But even more importantly, we need to continuously report, expose, reveal, communicate and evangelize the business results of SOA at high levels to ensure that SOA remains a strategic program level objective, not just a single project.
Khan: Can SOA drive greater business agility or does improved business agility necessitate a move to SOA?
Matsumura: People talk about Continuous Process Improvement for the business. You can continuously improve hard coded, tightly coupled processes -- it will just be dog slow and expensive as hell. Now implementing SOA requires endurance across multiple projects and you won't see agility until after your 3rd or 4th project. So organizations who can survive this soa adoption danger zone, they will have a sustainable competitive advantage over those who don't. In short, both statements are true, but without an eye on both sides you won't succeed at either (the SOA "platform" side AND the business agility/value side).
Khan: If you had to take a pause and assess the state of the SOA today, what do you see?
Matsumura: People have gone through all of the Kubler-Ross stages now and have come to acceptance of SOA. SOA isn't utopian in the sense that Adoption is painful and difficult. We have been through a lot of cycles where people look for escapist alternatives. But when you get down to it, IT System and Organizational sprawl is a disease, and if you don't cure this disease, your IT systems will become ossified, atherosclerotic, and will slowly become the cement boots that will drown your business. People are understanding both that the cure is as painful as the disease, or perhaps even more painful. It's just that they have now accepted that there aren't too many other options that make sense, at least to large enterprise. Smaller shops can try things like wholesale outsourcing, SaaS and WOA--but at certain levels of size and age you can't count on point solutions anymore.
Khan: Looking into your crystal ball, what do you see on the horizon for SOA and even beyond SOA?
Matsumura: Honestly I see as many failed efforts as successes. There will be some organizations that lose their nerve and declare victory without actually achieving it. Faking it and kidding yourself might even be worse than just quitting. I think there will be some new terms such as "cloud computing" that will emerge to describe new SOA production and consumption patterns. Lots of new aspects to consider including how SOA transforms IT infrastructure management including Virtualization. But SOA is going to be a significant long-haul effort. If enough people go into sour grapes failure mode, the word SOA may be supplanted by a sexier word. But the need to manage distributed heterogeneous business capabilities in spite of System and Organizational sprawl will continue to be the concern of large IT in the near to mid term. Eventually the SOA successes will power a dynamic assemble-to-order economy that will be almost unrecognizably well regulated, multidimensionally optimized and will drive customization down to the individual consumer level. What this will be called I can't say as yet.
Khan: Miko, thank you for your time and your enlightening perspectives on SOA.