Top 10 SOA Pitfalls: #3 - Missing skills

Last week Gero Vermaas explained the Incorrectly Applied CDM en this week we’ll continue with #3 - Missing skills

Just like any other paradigm, a level of new knowledge and experience is required. Unfortunately, SOA requires lots of new knowledge and experience. It requires a different way of thinking of more or less everyone involved. People are used to closed environments on both organisational and technical level; largely well protected from influences and unwanted dependencies with the outside world. It's all in their area of influence which makes achieving short term results relatively easy. I'm referring to silo applications where the world is complicated enough. From their view, SOA makes things even more complex. There should be awareness that there is lack of knowledge, experience and attitude and something should be done about this first. There is no real solution, except for the obvious one: educate everyone involved. Also, agile methodologies have proven to be effective in building up knowledge and experience.

Companies we come across introduce SOA in different ways:

• External consultant / architect advises the company to follow this new paradigm. If she is wise enough, she will first focus on the business case for SOA and make people understand why the company needs this architecture style. At the end, this makes things less confusing for people who don't understand SOA. At the same time, making an understandable business case for SOA is a daunting task. But very much worth the effort. At the same time, companies tend to approach SOA in their business cases as a thing which directly delivers value. They don't realize that SOA is just an architectural style and not a thing. Creating business case for SOA means making the match between the real business values and advantages of this architectural style.
• Product vendor has an ESB (Enterprise Service Bus), which seems to match with wanted features, which seems to be backed by real business needs. In the end, everybody is playing with the toy. Playing becomes difficult over time, because knowledge development is often driven by the tool. Non-IT people tends to feel left out and see SOA as something technical. Strangely enough, even they seem to be happy with the big toy without the understanding how the toy solves their problems. Another proof we all have something childish in ourselves.
• A group of people from within the company read the books, attend courses, conferences and so on and somehow realise SOA will solve many of their problems. They get together, start brainstorming and soon realize that this thing called SOA is huge and that there are many area's "where no man has gone before". According to them, if you want to do it well, you should spend a lot of time and effort in thinking and defining it. They act like researchers but without real plan covering the outline of this research, principles, results, criteria when the research is finished and so on. Another proof of childish behaviour. Children love to create something big, are satisfied if they learn something a long the way and be completely equanimous if the whole thing is not used or replaced with something else.

Let me be clear that there are many ways to introduce SOA and that it very much depends on the concrete situation. Even the toy-based approach is in some situations the best possible approach. But, each of them suffers from a lack of knowledge and skills in many area's:

Business Architecture
The term itself is quite new for many organisations and becomes important in SOA world. Creating business architecture in SOA means crossing the departmental boundaries, integrated business process modelling, creation of canonical data models, definition of business services (which are later translated into concrete services), defining different kind of security architecture, gathering business requirements and concerns of many stakeholders, and thinking in SOA design patterns.

Software Requirements Engineering
The toolbox of an requirements engineer is often limited to Use Cases. This methodology is not very suitable in SOA environment because it covers a very limited area. Quality (deprecated term: non-functional) requirements become more important because the complexity of a SOA architecture has larger impact on performance, availability, application management.

Project management
As a project manager, your project has budget and scope. It gets really fuzzy when people start requesting to build generic services and take care of their concerns. Your budget holder could also be reluctant to pay extra for something like this. Especially when you tell her what this means when the system is in production mode. A project manager should aim for two goals: usual delivery of the business product and "readiness" for SOA. These goals could conflict and project manager should be able to manage these conflicts. We see in practice that project managers simply ignore the SOA goals if the conflicts occur.

Software Architects and Developers
They require skills on EAI/SOA patterns, webservices, messaging, ESB's, BPM engines. And particularly difficult to find are specialists with experience in XSLT, Xquery, ebXML and WS-* stack.

Testing
Testers will also need to understand SOA and the exact implications of it on testing. Testing services independently of each other is relatively easy. But what about a complete business process, supply chain testing, transaction handling over multiple services when a transaction must be rolled back, asynchronous messaging, automated testing and so on.

IT management
An IT manager who does not really understand SOA sees it as introduction of some large and expensive infrastructure components which magically solves the integration problems. In large organisations there are several IT managers, one for each department. It proves almost impossible to introduce collective middleware. The problem is the assigned responsibility and ownership. One of the solutions is creation of a the separate Middleware Competence Center or Shared Service Center. This often is a complete new department. If you're lucky, the IT manager and her staff have SOA skills.

Collaborative attitude
Separate departments can have a protective attitude. Although everybody in the company will tell you that collaboration is good for everyone, they will first protect interests of the department before they address overall company interests. The root cause is the unclear advantages of collaboration. People tend to see disadvantages, e.g. additional dependencies.

If a company lacks knowledge in only one these areas, the whole SOA undertaking is in danger. The symptoms are long discussions, sometimes for a year or longer and the only results are large documents and/or some unused middleware infrastructure. The discussions focus on who should do what and how are responsibilities divided. The root problem is that these people have many years of experience, but don't realize they are lacking the skills needed for SOA and they should be doing something about this first.
Hiring experienced consultants to the job only solves the problem if everybody actively participates in the learning process and consultants have coaching role.
Lack of skills can be partially compensated by focusing on short term results and learning from mistakes, which is an Agile way of thinking.

Next week Viktor Grgic will take us to #2...