SOA Challenges

Service Oriented Architecture (SOA) places signficant priority on software quality and testing.

I just read a vendor whitepaper about SOA.  The paper emphasised a number of points about SOA and software quality, as indicated by the quote:

"Developing an SOA that guarantees service performance, scalable throughput, high availability, and reliability is both a critical imperative and a huge challenge for today’s large enterprises."

This whitepaper from Oracle confirms for me my experience that SOA is placing greater emphasis on the importance of software quality and testing.

Testing becomes the pivotal role in integration activity where SOA is used to glue together larger systems.  It is at testing where we see all the components and the fabric that binds them come together into a working solution.

Not only must testing support function, end-to-end, and acceptance testing of the SOA integration, but as this paper highlights other quality attributes are critical to the success of SOA:

• Performance
• Scalability
• High availability
• Reliability

These are difficult areas to test and evaluate particularly where SOA is integrating many and large separate applications.

My experience on testing projects involving SOA is:

• Building end-to-end test environments is difficult, and we must use stubbing and harness to contain the scope of testing.
• Performance testing is a must, as SOA communication between applications impact user response times significantly.
• SOA is distributed, which means vulnerability to failure of a single component can bring down other interdependent systems and applications.
• Performance monitoring should run tests of user experience periodically.  Many of these tests can be drawn from automated tests used for performance or functional testing.
• Testing is more technical, as tests are constructed around SOA messages.  Tool support (either commercial or open-source) is essential.  Frequently manual / business testers aren't suitable in this areas, as there is no user interface, and instead tests must be coded to send and retrieve messages.
• Asynchronous behaviour of messaging makes functional testing more complex.  We must deal with messages coming back in different orders, it is not always send then receive, other messages may be interleaved.

When adopting SOA, testing teams will need to evolve their strategy regarding how the evaluate and test.  New methodology and technology will need to be incorporated to address the points above.