Web services and SOA testing
August 30, 2005 “More and more today, Web services are accomplishing complex tasks and driving mission critical processes. I've always wondered if you ever explained to the business user or process owner how little of their process enabling technology is tested, what would their reaction be.” In this Q&A, Wayne Ariola, vice president of corporate development for Parasoft, explains how this principle needs to be applied to Web services and SOA testing. Q: In one of your recent announcements, you dropped the “P” to rename SOAP Test as SOA Test. But there’s obviously a lot more to it than dropping the P, right? Ariola: Basically, the name SOAP Test implied SOAP and WSDL services. And that’s not bad, and if we were known for that, that would be okay. However, it didn’t represent the breadth of messaging and transport protocols that are present in the product.  Q: In many ways, there’s more to Web services than SOAP and WSDL, right? Ariola: Two years ago, anybody looking at the concept of service-oriented architecture or Web services would be running around preaching “XML, SOAP/WSDL transactions” As people started to size the scope of what the service-oriented architecture was going to touch in their environment, they realized that they had reliable, time tested protocols implemented such as CORBA, JMS, MQ, and TIBCO’s Rendezvous. That’s why we dropped the P from SOAP Test. We have the capability to test multiple protocols throughout a service-oriented transaction. We felt like we were undercutting the capability of the product. Q: Your suite also covers testing for BPEL, Java applications, and Web applications. How does it all fit together? JTest, our Java testing product for automated code analysis and unit testing, has been on the market for eight years. People who are using Java-based services realize that they have an API that makes it easier to manage the introduction of Web services and SOA. However, they’re also finding as they begin to wrap their APIs as a Web service, they need to make sure the application is able to still perform as they originally intended – and more. So unit testing down to the component level becomes extraordinarily important around Java-based applications. They’re now asking an application to do things that it wasn’t originally designed for, at speeds or transaction volumes that it might not have been designed for. A product like JTest, which does automated unit testing, is valuable for that reason. We also do automated unit testing for C/C++ and .NET. So the combination of SOA Test testing the transport or message layer, then having JTest, C++ Test and .Test on the component or the app level, driving unit testing, is a very strong combination. Q: What kind of testing is important for SOA? Ariola: People are adopting a service-oriented architecture for a number of reasons. One of the biggest components is reuse; therefore there is a strong impetus for enterprises to ensure that their services are secure, reliable and complaint. This means that services must be tested for interoperability, unit tested to ensure that the operations are functioning as expected, penetration tested to eliminate any possible security vulnerabilities, scenario tested to cover specific orchestration and usage scenarios, load and performance tested and regression tested. For organizations that are driving for a SOA ROI with the re-use benefit, regression testing becomes a critical component of the quality process. Having a robust process to make sure that a business component not only meets the needs of the business process, but also meets the needs of multiple business processes throughout the environment is a critical path task. Also, many business processes extend past the organizations four walls that developers can control. They need an automated way to manage this expanded testing process, whether it’s stubbing out the service, or adding a data source to the service to see what the inbound messaging would look like, and how they’re handling it. They need a solution that is consistent, flexible and capable of automating the process to extend test coverage and for both positive and negative tests. Q: Web services and SOA architectures could get too complex for traditional means of testing, then? Ariola: If you had a Web application up and running, you could have a QA guy just start pounding on the application. Then to do load testing, you’re going to need some assistance with a piece of software, not only to produce the load requirements but also deliver realistic test scenarios. But with Web services, you don’t have the luxury of testing in a very complex orchestrated nature without the assistance of some sort of automation. You can construct a simple GUI to drive single test cases, but you’re going spend a significant amount of time reconstructing the interface to gain the necessary coverage. It is just too limiting compared to the breadth of processes or business scenarios that an organization should be testing. That is why I’m always wondering what if the process owner or end-user knew how little of their enabling technology is tested. Q: Because you’re dealing with a number of loosely coupled services? Ariola: Yes, absolutely -- that’s where the problem occurs. When it goes outside the four walls, and you expect data back from loosely coupled components, and you need to control the breadth of that process both synchronously and asynchronously across multiple partners, business vendors, or internal applications. Q: That sounds like quite a challenge. What kinds of tests are involved? Ariola: You need a series of steps, and where it starts out with analysis of the structure, interoperable checks, formatting checks, semantic checks, schema checks, the basic structure of the XML transaction, Web services, SOAP, WSDL, up and down the line, transport protocols, through unit testing to make sure that each of the individual components are operating the way you expect. Then there’s scenario testing, where you can actually construct the orchestrated service across multiple business partners, and stubbing out the appropriate services, setting the appropriate timing for synchronous and asynchronous flows. Then there’s security penetration testing, because at one point in the process, you might have access to a database, where another point in the process might be ripe for a SQL injection vulnerability. You need the ability to do penetration testing across the breadth of the business scenario, through regression testing.  Q: How frequently should you test? Ariola: Having the regression suites available and running nightly is a must for your mission critical operations and high volume scenarios. And this is the absolute minimum. This practice can really give the organization an early heads up to potential issues and changes. Remember if a service changes it now affects many people across the enterprise. Once services go live, post provisioning, you definitely need to be running a regressions suite on those services. We’ve seen situations where home office developed a certain set of operations, then they went in to change the operation. They didn’t really understand that one of their subsidiaries was also using that component as party of their business process. So home office has a process running, with no problem or effect. Yet their subsidiary is down, because operation number one changed. Q: Webservices.org recently conducted a survey on testing, and we found that while basic forms of testing are widespread, the numbers drop to 14% for more specialized forms of testing. Is that what you’re seeing? Ariola: Most people at this time are only invoking a limited number of services, where the level of complexity within their environment has not really reached the point where it necessitates a lot of automation, so they forego performance testing or robust load testing at this point in time. Q: Agreed. The survey also found that most companies only have, on the average, about five Web services. Ariola: In my opinion there is a tipping point where the complexity of the environment requires a robust solution. That number is between eight and ten services deployed. Then, the level of complexity increases to the point they start to investigate a more sophisticated means to manage the services. That’s the point where they start investing in robust testing solutions, where they start inquiring about whether a registry is an appropriate technology for them. It’s the point where they start looking at Web services management solutions to perhaps monitor their transactions. And surprisingly, it is the point where they begin to investigate security solutions for Web services such as Reactivity. Q: What is Parasoft’s largest customer site? Ariola: We have over 300 separate organizations using Parasoft SOA test. We have one client with over 75 services in place. Then we have another client with very few services in place, but the number of transactions is insane. Take Yahoo!, for example. They have a tremendous amount of transactions on just a handful of services.  Q: Which industry is taking the lead? Ariola: Our financial services clients are growing their services, and are now beginning to now formulate how they can better manage data, or supply their trading partners or their customers with better data by providing more services, now they have their initial services up and running. So we’re seeing high transaction volumes in financial services with the supplemental transactions now coming online. In insurance services, in comparison, we see a lower number of transactions, but extraordinarily large transaction messages. The orchestration complexity with the insurance industry is also very interesting. As you can imagine, the data needs to be accurate and the process sequence is critical. Luckily, Parasoft SOA Test handles this complexity and extends the test coverage automatically. Q: Financial services companies have a lot of legacy systems. Does this create more complexity in the testing process? Ariola: It depends. In most cases, there’s a lot of trust around these back-end systems. The reason why they’ve sustained through multiple evolutions of disruptive technology is because they’ve served their purpose, dependably. Usually people dip into the wrapper and test that, not necessarily the actual mainframe. However, in security and penetration testing, it does necessitate, at certain points, dipping back into that mainframe. Q: Who do you see as your competition in the Web services application testing market? There are a number of open-source testing tools. Ariola: Yes, our main competition is coming from open source. I guess you can call the IDEs the focus of open source today, such as Eclipse and the Eclipse plug-ins. The reason why Parasoft continues to grow is because we’re able to supply the organization a heterogeneous solution across multiple platforms for every developer, no matter what IDE or environment they’re working in. With open source and the IDEs, a single developer will be able to configure a rule set and run that rule set against their code, or they’ll be able to do a certain level of testing against their code. The open-source tools continue to look at individual operations or niche operations within the software development lifecycle. But that practice is not sustainable. If you have 100 developers, and five of them are using best practices to create great code, that means you have 95 developers creating some shoddy code. Parasoft has collaborative features, and capabilities to share and configure rule sets to meet the specific needs of particular groups. We can create unit test cases that are assets that can be shared throughout the environment -- a seamless quality process from design through development through QA. We offer a holistic end-to-end quality process.  Q: Do you offer the capability to build a repository of best practices for ongoing testing requirements? Ariola: Yes, we have always had an automated library of governance, policies and rules for best coding practices. In fact, in our language-level products and SOA Test, we ship over 600 basic rules that you can analyze and monitor your code. However, even though we have 600 rules or best practices, they’re worthless to you. The only way that they become valuable to you is if you take the time to configure them, so they make sense to your environment, not only logically per application you are building, but also per the people who have to abide by these guidelines. They need to understand the “why” and the “what” – “Why am I doing this?” “What does this mean to me when this rule fires?” Once you have that, you have an organization that understands that if I’m outside the parameters of what this rule base is structured for, then I’m in trouble. For Parasoft’s part, that means that we need to construct best practices and rules and policy guidelines that are very meaningful. We also make these best practices very, very simple to configure. For example, I can configure a new set of best practices in the product in about five minutes; this includes parameterizing specific rule constraints. We are an 18-year-old company, and we’ve invested a significant amount of time not only in pattern-matching capabilities, but also in constructing rule sets that are truly helping developers create better code. Source: Web Services.org USA & Canada, call toll free today at 1-800-547-4149 or just send us an email for additional information on any Web Services offered on this website. Have your website professionally optimized by the search engine positioning experts at Rank for $ales. If your site has dropped in rankings since November 16, 2003, contact the search engine positioning experts at Rank for Sales. Get your business or company listed in the Global Business Listing directory and increase your business. It takes less then 24 hours to get a premium listing in the most powerful business search engine there is. Click here to find out all about it. For the best technical information on hardware, software, Internet applications, e-Commerce, B2B, Web services or IT-related industry news, visit Tech Blog. Reciprocal Link Exchange Program: If your company is engaged in the business of Web Services, the development of related Internet application, ecommerce or B2B development, Internet security services, Web hosting services or is involved in professional Search Engine Optimization, My Web Services is seriously interested in a worthwhile Reciprocal Link Exchange Trading Program with your company. Click here to get all the details.
Sponsored by
Internet Trends
Sponsored by
LCWHG
Sponsored by
ISEN 
|
