Testing the Customer Experience

More service providers are starting to do customer-experience testing. Whether entering new market segments or fending off new competition in established arenas, service providers that take a more comprehensive approach to testing gain the assurance they need that new offerings won’t wreck their reputations with customers, their bottom lines or both.

Traditional enterprise testing seeks to answer whether applications and technologies behind a service launch will perform as intended. Customer-experience testing moves beyond that exercise, going further to evaluate whether the people working with the systems and the business processes defined for them are likely to prove successful.

There’s a standard suite of enterprise tests -- unit testing, string testing, system testing, integration testing, user-acceptance testing, stress-and-volume testing, 508-compliance testing, etc. -- that almost every service provider relies on to ensure the systems underlying an offering will perform as intended. These application and technology tests are critically important.

But as traditionally executed, these tests fail to show whether the service can be implemented and serviced successfully. A service provider must go further. The service provider also must evaluate people, processes, technology and the interrelationships among all three. In doing so, the service provider can eliminate logistical uncertainties and uncover operational complexities that could undermine profitability and ruin its reputation with customers.

Customer-experience testing -- delving more deeply into the service provider’s processes and internal and external user communities -- is designed to deliver these results and benefits.

Expanding the testing program to include customer-experience tests does not have to delay service introduction (and, in fact, might accelerate a successful launch). A service provider can perform the customer-experience tests concurrently with standard application and technology tests, and much of this process can take place during “soft” launch of a service, prior to mass-market rollout. It’s just a matter of interweaving the exercises at key points in the service lifecycle.

Start the customer-experience-testing process from the very outset, even as requirements are being identified, coding is beginning and methods and procedures are being developed. Map requirements to test scenarios. Define traceability and consider compliance. As soon as the test plan is completed, go ahead and assign resources per test.

Perform application testing for systems, training and processes as integration and user-acceptance testing of the system is taking place. At the same time, audit the integration tests to find opportunities for improvement in future test efforts. Use field tests, Web reviews and document audits to confirm regulatory compliance.

Between production release and customer availability is the time to determine whether the service is going to work satisfactorily in terms meaningful to the customer. Use the planned service as a customer might. Evaluate usage procedures and training materials.

Once the service is available, commence with fulfillment testing. Check downstream impacts of the service (across billing and management reporting, for example). Measure and document performance metrics.

Finally, throughout service updates and maintenance, continue with operations tests, validating inputs and outputs and measuring results against SLAs. Seek customer feedback, and make adjustments accordingly.

Relatively easy-to-uncover, simple-to-correct operational glitches can result in significant costs or poor customer reception if they are not identified and eliminated before a service goes live.

Carrier-to-carrier integration issues can be a source of these glitches. It was through pre-production customer-experience testing that one service provider recently discovered that 80,000 Primary Interexchange Carrier/Customer Account Record Exchange (PIC/CARE) orders had been processed incorrectly. The mistake was corrected before production, saving at least a month’s worth of lost revenue from 80,000 individual customers.

This is just one real-world example of how dovetailing customer-experience testing with standard application and technology tests has turned testing into a process that pays for itself. There are plenty of others. Through fulfillment testing, another service provider uncovered an instance where a phone number was listed incorrectly on a direct-mail marketing flier. Through operational testing, a third service provider streamlined its order-input processes and, in turn, halved opportunities for data-entry errors.

More service providers are finding that a comprehensive testing program -- seamlessly integrating application, technology and customer-experience tests -- can yield valuable differentiators in an intensifying competitive environment. Service providers can use test results to enhance customer service, cut prices thanks to process improvements and successfully bring services to market more quickly. No longer considered an unfortunate, inevitable cost of doing business, testing is emerging as a strategic business tool for service providers pursuing new revenue opportunities or facing new threats in once-safe market segments.

Joe Tedesco is CEO and founder of Architech Corp., a provider of tactical software solutions and customer-experience testing services. He can be reached at joe.tedesco@architechcorp.com.

Architech Corp. www.architechcorp.com

Share this article: Email, Slashdot, Digg, Del.icio.us, Yahoo!MyWeb, Windows Live Favorites, Furl
Add this article feed to: RSS, My Yahoo, Newsgator, Bloglines