Monitoring Service Compositions In MoDe4SLA: Design of Valdiation

In previous research we introduced the MoDe4SLA approach for monitoring service compositions. MoDe4SLA identifies complex dependencies between Service Level Agreements (SLAs) in a service composition. By explicating these dependencies, causes of SLA violations of a service might be explained by malfunctioning of the services it depends on. MoDe4SLA assists managers in identifying such causes. In this paper we discuss how to evaluate our approach concerning usefulness for the user as well as effectiveness for the business. Usefulness is evaluated by experts who are asked to manage simulated runs of service compositions using MoDe4SLA. Their opinion on the approach is an indicator for its usefulness. Effectiveness is evaluated by comparing runtime results of SLA management using MoDe4SLA with runtime results of unsupported management. Criteria for effectiveness are cost reduction and increase in customer satisfaction.