Quality properties of IEC 62559 use cases and SGAM models

Abstract

The Smart Grid Architecture Model (SGAM) and the IEC 62559-2 Use Case Template are essential representatives of the Requirements Engineering of energy systems to promote interoperability. In particular, the quality of the use case descriptions and SGAM models is crucial for the system understanding. In order to measure and assess the quality of given use cases of the SGAM models, the aim of this research is to determine those requirements and quality characteristics. Based on a literature review, general best practices for deriving use cases and SGAM models are obtained. The results can be used to concept far-reaching supportive systems or to evaluate IEC 62559 use case descriptions and SGAM models.

Introduction

The Use Case Methodology acc. to IEC 62559 and the Smart Grid Architecture Model (SGAM) developed via the Mandate M/490 represent essential techniques of Requirements Engineering in the Smart Grid context. A primary goal of these methods is the increase of the interoperability understanding by creating use case-descriptions as well as SGAM Models for the system-of-interest (Gottschalk et al. 2017). The development of such descriptions (refered in this paper as IEC 62559 use cases) and SGAM models often are essential parts of contracts in National and European research and innovation projects in the energy sector (e.g., the H2020 / Horizon Europe funding programs) (Schütz et al. 2023).

The quality of the IEC 62559 use cases and SGAM models can be decisive for the design and implementation of the system-of-interest. The relevance of increasing the quality characteristics becomes clear in the usage of use cases and models within or with other organizations. The identical understanding about the system-of-interest across all operating parties is essential and must be clear from the descriptions or model. Minimizing errors in the Requirements Engineering phase of a project is therefore essential. Errors that are not found in the phase, but during the implementation or even during the runtime lead to high additional costs, which can be up to 1000x on average (Haskins et al. 2004). An identification of the quality characteristics for adequate IEC 62559 use cases but also SGAM models can contribute to create a better awareness when creating the descriptions and models.

Objective of this research is the identification of the requirements and quality properties of use case descriptions (acc. to IEC 62559-2 use case template) and SGAM models (acc. to SGAM Reference Architecture). Due to the high interdependencies of the two artifact types, a joint consideration is to be performed. The following research questions are established from the objective:

RQ 1:

Which requirements can be identified for creating IEC 62559 use cases and SGAM-models?

RQ 2:

Which primary quality characteristics can be identified from the model requirements (from RQ 1) for creating IEC 62599 use cases and SGAM models?

The methodology of this study is based on the framework of Design Science framework as it underpins both practical as well as scientific relevance. This underlying paper builds on a model understandingacc. to (Rosemann 2003) andprinciples of propermodeling acc. to (Becker et al. 2012) to first show the origin of quality variants and typical characteristics within models. Other established quality characteristics from science and practice are also used for the analysis. The derivation of the relevant quality characteristics is performed by observing and analyzing a set of requirements for IEC 62559-2 use cases and SGAM models. Figure 1 visualizes the research project and its subjects.

Fig. 1

Placement of the research subjects

Related work

The importance of quality characteristics within the IEC 62559 use case methodology and SGAM framework has already been demonstrated within (Schütz et al. 2021). However quality characteristics were examined in terms of architectural design for interoperability. First metrics with the use of IEC 62559 use cases and SGAM models were brought to hand in the context of evaluating interoperability of architectures (van Amelsvoort et al. 2015). This study distinguishes itself to refer to the identification and metrication of quality characteristics for adequate IEC 62559 use cases and SGAM models, in terms of modeling, model understanding and the general quality dimensions.

The topic of consistency already played a relevant role for IEC 62559 use cases and SGAM as pointed out on existing literature (e.g. Gottschalk et al. 2017; Dänekas et al. 2014; Santodomingo et al. 2014). On the subject of the consistency of IEC 62559 use cases and SGAM, research has also already been conducted with regard to consistency checks (Gottschalk and Sauer 2015). Consistency is a part of the quality dimensions, but it is not the stand-alone relevant characteristic. In the work of Uecker et al. (2021), a proposal to extend the IEC 62559-2 use case template with assurance cases was disseminated to represent IEC 25010 quality attributes for smart grid (quality) requirements. However, the quality properties do not refer to the use cases themselves, but to those of the system-of-interest under consideration. Therefore, this work builds on considering and identifying quality dimensions in the context of Guidelines of Modeling (GoM).

Background

For the analysis of the requirements and quality characteristics of IEC 62559 use cases and SGAM models, it is first necessary to consider the understanding of the models in the research field of Information Systems (IS). The underlying model understanding provides the theoretical framework for identifying challenges for IEC 62559 use cases and SGAM models. The GoM relates to model understanding and define generally applicable guidelines for improving model quality. Those artifacts are to be considered as models but also semi-structured data in the traditional sense. Consequently, in this research it is essential to consider relevant quality characteristics of data for the selection.

Model understanding in information systems

The understanding of models within the field of IS builds on the generally accepted definition of a model. Models can be understood as simplifying or idealizing representations of the reality. The application of a methodology for abstracting the system elements and relationships from reality is necessary to master complexity. The term modeling addresses the process of creating models. A model is the result of a composition of two systems, namely the Object System and the Model System with the assistence of a Mapping Relation. The Object System represents a subjective interpretion of a purpose-oriented selected section of the real world which is also referred as Discource World. The Model System is a subjective representation of the Object System, which strictly conforms to a syntax (e.g., structure, notation, or a language). A Metamodel defines the syntax that can assess the consistency and completeness of the Model System. A Mapping Relation maps the relationship between the Object System and the Model System. Irrelevant facts of the Object System are minimized in complexity elimination and relevant facts of the Object System are minimized in diversity typification. The mapping relation presupposes semantic knowledge from the Metamodels. Subjective modeling goals (called Purpose Relations) have direct effect on both Object and Model Systems as well as the Mapping Relation. The Action Relation includes model-based design of the real world. Figure 2 represents the underlying structure of the model understanding (Rosemann 2003).

Fig. 2

Generic structure of the model understanding [Own representation acc. to (Rosemann 2003)]

Guidelines of modeling

The GoMs relate to model understanding acc. to (Rosemann 2003)and serve as a framework for adequately building models. In modeling systems, it is recognized that different modelers perceive an event differently and thus exhibit variances in the models. In order to minimize the deviations as far as possible, it is necessary to use the rules for modeling languages, which realize a good agreement between object and model system (Becker et al. 2012). Six Guidelines of Modeling have been arisen by the problem (Becker et al. 2012):

Guideline of Correctness:

Two subspecies of correctness are examined here: The syntactic and semantic correctness. A model is considered syntactically correct if all given rules of a modeling language (the metamodel) are observed. A model is considered semantically correct if the structural and behavioral correctness of the model is correct with respect to the object system represented via the Mapping relation function

Guideline of Relevance:

This principle deals with modeling the relevant aspects for the given modeling goal. Abstracting in a model means that not all aspects have to be modeled all the time.

Guideline of Profitability:

The modeling goal should be achieved with minimal effort. The principle specifies that a model should be refined until the costs of the refinement are equal to the resulting benefits from the refinement.

Guideline of Clarity:

The guidelines ensures to make a model easy to interpret, so that it is easy to read, clear, and understandable. The principle of clarity also aims to ensure that the hierarchy, layout design and filtering are appropriate for the addressee.

Guideline of Comparability:

Two aspects are illuminated here: First the processes of the real world and the imaginary world documented in the modeling language must be fully identical in the model. The second aspect aims at the comparison of models created over different modeling languages. These should be mutually translatable.

Guideline of Systematic Structure:

This principle addresses the requirement for inter-model consistency between structure and behavior models.

Data quality properties

Data quality properties are used to determine the quality of data. For this purpose, a large number of properties exist that are used both in industry and in science. Properties can be quantified on data via the use of appropriate evaluation metrics.

Quality properties acc. to ISO/IEC 25012

ISO/IEC 25012 defines general data quality characteristics in the field of software engineering. These are distinguished between inherent data quality characteristics, system-dependent data quality characteristics and the mix of both (ISO/IEC 2008). The ISO/IEC 25012 defines inherent data quality as referring to the degree to which data quality characteristics have intrinsic potential to satisfy implicit data needs. acc. to the standard, the propertiesAccuracy, Completeness, Consistency, Credibility and Currentness are considered.

Usage of data quality dimensions

A data quality dimension is defined as a characteristic that can be used to assess the fulfillment of data requirements and to measure and manage the data quality as well as the information quality (Sidi et al. 2012). Depending on the viewpoint (e.g. from the fields of IS, Data Quality, Software Engineering, Hardware development) different data quality characteristics are to be considered. In the research from (Sidi et al. 2012) executing a survey of data quality dimensions identifying 41 dimensions (e.g.Consistency, Accuracy, Completeness,...) from different fields of research. Frameworks like the 3QM-Framework are using those data quality dimensions based including GoMs for assessing the quality of process models 14 on a quantitative basis. Selected quality properties of the framework areSyntactic/Semantic Correctness, Relevance, Completeness, Flexiblity, Unambiguity and Understandability. For the metrication two ratios were used which apply to specified types of occurrences. The absolute value A refers to the number of occurring defects (e.g. missing components in the diagram) over the corresponding property (e.g. completeness) and The relative value $R = 1-\frac{A}{N}$ refers to the size of the set of absolute occurences.

Requirements analysis for IEC 62559 use cases and SGAM models

The creation of SGAM models and Use Case are subject to a set of requirements. In addition to the primary compliance with the rules on the syntactic and semantic level, further requirements for the creation can be established. Primary sources and fundamental literature on the methodologies represent essential information on the methodologies here, but also guidelines for building proper models. These are consulted during the requirements analysis to derive a set of rules. The literature considered is limited to those listed in Table 1. The extracted requirements for IEC 625592 Use Cases (U), SGAM Models (S) and generics (U/S) are listed in Table 2.

Table 1 Considered literature for the requirements analysis

Table 2 Identified requirements for creating IEC 62559 use cases and SGAM models

Deriving the proposed quality properties

From the determined requirements, but also through the GoMs and the existing quality properties relevant ones can be determined selectively for both IEC 62559 Use Cases and SGAM models.

Observed challenges with applying the view of the model understanding

Heterogeneity within the understanding of the model is a major source of error. This is illustrated in particular in the concretization of the structure from the model understanding for IEC 62559 use cases and SGAM models in Fig. 3. The Object System represents the interpretation of the real world. It consists of the modelling system components and relationships. The Model System represents the actual IEC 62559 use cases or SGAM model. The metamodel refers to the corresponding IEC 62559 use case-standard or the SGAM reference architecture.

Fig. 3

Concretization of the Structure of model understanding for the IEC 62559 use cases and SGAM models [extended from Rosemann (2003)]

The possible sources of error due to subjectivity, are marked with yellow flashes in the figure. Major problems in the creation of high-quality IEC 62559 use cases and SGAM models are the subjective interpretation of the Discourse World, the adequate transformation with the Mapping Relation and the checking of the structural and behaviour compliance as the checking may result to false-negative tests. Especially the cooperation of different stakeholders and actors might result to challenges that lead to semantic, syntactic and pragmatic barriers (Köhlke 2019). Examples could include a different understanding of the System-of-Concern and the methodologies, having different focus areas, mistake on the necessary degree of abstraction or communication barriers in general.

Requirements and GoMs mapping for deriving quality properties and -metrics

The next phase of the research consists in the analysis of the collected model requirements as well as the principles of modeling including the model understanding. Figure 4 represents a model visualization of the process. From the model requirements, quality dimensions can be identified that influence the quality of a use case or SGAM model inductively. While the requirements refer to concrete issues within the models, the quality dimensions only describe the quality on a generic level. Therefore, it became necessary to identify metrics for the corresponding quality characteristic inductively from the requirements using the terms acc. to Rosemann’s theory of model understanding (Rosemann 2003).

Fig. 4

Visualization of the procedure for deriving quality properties

For each identified quality attribute, it is then validated whether it relates to a GoM. Since metrics have a numerical, quantitative nature, it is also necessary to build an absolute and relative formula. The calculation basis of this research is mainly builds on the approach of the 3QM framework (Overhage et al. 2012). Our approach also counts the frequency of events in the absolute calculation basis and considers them in relation to the size of the total quantity in the relative calculation basis. From this methodology, Table 3 has been established, which visualize the the quality dimensions and their metrication and the considered requirements as well as related GoMs from the inductive deriving.

The general goal in modeling should be minimizing the error rate in IEC 62559 use cases and SGAM models in order for maximizing quality. The overall quality can be approximated by aggregating the quality dimensions metrics. An accurate calculation would mean that all quality dimensions must be known, which in the context of this research is limited only to the analyzed requirements. Furthermore, the present understanding of the model allows only a subjective interpretation of the reality, which leads to the fact that the truth can be approximated by the sample of interpretations. For the underlying optimization problem, the truth error function f(x) can be substituted with the approximation of an error function $\tilde{f}(x)$. The approximation of the error function to be minimized can be defined globally for all considered metrics from Table 3$m \in M$ for an IEC 62559 use case or SGAM model ($x \in U \cup S$) under consideration of a metric function $v(x, m) \in \mathbb {R}$ returning the value applying a metric m on a given input x and a weight function $w(m) \in \mathbb {R}$:

$$\begin{aligned} {\mathop {{{\,\mathrm{arg\,min}\,}}}\limits _{x \in U \cup S}} \tilde{f}(x) = {\mathop {{{\,\mathrm{arg\,min}\,}}}\limits _{x \in U \cup S}} \sum _{m \in M} v(m, x) * w(m) \end{aligned}$$

Table 3 Derived quality dimensions and their metrics for IEC 62559 use case and SGAM models acc. to a requirement mapping

Limitations and future work

The results of this study (i.e., both the requirements and quality attributes) provide generic attributes and quality characteristics derived from the (theoretical) literature on the IEC 62559 use case methodology, the SGAM framework and additional work. However, not all requirements are specific enough and offer room for interpretation, especially for the different use case types (high-level vs. low-level use cases vs. business use cases vs. system use cases), which can lead to further detailed requirements for each type. For example, the requirement for SGAM Models ID #9 and #10–#13 (see Table 2) are inpractical at their current state (as these are conscious ambiguous, unclear and not verifiable) as the granularity of different components depends on the objective and perspective of the respective use case description in scope and context. Here applies the system theoretical principle of systems engineering: “The level of detail required by the architect is only to the depth of an element or component critical to the system as a whole” (Maier and Rechtin 2021). Hence, a use case description has to be as generic as possible and as specific as needed (Cherns 1976). As a result, not every component requires the same level of depth or abstraction within the same use case description.

Nevertheless, the current shortcomings can be resolved with appropriate modelling / designing rules and principles for creating IEC 62559-based use cases and SGAM models. To address these shortcomings, follow-up research will identify and evaluate specific measurements for the selected quality attributes for IEC 62559 use cases and SGAM models considering different system levels. In addition, the quality characteristics were derived from the requirements in order to select the most essential ones. In the process, additional properties can also emerge when considering further types of literature. In particular, consideration of yet-to-be-created datasets for both IEC 62559 use case and SGAM models may result in additional properties.

Conclusion

Research aspects of this study were the investigation of requirements but also quality characteristics and metrics on (adequate) IEC 62559 use cases and SGAM models as core representatives of Smart Grid Requirement Engineering methods. The results can be applied in two primary ways. These can be used directly in the creation of future descriptions or models to ensure quality. Indirectly, metrics of the quality characteristics can be used to implement supporting systems [e.g. in Use Case Management Repository (UCMR)] which serve as the evaluation basis for existing and new IEC 62559 use cases as well as SGAM models. But also in follow-up studies, the proposed quality properties and metrics can be considered in the evaluation parts. Since the requirements and quality properties correspond to only a significant subset, further requirements and quality properties and metrics can be derived, especially by considering a data set of IEC 62559-2 use cases and SGAM models which are currently still widely distributed. In addition, follow-up studies build on the findings of this study to develop an intelligent system to improve these quality dimensions and evaluate it on a set of use cases and SGAM models.