Big data-oriented energy prosumption service in smart community districts: a multi-case study perspective

Abstract

The smart grid achieves bidirectional information and energy flow between energy consumer and utility grid, aiding energy users not only to utilize energy, but also to produce, sell, and share energy with other consumers or with the utility grid. This type of energy user is referred to as the “prosumer”. Thus, prosumer management structures are important within energy market. However, prior studies on energy sustainability has paid little attention on prosumer involvement and management. Likewise, the continuous growth of cities has increased data processing complexity. Consequently, processing and analysis of historical, online, and real-time streaming data from energy sensors and metering devices has become a major issue in smart cities. Therefore, this research aims to present an architecture based on big data to improve energy prosumption in smart community districts by applying enterprise architecture approach grounded on The Open Group Architecture Framework (TOGAF). Accordingly, qualitative methodology is adopted to collect data by employing case study by focus group interview from two energy companies in Norway to preliminarily validate the architecture. Findings from the case studies was demonstrated in ArchiMate modeling language to evaluate the applicability of the architecture. Moreover, findings from this study provides practical scenario that energy service providers can refer to in designing their own energy data platforms. Essentially, the architecture can be utilized as a guide to help municipalities and policy makers in creating approach for energy data analytics in smart community districts towards making decisions for future energy prosumption planning.

Introduction

Climate change is putting pressure on policy makers, governments, global industries, and the international community to deploy renewable energy sources and improve energy efficiency (Li et al. 2017; Anthony Jr et al. 2019). To attain the need for cleaner energy and comply with economic and ecologic demands, the electricity market structure is gradually transitioning from a centralized model to more decentralized and interactive approach based on smart grid technology where energy consumers may play a role as prosumers who produce, sell or share surplus energy (Wentland 2016). Thus, development of smart grid energy sharing operation depends on prosumers involvement (Bohnsack et al. 2014). Nonetheless, energy prosumption entails the collaboration and interactions between stakeholders which involves prosumers, energy service providers, municipality, energy market operator, etc. who all rely on the availability of information for energy monitoring and management capabilities (Espe et al. 2018). Hence, the challenge is to support prosumers and stakeholders in obtaining appropriate information in order to be more innovative, productive, and be able to make decisions which impact their social, environmental and economic state (Jnr et al. 2018).

Similarly, energy systems in smart city generate large amounts of data during their execution that contain important information that can be utilized to optimize or improve energy production and consumption (Wi et al. 2013). Current methods do not adequately address the effective collection, processing, and storage of this data (Espe et al. 2018). Such a challenge entails contemporary Information Communication Technology (ICT) solutions capable of storing and processing significant amount of energy related data to produce intelligent contextual information (Kotilainen et al. 2017). Although, due to the huge amount of data generated by sensor and metering devices in residential buildings, Electric Vehicles (EVs), and other physical devices linked with increased rates of data transmission, constitute big data issue (Dijk et al. 2013). Hence, there is need to process open and private, historical, online, and real time data from sensor and metering devices that can be used by applications to improve prosumption services. Accordingly, Enterprise Architecture (EA) concept is applied in this study, where EA describes the central structure of an enterprise and facilitates transformation by offering an inclusive view on as-is as well as to-be structures and processes (Pittl and Bork 2017). EA aims to manage and foster Information Technology (IT)/business alignment.

Moreover, over the years EA has become a well-known domain for business and IT system management serving the purpose of modelling complexities of the real world practically and ideally aiding stakeholders in enterprise to effectively plan, design, document, and communicate IT and business-oriented issues (Chen et al. 2008). Thus, EA is an ideal approach for smart cities as it provides decision support for stakeholders (O’Brien 2018) and is adopted in this study to model energy prosumption services seeing cities are large enterprise. This is because smart cities utilize ICT services to improve the quality of life of citizens and stakeholders similar to enterprise which provides services to stakeholders. Although, it is complex to manage these ICT services in smart cities. Thus, in order to address these complexities in cities EA frameworks are typically employed (Pourzolfaghar et al. 2019). Additionally, EA has been adopted in prior smart cities in the literature (Brand et al. 2015; Tanaka et al. 2018; Berkel et al. 2018; Pourzolfaghar et al. 2019) to resolve the concerns of stakeholders in managing the contextual definitions and requirements for smart city services and systems. According to Pourzolfaghar et al. (2019) EA adopted in smart city supports in deploying services in smart cities in achieving citizens’ concerns and needs.

Therefore, this study presents an architecture based on previous study (Petersen et al. 2019) to improve energy prosumption in smart cities by applying enterprise architecture approach grounded on The Open Group Architecture Framework (TOGAF) embedded with big data to process energy data for real-time decision-making towards improving prosumption services. The architecture facilitates the integration of new data sources, technologies to provide energy related services to support different types of analytics operations needed to effectively utilize available energy data. Besides, the architecture offers a digital ecosystem that enables data sharing which is synergically applied to create innovative electricity solutions for citizen engagement, energy monitoring, and evaluation at district and city level. The architecture provides the processed data to prosumers and stakeholders via third-party applications that employs Application Programming Interface (APIs) that use data-driven services to improve prosumption operations in smart cities.

This paper is part of the European Union (EU) H2020 funded +CityxChange project (https://cityxchange.eu/) which objective is to develop and implement Positive Energy Blocks and Districts (PEB/PED) in smart cities and communities as part of emission decrease to attain the Paris Climate Goals by 2050 and measure these out in accordance with the European Clean Energy Transition in municipalities (Ahlers et al. 2019). Respectively, PEB is defined by the EU as numerous buildings that dynamically manage their electricity consumption and the power flow between them and the entire energy system to annual accomplish positive electricity balance. The +CityxChange project employs an integrative method with focus on open innovation, city integration, and replicability (Petersen et al. 2019). The method entails the integrated planning and design of cities, creation and enabling of a common energy market, and communityxchange with all stakeholders of the city to create, connected and engage communities.

In regards to the EU H2020 funded +CityxChange project, this paper presents the initial validation of the applicability of the developed architecture aimed at visualizing how a positive energy blocks and districts in smart cities and communities can be achieved. The rest of this paper is structured as follows: Section 2 presents the literature review. Section 3 is the architecture and Section 4 is the research methodology. Section 5 is the results and Section 6 is discussion and implication. Lastly, Section 7 is the conclusion.

Literature review

This section presents an overview of enterprise architecture, background of the open group architecture framework, energy prosumption services in community districts, synergic symbiosis of smart grid, prosumers and energy market, and related works.

Overview of enterprise architecture

Enterprise architecture offers systematic support for IT and business structures by providing an aggregate and broad structure of an entire establishment based on stakeholders’ concerns (Aier et al. 2009). EA aims to document enterprise current structures relating to artifacts from IT and business and their inter-relationships, analysis of relationships and dependencies, planning and comparing future scenarios (O’Brien 2018). EA promotes the idea that an organization, as a complex system can be developed or improved in an orderly method to achieve improved results. EA can be utilized by analysts, designers, and managers to predict, plan, lead, develop, control, and manage organizations current process (Pittl and Bork 2017). It defines an organization in terms of its strategy, physical instantiations, structure, value streams, and information flows, as well as its business and transaction models (Bernus 2003).

EA conceptualizations or descriptions may be layered to depict specific types of relationships for instance, those between business services, applications, security, internal IT services, data storage, networking, etc. These descriptions are important when attempting to address issues in large complex settings (Aier et al. 2009). The blueprints created by EA provides a basis for preparing, optimizing, and modelling the performance of organization. Moreover, EA can be integrated with information from external sources like detailed data descriptions and network maps to improve analysis. EA also defines the physical technology, network infrastructure, and utility connections (Saat et al. 2009). Many establishments adopt EA as part of their IT management and planning operations in facilitating important role in strategic alignment, planning, and prioritization to improve social, environmental, and economic impacts (Rouhani et al. 2013).

In smart city context EA can provides information on the fundamental as-is energy prosumption model thus serving as an informational base for informed decisions for prosumers and stakeholders (Chen et al. 2008). EA deploy a high abstraction level solution that depicts the foundation for design. It is a kind of ‘skeleton’ that focuses on important features and characteristics of smart city. EA management can provide organized methodology that drives energy business operations by providing constructional principles based on stakeholders concerns for designing smart cities (Lagerström et al. 2009). Besides, EA can represent all the component involved in energy prosumption operation using graphics and schematics to highlight all the parts of objects and how they are interrelated to promote energy sustainability. EA also entails the view of the organizing logic for business procedures, information/data flow, application deployed, IT infrastructures and IT organizational regulations, policies, and technical requirements (Riege and Aier 2008). Generally, EA supports reasoning about the behavior, structure, and properties of system. It provides a blueprint from which energy prosumption operation can be developed supporting management of complexity and risks.

Background of the open group architecture framework (TOGAF)

The Open Group Architecture Framework (TOGAF) was first proposed in 1995 based on a framework for information management designed by the United States Department of Defense (The Open Group 2011). TOGAF has progressively expanded its scope from mainly management of IT towards a wider business-oriented goal. TOGAF describes quality of services and provides integrated information, management and IS standards for architecture development (Rouhani et al. 2013). TOGAF Architecture Development Method (ADM) offers an established and iterative approach for developing EA by deploying an architectural model, transitioning, governing, and designing architecture contents (Cameron and McMillan 2013). TOGAF activities entails an iterative cycle of continuous architecture description and understanding that supports enterprises to transform and managed their business targets. Although, each ADM processes are properly defined, it does not fully address the flexibility of implementation which is left for EA architects to deploy the needed activities (Rouhani et al. 2013). Likewise, TOGAF methodology does not considers some features such as the breadth of coverage, level of details, and degree of time horizon (Rouhani et al. 2013). TOGAF ADM cycle comprises of the following phases:

• Architecture vision which consists of the description of current and desired architecture of IT and business views.

• Business architecture which depicts the current state of business and examines gaps between the present and desired state.

• Information System (IS) architecture which specifies the data and IS infrastructure requirements.

• Technology architecture which consists of the infrastructure needed for implementation.

• Opportunities and solutions which comprises of assessment and selection of implementing options.

• Migration planning which target on prioritizing implementing projects in accordance with associated dependencies.

• Implementation governance which relates on governing EA project mainly on deploying and implementation.

• Architecture change management which involves future changes by employing repeated monitoring process in IT and business to create new deployments.

In contrast to other EA frameworks such as Zachman framework, Department of Defense Enterprise Architecture (DODAF), Gartner, Federal Enterprise Architecture (FEA), and The Smart Grid Architecture Model (SGAM) framework, TOGAF mainly aims to represent the organized goals and perspective of a single organization. Moreover, findings from Cameron and McMillan (2013) revealed that organizations employ TOGAF because it is flexible enough to adapt to their IT and business strategy and support EA development. The researchers mentioned that enterprises adopt TOGAF because it is not constrictive and rigid but can be customized and improved using external components.

Correspondingly, energy prosumption services in smart city is still emerging, since the transformation of traditional energy operations requires several business, services, technologies, applications and processing of voluminous data. Therefore, there is need for an architecture centered on data acquisition from sources and applications for supporting decision management of prosumption operations. Hence, this study opted to adopt TOGAF as it allows for extension from enterprise to smart city domain to improve energy prosumption services analogous to Brand et al. (2015); Tanaka et al. (2018) who adopted TOGAF for electric mobility and smart city governance respectively.

Energy prosumption services in community districts

A prosumer refers to an energy user who not only produces renewable energy but also sells or shares the surplus with energy buyers such as the utility grid and other energy consumers (Bellekom et al. 2016). The prosumers generate energy from renewable sources by deploying small-scale renewable infrastructure such as Photo-Voltaic (PV) solar arrays, micro-hydro power systems, and wind turbines (Kotilainen et al. 2017). Thus, a prosumer is an actor who utilizes power and meanwhile generates power at a certain time (Espe et al. 2018).

Figure 1 depicts the role of a prosumers, where the goal of prosumers in community districts differs from conventional energy consumers. This is because the traditional consumers only utilize energy from the grid (Brundu et al. 2016), whereas the prosumer generates, use and actively transfer or store excess electricity using energy storage system for future usage, or trade the surplus energy (Parag and Sovacool 2016). Prosumers play a significant role in the energy value chain by contributing towards energy flexibility, innovation, and value creation. Prosumer groups help enable an effective and sustainable energy sharing service (Espe et al. 2018). Thus, the prosumer role in the smart grid could become the most developed due to social, economic and environmental value derived (Methenitis et al. 2018).

Fig. 1

The role of a prosumers in community district

Accordingly, due to the relevance of prosumption in smart cities, there is a need to research on how to manage and improve prosumption services in smart cities (Parag and Sovacool 2016). Also, heterogenous data generated from metering devices and energy sensor can be used to provide value added information that can be utilized for local energy trading forecasting, statistics monitoring or energy usage, and sales management (Bohnsack et al. 2014). Furthermore, prosumption service in community district gives rise to the concept of energy marketplace where prosumers, energy consumers and various services that are provide are traded and distributed resulting to a renewable electricity system (Wentland 2016).

Synergic symbiosis of smart grid, prosumers and energy market

Energy systems universally are in a progression of profound revolution due to need to intensely decrease carbon emissions, enhance energy effectiveness and shift to renewable energy sources (Bellekom et al. 2016). Electricity production must be more flexible with alternating generation and should consider the optimized management of production and consumption of power (Bohnsack et al. 2014). Respectively, this has resulted to the deployment of smart grid technology which introduces intelligence, enables flexibility to the power grid, and allows bi-directional data and energy flows between suppliers and consumers as well as data on real time pricing of energy (Parag and Sovacool 2016). In the energy system prosumers are connected directly to the smart grid to facilitate direct energy sharing between individual prosumers and the utility grid, and energy-distribution decisions are made based on individual prosumer (Grijalva and Tariq 2011). In this scenario prosumer may assume different level of operations that can vary from generating energy for domestic use to sharing and selling excess energy through the grid to become an active member in the energy industry (Espe et al. 2018).

Figure 2 depicts an overview of smart grid, prosumers, and aggregator. The smart grid enables consumers to optimize their electricity usage and align it with their needs and, when appropriate, with their energy generation and storage preferences, while making profit (Parag and Sovacool 2016). The energy market comprises of grid-connected prosumers who are managed through an aggregator. The aggregator is responsible for capturing and analyzing energy data flow and making decisions accordingly to manage the community groups of prosumers who mutually offload and use their generated energy or trade off their surplus electricity to other energy buyers (for example individual consumers, energy retailers, or utility grid) (Bellekom et al. 2016). Yet, the concept of prosumers and energy market is still in its infancy, and thus, the existing literature has very little to offer, either by way of exploring related concepts or determining associated issues related to big data. Similarly, there has been very little research in developing architectures to model dynamic prosumer base and existing approaches are also not practical enough in resolving some important issues like supporting energy sustainability (Wentland 2016).

Fig. 2

Overview of smart grid, prosumers, and aggregator

Figure 3 shows the interaction of smart grid, prosumers and energy market. Electricity is produced by prosumers using PV solar systems in domestic household, and excess energy can be stored, sold, or shared with neighbors in the same neighborhood, depending on energy demand and supply. The produced energy units will be used to meet internal demand, while the surplus energy generated is sold to electricity market at the market clearing price. Although, energy produced from renewable sources such as solar as is the case of this study is based on a non-continuous nature and is influenced by weather conditions. These aspects include issues related to planning and dispatching of information derived from data sources to manage electricity market systems for market users such as prosumers, municipality, energy services providers, and energy retailers. Hence, this study adds to the body of knowledge and explores on how enterprise architecture can be adopted to process and analyze heterogenous data from metering devices and energy sensors in buildings and EVs to provide information for decision making to improve energy market and energy prosumption service relating to weather forecasting, trading monitoring, etc.

Fig. 3

Smart grid, prosumers and energy market

Related works (prior architecture frameworks)

Recently, different approaches have been proposed to address energy prosumption. In the following, the most relevant studies in this field are reviewed. Among these studies Pasetti et al. (2018) proposed a virtual power plant architecture to manage demand-side of smart prosumers. The architecture comprises of plant, local, supervisory, and aggregation levels which is grounded on a service-based approach. The researchers employed a multiple agent’s method to implement the enhanced management of prosumer’s assets to manage energy demand response requests. Likewise, Hwang et al. (2017) implemented an electricity prosumer business architecture using blockchain to improve safety and transparency. The architecture comprises of big data, Internet of Things (IoT), blockchain system, and energy prosumer. The authors focused to design a business model to enhance energy proficiency by analyzing consumer energy usage pattern.

In addition, Hertig and Teufel (2016) investigated prosumer involvement in smart grids in relation to prosumer and designed a prosumer behavior framework which comprises of factors that influences preference and decisions, prosumer, prosumer community, assets, and sociotechnical configuration. The authors suggested a comprehensive view of electricity prosumer behavior network to highlight the changing roles from a consumer to a prosumer in a decentralized energy market. Moreover, Kotilainen et al. (2016) designed a prosumer based digital energy ecosystem framework. The framework comprises of technology, energy, business, data acquisition, data analytics, and value-added services which is aligned with smart grid technologies to provide decentralized electricity production using renewable energy sources.

Vergados et al. (2016) researched on how to cluster prosumers into virtual microgrids to decrease cost for renewable energy trading markets. The researchers proposed a framework that comprises of Decision Support System (DSS) acquisition component, database, energy negotiation component, cloud engine, DSS algorithms, and Restful Application programming Interface (APIs). Moreover, they explored how to orchestrate energy prosumers located in Greece into virtual clusters, in order to reduce total energy cost and address market forecasting inaccuracies. Patti et al. (2015) developed a multi-service smart metering architecture for energy prosumers. The architecture layer comprises of integration, machine-to-machine (M2M), storage application, user interface platform, security, and cloud-based unit aimed at data collection and processing. Similarly, Rathnayaka et al. (2012) designed a framework for smart grid energy sharing in managing prosumers in smart grid. The framework comprises of smart infrastructure, bidirectional communication, intelligent data processing and control, protection, environment interaction, and prosumer management.

Correspondingly, CEN-CENELEC-ETSI Smart Grid Coordination Group (2012) designed the Smart Grid Architecture Model (SGAM) framework which has been popularly adopted in smart grid domain. SGAM framework was developed in 2012 based on the request of the European Commission (EU) in 2012 mainly as a standardized reference architecture model for smart grid systems to address complexity and improve interoperability of physical infrastructures. The framework can be deployed to plan, design, envision and validate smart grid use cases, systems, and architectures in addressing standardization and interoperability issues in a structured way. The SGAM consists of three dimensions (interoperability layers, zones and domains). The interoperability layers comprise of five layers (component, communication, information, function, and business). Also, the zones entail (process, operation, field, enterprise, station, and market) which reflect the hierarchy within energy systems management. The domains encompass of (customer premises, distributed energy resources, generation, transmission, and distribution) which represents all the phases in the energy conversion chain. The SGAM is mostly relevant as it supports the energy users, systems, and domain experts to better attain their tasks such as designing, modelling, visualizing, and analyzing interoperable and complex smart grid systems.

Also, Grijalva and Tariq (2011) deployed a prosumer-based smart grid architecture to facilitate sustainable energy industry. The authors provided a scalable and flexible prosumption business paradigm for different industry. The architecture layers comprise of device, local control, system control, and market to replace the conventional one-way, generation, transmission, distribution approach. Karnouskos et al. (2011) explored prosumer collaboration for effective energy management in smart grid districts and comprises of enterprise application, neighborhood energy management, and brokerage agent frontend. Their study aimed to provide information and communication-oriented technology for prosumers to buy and sell energy on online marketplaces. Besides, Lampropoulos et al. (2010) developed an approach for modeling the behavior of energy prosumers within the smart grid. The researchers link the factors (power system physical, prosumer aggregation, market, exogenous, policy and regulation, and prosumer) and related associations that influence load profile and provided a framework for presenting electricity prosumers behavior.

On the other hand, although the reviewed study explored big data management and processing paradigms to improve prosumption service performance, they lack proper handling and management of online, historical and streaming real-time data generated from residential buildings in community districts. Thus, there is need for a developed architecture that process heterogenous data from different nodes and sources. To address these issues, an architecture is presented in this study to manage processing, analyses and sharing of heterogeneous information related to energy prosumption in residential buildings and apartments in community districts. In addition, this study provides a digital data space of information that can be exploited to characterize and create awareness of the energy production and consumption profile of individual buildings in order to deploy more efficient energy policies.

Designed architecture

In order to manage energy sustainability in smart community districts, a key challenge remains to orchestrate energy prosumption services. Therefore, this study presents an architecture based on previous study (Petersen et al. 2019), towards big data to improve energy prosumption in community districts by applying enterprise architecture approach grounded on TOGAF for processing and analysis of historical, online, and real-time data from energy sensors and metering devices, etc. in monitoring and management of district energy market in smart cities. The architecture comprises of four layers (context, business, application and data processing, and technology) from TOGAF and another three layers (services, data space, and data sources) included to extend TOGAF as depict in Fig. 4.

Fig. 4

Designed architecture adapted from (Petersen et al. 2019)

Figure 4 presents the architecture which comprises of context, service, business, application, data space, technology, and data sources layers. Thus, each of the layers are discussed below;

Context layer

The focus of the context layer is to define appropriate scope, goals, etc. for the services in relation to citizens and stakeholders concerns and requirements. The context layer entails components to specify stakeholders’ concerns, quality factors, and associated drivers/enablers towards improving the quality of life (Pourzolfaghar et al. 2019). This layer aims to capture main information about drivers, priorities, and other important aspects in delivering effective prosumption services to citizens and stakeholders Thus, this layer presents the main objectives and goals to be attained (Abu-Matar and Davies 2017), which in this study encompasses managing prosumption services in achieving a positive community energy neighborhood.

Service layer

In this study, service refers to a unit of functionality that a system, organization makes accessible and which has some value for citizens and stakeholders in community districts (Jonkers et al. 2017). The services are activities required to achieve business processes, which are independent and encompasses features which add value to prosumers and stakeholder that use them (Barbosa et al. 2016). Service provided in smart cities supports current trends and can be provided by enterprises to their clients via applications (Tcholtchev et al. 2014; Junior 2019). Besides, service layer involves operations that supports businesses to deploy applications such as energy consumption measurement, optimization suggestions, weather forecasting, etc. (Moreno et al. 2016).

Business layer

This layer involves corporate actors and roles in virtual enterprises that collaborates to provide prosumption services to citizens realized by commercial processes (Jonkers et al. 2017). This layer depicts the interconnections of elements that exist in the enterprise, and specifies the logistic processes performed by executive roles according to organizational policies (Hirvonen and Pulkkinen 2004). In addition, the business layer stipulates the primary activities and interactions of enterprises connected with each other and provides the processes enterprises employs to meet their goals which are connected to the front-end services of the enterprise (Caetano et al. 2017).

Application and data processing

The application entails the logical structures of systems and their interconnections (Hirvonen and Pulkkinen 2004). This layer describes how business operations are supported by systems, along with the dependencies, behavior, and structure of the components (Caetano et al. 2017). Moreover, application layer is the heart of the architecture as it encapsulates the data and behavior, exposes services, and makes them accessible through interfaces. Hence, this layer offers access for prosumers via Extensible Markup Language (XML) or JavaScript Object Notation (JSON)-based RESTful API to manage policies aiming at optimizing energy production and consumption. It provides functional capabilities access to data visualizations, analytics, and reporting linked to underlying databases by using queries in retrieving JSON/XML results required to support prosumption services (Brundu et al. 2016). Additionally, this layer provides an application centric view of prosumption operations that helps to provide energy related trading services to prosumers thus enabling sustainable energy (Vilajosana et al. 2013). According to Mokhtari et al. (2019) application layer is responsible for APIs and standard to exchange data between smart service and systems to prosumers.

Data space layer

The data space layer has been proposed by prior study (Otto et al. 2017), where the authors recommended the importance of data space in smart city architecture. Moreover, this layer specifies, data flows, data elements, and the data interrelations required to facilitate application layer (Kirpes et al. 2019). As mentioned by Brundu et al. (2016) data layer provides access and exposes APIs for querying and discovering data sources that supports seamless data access by applications to optimize energy production and consumption services. Besides, data layer provides huge processed data storage of historical, online and real-time data storage to support prosumption operations (Silva et al. 2017). It also includes meta-data about energy sensors, metering devices, actuators and relation to appliances, buildings and grid’s substation (Patti et al. 2015). Moreover, this layer catalogues and stores processed and analyzed data with set of policies regulating their usage (Bibri and Krogstie 2017), such as public open data extracted from the web as well as data provided by municipal or energy service provider that can be used to produce valuable insights (Vögler et al. 2017).

Technology layer

The technology layer focus to support data space layer by providing infrastructure needed to run applications (Pourzolfaghar et al. 2019). It comprises of software (such as operating systems) (Hirvonen and Pulkkinen 2004), hardware (e.g. battery), data communications, and technological solutions employed for temporal storage, processing and handling real-time data generated from metering devices, sensors, appliances, EVs, residential buildings, etc. (Jonkers et al. 2017). In addition, the technology layer explicates how applications are supported by legacy systems and new infrastructures in terms of processing, communication, and storage (Caetano et al. 2017). This layer uses subscribe and publish protocols such as Message Queuing Telemetry Transport (MQTT), Advanced Message Queuing Protocol (AMQP), etc. to support prosumers and stakeholders to consume processed large sets of heterogenous data. Thus, efficiently managing potential noise in diverse plethora of energy related data produced (Khan et al. 2014).

Data sources layer

This layer interacts directly to the technology layer and comprises of structural elements such as physical devices (energy sensors, metering devices, smart appliances, EVs, etc.), buildings, equipment, facility, and material that generates data in volume, velocity, varieties, and veracity (Khan et al. 2014; Bibri and Krogstie 2019). The data generated from the physical devices includes their status (ON or OFF) and associated parameters that need to be sensed and delivered to the technology layers for further processing (Mokhtari et al. 2019). Besides, energy sensors and metering devices deployed collects real-time measurement regarding energy production and consumption within the residential buildings to perform demand-supply electricity balance analysis for prosumers and stakeholders (Simmhan et al. 2018). Also, this layer enables interoperability across the heterogeneous sources, both hardware and software (Patti et al. 2015). It deploys wireless communication technologies, such as Bluetooth, 2G/3G/4G connectivity, IEEE 802.15.4, ZigBee, LoRaWAN, etc. (Simmhan et al. 2018).

Research methodology

Our empirical research is based on qualitative approach. Over the years, qualitative research approach has become progressively prominent among academicians as an adopted paradigm used to confirm the validity of research projects. Yin (2013) maintained that research for social domain should be concerned with elaborating and developing theory from practice. Qualitative research method relates to the development of new knowledge or dimension of research that is usually informed and guided by some theoretical model suitable to the area of investigation (Junior et al. 2018). Thus, qualitative method confirms emergent theory that is developed from a synthesis of data that emerges from real world use in practice (Jnr et al. 2019). Although, qualitative method is challenging due to lack of control and uncertainty that creates anxiety for less experienced researchers (Yin 2013). Conversely, qualitative method allows researchers to use existing theory with their own theory to validate real world practice. Creating an opportunity to revalidate the theory and to further improve it.

Similarly, qualitative method is not suitable for detailed and rigorous theory testing but in general can help solve current practical problems such as managing energy prosumption services, while at the same time expanding scientific knowledge in smart communities. Furthermore, qualitative method is concerned with simply improving practice which in this study relates to energy prosumption services, and/or enriching real-world situation (energy districts) related to stakeholder’s problem (managing heterogenous data for prosumption operations). Qualitative method has been adopted by prior energy related studies (Beaume and Midler 2009; Mäkelä and Pirhonen 2011), and previous study on big data/smart city research (Weber et al. 2009; Lim et al. 2018). Thus, this study opted to employ qualitative method as it aids replicability, or demonstrability through multiple case studies analysis.

Accordingly, a multiple case study approach is adopted analogous to prior energy research (Mäkelä and Pirhonen 2011; Giordano and Fulli 2012; Weiller and Neely 2014; Wentland 2016; Haarstad and Wathne 2019) and smart city/EA studies (Viale Pereira et al. 2017; Lim et al. 2018). Then, data was collect using focus group interview as recommended by Spickermann et al. (2014); Kamargianni and Matyas (2017) from two energy service providers in Norway to verify the applicability of the architecture. As recommended by Junior et al. (2018) the case study method includes data gathering where data is collected using multiple case studies by focus group interview. Following, data feedback and analysis which aim to iterate, refine and document the findings from the data collection session using descriptive analysis. Subsequent, reporting of findings is carried out on how the applicability of the architecture is to be deployed using ArchiMate modelling tool as used in prior smart city studies (Pittl and Bork 2017; Berkel et al. 2018) to model practical cases based on the findings from the two case studies to verify the applicability of the architecture in real life energy scenario.

Background of modeling tool

In enterprise modeling several methods such as Semantic Object Model, 4EM, ArchiMate, etc. are employed to design multiple views or perspectives towards deriving a comprehensive and coherent description. However, this study requires a modeling tool that includes objects and relationships that can be utilized for big data energy services analysis. Specifically, this study opted for ArchiMate enterprise architecture modeling language because it is an object management group standard based on TOGAF and is widely used in industries (Berkel et al. 2018). ArchiMate is an independent and open modeling language for enterprise architecture that is supported by diverse tool vendors and consulting partnerships. ArchiMate is defined in a meta-model to fulfill most EA modeling tasks in providing instruments to support enterprise architects describe, analyze and visualize the relationships among domains in an explicit way (Lankhorst et al. 2010).

Moreover, Archimate has a data and services extension which aligns well with our research goal of data-oriented energy prosumption service. Also, ArchiMate provides a formal notation to represent the relationships between entities in a hierarchical approach in layers and aspects. Figure 5 depicts the modelling of the architecture in relation to findings from case study A and case study B current data oriented-energy prosumption service in ArchiMate. Therefore, Fig. 5 depicts the combined modelling of prosumption energy services based on findings from the focus group interview session, where the results are presented in layers to validate each of the seven layers in the architecture.

Fig. 5

Energy prosumption operation implementation

Case study protocol

We adopted multiple case studies approach to answer the ‘why’, ‘how’, and ‘what’ questions of energy prosumption operations within the real-life context of smart community districts (Yin 2013; Anthony Jr 2018). Thus, multiple-case study method is suitable for our research, as it provides in-depth information about energy prosumption case scenario in practice and it allows the rich presentation of evidence and a clear statement of theoretical arguments (Jnr et al. 2019). In this research, case studies not only describe energy prosumption case scenario but also validates the applicability of the architecture in explaining energy prosumption services. Hence, case study is suitable for this research as it enables us to probe for more information and clarification of our architecture grounded on TOGAF based on not just the perceptions of the interviewee but also on their experiences and opinions in real energy prosumption service in community districts. Accordingly, primary and secondary data was collected, where secondary data were mostly sourced from academic publications related to big data, energy prosumption, EA, and smart cities helped to conceptualize the architecture (see Literature review section). Primary data were obtained through a focus group interview session in the two selected case studies. A total number of five interviewees were involved in the two-focus group interview session, where data was collected for more than 2 h by the interviewers in the companies. The participants profile is shown in Table 1.

Table 1 Demographic data of interviewee

Table 1 shows that data was collected from five informants as recommended by (Yin 2013) who suggested that data should be collected from more than three participant in case study method. During the focus group session two interviewers from the +CityxChange project presented existing EA and demonstrated the basic idea behind the architecture to the selected experts and practitioners who were purposively chosen as participants based on their prior knowledge and experience in energy prosumption in smart cities. Accordingly, focus group was adopted as it allows researchers to carefully identify both the technical aspects related to energy prosumption implementation, as well as the interaction with the business aspect of the company. The interview was performed face-to-face in an informal style, where the interviewee discussed with the help of the architecture (see Fig. 4) to their current practice in relation to energy prosumption and trading in community districts towards achieving a positive energy neighborhood.

The interviewees commented and sketched a real-life practical use case scenario on energy usage and data generation from metering devices, energy sensors, EVs, buildings, etc. during the focus ground discussion. After the focus group session, the interviewers modelled the ideas presented in Microsoft word document which was later sent to one of the participants as a follow up and post confirmation to strengthen the findings. This provided an opportunity to iteratively refine and literately modify the designed use case scenario findings. Each interview was recorded manually to assist in understanding the context in which the interviews were conducted and expedite the procedure of transcription and analysis of data. The textual content was coded and systematically analyzed using descriptive analysis.

Findings

This Section presents the findings from the case studies focus group interview by stating the background of case studies, and implementation and evaluation.

Background of case studies

The selected two organizations are at forefront of sustainable energy in Norway and therefore are selected as case studies to be explored in verifying the architecture. To maintain the anonymity of the companies the names will be referred to as case study A and case study B. Case study A was founded 13 years ago in Norway and has grown to be a global establishment with more than 500 staffs and offices in 8 countries. With the goal to create solutions that both benefit the society and support sustainable development. Case study A is a leading provider of software solutions to mostly electricity companies, grid owners, and municipalities in achieving a sustainable and safe society. Case study A is currently transforming its operations into a data-oriented business models to improve innovation and expose prospects and value for data. Case study A offer solutions in close cooperation and exchange knowledge with their partners and users to improve innovation through collaboration with research organization and institutions. The organization aims to help attain an operational and strategic decisions based on access to accurate and timely data from integrated systems to help energy utilities work smarter, benefitting citizens, stakeholders, and the environment.

Respectively, case study B is paving way for a renewable urban community by promoting the production and utilization of renewable energy as part of the solution to address the United Nation (UN) climate panel goal of decreasing Carbon di Oxide (CO2) emissions in cities. Thus, case study B works to create value through environmentally friendly production of energy related services. Case study B has up to 400 employees distributed across generation of wind power and hydropower, distribution of electricity power, and development of progressive energy-related services aimed at attaining a viable, functioning, and efficient renewable society. The organization seek sustainable solutions in managing renewable resources in a climate-challenged world towards supporting positive societal development. In case study B, all energy production is renewable as a driving force for growth of green energy.

Implementation and evaluation

This sub-section presents how a positive energy blocks and districts in smart cities and communities can be achieved. Besides, the evaluation of the approach is designed in ArchiMate modeling tool.

Energy prosumption implementation

The prosumption energy service implementation is shown in Fig. 5, based on the findings from the focus group sessions.

Findings from Fig. 5 indicates that case study A and case study B collaborates to manage energy prosumption operations, where case study B communicates information of micro electricity payment transactions with the community district grid company in managing prosumers energy production and consumption. In addition, the community district grid company is connected to payment company and prosumers which are linked to case study B and case study A who provides the trade platform that sends and received energy flow from energy data management company that offers system operations for buildings in relation to assets within residentials buildings production and consumption for decision of assets usage.

Architecture evaluation

To evaluate the architecture grounded on TOGAF, data collected from the focus group interview was modelled in ArchiMate modelling tool to test the applicability of each layers of the architecture (see Fig. 4).

Figure 6 depicts the meta-model for energy prosumption operation in ArchiMate. It also shows how each of the individual layers are connected based on the flow of data and connection among the modelling objects. According, findings from each of the layers are discussed below,

Fig. 6

Meta-model for energy prosumption operation in ArchiMate

Context

Findings from the focus group session suggest that context layer comprises the main requirements, objectives, and targets to the attained for citizens, municipality, stakeholders, etc. This result is consistent with the view of Abu-Matar and Davies (2017) where the authors mentioned that context represents the main feature or capability to be provided such as the Key Performance Indicators (KPI) to be achieved in community district. The finding reveals that the requirements involve the installation of PV solar panels and storage battery. Moreover, the goal aims to attain increase in renewable energy production and increase prosumption service. Also, the target is to achieve total energy sustainability.

Service

On the service layer, finding reveals that this layer comprises of various services that enhance energy prosumption related operations that utilizes data into business layer to provide sophisticated functionalities as recommended by Karnouskos et al. (2011). The services include providing energy related information and enabling distributed positive energy blocks or districts.

Business

Regarding business layer, enterprises such as the district energy company, energy distributor, etc. involved in the energy ecosystems are expected to collaborate and co-operate in providing value to citizens and stakeholders (Jonkers et al. 2017). Therefore, our findings confirm the statement postulated by Caetano et al. (2017) that business layer comprises of a complex ecosystem that entails a wide range of organizations that work together to provide services such as public administrations, energy operators, energy providers, and so forth. Hence, the result indicates that case study A, case study B, local grid company, and prosumers collaborates to provide different roles such as system operation, energy market operation, and energy trading in community district.

Application

In the application layer, a set of tools, systems and APIs are deployed to manage and access data from the data sources layers enabling interoperability among different devices and technologies in supporting energy prosumption services (Brundu et al. 2016). Findings from this study suggest that case study A and case study B, local grid company, and prosumers uses applications such as trade platform application, system operation and dispatch operation, and via site Energy Management System (EMS) that exposes prosumption services to access data from the data space layer for energy prosumption services. As suggested by Silva et al. (2017) these applications are used by prosumers and stakeholders involved in energy trading to visualize energy production and consumption in a single building and across the district to provide real-time information from energy sensors and metering devices, etc. in the data source layer. The information is used for implementing control policies to improve energy demand and optimize energy consumption in community district for facility managers and energy suppliers in local grid to monitor and maintain energy distribution network. These applications also allow prosumers to create or add new (POST), read or retrieve (GET), update (PUT), delete (DELETE) resource information (Mokhtari et al. 2019), and provide awareness to prosumers regarding energy consumption and promote green energy-friendly behaviors (Patti et al. 2015).

Data space

Finding from the case studies confirms the importance of the data space layer which exposes energy service via APIs such as the site EMS API provided by the data management provider company (see Fig. 6) for discovering and querying data sources for providing seamless data access by applications layer based on different energy prosumption services. Moreover, findings from both case studies stated that the data layer comprises of other internal and external data sources such as transaction database, open database in Hadoop Distributed File System (HDFS), HBase, and meta data. The open database provides huge raw data storage for processed historical energy datasets as well as online and real-time data storage to support applications deployed for prosumption services (Brundu et al. 2016).

Also, in this layer the results confirm availability of meta-data which contain descriptive information and relationship regarding energy sensors, metering devices, EVs, appliances, etc. in individual residential building and the entire community districts (Patti et al. 2015). Our result is consistent with findings from Vögler et al. (2017) where the researchers stated that the data layer manages a wide range of data sources and procedures for providing citizens with data services that offers valuable insights. According to Chaturvedi and Kolbe (2018) this layer makes the architecture to be a self-sustainable via standardized APIs such as site EMS API that returns data in formats such as Extensible Markup Language (XML) and JavaScript Object Notation (JSON) that exploits available data metadata for supporting prosumption operations as recommended by Khan et al. (2014).

Technology

Findings from the case study confirm that the technology layer is important as it entails hardware and software involved in processing of ever-growing vast amount of data (generally discussed as big data). Our result indicates that this layer provides temporal data storage and handling plethora of generated diverse data. Additionally, the technology layer can process both static data such as historical energy data that isn’t regularly accessed and dynamic data that continually changes such as online and real-time data. The result suggest that this layer also comprises of payment infrastructure, local grid infrastructure, Vehicle to Grid (V2G) infrastructure, EMS, and site EMS such as Supervisory Control and Data Acquisition (SCADA).

Furthermore, this result is analogous with findings from Patti et al. (2015), where the authors mentioned that the technology layer provides mechanisms to proficiently mediate, transform, and analyze large sets of diverse data and allows for processing of both historical and streaming data, which are generated in smart cities. Accordingly, to process energy related data Hadoop, HDFS, HBase, MapReduce, Spark and Storm are incorporated in this technology layer to facilitate the historical, online, and real-time energy related data processing and storage. Hadoop is an open source component of MapReduce that was first released in 2007 by Apache that aids the distributed processing of huge data sets across clusters. It comprises of the distributed file system component called HDFS and MapReduce programming framework for processing big datasets. In order to address the limitations of Hadoop, a new cluster processing framework called Spark was developed to support the running of computations in memory using Resilient Distributed Datasets (RDDs) which provide faster computation times for iterative applications as compared to Hadoop.

MapReduce is employed in the technology layer to processes large volumes historical related energy data for Hadoop with a high fault tolerance. Thus, MapReduce on Hadoop is utilized for offline/static data processing. MapReduce assigns filtered data into diverse data sets and then linked mapped data are combined to create smaller datasets. MapReduce aims to perform offline processing on large sets of static data for enhancing prosumption services. Moreover, Apache Spark an open source cluster processing framework that uses in-memory computing technique and is deployed for real-time streaming data processing from the physical devices in the data source layer. It uses batch in-memory to perform micro-batch processing via Spark-streaming as it offers faster performance and is suitable for processing big data from real-time and online sources. Therefore, Spark is employed to analyze real-time data from physical devices in the data source layers to quickly provide meaningful information aiming to facilitate decision regarding prosumption requirements of citizens and stakeholders.

Similarly, Apache storm is a distributed powerful real time computing framework for processing streaming data in micro-batch processing. It can process enormous number of records per node per second on a cluster of modest size. It is used to process online data produced from applications such as the energy prosumption trade platform situated in the application layer. Besides, HBase is deployed which is a NoSQL repository HDFS that stores processed semi-structured energy data from historical, online, and real-time data sources in JSON files. HDFS is a native Hadoop data management system with high performance distribution, reliable, fault-tolerant, and scalable data storage to manage large volumes data files. The rationale for utilizing HBase is to improve energy data real-time lookups performance, server programming, and in-memory caching. Additionally, HBase aids usability improvements and is fault tolerant. Thus, HDFS is used for primary storage since HDFS implements distributed storage in Hadoop clusters to enable autonomous decision-making mechanism by supporting file creation, update, write once, read many, and delete operations in real-time via complete cluster. HDFS is configured on a local file systems of cluster nodes and is designed to store huge files that are appropriate for streaming data access.

Data sources

Finally, findings indicate that the data source layer is important as it comprises of physical energy related infrastructures that produce heterogenous data. This layer provides interoperability for managing deployed physical infrastructures that produce data using communication technologies such as wired and wireless networks protocols (Khan et al. 2014). Accordingly, our result suggest that the physical infrastructures comprises of EMS, metering devices, IoT devices, energy sensors, EVs, and residential buildings in community districts.

Discussion and implications

Discussion

This study presents an architecture based on big data to improve energy prosumption in smart community districts by applying enterprise architecture approach grounded on TOGAF standard. TOGAF standard was adopted in this study because its regarded as one of the widely employed EA standard due to flexibility, interoperability, best-fit model, compliance with EA industry standards, and IT-business alignment. Furthermore, findings from this study provides evidence for policy-making, municipal planning, and energy service providers in promoting renewable energy production and consumption in order to mitigate environmental issues caused by reliance on dependency of non-renewable energy sources.

Moreover, findings from this study creates an approach on harnessing energy data from physical infrastructures to improve energy prosumption. Additionally, findings from the focus group interview modelled in ArchiMate (see Fig. 6) provides an abstract level of energy services. Our findings enable city developers and energy service providers to replicate the design for rapid deployment of lightweight energy prosumption operation such as monitoring energy production and consumption, electricity trading, assessment of energy efficiency of domestic appliances.

Besides, our findings offer practical approach that can be adopted by enterprises and city operators to develop web and mobile applications that exploit integrated/published, open and private data, static, online, and real time data to provide energy related services to citizens. The architecture is embedded with big data technologies to process real-time, online, and historical data-driven services via applications for improving energy management and reducing energy consumption. Thus, providing opportunity to achieve long-term integrated approach for green electricity city district. In addition, this study provides a roadmap for the effective management of energy exchange between prosumers and optimization of local energy consumption. This enables prosumers to actively participate in energy market or energy brokers to contribute and sell flexibility.

Implications

Findings from this study has significant theoretical and practical implications for energy sustainability research.

Theoretical implications

Currently, in community districts domestic buildings are installed with metering devices and energy sensors connected in order to promote smart energy services in providing consumers, prosumers and energy service provider with easy and secure access to their energy consumption data in standardized format which supports data integration, sharing, and reuse to reduce environmental issues facing municipalities such as CO2 emissions (Rathnayaka et al. 2012). However, these metering devices and energy sensors in residential building and EVs produces data with variability, velocity, volume, and variety that needs to be stored, processed, analyzed, and utilized for making decision regarding to energy production and usage per time voltage, interval, etc. (Silva et al. 2017). Therefore, this study contributes to theory by providing guidance for future research on big data application on energy platforms to provide decentralized energy market.

Additionally, this study contributes to existing body of knowledge as it shows how available open energy data can be made accessible for prosumers based on information produced from physical infrastructures, historical data and applications that provides monitoring and control of energy trading. Likewise, this study provides recommendation on how big data tools (Hadoop, HDFS, HBase, MapReduce, Spark and Storm) are deployed via wired or wireless network to collect and process generated energy data to provide value added information to prosumers, energy service providers, stakeholders, etc. via an interactive data analysis environment to depict energy production and usage patterns with the involved tariff.

Practical implications

This project is a part of the +CityxChange project (https://cityxchange.eu/) which aims to develop and implement positive energy blocks and districts in smart cities and communities. Accordingly, this study offers an approach to achieve better energy management in neighborhood level. Thus, this study provides practical implications on how enterprises can collaborate to provide energy related services, and prosumers can share and trade surplus energy in an energy marketplace. Overall, findings from this study depicts that the presented architecture exposes discovered data as information via third-party applications that can be further exploited to improve energy related services in relation to energy prosumption.

Correspondingly, historical, real-time, and online data can be analyzed and used to depict energy charts that provides forecast of energy production/consumption based for weather and trading information to be used to support decisions in relation to energy reduction planning and investment in creating a growing number of innovative energy services for community districts and beyond (Zuccalà and Verga 2017). Also, we specify the role of prosumers in the smart grid based a bidirectional communication of energy and data flow with involved components, in relation to energy consumption and production. Moreover, an overview of the importance of local energy markets in enabling better energy management at community district level in buying and selling electricity in the local marketplace was presented. Besides, findings from this study depicts how the presented architecture based on seven layers allows for easy flow of energy related data and better collaboration among prosumers and stakeholders in order to promote more sustainable energy operations in community districts.

Conclusion

Evidently research and development in smart community districts is gaining importance in the global context. This is obvious due to increase of urban population which is expected to increase energy consumption in the next decades (Bellekom et al. 2016). Thus, prosumer involvement is important within energy market. However, prior studies on smart grid related to energy sustainability has paid little attention on prosumer involvement and management. Accordingly, this study explored the emerging role of prosumers who produce and consume energy and their contributions towards renewable energy sustainability. In addition, we identified big data as a promising field for exploitation in promoting energy prosumption, due to heterogeneous data generated by energy sensors, metering devices, EVs, appliance, etc. in residential building. In a bid to address the issues, this study presents an architecture developed in prior study (Petersen et al. 2019) grounded on TOGAF standard to manage data-oriented energy prosumption services in smart community districts. Additionally, the architecture provides a data-driven energy services that aggregate, enrich, and provide access to shared data or data source which is not adequately provided by prior architectures.

Furthermore, qualitative research was adopted, and data was collected by employing case study using focus-group interview from two energy companies in Norway to validate the applicability of the architecture. In addition, ArchiMate modelling tool was utilized to model findings from the case studies to evaluate each layer in the architecture in relation to energy prosumption operations. Findings from this study provides practical energy management solutions for energy prosumer, stakeholders, energy service providers, etc. using historical, online, and real-time data for management and monitoring operations. Moreover, findings from ArchiMate modelling indicates that the architecture supports energy data processing and exchange that helps to provide an efficient approach to deal with big data issues in future energy centric smart cities. In relation to prior prosumption approaches that uses limited standalone data processing and storage systems, the presented architecture employs an unlimited storage and processing system which comprises of Hadoop, HDFS, HBase, MapReduce, Spark and Storm that provides a platform that supports the replicability of the architecture in other countries cities to improve renewable energy sustainability.

While our study is qualitative in nature there is need to carry out further quantitative approach that statistically evaluates the architecture using experimental method of investigation. Moreover, findings from this study is from energy companies in Norway as such the findings cannot be generalized to other regions. Thus, longitudinal study is required in energy related areas in other Nordic and European countries (one lighthouse and other follower cities) involved in the +CityxChange project to further apply the architecture in their current energy context to evaluate the feasibility of our approach in energy companies based in those regions.