Abstracts from the Energy Informatics.Academy Asia 2021 conference and PhD workshop

s from the Energy Informatics.Academy Asia 2021 conference and PhD workshop Beijing, China. 29-30 May 2021 Published: 24 September 2021 Welcome message from the organizers at the Energy Informatics.Academy (EI.A) Asia 2021 Zheng Ma, Birte Holst Jørgensen, Guangchao Chen, Bo Nørregaard Jørgensen SDU Health Informatics and Technology, the Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark; Department of Wind Energy, Society, Market and Policy, Technical University of Denmark, Roskilde Denmark; College of Materials Sciences and Opto-Electronic Technology, University of Chinese academy of sciences, Bejing, China; SDU Center for Energy Informatics, the Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark Correspondence: Zheng Ma (zma@mmmi.sdu.dk) Energy Informatics 2021, 4(Suppl 1):Welcome Dear readers, This supplement of the proceedings of Energy Informatics.Academy (EI.A) Asia 2021 includes 3 poster abstracts from the EI.A Asia 2021 conference submission and 10 from the co-located EI.A SDC Asia 2021 Ph.D. workshop. The EI.A-SDC Asia 2021 Ph.D. workshop [1] is co-organized with Energy Informatics.Academy (EI.A) and the Sino-Danish Center (SDC) Sustainable Energy Systems, aims to support Ph.D. candidates in their research, offers them from the field a unique possibility to present and discuss, receive feedback, and exchange comments with peers and experienced researchers. The 13 poster abstracts cover three important aspects of the energy informatics domain (shown in Table 1): Energy systems Energy in buildings and industry Energy markets and business Table 1. Themes of the 13 poster abstracts from Energy Informatics. Academy (EI.A) Asia 2021 Theme Paper title Type Energy system Analysis and Application of Model Predictive Control in Energy Systems Ph.D. workshop Data-driven proactive and predictive maintenance of power distribution systems Ph.D. workshop Flexible time aggregation for energy systems Ph.D. Table 1. Themes of the 13 poster abstracts from Energy Informatics. Academy (EI.A) Asia 2021 (Continued) Theme Paper title Type

Summary: Model predictive control (MPC) allows efficient use of energy systems and can provide considerable energy savings. However, finding a proper configuration of MPC in specific energy systems remains challenging. This doctoral project aims to develop methods of deriving the best combination of models and optimization schemes for a given energy system. The project goal will be achieved by testing different models and optimization techniques of MPC in a virtual testbed. An analysis of how to choose models and corresponding optimization techniques will be conducted based on test results. Three case studies will be carried out to evaluate the proposed methodology. It is expected to advance knowledge of setting up appropriate MPC configuration for energy systems and speed up MPC transition from academic research to wider industry implementation. Keywords: Model predictive control, Energy systems, Simulation, Optimization

Problem statement
Energy shortage and climate change are global challenges as conventional energy sources are unsustainable and will lead to a large amount of greenhouse gas emissions as well as pollution. Energy consumption can be broken down in various sectors. Buildings, for instance, consume nearly 20%-40% of overall global energy and are responsible for about 33% of the world's CO 2 emissions annually [1]. In 2018, the industry sector accounted for 24% of global CO 2 emission and 37% of global energy use [2]. Saving energy in these sectors is therefore of extreme significance to achieve a sustainable development and environment-friendly future.
Targeting the goals of the Paris Agreement, smart and efficient energy systems provide a feasible solution in addition to increasing renewable energy penetration. Advanced control is a promising technique to be implemented in smart energy systems aiming to reduce energy consumption without sacrificing overall performance. The model predictive control approach has been gaining popularity among these advanced control strategies due to its potentially superior performance [3]. The concept of MPC is illustrated in Figure 1. Firstly, MPC requires an accurate controller model to precisely forecast a system's future states for a set of inputs. Secondly, with predicted states, given constraints and cost function, MPC uses a formulated optimal control problem that optimizes control variables for a specific horizon to reach a defined trajectory. Next, the obtained optimal control sequence is applied for first the control time-step rather than the entire optimization horizon. Lastly, the system will repeat the optimization process at each time step based on the feedback of the system's real states. A simple example of a building MPC is shown in Figure 2. The characteristics of having a predictive model, rolling optimization and feedback correction make MPC a strategy of higher control quality and it is extremely good at dealing with conflict objectives (i.e. saving energy without sacrifice occupants' comfort in buildings) and exploiting buildings' thermal inertia to obtain a better overall performance as compared to conventional control. However, the configuration of MPC (here it refers to the combination of control models and optimization schemes) in specific energy systems remains challenging. The process of a typical MPC implementation can be roughly divided into five steps shown below: Selection and development of models Defining the cost or objective function Selection of the optimization technique Programming of control logic and strategies Testing and commissioning of the control program Model selection and the corresponding optimization techniques significantly affect the overall performance of an MPC. Despite this, they are usually chosen based on researchers' experience and familiarity with given approaches. There is a lack of systematic studies comparing different modeling approaches and optimization techniques with respect to different applications. These facts bring up the research question: is it possible to find the best optimization scheme for a given type of energy system model?
To answer the research question, a systematic evaluation is needed to assess the potential optimization schemes and their performance on models in different contexts. The breakdown of several subquestions is listed below, which should be investigated and answered separately: What are the typical MPC models to be applied in selected energy systems? What are the general optimization techniques to be employed? How does the previous research work done by other researchers refer to a combination of MPC models and optimization schemes?
What are the most appropriate metrics/KPIs in relation to evaluate the performance of the selected MPC model and optimization scheme? For a selected optimization strategy, how to get a reliable control-oriented model of proper complexity and accuracy such that the MPC infrastructure achieves expected energy savings?
(The difference between the top and the bottom figure is one timestep) This doctoral project aims to develop a methodology that can provide guidance to find the best combination of MPC models with optimization schemes. The methodology is expected to be applicable in different energy systems, such as buildings, district heating networks and greenhouse energy systems. The developed methods should demonstrate feasibility in application in real cases as well.

State of The Art
Previously, researchers have tested MPC in different energy systems, such as solar tank [5], HVAC systems [6], latent heat thermal energy storage system [7] and power plant [8] etc. Rather than covering MPC applications in different energy systems, this paper presents a state of the art of one typical energy system, namely buildings, as a start point of analysis and application of MPC in energy systems. For many years, the academic area has intensified research on building MPC to balance energy consumption and cost, greenhouse gas emissions and occupant thermal comfort. There are some existing review papers concerning building model, MPC implementation, factors affecting MPC performance etc, which provide brief and direct insight to the research sector of building MPC. For instance, Rockett and Hathway reviewed MPC for nondomestic buildings from concept to implementation, merits, prospectives, challenges and related work in literature [9]. Modeling techniques used in building HVAC control systems were summarized in [10] and technologies utilized to predict building energy consumption were reviewed in [11]. Afram and Janabi-Sharifi reviewed not only control methods, with an emphasis on the theory and applications of model predictive control (MPC) for HVAC systems [3], but also MPC for HVAC systems using Artificial neural network (ANN) [12]. Additionally, the significance of occupancy for building MPC was addressed and relevant papers were summarized in [13]. Building modeling is a crucial part for MPC implementation as it significantly influences control performance [14]. Meanwhile, it is a tough and time-consuming task to develop proper and suitable model for mimicking building dynamics, which has been estimated to occupy 70%-75% of the MPC implementation effort [9]. Reducing the time and expertise for developing models will therefore improve scalability of MPC. The modeling approaches fall into three main categories, namely white-grey-and black-box model. White-box models describe buildings explicitly based on detailed physical processes and equations and detailed documentation of building properties is needed, which prohibits its wide usage. Comparisons of white-, greyand black-box models for building MPC are reported in [15] [16], but whatever model type, it should be considered as a good model as long as it can accurately capture building dynamics and predict system future states. Regardless of model type used for MPC, there are some other factors that heavily impact MPC performance. For instance, the impact of weather and occupancy disturbance on MPC performance are addressed in [17] and [18], respectively. The suitable model order for capturing the building dynamics is investigated in [19] [20]. The proper objective functions to be formulated in MPC is studied in [21]. A systematical analysis of practical factors affecting MPC performances is carried out in [22], those factors span from building design to model structure, model order, data set, data quality, identification algorithm and software tool-chain. Moreover, the undergoing research project IBPSA Project 1 is developing a simulation framework consisting of test cases and a software platform for the testing of advanced control strategies (BOPTEST-Building Optimization Performance Test) [23]. Although a considerable number of studies have been done to deploy well-performing MPC for buildings, they focus on developing a specific MPC configuration for a specific building energy system, there is still a lack of leading guidance on which MPC configuration performs the best in general. Therefore, more studies addressing the research question need to be carried out, which allow to enrich literature reference as well as filling up gaps before widely implementing MPC in real buildings.

Methodology
To fill the aforementioned research gaps of this field, the project will contribute by analyzing diverse combinations of MPC models and optimization schemes in energy systems. Figure 3 depicts the research workflow of the project. The methodology for the work involves studying the effect of model and optimization type on overall MPC performance. It is planned to analyze three case studies of different energy systems: (1) a building, (2) a district heating/ cooling network and (3) a greenhouse energy system. Building MPC will be appointed as typical energy system to be investigated and data of a live lab building (OU44) in SDU can be easily accessed. Modelling and optimization process will be implemented via Modelica and Python and the research will be organized in the following steps.
State of the art on building MPC models and optimization schemes. The first step is to do an exhaustive literature review identifying all available models, optimization schemes and their combinations that can affect building MPC performance. The study will focus on models and optimization schemes that are in common use. A review of metrics/KPIs applied to evaluate building MPC performance and the simulation tools to be employed for building MPC is also expected.
Development of a virtual building model which enables us to make numerical experiments. Then testing a matrix of a number of model types and optimization schemes. Based on different metrics/KPIs (energy saving, cost saving, thermal comfort improvement etc.), a comprehensive methodology on how to choose models and corresponding optimization techniques should be formulated.
Implementation of the framework in a real building. A case study of public energy applications should be conducted based on the formulated method. Performance should be compared between the specific building applied this method with one that does not use the method for guidance.
Employed strategy of MPC in other selected energy systems will be studied based on research results and experience gained from building experiments. The same research process of investigating methodology for building MPC shall be transferable to other energy systems.

Summary
The challenge of properly configuring MPC for a specific energy system hinders wide application of MPC in the energy sector.
The presented doctoral project attempts to develop a methodology of finding the best models and optimization scheme combinations for MPC applications in different energy systems. In this work, it is aimed to fill up the gap of guidance on MPC model and optimization scheme selection, which allow faster and proper MPC implementation without sacrificing performance. This methodology will advance knowledge of setting up appropriate MPC configuration for energy systems, which will boost the transition between intensive MPC academic research activities and wide implementations in industry.   Summary: Motivated by the inefficient preventive and corrective maintenance strategies typically employed in nowadays power distribution systems and the advances in machine learning, this poster paper proposes research that aims to enable the application of predictive maintenance strategies in power distribution systems. Predictive maintenance relies on continuous monitoring of the power system to provide timely fault warnings, so remedial actions can be taken before permanent failures occur. Because fault data can be scarce for power distribution systems and the employment of high-fidelity sensors can be lacking, this poster paper proposes the use of data-driven models that can be developed with limited fault data or using data with a low sampling frequency. Keywords: Predictive maintenance, Power distribution system, machine learning, Data Motivation Today's maintenance strategies of critical infrastructures, namely power distribution systems, are typically preventive, e.g. periodically scheduled maintenance, or corrective [1], meaning that maintenance actions are taken reactively as faults occur. In some instances, power system operators still rely on phone calls from interrupted consumers to identify that a fault has occurred [1]. The cost of interruptions, especially upstream in distribution systems can be high because a larger number of downstream consumers are affected. This emphasizes the need for proactive and predictive maintenance strategies, which aim at supplying timely warnings so that maintenance actions can target components with poor reliability and be taken before failures cause interruptions in the power system, thereby avoiding costs of lost load and dissatisfied consumers.

Related literature and theories
Predictive maintenance strategies can be divided into two broad categories, namely, condition monitoring and prognostics. Both approaches rely on sensors for continuous monitoring of the power systems. Condition monitoring tracks the gradual degradation of the health of the power system and typically also entails an element of diagnosis that is achieved by classifying the health and failure states of the system. On the other hand, prognostics revolve around predicting the future health state of the power system, detecting incipient faults early, or continuously predicting the remaining useful lifetime.
The digitalization trend makes the use of data-driven machine learning models for predictive maintenance more feasible. Data-driven models concern the use of historic data to develop models that can classify states of deteriorating health or make prognoses about future faults and abnormal events. Historic data can be divided into two categories, namely 1) meter data, which is continuous measurements of, e.g. current and voltage, and 2) Fault statistics or maintenance records, that describe the characteristics of faulted components, the time, cause, and consequence of the fault in addition to what remedial maintenance actions were taken to repair or replace the faulty component. Predictive maintenance can only be realized for incipient faults and gradually deteriorating system health.
Regarding this, several studies have shown that there are multiple fault states in power systems that develop gradually and whose gradual development can be measured in current, voltage, and electromagnetic signals [2][3][4]. Russel et al.'s study showed that, in some cases, permanent power system failures are preceded by partial discharges several weeks to months in advance [1], making it possible to take remedial maintenance actions in a timely manner. Zhang et al. developed a prediction model based on a long short-term memory (LSTM) neural network and support vector machines that use the features of the current, voltage, reactive and active power signals preceding the fault to make its prediction [5]. Similarly, in a recent work by Bang and Skydt et al. an LSTM network is used to give early fault warnings 10-15 minutes in advance in medium voltage distribution networks [6], this was achieved using a signal like the ones shown in Fig. 1. Both methods rely on historic meter data leading up to power system faults for the training of the data-driven models. Generally, having historic meter data of fault events is necessitated in the training of classification models applied to predictive maintenance, that discriminate between healthy and degraded health. Power systems are, nevertheless, reliable, and resilient, for which reason historic fault data can be scarce. The scarcity is further emphasized by the employment of preventive maintenance strategies. Because of fault data scarcity, some applications produce artificial meter data, using methods known as virtual sample generation (VSG), to increase the size of the training dataset [6,7]. VSG has proven useful in the context of predictive maintenance, even using simple Gaussian-based VSG methods, that produce artificial data without considering the correlation between variables [6]. Nonetheless, traditional training methods like the ones proposed by Zhang et al. and Bang and Skydt et al. still rely on historic meter data leading up to power system faults, so for new systems and new equipment that have not yet experienced any faults, these methods become impractical to apply. An alternative data-driven approach is to learn the healthy state of a system, in this case, the prediction of the model would deviate from the measured meter data more as a fault gradually develops and changes the behavior of the system. This approach has been applied for the predictive maintenance of photovoltaic arrays [8,9]. This can also be done with auto-encoders by monitoring the trend of the reconstruction error [10]. Predictive maintenance of power systems has also been done using fault statistics. Clavijo-Blanco and Rosendo-Marcías calculate failure rates of clusters of components using fault statistics, which are used to predict how many faults will happen in some time period [11]. This information is utilized to target maintenance actions at the most at-risk grid areas and evaluate the risk of operating the system in different configurations. An approach like this can be seen as a predictive alternative to statistical asset management, which looks back in time to see what component type or grid area have experienced most failures and target grid reinforcements in that way [12]. When meter data-driven maintenance strategies for power systems are used, they rely on high-fidelity measurements of voltage and current signals to detect and classify short-lived events whose lives span from a few milliseconds to seconds [2,4]. Despite the digitalization trend, there are still many power systems that do not possess high-quality sensor equipment, let alone the communication infrastructure to support it. In these systems, meter data is sampled at a lower rate and the sensor coverage is low. When the sampling rate is very low, common signal processing methods, such as frequency decompositions, become impossible to apply according to the Nyquist-Shannon sampling theorem [13]. This poses an additional challenge as frequency decompositions are used widely, to great effect, for feature extraction in machine learning e.g. in [14], where a cycle-by-cycle decomposition is used for features in a decision tree ensemble model used for early fault detection. Objective Based on the highlighted challenges of data-driven predictive maintenance in power distribution systems, the objectives of the research proposal are: 1. Identifying the potential of classes of methods for fault prediction in power systems with limited and low-frequency meter data 2. Developing methods for fault prediction in power systems with limited and low-frequency meter data 3. Field testing of the developed methods

Methodology
Guided by the intuition that training one model to work for multiple states of operation is a harder task than a single state of operation, data analysis methods will be applied with the aim of characterizing the operational states and grouping them by similarity, e.g. with time series-based clustering. If it is feasible to combine datasets from different components or operational states, the amount of training data is effectively increased. Additionally, the research will consider the possibility of using VSG to further expand the training dataset focusing on VSG methods that consider the correlations between variables. A schematic of the proposed workflow focused on data preprocessing can be seen in Fig. The novelty lies in the last two steps in the workflow, that both aim to increase the amount of data for model development. Prior to combining datasets and employing VSG, a functional base model will be created. The research will study the effects of the two proposed additions on the base model. The author considers different choices for the base model appropriate, e.g. auto-encoder, LSTM regression model, or generative adversarial networks, where the discriminator could be used to produce a likelyhood type measure of whether a sample represents a healthy system state or not. Typically, the generator of a generative adversarial network is a component of interest, which can be used to create artificial data, either used for VSG or fake imagery [15,16]. The research proposal also considers a fault statistics-based approach to predictive maintenance, that combines maintenance records with summary statistics from meter data and external factors, like weather effects to predict failure frequencies in power distribution systems. This approach poses an alternative to maintenance in statistical asset management schemes. This approach will make use of correlational analysis to find the best predictors of fault frequency.

Expected results
Based on the proposed additions to the preprocessing workflow, introduces in the previous section, the research is expected to provide knowledge on how the method used to combine datasets affects the generalization ability of the model and whether it is feasible to create one model for multiple modes of operation and/or for multiple system components. Additionally, the effects of VSG are to be studied, so that it becomes clear whether virtually created samples are equally good for training as real samples, and whether there is a limit to how many virtual samples it is feasible to use for model development. This knowledge is crucial for systems with low amounts of data because it allows for more efficient usage of available data. Regarding the statistics-based approach to predictive maintenance, the project aims to determine the best predictors of failure frequency in electrical components, study the maintenance applications of the prediction model, and compare predictive maintenance to statistical asset management. Summary: With high shares of renewable generation and a reliance on storage, modelling large scale energy systems is computationally challenging. One factor driving the complexity of these models is the need for a high temporal resolution over a long period; a typical baseline is modelling all 8760 hours in a year. While simple methods such as down-sampling and segmentation are effective at reducing the number of time-steps in a model, there is potential for more sophisticated simplifications. In this work, we propose a flexible time aggregation framework where individual components in the systems (e.g. generators, storage units) may be modelled at a lower time resolution. We base the method on the theory of aggregation in linear programming, giving the possibility for provable bounds on the resulting objective value. These ideas have only been explored in a limited fashion in the context of energy systems modelling, and we highlight their potential for large scale energy system models and the next steps for research. Keywords: Energy systems modelling, optimisation, time aggregation.

Motivation
In the Paris Agreement we have set out to achieve a carbon neutral world by mid-century, necessitating a global shift to renewable energy sources. One element enabling this transition is mathematical modelling in order to explore and design future energy systems. With accurate models, we can determine an optimal transition path and optimal mix of technologies which respects emission limits while minimising costs. While energy system expansion is not a new problem, the emergence of wind, solar and other intermittent renewable energy sources introduces new challenges. There is a need for flexibility in order to even out local mismatches in time between generation and demand, which can be provided by transmission capacity, sectoral coupling and energy storage units, among others. The less predictable generation and the use of storage in turn make newer energy systems significantly harder to model. This has spurred a renewed interest in the fundamentals of energy systems modelling and motivates the search for more efficient modelling techniques. In this paper, we are interested in the capacity expansion problem for large-scale energy networks at the transmission level, with a time scale of months to decades. Well-known models addressing this problem include TIMES [1], ReEDS [2], OSeMOSYS [3], PyPSA [4] and Calliope [5]. Theoretically, large energy systems can be modelled quite well with (linear) mathematical programs, allowing the optimisation of both operation and design of the system. However, the systems are large and complex enough that we run into computational limits when building detailed models. Therefore, we must make simplifications and corresponding compromises to accuracy. As an example, PyPSA-Eur [6] is a model of the European energy system consisting of hundreds of nodes over one year at hourly resolution, and takes in the order of one day and tens of gigabytes of memory to run. Recent advances in computational power are driven by increasing numbers of processing cores, which linear program solvers are not able to take advantage of effectively [7]. Thus, to improve the capabilities of energy system models, we focus on improvements to the models themselves. There are many different ways to reduce the complexity of energy systems models [7], and comprehensive models employ a combination of techniques suited to problem which the model addresses [8]. Basic factors to consider are level of technological detail, spatial resolution and temporal resolution. These factors all influence the size and complexity of the linear program defined by the model. In this work, we are mainly interested in temporal resolution. Large-scale energy systems are usually modelled at an hourly resolution at best. However, several methods exist to address the number of time steps in energy systems models [9], collectively referred to as time aggregation techniques. A uniform downsampling of the model (e.g. going from hourly to 4-hour time steps) is perhaps the most basic reduction. A more sophisticated method, called segmentation, is to aggregate adjacent time steps based on how similar the input data for the model is at these time steps. For example, the state of our system might not change much between 02:00 and 05:00 at night, allowing us to aggregate these three hours into one longer step. Finally, the most drastic simplification is to model only a select subset of representative time periods, for example a few typical days per season. However, implemented naïvely, a model using representative time periods can't keep track of the state of charge of storage between periods. While time aggregation techniques have been the object of much research, up-to-date surveys have still identified significant gaps in the literature. For example, [7] mentions the need for a more abstract or flexible approach to aggregation. Together with the lack of systematic comparison between different studies, we can conclude that there is no consensus yet on the best way of using time aggregation in energy systems modelling. The use of representative periods specifically is becoming questionable in energy systems which are influenced more and more by the weather. A traditional system with conventional power plants may have a few typical demand profiles for each season, and the behaviour of this system is almost solely determined by the demand input data. However, a system spanning a large geographical region with generation relying heavily on wind and solar power may not have any global typical periods which repeat at all. This is because the operation of such a system depends on many different external variables, and the probability that all these variables (and hence the  operation of the whole system) repeat themselves simultaneously in a typical pattern is very low. A similar problem has been identified in [9] in the context of adding representative extreme periods to a model. In this work, we do away with the traditional use of representative time periods. Instead, we investigate the idea of modelling individual components at different time resolutions, or indeed applying the technique of representative time periods to individual components. Some limited attempts at this have been made previously and are used for inspiration. However, we pursue a synthesised and general perspective on how to use variable and constraint aggregation techniques at the component level to reduce the temporal complexity of energy system models.

Related literature and theories
For comprehensive reviews on the topic of time aggregation in energy system models, we refer to [9][10][11]. In the following, we merely outline the most relevant work in our context. As mentioned, modelling (long term) storage presents a particular challenge and is one of the motivating factors for this work. However, while we abandon the model-wide use of representative periods, significant work has also been done recently on reconciling the use of representative periods with seasonal storage. In particular, a number of independent publications [12][13][14][15] have pioneered the idea of "linking" representative periods in some way, keeping track of the state of charge of storage between periods. The performance of some of these new methods has been compared on large instance in [16]. However, while linking representative periods takes care of the state of charge of seasonal storage, the more fundamental problem with representative periods (there being no repeating patterns in high-dimensional input data) remain. Existing literature is sparse on the topic of modelling individual components at different time resolutions. The work in [12,13] could be seen as modelling seasonal storage at a coarser resolution, but the idea is not investigated as such. To the authors' knowledge, the topic (in the context of energy systems research) has been the most thoroughly investigated in [17]. There, inspiration is taken from literature on the scheduling of chemical plant operations. But although the paper highlights the potential of separate time resolutions, the scope considered in [17] is fairly narrow. The instance on which various componentlevel time aggregation techniques are tried out is a simple single-node system. While the results are positive but not dramatic, it is acknowledged that the impact on larger systems may be quite different. Moreover, there is much potential for different component-level aggregation strategies.

Research questions
In this work, we address two questions. This formulation should generalise existing time aggregation approaches such as down-sampling, segmentation and representative time periods.
2. How do we choose appropriate simplifications at the component level, and what is their impact on the performance and accuracy of typical energy system models?
Of course, the proposed simplifications should be compared with existing approaches to evaluate their impact.

Methodology
While we want to simplify energy system models to reduce the computational burden, we should recognise that some components may suffer more from simplification than others. On one hand, for example, the operation of a nuclear plant or seasonal energy storage may not vary much throughout a day and could be modelled at 6hour intervals. On the other hand, the operation of batteries, which often work with daily cycles, may need to be modelled at an hourly resolution in order to take peaks in demand and production into account. There may also be components, such as gas turbines providing peak generation, whose operation may only need to be modelled in detail during hours or even seasons where peak demand typically occurs. As we outline in more detail below, varying the time resolution at a component level can be achieved by simply aggregating individual operational variables of the model. It is instructive to compare this approach visually with existing time aggregation techniques. The left drawing in Figure 1 illustrates the operational variables in a model without any time aggregation. Each box represents a variable, and we have arranged the variables by which time step and component they correspond to. The approach of this paper is illustrated on the right in Figure 1, highlighted by the red box. Here, we have aggregated some of the variables, but differently for each component. Contrast this with aggregation by segmentation and representative time periods, illustrated in Figure 2. There, variables have been aggregated uniformly for all the components.
As a theoretical underpinning, we can use the aggregation and disaggregation framework proposed in [18]. Simply put, we can aggregate variables x 1 , …, x n by replacing all their occurrences in a linear program by a new variable x. This produces a new linear program, potentially with a different optimal objective value. The idea is that if are all similar in an optimal solution, then replacing them by x won't change the objective value much. Dually, we can also aggregate constraints. In the context of energy system models, each operational variable often has one or more simple constraints associated with it. For example, the output of a power plant is at any given time limited by its maximum capacity. Now, if we have constraints x i ≤ b i for i = 1, …, n and we aggregate into , then we can simply replace the constraints by the single bound x ≤ min ði¼1;…;nÞ b i .
As we can see, any set of variables and constraints can be aggregated in theory. However, the natural question is which variables (and associated constraints) we can aggregate without changing the objective value too much. There are a few different kinds of aggregations which we could consider. For one, we can aggregate sets of variables, associated with one component, which represent contiguous sections of time. This is analogous to segmentation, but for a single component. Another approach is to use aggregation to create representative time periods for single components. Moreover, while we choose to focus on temporal resolution, variable aggregation may also be applied flexibly to components. For example, sets of components may be aggregated for specific time periods only. All in all, the approach outlined in this section answers our first research question in the affirmative.

Expected results
The second research question, asking how effective the flexible aggregation technique can be, is harder to answer. We plan to try out a variety of different aggregation techniques and levels of aggregation on large instances of energy systems in order to compare their performance and accuracy to that of a full resolution model. We will use the PyPSA-Eur as a reference and a basis for experimentation. In this way, we hope to elucidate which kinds of components deal well with which kinds of simplifications. The performance and accuracy of our tests will additionally be compared to existing time aggregation methods, and indeed incomplete solves of a full resolution model. Since our aggregation method is a strict generalisation of previous time aggregation approaches, our results in terms of performance and accuracy are expected to be at least as good as for previous approaches. What remains to be investigated is whether the flexible time aggregation approach leads to significant enough improvements to justify its use. Finally, following [18], it is also possible to calculate bounds on the objective function of aggregated models without solving the full resolution model. These bounds will be investigated and compared with empirical results in order to evaluate their usefulness. Summary: The activation of flexibility potentials and using them for stabilizing the power grid is a crucial component to overcome the challenges of the energy transition. Flexibility potentials are not only provided by centralized, large-scaled traditional power plants but also by private households. The increasing availability of electricpowered vehicles, photovoltaic systems, and energy storage systems in private households make such flexibilities even more available and increases the importance of private households. The role of households, therefore, changes from pure consumers to consumers and producers of electric power. These households are called prosumers.
The activation of small prosumer flexibilities can thereby make a decisive contribution to the systematic stabilization of local power grids. FlexChain research project goals are developing an easy-touse, decentralized efficient trading platform and generating incentives for private households to provide their prosumer flexibility to stabilize their local power grid. Keywords: Smart Grid; Flexibility; Home Energy Management System; Blockchain; Smart Contract

Introduction
The increasing spread of renewable energies is leading to a fundamental structural change in the energy sector. Centralized scheduled power plants are superseded increasingly by flexible, weather-dependent, and decentralized generators. Those can be in the form of wind and solar farms, among others. [1] The increasing use of small-scale generators in private households, such as photovoltaic systems, transforms consumers into consumers and producers in one, the so-called prosumers. [2] This change of consumers to prosumers leads to a continuing decentralization of power generation. Therefore, new requirements arise for the power grids. Those requirements demand new, intelligent solutions to ensure a stable basic electrical supply. The concepts of smart grids ideally try to avoid the cost-intensive and often lengthy implementation of infrastructure measures and the use of control measures by transmission system operators. [3] Gaps between available and the required amount of energy is dynamically closed in terms of time and location by exploiting flexibilities. These solution concepts build on the fact that there is a high willingness to activate flexibilities among prosumers in principle. [4] The research project "Blockchain-induced activation of small flexibilities in the low-voltage grid (FlexChain)" is funded by the German Federal Ministry for Economic Affairs and Energy (BMWi). It has started in September 2020 and will end in August 2023 after a planned duration of three years. The funding code is 03EI6036A.

Related Work
In addition to the traditional stabilization of the grid by large, centralized generators [5], the possibility of stabilizing the grid also by demand-side resources [6] is receiving more and more attention in research. These potential resources include households, which can be either traditional consumers or prosumers. The fact that households can be motivated for stabilization on the transmission grid level and the distribution grid level has been described by [7]. An estimation of the principal flexibility potential from household appliances such as washing machines or dishwashers was made by [8]. [9] also acknowledges high potentials for the supply of flexibilities by a household for electric cars and battery storage. In general, the versatile flexibility potential of households has been already discussed and presented several times. [10] [11] [12] To be able to control and determine these flexibility potentials of a household in a targeted manner [13], and [7] refer to the advantages of the use of Home Energy Management Systems (HEMS).
Due to the small size of the flexibilities of a household, the households can't participate directly in the classical electricity markets with their flexibility potentials. [14] An option is to use an aggregator, which offers the small flexibilities of several households aggregated on the classical electricity markets. But this option is also not purposeful if the flexibilities should stabilize the local grid. Such an approach has been described by [15] and [16]. [15] focuses specifically on the aggregation of multiple electric vehicles located in a large geographic area. The combination of the flexibilities makes it possible that the aggregator can eventually offer the electric cars to the electricity market as one load/storage unit with sufficient size. In [16], the total flexibilities of several households get offered aggregated in the energy market. A HEMS represents the interface between the aggregator and the flexibilities of households in this approach. New trading mechanisms have to be designed for local trading for many households offering their small flexibility potentials by themselves. [17] In the flexQgrid project, a trading mechanism is applied, which enables such peer-to-peer trading of small flexibilities between households within a low-voltage grid area. However, in flexQgrid, the flexibility trading does not specifically stabilize the power grid. Instead, the trading aims to utilize the maximum possible energy from renewable resources. [18] One example of an approach that pursues trading of small flexibilities, which are provided by households, for targeted stabilization of the local distribution grid is the "Altdorfer Flexmarkt". [19] The "Altdorfer Flexmarkt" is based on a central trading platform where households can trade their flexibility potentials with the grid operator. The particular challenge in developing such an approach is still the volume of transaction costs. If trading is to take place economically, the transaction costs must not exceed the business value of the traded flexibilities. The small size of the traded flexibilities leads to a low business value.

Objectives
The goal of the research project FlexChain is to realize an easy-touse, decentrally organized trading platform for small flexibilities. Through FlexChain, prosumers are to be encouraged to make their potential flexibilities available for the stabilization of the grid. The goal of FlexChain is to offer grid operators an alternative to conventional grid reinforcement using infrastructure measures to resolve congestion situations in the power grid. The focus of FlexChain is on the local stabilization of power grids in the low-voltage range. Using a blockchain-based solution for trading flexibilities will ensure simple manageability, low transaction costs, and permanent documentation of the trading processes. By using smart contracts, transactions are automated, transparent, and forgery-proof. The goal is to achieve the lowest possible transaction costs. Lower transaction costs lead to smaller, economically tradable flexibilities. This benefits households in particular. The realization of the research project relies on an interdisciplinary approach. Thereby, electrotechnical, information-technical, as well as socio-economic questions will be holistically answered.

Approach
The idea of FlexChain rests on the activation of small prosumer flexibilities in a market-based process. The goal is to identify, forecast, and activate the potentials of a household via a HEMS. The data exchange with both the metering point operator and the distribution grid operator via the existing smart-meter-gateway infrastructure. The intelligent local network station and the intelligent, automatic trading platform FlexChain are the upstream nodes. -Definition of an economic and legal framework for blockchainbased marketing of small flexibilities -Design and implementation of standardized trading processes to trade small flexibilities for the stabilization of the distribution grid -Generation of forecasts for individual households using a HEMS, as well as prognoses for the local low-voltage grid -Control of prosumers energy assets by the HEMS, for conducting of flexibility after a successful trade transaction -Developing the required blockchain technology and smart contracts that are necessary for automated trade execution -Consideration of the stakeholders of a decentralized trading platform and evaluation of the potential digital market and business models -Investigation and evaluation of the economic benefits of a blockchain-based trading platform for the grid-supporting use of small flexibilities from households The project goal will be evaluated in a field trial in addition to the research and development work. These will take part in a dedicated residential area connected to the transmission grid of the project partner Stadtwerke Saarlouis GmbH. Customer acquisition takes place by active inquiries. Incentives will be developed for that purpose. Those can be based, for example, on cross-selling effects or the development of innovative business models. In a follow-up evaluation, both the technical and the economic feasibility of the approach are determined.

Project Partners
The consortium of the FlexChain research project includes the August-Wilhelm-Scheer-Institut für digitale Produkte und Prozesse gGmbH (AWSi), Hager Electro GmbH und Co. KG (Hager), Stadtwerke Saarlouis GmbH (SW SLS), VIVAVIS AG (VIVAVIS) and OLI Systems GmbH (OLI). The AWSi acts as consortium leader in the research project FlexChain. The AWSi is responsible for the conception and implementation of the trading processes required to trade small flexibilities. The resulting subtasks contain the realization of an efficient matching algorithm and the specification of the therefore necessary smart contracts. Hager is responsible for activating the flexibility potential in households. For this purpose, the company's HEMS technology will be refined to include the required functionalities. New functions result from the determination of flexibility potentials of individual plants and devices and the calculation of the total flexibility of a household. The creation of flexibility offers and the retrieval of flexibility demands are also part of the required functionalities. The SW SLS assumes the role of grid operator and energy supplier. From the network operator's point of view, SW SLS provides the network information required for the simulations and tests planned in the project. Furthermore, the SW SLS is responsible for the organization and execution of the field test. For evaluating the market platform, the SW SLS takes the market view in their role as an energy supplier. VIVAVIS is responsible for configuring the blockchain gateway, as well as for the connection between blockchain gateway, and HEMS, and blockchain gateway and smart-meter-gateway. OLI brings its extensive experience in the design of blockchain systems to the FlexChain research project. The tasks of OLI consist of rolling out the necessary blockchain infrastructure, developing the blockchain gateway, and designing and implementing the interfaces between the smart contracts and the HEMS.

Results and Impacts
In the following part, the current concept status of the FlexChain trading process gets described. The HEMS installed in the households creates forecasts for the installed energy assets regarding their expected usage. The result of this is a baseline. This baseline gets transmitted to the grid operator. For the grid operator, these baselines are essential to simulate the grid. The grid simulation identifies potential grid bottlenecks to which the grid operator must respond. Grid bottlenecks generate demand for flexibilities. The grid operator sends this demand as a request to the blockchain, which leads to the creation of smart contracts. Via push messages, the affected HEMS get informed by the blockchain of the demand that has arisen. If a HEMS sees itself capable of responding to demand with an offer, it sends this information to the blockchain. A HEMS offer gets stored as a smart contract on the blockchain. When dynamically parameterizable criteria between supply and demand match, the matching occur and subsequently a transaction. Currently, two trading modes are conceptually supported. The first trading mode bases on a fixed price, given in the smart contract of a demand. The second trading mode bases on simplified auction trading. The aim is to keep transaction costs as low as possible. The successful completion of the research project will change the possibilities of grid stabilization in the long run. Grid stability can get achieved by the efficient activation of flexibilities instead of an expensive and lengthy conventional grid expansion. In the ideal case, active control measures by the grid operator are no longer necessary. As a result of the increasing distribution of electromobility, photovoltaic systems, and others, the research project participates in the rising flexibility potential. The politically promoted change from consumers to prosumers in private households also plays a decisive role.

Conclusion
The research project FlexChain offers grid operators an intelligent alternative to conventional grid reinforcement. FlexChain enables effective and efficient stabilization of the local energy grid by activating and economically trading small flexibilities. The focus is on the flexibilities of private households. Thus, active participation in stabilizing electric grids through private households is encouraged by FlexChain. Based on the increasing shift to weather-dependent, renewable energy, FlexChain can be a crucial grid stabilization tool for grid operators.

Declarations
Ethics approval and consent to participate Not applicable.

Consent for publication
Not applicable.

Availability of data and materials
Not applicable.

Competing interests
The authors declare that they have no competing interests.

Funding
This work is part of the research project "Blockchain-induced activation of small flexibility potentials in the low-voltage grid (FlexChain)" funded by the BMWi. Summary: The increased use of intermittent renewable sources in large power system, and more harmonics infused into distribution system; both prompt high distortion level in the power system. This paper analyzes electric power quality by applying D-STATCOM, one of the Flexible AC Transmission System (FACTS) devices, into 4.16kV distribution system. The new control method, named 'fake reference approach', coordinates with reinforcement learning; especially, Deep Deterministic Policy Gradient (DDPG) algorithm. The action vector In recent years, the continuous growth of the power industry has contributed to increased demand for high-precision equipment such as power electronic devices. The result of this trend is an increase in power demand while deteriorating the power quality of the distribution system [1,2]. The matters concerning the distribution system's power quality shall be as follows: Power factor degradation, Harmonic current introduction, Interruption, Instantaneous voltage drop [Sag], Instantaneous voltage rise [Swell], and Voltage unbalance. FACTS facilities have recently been introduced to mitigate/compensate for these power quality problems [3][4][5]. In addition, D-STATCOM, which operates on a Voltage Source Converter [VSC] basis, is installed parallel to the distribution system, acting as an active filter, and is flexible in responding to the external disturbances and load variation [6,7]. A D-STATCOM is a shunt-connected bidirectional converter-based device that operates as an impedance converter to utilize either inductive or capacitive electrical elements by revising its output voltage levels. D-STATCOM can solve the problem mentioned earlier; the poor load power factor, the poor voltage regulation, and the unbalanced loads [8]. D-STATCOM, consisting of three controllers (AC Voltage Regulator, DC Voltage Regulator, and Current Regulator), uses voltage source PWM[Pulse Width Modulation] inverters that utilizes the Insulated Gate Bipolar Transistor (IGBT) to compensate for the voltage in discrete space by generating reactive power as much as the system needed [9]. Thus, the study of enhancing control performance, such as making improvements on transient response, becomes more vital; especially in the emerging power system with a higher portion of renewable energy sources integrated [10]. The random and intermittent characteristic of renewable energy gives rise to significant challenges in terms of how to control bus voltage in case of power system collapse.
In accordance with the previous paragraph, this paper introduces a new control method that has difference from the existing control method and describes how the new control method will be applied with reinforcement learning. The concept of the new technique is to apply 'fake reference' instead of 'real reference' to the voltage controller, and the responses as observed value, follows the 'fake reference' rather than the 'real reference' even though it desires the 'real reference'. In section 2, DDPG algorithm and the fake reference that used in the model will be described. In section 3, 'Case study and results,' it shows transient response improvement, compared to conventional method. Furthermore, the case study is composed of programmable voltage source for testing that the model properly works at the intermittent power system conditions. Fake reference approach using ddpg algorithm In this paper, reinforcement learning is used to focus on voltage regulation. If controller designer originally wanted a power supply voltage of 1 pu, the theory is to set the voltage command to 1. However, our approach stems from the idea that a reference should be given instead of 1 pu even though the desired real reference is 1, and the result, response characteristic, would be more accurate and fast. In this paper, we called this as 'Fake reference'. In this paper, only reinforcement learning is the solution to this concept since the 'Fake reference' approach deals with an area where human cannot reach, and not be organized by formulas.
The Deep Deterministic Policy Gradient [DDPG] algorithm is an algorithm that is learned using artificial neural networks and reinforcement learning [1,11,12]. DDPG is a model-free offpolicy actor-critic algorithm that combines 'Deep Q Network[DQN]' with 'Deterministic Policy Gradient'. However, the DQN could solve the problem using high-dimensional observation spaces, but only discrete and low-dimensional action spaces. Therefore, DQN split action space into discrete action as a way to apply it to continuous domain problems. In addition, DQN is considered an episode as a bundle, but DDPG is not, so it is suitable for control in a variety of power system conditions. A major challenge for deep learning in continuous action space is how agent explore in the environment. DDPG learns through off-policy way by adding random noise vector for action exploration, as not taking action to get the best return, but sometimes taking random action to try new attempt. DDPG algorithm basically consists of state, action, and reward vector. These components let DDPG agent learn the value function from the reward vector through the observations by exploring the system's environment. Especially, storing information such as state, action, reward, and next state in mini-batch let agent can learn by sampling all the experience what it has accumulated so far, instead of learning only from recent experience. There are more details that explain the DDPG algorithm well about how it is updated the parameter as formulated by each process [13][14][15]. This paper introduces a technique for applying the actor-critic model-free policy gradient method as an algorithm that controls discrete action domains to achieve the system's desired objective. The unpredictable disturbance is definitely an obstacle in the control system, resulting in lagging performance in controller design. In a way, the fake reference is extra-disturbance and a technique for matching real references by giving them an extra-noise. However, this is not an unpredictable one but a predicted one under variable systematic conditions, which will be learned by reinforcement learning to form the appropriate fake reference.

Case study and result
In this proposed method IEEE 13-bus real unbalanced distribution network is taken as a base case which is shown in Fig 1. Data of line impedance and system information are referenced in the IEEE 13-bus system from Electric Power Research institute (EPRI). The goal is to prove that the proposed approach develops bus voltage control performance with an improved transient response. The system profile shown in Figure 1 is used to evaluate that performance in various conditions. The slack bus voltage is set at 120.6kV, and the transformer turn ratio is given as 120kV/4.16kV. In Figure 2, the voltage source at bus 650 is variable that designated in programming, and it fluctuates between 0.99 and 1.01. Experiment has proceeded on the assumption that D-STATCOM is installed in parallel to the bus 632, and electric circuit breaker is installed between 671 bus and 692 bus. The real reference of D-STATCOM is set at 1pu which is shown as blue line in the Figure 3. In Figure 3, it shows that DDPG approach commands a variable 'fake reference' when the programmable voltage source is changed. The reinforcement learning has been implemented for 500 iterations, and the action of reinforcement learning as a disturbance, develops transient response in control system. The response characteristics with and without D-STATCOM are shown in Figure 4. Figure 4 shows that a specific value is near 1 and not be controlled when without the D-STATCOM. Besides, the transient response is enhanced when the 'fake reference' control method is applied in comparison with existing control theory. This confirms that the model works adaptably with programmable voltage source that indicates various power system conditions.

Conclusion
In a condition that the power system gets more complicated these days, the role of FACTS devices becomes highlighted for the stability of power system. In this paper, simulation is conducted by a model of 4.16kV distribution system with D-STATCOM in Simulink. The proposed model makes fake reference to control bus voltage by reinforcement learning instead of real reference what exactly desired by control designer. The simulations confirm that the system voltage is more stable when operating D-STATCOM with proposed model. In this paper, only AC Voltage Regulation is covered. However, as described in the introduction, the more controllers, DC Voltage Regulation, and Current Regulator will be covered in a further study. Moreover, more case studies considering the variable AC voltage reference of D-STATCOM and OPF problem will proceed together.   Summary: The electricity consumption of industrial buildings with many office spaces is significantly affected by the behavioral patterns and activities of the building users. Furthermore, external factors such as temperature changes, humidity and grid data are some of the common parameters that could have an effect on the energy consumption. Hence, there is a need for developing sophisticated energy consumption optimization strategies while considering all the possible factors that might affect the energy consumption to help minimizing the running cost of the buildings. The goal of this paper is to propose an intelligent framework for the optimization of the energy consumption of an industrial building while maintaining the thermal comfort inside the building Keywords: Building energy management, Optimization, Artificial intelligence, Neuroevolutionary learning Motivation Over the past years, there has been a significant increase in the energy demand. This is mainly due to many industrial developments emerged as a result of the advancements in the technology as well as the population growth. When we observe the energy consumption statistics, it is evident that the building sector accounts around 40% of the total energy consumption in the European Union (EU) including Norway and 36% of the EÚs CO 2 emissions [1,2]. However, according to the U.S. Energy Information Administration, about 57% of building energy consumptionis due to heating, ventilation, and air conditioning (HVAC) system [3]. As a result, the designing of efficient building management systems has received a significant importance considering the fact that optimizing energy consumption is beneficial for both consumers and service providers. Building energy management system (BEMS) is a computerized system designed for monitoring and controlling energy related building services. BEMS connects various systems in buildings including lighting, HVAC systems along with their auxiliary units and provides a centralized platform to monitor energy consumption of the building while creating a platform to get insight on building's energy utilization [4,5]. Building energy optimization (BEO) is a process of identifying optimal design that minimizes the total cost of energy consumption in the building while maintaining the thermal comfort for building users. Energy optimization mainly based on physical models. However, the construction of accurate physical models is computationally expensive [6]. Therefore, the focus has been shifted towards data-driven methods for BEO. Numerous studies have been conducted on BEO. Genetic algorithm (GA), particle swarm optimization (PSO), generalized pattern search methods (GPS) and reinforcementlearning (RL) are the most common optimization methods found in literature [7,8].
Controlling heating and cooling inside the building is one of the main concerns when considering building energy systems. The HVAC system helps to maintain the indoor temperature and ensure thermal comfort inside the building by maintaining the set temperature. It is a well-known fact that the HVAC system has a strong correlation with the outside temperature. Moreover, external factors such as time of the day, type of the day (weekdays or weekends), number of people inside the building need to consider to ensure thermal comfort [9,10]. Therefore, to optimize the energy consumption of the building, one of the main objective functions is to reduce the heating and cooling energy consumption. The aim of this paper is to explore strategies to evaluate the existing approach for multi-objective BEO and incorporating additional innovations while considering the two objective functions cooling and heating energy consumption. Furthermore, we also plan to incorporate the external factors such as weather changes, time of the day into the optimization process so that we can further enhance the thermal comfort inside the building. The rest of the paper is organized as follows. In Section 2, we briefly describe the related literature and theories while the background related to the proposed framework is given in Section 3. The proposed framework is provided in Section 4 before the paper is concluded in Section 5.

Related literature and theories
Optimization of energy consumption is being extensively studied in literature. In the following, we have summarized the optimization strategies considered in the existing building energy systems. Genetic algorithm (GA) is one of the optimization methods utilized in relation to BEO [11]. In this direction, the authors in [12] adopted three black-box optimization methods -direct search, metaheuristics and model-based method and evaluated their performance when deployed in BEMS. In comparison, multi-objective GA optimization is more appropriate when compared to single objective optimization for BEO due to the fact that it can optimize several objective functions concurrently. Thus, a simulation-based predictive control (MPC) model is proposed in [13] to achieve GA based multi-objective optimization. Furthermore, the authors in [13] also considered the operating cost for space conditioning and thermal comfort for the optimization process while using GA with a combination of Energy-Plus simulation software 1 [14] and MATLAB 2 . In contrast to [13], an active archive non-dominated sorting GA (aNSGA-II) while using EnergyPlus is proposed in [15] to optimize the energy demand of existing residential buildings. In [16], a multi-criteria tool is developed with thehelp of an improved multi-objective GA (NSGA-II) [17] while coupling TRNSYS software 3 . Although GA has been widely adopted as an optimization strategy, there are many drawbacks associated with GA when used for optimization purposes including the associated computational cost when dealing with a large number of functions, slow convergence and the lower stability. However, it is possible to address these drawbacksby using simplified models and selecting small GA population while having a small number of generations to potentially reduce the computational cost as evident from the research work in [18,19]. Although the aforementioned options would lead to better computational efficiency, but such approaches could run the risk of over simplification leading towards inaccurate modeling of the building.
In [20], the authors have compared seven multi-objective evolutionary optimization algorithms used to solve the design problems in nearly zero building (nZEB). It is found that none of the tested algorithms mentioned above converge completely within 1800 evaluations and concluded that this is due to the stochastic nature of the population-based algorithms that cannot assure a global minimum within a finite number of evaluations. In addition, the performance of the three optimization algorithms namely, GA, PSO and sequential search algorithms are considered to optimizing the building envelop and HVAC system design for residential buildings in terms of computational time and cost reduction in [21]. Although they found that the GA and the PSO require less computational time to obtain optimal solutions than a sequential search algorithm, the optimal results obtained using the three aforementioned algorithms were almost similar. In recent years, reinforcement learning (RL) also known as model-free approach has drawn huge attention concerning BEO [22]. Authors in [23] proposed a dynamic pricing and energy consumption scheduling for a microgrid where the service provider acts as a broker between the customer and the utility company using the Qlearning algorithm. Furthermore, they considered multi-agent learning in smart gird including both the service providerśdynamic pricing and the customerśenergy consumption scheduling for their study. Moreover, RL-based BEMS with the integration of photovoltaic (PV) panels is proposed in [24,25]. The RF-based approach is considered superior in solving the decision-making problems with uncertainty such as energy generation of PV panels. Reducing energy cost and thermal comfort inside the building are main objective functions of these studies. In addition, Artificial neural network (ANN) coupled with the Q-learning algorithm is utilized in [26] whereas in [27], a RLbased approach is used to solve the residential scheduling or load commitment problem. A Deep reinforcement learning (DRL) method is used to perform online optimization of schedules of BEMS in [28]. The learning procedure is conducted using the deep Q-learning and deep policy gradient methods andthey have identified that the deep policy gradient method is more suitable for online scheduling.
Although extensive research work has been conducted on building energy management using RL, it is evident that the integration of renewable energy sources have not been considered in most of the studies. In addition, the multi-task RL and multi-objective RL algorithms are not considered in most of the work in the literature. Our intention is to identify what techniques have been utilized for the purpose of optimizing energy consumption as well as identifying the critical objective functions that must be considered when developing an efficient BEMS. We will use this knowledge as a basis and propose a framework and strategies to be investigated in future works for better optimization of energy consumption in an industrial building which includes office spaces and laboratories.

Background
In this section, we provide an overview of optimization and machine learning techniques that have been widely utilized for the purpose of efficient management of energy consumption. Genetic Algorithm (GA): GA is a classical evolutionary algorithm based on randomness along with the natural selection and the survival of the fittest which was first introduced by Charles Darwin [18,28]. When executing a GA, it randomly selects an initial population where the population size is equal to the number of solutions. Each individual in the population is considered as a solution to the problem that needs to be solved and represented with a chromosome. A fitness function is used to evaluate how each chromosome fits as a solution. Based on the fitness values parental chromosomes are selected and with the help of crossover and mutation operations, the next generation of solutions are constructed. The algorithm terminates when the population has converged meaning that it is not possible to develop off-springs or solutions that are significantly different from the previous generation [28].
Particle swarm optimization (PSO): PSO is a population-based stochastic algorithm inspired by the social behavior of bird flocking or fish schooling in nature [29,30]. PSO randomly selects the initial solutions which are called particles. In each iteration, the fitness value of each particle and the global best value of the population (global best) is calculated. The movement of the particles is influenced by the calculated values and this process is repeated until the particles reach the target. However, given that all the particles follow their own best and the global best particle, all particles may trap in local minima in the solution space [31]. Artificial Neural Networks (ANNs): ANNs are inspired by the human biological system; especially how the human brain process information and communicate via the nervous system. The basic structure of an ANN is consisting of an input layer, an output layer, a set of hidden layers along with a set of highly interconnected elements referred to as the neurons. These neurons transform a set of inputs to a set of desired outputs while the transformation is dependent upon the characteristics of the elements as well as the weights associated with the interconnections. Given that weights influences the output, it is necessary to adjust the weights and the thresholds accordingly, which is referred to as the learning process of an ANN [32,33]. Deep neural network (DNN) is an ANN with multiple hidden layers between the input and the output layers and this allow to learn more complex patterns. Thus, DNNs can be used to model complex non-linear relationships [34]. Reinforcement Learning (RL): RL also known as the model-free control approach, is a form of machine learning technique which does not require any model to ensure optimal control of the system. RL consists with an agent that interacts with its environment and learn the type of the action depending on the state of the environment. During every step of the learning process, an agent chooses an action based on its current policy and the current state of the environment. Then, the environment returns a reward and the new state to the agent. The agent adjusts its policy with regards to the state, action and the reward. After successive iterations, the agent can determine the optimal policy with maximum reward [24,25,26].

Methodology
In this section, we provide an overview of the proposed framework which is illustrated in Fig.1. The framework consists of four main modules -input data, data handling, ML model and optimization.

Input data
This module aims to collect relevant input data and feed them into the data handling module. There are two inputs to the systembuilding data and external information.
Building data: Many systems integrated with buildings such as HVAC systems, power systems, security systems. These systems contain various sensors and meters which provide energy-related data such as temperature and electricity consumption. To ensure thermal comfort inside the building, it is necessary to adjust the temperatures accordingly. Therefore, we consider temperature and energy consumption data as inputs to our model. External information: We define external information as the outside factors that affect the building energy consumption. Hence, in our model, we are going to consider outside temperature, humidity, wind speed, solar irradiance, as the environment induced external factors. In addition, the grid data such as electrical load, peak load, electrical price variation during the day will also be considered as external input data in our model.

Data handling
This module is responsible for data collection, pre-processing, storing, normalizing, and feature extraction. The data collector collects both building data and external data. Considering the fact that the raw input data is incomplete due to noise and inconsistencies, it is necessary to perform pre-processing of data. Then, feature extraction and data normalization is carried out to ensure that that the data is conditioned properly before being fed to the ML model. In the following, we have described the tasks associated with data preprocessing, feature extraction and normalization. Data pre-processing: Data pre-processor helps to perform transforming and scaling the data set before data feed to the ML model. There are several steps associated with data pre-processing: Data cleaning: Data cleaning is responsible for identifying, smoothing, filtering outliers and noisy data as well as resolving the inconsistencies of the data.
Data integration: Data integration refers to the process of combining data from different sources. It is necessary to perform this task to ensure that all data is compatible, with no conflicts between data from different databases. In addition, data integration ensures data redundancy.
Data reduction: The task of data reduction is to select a subset of data from the dataset to be used for the experiments. The reduction is performed in such a way that it will not drastically affect the results in comparison to the utilization of the complete dataset for the experiments.
Data discretization: Data discretization is carried out to replace numerical attributes with nominal ones.
Data normalization and feature extraction: We carry out data normalization to convert the inputs to a common scale. This process is required if the input features have different value ranges. Feature extraction is the process of selecting a subset of relevant features and thereby reducing the number of features in a data set. This would help in avoiding over-fitting, improving generalization along with reducing the training time.

ML model
The ML model is used to predict the heating and cooling energy demand for a pre-defined period of time with the help of the historical data. Before feeding historical data to ML model, we need to evaluate and filter appropriate ML models relevant to the problem domain. Then, we split the data set, so that 80% of data isused for training and cross-validation, while the remaining 20% is kept for testing. Thereafter, the training data is used as the input to train the algorithm. It is intended to incorporate NEA for training and optimization phase of our framework. After completion of the training phase, we use test data to validate the trained model before applying it to our problem.
To get the estimated values, the new data will be fed to the trained model after performing data handling which includes pre-processing, data normalization, and feature extraction. The estimated values of energy consumption are then utilized for optimizing the energy consumption by creating efficient schedules for heating and cooling of the building in near future. Optimization Next we focus on to find a way to minimize the total heating and cooling energy consumption of the building while maintaining thermal comfort inside the building. To achieve this, we will formulate an optimizer as mentioned below. Assume that the comfortable range of temperature inside the building is given by [Tmin,Tmax]. Then, we can formulate the optimization problem for the consumption of heating and cooling energy as, where, T opt , T k Tˆ, T org represent the optimal temperature, temperature at time k, temperature set point, and the original temperature at given settings respectively.
Moreover, E heating andE cooling represent the energy consumption of heating and the cooling energy consumption respectively. Thereafter, we need to minimize the cost of energy in the building. To achieve this, first, we need to inspect the types of loads in cooperate with the BEMS. We can divide the load into two types, fixed loads and flexible loads. Fixed loads: The fixed loads include security systems, alarm systems and surveillance systems. These loadshave a critical energy demand requirement that needs to be satisfied when considering the load distribution. Flexible loads: The flexible loads include the devices that can operate during off-peak hours when the costis low and the manageable loads such as heaters, fans, lights. The operation of these devices can be scheduledduring the day.
Then the optimal cost of energy for theithconsumer can be calculated as, Where Q c , Q u , P G , P Fixed , P Flex , S d represent the price that the utility company pays for a customer, electricity price of the utility company, power generation, fixed power consumption, flexible power consumption, and the status of the electrical devices "On/Off" respectively. Moreover, n is the number of electrical devices.

Conclusion
The objective of this paper is to present possible strategies for an energy consumption optimizing framework in an industrial setting having office spaces and laboratories. The proposed framework suggests workflows for collecting the input data and the preprocessing steps: cleaning, normalization and feature extraction. Thereafter, the utilization of ML and neuro-evalutionary techniques are proposed for prediction of future energy loads. Then, we aim to identify optimization techniques for better scheduling of heating and cooling energy demand for a period of time ahead. Finally, we try to minimize the total cost of energy for a particular consumer. Key research areas identified through this paper is using machine learning techniques and neuro-evaluation inrelation to energy consumption optimization.   [4,5]. However, the use of such solution is still mostly limited to passive applications while the use of PCM as part of a standalone cooling/ventilation solution is very limited and needs additional investigation.

Related literature and theories
Several studies have investigated PCM-based cooling solutions for meeting the cooling demand. Experimental and numerical investigations of an active cooling application utilizing PCM in a PCM-water heat exchanger was carried out by Roccamena et al. [6,7]. Different control strategies for reduction of peak power consumption were tested for an air-PCM heat exchanger cooling device, which was placed directly in the supply air stream of a ventilation unit by Mankibi et al. [8] and Stathopoulos et. al [9]. Building up on the block of investigations, this study will present the design, modeling and performance simulation of an innovative air-PCM heat exchanger-based cooling solution for ventilation in office buildings under Danish climatic conditions. The storage potential in the climate module will provide flexibility options in the building sector that can balance the electricity grid and aid in reducing the fluctuations in supply and demand. This will be enhanced by implementing intelligent control strategies that will enable interaction with the demand response market [10,11]. The climate module also allows for load shifting strategies to be employed with intelligent forecasting of production patterns and local weather conditions. System modeling To quantify the added benefit of the climate module and demonstrate the expected performance of the concept, a dynamic energy performance model for the room and the overall ventilation unit, consisting of the heat exchanger and the PCM climate module is developed. The interaction between these is illustrated in Fig. 4. The PCM module is modeled using a 2D numerical modeling approach that assumes uniform behavior from each PCM plate and therefore allows modeling of only a single PCM plate. The numerical model uses the ODE15s solver in MATLAB to solve the finite difference formulation of the discretized 2D heat equation for the heat transfer dynamics in the PCM material. The heat transfer in the climate module is calculated through estimation of the convective heat transfer coefficient, using correlations for the Nusselt number for flow between two isothermal plates [12]. The latent part of the PCM is modelled using an apparent heat capacity method, where the specific heat capacity includes the latent heat of the PCM and the phase change temperature [13]. The PCM model considers the hysteresis of the PCM, meaning the non-isothermal phase change and different solidification and melting temperatures, as shown in fig. 5. An R2C2 room model is developed to evaluate the behavior of the room to be cooled under different climatic conditions, internal loads and operation schedules. [14].
The ventilation unit has a cross flow heat exchanger with an overall heat transfer coefficient calculated based on the NTU method [16]. The modeling of the room and the ventilation unit is developed in MATLAB. The evaluation of the performance is done using two Key Performance Indicators (KPI), thermal comfort violations and energy consumption.  Fig. , with the modeled PCM temperature, the ambient air temperature and the resulting supply air temperature. This period resulted in total thermal comfort violation of 94°Ch. This suggests that the system is highly sensitive to the ambient environment, which needs to be further considered in the final design phases of the module.

Case study
To evaluate the energy use of the proposed solution, the energy consumption from the fans and the added pressure drops in the PCM module are considered. The energy consumption for a year was compared to that of a DX technology. The DX system is assumed to have a pressure drop of 15 Pa and an average coefficient of performance of 2 over the entire year. The comparison can be seen in Fig. . A substantial saving in the electricity consumption is achieved with the PCM solution with a consumption of 109 kWh where the DX system consumes 220 kWh of electricity on an annual basis.

Expected results
The developed model is planned to be used as a basis for detailed design analysis and actual production of the proposed PCM cooling unit, allowing the assessment of important design and operational parameters along with control strategies. Investigations of demand response is expected to show an advantage of this technology compared to conventional cooling technologies. Parametric analysis of design parameters for the module and of control set points for the control of the module is expected to yield optimal conditions and increase the performance of the system both in terms of efficient utilization of resources and the archived thermal comfort. The technology is a contender for the DX-based cooling technologies that are in use currently. It is expected that the PCM based cooling concept will allow for more efficient operation than the DX solutions, but that the PCM technology will not be able to maintain as high level of thermal comfort at all times as the DX-based solutions.

Conclusion and Discussion
The development of a cooling concept utilizing a PCM module for comfort cooling is described. The boundary conditions are found to be crucial for the thermal comfort of the considered room, resulting in substantial comfort violations during periods of high ambient temperatures. This issue can be counteracted by using a PCM with a higher melting temperature resulting in better performance during extreme conditions but worse performance in normal operation conditions. The result is a tradeoff between a reduction in the maximum thermal comfort violation and better average thermal comfort. More PCM mass could also be employed to increase the usage time of the PCM unit. However, this will increase the investment costs of the module and lower its overall cooling yield/kg of PCM. Substantial energy savings are found with the PCM based solution in comparison with the conventional solution. Part of these savings are due to the PCM based solution lacking the same performance as the conventional technology and it is therefore possible that the benefit in the current configuration are overstated when considering the consumption pr. energy delivered. Future work will investigate the performance under different climate zones where it is expected that hotter climates in Southern Europe, the middle east or Asia will prove more fruitful for the proposed solution and will yield an increased efficiency due to a higher cooling demand.   Summary: The utilization of demand-side flexibility is vital in future energy systems where the supply-side relies on renewable energy sources. In this transition, it is widely recognized that the building sector will hold a key role in providing demand-side flexibility, being currently responsible for 40% of total energy consumption in the EU.
To provide this flexibility, advanced building control must be made more accessible considering that most buildings today still use inflexible rule-based control. Currently, implementation of advanced building control is associated with cumbersome and time-consuming manual work which requires expert knowledge. In this work, a datadriven and autonomous approach is therefore proposed, in the form of three general project objectives, to overcome these barriers. Namely, building energy modelling and performance data monitoring and evaluation algorithms will be identified to support energy flexibility identification and quantification. Subsequently, an automated building flexibility methodology will be developed to form the basis for energy operational optimization. Finally, this methodology will be implemented and tested in case studies to assess the impact on the energy system operation.
Keywords: Demand-side, Flexibility, Optimization, Building, Automated, Data-driven, Machine-learning Motivation The increasing integration of renewable energy sources (RES) is a necessary step towards low emission energy systems. However, energy systems with a production portfolio comprised majorly of RES such as wind or solar power provide considerably less flexibility on the supply-side compared to traditional production units such as coal-fired plants. In parallel to this transition of the supply-side, the demand-side is also seeing extensive electrification with e.g. electric vehicles and heat pumps. Therefore, with the expected increase in electricity consumption and decrease in supply-side flexibility, future energy systems are expected to face considerable challenges balancing production and demand. While extensive research efforts have been performed on the supply-side for optimal operation and allocation of resources, it has for many years been under the assumption of an inflexible demand. Therefore, there is currently an unexploited flexibility potential on the demand-side, especially within the building sector, which is vital to utilize to deal with the before-mentioned system-balancing challenges. In recent years, much attention has already been put on the building sector by the EU, acknowledging the high potentials for cost-effective energy savings Eur18lg. It has been estimated that buildings are responsible for 40% of energy use in the EU MJr17lg with electricity accounting for over 48% of the total energy consumption KOA16lg. With such a large share of the total energy use, it is thus essential to consider buildings as one of the major sources of flexibility in future energy systems KOA16lg.

Related literature and theories
Demand-side flexibility can be directly related to the portion of demand in the energy system that can be reduced, increased, or shifted within a specific duration. In a recent report [4], demandsside flexibility was further divided into two categories; explicit and implicit demand-side flexibility. Explicit demand-side flexibility reflects dispatchable energy managed by aggregators acting on the wholesale, balancing, and reserves markets. Implicit demand-side flexibility, on the other hand, is related to the change in energy consumption based on price signals. One of the widely implemented approaches dealing with these concepts is Demand Response (DR). A commonly used technique within DR is load shifting where the consumers willingly shift their consumption away from peak consumption hours and/or towards peak production hours, with some form of monetary compensation Nik17lg. The benefit of load shifting is that the net final consumption is unchanged and that no energy storage conversion losses are introduced Pet15lg. Contrary, it is completely dependent on the consumers' willingness to shift their demand, which is one of the major barriers to broadly adopt this concept Nik17lg. Therefore, a major part of current research targets the use of energy storage, where the disruption of the consumer consumption habits is avoided at the cost of storage conversion losses. In the report Examples of Energy Flexibility in Buildings, developed as part of the Annex 67 initiative, the sources of flexibility were identified majorly as energy storage approaches. Among the cases studied the thermal mass of the constructions was considered in 66% of the cases, thermal storage in 34%, battery storage in 29%, and fuel switch in 6% Int19lg. For buildings, these numbers give a good indication of where the flexibility potential currently is believed to be found. As indicated by these numbers, the use of the thermal mass in constructions is a popular approach, as there are no investment or operational costs associated with this source of flexibility, all other things being equal. The method works by storing energy in the envelope of the building by manipulating the indoor temperature as illustrated in Fig. 8. In Fig. 8 the setpoint of a building is controlled according to a simple rule-based controller with two price thresholds determining three different temperature setpoints. As seen from the figure, the setpoint is decreased when the electricity price is higher than the high price limit and increased when the electricity price is lower than the low price limit, while it is kept constant inbetween. Although this type of control is very simple to implement and has demonstrated flexibility benefits compared to a constant setpoint controller Hic18lg, it doesn't take into account  the operational costs of the building. Ensuring that complex and often highly non-linear building systems are controlled reliably and effectively, considering both price and CO2 signals, poses significant requirements to the control algorithmThi19lg. Recently, Model Predictive Control (MPC) has been demonstrated to be a promising candidate for this purpose. This control method is based on a repeated optimization scheme for a given time horizon, where only the first part of the solution is implemented in the operation strategy for each time step. Contrary to rule-based control, MPC is thus able to account for future system states as a consequence of current control actions using a model of the system and typically weather and price forecasts. Though MPC has shown significant cost and efficiency improvements in both simulation environments and real case studies Chr20lgmDat, it has not been adopted widely in commercial applications due to significant hardware, software, and technical knowledge requirements Jánlg. This problem was also encountered in a recent case study, which found that employing MPC in a Swiss office building resulted in higher total costs than the operational savings gained compared to a traditional controller Dav16lg. Because of this, most energy service controllers are currently rule-based Datlg, although it is expected that model-based methods such as MPC will be utilized to a much larger extent in the future Int19lg. Therefore, to quantify and utilize the available demand-side flexibility, there is a need for making advanced building control more accessible in both residential and commercial applications. Currently, one of the major barriers, to implementing model-based control, is the expensive and labour-intensive task of manually creating and calibrating an accurate building energy model. Additionally, the model complexity must be constrained to allow for operational optimization, which typically has been handled by using model reduction techniques to reduce the order of the model Col13lg. However, this adds additional work to the model development. In the context of holistic modelling of energy systems, Reynolds et al. Rey18lg reviewed white, grey, and black box energy modelling techniques for conversion technologies, forecasting, and buildings. In the study, it was concluded that for real-time optimization purposes, white box modelling is not ideal, being computationally very expensive. The paper instead recommended data-driven, grey, or black-box models claiming that especially machine learning methods perform very well concerning the important trade-off between computational complexity and prediction accuracy. Additional advantages for datadriven modelling are discussed in Fra18lg; firstly, data-driven models are scalable, which means that the same model structure and model development methodology could be applied to similar systems. This means that no assumptions or approximations must be made explicitly by the modeler, which might make it possible to develop a methodology allowing for effective and automated model development and optimization. In addition, such models are flexible and adaptable, allowing implementation in various applications and building types with minimal manual work or modifications introduced. In this regard, the success of this modelling approach is completely dependent on both the availability and quality of data, but not as dependent on the domain-specific knowledge and skill of the modeler, as for first-principles modelling. Using data-driven techniques for automation is not a novel idea, but has been a popular research topic in the industry with the concept of Industry 4.0 Dje19lg. In recent years, the concept of a Digital Twin (DT) has also emerged with successful implementations in sectors such as manufacturing and production Fei19lg. In the energy building sector, it is still in its early stages and there is still no consensus on a specific definition of DTs among researchers. However, a DT is generally understood as a digital representation of a system that continuously adapts to the real system through the use of data collected on-site Raf20lg. According to Fei19lg, a DT can thus be divided into three main components; the actual system, the digital representation, and the data flow linking these two components together. Even though data-driven modelling and control have shown promising results in the past years, there are still critical questions left unanswered as presented in Datlg, a review on data-driven predictive control in buildings. Firstly, it is not yet well understood which data sets and features are relevant for data-driven building energy modelling and how the selection of these differ, considering different types of buildings. Hence, it is also not well known under which conditions data-driven modelling and control methods scale to different building types and how the quality and quantity of data affect the performance of data-driven approaches. This is required to understand relevant use-cases and requirements for sensory placements in the buildings. Finally, data-driven black-box models such as Neural Networks, Support Vector Machines, etc. usually are highly non-linear models, which makes them hard to implement in model-based control schemes such as MPC, due to the often non-convex solution space. Although several studies have demonstrated successful implementations [20,21,22,23], there is still no consensus on an approach that generalizes across multiple types of buildings and datasets. Therefore, further research must be conducted to find how the developed data-driven models can be used for operational optimization in buildings.

Project objectives
Based on the aforementioned potential of data-driven methods, this research proposal will seek to accomplish the following objectives: 3. Identification of building energy modelling and performance data monitoring and evaluation algorithms to support energy flexibility identification and quantification within a holistic DT environment. 4. Design and development of an automated building flexibility methodology to form the basis for energy operational optimization. 5. Implementation of the automated methodology in multiple case studies to assess the impact on the energy system operation.

Methodology
In objective 1, the use of machine-learning and statistical methods will have a major role in modelling specific components or to create whole building energy models as an attempt to automatize the modelling phase. The identified methods will be tested in both simulation environments as well as in real case studies where conditions will be established, for the type of systems as well as the quality and the amount of data for which the data-driven modelling methods can be successfully implemented. Here, an important judgment condition for the model performance is the generalizability and robustness of the datadriven models, i.e. how well it performs outside the range of operating conditions upon which the model was trained. Using the data-driven methods identified in objective 1, an automated methodology is developed in objective 2. Here, the methodology will define applicable data-driven methods along with necessary model inputs, data quality, and quantity for different building system components or whole building models. The methodology will make it possible to construct an algorithm, which can automatize a major part of the model development phase. It is furthermore investigated how these data-driven models can be implemented in operational optimization. Objective 3 will be completed in cooperation with the industry through a series of case studies in the form of buildings or clusters of buildings. Hence, the validity of the developed automated methodology will be demonstrated and tested in real-life applications.

Expected results
During the project, it is not expected that the identified data-driven models will necessarily outperform more traditional and well-calibrated white-box models in terms of prediction accuracy. However, the goal is that the developed methodology will contribute with a more autonomous and flexible building energy modelling than traditional modelling approaches, which fits well within a dynamic DT environment. Coupling the developed models with model-based control such as MPC, many of the existing barriers discussed in this work, which prohibit the utilization of building demand-side flexibility, will be removed. Summary: Due to the continuous integration of fluctuating renewable energy sources in the energy system, the security of supply is challenged. For stability and security of supply in the future energy system, it is necessary to integrate consumers through demand response. Industrial consumers have been identified as a group of consumers with significant energy flexibility potentials. However, industries hesitate to adopt energy flexibility measures due to uncertainties concerning product quality and the overall impact on production flow. This paper presents the ongoing research in developing a digital twin framework for risk mitigation in the production flow by bridging the gap between risk mitigation and demand response participation. As a result, the research provides a novel approach for production flow risk assessment and energyaware production strategies. Using agent-based modeling as a foundation, a digital twin framework provides a robust industrial production solution. The digital twin components have been identified using agent-based modeling abstractions that govern production flow, including process, environment, product, batch, planning, conveyor, and transportation. The framework has successfully been applied across several industries for evaluating energy flexibility potentials while focusing on production constraints. Keywords: Industry 4.0, Digital Twin, Energy Flexibility, Production Process, Agent-based modeling

Motivation
The increasing renewable energy resources in the energy system create challenges in the electricity grids, e.g, security of supply and balancing the production and demand side [1]. An established method for balancing the electricity grid is consumers' active use of dispatchable loads through demand response (DR) programs [2]. The International Energy Agency (IEA) recommendations to increase the DR utilization prioritize large loads that are easy to manage and continuously evaluating the potential for new demand-side flexibility services [3]. A sector with significant loads is the industrial sector accounting for 37 % of the total final energy use in 2018 [4]. Hence, there is an interest in utilizing the industrial loads intelligently to coincide with the energy system's need for stability [5]. Especially, many countries, e.g., Denmark, have been promoting DR via electricity markets [6]. However, compared to the energy flexibility in buildings, industries have more barriers and constraints to adopt DR [7,8]. A study by [9] shows that 83 % of the industries have expressed concerns regarding the interruption of production, 78 % have been concerned about the impact on product quality, and a smaller proportion of industries also reported concerns about comfort and lack of business cases. A digital twin (DT) is defined as "a digital representation of a realworld entity or system. The implementation of a digital twin is an encapsulated software object or model that mirrors a unique physical object, process, organization, person or other abstraction" [10]. DTs allow users to reflect the current system operation and perform scenario testing on production policies to predict future expected system states [11]. Therefore, DTs potentially bridge the gap between industrial hesitance to adopt DR solutions due to the unforeseen impact on the production flow and the need for flexible assets in the evolving energy system. However, little literature has focused on the digital twins for the entire production processes. Furthermore, although some studies have focused on energy flexibility in the industrial processes, e.g, cement production [12] and water supply [13], the literature has only focused on the implicit DR with hourly electricity prices and financial gains or CO2 reduction [14]. Industrial processes and product quality have not yet been well investigated.
Research aim Therefore, this Ph.D. research aims to develop a digital twin framework for industrial production processes that can accurately depict the effects of factors influencing the production flows and quantify the potentials for energy efficiency and demand response participation. The framework is expected to provide a generic approach that can be used in varying spatial and temporal production systems. Through AI and IoT, the DT can be fed with historical and real-time data that allows testing varying production strategies with minimal risk to the products. Hence, the DT enables the facility to integrate energy flexibility measures while also accounting for product quality and flow. The DT is created to incorporate specified external data flows, i.e., electricity prices, CO 2 emissions, and weather forecasts. Based on the current findings, several key framework components have been identified. Using a productcentric approach ensuring that the quality is not compromised. The novelty of the presented research is the combination of I4.0 and Big data analytics for developing a digital twin framework that includes energy system considerations and reflects the effects of energy flexibility measures. The framework can be leveraged for optimizing production flow under energy system constraints that can enable the facilities to minimize environmental impacts while maintaining the performance of the productions.

Related literature
A scoping review is conducted in this Ph.D. research to exam the potential for industrial energy flexibility evaluation reviewed the current trends in assessing industrial processes' flexibility potentials.
The results show that the majority of existing studies have focused on a single process and used optimization as a means of matching a production strategy with the state of the electricity market [15]. Focusing on a single process is stated to be a concern as decisions regarding energy flexibility may propagate to other parts of the system. Hence, the entirety of the production flow needs to be included. To enable mirroring the actual facility operation, simulation becomes an increasingly viable solution. A study by [16] compares optimization and simulation for flexible energy production, and shows that simulation could provide optimal results 80 % of the time compared to optimization while having significantly lower computation time. The acceptance rate of simulation is shown higher than optimization due to the ability to perform a visual inspection [16].
To develop a DT framework for production flows that can accurately predict production time and expected completion, it is essential to integrate product state and quality with the production process. An example is shown in the study of [17] that uses a multi-agent-based simulation to examine implicit DR participation's potentials using industrial brewery fermentation tanks. The study shows the DR potentials within the brewery process while still adhering to the productspecific quality restraints. As an initial development of a greenhouse DT, a data architecture is proposed for collecting critical information exchange in a Smart Industry Architecture Model [11]. The possibilities of creating a self-configuring DT that enables the solution's adaptability and versatility [18]. This research project targets the needs of realistic industrial implementations supported by increased interdisciplinary collaboration stated in the scoping review and utilizes the simulation's benefits stated in [16]. The developed DT follows the principles outlined in [18] to further contribute to the framework capabilities and increase the solution's robustness.

Methodology
The DT framework includes the modeling methods of agent-based modeling, discrete event simulation, and system dynamics. The simulations are developed by using software called AnyLogic which allows multimethod modeling. The main methods applied in the PhD research are: Agent-based modeling: The DT framework is built based on a multiagent system (MAS) approach. Agent-based modeling (ABM) focuses on the behaviors of individual agents in a system [19]. An agent refers to an entity that adapts to its internal and external environment by a given objective [20]. Therefore, the MAS's behavior becomes a result of the combined individual agent behaviors, which are known as emergent phenomena [21,22]. Furthermore, ABM can ease the manipulation of agent population sizes to fit specific use cases [19]. Discrete event simulation: can represent a discrete sequence of events in mutually exclusive intervals [23]. Discrete event simulation follows a stochastic behavior in which randomness is introduced through statistical distributions; furthermore, state changes occur at irregular discrete time steps [24]. Discrete event simulation is suitable for applications that follow a sequence of operations commonly observed in production facilities and logistics [25]. In this project, the discrete event simulation is used within specific agents with deterministic behavior, i.e., production flows. System dynamics: follow a deterministic behavior in which state changes occur in continuous equally spaced discrete time steps [24]. System dynamics allows qualitative data to be considered and is often used to model the internal behavior patterns that arise from the interaction of subsystems [26]. In this project, system dynamics are used to represent agents that experience changes observed in regular time steps, e.g., temperature. Big data analytics: an essential part of transforming the developed simulation model into a digital twin is through the use of big data analytics. Big data analytics stem from the continuous flow of data found in I4.0 facilities, which provides an opportunity to capture complex system dynamics and enable data-driven decision-making [27]. The application of Big data analytics can be classified as descriptive, predictive, and prescriptive depending on the technique used for the data analysis [28]. In a true digital twin that mirrors the actual production, the simulation model can be used as a testbed for running what-if scenarios and testing policies. This allows checking specific modes of operation with minimum risk to the actual production flow. In this project, the DT framework is developed to integrate information from other systems and DTs through a horizontal integration approach. By incorporating multiple DTs into one, the specific facility's combined operation is represented, and the potential changes to one part of the system can be reflected in other parts of the system through the data exchange [11,29]. Hazelcast (an in-memory computing platform) has been identified as a viable solution for DT integration, providing a distributed data structure for exchanging information across clients. Furthermore, in-memory computing offered by Hazelcast is significantly faster compared to stream processing [30]. To validate and ensure the adaptability of the framework, the framework will be tested on several distinct types of production processes including an industrial cooling process, brewery, and several industrial greenhouses. To validate the framework, the developed digital twins will be compared with the physical twins to ensure that this is accurately depicted before performing further testing and optimization. The verification of the digital twins will furthermore be evaluated in terms of accurately demonstrating and predicting the consequences within the model due to changes in the physical twins, i.e. if a change is performed within the digital twin and subsequently performed in the real production, are the two outcomes similar.

Expected results
The DT framework developed in this Ph.D. research project includes the following agents (shown in Table 1), and the agents can be viewed as abstractions that can be created based on the given input parameters, e.g., processing time, dimensions, quality requirements, etc. Meanwhile, agent communications according to the given rules (shown in Table 2). The framework with related agents and agent communication is illustrated in Figure 1. The framework provides standardized building blocks that may be combined in numerous ways to reflect the chosen production process. Figure 1 shows that the central core component within the framework is the product that travels through the production steps. Multiple individual products can be collected in a batch that holds logistical information for the products. The products are placed in processes, conveyors, etc., which are placed within an environment. The resources are placed on the top level as the different environments and processes may share the available resources. The framework should be considered a preliminary setup as it will possibly change throughout the research described. The research results are expected to enable industrial production facilities to transition efficiently into DT use for production optimization through the developed framework. Using the developed DT framework approach, the DT solution can be easily adapted to fit the specific facilities' varying layouts and parameters. The framework has been applied in industrial greenhouse production, a cooling house facility, and a brewery to validate the framework's adaptability. An example of the greenhouse production facility is shown in Fig. and Figure 3. Summary This paper presents the ongoing PhD research in developing a framework for developing industrial production process digital twins. The PhD research project was initiated during 2020 and will be conducted until 2023. In the current state of the research, an initial framework is being built for industrial greenhouse production facilities. Once the greenhouse production framework has been completed, the concepts applicable across various types of production facilities will be collected in a generalized digital twin framework for industrial production processes. Subsequently, the framework will be tested on other production facilities, which will validate the adaptability of the framework. In this stage, the framework may be extended to include additional required functionality found in the other production facilities. Conclusively, this will result in a framework for developing digital twins in production facilities.

Agent Description
Process Agent is a representation of a generic production process with an arbitrary production time. The process agent holds a specific number of product agents, corresponding to the maximum number of products that the specific process can contain.
Environment Agent contains the environmental production parameters for a given part of the system. The environment is an essential agent for assessing the relationships between the production environment and the product qualities. In a specific production facility, individual parts of the system can provide various production parameters to the environment agent through IoT devices. The production environment is typically controlled through a controller which adheres to specific setpoints. The environment's behavior can be influenced by processes, products, and resources which it contains.
Product Agent is the central object moving through the production facility's pre-defined production stages. The product is affected by the production environment and the process agents. The product's quality is monitored throughout the production flow to ensure that it is kept within the specified restrictions.
Batch Agent is highly related to the product agents but functions as a collective reference point for all product agents within the same batch. A batch may hence be constituted of multiple product agents that adhere to the same logistical parameters. Hence, within the batch agent, information such as product deadline, customer, price, etc., is stored.
Conveyor Agent moves the agents within the facility. The conveyors are used within production systems to move the products between processes. The conveyor works autonomously and requires no resource to function. The conveyor agent can function either as an accumulating type or a fixed distance type depending on the specific conveyor used.
Transportation Agent is similar to the conveyor agent but is governed by a resource need to function. Hence, the transportation agent requires an available resource to be active, e.g., a forklift needs an operator to work.
Ressource Agent represents the resources required to operate the facility. Parts of a facility may be automated and operated independently of available resources, whereas other processes require available resources to function. Personel is an example of a resource agent required to operate specific machinery or, e.g., operate a forklift. Parts of the production flow may hence be limited to resource availability.
Planning Agent is the central control unit that determines the movement of products and the facility's operation. In practice, the planning agent can be integrated with the facility's internal signals and is thereby a connection placeholder. However, the planning agent was constructed to allow for scenario testing in which the operation of the DT is decoupled from the actual operation.

Related agents Their primary communication
Product and process agents Generally, the framework recognizes that a product should pass through a number of production processes to be completed. Once the product has completed a process, it is transferred. The products will transfer to the next production step through a conveyor or other means of transportation between the processes.

Product and conveyor/ transportation agents
To transfer the products within the production, the products will be moved by either conveyor or transport. The conveyor may hold multiple product agents at the same time. Unless otherwise specified, the movement will adhere to FIFO.
Product and batch agents All product agents within the production are assigned a batch. The batch holds logistical information that governs the start date, deadline, etc. The batch can, in some sense, be considered a parent class from which the products inherit logistical information. The relationship is a one-tomany in which a batch can hold multiple product agents, but a product agent can only respond to one batch agent.

Process and environment agents
Within the framework, the product, process, transportation, and conveyor agents are all associated with a local environment. The relationship follows a one-to-many principle in which an environment can hold multiple of the previously mentioned agents. However, each of the agents can only respond to one environment. The features of the environment may vary based on the case, but examples of environment features include temperature, light levels, carbon dioxide concentrations etc. Furthermore, the environment agent holds the possibility for external data integration through, e.g., IoT sensors, which allow real-time monitoring of the specific environment.
Resource and other agents A resource is considered any entity required to act. Furthermore, the individual resource units can be assigned varying parameters related to specific task availability. This allows differentiating the resource population and having multiple resource populations within the DT. E.g., a specific task within a production facility may require personnel with specific knowledge, certification, etc., to perform. This can be captured through the different populations. Tasks will be assigned based on user input as the tasks required within specific facilities can vary greatly. The tasks can be created to use specific resource units and a specified number of resource units.

Planning and other agents
The planning agent is the central operational component from which several decisions are taken. The planning agent monitors the current state and number of products staying in the processes, conveyors, and transportation agents and chooses when and where to move the product agents. The planning agent refers to the batches associated with the products To determine if the products within the batch are on schedule or delayed. The planning agent is also the external communication point from which information within the DT is communicated to existing control systems, EPR, etc.    Summary: Faults and anomalies in buildings are among the main causes of occupant discomforts or energy inefficiencies. Therefore, early fault and anomaly detection are important for improving buildings operation. A statistical process monitoring measure for room-level condition monitoring of the building is proposed in this paper. The proposed measure uses multivariate statistics to detect faults and anomalies and therefore helps to improve the performance of the smart buildings. This method firstly employs principal component analysis for dimensionality reduction. After the projection of the observations into a principal component subspace, the method uses Hotelling's T squared statistics to detect deviations in the principal component subspace. The approach has been tested on the real building which is located in Odense, Denmark and the results are presented. The results have shown that the method could detect faults and anomalies successfully without requiring sophisticated and computationally expensive training. The method is scalable, adjustable, and implementation-wise simple. Therefore, it is potentially suitable for wide adoption in practice. Keywords: Fault detection, anomaly detection, building operation, PCA, statistical process monitoring Introduction The building sector consumes 40 per cent of the total energy consumption worldwide and contributes to an average of 30 per cent of global carbon emissions [1]. Energy-related CO2 emissions from buildings have increased over the last few years after being flattened from 2013 to 2016. In 2019, the emissions have reached a new record of 10 Gt CO 2 [2]. Hence, the urgent need to reduce CO2 emissions from buildings is a large market opportunity, but it also presents many barriers [3]. Buildings are part of complex ecosystems where the primary uses of the buildings are more important than their energy use. This is for instance the case for commercial buildings like retail stores [4] and hospitals [5]. In the past, commercial and public buildings only had manual lighting switches in each room and a centralized air conditioning (HVAC) system operating with fixed schedules. However, in the last decade, the complexity of buildings has increased enormously. Modern buildings are becoming smart buildings with automatically and centrally managed building management systems (BMS), which controls all building subsystems such as lighting, heating, ventilation, and air conditioning (HVAC). The current state of the building is recorded through a complex network of sensors and meters, which measure indoor conditions such as temperature and air quality, and operating quantities such as ventilation rate, light intensity level, heating, and cooling signals. Managing building energy consumption and sustaining an ideal indoor climate for occupants require extensive monitoring and sensing mechanisms within the building compound and outdoor [6]. This is to ensure that accurate information can be obtained on the overall energy consumption and its indoor climate. Every single component of every building subsystem is subject to wearing, misconfiguration, and, in general, faults. A fault is any instance of a component that does not perform its task as expected. Isermann defines a fault as 'an unpermitted deviation of at least one characteristic property of a variable from acceptable behavior. Therefore, the fault is a state that may lead to a malfunction or failure of the system' [7]. Examples of faults are a stuck sensor that returns constant readings regardless of the actual measurement or a noisy sensor that returns inaccurate readings. Faults impact building operations in two different, but not necessarily distinct, ways. They may cause occupants discomfort or energy waste. Occupancy discomfort happens when the building's operation is degraded such that the indoor conditions are no longer within the acceptable range for human's comfortable conditions. A broken heating system during winter, for example, could cause the indoor temperature to fall below 15°C. Energy waste, on the other hand, happens when the system consumes more energy than it should according to its design. An example is the simultaneous heating and cooling of space. The two aspects often oppose each other, i.e. a fault causes either occupant discomfort or energy waste, but sometimes a fault results in both. E.g. broken lights, while causing discomfort, actually result in energy savings. Similarly, insufficient insulation does not affect occupants, if the increased heating results in appropriate indoor temperature. On the other hand, a setpoint wrongly set to 10°C during the summer has the double effect of making a room extremely uncomfortable, and of unnecessarily increasing cooling. Faults causing occupants' discomfort are usually easily detectable, at least because occupants would complain to the building management. Faults causing energy waste, on the other hand, are more subtle and difficult to detect. If no system is set up to monitor a building and no maintenance operation is scheduled, faults can go unnoticed for a very long time. For this reason, most research about faults is done in the context of energy waste [8,9]. To reduce the occupants' discomfort and complaints, as well as to save energy and to prevent total system failure, it is important to detect faults and anomalies as early as possible.
The so-called fault detection and diagnosis (FDD) methods for buildings have been reviewed in [10,11,12,13]. In general, the methods are divided into process-history-based, quantitative-model-based, and qualitative-model-based methods, and are further divided according to the specific technique. Each of these families of methods has different advantages, disadvantages, and trade-offs, as well as implementation constraints and caveats. The performance of the model-based methods relies on the accuracy of the model. However, accurate building modelling is laborious. Therefore, model-based FDD methods such as [14,15,16] are hardly scalable. The processhistory-based or data-driven FDD methods have shown great potentials [17]. However, they rely on data and their performance is compromised if the training data set is not of sufficient quality [18]. It is therefore important that the data which is used in the FDD design process of such methods being validated [19]. A data-driven measure for room-level condition monitoring of the building is proposed in this paper. The proposed measure uses multivariate statistics to detect faults and anomalies at room level in smart buildings. This method firstly employs principal component analysis to project the observations into a principal component subspace and uses Hotelling's T 2 statistics to detect deviations in principal component subspace. The approach has been tested on a real building in Denmark. The results have shown that the method could detect faults and anomalies successfully. The method is scalable, adjustable, and, implementation-wise, simple. Therefore, it is potentially suitable for wide adoption in practice.
In the remainder of the paper, we first review T 2 multivariate statistics which is used in this paper. Then a PCA-based condition indicator is proposed along with appropriate thresholds for the T 2 statistics in terms of the level of significance. Afterwards, the case study building is described and the results of anomaly and fault detection are presented and discussed. The last section concludes the paper.
Hotelling's T 2 multivariate statistics As opposed to univariate statistics, Hotelling's T 2 multivariate statistics take into account the correlations between the variables. This feature makes it suitable in applications within fault detection and diagnosis [20,21]. In the following, we briefly describe Hotelling's T 2 multivariate statistics. Let the data be presented in a matrix Ψ ∈ ℝ n × m with the following structure: Energy Informatics 2021, 4(Suppl 1):30 where ψ ij is associated with the i'th observation of the j'th observation variable.
The sample covariance matrix of the given data set (1) is obtained as: Eigenvalue decomposition of the sample covariance matrix S cov results in: This divulges the correlation structure for the covariance matrix, where Δ is a diagonal matrix and W is orthogonal. The Hotelling's T 2 multivariate statistics is defined as: where and ψ is an observation vector.
In this section, a condition indicator is presented which uses PCA for condition monitoring. PCA projects the data into lower-dimensional space and produces representations of the data which better generalize to data independent of the training set compared with using the total dimensionality of the observation space. This inspires the use of PCA in this method. Let our training data be presented in a matrix form described in (1). The so-called loading vectors w ∈ ℝ n are needed to be obtained by solving the stationary points of the following optimization: This optimization can be solved by Singular Value Decomposition (SVD): The loading vectors w are the orthonormal column vectors of W. The loading matrix P is constructed by stacking the first r columns of W. The loading matrix enables to project the observation vectors ψ ∈ ℝ m into lower dimensional space of the so-called scores t ∈ ℝ r : The condition indicator κ, therefore, can be defined as T 2 statistic for the lower-dimensional space: where Σ r is the leading r × r sub-matrix of Σ , i. e., it composed of the first r rows and columns Σ. Appropriate thresholds for the condition indicator κ based on the significance level, α, can be obtained from: where F α (r, n − r) is the upper 100α % critical points of the Fdistribution with r and n − r degrees of freedom. The value of the threshold is used to detect faults and anomalies. For an observation vector from the testing data set, if the value of the condition indicator is less than the threshold, the data is related to in control (normal) situation otherwise the observation is associated with out of control (faulty or abnormal) situation.

ROOM-LEVEL FAULT AND ANOMALY DETECTION
This section is divided into two parts. In the first part, the case study and the available data are introduced. In the second part, the implementation of the method is described and the results are presented and discussed.

Description of the Case Study and Data
The case study in this work is a room in OU44 building at the University of Southern Denmark. OU44 building ( Figure 1) is a fourstory building which is located on the Odense campus. The floor area of the building is 8500 m 2 and is one of the most energy-efficient buildings in Denmark [22]. The test room which has been selected for this work is a study zone located on the second floor. We used 30240 samples of minutely measured data for this study (21 days during the spring period). The first half of the measured data is used for training and the second half for testing. The data includes outdoor temperature, global horizontal solar radiation, indoor temperatures, VAV damper positions, radiator valve positions, and occupancy counts obtained by stereo vision cameras. The study zone is 125 m 2 and is usually open day and night for student use.

Implementation, Results, and Discussion
To design the PCA-based condition indicator, we first normalize the training data set which is composed of 15120 samples, and then by following the method described in the last section, we find the appropriate projection matrix to reduce the dimension to two i. e. r = 2 and calculate κ. For each observation vector, the value of κ indicates the condition of the room. If the condition indicator κ is less than the threshold, the situation is identified as normal, otherwise, the abnormal/faulty situation is detected. The threshold κ α = 5.9934 which is obtained according to (10) with r = 2, n = 15120 and α = 0.05. The results of the implementation of the method are shown in Figure 2. As it is clear in Figure 2, three main anomalies are identified successfully. The most significant anomaly is detected for sample intervals (8900,9950). The main cause for this anomaly is a significant drop in temperature due to all windows being left open. This is clear from Figure 3. In addition, within this interval, the radiator was fully open while the room was unoccupied for almost the whole interval. The same heating issue exists for the second most significant anomaly. However, in the last part of the period, the room is quickly over-occupied with the number of students which are more than the usual capacity of the room. The third most significant anomaly is due to the opening of the window. The main reason that it does show as significant as others are that the room has not been unoccupied and the room occupancy and the outdoor temperature have increased.
The proposed method has identified the main anomalies and is simple to implement as it does not require developing building models. The approach is adjustable to different types of sensors and buildings. The method will be improved if it is empowered by an approach for quantifying the degree of importance and significance of anomalies. In this case, the facility managers can better prioritize the anomalies to be addressed for improving the operations and maintenance. This is in particular helpful when a building has many rooms to monitor.

Conclusions
In order to improve the buildings' operation, it is important to detect faults and anomalies as early as possible. To this end, process monitoring measures play a key role. A data-driven room-level condition monitoring measure has been proposed in this paper. The proposed measure only needs reading from sensors to detect faults and anomalies in smart buildings. This method finds the optimal project matrix for the projection of the observations to the lower dimension and uses Hotelling's T 2 statistics to detect deviations in the lower dimension subspace. The method has been tested on the real teaching facility in Denmark and the results have been promising. The method has shown that it could detect faults and anomalies successfully without the need for complex and computationally expensive training. The method is scalable, adjustable, and is easy to deploy. For future work, the natural next step would be to improve the method by integrating an approach for quantifying the degree of importance and significance of the anomalies.  Summary: To achieve national and international climate goals, huge investments are expected in the transition to a low carbon society. Due to large investments and high failure rates, avoiding risks (especially the value chain risk) evaluating the energy flexibility solutions and their impact on the energy ecosystem is essential. This PhD research project aims to develop an agent-based simulation framework for evaluating energy flexibility solutions and adoption strategies in a given energy ecosystem. The simulation framework consists of two sub-frameworks. One identifying and implementing energy flexibility solutions to the agent-based simulation. Another for identifying and implementing adoption strategies to the agentbased simulation. To show proof-of-concept of the developed framework, agent-based simulations of a case study are developed based on the model framework. The case study is an investigation of electric vehicle charging in a Danish electricity distribution grid. The research project's outcomes of evaluations and recommendations of energy flexibility solutions will contribute to the climate goals. Keywords: energy flexibility solutions, agent-based simulation, innovation adoption, generic framework Motivation Grid balancing becomes more and more challenging due to the increasing share of non-dispatchable energy production from renewable energy resources, e.g., wind and solar [1]. One of the solutions is to utilize the energy flexibility on the consumption side [2]. New energy flexibility solutions are introduced to the market to activate the flexibility potentials, e.g., virtual power plants and distributed energy resources [3]. However, there are many kinds of solutions, and some are more efficient but also more complex than others [4]. The failure rate to launch new products/solutions to the market is still high, and the energy flexibility solutions usually require large investments [5,6]. The market adoption rate is also usually slow and depends on different market segments and business models that companies apply [6,7]. Therefore, to avoid the risk (especially the value chain risk) and to evaluate the impacts of energy flexibility solutions on the energy ecosystem, this research project aims to develop an agent-based simulation framework for evaluating energy flexibility solutions and adoption strategies in a given energy ecosystem. This research applies agent-based simulation to evaluate energy flexibility solutions, adoption strategies, adoption rate, and business opportunities in a given energy ecosystem. The research project outcomes will support the ambitious climate goals for Denmark [8] and the Paris Agreement [9] [10]. The developed agent-based simulation framework will also assist the understanding of what-if scenarios, e.g., how to positively affect the adoption of smart energy solutions by overcoming the adoption barriers and turning their solutions/innovations into adoption triggers [7]. Consumers' behaviors, including the adoption rates and adoption speeds and their differences and similarities in the different energy ecosystems (e.g. USA and Denmark), are also investigated in the project.

Research Objectives
With a case study of the Danish electricity ecosystem, the research objectives are: 1. Develop a generic agent-based simulation framework for energy flexibility solutions in a targeted energy ecosystem 2. Design a generic agent-based simulation framework for adoption strategies of energy flexibility solutions.

Evaluate the adoption of energy flexibility solutions' impacts on the energy ecosystem
The planned research process to achieve the above research objectives is illustrated in Fig. . The first objective requires methodologies for energy flexibility selection and implementation in agent-based simulation. The development of a methodology for selecting relevant solutions starts by identifying the energy ecosystem and its CSTEP ecosystem factors [11,12]. CSTEP factors cover Climate & environment, Social culture, Technology, Economy & finance, and Policy & regulation. The next step is to investigate the State-ofthe-Art (SoA) solutions. Last, the SoA solutions are evaluated and selected. The development of a methodology for implementing the solutions into an agent-based simulation starts by selecting the energy ecosystem and implementing it to the simulation model. Next, the agent-based model is designed and solutions are implemented. Last, a generic agent-based simulation framework for algorithm implementation is developed. To achieve the second objective, methodologies for adoption strategy selection and implementation in the agent-based simulation are required. The methodology of adoption strategy selection is developed starting by conducting a SoA analysis of adoption strategies for energy-related solutions. Last, the adoption strategies are evaluated and selected for implementation. The adoption strategy implementation is developed by designing an agent-based model for implementing the strategies. Last, a generic agent-based simulation framework for adoption strategy implementation is developed. The third and last objective is achieved by identifying the ecosystem impacts based on simulation results. This is done through the implementation of energy flexibility solutions and adoption strategies in the agent-based ecosystem model. The ecosystem for simulation is represented by a case study providing proof-of-concept of the developed frameworks. Next, an ecosystem impact indicator application is developed. Next, hypotheses and scenarios are designed to evaluate the ecosystem impacts. Next, the hypotheses are tested through scenarios. The results are generated for all scenarios and the raw results are interpreted through visualization in form of graphs and relevant figures. Last, the results are analyzed and CSTEP factor analysis is conducted based on the analyzed results. The right-hand side of Fig. represents the three objectives. Their relations are shown with the arrows that the two first objectives can be made separately but are both implemented and tested to achieve the last objective. The same approach is used to describe the relationship of the small blocks on the left-hand side. The evaluation and selection of SoA solutions and adoption strategies are related to the development of the ecosystem impact indicator application. The same applies to the energy flexibility and adoption strategies implementation in the agent-based ecosystem model. This block is dependent on the agent-based modeling design and implementation of the solutions and adoption strategies.

Related Literature and Theories
The main literature and theories identified to be related to this research are described in this section. The literature and theories are mainly in the subjects of energy ecosystem, energy flexibility, energy flexibility solutions, and innovation adoption.

Energy ecosystem
In this research project, the concept of energy ecosystem is used to investigate the complex social energy system [12]. The research in the energy ecosystem has been mainly discussed by the SDU Center for Energy Informatics, and applied in the fields of energy in buildings and microgrids, e.g., [1,13,14]. According to [15], 'a targeted ecosystem is a completed business ecosystem within a defined boundary, with elements of actors, roles, and interactions, and actors create values and interact with others to complete value flows'. Therefore, the energy ecosystem in this Ph.D. research project refers to distribution grid ecosystem including actors of the Transmission System Operator, Distribution System Operator (DSO), electricity supplier, and domestic consumers; objects of the electricity grid, DataHub, EVs, and EV charging boxes; and including all five interactions of monetary, data, information, and good/product flows, and intangible interactions between actors and objects. Energy flexibility Energy flexibility is a key element in this research. Energy flexibility on the demand side means that the energy consumers shift their consumption from one period to another period that has more electricity production by renewable energy sources [16]. These periods are related to the periods where the price is the cheapest [17]. The prices give the consumers incentives to move its load to periods with lower prices, hence, helping the grid and achieve an economic benefit. This is called Demand Response (DR) which is defined by the European Commission as "voluntary changes in consumers' electricity usage patternsin response to market signals". Energy flexibility solutions Energy flexibility solutions are in this research considered as solutions that activate potential flexibility at the consumers. This could be smart meters together with an hourly price scheme allowing the consumer to do DR or an algorithm that automatically utilizes the flexibility to benefit consumers. The term "solution" covers algorithms/software/service, regulations, and business models.

Innovation adoption
Several innovation diffusion/adoption theories exist, such as "Diffusion of Innovation Theory" by Rogers 1960, "Interorganizational relationship theory" by Clark 1965, "Theory of Reasoned Action" by Fishbein and Ajzen 1975, etc. This research currently uses Rogers' innovation diffusion theory [18] to identify consumers' adoption behaviors and to find the adoption rate of new energy flexibility solutions, e.g., [19,20]. According to [18], the adoption rate is defined as: "the relative speed with which an innovation is adopted by members of a social system. It is generally measured as the number of individuals who adopt a new idea in a specific period, such as a year. So, the rate of adoption is a numerical indicator of the steepness of the adoption curve for an innovation." The adoption curves can be seen in Fig. . There are many factors have been shown that influence consumers to adopt energy flexibility [21].

Methodologies
The methodologies used in this research project can be divided into sub-sections: Scoping review, Agent-based simulation, and case studies. The methodologies are what are used to conduct the research and to achieve the expected results.

Scoping review
The scoping review methodology is used to identify the SoA solutions both in energy flexibility solutions and adoption strategies. Based on the scoping review result and CSTEP ecosystem factor analysis, this project also develops a methodology to evaluate and select the SoA solutions and adoption strategies that match the needs and criteria of a targeted energy ecosystem, e.g., [22].

Agent-based simulation and modeling
This research uses agent-based simulation as the main method for developing, testing, and validating the model framework. Agentbased simulation is a relatively new method to simulate real-life systems compared to system dynamics and discrete event modeling. The increased use of agent-based simulation happens due to the desire to get deeper insights into simulated systems. Furthermore, the growth in CPU power has influenced the increase as agent-based models demand high CPU power capacity. Agents can represent many different things in an agent-based model such as energy market stakeholders which represents agents in this research [23]. An agent-based simulation is an artificial intelligence method that allows software agents to behave close to real-life entities. Agents operate in an environment and behave and react to different external events. This behavioral knowledge is fed into the agent logic through data (e.g. historical data for how an agent reacts to specific changes in the environment) [24]. The software used for the agent-based simulation is called AnyLogic and is a unique simulation software tool that supports system dynamics, discrete event, and agent-based modeling as simulation modeling methods [23].
In this research project, agents represent actors in an energy ecosystem such as a DSO [15]. The agent-based simulation method makes it possible to study the collective behaviors of agents [25]. Hence, identifying emergent behavior and interference. Emergent behavior is behavior that arises out of the interactions between parts of a system. This behavior cannot easily be predicted or extrapolated from the behavior of those individual parts. Emergent interference is an undesirable behavior that arises out of the interactions. Emergent interference is important to identify as this can lead to dysfunctional system behavior and in the worst case cause a severe system failure.

Case study
To show proof-of-concept of the developed frameworks a radial distribution grid below a 10 kV transformer of 137 domestic consumers in Denmark is chosen as the case study EVs are chosen in the case study due to the DSOs' concern regarding the impacts of EVs on the distribution grid stability. Therefore, this research aims to investigate how the adoption of EVs affects the grid. In this research project, Electric Vehicles (EVs) are considered as a flexible demand. The EV is considered as having high potential as a flexible load due to its high consumption [26]. EVs are flexible as they do not need to charge immediately when the owners arrive home, and the EVs' charging can be shifted to times when the electricity price is low [27]. It is expected that the number of EVs will increase over time due to the goal of having one million EVs in Denmark by 2030 [28]. This calls for a solution that can utilize the EVs' flexibility to balance the power grid. Furthermore, flexibility should be utilized to avoid overload in the grid. The EV types are represented by the top five EVs sold in Denmark in 2019. The types are important as they vary in battery capacity, mileage, and charging rate. The energy flexibility solutions are represented by smart charging algorithms. The evaluation is made from the DSO's point of view, hence keeping grid stability instead of minimizing consumer cost is prioritized. To evaluate energy flexibility solutions and adoption strategies several hypotheses are designed. Two hypotheses are tested through scenarios designed to answer the hypotheses. Data for the research is mainly obtained from the Danish DSO -TREFOR [29]. Household consumption data including grid constraints are given from their distribution grid. Data for EVs and consumers' driving patterns are obtained from the literature. Furthermore, data is going to be obtained using qualitative and quantitative interviews used to identify consumer adoption behavior.

Accomplished results
The energy ecosystem built up around EV home charging is created. The stakeholders' logic, communication flows, and the environment are identified and implemented into a model in AnyLogic. The model comprises the fundamental agent-based model that is going to evaluate the energy flexibility solutions. The adoption rate in the fundamental model is based on historical data for EV adoption in Denmark. The results generated by the model so far are the times when the EVs are expected to overload the transformer with and without the use of smart charging. The smart charging in the fundamental model makes sure the EVs are charging when the electricity price is lowest within the time it is connected to the charger. When using simple charging, the charging starts when the EV arrives home which is based on the domestic consumption pattern. A large increase in consumption after 12 noon is indicating that the EV has arrived home. Furthermore, the results show how many EVs the current grid can handle with simple and smart charging. The result can be seen in Table . The results show that smart charging can have more EV charges simultaneously. However, the overload occurs much faster and more frequently as the EV consumption is placed in the same period. Hence, this strategy is not durable in the long term. Suitable smart charging strategies are identified and several are implemented in a newer version of the fundamental model.

Future works
The future work consists of developing frameworks for implementing energy flexibility solutions in the model and for identifying and implementing adoption strategies. The framework is designed to develop a generic model that can be adjusted for a given energy flexibility solution in a targeted energy ecosystem. The framework includes methodologies for adjusting the model for different energy flexibility solutions and adoption strategies in a defined ecosystem. This is done by use of a modular setup approach. Each module represents a part of the ecosystem e.g. an electricity consumer. The modules have some defined inputs and outputs making it possible to add and remove modules relatively easy to reflect a specific ecosystem, solution and adoption strategy. The methodologies should be the fundamental parts for developing the frameworks in objectives 1 and 2. The model is going to be modified to enable consumers to choose between available energy flexibility solutions in the same simulation. This should reflect a future in which the consumers can choose between more than one solution and how this is going to impact the grid.
The results from the case study simulations are expected to be a variety of dates when the first overload is expected. With the current grid, it is not expected to be able to handle 100% EV adoption without a compromise of the consumers' convenience. The results will show the best charging strategy to prolong the period it takes before overloading the grid. Meanwhile, the charging algorithms' performance will be evaluated, such as the computation cost and number of EVs to support, etc. The adoption strategies are evaluated based on how the adoption rate impacts the ecosystem. The results can help the DSO choose its strategy to how and when to improve the grid. Furthermore, it is expected that the results will suggest possible regulation changes to improve strategies. The developed agent-based simulation framework can be modified to apply to different energy ecosystems, e.g., district heating or sector coupling, for different solutions and business models, e.g., photovoltaic and heat pumps. Meanwhile, the developed framework can be applied to other geographic-different energy ecosystems, and also be possible to support the cross-national comparisons. Introduction In Japan, photovoltaic power (PV) generation has been boosted by feed-in-tariff (FIT) introduced in 2012, and its installed capacity has exceeded 50GW which is almost one-third of the country's peak demand. The Japanese Government had decided to change the system to feed-in-premium (FIP) for the new installation after 2022, in which PV systems will be treated as ordinary power generators and subjected to the requirement that the generated kWh should be equal to the planned value in average over 30 min. If the actual kWh is smaller than the planned one, PV owner has to pay the penalty determined by the total demand-supply balance of the power system. If the actual value is larger, excess electricity is purchased at somewhat low price determined by the wholesale price in the same time slot. Under the rule of FIP, because of the imbalance compensation mechanism mentioned above, there is a strong incentive for PV owners to have a battery energy storage (BES) to minimize the imbalance. On the other hand, if the PV owner possesses BES, it can also be used to charge or discharge according to the market price to improve the profit while contributing to the planned power generation. Thus, the strategy how to operate the BES is crucially important from the viewpoint of PV generation business assisted by BES. In this study, we considered an operation of a battery-assisted PV system (BAPV) and examined the five BES operation strategies to compare the expected business profitability based on the actual market price and PV generation considering the bidding process with the forecasted values.

Numerical Simulation Model
Operation flow to participate in the wholesale energy market with BAPV is described below. The first step is to forecast intra-day market price by means of the persistence model which utilizes the past actual values a week ago. At next, on the day of the event, PV power generation forecasting was conducted by the deadline for the bidding (gate closure), which is 60 minutes before the delivery time, and bid to the intra-day market [1] . Finally, at the actual delivery, the BES will be controlled so as to compensate the difference between the bidding value and actual PV generation. It is also assumed that all of the bids will be executed and that the PV power generation will not be subject to output curtailment.

Forecasting of PV power generation
It is generally known that as the forecast lead time of PV power generation becomes shorter, the forecast accuracy becomes higher. Therefore, the intra-day market which has a short time span (60 minutes) between the trading gate closure and actual delivery, was studied in this paper. The LSTM model was adopted to forecast 60 minutes ahead of actual solar irradiance using the actual log data as   input data and convert it into the PV power generation. The LSTM model is a kind of deep learning algorithms dealing with a time series analysis and reported to provide good performance for renewable energy power generation forecasting [2]- [7] . Forecasting of wholesale electricity market price It is necessary to forecast the intra-day wholesale market price to determine the optimal operation scheduling of BES. Therefore, the persistence model, which uses the past values as the forecasted values, was adopted to forecast the day ahead of market price. In this paper, the past values from seven days ago is used as the forecasted values.
Optimal bidding scheduling of BES The daily optimal operation scheduling method of BAPV was proposed in previous studies [8]- [15] . In this paper, the optimal bidding scheduling was determined by solving a mixed integer linear programming problem formulated as follows. The optimization variables are the BES charge and discharge power output and the flag variable, and the time slots are 48 frames per day in 30-minute time increments. Equation (1) indicates the objective function. Equation (2) indicates the equation for the amount of power transmitted to the grid. Equation (3) indicates the maximum and minimum limit constraints of the power transmitted to the grid. Equation (4) indicates the maximum and minimum limit constraints of the flag variable. Equation (5) and (6) indicate the maximum and minimum limit constraints for the BES charge and discharge power output, respectively. Equation (7) indicates the equation for the state of charge (SOC) of BES. Equation (8) indicates the maximum and minimum limit constraints for the SOC of BES. Equation (9) indicates the beginning and end matching constraints (=50%) for the SOC of BES at 00:00 and 24:00. The surplus and deficiency imbalances are calculated from the difference between the actual PV generation values and schedule values submitted to the market in each time frame, and the BES are controlled to compensate for the surplus and deficiency imbalance within the maximum and minimum limit constraints of the rated output and SOC. The BES is assumed to be able to control charge and discharge power output in a time cycle on the order of milliseconds, and there is no control delay at the one-minute time granularity handled in this study. In addition, SOC adjustment is not performed.

Numerical simulation conditions
The equipment configuration includes one each of PV and BES. Table  1 summarizes the study cases covered in this paper. The PV rated output is 1,000 kW. The PV power generation data was converted from the solar irradiance data measured by an irradiator installed at the EMS Demonstration Center of Waseda University in Tokyo, Japan. The LSTM model was used to forecast PV power generation. The BES was examined for each case where the rated output was changed from 0 to 1,000 kW in 100 kW increments and the rated capacity was changed from 0 to 1,000 kWh in 100 kWh increments. The one-way charge/discharge efficiency was set to 90%, and the SOC utilization range was set to 20-80%. The initial SOC was set at 50%. For cases 2 and 3, where no imbalance compensation is performed, the BES is controlled according to the contents of the sell/buy bids for the BAPV, while for cases 4 and 5, where imbalance compensation is performed, the BES is controlled so that the imbalance is compensated sequentially, as described above. The evaluation period is one year (from January 1, 2019 to December 31, 2019). The time granularity is 1 minute. As for the market price, we used the average price of the Japan JEPX intra-day market. Two cases of market price forecast for the intra-day market were considered: Persistence model and true value (no prediction error). Annual electricity sales revenue was adopted as the evaluation index and calculated using equation (10). MATLAB was used as the programming language, YALMIP as the optimization modeling tool, and CPLEX as the optimization solver. Where, Rev:yearly total income (YEN/year), Rev A :yearly regular income (YEN/year), Rev B :yearly surplus imbalance income, Rev C :yearly sufficient imbalance penalty (YEN/year). Table 2 shows the accuracy of the 60-minute ahead of solar irradiance forecasts based on the LSTM model. By coarsening the time granularity at which the forecast accuracy is evaluated from 1 minute to 30 minutes, we confirmed that the forecast accuracy is improved by about 18% in MAE due to the offsetting effect of forecast outliers in the vertical direction. The forecasting accuracy of the day ahead of intra-day market price based on the persistence model is shown in Table 3. It is confirmed that the MAE is about 2.1 YEN/kWh. To confirm the typical behavior of the BES control, the dispatch control of BES on a typical day for Cases 2, 4, and 5 are shown in Figures 1, 2, and 3. These are the results when the persistence model is used and BES rated output is 1,000 kW and rated capacity is 1,000 kWh. As can be seen from Fig 1, BES are charged and bid for buy during times when market prices are low, while they are discharged and bid for sale during times when market prices are high, and the difference in market prices is used to maximize business revenue. In addition, as can be seen from Fig 2, the BES do not participate in the bidding process, but are controlled to sequentially compensate for the surplus and deficiency imbalance, which is the difference between the actual and planned values of PV generation. It can be confirmed that the imbalance is compensated within the range that does not deviate from the rated output and SOC maximum and minimum limit constraints of the BES. As can be seen from Fig 3, the BES is controlled to participate in the bidding process and to sequentially compensate for the imbalance, which is the difference between the actual and planned values seen in the combined PV and BES. The business profitability of the PV power generation alone in Case 1 is shown in Table 4, which confirms that the annual electricity sales revenue in 1kW units is 6,270 yen. If we can improve the accuracy of the forecast, it is conceivable that we can further improve the annual electricity sales revenue by reducing the deficiency imbalance settlement. Next, Table 5 shows the business profitability in each case entered with PV and BES. The figures are listed as the range of values when the cost effectiveness is highest and lowest when the rated output and rated capacity of the BES are changed. In the case of the dataset used in this study, the BES was found to be most cost-effective in Case 4 when they did not participate in the bidding process and maximally perform planned power generation that was devoted solely to imbalance compensation. From now on, VRE penetrates more to power system and possibly lead to higher volatility. In the market price as well as elevated imbalance penalty as considered in Japan. Including these circumstances, our proposed study framework is applicable to evaluate various cases.   The research mainly focuses on industrial consumers with a large diversity in processes and consumption profiles allowing for different flexibility implementations. The project develops a digital twin framework based on agent-based modelling to understand and simulate the industrial consumers' market participation. The framework is developed and tested using case studies from the Danish and Chinese electricity markets. This framework includes demand response strategies based on market properties, consumer flexibility properties and consumer preferences. The simulation results from the case studies allow to establish general recommendations for demand response strategies within different market contexts. Therefore, the framework can facilitate industrial consumers' evaluation of their strategies for demand response participation. Keywords: agent-based model; demand response; market participation strategy; industrial consumer, digital twin Motivation Demand response, defined as "a shift of electricity usage in response to market price signals or certain requests" [1], is considered a promising solution to improve the integration of variable renewable energies (VRE) in the energy system [2]. Industrial consumers have the potential to provide market-based demand response due to their characteristic electricity consumption features. In many countries, industrial consumers with large energy consumption are allowed to participate directly in wholesale markets without passing through intermediary market actors such as aggregators or retailers [3]. This allows industrial consumers to keep control over their process scheduling and simplifies regulatory requirements. Additionally, many industrial consumers have been monitoring their production processes in real-time at a fine time resolution [4], which is a key requirement for successful demand response implementation. The profit-driven mindset of industrial consumers also gives a more rational reaction to price variations compared to residential consumers [5], which creates less uncertainty in modelled results. However, power consumption adjustments of industrial processes require deep domain knowledge, due to process requirements such as high timing precision or temporal interdependencies [3]. Industrial consumers can quickly consider the uncertain impact of such consumption adjustments as unwelcome operational risks [6].

Numerical simulation results
Additionally, the lack of clear financial returns from short-term market participation increases consumers' reluctance towards market participation [7]. In [8], a survey on industrial demand response potential underlines the need for tools to demonstrate to all involved actors the economic benefits of implementing the identified flexibility in electricity markets. Many market bidding optimisation strategies are proposed in the literature to prove and improve the financial viability of demand response market participation, such as [9][10][11] . In these studies, which often use mathematical programming, the main objective is to maximise savings or revenues from market participation. The consumers then adapt their consumption based on the optimised schedules. Yet the following work argues that this optimisation approach is not representative of industrial consumers' operational constraints. Indeed, electricity market participation is not the main source of revenue for industrial consumers, and therefore cannot be the main driver of their load schedule. In [12], Lund et al. put optimisation approaches, based on one or few success criteria, in contrast with simulation approaches, which allow to compare scenarios according to several criteria. Lund et al. refer to simulations as being descriptive, or analytical, processes, as opposed to prescriptive optimisation processes [12]. A simulation-based approach varies key parameters in the model and assesses the impacts of these decisions through scenario comparison. To make such simulations as realistic as possible, large amounts of data can be fed into the simulation, in which case it can be referred to as a digital twin.
In this context, this work proposes to develop a digital twin framework to simulate the participation of industrial consumers in electricity markets. The simulation outputs will allow to evaluate the benefits and constraints for industrial consumers to offer flexibility in a given electricity market. Meanwhile, demand response strategies can be recommended to industrial consumers based on their preferences and market specific requirements, which allows more stakeholder-targeted business development, as suggested in [13].
There are many external factors that influence industrial consumers' participation in the demand response, especially market conditions. To investigate how industrial consumers' preferences, benefits and barriers are influenced by the market conditions, this PhD research project selects two electricity markets for testing and comparison: Denmark and China. The Danish and Chinese electricity markets present the typical unbundling and bundling electricity markets. Denmark has been part of the pioneering Nordic electricity market deregulation since the 1990s, which has brought many different submarket participation options to consumers [14]. In Denmark, energy flexibility solutions are promoted due to the large share of VREs in the country's electricity production and due to the decommissioning of traditional fossil-fuel based dispatchable generators [15]. The combination of high VRE share and high market liberalisation therefore provides a strong base for market-based demand response initiatives. In China, the electricity markets are still with centralised price setting and monopolised by state owned utilities [16]. Meanwhile, industrial consumers represent 60% of the country's electricity consumption [17], yet their electricity costs are 50% higher than for US industries [18]. Therefore, industrial consumers could have motivations to participate in demand response to reduce their operation costs. Industrial consumers' market participation in the Danish and Chinese electricity markets is quite different due to the different market rules and their corresponding market participation strategies [14]. For this reason, a digital twin framework that supports the digital representation of electricity markets is beneficial for industrial consumers to understand different electricity market conditions, investigate participation strategies and optimize their benefits based on preferences and production constraints. However, no literature has focused on a digital twin framework for electricity markets, especially not for the investigation of industrial consumers' participation in different electricity markets.

Research question and objectives
This PhD research project aims to contribute to a digital twin platform development of the smart energy ecosystem by the SDU Center for Energy Informatics, specifically developing a digital twin  framework for the energy markets, to facilitate industrial consumers' participation in electricity markets with their energy flexibilities. To fulfil this aim, three research objectives are designed in this PhD research project: The first objective is to investigate industrial consumers' flexibility properties for participation in a selected electricity market. From the consumers' side, different electricity consumption profiles imply that different flexibility properties are investigated. From the electricity market side, market liberalisation has caused several submarkets to emerge [19]. Depending on the services provided to the grid, these submarkets have different participation requirements and price dynamics which are categorized in this research project. The second research objective is to create a model framework to represent the selected electricity market ecosystem for testing different combinations of market participation strategies under various industrial consumers' operational conditions, without redefining the problem every time.
The third objective is to develop a general demand response strategy framework. This framework is expected to be adaptable to different industrial consumer types in different market contexts. In particular, the influence of two different electricity market structures on various demand response strategies are investigated: the Danish and the Chinese electricity market.

Related literature and theories
Agent-based modelling (ABM) is a simulation approach in which independent agents are assigned their own logic and interact in an evolving environment [20]. It is particularly suitable to study the impacts of interaction structures and agents' individual behaviour on market results [21]. ABM, used in this work to set up the simulation, has been extensively used in the literature to study consumer participation in electricity markets [22]. ABM models for electricity markets, such as MASCEM [23], model the impacts of sellers and buyers on the overall market dynamics, and therefore have a marketcentric perspective. Others, such as [24] and [25], take a more consumer-centric approach, and develop market bidding strategies based on the consumer agents' constraints. In [26], the differences in optimisation results between aggregator-side fleet management and individual consumer management in an ABM are shown. The bidding strategies suggested in the ABM models in [24][25][26] still implement the logic of their agent using optimisation programming methods. More heuristic demand response strategies in ABM are implemented in [20,27,28], which model different industrial processes such as water reservoir pumping, greenhouse lighting and fermentation cooling. In these cases, the agent actions are based on received price forecasts and current state of operation, which allows to model decision processes closer to real-time operations. This research aims to contribute to the literature by providing a digital twin framework for evaluating the industrial consumers' demand response participation. The framework takes into account the flexibility properties and behaviour of the consumer, as well as the properties of the specific electricity market analysed, to provide a successful market participation strategy.

Methodology
The development of the industrial consumers' demand response strategy framework is based on a digital twin framework that consists of three parts (shown in Fig. ): Part 1-process-to-market mapping Firstly, a market screening process is done before simulating consumers' participation in a selected electricity market. This requires firstly to map the given industrial consumer's flexibility properties on one side, and the market participation possibilities on the other side. The parameters describing consumer flexibility are obtained by analysing and categorising different industrial processes. In the meantime, the submarkets' properties are identified, both in terms of participation requirements and price dynamics. Data analysis of market prices allows to identify price signals which have the most impact on demand response savings, such as, for example, the price variability within a day. More insights on relevant market signals are given from the simulation results in Part 2. This interdependence between Part 1 and Part 2 presents the importance of the iterative procedure in this market evaluation framework. Once the process-tomarket mapping is done, the analysed industrial process is matched with the most relevant submarkets for market participation based on a decision tree selection process. Part 2-market participation simulation framework Based on the selected markets' properties, the market ecosystem with all relevant actors and their interactions is identified, following the business ecosystem modelling framework outlined in [29]. This ecosystem mapping defines the structure of the agent-based model (ABM) used in this research to simulate industrial consumers' market participation with their flexibility. To develop the model framework, the business ecosystem architecture development presented in [30] is followed, which provides a methodology to configure roles, actors, and interactions in a given business ecosystem. Different submarkets are configured as agents with their own roles that separate the market operators' logic from the bidders' logic. This role-based approach provides more flexibility in the agent-based models for testing different market participation strategies. The market participation strategies are strongly influenced by the submarket's rules and price dynamics, but also by the consumers' operational requirements and preferences. The implemented and evaluated strategies should therefore be adaptive, based on consumers' real-time operational conditions. Examples of consumer preferences can be risk willingness, expectations on savings, or focus on carbon emissions reduction. A more detailed list of influential factors for market participation is given in [31], which will be used for defining agents' behaviour and bidding logic.
To reflect current market conditions, cleared market prices are based on historical data. This implies that industrial consumers in the model make rational decisions, based on the discussion with some companies in the project. Therefore, the PhD research focus differs from other researches by emphasizing on the industrial consumers' benefits from market participation, instead of focusing on the system's benefits from demand response participation. Part 3-demand response strategy framework In part 3, the model framework is further developed, tested, and validated by iterating over different industrial processes (thermal, mechanical, electromagnetic, etc.), different operational requirements (batch, continuous, nightshifts, single process, chain process, etc.) and different market contexts. The first two parts are initially developed in a Danish market context which represents unbundled and internationally interconnected electricity markets. In this part, market participation of these different industrial processes is also tested in the Chinese context. Due to the Chinese government's regional approach to market liberalisation [32], a specific province is selected as a case study. The case study selection is based on several factors, such as the province's demography, geography, economic activity, electricity market liberalisation stage, and data availability. As shown in Figure 1, the research elements of electricity market categorisation (in Part 1), market properties (in Part 2), and market structure comparison (in Part 3) will be adjusted based on the Danish and Chinese electricity market conditions. Furthermore, the correlated elements, e.g., ecosystem mapping will be adjusted accordingly [30]. Meanwhile, industrial consumers' preferences and potential biding strategies in the literature, e.g., [9,33], will be tested in the simulations for both Danish and Chinese electricity markets. The comparison of the simulation results will explain how these factors influence industrial consumers' participation in demand response and financial gains. The developed digital twin framework for the energy markets allows to adjust the agent logic and communications according to the digital twin design. Therefore, the two digital twins of the Danish and Chinese electricity markets will be implemented according to the framework design with specifications adapted to each market. This approach not only ensures the harmonisation of the digital twin design and development, but also comparability between two digital twins.

Preliminary results
The preliminary findings of the PhD research project can be summarised as: Part 1-process-to-market mapping The main result from the first part of the proposed framework is a categorisation process that matches industrial consumers' flexibility potentials in their production processes with the most suitable submarkets for demand response participation. More specifically, on one side, it provides a method for categorising industrial processes based on their flexibility potentials. On the other side, it provides a method for categorising electricity submarkets which shows the participation options for industrial consumers. These two classification methods allow to standardise inputs for a market selection framework, which is the output of Part 1.
A preliminary analysis of Danish industrial processes' suitability for market participation underlines that the energy-restricted flexibility of consumers creates different barriers compared to the capacityrestricted flexibility of generators. In many cases, time-shift capability and response duration can reduce the number of submarket options for industrial consumer participation. From the market side, submarket price variability, activation frequency and liquidity also have a large influence on submarket suitability, but no submarket is consistently better over all these parameters. The choice of relevant submarket therefore also depends on the industrial consumer's bidding preferences. Part 2-market participation simulation framework The main result from the second part of the framework is a systematic approach for simulating and evaluating industrial consumers' market participation. This approach includes 1) a framework composed of five elements: ecosystem mapping, consumer flexibility properties assignment, consumer preferences assignment, market properties assignment and participation strategy selection; 2) multi-agent-based simulations to test and validate the framework. The validated model framework for industrial consumers' market participation evaluation is the output of Part 2. Preliminary results on energy-based markets in the Danish context show that, based on historical market data, significant savings from demand response would only be obtained when bidding in multiple submarkets. For rational agents with a risk-neutral behaviour and no forecast uncertainty, the potential benefits of participation in multiple markets are considerable. In reality, market clearings can deviate from forecasts and participation in more submarkets puts more constraints on the industrial process schedules, which significantly increases the financial and operational risk for industrial consumers. Agents with different risk willingness could be attracted to different participation options. Preliminary results in parts 1 and 2 show that the behavioural aspect of the consumer plays a large role in the market bidding strategy and must be explicitly included in simulations. Future works The model obtained so far will be generalised to develop the demand response strategy framework in part 3. From the industrial process perspective, more processes are first tested in the Danish context, showing the impact of different flexibility types on market participation. Then, from a market perspective, the agent-basedmodel is adapted to the Chinese market to evaluate the impact of market structure on market participation. The comparison of results will then identify the main drivers which influence a demand response market participation strategy.