Identification and classification of heat pump problems in the field and their implication for a user-centric problem recognition

Heat pumps are at the heart of the transition to sustainable heating in buildings. Yet, minor installation and setting errors add to unnoticed performance drops over the system’s lifetime. With the advent of smart meters that constantly measure electricity consumption, data patterns of heat pumps have become available, even for the many not connected to the Internet. These data hold the potential to monitor heat pumps continuously, identify issues, and thus assist energy consultants and heat pump owners in lifting hidden conservation potential. Yet, research and practice lack an overview of specific problems that could help in this task. In a mixed-method approach, this study investigated 228 protocols of on-site heat pump inspections in Switzerland and found 47 problem classes with varying frequencies. Based on this empirical data and expert interviews, a classification scheme for heat pump issues is proposed and validated. It uncovers the cause of problems, how and by whom they can be recognised and solved, and potential benefits. The work demonstrates that (i) several problems are likely to create smart meter patterns and that (ii) heat pump owners could be involved in the problem recognition and solving process if they get guidance (i.e. simple rules and instructions). Finally, this study discusses implications for developing information systems to automate and assist the recognition and solving of problems. Such information systems may raise not only the attention of heat pump owners but also trigger desired actions (i.e. request consultancy, inspect heat pump themselves).


Introduction
About a quarter of the world's final energy consumption is attributed to heating in residential buildings (IEA 2021). Motivated by the Paris Agreement and pushed by geopolitical reasons, replacing traditional gas-and oil-based heating systems is a top priority of policymakers (e.g. GEG 2020; Kanton Zurich 2021;BEIS 2021). For this purpose, heat pumps are one of the most important alternatives, as they can harvest renewable energy sources from the air or ground using electricity.
Triggering actions for improving heat pump efficiency will, in many cases, require it to be externally initiated. This could be possible with monitoring technologies that rely on heat pumps with Internet-of-Things (IoT) connectivity. Although such modules have become more common, they are not available for a large number of already installed heat pumps. Such modules are often used to prevent breakdowns (and unsatisfied customers) but less to improve efficiency. Moreover, overarching standards for this measuring technology are not yet finished (Nowak 2021; Scotton and Nowak 2021).
This study explores a different approach and focuses instead on the existing smart grid technology from the electricity sector. As a central component of the smart grid, smart meters measure the electricity consumption of residential buildings and the heat pump several times an hour and communicate data to grid operations. Smart meters are installed in each building in the EU (EC 2014) and thus will be available also for the large stock of existing heat pumps and those currently on the market regardless of their IoT connectivity.
Therefore, this study is rooted in the two fields of energy informatics and information systems, which both have a strong tradition of putting forth the capabilities of the smart grid to improve the efficiency of energy systems (Goebel et al. 2014;Ketter et al. 2018). Several applications (e.g. load disaggregation, feedback interventions) demonstrated how information could be retrieved and used to conserve electricity in the residential sector and, therefore, confirmed the inherent value of smart meter data that is processed with advanced data analytics methods (Zhou and Yang 2016;Zhang et al. 2018). With a particular focus on heat pumps, several works demonstrated that smart meter data could be used to predict basic characteristics of heat pumps (e.g. Fei et al. 2013;Hopf et al. 2018;Weigert et al. 2020;Brudermueller et al. 2022) and even recognise heat pumps with a high saving potential . This study goes one step further and uses the empirical insights from on-site visits and answer the following research question: RQ2: How can heat pump owners recognise heat pump problems, and how can information systems (i.e. smart-meter-based detection and assisted recognition) support this process?
The answer to this RQ can help develop information systems that automate problem detection and assist heat pump operators and efficiency consultants. Automated approaches that reveal possible problems can be the starting point to enable energyaware behaviour among heat pump owners and thus trigger desired actions such as seeking solutions or professional help. In easy cases, such information could assist users in problem recognition and avoid unnecessary and expensive on-site visits from professionals who are often unavailable due to a shortage of skilled workers (Nowak 2021;Ecoplan 2021;Branford and Roberts 2022;Hilpert 2022). In complex cases, energy consultants could benefit from additional information about possible problems before conducting on-site visits where they manually inspect the system.
Since problem recognition is only the first step in achieving energy efficiency, this study also answers the following research question: RQ3: How can heat pump problems be solved, and how can information systems support this process?
The rest of this article is structured as follows. The "Related work" section introduces heat pump problems and recognition approaches. The "Research approach and data" section describes the data collection process, introduces the investigated sample and presents the mixed-method approach for the analysis. The "Findings" section shows a classification scheme for heat pump problems and presents the results of applying it to the given dataset. Section "Discussion: information systems for heat pump problem recognition and solving" discusses the implications of the findings for developing information systems to identify and lift potentials. "Limitations and future research" names limitations to this study and opportunities for future research. The "Conclusion" section concludes the study.
This study contributes to the scientific body of knowledge by presenting typical heat pump problems and analysing them from three dimensions (problem recognition, solving, and potential benefits). The study gives insights to researchers and practitioners into which heat pump problems are suitable candidates for developing information systems that (i) use the existing smart meter infrastructure for automatic heat pump problem detection, (ii) assist heat pump owners in manually recognising, and (iii) solving problems.

Related work
The following section presents what is known about heat pump performance, problems and their possible causes, approaches to recognise them, and how a classification scheme can help to generate new insights about heat pump problems in the field.

Problems with heat pump installations
Several studies report substantial underperformance of heat pumps in residential buildings in different countries and for different types of heat pumps. Ground source heat pump installations could often not meet the expected performance and were often below national energy standards in China (Qiao et al. 2020;Gao et al. 2021), the US (Puttagunta et al. 2010;Yin et al. 2019), and Europe (Caird et al. 2012). Similar findings were reported for air source heat pumps in Europe (Caird et al. 2012;Chesser et al. 2021;O'Hegarty et al. 2022). A meta-review considering more than 600 European installations revealed that the performance of heat pumps varied widely, indicating that heat pumps are sensitive to poor design, installation, and planning (Gleeson and Lowe 2013). In addition to these problem categories, the literature names hardware and setting problems but also situations that favour the emergency of problems (i.e. inexperience of planners, installers, and users).
Hardware problems were, for example, investigated in fault reports that have been reported to manufacturers (Madani and Roccatello 2014). Common and costly faults were associated with the heat pump and connected components, such as control and electronics, shuttle valves, and domestic hot water tanks. The authors conclude that many issues could be avoided at a low cost if manufacturers increase the quality of components. Other studies simulate the influence of hardware failures and soft errors typically caused by incorrect planning and installation practices or missing maintenance (e.g. Domanski et al. 2014;Winkler et al. 2020;Bellanco et al. 2022). The authors name substantial performance drops caused by air duct or valve leakage, refrigerant over-and undercharge, oversized systems, the existence of non-condensable gases, low airflows, and evaporator fouling. In a series of interrelated studies, such problems were also found in the field (Russ et al. 2010;Miara et al. 2011Miara et al. , 2017Günther et al. 2014Günther et al. , 2020. The studies name, for example, oversized equipment, not correctly-mounted sensors, not-closing three-way valves, and dirty air ducts and evaporators. Moreover, the studies above name various setting problems of heat pumps, such as unnecessary high flow temperatures or summer operation. They found heat circulator pumps in the connected distribution system that run too often and at too high temperatures in buffer and domestic hot water tanks. In a few cases, there was serious overconsumption of electrical backup heaters related to legionella control systems (Günther et al. 2020). Such systems ensure killing bacteria by increasing the temperature of the hot water tank frequently to 60-70 °C (Van Kenhove et al. 2019). If the heat pump must increase the hot water to such high temperatures, some systems reach their performance limit and need support from the backup heater. Other systems prevent legionella in the hot water distribution by avoiding water stagnation (i.e. hot water circulation pumps) or keeping pipes hot (i.e. pipe heating cables). Legionella control systems have in common that they can consume large amounts of energy which makes it necessary to operate them often and at temperatures high enough to prevent bacteria but not too often to avoid energy waste (Nipkow and Lingenhel 2002;EnergieSchweiz 2016).
The above-mentioned problems often go unnoticed and are favoured if planners, installers, and homeowners are inexperienced with heat pumps. It was noted that some planners find it "difficult and time-consuming to calculate the energy efficiency correctly" (p. 6), especially if buildings are not well-insulated (Decuypere et al. 2022). This could lead to installed systems that are oversized or undersized. Installers may also lack the qualifications to install systems and set parameters correctly. For example, it was noted that installers' qualification in the UK often relies on proof of "experience" and less on formal heat pump qualification (Caird et al. 2012;Gleeson 2016). This situation seems to exist even after ten years (Branford and Roberts 2022). For the Belgian market, Decuypere et al. (2022) observed that some installers "struggle with keeping pace with the rapid evolution of this technology" (p. 6). Likewise, the European Heat Pump Association named upskilling of installers and planners a critical factor in ensuring the correct installation and service of heat pumps (Nowak 2021). Problems caused by the inexperience of planners and installers can be, therefore, also to be expected in new installations. Finally, inexperienced homeowners are an important factor for the efficiency of heat pumps. In the sales process, homeowners seem to have "difficulties understanding how a heat pump works, even after the installer's explanation" (Decuypere et al. 2022, p. 6). Miara et al. (2017) note that some heat pump operators in their study lacked the knowledge of their system while others were willing to "improve the heat pump's efficiency actively" (p. 7). Users with a greater understanding also had higher system efficiencies than those with less understanding (Caird et al. 2012;Qiao et al. 2020;Gao et al. 2021).
Problems related to heat pumps can occur in the heat pump itself and in different components and settings that form the heating system (e.g. the heat pump, additional heaters, the heat distribution, and the hot water system). While hardware problems and their possible influence on the heat pump performance were the focus of several works, only a few studies consider also setting errors. Such setting problems have the potential to be solvable cheaply and easily, given that they typically do not require expensive spare parts. Moreover, setting errors may be beneficial to increase the system performance and ensure residents' health, which has not been considered in studies so far. Since the inexperience of installers can lead to several problems, it is also of high interest when problems typically occur. Finally, the influence and interest of users on the performance were noted. However, less is known about which problems they actually can influence without external experts. Thus, this study provides a comprehensive overview of heat pump problems and related components with a special focus on settings, investigates when problems typically occur, where they originate, how they can be solved, and the potential of heat pump owners in problem recognition and solving.

Recognition of heat pump problems
There are two basic approaches to identify the above-mentioned unnoticed flaws: manual recognition and automated approaches. Manual recognition lies in humans' hands and consists of on-site manual inspection. Installers or energy consultants typically do this after the installation or upon request if the homeowner is aware of possible problems. However, this is, as described above, often not the case and makes automated approaches attractive that are realised with information systems and monitor a heat pump continuously. Such fault detection and diagnostics (FDD) systems hold the potential to trigger efficiency actions of homeowners, installers, and energy consultants. Both recognition approaches process data (e.g. sensor data, visual data) to derive information about the state of a component for having a basis for decision-making and further actions (e.g. exchange components, set parameter to a specific value, call expert). Thus, it is necessary to measure data that is somehow related to the cause of a problem and to have a model or rules which are either based on a priori knowledge (i.e. knowledgebased) or derived empirically (i.e. data-driven) (Katipamula and Brambley 2005). Recent research activities in the FDD field preferred empirically driven methods in which black box models such as classification are often used (Kim and Katipamula 2018;Zhao et al. 2019). Data-driven methods use sensor data to learn rules about an error or fault. They are often built without a deep understanding of the causal relationships among faults, and humans cannot understand them well (Zhao et al. 2019). However, data-driven methods often outperform knowledge-driven-based methods in terms of accuracy and even work with fewer sensors (Zhao et al. 2019). Similar was recently reported in the energy informatics field, where data-driven methods were superior to engineering-based methods in terms of accuracy and even compensated for difficulties in data collection (Wederhake et al. 2022).
Still, for the development of data-driven models, it is necessary to have enough ground truth data on "good" and "bad" states (i.e. fault existent or not) available together with data that somehow contain patterns (i.e. sensor data) related to these states. Collecting both types of data is challenging, as described below.

Labelled data
Ground truth data (i.e. labels) on faults can be either collected from maintenance or service reports in the field, generated in a laboratory environment, or simulated with fault modelling (Li and O'Neill 2018). The latter two are often not practical: Fault models for non-traditional HVAC systems such as ground source heat pumps are rare (Li and O'Neill 2018), and models based on simulated data showed low transferability when applied in practice (Bode et al. 2020). Thus, collecting labels in the field is a useful, even though a costly alternative, to including real operational conditions that "span across different vendors, systems, or physical elements" (Li and O'Neill 2018, p. 954). This study will show the outcome of collecting labels in the field and provide a comprehensive overview of problems and their frequency typically found in residential heat pumps.

Sensor data
The FDD literature of the last decades names the cost associated with measuring, collecting, and storing sensor data as an important challenge (e.g. Nowotny 1985; Katipamula and Brambley 2005;Kim and Katipamula 2018;Zhao et al. 2019). Such sensors measure, for example, pressures, temperatures, and the power of different components and often contain patterns related to faults. From a practical perspective, every sensor adds additional cost to the system. Limiting the number of sensors is crucial, given that building owners are sensitive to the initial costs (Zhao et al. 2019). Even though manufacturers integrate sensors today more frequent in residential heat pumps, they have just started to set the technological basis for connectivity and data storage. IoT connectivity is today proprietary, depends on a specific manufacturer, and must often be ordered extra by the customer if available at all. Industry and policymakers are still developing overarching standards for smart appliances and building automation and control systems (Nowak 2021; Scotton and Nowak 2021). Thus, these to-be-defined technical functionalities and standards will be only available for future heat pumps but not for the large number of systems that are already or will be installed by then.
From a theoretical perspective, there are different possibilities to reduce the number of required sensors. Bellanco et al. (2021) suggested using virtual sensors that are "based on low-cost direct measurements to characterize equipment operation and estimate virtual measurements indirectly" (p. 8). The basic idea is to use only a few sensors that co-measure faults. A commonly used sensor is the electrical power consumption of the compressor (Bellanco et al. 2021). This sensor provides important features to detect several faults (Bellanco et al. 2022). In a sophisticated form, one can use only a single coarse sensor: a so-called smart meter. This sensor measures the overall consumption of a household, typically several times an hour. Using the overall power consumption rather than only the consumption of a compressor also has the advantage of encompassing further electricity consumers related to the heating system (e.g. heat circulator pumps, electrical backup heater) that influence the overall performance.
While the above-mentioned articles are connected to the HVAC and refrigeration community, the use of smart meters in a smart grid to putting forth the efficiency of energy systems has a strong tradition in the fields of energy informatics and information systems (Goebel et al. 2014;Ketter et al. 2018). This measurement technology is standardized, mandated to be rolled out in many places in the EU (EC 2014), and will be available to measure all heat pumps-even the old ones. Most applications that use smart meters in the context of heat pumps focus on their role in power grids, the integration of renewable energy sources, and operation under variable electricity prices (Fischer and Madani 2017). However, several applications (e.g. load disaggregation, feedback interventions) demonstrated how information could be retrieved and used to conserve electricity in the residential sector and, therefore, confirmed the inherent value of smart meter data that is processed with advanced data analytics methods (Zhou and Yang 2016;Zhang et al. 2018). Several works use smart meter data with different resolutions in the context of heat pump efficiency. Load disaggregation techniques such as non-intrusive-load monitoring approaches were proposed based on high resolutions to detect faulty heat pump states (Armstrong et al. 2006;Sawyer et al. 2009). At lower resolutions (i.e. four measurements the hour), smart meter data were used to predict basic characteristics of heat pumps, such as their existence (Fei et al. 2013), their heat reservoir and age (Weigert et al. 2020), their modulation type (Brudermueller et al. 2022), and their primary purpose (i.e. space or water heating) (Hopf et al. 2018). It was also shown that smart meter data could improve the efficacy of energy conservation campaigns if used to pre-select heat pump installations .
Thus, to cope with the practical challenge that only a few connected systems provide sensor data and in line with the research that aims to reduce the number of required sensors, this study investigates if typical heat pump problems create patterns in coarsegrained smart meter data.

Research approach and data
This study uses a mixed-methods approach to reveal typical problems of heat pumps in the field and to answer how problems can be recognised and solved. The empirical data studied are heat pump inspection reports and expert interviews. This section describes how the qualitative data was gathered and analysed to extract heat pump problems, build a classification scheme, and classify heat pump problems.

Heat pump inspection reports
The heat pump inspection reports stem from a Swiss utility that provides electricity and offers their customers efficiency consulting services for heat pumps. The utility was also interested in using its smart meter infrastructure to identify heat pump problems. The service included a full on-site investigation of the heat pump and directly connected components such as the heat distribution and hot water system. Within a 90-min visit, the consultants checked all components for efficiency and conducted simple-to-solve measures to improve efficiency immediately (e.g. settings on the heating control unit or other components). The experts also recommended further actions that may include spare parts or changes to the system (e.g. insulating pipes, exchange heat circulator pump) but did not carry these out. Furthermore, they trained heat pump users.
The consultants documented the conducted steps in a questionnaire-like report and handed them out to the client. The report has four main sections and describes (i) building characteristics, (ii) characteristics of the existing heat pump, hot water, and heat distribution system, (iii) an audit of the installation, its efficiency and meaningfulness of settings, and (iv) general hints on the effects of typical settings on the system efficiency. The audit section (iii) documents identified problems for different components and were the focus of this study (see Figs. 8,9,and 10 in appendix A for an excerpt).
The sample of heat pump reports consists of 228 reports from 2015 to 2021 and documents heat pump inspections in residential buildings, see Table 1. Most inspections cover single-family houses (91%) and a few apartment houses. About half of the investigated systems are air source heat pumps (54%), while the remaining are ground source heat pumps. Most air source heat pumps were indoor units (65%), followed by outdoor units (20%) and split units (15%). Most ground source heat pumps are vertical systems (92%), and the remaining horizontal or water-based systems. The average inspection occurred in a building with 3.1 residents, with 243 m 2 of heated space area available and a heat pump with a capacity of 12.8 kW.

Expert interviews
The utility also provided the expertise of two heat pump experts with more than 10 years of experience in energy efficiency consulting, which resulted in five expert interviews with more than ten hours of material. The interviews included an open part and semistructured questions. Before the interviews, the author accompanied two on-site heat pump inspections to include empirical observations. Appendix B provides the interview guide used in the expert interviews and a list of the conducted interviews (Table 6).

Mixed-methods approach
Figure 1 shows how the empirical data was used in a three-step mixed-method analysis to generate, improve, and validate four constructs: a list with problem classes, a classification scheme, a list with classified problem classes, and descriptive results.
In the first step, typical problems were extracted from the reports. While the reports contain several problems in the form of structured elements (i.e. checkboxes), various problems were hidden in unstructured text fields (i.e. qualitative data). Common text analysis approaches that allow to extract patterns and categories (i.e. problem classes) from qualitative data are the thematic analysis (Kuckartz and McWhertor 2014) and content analysis (Krippendorff 2018). The latter not only allows to be applied on any material that "may be considered as text" (Krippendorff 2018, pp. 24-25) such as the heat pump inspection reports but also allows to quantify data (i.e. measuring the frequency of problem classes) (Vaismoradi et al. 2013). This feature is essential for this study which aims to recommend which problem classes might be valuable candidates for recognition using automatic approaches. In this respect, deriving enough labels for machine learning is crucial (see related work above) and has been described as a practice similar to conducting a content analysis (Geiger et al. 2020). Hence, this study applied content analysis to extract problems. For this purpose, the problem audit part of the reports was In the second step, the author generated a nascent version of the classification scheme for heat pump problems building up on the list with problem classes and informed by the literature. Both constructs were improved and validated during the open part of the expert interviews. The consultants were encouraged to sharpen each problem class, resulting in new or higher-order problem classes. The classification scheme was improved by including empirical observations. For example, during the accompanied inspections, it was observed that the consultants not only aimed to improve energy efficiency but also settings to avoid legionella. Insights from the interviews led, for example, to the separation of the actors (i.e. hardware installers and technicians) able to identify or solve problems depending on different levels of skills. In the semi-structured interview part, the experts were asked to apply the classification scheme to the problems found. The problem list ordered the topics, and the classification scheme served as a semi-structured questionnaire. This allowed the experts to classify problems in a multiple-choice style but left room for open answers (Ciesielska and Jemielniak 2018). Finally, a quantitative descriptive analysis examined the final list of classified problem classes.

Findings
This section gives an overview of the extracted problem classes, presents the classification scheme, and shows the results of applying the classification for the given dataset.

Problem list and classification scheme
The content analysis of the 228 investigated reports revealed 47 problem classes, see Table 2, that occurred 968 times (problem incidents). Each problem class is referenced in the remaining article using a unique identifier 1 which allows to locate them in the complete list of problem classes in Table 7 in appendix C.
The analysis revealed two types of problem classes, general problems, and underlying problems. Experts find general problems at the beginning of the inspection, in which they pre-evaluate the expected with the actual consumption of the building (given factors such as the building size, insulation standard), the sizing of the heat pump, and the customer's experience with the heating system. Three general problems were noted: In 19% of the cases, the consultants found that heat pumps required too much energy (#G2). In 5%, the capacity was clearly too small or too big (#G3), and in more than 40%, users had barely any knowledge of the heating control system and needed training (#G1). At least one of these general problems occurred in 57% of the inspections. Hence, general problems motivate further inspection activities, in which the second type, underlying problems, are revealed that require checks on individual components. Underlying problems can be recognised at a specific component, solved with a specific measure, and lead to potential benefits. The consultants reported 822 underlying problems that can be assigned to 44 problem classes. On average, consultants found 3.6 underlying problems per investigated installation. Underlying problems are the focus of this study since they allow to derive actionable efficiency measures. The following paragraphs introduce the classification scheme ( Fig. 2) which is supposed to shed light on the underlying problems. It allows to classify heat pump problems according to ten questions in the three dimensions, problem recognition, problem solving, and potential benefits.
The first dimension, problem recognition, starts with investigating the causal component (1) of problems. Problems can occur in heat producers such as the heat pump (air source, ground source) and additional heaters (i.e. the electric backup heater). The produced heat flows through the heat distribution system driven by the heat circulator pump within a closed heating circuit through pipes, thermostatic radiator valves, and a place of heat extraction (i.e. underfloor heating, radiators). Pressure gauges (i.e. manometer) and the expansion valve keep the pressure at a favourable level, while the heat storage allows a delayed use of heat energy. Another system that heat producers often feed is the hot water system. It consists of hot water storage and components that prevent (i.e. circulation pump, pipe heating) and kills (i.e. disinfection system) legionella. The heating control system is the last component-and the entire system's "brain". It uses measured information to schedule heat producers and connected components. This system often has a small screen, modern systems even an interface for external access, and can be programmed and parameters adjusted.  The temporal incidence (2) takes on the fact that some problems likely have been existing since the installation, while others occur slowly over time or abruptly after changes to the system. Knowledge of the temporal incidence can help, for example, to better understand if an after-installation check is necessary. However, even a perfect installation will not be problem-free over the system's lifetime. It also answers which problems benefit from continuous monitoring.
Components for recognition (3) describe the main component that allows problem detection. In many cases, problems can be identified in the causing component. For example, pipes that are not insulated are at the same time the cause of the problem, and a quick look at the pipes also allows problem recognition. Other problems are recognisably based on sensor data in the heating control system.
Information on components for recognition is also essential for developing information systems that guide human actors and make them problem recognisers (4). Different actors are possible: There are hardware installers (with a solid understanding of heat pumps), technicians from the manufacturer (with specialised knowledge), and energy consultants (with a solid experience of heat pumps, which are independent of the hardware installer or manufacturer). These three roles are all external experts that must be mandated first by the heat pump owner. Thus, this study focuses on the guided user who typically owns a heat pump.
Some problems might be not only recognisable by humans (i.e. by visual inspection of components) but also by automated approaches that monitor the system based on smart electricity meter data. For example, an overuse of the electrical backup heater might be recognisable with the help of analytical systems. To reveal the opportunities of such automated approaches, the consultants assessed if a problem is probably recognisable or probably not recognisable based on smart meter data (5).
The second dimension, problem solving, examines primary solution approaches (6) which refer to settings on the heating regulation, installations of spare parts/service or if there is considerable effort needed. Knowledge of the solution approach also sheds light on typical problem solvers (7). The actors involved are similar to problem recognisers (fourth question). The last topic in this dimension is the implementation costs (8). The consultants gave a rough estimate of the time needed to solve the problem, including the cost of spare parts, which allows for separating cheap from expensive-to-solve tasks.
The third dimension covers the potential benefits of solving problems in the investigated heat pump installations. Such lie in the energy consumption (9), which can decrease, not change, or increase and in the health of residents (10), which may be positively affected or not affected. Table 3 shows the frequency of the answers to questions in the dimension problem recognition. The following paragraphs refer primarily to the share of problem incidents.

Causal component
Understanding the cause of a problem can support experts in thinking through how typical problems can be identified and what components are vulnerable to potential problems. Figure 3 shows the frequency of the problem incidents per main and sub-component.
The most frequent component is the heating control system (35%), which is prone to incorrect or non-optimal settings of the heating curve (#HC3-4), the heat limit (#HC5), or the night setback (#HC6). In some cases, the interface for retrieving and manipulating such settings was not accessible (#HC1-2). Issues caused by hot water production (31%) are often related to settings that influence the water quality, for example, in the hot water tank, the water circulation, and the disinfection (#HW1-5, #HW8, #HW11). Moreover, operation times were not scheduled to low-cost tariff times (#HW12) or ran more often than needed to ensure high water quality (#HW6-7, #HW9). The heat distribution causes 19% of the reported problems covering inefficient (#HD8) or incorrectly set speed of heat circulator pumps (#HD9). Other problems are broken expansion tanks (#HD3) and thermostatic valves (#HD10), or wrong preset pressure (#HD1-2). Pipes might not be fully insulated or have a leakage (#HD11, #HD7). In a few cases, the heating water pressure was wrong (#HD4-5), or the pressure meter was broken or misadjusted (#HD6).
Although heat pumps cause 30% of the problem classes, problems rarely occur (12%). Most incidents involve refrigerants for which refill is prohibited (#HP8), refrigerants that must be reported to authorities (#HP9) or in which the leakage check is overdue (#HP10). Few systems also showed profound fault messages (#HP6) or had incorrectly mounted sensors (#HP7). Air source heat pumps can have partly blocked air ducts (#HP2) or too close distances (#HP5). Cases of iced-up heat pumps were rare (#HP4). Outside mounted air source heat pumps are often dirty (#HP3), sometimes with consequences on the evaporator (#HP1). The brine used in ground source heat pumps can have a non-optimal pressure (#HP12), a too low temperature (#HP13), or might not be frost-proof (#HP11). Finally, additional heaters that support the heating system in a bivalent mode can cause problems, although rarely found (3%). Still, their unintended use (#AH1-2) may lead to substantial energy use and high costs.

Temporal incidence
Knowledge about problem causes also helps to answer when problems typically occur. In 68% of the cases, problems likely have been existing since installation (Table 3). This category contains, for example, suboptimal settings on the heating control system (#HC2-6) and the hot water production (#HW2-7, #HW9, #HW11-12), which can cause energy waste. In addition, the consultants found incorrect settings in the heat distribution system, such as a wrong preset pressure in the expansion tank (#HD1-2, #HD9, #HD11). In several cases, they found hardware installation flaws, such as not fully insulated pipes (#HD11) and incorrectly mounted temperature sensors (#HP7). In a few instances, companies forgot to register refrigerants to the authorities (#HP9). Even if a system is well-planned, correctly mounted and has optimal settings, some problems can occur over time (28%). For example, the hot water storage calcifies over several years (#HW1), the water pressure decreases to a point where it is too low (#HD4), and heat pump components get dirty (#HP1, #HP3). Moreover, components are ageing materially and technologically. For example, thermostatic valves or rubber in the expansion tanks eventually become solid and break (#HD3, #HD10). Even well-functioning components grow old technologically, and authorities consider heat circulation pumps inefficient (#HW10) or assess refrigerants differently regarding ecological compatibility and forbid their refill (#HP8).
Finally, a few problems (4%) occur abruptly or after system changes. Examples are a heating circuit leakage (#HD7) and major faults on the heat pump (#HP6, #HC1), which may activate the emergency heating mode (#AH1). Additionally, system changes can cause problems. For example, the refill of heat water may result in a pressure that is too high and air within the system (#HD5).

Components for recognition
Based on the previous sections, which described where and when problems could occur, this section focuses on components that help in the recognition process. Retrieving information about system and error states is, for most problems found in this study (44%), possible using the interface of the heating control system (Fig. 4). The interface shows, for example, settings of the heat regulation (#HC3-6), additional heaters (#AH1-2), and often also the hot water production (#HW5-6, #HW12). It also shows errors such as too low brine temperatures or similar (#HP6, #HP13). The presentation of such information depends on technological maturity: First-generation systems signal errors with a warning light on the heat pump, while modern systems store errors into memory and push them to a smartphone application or even to the manufacturer. As the heating control system provide unique functions and allows to retrieve and manipulate different kinds of information, it is important to design those interfaces in an understandable and accessible way for all actors. However, the consultants reported that this is not always the case: Some settings are tricky to retrieve and hidden behind an expert interface that requires passwords the owner does not know (#HC2). Consultants reported that they maintain a database with manuals and default passwords, which helps sometimes.
Another 21% of the problems can be detected in the heat distribution system. Quick visual inspections of sub-components and simple tests are often sufficient. For example, the pointer of a pressure gauge (i.e. manometer) shows abnormal states via a red zone (#HP12), and gaps on otherwise insulated pipes are clearly visible (#HD11). A simple knock-test allows for identifying a faulty expansion tank (#HD3). Other problems require additional information for evaluation. For example, the correct preset pressure of the expansion tank (#HD1-2) must be calculated based on the highest point within the system. Similar appears to the optimal speed of the heat circulator pump (#HD9).
About 19% of the problems can be recognised in the hot water system. For example, a calcified hot water storage (#HW1) can be recognised via visual inspection or estimated by the timespan since the last descaling, which is often noted on the storage. In addition, the thermometer in the tank allows retrieving the temperature, and thus with knowledge of recommended temperature ranges, also the evaluation of wrong temperatures (#HW2-3). About 9% of the problems can be recognised in the heat pump, such as incorrectly mounted sensors (#HP7). Moreover, some homeowners carelessly place garden furniture on the air ducts or put a shrub or compost close enough to outside mounted heat pumps, leading to significant performance drops (#HP2).

Humans as problem recogniser
External experts (i.e. hardware installers, technicians, consultants) are necessary to recognise about 55% of the problems. Expert knowledge seems necessary to retrieve and evaluate settings in the heating control system (#HC1-6, #AH1-2) and if context information is needed. This applies, for example, for several heat pump problems (#HP1, #HP7, #HP9, #HP11), if one needs to specify the right pressure in the expansion tank (#HD1-2), or in cases with incorrect settings related to the hot water production (#HW3, #HW5-7, #HW10-12).
According to the energy consultants, homeowners can find several simple problems requiring little contextual knowledge if they are provided with simple rules. This applies, for example, to calcified hot water storage (#HW1), too high temperatures (#HW2) and Components and sub-components that can be used to identify problems other problems in the hot water system (#HW8-9). Guided users can identify faulty states visible on the heat pump, such as blocked air ducts or heat pumps that ice up (#HP2-5, #HP8, #HP10), and such visible in the heat distribution, like inefficient heat circulator pumps, not insulated pipes, and broken or pressure gauges in the red (#HD4-6, #HD8-11). Other problems create perceptible noises. Guided users can, for example, perceive air within the heat distribution system from a gurgling noise (#HD5) or conduct a simple knock-test and listen to a dull noise to detect a faulty expansion tank (#HD3). Overall, guided heat pump owners should be able to recognise problems in 45% of the problem incidents (Table 3).
Although some problems require experts to check settings, guided users might be able to collect proxy information that is equally informative for problem recognition. In case of recognising too high heating curves, the consultants referred to the room temperature they percept (#HC3): "on-site, while you arrive, you already feel that it's too warm […] The colder the outside temperature, the warmer the room temperature. […] This would be a typical sign that the heating curve is too high" (I4). Hence, even in the case of complex problems, it seems possible to derive simple rules that allow involving guided users in the problem recognition process.

Smart meter patterns
Besides manual problem recognition based on human senses and expertise, thirteen different problems (44% of the cases, see Table 3) are likely to create patterns in smart meter data, making automated recognition possible. Patterns may occur in the heat regulation, the heat source, electrical backup heating, and hot water production. However, their identification in typical smart meter data that mix the consumption of several devices requires context information, which is described in the following.
Typically, a heat pump operates about 2000 h per year, about 5.5 h per day. However, operation hours vary within a year, even within a day, creating helpful patterns. For example, winter consumption is greater since heat pumps primarily operate in the heating period and only for hot water production in the summer. One consultant mentioned that such differences in the transition between the heating and non-heating period (vice versa) create patterns related to a wrong heat limit (#HC5) "because the heating system switches on earlier [in the autumn]. One would perhaps still have to use climate data. […] if you notice that the heating is still on in June, then something is wrong" (I4). Daily differences, by contrast, are helpful to detect an enabled night setback (#HC6) "because it runs less at night and more during the day" (I4). One consultant explained that this setting could lead to a strong pattern every morning since "you kind of cool down the house at night, and you have to catch up with it during the day […] to achieve this [temperature] again during the day, you must automatically set the heating curve higher" (I6). This effect is intensified for low-dimensioned heat pumps since a strong ramp-up in the morning must be supported by an electrical backup heater resulting in further patterns.
Furthermore, the consultants referred to the continuous operation of heat circulator pumps (#HD8) and pipe heating cables (#HW7) which can increase the base load curve considerably since "each metre has about 7-10 watts. […] If there are 20 m, you might already see it" (I4).
Three heat pump problems related to the heat source (#HP2, #HP5, #HP13) likely create smart meter patterns. In the case of blocked or too close mounted air ducts, one consultant mentioned that "the increase in energy is greater compared to a normal system. Because the longer the system runs, the faster the air cools down due to the air shortcircuit. And in principle, the supplementary heating or the emergency heating is probably also switched on then. If the air gets below minus 8 degrees [Celsius], the power is no longer sufficient, and then the system is reheated electrically. For me, by the way, that's a classic you could recognise" (I4).
Both consultants expect that electrical heaters for backup and disinfection purposes create strong smart meter patterns (#AH1-2). They argue that the large power of 6-9 kW and their timing help to locate them in smart meter data: "If it suddenly runs, you can see that […]. For me, that would be the classic case of sending a message [to the customer]. Look, there's something wrong. Go to the basement" (I5). If problems with resistance heaters exist, they are likely triggered frequently; as one consultant mentioned, "In the worst case, it happens every night" (I6). Moreover, electrical heaters typically only operate outside of the gridlock times, which are supposed to protect the grid in times of heavy usage; as one consultant explains: "The electrical heater for legionella control […] is only released by the grid at midnight, twelve or one o'clock until 6 o'clock in the morning. […] legionella control can only be done at night" (I6). Gridlock times are often mandatory for specific power ranges, depending on the local grid operator and the electricity tariff. In the case of the investigated utility, gridlock also applied for "heat pumps with a power rating of 4 kilowatts or more must also be locked, i.e. at lunchtime when you cook or bake a lot so that the heating doesn't start as well" (I6). Hence, information about local gridlock times helps identify patterns related to unintended heating (#AH1-2) or frequent disinfection (#HW6). Conversely, missing patterns might indicate non-existent (#HW4) or disabled disinfection programs (#HW5).
Finally, knowledge about specific tariffs is also helpful to optimise cost in case of tariffs with variable prices, i.e. to avoid non-aligned hot water production scheduling (#HW12). This cost factor is important since heat pumps run for "hot water production about 300 h a year for four people" (I7). One consultant explained, "Let's say the electric heater is enabled at 11 p.m. And the heat pump is programmed to make the hot water from 4 a.m., which would be nonsense because then the electric heater would heat up at 11 p.m. When the heat pump is enabled, it does nothing because the electric heater has already done it and then you have lost efficiency again" (I5).

Problem solving
This section focuses on typical solutions for problem solving, the actors who can solve the problems, and the associated implementation costs. Table 4 shows the frequency of the answers to questions in the dimension problem solving. Figure 5 shows all questions of this dimension and problem classes.

Solution approach
The right solution approach for each problem depends very much on individual installations (see appendix Table 7 for typical solutions in practice). However, two general solution approaches are often feasible. About half of the problems (Table 4) can be  Fig. 5 The matrix shows the dimension problem solving according to the solution approach and cost in CHF. Colours indicate the problem solver. Bubble sizes indicate the relative frequency of problems occurrence. Bubbles are ordered by cost. The solution approach "considerable effort needed" was removed from this Figure. solved with service tasks, including spare parts. It involves typical maintenance tasks such as cleaning or descaling components (#HP1, #HP3, #HW1), checking and refilling refrigerants (#HP8-10), brine (#HP11-12), or heat water (#HD4-5). Furthermore, it encompasses the installation replacement of broken, inefficient, or incorrectly mounted components (#HD3, #HD7-8, #HD10, #HP7). Finally, this category embodies retrofitting disinfection function (#HW4) or automatic timers that allow better scheduling (#HW7, #HW9). The other half of the problems could be solved by changing heating control settings or such related to the heat distribution and hot water system. Important parameters are often related to the heating curve (#HC3-4), the heat limit (#HC5), the night setback (#HC6), and hot water production settings (#HW1-10). The consultants found as well overambitious settings of disinfection programs and circulation that operates too often (#HD9, #HW11) and such not carried out frequently enough (#HW5, #HW8). The optimal settings depend on several contextual factors such as the building (e.g. size, insulation), the heating system (e.g. underfloor heating, radiators or both used), the hot water system (i.e. storage size) and the residents (e.g. showering behaviour, age of occupants).
Knowledge about such contextual factors is not only essential to aim for optimal settings but to achieve acceptance from residents who could perceive the noise level of air source heat pumps or optimal room temperature differently. The consultants explained this for the optimisation procedure of heating curves where they reduce the curve together with the user, which can initially lead to too low temperatures. However, the consultants also explain to the user how to gradually increase parameters where both efficiency and comfort are satisfied. This procedure is repeated over multiple days since underfloor heating just slowly responds to new settings. Therefore, consultants must anticipate the occupants' impatience about temporarily colder room temperatures since frustration can lead to a reset of the curve close to the original one, i.e. little optimisation. The consultants reported three strategies to cope with this optimisation-/acceptance-dilemma: First, they start with settings causing rather low temperatures in the beginning-what can be positively influenced by the user-instead of starting with high temperatures that must be reduced. Second, they choose initial settings dependent on the willingness of the user to improve efficiency. Third, they recommend starting with parameter adjustment in large steps, which resolves uncomfortable room temperatures quickly and proceed with smaller steps.
Finally, only a tiny fraction of problems (1%) needs considerable effort to be solved. This category applies to inside-mounted heat pumps with air ducts (air intake and exhaust) mounted to close, leading to short circuit flow (#HP5). Often, significant construction measures are necessary to solve such situations.

Problem solver
The consultants named four actors able to solve problems: energy consultants, hardware installers, technicians, and guided users, all with different skills and access to spare parts. About 8% of the problems need extensive expertise from energy consultants, while 43% of the cases need hardware installers. Tasks with high complexity are assigned to technicians who solve a variety of problem classes (about 20%) which do not occur very often (7%). Since experts are rare, the potential of the guided user is of special interest.
The consultants expect that guided users (i.e. heat pump owners) could solve 42% of the problems. Figure 5 shows that the main solution approach is clearly connected to the different actors: Energy consultants and guided users can adjust most setting tasks (i.e. on the heating control system). Guided users can decrease the temperature of the heat storage (#HW2), the speed of the heat circulator pump (#HD9) and enable an existing pipe heating function (#HW8), and, most importantly, set parameters in the heating control system (#HC3-6). Of course, this presupposes that users know meaningful parameters that require external input from energy consultants or information systems. For example, one consultant noted that users need instructions to set the speed of the heat circulator pump (#HD9), which depends on the device and must be known upfront. Additionally, guided users can solve some types of service tasks, for example, refill or drain heat water, deflate the pipes (#HD4-5), and unblock air ducts (#HP2).

Implementation costs
This subsection presents coarse cost estimates for each found problem and calculates the overall cost for each installation if all found problems are solved. For this purpose, the consultants estimated the time required for solving and spare parts costs. The time was multiplied by typical Swiss hourly rates for the different experts: 150 CHF/h (160 USD/h) for the consultant and the technician; 80 CHF/h (85 USD/h) for the hardware installer; Labour and spare parts costs were added to get the total estimate for each problem class. Figure 5 shows the cost of solving each problem class and relates them to the main solution approach. About 57% of the problem incidents can be solved with an investment of less than CHF 50. These problems mainly involve parameter optimisation (#HC3-6, #HD9, #HW2-3, #HW5-6, #HW8, #HW11-12, #AH2) and many problems could be alternatively solved by guided users (#HC3-6, #HW2, #HW8, #HD9). Several service tasks (11%) cost below CHF 100 and are neither time-consuming nor do they require expensive spare parts. Such tasks involve, for example, installing a simple timer that schedules pipe heating (#HW7) or a circulation pump (#HW9). Even though most service tasks require experts, guided users can solve some tasks independently (#HD4-5, #HP2, #HC2). About 7% of the found problems cost between CHF 100 and CHF 499. Such problems involve, for example, repairs of faulty expansion tanks (#HD3) and maintenance tasks like cleaning evaporators (#HP1). Finally, 25% of the problems require a significant investment of more than CHF 500. These are, for example, costly maintenance tasks such as descaling hot water storage (#HW1) and retrofitting system components or refrigerants (#HW4, #HD8, #HP8). Summing up, most optimisation tasks are cheap, while costs associated with service tasks (including spare parts) vary greatly.
Next, the estimates are used to calculate the total costs for each installation, see the boxplot in Fig. 6, assuming that heat pump owners let all problems be solved. The median costs and thus half of the investigated installations require less than CHF 686 (USD 731). The other half had more expensive tasks requiring significant labour and expensive spare parts. However, the overall cost may be lower in practice since not all found problems are urgent and must be solved immediately. For example, the retrofit of a disinfection function for hot water (#HW4)-even though recommended in many cases-is not always urgently required if the heat pump can produce high output temperatures. On the other hand, the heat pump owner may not pay for all measures since some tasks related to major problems or installation errors (#AH1, #HP4-7, #HP9, #HC1) may fall within the warranty.

Potential benefits
Potential benefits of solving problems are energy conservation and ensuring the health of residents (see Table 5 for the frequency of the answers to both questions). Both topics are closely linked and must be weighed against each other, as explained below.
Typically, heat pump conservation potential is given if the temperature difference between the heat source and the output temperature is considerably large. This is given if settings result in higher output temperatures than necessary to ensure comfort and if the heat source temperature is low or cannot be transferred right. The former was often found as a consequence of incorrect or non-optimal settings of the heating curve, the  heat limit, or the night setback (#HC3, #HC5-6). Heat transfer problems for air source heat pumps were found as consequences of blocked air ducts (#HP2), too close distance between intake and exhaust (#HP5), dirty evaporators (#HP1), and icing (#HP4). Energy consumption increases for ground source heat pumps occur if the brine's pressure or temperature is too low or not frost-proof (#HP11-13).
Even if the heat pump works efficiently, conservation potential is often given for connected components that are inefficient, run too long at a too high level, or are triggered too often. Inefficiency is, for example, given in not-insulated pipes (#HD11) or for technologically old components operating below today's efficiency standards (#HW10). A too-long operation can appear for backup resistance heaters (#AH1-2) that should support heat pumps only in urgent cases. Scheduled operation is helpful in the heat distribution (#HD8-9) and the hot water system (#HW6-7, #HW9, #HW11). To stick with the rather extreme example of the energy consultant-a permanent running 20-m pipe heating cable needing 7-10 Watt per meter (I4) would sum to 1226-1752 kWh per year. Thus, the consultants recommend scheduling cables with a timer (i.e. half-day use). Overall, 65% of the cases found have the potential to lead to energy conservation.
Few to no conservation potential or an unclear direction of influence was found for 25% of the investigated cases (#HP8-10, #HD3-6, #HD10). For example, calcification (#HW1) of hot water storage fed by a heat pump does not necessarily lead to energy losses, which is contrary to traditional resistance-heater-based hot water storage. Finally, this category also contains an example of a problem that influences the energy costs but not the overall energy consumption (#HW12).
An increase in energy consumption is likely in 10% of the instances found. With one exception that refers to too low heating curves (#HC4), all problems are related to hot water production (#HW3-5, #HW8) and may also affect the health of residents. In a few cases, the pipe heating cable was disabled by the customer (#HW8), the heat storage temperature was too low (#HW3), or the disinfection program was disabled (#HW5). Frequently, the disinfection function was not installed (#HW4). Even though such settings lead to energy conservation, they also can lead to legionella-induced health issues. Careful energy conservation can be achieved if hot water pipes, circulation, and disinfection programs are neither disabled nor run continuously and if the hot water temperature ranges between 55 and 60 °C.

Discussion: information systems for heat pump problem recognition and solving
This study aims to prepare the ground to develop information systems that support recognising and solving heat pump problems. It ultimately contributes to the research activities in the field of energy informatics and information systems (Goebel et al. 2014;Ketter et al. 2018) by investigating the role that smart meters and heat pump owners can play in improving the efficiency of heat pumps.
For this purpose, this study answered, as a first research question, what typical problems exist that experts found in an energy efficiency campaign for heat pumps. The analysis revealed three general problem classes that primarily motivate inspection activities, but also made 44 underlying problem classes visible that vary considerably in their frequency, differ in the way they can be recognised, and solved, and in terms of potential benefits that result from solving them.
Hence, this study shed light on such underlying problems by developing and applying a classification scheme consisting of three dimensions and ten topics based on the expertise of energy consultants. Two topics are related to energy efficiency and health and help to describe the dimension potential benefits of solving heat pump problems (RQ1). Five topics refer to the problem recognition dimension, particularly when problems occur, where they originate, and how and who can recognise them (RQ2). Finally, three topics refer to the problem solving dimension and include problem-solution approaches, the problem solver and the associated costs (RQ3). The descriptive analysis of these topics clearly suggests the need for developing information systems that monitor and improve heat pump efficiency. Many of the found problems likely lead to energy waste (65% of the problem instances), often exist since installation (68%), and are caused by relatively inexpensive to solve incorrect settings (47%).
Due to their different characteristics, not all problems are similarly interesting or suitable for developing information systems that support energy consultancy services. Therefore, the following section discusses the three dimensions of the findings section altogether and answers which problems should be focused on when developing information systems that automate or assist the problem recognition process (RQ2) or the problem solving process (RQ3). Finally, it discusses implications for developing such information systems and thus provides a possible research agenda in this field.

Automated heat pump problem recognition
The cornerstone for developing data-driven automated problem detection is the collection of sensor data and sufficient labels from the field, which is both challenging and costly (e.g. Nowotny 1985; Katipamula and Brambley 2005;Kim and Katipamula 2018;Li and O'Neill 2018;Zhao et al. 2019). Both aspects are discussed in the following to reveal suitable candidates for automated problem recognition based on smart meter data.
Regarding the collection of sensor data, the findings of this study suggest that smart electricity meters can be an effective alternative to multiple sensing on heat pumps. Thirteen problems (ten with relevance for energy conservation) likely cause consumption patterns or strong change events in smart meter data (see Fig. 7). They represent about 44% of the found problems and are potential candidates for developing information systems that automate heat pump problem recognition. However, one must also consider their suitability for training data collection (see below). Using smart meter data is advantageous over IoT-based multiple sensing on heat pumps since the smart meter infrastructure is already standardised and mandated to be rolled out widely (e.g. EC 2014). Most importantly, data will even be available for a large number of old heat pumps and those currently on the market, for which neither multiple sensing nor the required IoT functionality is available.
Still, expert interviews in this study revealed that smart meter data alone are not enough for pattern recognition. This is because smart meters often mix the heat pump's measurement with other electrical consumers, patterns will only become visible if contextual information is available. What can help to isolate patterns is knowledge about specific periods in which problems can occur. Such periods include the transition phase between the heating and non-heating period, and day and night differences. Moreover, heat pumps and electric backup heaters should only create patterns outside of gridlock times that often depend on the electricity tariff (i.e. cheaper tariffs typically have more extended lock periods). Information about dynamic pricing tariffs (i.e. high/low tariff times) can help to identify non-optimal production times. Finally, climate data such as outside temperature is crucial for several problems and is often available from external providers at a high spatial and time resolution, similar to smart meter data. Therefore, developers of automated approaches should use smart meter data and contextual information to isolate patterns and tailor features required for machine learning for specific heat pump problems.
Regarding training data collection, the results emphasize that the manual extraction of labels from heat pump inspection reports, as conducted in this study, can be a viable alternative to generating labels in a laboratory or simulation with fault modelling. The average inspection revealed 1.6 problems that likely create smart meter patterns. Moreover, large single-event probabilities exist for the five most frequent problems that create smart meter patterns (#HC5, #HW4, #HC5, #HD8, #HW7), which range between 15 and 39%, on average, 24%. Thus, developers of information systems for automated heat pump problem recognition should focus on these problem classes. Based on these large single-event probabilities and the fact that an inspection reveals more than one problem, only hundreds and not thousands of inspections must be conducted to reach a training database similar to previous studies that successfully predicted heat pump characteristics (Fei et al. 2013;Hopf et al. 2018;Weigert et al. 2020;Brudermueller et al. 2022). Still, a key requirement for the generation labels is a standardised, digitised, and structured documentation of problems rather than one that is paper-based and unstructured (i.e. text fields).
Information systems that implement automated detection could support energy consultancy in several ways. First, consultants could use prediction scores about possibly occurring problems to make an informed decision about the benefit of inspection before the expensive on-site visit. Second, they can prioritize on-site visits to reduce their effort per saved kWh. Third, consultants might save time during the inspection since they can skip elements of the manual problem recognition process. Fourth, automated approaches could pave the way from one-time inspections to continuous inspections, which would also help to identify problems that occur slowly over time (about 28% of the found problems).
Besides improving the information base for energy consultants, automated approaches can also raise attention among heat pump owners who are often not aware of the existence of problems that affect energy consumption, i.e. by "sending a message [to the customer]. Look, there's something wrong. Go to the basement" (I5). Thus, automated approaches can also be an entry point for homeowners to pre-check the heat pump and components themselves. Triggering manual inspections is also important since automated approaches based on smart meter data alone will not be sufficient to detect all found problems. About 31 found problems, representing 56% of the cases, will likely not create smart meter patterns.

Assisted heat pump problem recognition
Where the capabilities of automated approaches end, information systems for assisted manual recognition can help. To this end, this study also investigated the potential that lies in the hand of the common homeowner in problem detection and solving. The findings suggest that manual recognition by non-professionals seems possible for 22 different problems (ten with relevance for energy conservation), representing about 45% of the found problems, see Fig. 7. Developers of information systems for assisted heat pump problem recognition should focus on these problem classes. Guided heat pump owners can conduct a series of simple visual inspections and use their ears and their temperature senses to recognise problems in the heat pump, the heat distribution, and the hot water production. However, they need access to expert knowledge for these tasks. Information systems could assist them by providing checklists containing simple rules (i.e. "if component X looks like Y, then problem Z exist"), self-explanatory pictures of dos and don'ts (i.e. don't put furniture on the air ducts), and how-to videos. Finally, information systems for assisted recognition can fulfil another important aspect. They can return homeowners' information about confirmed problems to the automated detection system. Thus, homeowners could be generators of ground truth data (i.e. labels) for automated approaches. In that respect, automated recognition and assisted recognition could mutually fertilise each other.

Assisted heat pump problem solving
Further down the road, information systems could also assist users in solving problems. Figure 7 shows that heat pump owners can solve eight different problems (#HD4-5, #HD9, #HW2, #HW8, #HP2, #HC5-6) that they can detect manually or are potentially recognisable from smart meter data. Five are relevant for energy conservation (#HD9, #HW2, #HP2, #HC5-6) and are the most promising candidates for information systems developers for assisted heat pump problem solving. Guided heat pump owners can detect and solve the named problems end-to-end and energy consultants can be relieved in such simple cases. Information systems can assist homeowners in solving problems in two respects. First, they could provide step-by-step instructions, references to handbooks, and videos explaining how problems can be solved. Second, they can reduce the search costs associated with looking for solutions. Since several setting problems occurred very often, and 40% of the users had barely any knowledge of the heating control system and needed training, system developers should focus on assistance solutions that give instructions and explain settings. It might be critical to customise instructions to specific heat pump products and components, as accessing and using interfaces were named as an important practical challenge. Moreover, system developers should provide clear explanations about the optimisation procedure for settings that could temporarily lead to lower room temperatures to anticipate the possible impatience of users. For complex situations, information systems could enable live assistance, for example, involving energy consultants remotely. More advanced and promising approaches could be chatbots that promote climate-friendly behaviour (Hillebrand and Johannsen 2021) or augmented reality that supports maintenance tasks (Quandt et al. 2020).

Limitations and future research
This study is one of the first that proposes and applies a classification scheme for heat pump problems in the field to investigate the potential of smart meter data for automated detection and the potential of heat pump owners in detection and solving. Through its exploratory nature, this study comes with several limitations that simultaneously prepare the ground for future research.
This study used data from a real heat pump inspection campaign that a utility company had already conducted. Thus, the influence on selecting participants that received such an inspection was limited. For example, it is not clear what motivated heat pump owners to participate in the campaign. Future research should, therefore, include factors like heat pump owners' environmental attitude, sensitivity to heating costs, and level of experience with heat pumps that could lead to a sample bias. Furthermore, the extraction of heat pump problems led to several issues that depend on national regulations. For example, Swiss regulations require reporting of the used refrigerant (#HP9), prohibit certain types of refrigerants for a refill (#HP8), and specify periods for leakage checks (#HP10). Such regulations might be different or irrelevant in other countries.
Acquiring experts to evaluate and apply the classification scheme was difficult, especially since there were ten questions to discuss for about fifty problem classes (requiring about five hours per consultant). However, two experts were willing to be interviewed. Future work could extend the expert base. Another limitation is the discriminatory power of the classification scheme regarding the actors for recognition and solving. It could be that problem classes classified as only recognisable or solvable by external experts can, in simple instances, also be recognised or solved by guided users. In challenging cases, problem classes typically recognisable or solvable by guided users might require professionals. The same applies to the three professional actors, i.e., experienced installers might also be able to perform technicians' tasks. What information (e.g. textbased instructions, pictures, videos) helps heat pump owners best to recognise and solve specific problems also remains future work.
Finally, the expert's assessment that problems create smart meter patterns does not necessarily imply that problems are predictable using machine learning methods. Besides the existence of patterns shown in this study, several factors influence predictability. This includes a problem-specific feature design, appropriate algorithms and parameters, and most importantly, having enough training data already discussed above. Furthermore, there might be the case that some problems generate patterns that were not easily derivable for the energy consultants. Hence, for some problems that occur frequently, and thus having a solid training database, there still might be a chance of predictability. Future work, therefore, should apply machine learning models and confirm the predictability of problems. The results of this study show with which problems to start.

Conclusion
This paper provides an empirically based overview of heat pump problems from different angles to prepare the ground for developing digitally supported and scalable energy consultancy services.
To overcome the limitations of previous studies that focus on approaches for recognising heat pump problems based on multiple-sensing (often not available in existing installations), and that see problem-solving primarily in the hand of experts (often scarce in the market), this study investigates two complementary alternatives: the role of data-driven recognition using a single-sensing approach based on smart meter data, and the role that typical heat pump owners can play in manual recognition and solving. To that end, this mixed-method study provides three important contributions to energy informatics and information systems research.
First, by applying a content analysis on 228 protocols from a heat pump inspection campaign, this study lists 47 problem classes and their frequencies. Knowledge about the frequency of typical problems is important for developing automatic recognition approaches since collecting enough labels is costly but crucial for machine learning.
Second, this study proposes and applies a classification scheme based on expert interviews to shed light on the found heat pump problems in three dimensions relevant to energy consultancy (recognition of problems, solving, and potential benefits). The classification showed that many of the problems found stay unnoticed, likely have been existing since the installation, and can lead to lower system efficiencies in many cases. The analysis ultimately highlights the importance of two major problem classes: those that are particularly complex to identify and fix and justify the time investment of hard-toget experts in on-site visits and those that can be recognised and solved by heat pump owners with little effort. In both cases, heat pump owners need to be triggered to tackle the efficiency potential of their installation and either request assistance from external experts or carry it out themselves.
Third, this study strengthens the idea that triggering such desired actions among heat pump owners seems possible with information systems that allow automated recognition of problems or assist users in recognition and solving. In particular, this study shows that (i) monitoring the systems' efficiency in operation based on the existing infrastructure of smart electricity meters is very likely tangible for several efficiencyrelevant problems and, based on this, (ii) the possibility of involving heat pump owners

B. Expert interviews
This part of the appendix contains the interview guide, and a list with the conducted interviews ( Table 6). The interviews followed the basic structure below:

Construct improvement and validation (open part)
Improving and validating the classification scheme: The interviewer provides the expert with a nascent version of the classification scheme with the three dimensions problem recognition, problem solving, and potential benefits.
1. Please think about following dimension "X" in the classification scheme (e.g. problem recognition). a) Do you agree/disagree with the categories (including sub-categories) of the classification scheme? b) How would you distinguish/extend the given categories (main, sub, sub-sub categories)? Improving and validating the list with problem classes: The interviewer provides the expert with a nascent version of the problem list, including a preliminary problem class description and, if necessary, references to the report's origins and exemplary text passages.
2. Please consider the following preliminary problem class description "X". a) Do you agree/disagree with the problem class description, and how would you change it? b) Are there problems that justify a distinct or higher-order problem class? c) What is a typical solution for this problem class?

Application of the classification scheme (semi-structured part)
The interviewer provides the expert updated versions of the classification scheme and problem list. Please answer the following questions for each problem class.   b) Temporal incidence: When does the problem occur? c) Components for recognition: Which component can be used to identify the problem? d) Problem recogniser: Who can recognise the problem? e) Smart meter patterns: Do problems create patterns in smart meter data? 4. Dimension "Problem solving" a) Solution approach: What primary action does it take to solve the problem? b) Problem solver: Who can solve the problem? c) Implementation costs: What does it cost to solve the problem? 5. Dimension "Potential benefits" a) Energy consumption: How does the energy consumption change after problem solving? b) Health: Does problem solving positively impact the health of residents?

C. Heat pump problems
This part of the appendix contains the list of the found problem classes in heat pump installations (see Table 7).