Computers in Industry 65 (2014) 1126–1135 Contents lists available at ScienceDirect Computers in Industry journal homepage: www.elsevier.com/locate/compind Comparing a knowledge-based and a data-driven method in querying data streams for system fault detection: A hydraulic drive system application Ahmad Alzghoul a,*, Björn Backe b,1, Magnus Löfstrand c,2, Arne Byström d,3, Bengt Liljedahl d,4 a Department of Information Technology, Division of Computing Science, Room POL ITC 19134, Box 337, 751 05 Uppsala, Sweden Division of Computer Aided Design, Luleå University of Technology, Room E218D, SE-97187 Luleå, Sweden c Department of Information Technology, Division of Computing Science, Room POL ITC 19111, Box 337, 751 05 Uppsala, Sweden d Bosch Rexroth Mellansel AB, SE-895 80 Mellansel, Sweden b A R T I C L E I N F O A B S T R A C T Article history: Received 9 January 2013 Received in revised form 18 April 2014 Accepted 19 June 2014 Available online 5 July 2014 The field of fault detection and diagnosis has been the subject of considerable interest in industry. Fault detection may increase the availability of products, thereby improving their quality. Fault detection and diagnosis methods can be classified in three categories: data-driven, analytically based, and knowledgebased methods. In this work, we investigated the ability and the performance of applying two fault detection methods to query data streams produced from hydraulic drive systems. A knowledge-based method was compared to a data-driven method. A fault detection system based on a data stream management system (DSMS) was developed in order to test and compare the two methods using data from real hydraulic drive systems. The knowledge-based method was based on causal models (fault trees), and principal component analysis (PCA) was used to build the data-driven model. The performance of the methods in terms of accuracy and speed, was examined using normal and physically simulated fault data. The results show that both methods generate queries fast enough to query the data streams online, with a similar level of fault detection accuracy. The industrial applications of both methods include monitoring of individual industrial mechanical systems as well as fleets of such systems. One can conclude that both methods may be used to increase industrial system availability. ß 2014 Elsevier B.V. All rights reserved. Keywords: Fault detection Data-driven Knowledge-based Data stream mining Data stream management system Product development 1. Introduction Industrial companies seek to increase the availability of their product and, thereby, improve product quality. Product availability can be improved in the design phase, in the testing and refinement phase, or in the operational phase, i.e. when the product is in use * Corresponding author. Tel.: +46 018 471 4026. E-mail addresses: ahmad.alzghoul@it.uu.se (A. Alzghoul), Bjorn.Backe@ltu.se (B. Backe), Magnus.Lofstrand@it.uu.se (M. Löfstrand), Arne.Bystroem@boschrexroth.se (A. Byström), Bengt.Liljedahl@boschrexroth.se (B. Liljedahl). 1 Tel.: +46 0920 49 2439. 2 Tel.: +46 018 471 4020. 3 Tel.: +46 660 87062. 4 Tel.: +46 660 87123. http://dx.doi.org/10.1016/j.compind.2014.06.003 0166-3615/ß 2014 Elsevier B.V. All rights reserved. [1,2]. Availability of industrial systems can be improved by detecting and diagnosing the failures at an early stage [3]. Fault detection and diagnosis methods have been classified in different ways [4–6]. Zhang and Jiang [5], divided the fault detection methods into model-based and data-based methods as illustrated in Fig. 1. Then, Zhang and Jiang [5] divided the modelbased and the data-based methods into two groups: quantitative and qualitative methods. On the other hand, Chiang et al. [6] classified fault detecting and diagnosing methods into three categories: data-driven methods, analytical-based (model-based) methods, and knowledge-based methods. Fig. 1 shows that the model-quantitative-based, data-quantitative-based, and the combination of model-qualitative-based and data-qualitative-based methods in the Zhang and Jiang [5] model can be seen as analytically based, data-driven, and knowledge-based methods, respectively, in the Chiang et al. [6] model. A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 1127 Fig. 1. Classification of fault detection and diagnosis methods (adapted from Zhang and Jiang [5]). The analytical approach uses first principles to construct mathematical models of the system [6]. Therefore, the analytically based methods incorporate the physical understanding of the system into the fault detection and diagnosis process [6,7]. According to Ref. [6] most of the analytically based methods are based on parameter estimation, observer-based design and parity relations. The analytical model is, according to Chiang et al. [6], not adequate for large-scale and complex systems. However, analytically based methods, when applicable, outperform the data-driven methods, as the former incorporate the physical understanding [6]. Knowledge-based methods use qualitative models in fault detection and the diagnosis process [6]. According to Ref. [8], most of the knowledge-based methods are rule-based expert systems. As the inferred rules are based on the historical failure cases and engineers’ experience, it is difficult to search a wide range of yieldloss cases beyond the engineers’ current knowledge [8]. The knowledge-based method is suitable when the detailed mathematical model is not available and when the number of inputs, outputs and states of a system is relatively small [6]. However, the knowledge-based method became more applicable to complex systems with the help of software packages [6]. A survey of knowledge-based fault diagnosis methods was performed by Zhu and Yu [9] and Venkatasubramanian et al. [10]. Fig. 1 shows that knowledge-based methods may be either qualitative or quantitative. It also shows that knowledge-based methods can be either qualitative, model-based (such as causal models) or qualitative, data-based (such as expert systems). In this paper, we have used causal models since the industrial partners are experts concerning the function of their hydraulic drive system, have a good historical knowledge, and are familiar with the fault tree analysis method (FTA) [11] used. Data-driven methods use the product lifecycle data for fault detection and do not require first-principles models. Therefore, data-driven methods can be applied to large-scale and complex systems and save time and cost, which is required for the development of first-principles models [6]. Data-driven methods are preferred when the product data is available while the system model is not [7]. In addition; data-driven methods have the ability to capture information and provide knowledge which is beyond the engineers’ current knowledge [8]. However, the performance of data-driven methods is based on the quality and the quantity of the collected data [6]. A survey of data-driven prognostics methods can be found in Refs. [10,12]. Data-driven methods can, according to Fig. 1, be divided into two groups: statistical and non-statistical methods. Principal component analysis (PCA) [13] and partial least squares (PLS) [14] are examples of statistical-based data-driven methods. On the other hand, artificial neural networks (ANN) [15], self-organized map (SOM) [16], and K-nearest neighbors [17] are examples of non-statistical data-driven methods. In this work, principal component analysis was selected as a data-driven method because it has been successfully used to detect faults in general, by researchers such as Villegas et al. [18], Tharrault et al. [19], Russell et al. [20], Kresta et al. [21], and Piovoso et al. [22]. Nowadays, the volume of the generated data is increasing. It is expected that the volume of generated data will exceed the available storage [23,24]. Therefore, fault detection methods need not only achieve high classification accuracy in detecting failures, but must also be fast enough to identify faults from continuous and fast-arriving data streams. A number of researchers have discussed the issue of detecting failures in data streams. A review of such application can be found in Alzghoul and Löfstrand [23]. Examples of research concerning data stream mining include [3,23,25–27]. Karcal [26], used a multivariate statistical process monitoring technique to detect change in the sensor data stream. Kargupta [25], developed the VEhicle DAta Stream mining (VEDAS) system for realtime vehicle-health monitoring. The proposed system was based on DSMS and a data stream mining (DSM) method. Matthews and Srivastava [27], applied different data-driven methods on continuous data stream from a solid rocket motor for anomaly detection. The authors of this paper have presented a number of related works [3,23,28,29] and in this paper, we report some verification and validation activities. In Alzghoul and Löfstrand [23], the authors investigated the possibility of increasing availability through the use of data stream mining and DSMS technologies. The authors reviewed the DSM algorithms and their applications in monitoring industrial systems. Also, they developed a fault detection system based on the DSM and the DSMS technologies. The fault detection system was developed based on three different DSM classification algorithms: One-class support vector machine (OCSVM), polygon-based classification algorithm and grid-based classification algorithm. The developed fault detection system was tested using data collected from hydraulic motors. The results showed that the three algorithms achieved good performance in detecting faults from sensor data streams. In Alzghoul et al. [3], the authors utilized data stream prediction methods to forecast the future data stream. In Ref. [3], they developed a fault detection system based on data stream forecasting, data stream mining and data stream management systems. The developed fault detection system was able to predict system faults based on a forecasted data stream. Furthermore, in Ref. [28] Alzghoul et al., discussed the potential industrial use of the proposed fault detection system which, in turn, was presented. Löfstrand et al. [29], proposed a model for predicting and monitoring industrial system availability [29]. They suggested using the IF-THEN-ELSE statements, which are based on FTA, for development of queries for system monitoring. However, developing and testing such queries was not done in Ref. [29]. In this work, queries based on FTA and PCA 1128 A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 methods were implemented and tested. Thus, the model presented in Ref. [29] is partly tested and verified which can be considered as a novelty of this work. Another novelty of this paper is testing and comparing a data-driven method (PCA), which was not tested in Refs. [3,23], to a knowledge driven method (FTA) in querying data streams for monitoring a hydraulic drive system application. Several authors, as identified above, discuss various fault detection methods, their advantages and disadvantages. However, few authors discuss applying specifically knowledge-based and data-driven methods for searching high-volume data streams. In addition, to our knowledge, no authors have compared the performance of different fault detection methods in searching high-volume data streams. The authors have worked closely with Bosch Rexroth Mellansel AB [30] (BRMAB, formerly Hägglunds Drives AB), to such an extent that their representatives are co-authors of this paper. BRMAB manufactures low-speed, high-torque hydraulic drive systems and are interested in improving the availability of their drive systems through monitoring. In this industrial context we investigated the functionality and performance of two different fault detection methods, as reported in Section 6. The first method, Principal Component Analysis is, in terms of Zhang and Jiang [5], a data-based quantitative method, while Chiang et al. [6], consider it a data-driven method. The second method, Fault Tree Analysis (FTA) is, in terms of Zhang and Jiang [5], a model-based qualitative method, while Chiang et al. [6], consider it a knowledge-based method. Using sensor data collected from a real BRMAB hydraulic drive system, the aim of this work is to partly validate the model presented by Löfstrand et al. [29]. The aim is also to test and compare the functionality and performance when applying a knowledge-based method (FTA) and a data-driven method (PCA) to query data streams, with the purpose of fault detection. Therefore, we developed a fault detection system based on DSMS technology to test the two fault detection methods. In Section 2 (research method), Fig. 2 shows the process of developing the fault detection functions, while Fig. 3 in Section 3 (fault detection system architecture) shows the architecture of the developed fault detection system. In addition, we discuss and summarize the advantages and disadvantages of using the knowledge-based and the data-driven methods (see Section 6). Results suggested that both methods, when implemented in the DSMS, are sufficiently fast and can produce good classification accuracy. As reported in Section 5, it is clear that the two approaches produce comparable and acceptable results in terms of accuracy and speed. This is also evident from reviewing Tables 1–5; thus, the developed models are verified and the model in Ref. [29] is partly verified. Furthermore, PCA is shown in this paper to be a suitable data-driven method for searching data streams from a hydraulic motor. Comparing to previous work by the authors, in Refs. [3,23], the authors showed that three other data driven methods are suitable. Furthermore, in Refs. [3,23], the data originated from a laboratory test (tank test hydraulic drive system) while in this paper, the data originate in a shredder application in use rather than from a lab test. From these tests, it is indicated that data driven methods in general, are suitable and generally successful for searching data streams. In particular, the authors feel that the four data-driven methods tested (three in Ref. [3,23] and one in this paper) are now validated when applied on data from hydraulic drive systems. Also the requirements for successful industrial implementation of the two methods were identified and described in Section 6. Furthermore, it was found that the performance of the fault detection models can be improved by investigating the misclassified data. Also, it was shown that sensor data can be used to improve the performance of the knowledge-based method by tuning its model parameters. 2. Research approach and case study Bosch Rexroth Mellansel AB (BRMAB) is a Swedish company which manufactures low-speed, high-torque hydraulic drive systems. BRMAB hydraulic drive systems are used in industries including: mining, recycling, pulp and paper and construction. BRMAB are interested in improving the availability of their drive systems through system monitoring. The work in this paper relates to a full-scale BRMAB hydraulic drive system, a shredder application used to crush waste wood as shown in Fig. 2. Qualitative data collection was done in collaboration with other researchers and BRMAB engineers. The data collection involved semi-structured and open-ended interviews [31] and data analysis involved using matrices [32]. BRMAB staff developed the baseline knowledge for the implementation of the applied fault detection methods, through an iterative process, together with the authors. The iterative process allowed for collaborative analysis throughout the process. The initial specific data collection comprised about three months while the entire research process, which forms the basis of this paper, comprised ca. 24 months. The first three month, were used for setting up the research team as well as the industrial team. The main research team consisted of the authors; one PhD researcher and two PhD students, one of which has since been awarded a PhD (i.e. Alzghoul). The industrial team consisted of two senior development engineers from Bosch Rexroth Mellansel AB (Liljedahl and Byström), each of which with about 30 year experience at the company. Of the industrial representatives, one (Liljedahl) has been responsible for product development with respect to the BRMAB hydraulic drive system while the other (Byström) has been responsible for systems and control development related to the hardware. Both industrial representatives currently hold positions concerned with long-term development at BRMAB (see biographies). Additional industrial representatives were brought in when further specific knowledge was required and when verification of pervious finings was carried out. Such instances occurred when:  Fault trees were verified  Cause and effect relationships among monitored parameters were decided  After various calculations of importance had been carried out, including heat dissipation in a cooler for example. Over the course of the herein reported study, about 10 face to face meetings, meetings using video conference software (on average Fig. 2. The shredder application. A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 every three weeks, www.alkit.se) and phone meetings (every two weeks) were carried out. 2.1. Formulating the fault detection models Sensor data was collected from the BRMAB shredder application to develop and test different fault detection methods being used to monitor in air–oil cooler functionality in the hydraulic drive system. Several variables which are associated with the cooler system functionality were considered. The process of building data-driven models and knowledgebased models is illustrated in Fig. 3. The data collection phase (as presented at the top of Fig. 3) was required to: enable a good understanding of the particular monitored system, identify which faults are of interest to monitor by the industrial partner, guide the choice of parameters to monitor, and to obtain other information which may be required. In the case of the data-driven method, if the required quantitative data is available for it to be used effectively, it likely does not require all the collected qualitative data. The next step in the data-driven method was to train the data-driven algorithm offline using a sample of the collected sensor data. The training was done using Matlab. Next, the resultant function was exported and formulated in SCSQL query language [33] which was implemented in the SuperComputer Stream Query processor (SCSQ) data stream management system [34]. In the case of the knowledge-based method, fault tree analysis (FTA) was performed using the collected data and heuristic knowledge by industrial representatives and researchers in collaboration. Thereafter, the fault detection function was formulated based on the FTA results. Further descriptions of the applied methods, as presented in Fig. 2(a) and (b), are presented in Sections 2.1.1 and 2.1.2. 1129 2.1.1. Fault detection using the knowledge-based method Before implementing the knowledge-based method (see Fig. 3b), based on [35], extensive data collection was performed. Interviews and workshops with a group of BRMAB representatives were conducted to elicit common faults affecting system reliability [36]. Secondary industrial data, such as technical specifications and hydraulic schemes, was collected and analyzed for system understanding [37]. Based on previous faults, found through interviews, a fault tree analysis (FTA) [38,39] was performed. The FTA was used to determine elements of the system that need to be monitored. A list of identified measurement points was made, which was used to detect critical faults through analyzing the FTA basic events. Thereafter, the structure and logic of the FTA was used to develop the relationships between the measurement points; that is, defining causal relationships between various parameters and parameter sets. Further information regarding the data collection and analysis for the knowledge-based method discussed here can be found in Ref. [29]. 2.1.2. Fault detection using the data-driven method (PCA) In this paper, principal component analysis (PCA) was used as the data-driven method. PCA produces a lower dimension of the data set and has the ability to increase the sensitivity of the process monitoring by applying two different measures, one for monitoring systematic trends of the process subspace and one for random noise subspace [6]. Principal component analysis is an unsupervised linear dimensionality reduction technique. PCA projects the data onto the orthogonal directions of maximal variance. The projection accounting for most of the data variance is called the first principal component [40]. PCA was used as a fault detection method as follows: Let X be data matrix of size n  m, where n is the number of data points and m stands for data dimensionality. X corresponds to data Fig. 3. Building fault detection function for this work using (a) data-driven method (b) knowledge-based method. 1130 A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 which represent the normal behavior of a system, i.e. no faults. Then, to build a fault detection system using PCA, one can follow the steps in List 1 (adapted from [41]): List 1 1. Calculate vectors for the mean (m) and the standard deviation (s) of X. 2. Normalize X to zero mean and unit variance using the calculated (m) and (s). 3. Calculate the PCA. 4. Determine the required number of principal components. 5. Calculate the control limits for T2 statistic and Q statistic. More details for the calculation of PCA, T2 statistic limit, and Q statistics limit can be found in Ref. [41] or [6]. T2 statistic and Q statistic are the control limits which will be used for online monitoring. List 2 shows the steps which can be used to test whether a new observation x represents a normal or abnormal (fault) behavior of the monitored system: List 2 1. Normalize x using the calculated mean (m) and variance (s) vectors in step (1) in List 1. 2. Calculate the T2 statistic and Q statistic for the normalized observation x using the obtained PCA model. 3. Check if the calculated T2 statistic and Q statistic are within their limits, i.e. the control limits calculated in step 5. 4. Consider observation x as fault data if it has T2 statistic violation or Q statistic violation. Fault detection based on PCA can be used to search the data stream using data stream management systems. The five steps in List 1 can be done offline to obtain the control limits for T2 statistic and Q statistic. Thereafter, a data stream management system can be used to do the calculations for testing new observations (in terms of data stream) according to the steps in List 2. The control limits T2 statistic and Q statistic can be updated offline. However, one could also use incremental principal component analysis (IPCA) [42], so that the fault detection system can be updated online. The following section presents the architecture of the fault detection system which was used to test the knowledge-based fault detection and the data-driven fault detection functions. 3. Fault detection system architecture As previously mentioned, some authors have discussed the issue of detecting failures in high volume data streams from equipment in use. Alzghoul and Löfstrand [23], Alzghoul et al. [3], and Kargupta [25], found that data stream management system (DSMS) technology has the potential to support scalability issues. DSMSs have the ability to manage data streams and apply continuous queries over input data streams. DSMS technology can be used to implement and test different fault detection methods in searching high-volume data streams. Therefore, the developed fault detection system is based on the DSMS technology. In order to test the data-driven and knowledge-based models, we used the architecture of the fault detection system illustrated in The two different fault detection functions (i.e. based on PCA and FTA) were developed as illustrated in Fig. 3. The fault detection functions are formulated into code using the DSMS query language. The DSMS is responsible for controlling, managing, and applying continuous queries over input data stream [25]. In this paper, the SuperComputer Stream Query processor (SCSQ) [34] was used as the DSMS. If the function output indicates a fault occurrence, then an alert is activated. Fig. 4 shows that there are two phases for the fault detection system: the online and offline phases. Building the fault detection function and formulating the mathematical functions as code are done in the offline phase, while receiving data stream from the monitored system and applying the fault detection function on the incoming data stream are done in the online phase using DSMS. However, updating the fault detection system can be done either offline or online based on which fault detection methods and algorithm are used. For instance, PCA can be updated offline by rebuilding the PCA model using recent data. On the other hand, it can be updated online through the use of incremental PCA (IPCA) algorithm [42]. 4. The data set The example query developed and used in this paper concerned fault detection in air–oil cooler, part of a hydraulic drive system Fig. 4. The architecture of the fault detection system. 1131 A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 powering the shredder application. Several variables which are associated with the cooler system functionality were considered. The cooler fan in the system is activated for different periods of time, depending on ambient temperature and system load. The cooler monitoring process will be activated when the cooler fan is switched on, whenever cooling is needed. Consequently, the part of the collected data representing when the cooler fan was switched on was used for training and testing. The data was collected once every 0.1 s for two days (06.00–14.00 day one and 06.00–18.00 day two). The data set includes both normal and abnormal data from the monitored system. The faulty data was created by BRMAB engineers in collaboration with the authors. The two failure modes which were practically simulated are: fully clogged cooler and half fully clogged cooler. The faulty data was practically created; that is, the data from different sensors was collected during the time the hydraulic drive system was running in the two failure modes. In order to build the data-driven model, i.e. applying the datadriven method, training data was required. Therefore, a sample of data was used to train and test the data-driven method (PCA). The normal data was arbitrarily selected; one sample from day one and one sample from day two. The abnormal data represented two simulated failure modes. The sample from the two days contained 4146 data points which correspond to normal behavior and 13,986 data points which correspond to abnormal data. Training PCAbased fault detection requires only normal data points, as mentioned in Section 2.1.2. However, a test was needed to examine the performance of the PCA algorithm. We trained and tested the algorithm using different training data set sizes (20% and 50% of the total normal data). The selection of the best PCA model was based on the classification accuracy of the remaining normal data points, i.e. 80% and 50% of 4146, and all abnormal data set (13,986 data points). In the case of the knowledge-based method, all data points (normal and abnormal) were used for testing, as the knowledge-based method does not require training. 5. Results This section presents, compares and discusses the results of testing the data-driven model (PCA) and the knowledge-based model (FTA). 5.1. Results from testing the data-driven fault detection method To test the data-driven model, PCA was first trained using normal data points as discussed in Section 4. Repeated random sub-sampling validation was used to assess the accuracy of the PCA model. Table 1 shows the accuracy of the PCA model at three different tests when 50% of the normal data points (50% out of 4146 = 2073) was used as training, while the rest was used for testing, i.e. 2073 normal data points and 13,986 abnormal data points. Table 1 shows that PCA was able to achieve high classification accuracy in detecting abnormal data in three different tests. PCA was able to detect on average 13,983 out of 13,986 abnormal data points, achieving 99.98% classification accuracy. PCA achieved less accuracy, 95.85% on average, in classifying normal data points. Thereafter, we decreased the number of training data points, as it is faster to train the PCA algorithm using the smaller data set. A smaller training data set size (20% of the normal data) was used to assess the performance of the PCA algorithm. Table 2 shows the performance of PCA when 20% of the normal data points (20% out of 4146 = 829) was used for training. Table 2 shows that PCA achieved high classification accuracy in detecting abnormal data, even if the number of training data points was reduced. Since the training data set was decreased and the PCA model was still achieving good classification accuracy, the Table 1 Accuracy of the PCA model at three different tests when 50% of the normal data points (2073 data points) was used for training. Classification accuracy of normal data Classification accuracy of abnormal data Test 1 Test 2 Test 3 95.18% 96.14% 96.24% 99.98% 99.98% 99.97% Average 1987/2073 = 95.85% 13,983/13,986 = 99.98% Table 2 Accuracy of the PCA model at three different tests when 20% of the normal data points (829 data points) was used for training. Classification accuracy of normal data Classification accuracy of abnormal data Test 1 Test 2 Test 3 97.29% 96.20% 95.99% 99.96% 99.98% 99.97% Average 3200/3317 = 96.47% 13,982/13,986 = 99.97% potential problem of overfitting was proven to be a non-issue. When comparing the three tests in Tables 1 and 2, we noticed that Test 1 in Table 2 achieved the best performance in classifying normal data points and almost similar results in classifying abnormal data points. Therefore, the PCA model, which was built using the training data in Test 1 in Table 2, was used for testing all collected data (06.00–14.00 day one and 06.00–18.00 day two). 5.2. Results from testing the knowledge-based fault detection method As mentioned previously, knowledge-based methods do not require training. The knowledge-based method was tested using all normal data points (4146) and abnormal data points (13,986). Therefore, only one test was applied, i.e. all data was used as testing data set, unlike the data-driven method where the result involved applying different training and testing data sets. Table 3 presents the classification accuracy of the applied knowledge-based method. Table 3 shows that the knowledge-based method achieved high classification accuracy of abnormal data and good performance in classifying normal data. In terms of performance, the applied knowledge-based method and data-driven method are quite similar when comparing the results obtained by the knowledgebased method and results obtained by PCA. The result obtained by PCA was slightly better than the results obtained by the knowledge-based method. Tuning parameters or thresholds which are involved in the knowledge-based method may change the results. Therefore, Table 3 Knowledge-based method classification accuracy. Knowledge-based method (before tuning) Classification accuracy of normal data Classification accuracy of abnormal data 3976/4146 (95.90%) 13,977/13,986 (99.94%) Table 4 Knowledge-based method classification accuracy before and after tuning. Knowledge-based method (before tuning) Knowledge-based method (after tuning) Classification accuracy of normal data Classification accuracy of abnormal data 3976/4146 (95.90%) 13,977/13,986 (99.94%) 4116/4146 (99.28%) 13,973/13,986 (99.91%) 1132 A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 the knowledge-based method was tested by trying different thresholds. It was found that tuning the threshold increased the classification accuracy of normal data points from 95.90% to 99.28% and slightly reduced the classification accuracy of abnormal data points from 99.94% to 99.91%, as shown in Table 4. 5.3. Investigating misclassified data To improve the performance of both methods the misclassified data was investigated. It was found that the misclassified data corresponded to the first few seconds after the cooler was switched on. To overcome this problem, one can either skip the first few seconds of data or make a condition that the system give an alert when faulty data is present for more than a specific period of time, e.g. faulty data occurring continuously for more than 60 s. The second alternative was selected in this work where the time threshold was 15 s. The second alternative is also useful for eliminating outliers, if they exist. A time threshold of 15 s will not affect the system negatively, since it heats much slower than that, even during high power usage. Once the cooler fan switches on, the temperature of the oil from the cooler reaches a steady-state value after some time. This time is dependent on the oil volume of the cooler and the oil flow passing through. 5.4. Results of testing all data The final test was performed using the alert condition with time threshold 15 s and using all collected data (data collected during two days). Fault detection functions developed using knowledge-based and data-driven methods were tested using the proposed fault detection system illustrated in Fig. 4. It was found that both methods were able to detect all faults without any false alert. Also, both methods were adequate to use online, i.e. they can be used to search the data stream, since they are faster than the arriving data stream. 6. Discussion The developed models were tested using a data set containing both faulty and non-faulty data points. The time when faults appear as well as when faults disappear in the data set, are known (i.e. the data-label is known). Comparing the outputs from the developed models with the pre-known data-label showed relatively speaking, good accuracy, as shown in Section 5, thus verifying the developed models. The results showed that both the knowledge-based and the data-driven method achieved high classification accuracy and were applicable for searching the data stream. However, through literature and based on development, testing and the results of both methods, several differences were noticed. Table 5 shows a summary of these differences. Rows 1–8 of Table 5 have been concluded (by the authors) to be true. Whereas, row 9 i.e. Area of application, is based on the research by other researchers [6,7,44,45]. The results based on the BRMAB case, show the ability of both methods to query data streams. Note that, the ability of the knowledge-based method to be applied as queries, using a data stream management system to detect faults in high volume data streams, has not been found extensively in literature based on the authors review. Table 5 shows that building and updating data-driven models requires less time and cost than building and updating knowledge-based models, these findings are also supported by other researchers in Refs. [6,43–45]. For example, Chiang et al. [6], Bratina et al. [45], and Das et al. [44] agree that data-driven methods are easy to develop and implement. Data-driven methods have the ability to provide new information which may not be presented in knowledge-based methods, which are solely based on heuristic knowledge. Such new information may include new types of out-of-bounds data sets [8]. Chih-Min and Yun-Pei [8] and Lou et al. [46] discussed that, in some situations, knowledge-based methods cannot find new faults. The authors of this paper would agree with Chin-Min and Yun-Pei that the cause and effect relations are of great importance for creating the knowledge-based models. Data-driven methods can be easily adapted to the changes in the monitored system, for example, by using incremental principal component analysis (IPCA) instead of, or in conjunction with, PCA. According to Luo et al. [46], and supported by results as presented in this paper, incorporating a level of adaptiveness in a knowledge-based method is harder to accomplish than in a data-driven method. The accuracy of the data-driven methods (and the accuracy of knowledge-based methods) depends on the applied algorithm; thus, the results may vary from one algorithm to another. For example, Alzghoul and Löfstrand [23] tested different data stream mining algorithms and the algorithms achieved different results. Building a data-driven model does not require specific engineering knowledge. However, data-driven models cannot be built without data sets. According to Chiang et al. [6] and ChihMin and Yun-Pei [8], and Das et al. [44], data-driven methods need only data and do not require deep knowledge about the monitored system. On the other hand, knowledge-based methods achieved good performance in detecting pre-known faults as shown in this paper and discussed by Chiang et al. [6] and Chih-Min and Yun-Pei [8]. Also, engineering knowledge can be used to build the fault detection model without the need of a data set [8]. However, as shown in this work, sensor data can be used to improve the performance of the fault detection model by, for example, tuning model parameters as it was shown in Section 5.2. Table 5 Comparison between knowledge-based and data-driven methods, results are based on literature and BRMAB case. Knowledge-based method Data-driven method Development cost Development time Fault detection accuracy High [43] Long [43] High [6,8] Adaptiveness Time and cost expensive compared to the data-driven method [46] Not necessary [6,8] High No [8,46] Yes [6,8,47] Can be used for both small and large-scale systems using software packages [6,7] Cheap [6,44,45] Short [6,44,45] Based on the algorithm [23] and the quantity and quality of data [6] Based on [6,44,45], plausibly cheap and fast Data set availability Speed and ability to query data streams Ability to detect new information Require deep system-specific knowledge Area of application Necessary [6–8,44,47] Generally high (depending on the algorithm) [23] Yes [8] No [6,8,44] Can be used for both small and large-scale systems [6,7,44,45] A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 Clearly, the quality of knowledge based methods in general is dependent on the validity and reliability on the information from the industrial informants, which in these cases was very good. One might conclude that any knowledge based method when used is capable of correctly capturing the relevant information from the informants, would likely generate comparable quality results. On the other hand, the performance of datadriven methods in general (in addition to the data-driven methods tested in Refs. [3,23] and in this paper), is based on the quality and the quantity of the collected data [6]. Also, the impact of the input variables on the target affects the performance of the data-driven methods. Furthermore, some of the data-driven methods such as support vector machine take the advantage of using nonlinear kernels to map the data to a very high dimensional space. For example, it was shown in Ref. [23] that the one-class support vector machine outperformed the polygon-based and grid-based methods because it maps the data into a higher dimension. Every fault detection technique has its own advantages and disadvantages. Therefore, it could be beneficial to combine different techniques to bring their advantages together. For example, one could achieve the advantages of knowledge-based methods in detecting known faults, and data-driven methods for detecting new faults, thereby achieving a better understanding of the system by combining the two techniques. In any case, we show that the accuracy of the knowledge-based method is for all intents and purposes the same as that of the data-driven method. This is likely because the interviewees and participating company partners have an exceptional system understanding, and it may also be due to the chosen research method. The high accuracy of the knowledge-based results is like due to:  The hydraulic theory, on which the queries are based, was wellknown for the authors. o However, to implement the theory correctly, good system knowledge and experience are required  BRMAB has exceptional system knowledge and experience, since the company has been in business over 100 years and the engineers in charge have worked over 30 years.  The authors have some experience regarding hydraulic theory and furthermore, the research method was iterative in that many development and feedback loops were carried out with researchers and industrial representatives collaboratively. The data-driven accuracy is high since it is based on the PCA model which is sensitive to the abnormal behavior and had been successfully applied in the monitoring of complex systems. In addition, the quality and quantity of the collected data was good for the authors to be able to produce a data-driven model with high performance. (As previously stated, the performance of datadriven methods is based on the quality and the quantity of the collected data [6].) In more detail, the data was ‘‘good’’ in the sense that the authors were aware when, i.e. at what time stamp, the technical system was manipulated based on common faults like a clogged cooler, to create sensor data corresponding to such faults. Furthermore, the system manual from BRMAB states the permitted ranges of oil temperatures for example, which is true for all relevant variable ranges. In addition, the quantity of data used for training clearly was enough since the accuracy turned out to be relatively high (i.e. For example on average 99.97% for the abnormal data, see Table 2). The results suggest that the developed models are applicable in monitoring the cooler functionality of hydraulic drive systems. The performance of the developed models may also indicate a possible 1133 success in monitoring other hydraulic drive functions and even other product functions. In general, it is plausible that the two compared approaches for fault detection can be used successfully in industry, when the following conditions are met. The knowledge-based method requires:  sensor data to tune the knowledge-based model parameters  a known data-label for model verification (i.e. our knowledge regarding when faults appear and disappear).  good system knowledge (provided here by BRMAB personnel)  knowledge regarding normal and abnormal behavior among the various monitored system parameters  that the technical system allow for formulating the continuous queries (i.e. ‘‘rules’’); therefore, too complex systems may need to be simplified The data-driven method requires:  a sample of sensor data (with a known data-label) for training  a sample of faulty data for verification and/or training  brief knowledge about the monitored system and most common faults that will give a good understanding of the achieved results and possibly increase the accuracy (for example, investigating misclassified data)  a good implementation of the data-driven algorithm that may affect both speed and accuracy The two verified models can be used to monitor industrial systems online, thereby increasing availability and saving maintenance time and cost. The two verified models are useful for monitoring a fleet of units or a single high-value unit. 7. Conclusions As discussed previously, increasing availability of products is of great interest for industrial companies. Availability of industrial products can be improved by fault detection and diagnosis. Fault detection and diagnosis methods can be divided into three categories: data-driven, analytical-based, and knowledge-based methods. In this work, monitoring cooler functionality of BRMAB hydraulic drive systems was investigated. Two fault detection techniques, i.e. data-driven (PCA) and knowledge-based (FTA) methods, were designed, implemented and tested. The performance of the two methods and their ability to search the data stream were compared. Also, the advantages and disadvantages of both methods were discussed. It was found that both methods achieved good performance in detecting faults (around 99% classification accuracy of abnormal data) and both methods are fast enough to be used for querying the data stream. The performance of the fault detection models was improved by investigating the misclassified data. Also, it was shown that sensor data can be used to improve the performance of the knowledge-based method by tuning its model parameters. The data-driven methods surpassed the knowledge-based methods in model development time and cost, they are easy to retrain and able to detect faults beyond the engineers’ knowledge. On the other hand, the knowledge-based models can be developed without sensor data and still achieve high accuracy in detecting historical faults, provided that they are based on accurate engineer (i.e. informant) knowledge. Hence, both approaches may, if the conditions discussed above are met, be used to predict and identify faults in hydraulic drive systems or comparable applications in use. In terms of the product development process, if the methods are used in the testing and refinement stage, their use may improve 1134 A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 the associated prototypes and may thereby improve the quality of the product at an earlier stage of development. Naturally, the two approaches may be used in the late stage of a product lifecycle to improve the design of the next updated product version. Acknowledgments The authors wish to acknowledge the following three organizations: The EU FP7 Project SmartVortex, The Faste Laboratory at Luleå University of Technology funded by the Swedish Governmental Agency for Innovation Systems (VINNOVA: 2012-00705) and the SSPI project (Scalable search of product lifecycle information) funded by the Swedish Foundation for Strategic Research (SSF: RIT08-0041). References [1] D.G. Ullman (Ed.), The Mechanical Design Process, 4th ed., McGraw-Hill Science, New York, 2010. [2] I. Alsyouf, A. Alzghoul, Soft computing applications in wind power systems: a review and analysis, in: European Offshore Wind Conference and Exhibition, Stockholm, Sweden, 2009. [3] A. Alzghoul, M. Löfstrand, B. Backe, Data stream forecasting for system fault prediction, Computers and Industrial Engineering 62 (4) (2012) 972–978. [4] V. Venkatasubramanian, R. Rengaswamy, K. Yin, S.N. Kavuri, A review of process fault detection and diagnosis part I: quantitative model-based methods, Computers and Chemical Engineering 27 (3) (2003) 293–311. [5] Y. Zhang, J. Jiang, Bibliographical review on reconfigurable fault-tolerant control systems, Annual Reviews in Control 32 (2) (2008) 229–252. [6] L.H. Chiang, E. Russell, R.D. Braatz, Fault Detection and Diagnosis in Industrial Systems, Springer Verlag, London, 2001. [7] C. Sankavaram, B. Pattipati, A. Kodali, K. Pattipati, M. Azam, S. Kumar, M. Pecht, Model-based and data-driven prognosis of automotive and electronic systems, in: IEEE International Conference on Automation Science and Engineering, 2009. CASE 2009, 2009, 96–101. [8] F. Chih-Min, L. Yun-Pei, A Bayesian framework to integrate knowledge-based and data-driven inference tools for reliable yield diagnoses, in: Simulation Conference, 2008. WSC 2008. Winter, 2008, 2323–2329. [9] D. Zhu, S. Yu, Survey of knowledge-based fault diagnosis methods, Anhui Gongye Daxue Xuebao (Journal of Anhui University of Technology), 19 (3) (2002) 197–204. [10] V. Venkatasubramanian, R. Rengaswamy, S.N. Kavuri, K. Yin, A review of process fault detection and diagnosis: part III: process history based methods, Computers and Chemical Engineering 27 (3) (2003) 327–346. [11] J.B. Fussell, G.J. Powers, R.G. Bennetts, Fault trees – a state of the art discussion, IEEE Transactions on Reliability R-23 (1) (1974) 51–55. [12] M.A. Schwabacher, A Survey of Data Driven Prognostics, AIAA Infotech@Aerospace, 2005. [13] K. Pearson LIII, On lines and planes of closest fit to systems of points in space, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2 (11) (1901) 559–572. [14] S. Wold, C. Albano, W. Dunn III, U. Edlund, K. Esbensen, P. Geladi, S. Hellberg, E. Johansson, W. Lindberg, M. Sjöström, Multivariate data analysis in chemistry, in: Chemometrics, Springer, 1984, pp. 17–95. [15] W.S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics 5 (4) (1943) 115–133. [16] T. Kohonen, Self-organized formation of topologically correct feature maps, Biological Cybernetics 43 (1) (1982) 59–69. [17] B.V. Dasarathy, Nearest Neighbor ({NN}) Norms:{NN} Pattern Classification Techniques, 1991. [18] T. Villegas, M.J. Fuente, M. Rodrı́guez, Principal Component Analysis for Fault Detection and Diagnosis. Experience with a pilot plant, 2010, 147–152. [19] Y. Tharrault, G. Mourot, J. Ragot, Fault Detection and Isolation with Robust Principal Component Analysis, 2008, 59–64. [20] E.L. Russell, L.H. Chiang, R.D. Braatz, Fault detection in industrial processes using canonical variate analysis and dynamic principal component analysis, Chemometrics and Intelligent Laboratory Systems 51 (1) (2000) 81–93. [21] J.V. Kresta, J.F. Macgregor, T.E. Marlin, Multivariate statistical monitoring of process operating performance, The Canadian Journal of Chemical Engineering 69 (1) (1991) 35–47. [22] M. Piovoso, K. Kosanovich, R. Pearson, Monitoring process performance in realtime, IEEE (1992) 2359–2363. [23] A. Alzghoul, M. Löfstrand, Increasing availability of industrial systems through data stream mining, Computers and Industrial Engineering 60 (2) (2011) 195–205. [24] Data, data everywhere (a special report on managing information), The Economist 2010. [25] H. Kargupta, VEDAS: a mobile and distributed data stream mining system for realtime vehicle monitoring, in: SIAM Proceedings Series, 2004, pp. 300–311. [26] S.C. Karacal, Mining machine data streams using statistical process monitoring techniques, in: IIE Annual Conference and Expo, 2007, 1290–1297. [27] B. Matthews, A.N. Srivastava, Comparative analysis of data-driven anomaly detection methods, in: JANNAF Conference on Propulsion Systems, Proceedings of the Joint Army Navy NASA Air Force Conference on Propulsion, Orlando, FL, 2008. [28] A. Alzghoul, M. Löfstrand, L. Karlsson, M. Karlberg, Data stream mining for increased functional product availability awareness, in: J. Hesselbach, C. Herrmann (Eds.), Functional Thinking for Value Creation, Proceedings of the 3rd CIRP International Conference on Industrial Product Service Systems, Technische Universität Braunschweig, May 5th–6th, Springer Berlin Heidelberg, Braunschweig, Germany, 2011, pp.237–241. [29] M. Löfstrand, B. Backe, P. Kyösti, J. Lindström, S. Reed, A model for predicting and monitoring industrial system availability, International Journal of Product Development 16 (2) (2012) 140–157. [30] Boschrexroth.com, (2014). Bosch Rexroth. The Drive & Control Company. - Bosch Rexroth AG. Available at: http://www.boschrexroth.com/ [Accessed Jun. 2014]. [31] M.Q. Patton, Qualitative Evaluation and Research Methods, Sage, London, UK, 1990. [32] M. Miles, M. Huberman, An Expanded Sourcebook – Qualitative Data Analysis, 2nd ed., Sage, Thousand Oaks, CA, 1994. [33] E. Zeitler, T. Risch, Using stream queries to measure communication performance of a parallel computing environment, IEEE (2007) 65. [34] E. Zeitler, T. Risch, Processing high-volume stream queries on a supercomputer, in: IEEE Proceedings of the 22nd International Conference on Data Engineering Workshops, 2006. [35] R. Isermann, Fault-Diagnosis Applications Model-Based Condition Monitoring: Actuators, Drives, Machinery, Plants, Sensors, and Fault-tolerant Systems, Springer Berlin Heidelberg, Germany, 2011. [36] R.K. Yin, Case Study Research, 4th ed., Sage publications, 2008. [37] M.B. Miles, A.M. Huberman, Qualitative Data Analysis: An Expanded Sourcebook, 2nd ed., SAGE Publications, Thousand Oaks, CA, 1994. [38] J.D. Andrews, T.R. Moss, Reliability and Risk Assessment, Professional Engineering Publishing Limited, Bury St Edmunds, 2002. [39] NUREG-0492, Fault Tree Handbook, Nuclear Regulatory Commission, Washington, 1981. [40] J. Hu, J. Si, B.P. Olson, J. He, Feature detection in motor cortical spikes by principal component analysis, IEEE Transactions on Neural Systems and Rehabilitation Engineering 13 (3) (2005) 256–262. [41] D. Garcıa-Alvarez, Fault detection using Principal Component Analysis (PCA) in a Wastewater Treatment Plant (WWTP), in: 62th Int. Student’s Scientific Conf., Saint-Petersburg, 2009. [42] H.V. Neto, U. Nehmzow, Incremental PCA: An Alternative Approach for Novelty Detection, Towards Autonomous Robotic Systems, 2005. [43] V. Uraikul, C.W. Chan, P. Tontiwachwuthikul, Artificial intelligence for monitoring and supervisory control of process systems, Engineering Applications of Artificial Intelligence 20 (2) (2007) 115–131. [44] A. Das, J. Maiti, R.N. Banerjee, Process monitoring and fault detection strategies: a review, International Journal of Quality and Reliability Management 29 (7) (2012) 720–752. [45] B. Bratina, N. Muškinja, B. Tovornik, Design of an Auto-associative Neural Network by Using Design of Experiments Approach, Neural Computing and Applications 19 (2) (2010) 207–218. [46] J. Luo, S. Ghoshal, A. Mathur, K.R. Pattipati, Adaptive maintenance knowledge bases for field service, in: 2007 IEEE Aerospace Conference, Big Sky, MT, (2007), pp. 1–11. [47] H. Wang, T.-Y. Chai, J.-L. Ding, M. Brown, Data driven fault diagnosis and fault tolerant control: some advances and possible new directions, Acta Automatica Sinica 35 (6) (2009) 739–747. Ahmad Alzghoul is currently a researcher at Uppsala University, Department of Information Technology, Division of Computing Science. He has an educational background in the field of computer science and engineering. He received a Ph.D. in Computer Aided Design at Luleå University of Technology, Division of Product and Production Development, Sweden. He received his M.Sc. degree in Computer Engineering (Intelligent Systems) from Halmstad University, Sweden, and his M.Sc. degree in Software Engineering from Linnaeus University, Sweden. His main research interests include data mining and industrial data mining applications. A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135 Björn Backe (M.Sc.) is a Ph.D. student in Computer Aided Design at Luleå University of Technology, Division of Product and Production Development. His main research interests include hardware reliability. Magnus Löfstrand has an educational background in mechanical engineering and was awarded his Ph.D. in Computer Aided Design (CAD) at Luleå University of Technology, Sweden in 2007. After serving as assistant professor in CAD, he is now employed as a senior researcher at Uppsala University, Department of Information Technology, Division of Computing Science, Uppsala DataBase Laboratory. He has an academic background in the work process description, refinement and simulation based on product development literature. He also has experience of research concerning, and equipment management for, distributed collaboration over IP networks. He is involved in research concerning system availability (maintainability and reliability) and the use of DSMS (Data Stream Management System) and DSM (Data Stream Mining) in engineering applications, often in the context of enabling larger industrial service content in industrial systems. 1135 Arne Byström, B.Sc., has experience from working in the Swedish high technological industry, mainly within R&D, for more than 37 years. He previously, for over ten years, served as manager for customer control system order development. He has good knowledge of elliciting and interpreting customer needs and has experience from control related problem solving in customer applications around the world. Today, he works for Bosch Rexroth Mellansel AB as Technical Product Manager Systems, in the application area of electrohydraulic drive control and drive monitoring systems. Bengt Liljedahl has a B.Sc. in Mechanical Engineering from 1973 and a diploma in Mechanical Engineering at University level, received in 1978. He has over 38 years of experience from working in Swedish Industry, mainly within R&D. He currently serves as Technical Product Manager at Bosch Rexroth Mellansel AB (former Hägglunds Drives AB). He has since 1989 coordinated and built a network of close cooperation with Swedish Universities to increase the competence of Bosch Rexroth Mellansel AB in 3 main areas, Tribology, Product Development and Material Science. For his longstanding and successful research collaboration, he was awarded an Honorary Doctorate at Luleå University of Technology in June of 2009.