Computers in Industry 65 (2014) 1126–1135
Contents lists available at ScienceDirect
Computers in Industry
journal homepage: www.elsevier.com/locate/compind
Comparing a knowledge-based and a data-driven method in querying
data streams for system fault detection: A hydraulic drive system
application
Ahmad Alzghoul a,*, Björn Backe b,1, Magnus Löfstrand c,2,
Arne Byström d,3, Bengt Liljedahl d,4
a
Department of Information Technology, Division of Computing Science, Room POL ITC 19134, Box 337, 751 05 Uppsala, Sweden
Division of Computer Aided Design, Luleå University of Technology, Room E218D, SE-97187 Luleå, Sweden
c
Department of Information Technology, Division of Computing Science, Room POL ITC 19111, Box 337, 751 05 Uppsala, Sweden
d
Bosch Rexroth Mellansel AB, SE-895 80 Mellansel, Sweden
b
A R T I C L E I N F O
A B S T R A C T
Article history:
Received 9 January 2013
Received in revised form 18 April 2014
Accepted 19 June 2014
Available online 5 July 2014
The field of fault detection and diagnosis has been the subject of considerable interest in industry. Fault
detection may increase the availability of products, thereby improving their quality. Fault detection and
diagnosis methods can be classified in three categories: data-driven, analytically based, and knowledgebased methods.
In this work, we investigated the ability and the performance of applying two fault detection methods to
query data streams produced from hydraulic drive systems. A knowledge-based method was compared to a
data-driven method. A fault detection system based on a data stream management system (DSMS) was
developed in order to test and compare the two methods using data from real hydraulic drive systems.
The knowledge-based method was based on causal models (fault trees), and principal component
analysis (PCA) was used to build the data-driven model. The performance of the methods in terms of
accuracy and speed, was examined using normal and physically simulated fault data. The results show
that both methods generate queries fast enough to query the data streams online, with a similar level of
fault detection accuracy. The industrial applications of both methods include monitoring of individual
industrial mechanical systems as well as fleets of such systems. One can conclude that both methods
may be used to increase industrial system availability.
ß 2014 Elsevier B.V. All rights reserved.
Keywords:
Fault detection
Data-driven
Knowledge-based
Data stream mining
Data stream management system
Product development
1. Introduction
Industrial companies seek to increase the availability of their
product and, thereby, improve product quality. Product availability
can be improved in the design phase, in the testing and refinement
phase, or in the operational phase, i.e. when the product is in use
* Corresponding author. Tel.: +46 018 471 4026.
E-mail addresses: ahmad.alzghoul@it.uu.se (A. Alzghoul),
Bjorn.Backe@ltu.se (B. Backe), Magnus.Lofstrand@it.uu.se (M. Löfstrand),
Arne.Bystroem@boschrexroth.se (A. Byström), Bengt.Liljedahl@boschrexroth.se
(B. Liljedahl).
1
Tel.: +46 0920 49 2439.
2
Tel.: +46 018 471 4020.
3
Tel.: +46 660 87062.
4
Tel.: +46 660 87123.
http://dx.doi.org/10.1016/j.compind.2014.06.003
0166-3615/ß 2014 Elsevier B.V. All rights reserved.
[1,2]. Availability of industrial systems can be improved by
detecting and diagnosing the failures at an early stage [3].
Fault detection and diagnosis methods have been classified in
different ways [4–6]. Zhang and Jiang [5], divided the fault
detection methods into model-based and data-based methods as
illustrated in Fig. 1. Then, Zhang and Jiang [5] divided the modelbased and the data-based methods into two groups: quantitative
and qualitative methods. On the other hand, Chiang et al. [6]
classified fault detecting and diagnosing methods into three
categories: data-driven methods, analytical-based (model-based)
methods, and knowledge-based methods. Fig. 1 shows that the
model-quantitative-based, data-quantitative-based, and the combination of model-qualitative-based and data-qualitative-based
methods in the Zhang and Jiang [5] model can be seen as
analytically based, data-driven, and knowledge-based methods,
respectively, in the Chiang et al. [6] model.
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
1127
Fig. 1. Classification of fault detection and diagnosis methods (adapted from Zhang and Jiang [5]).
The analytical approach uses first principles to construct
mathematical models of the system [6]. Therefore, the analytically
based methods incorporate the physical understanding of the
system into the fault detection and diagnosis process [6,7].
According to Ref. [6] most of the analytically based methods are
based on parameter estimation, observer-based design and parity
relations. The analytical model is, according to Chiang et al. [6], not
adequate for large-scale and complex systems. However, analytically based methods, when applicable, outperform the data-driven
methods, as the former incorporate the physical understanding [6].
Knowledge-based methods use qualitative models in fault
detection and the diagnosis process [6]. According to Ref. [8], most
of the knowledge-based methods are rule-based expert systems.
As the inferred rules are based on the historical failure cases and
engineers’ experience, it is difficult to search a wide range of yieldloss cases beyond the engineers’ current knowledge [8]. The
knowledge-based method is suitable when the detailed mathematical model is not available and when the number of inputs,
outputs and states of a system is relatively small [6]. However, the
knowledge-based method became more applicable to complex
systems with the help of software packages [6]. A survey of
knowledge-based fault diagnosis methods was performed by Zhu
and Yu [9] and Venkatasubramanian et al. [10].
Fig. 1 shows that knowledge-based methods may be either
qualitative or quantitative. It also shows that knowledge-based
methods can be either qualitative, model-based (such as causal
models) or qualitative, data-based (such as expert systems). In this
paper, we have used causal models since the industrial partners are
experts concerning the function of their hydraulic drive system,
have a good historical knowledge, and are familiar with the fault
tree analysis method (FTA) [11] used.
Data-driven methods use the product lifecycle data for fault
detection and do not require first-principles models. Therefore,
data-driven methods can be applied to large-scale and complex
systems and save time and cost, which is required for the
development of first-principles models [6]. Data-driven methods
are preferred when the product data is available while the system
model is not [7]. In addition; data-driven methods have the ability
to capture information and provide knowledge which is beyond
the engineers’ current knowledge [8]. However, the performance of
data-driven methods is based on the quality and the quantity of the
collected data [6]. A survey of data-driven prognostics methods can
be found in Refs. [10,12]. Data-driven methods can, according to
Fig. 1, be divided into two groups: statistical and non-statistical
methods. Principal component analysis (PCA) [13] and partial least
squares (PLS) [14] are examples of statistical-based data-driven
methods. On the other hand, artificial neural networks (ANN) [15],
self-organized map (SOM) [16], and K-nearest neighbors [17] are
examples of non-statistical data-driven methods. In this work,
principal component analysis was selected as a data-driven
method because it has been successfully used to detect faults in
general, by researchers such as Villegas et al. [18], Tharrault et al.
[19], Russell et al. [20], Kresta et al. [21], and Piovoso et al. [22].
Nowadays, the volume of the generated data is increasing. It is
expected that the volume of generated data will exceed the
available storage [23,24]. Therefore, fault detection methods need
not only achieve high classification accuracy in detecting failures,
but must also be fast enough to identify faults from continuous and
fast-arriving data streams. A number of researchers have discussed
the issue of detecting failures in data streams. A review of such
application can be found in Alzghoul and Löfstrand [23]. Examples
of research concerning data stream mining include [3,23,25–27].
Karcal [26], used a multivariate statistical process monitoring
technique to detect change in the sensor data stream. Kargupta
[25], developed the VEhicle DAta Stream mining (VEDAS) system
for realtime vehicle-health monitoring. The proposed system was
based on DSMS and a data stream mining (DSM) method.
Matthews and Srivastava [27], applied different data-driven
methods on continuous data stream from a solid rocket motor
for anomaly detection.
The authors of this paper have presented a number of related
works [3,23,28,29] and in this paper, we report some verification
and validation activities. In Alzghoul and Löfstrand [23], the
authors investigated the possibility of increasing availability
through the use of data stream mining and DSMS technologies.
The authors reviewed the DSM algorithms and their applications in
monitoring industrial systems. Also, they developed a fault
detection system based on the DSM and the DSMS technologies.
The fault detection system was developed based on three different
DSM classification algorithms: One-class support vector machine
(OCSVM), polygon-based classification algorithm and grid-based
classification algorithm. The developed fault detection system was
tested using data collected from hydraulic motors. The results
showed that the three algorithms achieved good performance in
detecting faults from sensor data streams. In Alzghoul et al. [3], the
authors utilized data stream prediction methods to forecast the
future data stream. In Ref. [3], they developed a fault detection
system based on data stream forecasting, data stream mining and
data stream management systems. The developed fault detection
system was able to predict system faults based on a forecasted data
stream. Furthermore, in Ref. [28] Alzghoul et al., discussed the
potential industrial use of the proposed fault detection system
which, in turn, was presented. Löfstrand et al. [29], proposed a
model for predicting and monitoring industrial system availability
[29]. They suggested using the IF-THEN-ELSE statements, which
are based on FTA, for development of queries for system
monitoring. However, developing and testing such queries was
not done in Ref. [29]. In this work, queries based on FTA and PCA
1128
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
methods were implemented and tested. Thus, the model presented
in Ref. [29] is partly tested and verified which can be considered as
a novelty of this work. Another novelty of this paper is testing and
comparing a data-driven method (PCA), which was not tested in
Refs. [3,23], to a knowledge driven method (FTA) in querying data
streams for monitoring a hydraulic drive system application.
Several authors, as identified above, discuss various fault
detection methods, their advantages and disadvantages. However,
few authors discuss applying specifically knowledge-based and
data-driven methods for searching high-volume data streams. In
addition, to our knowledge, no authors have compared the
performance of different fault detection methods in searching
high-volume data streams.
The authors have worked closely with Bosch Rexroth Mellansel
AB [30] (BRMAB, formerly Hägglunds Drives AB), to such an extent
that their representatives are co-authors of this paper. BRMAB
manufactures low-speed, high-torque hydraulic drive systems and
are interested in improving the availability of their drive systems
through monitoring. In this industrial context we investigated the
functionality and performance of two different fault detection
methods, as reported in Section 6.
The first method, Principal Component Analysis is, in terms of
Zhang and Jiang [5], a data-based quantitative method, while
Chiang et al. [6], consider it a data-driven method. The second
method, Fault Tree Analysis (FTA) is, in terms of Zhang and Jiang
[5], a model-based qualitative method, while Chiang et al. [6],
consider it a knowledge-based method.
Using sensor data collected from a real BRMAB hydraulic drive
system, the aim of this work is to partly validate the model
presented by Löfstrand et al. [29]. The aim is also to test and
compare the functionality and performance when applying a
knowledge-based method (FTA) and a data-driven method (PCA)
to query data streams, with the purpose of fault detection.
Therefore, we developed a fault detection system based on DSMS
technology to test the two fault detection methods. In Section 2
(research method), Fig. 2 shows the process of developing the fault
detection functions, while Fig. 3 in Section 3 (fault detection
system architecture) shows the architecture of the developed fault
detection system. In addition, we discuss and summarize the
advantages and disadvantages of using the knowledge-based and
the data-driven methods (see Section 6).
Results suggested that both methods, when implemented in the
DSMS, are sufficiently fast and can produce good classification
accuracy. As reported in Section 5, it is clear that the two approaches
produce comparable and acceptable results in terms of accuracy and
speed. This is also evident from reviewing Tables 1–5; thus, the
developed models are verified and the model in Ref. [29] is partly
verified. Furthermore, PCA is shown in this paper to be a suitable
data-driven method for searching data streams from a hydraulic
motor. Comparing to previous work by the authors, in Refs. [3,23],
the authors showed that three other data driven methods are
suitable. Furthermore, in Refs. [3,23], the data originated from a
laboratory test (tank test hydraulic drive system) while in this paper,
the data originate in a shredder application in use rather than from a
lab test. From these tests, it is indicated that data driven methods in
general, are suitable and generally successful for searching data
streams. In particular, the authors feel that the four data-driven
methods tested (three in Ref. [3,23] and one in this paper) are now
validated when applied on data from hydraulic drive systems. Also
the requirements for successful industrial implementation of the
two methods were identified and described in Section 6. Furthermore, it was found that the performance of the fault detection
models can be improved by investigating the misclassified data.
Also, it was shown that sensor data can be used to improve the
performance of the knowledge-based method by tuning its model
parameters.
2. Research approach and case study
Bosch Rexroth Mellansel AB (BRMAB) is a Swedish company
which manufactures low-speed, high-torque hydraulic drive
systems. BRMAB hydraulic drive systems are used in industries
including: mining, recycling, pulp and paper and construction.
BRMAB are interested in improving the availability of their drive
systems through system monitoring. The work in this paper relates
to a full-scale BRMAB hydraulic drive system, a shredder
application used to crush waste wood as shown in Fig. 2.
Qualitative data collection was done in collaboration with other
researchers and BRMAB engineers. The data collection involved
semi-structured and open-ended interviews [31] and data analysis
involved using matrices [32]. BRMAB staff developed the baseline
knowledge for the implementation of the applied fault detection
methods, through an iterative process, together with the authors.
The iterative process allowed for collaborative analysis throughout
the process.
The initial specific data collection comprised about three
months while the entire research process, which forms the basis of
this paper, comprised ca. 24 months. The first three month, were
used for setting up the research team as well as the industrial team.
The main research team consisted of the authors; one PhD
researcher and two PhD students, one of which has since been
awarded a PhD (i.e. Alzghoul). The industrial team consisted of two
senior development engineers from Bosch Rexroth Mellansel AB
(Liljedahl and Byström), each of which with about 30 year
experience at the company. Of the industrial representatives,
one (Liljedahl) has been responsible for product development with
respect to the BRMAB hydraulic drive system while the other
(Byström) has been responsible for systems and control development related to the hardware. Both industrial representatives
currently hold positions concerned with long-term development at
BRMAB (see biographies).
Additional industrial representatives were brought in when
further specific knowledge was required and when verification of
pervious finings was carried out. Such instances occurred when:
Fault trees were verified
Cause and effect relationships among monitored parameters
were decided
After various calculations of importance had been carried out,
including heat dissipation in a cooler for example.
Over the course of the herein reported study, about 10 face to face
meetings, meetings using video conference software (on average
Fig. 2. The shredder application.
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
every three weeks, www.alkit.se) and phone meetings (every two
weeks) were carried out.
2.1. Formulating the fault detection models
Sensor data was collected from the BRMAB shredder application to develop and test different fault detection methods being
used to monitor in air–oil cooler functionality in the hydraulic
drive system. Several variables which are associated with the
cooler system functionality were considered.
The process of building data-driven models and knowledgebased models is illustrated in Fig. 3.
The data collection phase (as presented at the top of Fig. 3)
was required to: enable a good understanding of the particular
monitored system, identify which faults are of interest to
monitor by the industrial partner, guide the choice of
parameters to monitor, and to obtain other information which
may be required.
In the case of the data-driven method, if the required
quantitative data is available for it to be used effectively, it likely
does not require all the collected qualitative data. The next step
in the data-driven method was to train the data-driven
algorithm offline using a sample of the collected sensor data.
The training was done using Matlab. Next, the resultant function
was exported and formulated in SCSQL query language [33]
which was implemented in the SuperComputer Stream Query
processor (SCSQ) data stream management system [34]. In the
case of the knowledge-based method, fault tree analysis (FTA)
was performed using the collected data and heuristic knowledge by industrial representatives and researchers in collaboration. Thereafter, the fault detection function was formulated
based on the FTA results. Further descriptions of the applied
methods, as presented in Fig. 2(a) and (b), are presented in
Sections 2.1.1 and 2.1.2.
1129
2.1.1. Fault detection using the knowledge-based method
Before implementing the knowledge-based method (see
Fig. 3b), based on [35], extensive data collection was performed.
Interviews and workshops with a group of BRMAB representatives were conducted to elicit common faults affecting system
reliability [36]. Secondary industrial data, such as technical
specifications and hydraulic schemes, was collected and analyzed
for system understanding [37]. Based on previous faults, found
through interviews, a fault tree analysis (FTA) [38,39] was
performed. The FTA was used to determine elements of the
system that need to be monitored. A list of identified measurement points was made, which was used to detect critical faults
through analyzing the FTA basic events. Thereafter, the structure
and logic of the FTA was used to develop the relationships
between the measurement points; that is, defining causal
relationships between various parameters and parameter sets.
Further information regarding the data collection and analysis
for the knowledge-based method discussed here can be found in
Ref. [29].
2.1.2. Fault detection using the data-driven method (PCA)
In this paper, principal component analysis (PCA) was used as
the data-driven method. PCA produces a lower dimension of the
data set and has the ability to increase the sensitivity of the process
monitoring by applying two different measures, one for monitoring systematic trends of the process subspace and one for random
noise subspace [6]. Principal component analysis is an unsupervised linear dimensionality reduction technique. PCA projects the
data onto the orthogonal directions of maximal variance. The
projection accounting for most of the data variance is called the
first principal component [40]. PCA was used as a fault detection
method as follows:
Let X be data matrix of size n m, where n is the number of data
points and m stands for data dimensionality. X corresponds to data
Fig. 3. Building fault detection function for this work using (a) data-driven method (b) knowledge-based method.
1130
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
which represent the normal behavior of a system, i.e. no faults.
Then, to build a fault detection system using PCA, one can follow
the steps in List 1 (adapted from [41]):
List 1
1. Calculate vectors for the mean (m) and the standard deviation
(s) of X.
2. Normalize X to zero mean and unit variance using the calculated
(m) and (s).
3. Calculate the PCA.
4. Determine the required number of principal components.
5. Calculate the control limits for T2 statistic and Q statistic.
More details for the calculation of PCA, T2 statistic limit, and Q
statistics limit can be found in Ref. [41] or [6]. T2 statistic and Q
statistic are the control limits which will be used for online
monitoring. List 2 shows the steps which can be used to test
whether a new observation x represents a normal or abnormal
(fault) behavior of the monitored system:
List 2
1. Normalize x using the calculated mean (m) and variance (s)
vectors in step (1) in List 1.
2. Calculate the T2 statistic and Q statistic for the normalized
observation x using the obtained PCA model.
3. Check if the calculated T2 statistic and Q statistic are within their
limits, i.e. the control limits calculated in step 5.
4. Consider observation x as fault data if it has T2 statistic violation
or Q statistic violation.
Fault detection based on PCA can be used to search the data
stream using data stream management systems. The five steps in
List 1 can be done offline to obtain the control limits for T2 statistic
and Q statistic. Thereafter, a data stream management system can
be used to do the calculations for testing new observations (in
terms of data stream) according to the steps in List 2.
The control limits T2 statistic and Q statistic can be updated
offline. However, one could also use incremental principal
component analysis (IPCA) [42], so that the fault detection system
can be updated online.
The following section presents the architecture of the fault
detection system which was used to test the knowledge-based
fault detection and the data-driven fault detection functions.
3. Fault detection system architecture
As previously mentioned, some authors have discussed the
issue of detecting failures in high volume data streams from
equipment in use. Alzghoul and Löfstrand [23], Alzghoul et al. [3],
and Kargupta [25], found that data stream management system
(DSMS) technology has the potential to support scalability issues.
DSMSs have the ability to manage data streams and apply
continuous queries over input data streams. DSMS technology
can be used to implement and test different fault detection
methods in searching high-volume data streams. Therefore, the
developed fault detection system is based on the DSMS technology.
In order to test the data-driven and knowledge-based models, we
used the architecture of the fault detection system illustrated in
The two different fault detection functions (i.e. based on PCA and
FTA) were developed as illustrated in Fig. 3. The fault detection
functions are formulated into code using the DSMS query language. The
DSMS is responsible for controlling, managing, and applying continuous
queries over input data stream [25]. In this paper, the SuperComputer
Stream Query processor (SCSQ) [34] was used as the DSMS. If the
function output indicates a fault occurrence, then an alert is activated.
Fig. 4 shows that there are two phases for the fault detection
system: the online and offline phases. Building the fault detection
function and formulating the mathematical functions as code are done
in the offline phase, while receiving data stream from the monitored
system and applying the fault detection function on the incoming
data stream are done in the online phase using DSMS. However,
updating the fault detection system can be done either offline or online
based on which fault detection methods and algorithm are used. For
instance, PCA can be updated offline by rebuilding the PCA model
using recent data. On the other hand, it can be updated online through
the use of incremental PCA (IPCA) algorithm [42].
4. The data set
The example query developed and used in this paper concerned
fault detection in air–oil cooler, part of a hydraulic drive system
Fig. 4. The architecture of the fault detection system.
1131
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
powering the shredder application. Several variables which are
associated with the cooler system functionality were considered.
The cooler fan in the system is activated for different periods of
time, depending on ambient temperature and system load. The
cooler monitoring process will be activated when the cooler fan is
switched on, whenever cooling is needed. Consequently, the part of
the collected data representing when the cooler fan was switched
on was used for training and testing. The data was collected once
every 0.1 s for two days (06.00–14.00 day one and 06.00–18.00 day
two). The data set includes both normal and abnormal data from
the monitored system. The faulty data was created by BRMAB
engineers in collaboration with the authors. The two failure modes
which were practically simulated are: fully clogged cooler and half
fully clogged cooler. The faulty data was practically created; that is,
the data from different sensors was collected during the time the
hydraulic drive system was running in the two failure modes.
In order to build the data-driven model, i.e. applying the datadriven method, training data was required. Therefore, a sample of
data was used to train and test the data-driven method (PCA). The
normal data was arbitrarily selected; one sample from day one and
one sample from day two. The abnormal data represented two
simulated failure modes. The sample from the two days contained
4146 data points which correspond to normal behavior and 13,986
data points which correspond to abnormal data. Training PCAbased fault detection requires only normal data points, as
mentioned in Section 2.1.2. However, a test was needed to
examine the performance of the PCA algorithm. We trained and
tested the algorithm using different training data set sizes (20% and
50% of the total normal data). The selection of the best PCA model
was based on the classification accuracy of the remaining normal
data points, i.e. 80% and 50% of 4146, and all abnormal data set
(13,986 data points). In the case of the knowledge-based method,
all data points (normal and abnormal) were used for testing, as the
knowledge-based method does not require training.
5. Results
This section presents, compares and discusses the results of
testing the data-driven model (PCA) and the knowledge-based
model (FTA).
5.1. Results from testing the data-driven fault detection method
To test the data-driven model, PCA was first trained using
normal data points as discussed in Section 4. Repeated random
sub-sampling validation was used to assess the accuracy of the PCA
model. Table 1 shows the accuracy of the PCA model at three
different tests when 50% of the normal data points (50% out of
4146 = 2073) was used as training, while the rest was used for
testing, i.e. 2073 normal data points and 13,986 abnormal data
points.
Table 1 shows that PCA was able to achieve high classification
accuracy in detecting abnormal data in three different tests. PCA
was able to detect on average 13,983 out of 13,986 abnormal data
points, achieving 99.98% classification accuracy. PCA achieved less
accuracy, 95.85% on average, in classifying normal data points.
Thereafter, we decreased the number of training data points, as
it is faster to train the PCA algorithm using the smaller data set. A
smaller training data set size (20% of the normal data) was used to
assess the performance of the PCA algorithm. Table 2 shows the
performance of PCA when 20% of the normal data points (20% out
of 4146 = 829) was used for training.
Table 2 shows that PCA achieved high classification accuracy in
detecting abnormal data, even if the number of training data points
was reduced. Since the training data set was decreased and the PCA
model was still achieving good classification accuracy, the
Table 1
Accuracy of the PCA model at three different tests when 50% of the normal data
points (2073 data points) was used for training.
Classification accuracy
of normal data
Classification accuracy
of abnormal data
Test 1
Test 2
Test 3
95.18%
96.14%
96.24%
99.98%
99.98%
99.97%
Average
1987/2073 = 95.85%
13,983/13,986 = 99.98%
Table 2
Accuracy of the PCA model at three different tests when 20% of the normal data
points (829 data points) was used for training.
Classification accuracy
of normal data
Classification accuracy
of abnormal data
Test 1
Test 2
Test 3
97.29%
96.20%
95.99%
99.96%
99.98%
99.97%
Average
3200/3317 = 96.47%
13,982/13,986 = 99.97%
potential problem of overfitting was proven to be a non-issue.
When comparing the three tests in Tables 1 and 2, we noticed that
Test 1 in Table 2 achieved the best performance in classifying
normal data points and almost similar results in classifying
abnormal data points. Therefore, the PCA model, which was built
using the training data in Test 1 in Table 2, was used for testing all
collected data (06.00–14.00 day one and 06.00–18.00 day two).
5.2. Results from testing the knowledge-based fault detection method
As mentioned previously, knowledge-based methods do not
require training. The knowledge-based method was tested using all
normal data points (4146) and abnormal data points (13,986).
Therefore, only one test was applied, i.e. all data was used as testing
data set, unlike the data-driven method where the result involved
applying different training and testing data sets. Table 3 presents the
classification accuracy of the applied knowledge-based method.
Table 3 shows that the knowledge-based method achieved high
classification accuracy of abnormal data and good performance in
classifying normal data. In terms of performance, the applied
knowledge-based method and data-driven method are quite
similar when comparing the results obtained by the knowledgebased method and results obtained by PCA. The result obtained by
PCA was slightly better than the results obtained by the
knowledge-based method.
Tuning parameters or thresholds which are involved in
the knowledge-based method may change the results. Therefore,
Table 3
Knowledge-based method classification accuracy.
Knowledge-based method
(before tuning)
Classification accuracy
of normal data
Classification accuracy
of abnormal data
3976/4146 (95.90%)
13,977/13,986 (99.94%)
Table 4
Knowledge-based method classification accuracy before and after tuning.
Knowledge-based method
(before tuning)
Knowledge-based method
(after tuning)
Classification accuracy
of normal data
Classification accuracy
of abnormal data
3976/4146 (95.90%)
13,977/13,986 (99.94%)
4116/4146 (99.28%)
13,973/13,986 (99.91%)
1132
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
the knowledge-based method was tested by trying different
thresholds. It was found that tuning the threshold increased the
classification accuracy of normal data points from 95.90% to 99.28%
and slightly reduced the classification accuracy of abnormal data
points from 99.94% to 99.91%, as shown in Table 4.
5.3. Investigating misclassified data
To improve the performance of both methods the misclassified
data was investigated. It was found that the misclassified data
corresponded to the first few seconds after the cooler was switched
on. To overcome this problem, one can either skip the first few
seconds of data or make a condition that the system give an alert
when faulty data is present for more than a specific period of time,
e.g. faulty data occurring continuously for more than 60 s. The
second alternative was selected in this work where the time
threshold was 15 s. The second alternative is also useful for
eliminating outliers, if they exist. A time threshold of 15 s will not
affect the system negatively, since it heats much slower than that,
even during high power usage. Once the cooler fan switches on, the
temperature of the oil from the cooler reaches a steady-state value
after some time. This time is dependent on the oil volume of the
cooler and the oil flow passing through.
5.4. Results of testing all data
The final test was performed using the alert condition with time
threshold 15 s and using all collected data (data collected during two
days). Fault detection functions developed using knowledge-based
and data-driven methods were tested using the proposed fault
detection system illustrated in Fig. 4. It was found that both methods
were able to detect all faults without any false alert. Also, both
methods were adequate to use online, i.e. they can be used to search
the data stream, since they are faster than the arriving data stream.
6. Discussion
The developed models were tested using a data set containing
both faulty and non-faulty data points. The time when faults
appear as well as when faults disappear in the data set, are known
(i.e. the data-label is known). Comparing the outputs from the
developed models with the pre-known data-label showed
relatively speaking, good accuracy, as shown in Section 5, thus
verifying the developed models.
The results showed that both the knowledge-based and the
data-driven method achieved high classification accuracy and
were applicable for searching the data stream. However, through
literature and based on development, testing and the results of
both methods, several differences were noticed. Table 5 shows a
summary of these differences.
Rows 1–8 of Table 5 have been concluded (by the authors) to be
true. Whereas, row 9 i.e. Area of application, is based on the
research by other researchers [6,7,44,45]. The results based on the
BRMAB case, show the ability of both methods to query data
streams. Note that, the ability of the knowledge-based method to be
applied as queries, using a data stream management system to detect
faults in high volume data streams, has not been found extensively in
literature based on the authors review. Table 5 shows that building
and updating data-driven models requires less time and cost than
building and updating knowledge-based models, these findings are
also supported by other researchers in Refs. [6,43–45]. For
example, Chiang et al. [6], Bratina et al. [45], and Das et al. [44]
agree that data-driven methods are easy to develop and implement. Data-driven methods have the ability to provide new
information which may not be presented in knowledge-based
methods, which are solely based on heuristic knowledge. Such new
information may include new types of out-of-bounds data sets [8].
Chih-Min and Yun-Pei [8] and Lou et al. [46] discussed that, in
some situations, knowledge-based methods cannot find new
faults. The authors of this paper would agree with Chin-Min and
Yun-Pei that the cause and effect relations are of great importance
for creating the knowledge-based models. Data-driven methods
can be easily adapted to the changes in the monitored system, for
example, by using incremental principal component analysis
(IPCA) instead of, or in conjunction with, PCA. According to Luo
et al. [46], and supported by results as presented in this paper,
incorporating a level of adaptiveness in a knowledge-based
method is harder to accomplish than in a data-driven method.
The accuracy of the data-driven methods (and the accuracy of
knowledge-based methods) depends on the applied algorithm;
thus, the results may vary from one algorithm to another. For
example, Alzghoul and Löfstrand [23] tested different data
stream mining algorithms and the algorithms achieved different
results. Building a data-driven model does not require specific
engineering knowledge. However, data-driven models cannot be
built without data sets. According to Chiang et al. [6] and ChihMin and Yun-Pei [8], and Das et al. [44], data-driven methods
need only data and do not require deep knowledge about the
monitored system.
On the other hand, knowledge-based methods achieved good
performance in detecting pre-known faults as shown in this paper
and discussed by Chiang et al. [6] and Chih-Min and Yun-Pei [8].
Also, engineering knowledge can be used to build the fault
detection model without the need of a data set [8]. However, as
shown in this work, sensor data can be used to improve the
performance of the fault detection model by, for example, tuning
model parameters as it was shown in Section 5.2.
Table 5
Comparison between knowledge-based and data-driven methods, results are based on literature and BRMAB case.
Knowledge-based method
Data-driven method
Development cost
Development time
Fault detection accuracy
High [43]
Long [43]
High [6,8]
Adaptiveness
Time and cost expensive
compared to the data-driven
method [46]
Not necessary [6,8]
High
No [8,46]
Yes [6,8,47]
Can be used for both small
and large-scale systems using
software packages [6,7]
Cheap [6,44,45]
Short [6,44,45]
Based on the algorithm [23] and the quantity
and quality of data [6]
Based on [6,44,45], plausibly cheap and fast
Data set availability
Speed and ability to query data streams
Ability to detect new information
Require deep system-specific knowledge
Area of application
Necessary [6–8,44,47]
Generally high (depending on the algorithm) [23]
Yes [8]
No [6,8,44]
Can be used for both small and large-scale
systems [6,7,44,45]
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
Clearly, the quality of knowledge based methods in general is
dependent on the validity and reliability on the information
from the industrial informants, which in these cases was very
good. One might conclude that any knowledge based method
when used is capable of correctly capturing the relevant
information from the informants, would likely generate comparable quality results. On the other hand, the performance of datadriven methods in general (in addition to the data-driven
methods tested in Refs. [3,23] and in this paper), is based on the
quality and the quantity of the collected data [6]. Also, the
impact of the input variables on the target affects the
performance of the data-driven methods. Furthermore, some
of the data-driven methods such as support vector machine take
the advantage of using nonlinear kernels to map the data to a
very high dimensional space. For example, it was shown in Ref.
[23] that the one-class support vector machine outperformed
the polygon-based and grid-based methods because it maps the
data into a higher dimension.
Every fault detection technique has its own advantages and
disadvantages. Therefore, it could be beneficial to combine
different techniques to bring their advantages together. For
example, one could achieve the advantages of knowledge-based
methods in detecting known faults, and data-driven methods
for detecting new faults, thereby achieving a better understanding of the system by combining the two techniques. In any
case, we show that the accuracy of the knowledge-based
method is for all intents and purposes the same as that of the
data-driven method. This is likely because the interviewees and
participating company partners have an exceptional system
understanding, and it may also be due to the chosen research
method. The high accuracy of the knowledge-based results is
like due to:
The hydraulic theory, on which the queries are based, was wellknown for the authors.
o However, to implement the theory correctly, good system
knowledge and experience are required
BRMAB has exceptional system knowledge and experience, since
the company has been in business over 100 years and the
engineers in charge have worked over 30 years.
The authors have some experience regarding hydraulic
theory and furthermore, the research method was iterative
in that many development and feedback loops were carried
out with researchers and industrial representatives collaboratively.
The data-driven accuracy is high since it is based on the PCA
model which is sensitive to the abnormal behavior and had been
successfully applied in the monitoring of complex systems. In
addition, the quality and quantity of the collected data was good
for the authors to be able to produce a data-driven model with high
performance. (As previously stated, the performance of datadriven methods is based on the quality and the quantity of the
collected data [6].) In more detail, the data was ‘‘good’’ in the sense
that the authors were aware when, i.e. at what time stamp, the
technical system was manipulated based on common faults like a
clogged cooler, to create sensor data corresponding to such faults.
Furthermore, the system manual from BRMAB states the permitted
ranges of oil temperatures for example, which is true for all
relevant variable ranges. In addition, the quantity of data used for
training clearly was enough since the accuracy turned out to be
relatively high (i.e. For example on average 99.97% for the
abnormal data, see Table 2).
The results suggest that the developed models are applicable in
monitoring the cooler functionality of hydraulic drive systems. The
performance of the developed models may also indicate a possible
1133
success in monitoring other hydraulic drive functions and even
other product functions.
In general, it is plausible that the two compared approaches for
fault detection can be used successfully in industry, when the
following conditions are met.
The knowledge-based method requires:
sensor data to tune the knowledge-based model parameters
a known data-label for model verification (i.e. our knowledge
regarding when faults appear and disappear).
good system knowledge (provided here by BRMAB personnel)
knowledge regarding normal and abnormal behavior among the
various monitored system parameters
that the technical system allow for formulating the continuous
queries (i.e. ‘‘rules’’); therefore, too complex systems may need to
be simplified
The data-driven method requires:
a sample of sensor data (with a known data-label) for training
a sample of faulty data for verification and/or training
brief knowledge about the monitored system and most common
faults that will give a good understanding of the achieved results
and possibly increase the accuracy (for example, investigating
misclassified data)
a good implementation of the data-driven algorithm that may
affect both speed and accuracy
The two verified models can be used to monitor industrial
systems online, thereby increasing availability and saving maintenance time and cost. The two verified models are useful for
monitoring a fleet of units or a single high-value unit.
7. Conclusions
As discussed previously, increasing availability of products is of
great interest for industrial companies. Availability of industrial
products can be improved by fault detection and diagnosis. Fault
detection and diagnosis methods can be divided into three
categories: data-driven, analytical-based, and knowledge-based
methods.
In this work, monitoring cooler functionality of BRMAB
hydraulic drive systems was investigated. Two fault detection
techniques, i.e. data-driven (PCA) and knowledge-based (FTA)
methods, were designed, implemented and tested. The performance of the two methods and their ability to search the data
stream were compared. Also, the advantages and disadvantages of
both methods were discussed.
It was found that both methods achieved good performance in
detecting faults (around 99% classification accuracy of abnormal
data) and both methods are fast enough to be used for querying the
data stream. The performance of the fault detection models was
improved by investigating the misclassified data. Also, it was
shown that sensor data can be used to improve the performance of
the knowledge-based method by tuning its model parameters.
The data-driven methods surpassed the knowledge-based
methods in model development time and cost, they are easy to
retrain and able to detect faults beyond the engineers’ knowledge.
On the other hand, the knowledge-based models can be developed
without sensor data and still achieve high accuracy in detecting
historical faults, provided that they are based on accurate engineer
(i.e. informant) knowledge. Hence, both approaches may, if the
conditions discussed above are met, be used to predict and identify
faults in hydraulic drive systems or comparable applications in use.
In terms of the product development process, if the methods are
used in the testing and refinement stage, their use may improve
1134
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
the associated prototypes and may thereby improve the quality of
the product at an earlier stage of development. Naturally, the two
approaches may be used in the late stage of a product lifecycle to
improve the design of the next updated product version.
Acknowledgments
The authors wish to acknowledge the following three organizations: The EU FP7 Project SmartVortex, The Faste Laboratory at
Luleå University of Technology funded by the Swedish Governmental Agency for Innovation Systems (VINNOVA: 2012-00705)
and the SSPI project (Scalable search of product lifecycle
information) funded by the Swedish Foundation for Strategic
Research (SSF: RIT08-0041).
References
[1] D.G. Ullman (Ed.), The Mechanical Design Process, 4th ed., McGraw-Hill Science,
New York, 2010.
[2] I. Alsyouf, A. Alzghoul, Soft computing applications in wind power systems: a
review and analysis, in: European Offshore Wind Conference and Exhibition,
Stockholm, Sweden, 2009.
[3] A. Alzghoul, M. Löfstrand, B. Backe, Data stream forecasting for system fault
prediction, Computers and Industrial Engineering 62 (4) (2012) 972–978.
[4] V. Venkatasubramanian, R. Rengaswamy, K. Yin, S.N. Kavuri, A review of process
fault detection and diagnosis part I: quantitative model-based methods, Computers and Chemical Engineering 27 (3) (2003) 293–311.
[5] Y. Zhang, J. Jiang, Bibliographical review on reconfigurable fault-tolerant control
systems, Annual Reviews in Control 32 (2) (2008) 229–252.
[6] L.H. Chiang, E. Russell, R.D. Braatz, Fault Detection and Diagnosis in Industrial
Systems, Springer Verlag, London, 2001.
[7] C. Sankavaram, B. Pattipati, A. Kodali, K. Pattipati, M. Azam, S. Kumar, M. Pecht,
Model-based and data-driven prognosis of automotive and electronic systems, in:
IEEE International Conference on Automation Science and Engineering, 2009.
CASE 2009, 2009, 96–101.
[8] F. Chih-Min, L. Yun-Pei, A Bayesian framework to integrate knowledge-based and
data-driven inference tools for reliable yield diagnoses, in: Simulation Conference, 2008. WSC 2008. Winter, 2008, 2323–2329.
[9] D. Zhu, S. Yu, Survey of knowledge-based fault diagnosis methods, Anhui Gongye
Daxue Xuebao (Journal of Anhui University of Technology), 19 (3) (2002) 197–204.
[10] V. Venkatasubramanian, R. Rengaswamy, S.N. Kavuri, K. Yin, A review of process
fault detection and diagnosis: part III: process history based methods, Computers
and Chemical Engineering 27 (3) (2003) 327–346.
[11] J.B. Fussell, G.J. Powers, R.G. Bennetts, Fault trees – a state of the art discussion,
IEEE Transactions on Reliability R-23 (1) (1974) 51–55.
[12] M.A. Schwabacher, A Survey of Data Driven Prognostics, AIAA Infotech@Aerospace, 2005.
[13] K. Pearson LIII, On lines and planes of closest fit to systems of points in space, The
London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2
(11) (1901) 559–572.
[14] S. Wold, C. Albano, W. Dunn III, U. Edlund, K. Esbensen, P. Geladi, S. Hellberg, E.
Johansson, W. Lindberg, M. Sjöström, Multivariate data analysis in chemistry, in:
Chemometrics, Springer, 1984, pp. 17–95.
[15] W.S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous
activity, Bulletin of Mathematical Biophysics 5 (4) (1943) 115–133.
[16] T. Kohonen, Self-organized formation of topologically correct feature maps,
Biological Cybernetics 43 (1) (1982) 59–69.
[17] B.V. Dasarathy, Nearest Neighbor ({NN}) Norms:{NN} Pattern Classification
Techniques, 1991.
[18] T. Villegas, M.J. Fuente, M. Rodrı́guez, Principal Component Analysis for Fault
Detection and Diagnosis. Experience with a pilot plant, 2010, 147–152.
[19] Y. Tharrault, G. Mourot, J. Ragot, Fault Detection and Isolation with Robust
Principal Component Analysis, 2008, 59–64.
[20] E.L. Russell, L.H. Chiang, R.D. Braatz, Fault detection in industrial processes using
canonical variate analysis and dynamic principal component analysis, Chemometrics and Intelligent Laboratory Systems 51 (1) (2000) 81–93.
[21] J.V. Kresta, J.F. Macgregor, T.E. Marlin, Multivariate statistical monitoring of
process operating performance, The Canadian Journal of Chemical Engineering
69 (1) (1991) 35–47.
[22] M. Piovoso, K. Kosanovich, R. Pearson, Monitoring process performance in realtime, IEEE (1992) 2359–2363.
[23] A. Alzghoul, M. Löfstrand, Increasing availability of industrial systems through data
stream mining, Computers and Industrial Engineering 60 (2) (2011) 195–205.
[24] Data, data everywhere (a special report on managing information), The Economist
2010.
[25] H. Kargupta, VEDAS: a mobile and distributed data stream mining system for realtime vehicle monitoring, in: SIAM Proceedings Series, 2004, pp. 300–311.
[26] S.C. Karacal, Mining machine data streams using statistical process monitoring
techniques, in: IIE Annual Conference and Expo, 2007, 1290–1297.
[27] B. Matthews, A.N. Srivastava, Comparative analysis of data-driven anomaly
detection methods, in: JANNAF Conference on Propulsion Systems, Proceedings of the Joint Army Navy NASA Air Force Conference on Propulsion,
Orlando, FL, 2008.
[28] A. Alzghoul, M. Löfstrand, L. Karlsson, M. Karlberg, Data stream mining for
increased functional product availability awareness, in: J. Hesselbach, C. Herrmann (Eds.), Functional Thinking for Value Creation, Proceedings of the 3rd CIRP
International Conference on Industrial Product Service Systems, Technische
Universität Braunschweig, May 5th–6th, Springer Berlin Heidelberg, Braunschweig, Germany, 2011, pp.237–241.
[29] M. Löfstrand, B. Backe, P. Kyösti, J. Lindström, S. Reed, A model for predicting and
monitoring industrial system availability, International Journal of Product Development 16 (2) (2012) 140–157.
[30] Boschrexroth.com, (2014). Bosch Rexroth. The Drive & Control Company. - Bosch
Rexroth AG. Available at: http://www.boschrexroth.com/ [Accessed Jun. 2014].
[31] M.Q. Patton, Qualitative Evaluation and Research Methods, Sage, London, UK,
1990.
[32] M. Miles, M. Huberman, An Expanded Sourcebook – Qualitative Data Analysis,
2nd ed., Sage, Thousand Oaks, CA, 1994.
[33] E. Zeitler, T. Risch, Using stream queries to measure communication performance
of a parallel computing environment, IEEE (2007) 65.
[34] E. Zeitler, T. Risch, Processing high-volume stream queries on a supercomputer,
in: IEEE Proceedings of the 22nd International Conference on Data Engineering
Workshops, 2006.
[35] R. Isermann, Fault-Diagnosis Applications Model-Based Condition Monitoring:
Actuators, Drives, Machinery, Plants, Sensors, and Fault-tolerant Systems, Springer Berlin Heidelberg, Germany, 2011.
[36] R.K. Yin, Case Study Research, 4th ed., Sage publications, 2008.
[37] M.B. Miles, A.M. Huberman, Qualitative Data Analysis: An Expanded Sourcebook,
2nd ed., SAGE Publications, Thousand Oaks, CA, 1994.
[38] J.D. Andrews, T.R. Moss, Reliability and Risk Assessment, Professional Engineering Publishing Limited, Bury St Edmunds, 2002.
[39] NUREG-0492, Fault Tree Handbook, Nuclear Regulatory Commission, Washington, 1981.
[40] J. Hu, J. Si, B.P. Olson, J. He, Feature detection in motor cortical spikes by principal
component analysis, IEEE Transactions on Neural Systems and Rehabilitation
Engineering 13 (3) (2005) 256–262.
[41] D. Garcıa-Alvarez, Fault detection using Principal Component Analysis (PCA) in a
Wastewater Treatment Plant (WWTP), in: 62th Int. Student’s Scientific Conf.,
Saint-Petersburg, 2009.
[42] H.V. Neto, U. Nehmzow, Incremental PCA: An Alternative Approach for Novelty
Detection, Towards Autonomous Robotic Systems, 2005.
[43] V. Uraikul, C.W. Chan, P. Tontiwachwuthikul, Artificial intelligence for monitoring
and supervisory control of process systems, Engineering Applications of Artificial
Intelligence 20 (2) (2007) 115–131.
[44] A. Das, J. Maiti, R.N. Banerjee, Process monitoring and fault detection strategies: a
review, International Journal of Quality and Reliability Management 29 (7) (2012)
720–752.
[45] B. Bratina, N. Muškinja, B. Tovornik, Design of an Auto-associative Neural Network
by Using Design of Experiments Approach, Neural Computing and Applications 19
(2) (2010) 207–218.
[46] J. Luo, S. Ghoshal, A. Mathur, K.R. Pattipati, Adaptive maintenance knowledge
bases for field service, in: 2007 IEEE Aerospace Conference, Big Sky, MT, (2007),
pp. 1–11.
[47] H. Wang, T.-Y. Chai, J.-L. Ding, M. Brown, Data driven fault diagnosis and fault
tolerant control: some advances and possible new directions, Acta Automatica
Sinica 35 (6) (2009) 739–747.
Ahmad Alzghoul is currently a researcher at Uppsala
University, Department of Information Technology,
Division of Computing Science. He has an educational
background in the field of computer science and
engineering. He received a Ph.D. in Computer Aided
Design at Luleå University of Technology, Division of
Product and Production Development, Sweden. He
received his M.Sc. degree in Computer Engineering
(Intelligent Systems) from Halmstad University,
Sweden, and his M.Sc. degree in Software Engineering
from Linnaeus University, Sweden. His main research
interests include data mining and industrial data
mining applications.
A. Alzghoul et al. / Computers in Industry 65 (2014) 1126–1135
Björn Backe (M.Sc.) is a Ph.D. student in Computer
Aided Design at Luleå University of Technology,
Division of Product and Production Development. His
main research interests include hardware reliability.
Magnus Löfstrand has an educational background in
mechanical engineering and was awarded his Ph.D. in
Computer Aided Design (CAD) at Luleå University of
Technology, Sweden in 2007. After serving as assistant
professor in CAD, he is now employed as a senior
researcher at Uppsala University, Department of
Information Technology, Division of Computing Science, Uppsala DataBase Laboratory. He has an academic
background in the work process description, refinement and simulation based on product development
literature. He also has experience of research concerning, and equipment management for, distributed
collaboration over IP networks. He is involved in
research concerning system availability (maintainability and reliability) and the
use of DSMS (Data Stream Management System) and DSM (Data Stream Mining) in
engineering applications, often in the context of enabling larger industrial service
content in industrial systems.
1135
Arne Byström, B.Sc., has experience from working in
the Swedish high technological industry, mainly within
R&D, for more than 37 years. He previously, for over ten
years, served as manager for customer control system
order development. He has good knowledge of elliciting
and interpreting customer needs and has experience
from control related problem solving in customer
applications around the world. Today, he works for
Bosch Rexroth Mellansel AB as Technical Product
Manager Systems, in the application area of electrohydraulic drive control and drive monitoring systems.
Bengt Liljedahl has a B.Sc. in Mechanical Engineering
from 1973 and a diploma in Mechanical Engineering
at University level, received in 1978. He has over 38
years of experience from working in Swedish Industry, mainly within R&D. He currently serves as
Technical Product Manager at Bosch Rexroth Mellansel AB (former Hägglunds Drives AB). He has since
1989 coordinated and built a network of close
cooperation with Swedish Universities to increase
the competence of Bosch Rexroth Mellansel AB in 3
main areas, Tribology, Product Development and
Material Science. For his longstanding and successful
research collaboration, he was awarded an Honorary
Doctorate at Luleå University of Technology in June of 2009.