An Extension to Deng’s Entropy in the Open World Assumption with an Application in Sensor Data Fusion

Quantification of uncertain degree in the Dempster-Shafer evidence theory (DST) framework with belief entropy is still an open issue, even a blank field for the open world assumption. Currently, the existed uncertainty measures in the DST framework are limited to the closed world where the frame of discernment (FOD) is assumed to be complete. To address this issue, this paper focuses on extending a belief entropy to the open world by considering the uncertain information represented as the FOD and the nonzero mass function of the empty set simultaneously. An extension to Deng’s entropy in the open world assumption (EDEOW) is proposed as a generalization of the Deng’s entropy and it can be degenerated to the Deng entropy in the closed world wherever necessary. In order to test the reasonability and effectiveness of the extended belief entropy, an EDEOW-based information fusion approach is proposed and applied to sensor data fusion under uncertainty circumstance. The experimental results verify the usefulness and applicability of the extended measure as well as the modified sensor data fusion method. In addition, a few open issues still exist in the current work: the necessary properties for a belief entropy in the open world assumption, whether there exists a belief entropy that satisfies all the existed properties, and what is the most proper fusion frame for sensor data fusion under uncertainty.


Introduction
Uncertain information processing plays a key role in complex systems of many fields such as sensor networks [1,2], pattern recognition [3,4], decision-making [5,6], supply chain network management [7,8], complex network [9] and target tracking [10,11]. Uncertain information may come from sensors with different credibilities and experts's subjective judgement. The heterogeneous sources and reliable degree increase the complexity and uncertainty of information process. The Dempster-Shafer evidence theory (DST) [12,13] has a promising efficiency in uncertain information processing such as information fusion [14,15]. However, there are still a few open issues in the DST framework that need further study. Firstly, the approaches of managing the conflicting belief masses still needs further refining [16,17]. Secondly, the reasonable ways of generating the mass functions for the practical applications [18,19]. Thirdly, uncertainty quantification with the possible measures in the DST framework [20,21], and the necessary properties a new belief entropy should obey [22][23][24]. Fourthly, rules of combining the body of evidence vary under different circumstances [25][26][27]. Inspired Definition 1. Assume that Ω= {θ 1 , θ 2 , . . . , θ i , . . . , θ N } is a nonempty set with N mutually exclusive and exhaustive events, Ω is the frame of discernment (FOD). The power set of Ω consists of 2 N elements denoted as follows: If m (A) > 0, then A is called a focal element. m (A) indicates the support degree of the evidence on the proposition A.

Definition 3.
A body of evidence (BOE), also known as a basic probability assignment (BPA) or basic belief assignment (BBA), is defined as the focal sets and the corresponding mass functions: where is a subset of the power set 2 Ω .
where k is a normalization factor defined as follows: It should be noted that the classical definitions of DST are defined in the closed world. In the open world assumption, Dempster's rule of combination is extended and named as the generalized combination rule (GCR) by Deng in [30]. Definition 6. In [30], the fusion result of two empty sets is defined as ∅ 1 ∩ ∅ 2 = ∅, which means that the intersection between two empty sets is still an empty set. Given two BPAs (m 1 and m 2 ), the generalized combination rule (GCR) is defined as follows:

Shannon Entropy
As the information entropy for uncertainty measure, Shannon entropy has been applied and generalized in many areas such as complexity network [46][47][48].
Definition 7. Shannon entropy is defined as [49]: where N is the number of basic states, p i is the probability of state i, p i satisfies If the unit of information is bit, then b = 2. In this case, Shannon entropy is:

Deng Entropy
As an extension of Shannon entropy in the framework of DST, Deng entropy is proposed in [21]. Some properties and behaviors are discussed in [21,24]. The application of Deng entropy can be found in [45,50].

Definition 8.
In FOD X, Deng entropy, denoted as E d , is defined as: where |A| denotes the cardinality of the proposition A.
According to [21], the Deng entropy has some advantages in some cases in comparison with some other uncertainty measures in Table 1. Table 1. Uncertainty measures in DST framework.

Uncertainty Measure Definition
Hohle's confusion measure [32] However, Equation (10) will be unavailable if |A| =0. Thus, the uncertainty measure in the closed world of the DST framework should be extended. In the open world assumption [19,30,52,53], the uncertain information represented by the nonzero mass function of the empty set and the incomplete FOD should be handled properly and cautiously.

New Uncertainty Measure in the Open World
In the DST framework, the uncertain information is modelled not only by mass functions, the FOD is also an important source of uncertainty [40]. In addition, in the open world assumption, the mass value of the empty set may not be zero, which also indicates the incompleteness of the FOD [30]. With this background, how to measure the uncertain degree in the open world assumption of the DST framework is a new perspective and an important issue. According to literature review, no existing uncertainty measure addresses this problem, which is the reason for this work. Example 1. Consider a set of BPAs with the FOD X = {a, b}, the mass functions with nonzero mass value of the empty set: It is obvious that the Deng entropy E d in Equation (10) is not available for the uncertainty measure of BPAs in this case. The denominator of the log function with respect to m (∅) = 0.2 will be (2 0 − 1 = 0), which is illegal. This is because the Deng entropy is only based on the mass function of the focal element and the cardinality of the corresponding proposition. In the open world assumption, the mass value of empty set may not be zero. In addition, how to define the element number in incomplete FOD is also an open issue. The same question also exists in other uncertainty measures listed in Table 1. The works in [37,38,40,42] also pay no attention to the possible nonzero mass function of the empty set as well as the possible incomplete element number in the FOD. A new uncertainty measure which is extended from the Deng entropy in the closed world, named the extension to Deng's entropy in the open world assumption, is proposed especially for the problems mentioned above.

An Extension to Deng's Entropy in the Open World Assumption
where |A| is the cardinality of the proposition A, X is the FOD, |X| denotes the certain element number in the FOD, m (∅) |X| is proposed to denote the uncertain element number in the FOD with respect to the corresponding proposition (A). ' ' is the symbol of the ceiling function, which means the smallest integer that is no smaller than the independent variable, e.g., 0.3 =1.
The extended measure addresses three parts of uncertainty in the DST framework, including the uncertain information expressed by the mass functions of focal elements, the mass function of the empty set and the possible incompleteness of the FOD. In detail, inspired by the existed uncertainty measures and the Deng entropy, the EDEOW handles two aspects of uncertainty according to the following methods:

•
In the closed world where m (∅) =0, the uncertainty represented by the mass function m (A) of the focal element as well as the corresponding cardinality |A|. • In the open world where m (∅) = 0, the nonzero mass function m (∅) of the empty set can be an indicator of the completeness or incompleteness of the FOD; currently, m (∅) |X| is chosen to express this uncertainty.
It should be noted that, in the EDEOW defined in Equation (12), the proposition A is no longer limited as a traditional focal element, it can also be an empty set ∅ which means uncertainty in the FOD [30]. In addition, apart from the m (∅) |X| , there must exist many types of expressions to express the incompleteness of the FOD. (11), with the EDEOW, the uncertainty degree of the BPAs can be calculated as follows:

Recall the BPAs in Equation
With the proposed EDEOW, the problem in Example 1 can be handled. The BPAs with a nonzero mass function of the empty set can be handled now with the extended measure.

Numerical Example and Discussion
Example 2. In FOD X = {a}, the mass functions are: According to the BPAs in Equation (14), the mass value of the empty set is 0, which indicates the BPAs are assigned in the closed world. The uncertain degree with Shannon entropy H, Deng entropy E d and the EDEOW E edeow can be calculated respectively as follows: Obviously, the mass function m ({a}) = 1 assigns a belief of 100% on the proposition {a}, which means the uncertain degree of the proposition is 0. In this case, the measuring result of the EDEOW is consistent with that of Shannon entropy and Deng entropy.
The mass value of the empty set is 0, the BPAs are assigned in the closed world. The uncertain degree measured by H, E d and E edeow can be calculated respectively as follows: According to the measuring results shown in Equations (15) and (17), if a mass function is assigned on the single subset, then the EDEOW can be degenerated to Deng entropy in the closed world. More importantly, the EDEOW satisfies the property of probabilistic consistency if the BPAs are only assigned on the single subset in the closed world. It should be noted that Shannon entropy and Deng entropy are not available if the BPAs are assigned in the open world where the mass value of the empty set is nonzero; as is shown in Example 1 and the following Example 4.

Example 4.
In a changing FOD |X|, consider the mass functions given as follows: The mass value of the empty set is 0.5, the BPAs are assigned in the open world assumption. The uncertain degree measured by H, E d and E edeow are presented in Table 2. Calculation results show that Shannon entropy E d cannot reflect the changes of the cardinality in the FOD |X| (even if we treat the empty set ∅ as an uncertain proposition with nonzero set to make this function applicable in this case), while the Deng entropy is not applicable in this case for the reason that m (∅) = 0. Only the EDEOW can successfully express the enlarging in the FOD as the value of E edeow (m) increases with the increasing of the |X|.
The element number in the proposition Y changes from 0 to 14, as is shown in Table 3. If the element number of Y is 0, which means Y is an empty set and the FOD may be incomplete, the BPAs are assigned in the open world assumption. In this case, the uncertainty measures E d , E Y , E DP , C H , D KR , S KP and TC GP , which are defined in the closed world is not applicable (N/A). Mathematically, E Y and C H can be applied to calculate the uncertain degree if and only if the constraint of "BPAs are for focal element" is ignored which means a possible modification of the definition of Yager's dissonance measure E Y and Hohle's confusion measure C H . If the element number of Y changes from 1 to 14, then all the uncertainty measures presented in the Section Preliminaries are available for measuring the uncertain degree. The uncertain degree of the BPAs with different uncertainty measures are presented in Table 3, where there is a large discrepancy among the values of the uncertainty measures especially for the proposition Y = ∅. Compared with the analysis in [21], the new changes exist in the nonzero mass value of the empty set. The E edeow is the only proper measure in this case compared with other measures listed in Table 3. Of course, we also believe that there are new proper measures for this case since a new measure is always being proposed, e.g., the new entropy in [23]. Figure 1 presents the uncertain degree of different uncertainty measures visually. Intuitively, if a big mass value is assigned on the empty set, which means a big uncertain degree in the FOD, in this case, the EDEOW can measure the uncertain degree. It seems that Yager's dissonance measure E Y and Hohle's confusion measure C H can be generalized to measure the uncertain degree in the open world assumption where the mass value of the empty set is nonzero. However, Figure 1 shows that the uncertain degree measured by E Y and C H does not increase along with the increasing element number in the proposition Y. The E d , E DP , D KR , S KP and TC GP are all not available for uncertainty measure in the open world assumption because of the limitation in the log function of the definitions. Above all, the other uncertainty measures in Table 1 can only be applied in the closed world. Only the EDEOW can successfully measure the uncertainty degree of belief functions in this case. In addition, the EDEOW is identical to Deng entropy in the closed world, which ensures a successful possible extension of the Deng entropy.

A Discussion on the Properties of the Extended Measure
As is discussed in [24], the Deng entropy does not match some of the essential properties for a uncertainty measure in the DST framework. In detail, the Deng entropy satisfies the property of 'probabilistic consistency', but the properties such as the 'set consistency', the 'subadditivity', the 'additivity' and the 'monotonicity' are all broken by the Deng entropy. In addition, the range of the Deng entropy is greater than [0, log 2 |X|]. Since the EDEOW is just a simple extension of the Deng entropy, the EDEOW inherits the shortcomings of the Deng entropy with respect to these properties; this should be addressed in the following work.
We noticed that there are new rules of properties defined in a recent research [23], which should be taken into consideration in the ongoing work. Although the extension to Deng's entropy in the open world assumption only satisfies the property of 'Probabilistic consistency', we noticed that the newly defined measure in [23] does not satisfy the 'subadditivity' property, and the distance-based measure in [20] does not satisfy the properties of 'probability consistency' and 'set consistency'. In short, the property of the belief entropy is still an open issue in the closed world as well as the open world assumption in the DST framework.

EDEOW-Based Uncertain Information Fusion Approach
An uncertain information fusion approach based on the EDEOW is proposed to illustrate the usefulness and applicability of the extended measure. The framework of the new approach based on the EDEOW is presented in Figure 2, which is a modification of the methods in [44,54]. Firstly, the uncertain information in the closed world and the open world assumption are modelled as BPAs in DST framework. Then, the EDEOW is adopted to measure the uncertain degree of the BPAs without distinguishing the difference of belief functions in the closed world or the open world assumption, which is accomplished by the advantages that the EDEOW is the extension of an uncertainty measure from the closed world. After that, the uncertain degree measured by the EDEOW is used as the weight of each BPA for modification of the BPAs. Finally, the generalized combination rule in [30] is adopted to combine the BPAs. As a result, applications will be based on the fusion results, such as decision making and fault diagnosis. The case study in [55] is adopted and modified for verifying the effectiveness of the extended measure, as well as illustrating the EDEOW-based information fusion approach in Figure 2. According to experience and historical data, there are three types of identified fault types in the motor rotor denoted as F 1 = {Rotor unbalance}, F 2 = {Rotor misalignment} and F 3 = {Pedestal looseness} respectively. The vibration signal is collected by three acceleration sensors placed in different positions. The acceleration sensors can collect the signals at different frequencies denoted as Freq1, Freq2 and Freq3, the signals will be used as the judgement variables of fault types. The monitoring results of sensors are modelled as BPAs in Table 4 adopted from [55]. Table 4. Data for fault diagnosis modelled as BPAs [55].

Freq1
Freq2 Freq3 For each frequency, the BPAs reported by three sensors are denoted as m s 1 (·), m s 2 (·) and m s 3 (·). {F1, F2, F3} is the FOD of this application in the closed world. Here, in this paper, in order to adapt the experiment data for the application of the extended measure in the open world, the belief functions of {F1, F2, F3} are assumed to be assigned to the empty set ∅, which extends the uncertainty of the FOD from the closed world to the open world. This is reasonable, because there may exist unknown fault types.

Uncertainty Measure of BPAs with EDEOW
In real applications, the reliability of each sensor is unknown. Thus, the uncertain degree of sensor reports should be measured properly. In the DST framework, the belief entropy is proposed for measuring the uncertainty of BPAs. Once the sensor reports are modelled as BPAs, the uncertain degree of sensor reports can be measured based on the EDEOW in Equation (12). For example, for the BPAs of Freq1, the uncertain degree with the EDEOW is calculated as follows: The uncertain degree of Freq2 and Freq3 can also be calculated by Equation (12). The results are presented in Table 5.

EDEOW-Based Modification of BPAs
The EDEOW of each BPA is used as the weight factor of each sensor report. With a process of normalization, the weight of each BPA in Freq1 is calculated as follows: Similarly, the weight of BPAs in Freq2 and Freq3 can be calculated. After normalization, the weight of each BPA for Freq1, Freq2 and Freq3 is listed in Table 6. The modification of BPAs for each frequency can be calculated with the following equation: Based on the normalized weight factor in Table 6, with Equation (22) The modification of BPAs for Freq2 and Freq3 can be calculated with Equation (22). The BPAs after modification of each frequency is shown in Table 7.

Generalized Combination Rule-Based Data Fusion
In the open world assumption, classical Dempster's rule of combination is not applicable [30]. In this paper, the generalized combination rule in [30] is chosen for data fusion in the proposed approach. Since the original n sets of BPAs have been modified as one set of BPAs by EDEOW-based weight factors, the modified BPAs should be fused (n − 1) times according to the chosen combination rule.
There are three sets of BPAs before modification. Thus, the modified BPAs should be combined two times with generalized combination rule in Equation (7). For frequency Freq1, the fusion results are shown as follows: The BPAs of Freq2 and Freq3 are also fused three times with the generalized combination rule, the results are shown in Table 8. With the fusion results shown in Table 8, F2 significantly has the highest support degree among all the frequencies, therefore, we can judge that the fault type is F2. The experiment results are consistent with [54,55], which verifies the effectiveness of the EDEOW. In addition, the proposed method has a higher support degree on the recognized fault type F2 than that in [54,55], which is good for decision-making by engineers in real applications.

Open Issues for Future Work
There is no universally accepted measure for uncertainty quantification in the DST framework. Many new measures are still being proposed within one year [23,56]. To match the open world assumption [19,30,52,53], an extended measure for quantification of uncertain degree in the DST framework is proposed in this paper. It should be noted that the extended measure is a simple extension of the Deng entropy. A lot of open issues exist in the extended measure as well as the other measures for the open world assumption in the DST framework.
The first one exists in the scope of the uncertainty measures in the DST framework. According to the current research, we find that the theory of belief entropy or uncertainty measures in the DST is still not solid and needs further deep research. We suggest that the following research work on this topic should take into consideration the open world assumption.
The second open issue exists in the properties of the extended measure, which is a shortcoming inherited from the Deng entropy. According to the research work in [24], the Deng entropy only satisfies the property of probabilistic consistency with respect to the five requirements for a total uncertainty measure. The following work should focus on improving the measure or developing a totally new uncertainty measure for the open world assumption by taking into consideration all of the properties discussed in [22][23][24]57].
Thirdly, the following work needs to investigate what happens if the mass on the empty set is not null with different size of the universe because the new measure in the open world must address these two parameters. In addition, what is the meaning of having an entropy measure that changes in accordance with the cardinality of the universe? For instance, for a FOD |X|, we will have the same measure result E edeow (m) = 0 for the mass function m (∅) = 1 and m ({a}) = 1. Currently, we have difficulty answering all of these questions in this simple extended measure.
Fourthly, there are still no universally accepted properties for a belief entropy or uncertainty measure in the closed world and for the open world assumption, which is a big problem for developing a new belief entropy. For example, even the newly defined measure in [23] does not satisfy the 'subadditivity' property. Another example is that the measure in [20] does not satisfy the properties of 'probability consistency' and 'set consistency'. We believe that there are new properties that should be obeyed by the measures in the open world assumption.
Finally, in the application of sensor data fusion, fusion frame and combination rule need further study. There are more fusion methods and the combination rules in the research works [25,26,53,58] that need to be investigated cautiously.

Conclusions
An extended uncertainty measure for belief structures in the open world assumption, named the EDEOW, is proposed in this paper. The extended measure can successfully quantify the uncertain degree of belief structures not only in the closed world, but also in the open world. With the extended measure, more uncertain information in DST framework is taken into consideration while applying information processing, including the possible incomplete FOD and the nonzero mass function of the empty set, of which both are sources of uncertainty in the DST framework in the open world assumption. To verify the usefulness and applicability of the extended measure, the EDEOW is