Performance Evaluation of Public-Private Partnership Projects from the Perspective of Efficiency, Economic, Effectiveness, and Equity: A Study of Residential Renovation Projects in China

The performance evaluation index system was established on the basis of 4E (Efficiency, Economic, Effectiveness, Equity) theory to improve performance evaluation of Public-Private Partnership Projects for Residential Renovation (4P2R). Moreover, the index weighting model was demonstrated based on Ordered Weighted Averaging (OWA). Finally, the cloud model was used for performance evaluation, through the cloud uncertainty reasoning to determine the quantitative performance evaluation index value, the positive reverse cloud to calculate the qualitative performance evaluation index value, and the normal cloud to measure the project performance. The proof-of-concept application shows that the performance evaluation index system and evaluation model can better identify the deficiencies in the project, and provide targeted improvement measures for similar projects in the future.


Introduction
Local governments in China have been establishing Public-Private Partnership (PPP) projects owing to the increasing popularity and applicability of PPP model since September 2014. As of February 2018, the National PPP Integrated Information Platform Project Library of China has included 7477 PPP projects that have~$1.8 trillion investment in total. With the ever-increasing demand for renovating old residential areas, Beijing, Harbin, and other regions in China have applied the PPP model to broaden financing channels and ease financial pressure. The main purpose of 4P2R is to renovate residential areas with long construction times, backward facilities, and imperfect property management, and solve the problems of "dirty, chaotic, and bad" neighborhoods [1]. By doing so, 4P2R can improve supporting facilities, achieve full coverage of property management, optimize the living environment, and enhance the residents' quality of life and well-being [2,3].
Social capital has a significant role in PPP for the renovation of old residential areas (4P2R) through broadening the project financing channels, solving the project funding gap, and resolving the local government financial risks [4]. However, 4P2R has encountered problems such as slow progress, low public coordination, difficulty in implementing preferential policies, and low social responsibility. In addition, the above problems may cause quality risk, cost risk, and schedule risk in PPP projects. These risks may lead to issues such as quality defects, project funds waste, and delays in construction schedules [5]. In this regard, how to boost the investment benefits of 4P2R has attracted the attention of the academic community and industry [6]. Moreover, an effective performance evaluation

4E Theory
The 4E theory has been developed from 3E theory and the 3E system. The 3E theory that was proposed by Checkland [20] includes three aspects: economy, efficiency, and effectiveness. The economy measures its output; efficiency reflects the utilization of resources by the system; and effectiveness reflects the role of the lower-level system to the superior system in terms of output. The 3E theory is the proposed method of cognition and it can solve complex problems based on the soft system methodology from the systems perspective [21]. This method is used to analyze and understand the complex environment and problems faced by the system; consequently, logical and practical solutions can be developed [22]. At the same time, it can solve complex situations where traditional evaluation systems fail to be applied [23].
3E system is a project performance audit system based on 3Es-efficiency, economy, and effectiveness-as defined by the International Organization of Supreme Audit Institutions (INTOSAI) [24]. Efficiency measures the use of project resources and cost efficiencies, such as the project's actual duration and qualified quality [25]. Economy focuses on the economics of the use of project resources and the economics of management, such as the use of project funds and the changes in the cost of project operations [26]. Effectiveness describes the completion of the expected goals from the perspective of project output and measures the relationship between actual benefits and expected benefits, such as product quality and efficiency, and increased employment [27].
With the development of performance evaluation theory, many scholars argued that the 3E evaluation system neglect fairness and democracy [28]. They built a 4E system to evaluate project performance by incorporating the fourth E: equity [29]. Equity can reflect the impact of a project on participants and society, including economic, social, and cultural fairness [30]. On the other hand, 4P2R makes use of a substantial investment, an extended period of construction and operation, and a large number of stakeholders, and it is easy to be affected by the internal and external environment. Therefore, we can comprehensively measure the performance level of 4P2R through the 4E theory.

Performance Evaluation Index System from the Perspective of 4E
An index system can be particularly important for evaluating the performance of 4P2R. It is vital to use indicators to reflect the intrinsic characteristics of 4P2R. However, due to the short implementation time of PPP projects, China has not yet established a unified index system for the measurement of 4P2R's performance.
In the index system for evaluating the 4P2R's performance in its construction stage, the principles of systematic, scientific, comparable as well as quantifiable, should be followed. We should also consider factors such as the construction purpose and characteristics of 4P2R, the diversity and particularity of stakeholders, and the degree of difficulty in obtaining relevant data. This study follows the research route of "Literature Survey → Field Investigation → Expert Inquiry → Indicators Selection" to determine the index system. First, we determined the objectives and indicators of performance evaluation through literature survey and field investigation. Then, each expert in the expert group independently evaluated the indicator through a questionnaire. To ensure the rationality and reliability of the indicator selection, we used a face-to-face interview approach to communicate with each expert after we collected data to confirm the evaluation results. They can provide professional advice for determining the indicator system by their rich experiences in the field of PPP projects. The expert group with a total of 14 members consisted of four scholars who also have many years of consulting experiences in PPP projects, five government staff and five project managers, all of whom have at least five years of comprehensive experience in PPP projects. Finally, we built an index system using four indicators that include efficiency indicators, equity indicators, economic indicators, and effectiveness index to evaluate the 4P2R performance (Table 1).

Infrastructure
The maintenance and renovation of water, electricity, heating, and gas (V48) The renovation of parking lots (V49) The reform of fire protection facilities (V410) The upgrading of residential roads (V411) The reconstruction of rain and sewage discharge facilities (V412)

Operating maintenance
The maintenance of public facilities and equipment (V413) The maintenance of roads and ancillary facilities (V414) The quality of greening management service (V415) The quality of environmental health services (V416) The quality of security services (V417) The 4P2R performance evaluation indicators are divided into four categories: efficiency, equity, economy, and effectiveness. By fully characterizing the intrinsic characteristics of 4P2R, the performance level of 4P2R can be measured comprehensively: (1) Efficiency indicators (F1): Efficiency indicators are mainly applied for reflecting the 4P2R performance in the transformation process regarding quality, schedule, cost, and safety management. In general, the objectives of project management include quality, progress, cost, and safety. This article selects indicators to measure the completion efficiency of the four major goals, such as the qualified rate of project quality (V11), the level of project safety management (V12), project schedule payment rate (V13), and progress completion rate (V14). The indicator (V11) is used to measure the qualified rate of 4P2R. The indicator (V12) is used to assess the safety management of 4P2R during the construction phase. The indicator (V13) is used to evaluate the timely payment of the 4P2R in the construction stage. The indicator (V14) is used to measure the completion degree of the project schedule.
Among them, qualified rate of project quality = Qualified number of single projects/Total number of single projects; project schedule payment rate = total progress paid/total amount of project investment completed; progress completion rate = actual construction time/the entire project schedule. (2) Equity indicators (F2): From the perspective of stakeholders, this type of indicators mainly analyzes project risk sharing, concession period, charge mechanism, as well as the satisfaction of stakeholders from the project. This paper selects six indicators: the rationality of risk sharing, concession period, and charge mechanism, in addition to the satisfaction of the public, government, and social capital. 1 The rationality of risk sharing (V21) measures the rationality of risk sharing between the government and social capital in 4P2R; 2 The rationality of concession period (V22) measures the rationality of the 4P2R concession period; 3 The rationality of charge mechanism (V23) measures the rationality of charging mechanism during the concession period after completing the renovation of the old residential areas; 4 Public, government and social capital satisfaction are used to assess the satisfaction of relevant stakeholders from 4P2R. Among them, the satisfaction level of the public (V24) and government (V25) focus on measuring the overall satisfaction of government departments and community owners on the effect of old residential areas renovation. Social capital satisfaction (V26) focuses on measuring the implementation of preferential government policies, and the cooperation of public with the project. and its contribution to the regional economic development. In this work, we chose seven indicators such as tax contribution rate, asset-liability ratio, investment profit rate, employment contribution rate, operating cost change rate, actual investment change rate, and degree of regional development. 1 Tax contribution rate (V31) is used to measure the contribution of 4P2R to the tax revenue in the region; 2 Asset-liability ratio (V32) is used to measure the debt level of 4P2R; 3 Investment profit rate (V33) is used to measure the profitability of 4P2R; 4 Employment contribution rate (V34) mainly measures the contribution rate of 4P2R to the public employment; Among them, tax contribution rate = project tax amount/total regional tax revenue; asset-liability ratio = total project liabilities/total project assets; investment profit rate = the annual net income of the project/the project's total investment; employment contribution rate = the number of employees increased by the project/the number of employees in the region where the project is located; actual investment change rate = (completed settlement price-planned investment)/planned investment.
(4) Effectiveness index (F4): F4 indicators focus on assessing the renovation and remediation of 4P2R. During index selection, the research team conducts on-site investigations, consults literature, as well as academics, government staff, and project management personnel in related fields. The renovation and rectification of old buildings can be summarized as housing repair, living environment, infrastructure and operating maintenance. 1 To achieve the integrity of the house and the improvement of its functionality, the main outline of house repair is to demolish, renovate and maintain the house. We based our evaluation on the following indicators: energy-saving transformation (V41), internal repair (V42), external repair (V43), and waterproofing and maintenance of roofs (V44); 2 The main purpose of environmental renovation is to provide a livable environment for the old community residents. The main content is to clean up illegal buildings and structures, upgrade the district greening and improve sanitation facilities. Community greening (V45), illegal buildings (V46) and sanitation facilities (V47) have been included in this category; 3 Infrastructure is an engineering facility that provides services for residents. The transformation of infrastructure in old communities involves water, electricity, heating, gas, roads, parking lots and firefighting facilities. In this paper, six indexes were selected: the maintenance and renovation of water, electricity, heating and gas (V48), the renovation of parking lot (V49), the reform of fire protection facilities (V410), the upgrading of residential roads (V411) and the reconstruction of rain and sewage discharge facilities (V412); 4 The operation and maintenance management is critical for ensuring the normal operations of old communities, involving the maintenance of public facilities and equipment, roads and ancillary facilities, environmental health, greening, and security. Here, we focused on public facilities and equipment (V413), the maintenance of roads and ancillary facilities (V414), the quality of greening management service (V415), the quality of environmental health services (V416), and the quality of security services (V417). The indicators and their function are shown in Table 2.

V41
To measure the energy-saving transformation of 4P2R V42 To assess the internal repair of 4P2R V43 To evaluate the external repair of 4P2R V44 To measure the repair of roofing and waterproof V45 To evaluate the community greening of 4P2R V46 To measure the dismantling of illegal buildings in 4P2R V47 To assess the arrangement of sanitation facilities V48 To assess the maintenance and renovation of water, electricity, heating and gas V49 To evaluate the renovation of parking lot V410 To measure the reform of fire protection facilities V411 To assess the upgrading of residential roads V412 To evaluate the reconstruction of rain and sewage discharge facilities V413 To assess the maintenance of public facilities and equipment V414 To evaluate the maintenance of roads and ancillary facilities V415 To measure the quality of greening management service V416 To assess the quality of environmental health services V417 To evaluate the quality of security services

Research Methods
This section explains the research methods used in the paper, including OWA operator and the cloud model. First, we analyzed the applicability of the methods. Subsequently, we evaluated the steps of OWA operator weighting method. Finally, we discussed the concept and application of the cloud model and three cloud generators.

OWA Operator Weighting Method
The scientific rationale for index weight calculation is critical to ensure the accuracy of the evaluation results [31]. The weighting methods include objective weighting methods and subjective weighting methods. (1) Objective empowering methods, such as entropy weighting method [32] determine the index weight according to the degree of dispersion of the index. The degree of dispersion is proportional to the weight of the index. When the degree of dispersion is 0, the index does not work in the evaluation, and it is challenging to address the subjective preferences of the decision maker [33]; (2) Subjective weighting methods, such as Analytic Hierarchy Process (AHP) [34], could indicate decision makers' subjective preferences. However, when the number of decision data made by decision makers is less, the maximum or minimum value has a greater negative impact on weighting and can reduce the objective determination of the weight value [35]. In other words, if the traditional weighting methods are used to assign weights to the indicators, deviations can easily affect the accuracy of the Sustainability 2018, 10, 1951 7 of 21 evaluation results. Therefore, numerous indicators should be taken into account to come up with an improved performance evaluation index system.
To solve the problems mentioned above, we used the OWA operator to weight the indicators. The calculation process of OWA operator weighting method is relatively simple and can better reflect the preference of decision makers [36]. The OWA operator weighting method has been applied successfully in the field of credit rating [37], strategic decision [38] and safety evaluation [39]. It is best suited for solving uncertain decision-making problems and can reduce the negative impact of the maximum and minimum values of decision data on the outcome [40].

Performance Evaluation Method Based on Cloud Model
Performance indicators are grouped into indicators that are qualitative and those that are quantitative. The values of qualitative indexes are determined by the expert group only once [41]. Affected by personal knowledge and experience, the scores given by experts have different degrees of randomness and ambiguity [42]. As a result, there are some deviations in the performance index values. These deviations can considerably affect the accuracy of performance evaluation results. The performance evaluation index system constructed in this paper includes many qualitative performance evaluation indicators. Therefore, scientific determination of the qualitative index values is key to ensuring the accuracy of project performance evaluation results.
We used the cloud model to measure index values and evaluate project performance. This model [43] converts between qualitative descriptions and quantitative ones; it has been applied to the fields of land evaluation [44], urban traffic flow forecasting [45], and smart distribution grid evaluation [46]. (1) To determine the value of qualitative indicators, we determined the scores of the performance indicators after repeated evaluations of the expert group through the combination the cloud model with the Delphi method. In the above process, we used the cloud diagram to visually control the convergence speed, quality, and direction of the expert scores, thereby ensuring the reliability of the scores; (2) In determining the value of qualitative indicators, we used the cloud uncertainty reasoning model to determine the value of quantitative indicators and maintain the consistency between the qualitative and quantitative indicators.

OWA Operator Weighting Method
Ordered Weighted Averaging (OWA) is an information aggregation method introduced by Yager [47]. It uses a particular method to reorder input elements based on attribute values and assigns different aggregation weights according to the location of data [48]. Many scholars have conducted in-depth research on the extended algorithm of OWA operator. The combination number OWA operator weighting model adopted in this paper was based on the classical OWA operator theory and used the combination number as the weighted vector to complete the data aggregation [49]. This method weights the decision data by the number of combinations and calculates each indicator's weight. The specific steps are listed below: (1) First, we sorted the decision data (e 1 , e 2 , · · · , e n ) of the indicators from the largest to the smallest, then numbered the data (g 0 , g 1 , · · · , g n−1 ) beginning from 0, and obtained the result (g 0 , g 1 , · · · , g n−1 ) that turned out to be g 0 ≥ g 1 ≥ · · · ≥ g k ≥ · · · ≥ g n−1 . (2) The data g k was weighted by the number of combinations, and the equation ∑ n−1 i=0 C i n−1 = 2 n−1 was obtained based on the number of combinations. The weight vector was determined by: (3) We weighted the decision data according to the weight vector β τ+1 , and the absolute weights of the indicators were determined by: (4) The relative weights of the calculated indicators were determined by: Through the above steps, the weight vectors of the indicators of each level were obtained, thereby completing the weighting of the index system.

Cloud Model
Based on the theoretical knowledge of fuzzy mathematics and probability theory, Li et al. [50] proposed the cloud model. As a transformation model that is uncertain, this cloud model deals with both qualitative concepts and quantitative descriptions [51]. It can represent the process from qualitative concept to quantitative description and from quantitative description to qualitative conception [52]. The cloud model can prevent information loss due to the transformation processes and increase objectivity in comparison to other methods [53].
The cloud model uses three digital eigenvalues to represent language values: Expectation (Ex) and Entropy (En) as well as Hyper entropy (He) [54]. Located in the center of gravity in the domain space, expectation (Ex) is a sample value typically reflecting a qualitative concept. Entropy (En) is used to describe the degree of uncertainty for a qualitative concept. This qualitative concept's fuzziness and randomness are used to determine the Entropy (En). Hyper entropy (He) is used to describe the degree of entropy's uncertainty, which can be regarded as entropy's entropy to reflect the thickness of clouds.
The cloud generation algorithm can be implemented in software and solidified into hardware. As a cloud generator, it comprises the positive cloud generator, reverse cloud generator, single condition, and single rule generator.
(1) The positive cloud generator [55] It is a process transferred from qualitative concept to quantitative representation. The model is used to generate cloud droplets from the cloud's digital eigenvalues, as shown in Figure 1.
(3) We weighted the decision data according to the weight vector , and the absolute weights of the indicators were determined by: (4) The relative weights of the calculated indicators were determined by: Through the above steps, the weight vectors of the indicators of each level were obtained, thereby completing the weighting of the index system.

Cloud Model
Based on the theoretical knowledge of fuzzy mathematics and probability theory, Li et al. [50] proposed the cloud model. As a transformation model that is uncertain, this cloud model deals with both qualitative concepts and quantitative descriptions [51]. It can represent the process from qualitative concept to quantitative description and from quantitative description to qualitative conception [52]. The cloud model can prevent information loss due to the transformation processes and increase objectivity in comparison to other methods [53].
The cloud model uses three digital eigenvalues to represent language values: Expectation (Ex) and Entropy (En) as well as Hyper entropy (He) [54]. Located in the center of gravity in the domain space, expectation (Ex) is a sample value typically reflecting a qualitative concept. Entropy (En) is used to describe the degree of uncertainty for a qualitative concept. This qualitative concept's fuzziness and randomness are used to determine the Entropy (En). Hyper entropy (He) is used to describe the degree of entropy's uncertainty, which can be regarded as entropy's entropy to reflect the thickness of clouds. The cloud generation algorithm can be implemented in software and solidified into hardware. As a cloud generator, it comprises the positive cloud generator, reverse cloud generator, single condition, and single rule generator.
(1) The positive cloud generator [55] It is a process transferred from qualitative concept to quantitative representation. The model is used to generate cloud droplets from the cloud's digital eigenvalues, as shown in Figure 1. The algorithm for positive cloud generator is shown below:

Algorithm 1
Input: The positive cloud generator's digital eigenvalues (Ex, En, and He) are used as inputs. N is set as the number of cloud drops. Output: N and its corresponding membership degree.
Step 1: The normal random number (En * ) is generated by taking En as expected value and He as mean square error, where En * = (En, He); Step 2: The normal random number (x) is produced by taking Ex as the expected value and En * as The algorithm for positive cloud generator is shown below:

Algorithm 1
Input: The positive cloud generator's digital eigenvalues (Ex, En, and He) are used as inputs. N is set as the number of cloud drops. Output: N and its corresponding membership degree.
Step 1: The normal random number (En * ) is generated by taking En as expected value and He as mean square error, where En * = N(En, He); Step 2: The normal random number (x) is produced by taking Ex as the expected value and En * as mean square error, where x = N(Ex, En * ); Step 3: Calculate y = exp − (x i −Ex) 2 2(En * ) 2 ; Let (x,y) be a cloud drop, where x represents a specific quantitative value of the qualitative concept C and y represents the membership degree of the qualitative concept; Step 4: Repeat Step 1 to Step 3, the operation stops after generating the set of n cloud droplets.
(2) The reverse cloud generator As a transformation model, the reverse cloud generator implements quantitative values to qualitative concepts [56]. The precise data can be converted into qualitative concepts as digital eigenvalues (Ex, En, He), as shown in Figure 2. ; Let (x,y) be a cloud drop, where x represents a specific quantitative value of the qualitative concept C and y represents the membership degree of the qualitative concept; Step 4: Repeat Step 1 to Step 3, the operation stops after generating the set of n cloud droplets.
(2) The reverse cloud generator As a transformation model, the reverse cloud generator implements quantitative values to qualitative concepts [56]. The precise data can be converted into qualitative concepts as digital eigenvalues (Ex, En, He), as shown in Figure 2. The algorithm for reverse cloud generator is shown below:

Algorithm 2
Input: Enter n cloud drops, e.g., sample data ( = 1,2, ⋯ , ) Output: The cloud model's digital eigenvalues (Ex, En, He) Step 1: Take = ∑ as the estimate of Ex; Step 2: After removing the point of y>0.999, the number of remaining cloud droplets is m.

;
Step 4: Obtain from = ∑ * , where is an estimate of En; Step 5: Obtain from = ∑ ( * − ) , where is an estimate of He; (3) The single condition and single rule generator Based on the positive cloud generator, we constructed X-conditioned cloud generator CG and Y-conditioned cloud generator CG , respectively, based on the single condition and single rule, and merged them to form a single condition with a single rule generator [57] in Figure 3. The algorithm for the single condition and single rule generator is shown below:

Algorithm 3
Input: The quantitative data , the digital eigenvalues (Ex , En , He ) of X-conditioned cloud generator CG , the digital eigenvalues (Ex , En , He ) of Y-conditioned cloud generator CG . Output: Under the condition that the degree of certainty is u, the qualitative concept value of The algorithm for reverse cloud generator is shown below:

Algorithm 2
Input: Enter n cloud drops, e.g., sample data x i (i = 1, 2, · · · , n) Output: The cloud model's digital eigenvalues (Ex, En, He) Step 1: TakeÊx = 1 n ∑ n i=1 x i as the estimate of Ex; Step 2: After removing the point of y>0.999, the number of remaining cloud droplets is m.

;
Step 4: ObtainÊn fromÊn = 1 m ∑ m i=1 En * i , whereÊn is an estimate of En; whereĤe is an estimate of He; (3) The single condition and single rule generator Based on the positive cloud generator, we constructed X-conditioned cloud generator CG A and Y-conditioned cloud generator CG B , respectively, based on the single condition and single rule, and merged them to form a single condition with a single rule generator [57] in Figure 3. ; Let (x,y) be a cloud drop, where x represents a specific quantitative value of the qualitative concept C and y represents the membership degree of the qualitative concept; Step 4: Repeat Step 1 to Step 3, the operation stops after generating the set of n cloud droplets.
(2) The reverse cloud generator As a transformation model, the reverse cloud generator implements quantitative values to qualitative concepts [56]. The precise data can be converted into qualitative concepts as digital eigenvalues (Ex, En, He), as shown in Figure 2. The algorithm for reverse cloud generator is shown below:

Algorithm 2
Input: Enter n cloud drops, e.g., sample data ( = 1,2, ⋯ , ) Output: The cloud model's digital eigenvalues (Ex, En, He) Step 1: Take = ∑ as the estimate of Ex; Step 2: After removing the point of y>0.999, the number of remaining cloud droplets is m.

;
Step 4: Obtain from = ∑ * , where is an estimate of En; Step 5: Obtain from = ∑ ( * − ) , where is an estimate of He; (3) The single condition and single rule generator Based on the positive cloud generator, we constructed X-conditioned cloud generator CG and Y-conditioned cloud generator CG , respectively, based on the single condition and single rule, and merged them to form a single condition with a single rule generator [57] in Figure 3. The algorithm for the single condition and single rule generator is shown below:

Algorithm 3
Input: The quantitative data , the digital eigenvalues (Ex , En , He ) of X-conditioned cloud generator CG , the digital eigenvalues (Ex , En , He ) of Y-conditioned cloud generator CG .  The algorithm for the single condition and single rule generator is shown below:

Algorithm 3
Input: The quantitative data x A , the digital eigenvalues (Ex A , En A , He A ) of X-conditioned cloud generator CG A , the digital eigenvalues (Ex B , En B , He B ) of Y-conditioned cloud generator CG B . Output: Under the condition that the degree of certainty is u, the qualitative concept value of Y-conditioned cloud generator CG B is x B .
Step 1: The normal random number En * A is generated by taking En A as expected value and He A as mean square error; Step 2: The degree of certainty is calculated by Step 3: The normal random number En * B is generated by taking En B as expected value and He B as mean square error; Step 4: If the X-conditioned cloud generator CG A is activated and then it rises in the direction of x A ≤ Ex A , Y-conditioned cloud generator is activated, and the certainty value is Similarly, if the X-conditioned cloud generator CG A is activated and then it drops in the direction of x A > Ex A , Y-conditioned cloud generator is activated and the certainty value is

Research Process
The research process on performance evaluation of 4P2R is shown in Figure 4. The detailed description of the research process is given from Section 5.2 to Section 5.5. Y-conditioned cloud generator CG is .
Step 1: The normal random number * is generated by taking En as expected value and He as mean square error; Step 2: The degree of certainty is calculated by u = exp − ( ) ( * ) ; Step 3: The normal random number En * is generated by taking En as expected value and He as mean square error; Step 4: If the X-conditioned cloud generator CG is activated and then it rises in the direction of ≤ , Y-conditioned cloud generator is activated, and the certainty value is = − * × √−2 . Similarly, if the X-conditioned cloud generator CG is activated and then it drops in the direction of > , Y-conditioned cloud generator is activated and the certainty value is = + * × √−2 .

Research Process
The research process on performance evaluation of 4P2R is shown in Figure 4. The detailed description of the research process is given from Section 5.2 to Section 5.5.

Determination of the Performance Evaluation Index System of 4P2R
We

Using OWA Operator Weighting Method to Produce the Index Weights
The OWA operator weighting method was employed for determining the performance evaluation indicators weights. First, the OWA operator was used to obtain the weight value ∀ * of the first-level indicator. Then, we calculated the weight values of the secondary indicators where the second-level indicators belong. Finally, we used W = ∀ * • to calculate the final weight of the

Determination of the Performance Evaluation Index System of 4P2R
We established the Performance Evaluation Index System including four first-level evaluation indicators and 34 second-level evaluation indicators. The indicator set F is a set of first-level performance evaluation indicators F i , where F = {F 1 , F 2 , · · · , F i }; the indicator set V i (i ∈ [1, t]) is a set of second-level performance evaluation indicators V ij (j ∈ [1,

Using OWA Operator Weighting Method to Produce the Index Weights
The OWA operator weighting method was employed for determining the performance evaluation indicators weights. First, the OWA operator was used to obtain the weight value ∀ * i of the first-level indicator. Then, we calculated the weight values Z e of the secondary indicators where the second-level indicators belong. Finally, we used W = ∀ * i ·Z e to calculate the final weight of the secondary indicator.

Index Value Measurement Based on Cloud Uncertainty Reasoning and Positive Reverse Cloud
We used the cloud uncertainty reasoning to determine the quantitative performance evaluation index value, and the positive reverse cloud to calculate the qualitative performance evaluation index value.

Quantitative Index Measurement Based on Cloud Uncertainty Reasoning
The steps through which we established quantitative performance index value are outlined below: (1) Determining the comment set of the indicator and digital eigenvalues Through cloud uncertainty reasoning model, quantitative index values were measured using the single condition and single rule generator. We used the comment A k to describe the quantitative indicators V ij , in which the comment was A = (A 1 A 2 A 3 · · · A k ), and the digital eigenvalues ij . If the quantitative index value was x ij(s) , after calculation by the positive cloud generator, the degree of membership of the index value x ij(s) was 4, which corresponds to the comment A k .
(2) Determining the comment set of the score and digital eigenvalues After calculating the degree of membership of index value A k , we determined the comment set of score and the corresponding digital eigenvalues to determine the corresponding percentage value of the index value A k . We used the comment B d to describe the percentage value of the index V ij , in which the comment was B = (B 1 B 2 B 3 · · · B d ), and the digital eigenvalues of the comment A k was SC B d Ex B d , En B d , He B d .

(3) Measuring the value of the evaluation index
The cloud uncertainty reasoning was used to calculate the measurement value r ij(s) of the evaluation index. Among them, the form of uncertainty reasoning was; if x ij(s) ∈ A k , then r ij(s) ∈ B d , since there is a one-to-one relationship between k and d.

Qualitative Index Measure Based on Positive Reverse Cloud
The steps by which we obtained qualitative performance index value are described below: (1) Qualitative numerical conversion to qualitative concept based on reverse cloud First, we formed a group of experts; then, evaluation criteria were formulated, and a set of precise data was formed after the experts gave their scores; finally, we used the reverse cloud generator to transform the set of precise data into digital eigenvalues (Ex, En, Hê) that can be used to represent qualitative concepts.
(2) Transforming from qualitative concept to quantitative representation based on positive cloud Based on the digital eigenvalues (Ex, En, Hê), we ran the positive cloud generator to generate cloud droplets and formed cloud maps. Then, the mapping of qualitative concepts to quantitative representations was completed. In combination with the cloud map, we visually analyzed the gravity center of discourse domain, the range of the number domain, the condensation degree of cloud droplet, and the clouds' thickness.

(3) Determining the qualitative index value based on the Delphi method
To reasonably determine the qualitative index value, we used the cloud model to visually control the convergence speed, quality, and direction of the expert experience; and we used the Delphi method to collect feedback for the expert group on the score's details. After repeated evaluations, it was unanimously approved by the expert group to obtain the Expectation (Ex * ) and Entropy (En * ) as well as Hyper entropy (He * ) of the qualitative index value, and the Expectation (Ex * ) was used as the quantitative value of the index.

Project Performance Evaluation Based on Normal Cloud Model
(1) Building the domain of project performance indexes and the domain of comments The index domain of project performance evaluation was described as V i = V i1 V i2 V i3 · · · V in i , and the comment domain was D = (D 1 D 2 D 3 · · · D θ ).
(2) Determining standard normal cloud for project performance evaluation indicators Set x 1 ij and x 2 ij respectively as the indicator's upper and lower boundary values to the level of comment V in i . Note the indicator is V ij . Then, we measured the qualitative concept of the index V ij corresponding to the comment level D θ of performance evaluation, which can be characterized by the normal cloud. Among them, Because the boundary value is a transition value from one level to another level, it is a fuzzy boundary and should belong to the corresponding two levels at the same time, that is, the degrees of membership of the two levels are equal. Thus, we can see: where He ij is used to describe the entropy's uncertainty, which is entropy's entropy to reflect clouds' thickness. In general, He ij should be adjusted according to the uncertainty of the evaluation index.
In general, He ij should be adjusted according to the uncertainty of En ij , and its value has a certain relationship with En ij : He ij = mEn ij . The range of m is from 0.05 to 0.2 and needs to be adjusted according to the actual situation [58].
(3) Performance evaluation of a single index According to each index value of the item to be evaluated, a positive cloud generator was used to determine the membership matrix∂ corresponding to each index for each level, e.g.,∂ = ∂ ij n×m . For the improvement of the credibility of the assessment, the forward cloud generator should be run N times, and the mean value of the membership degree should be calculated as the evaluation result using ∂ ij = ∑ n k=1 ∂ k ij /n.
In Equation (7), ϕ j (j = 1, 2, · · · , θ) refers to the degree of membership of the j-th comment. Then, based on the principle of the membership's maximum degree,φ θmax is the result of project performance evaluation.

Case Study
Zibo in Shandong, China aims to carry out a total of 241 residential renovation projects. The project involves a total of 174,000 households and a total area of 13.477 million square meters. The main renovations are housing repair, living environment, infrastructure, and operating maintenance. Among them, the ZD district involves 53 residential areas with a total investment of about~$103 million. The PPP model was adopted to renovate the residential areas in two batches. The first batch of 28 projects was completed by the end of 2016, and the second batch of 25 project plans are due to be completed by the end of 2018. We studied the representative residential area E in the ZD district as the proof-of-concept demonstration of our performance evaluation index system and performance evaluation model.
(2) Scoring the importance of indicators First, six experts were invited to form an expert group, including government department staff and university scholars. The expert group scored the first-level indicators according to the evaluation criteria. For the sake of simplicity, all scores were taken as an integer multiple of 0.5, e.g., 3.5, 5, and 7.5 (Table 3). (3) Calculating the weight vector First, we sorted the decision data of the indicators from the largest to the smallest and took indicator F1 as an example to obtain g k = (6.5, 6, 6, 5.5, 5, 5). Since the number of experts n was 6, the weight vector was calculated according to Equation (1)  The weighting method for second-level indicators was the same as described above, and the results were provided in Table 4.  project's actual situation. Taking the indicator of "Qualified Rate of Project Quality (QRPQ)" as an example, the natural property value of this indicator for the old residential area E was determined to be 0.94. We used the comment to describe the percentage value of the index, where the comment was "Very High, High, Middle, Low, Very Low" and the digital eigenvalues of the comment was "(0.95, 0.0083, 0.001), (0.9, 0.0083, 0.001), (0.85, 0.0083, 0.001), (0.8, 0.0083, 0.001), (0.75, 0.0083, 0.001)". The cloud chart is shown in Figure 5. (2) Determining the comment set of score and digital eigenvalues For ensuring consistency between index level and score, the comment level of the quantitative index should have a positive correlation with the score level. That is, the review level increases with the score. We used the comment to describe the percentage value of the index, where the comment was "Very High, High, Center, Low, Very Low" and the digital eigenvalues of the comment was "(90, 3.33, 0.02), (70, 3.33, 0.02), (50, 3.33, 0.02), (30, 3.33,0.02), (10, 3.33, 0.02)". The cloud uncertainty (2) Determining the comment set of score and digital eigenvalues For ensuring consistency between index level and score, the comment level of the quantitative index should have a positive correlation with the score level. That is, the review level increases with the score. We used the comment to describe the percentage value of the index, where the comment was "Very High, High, Center, Low, Very Low" and the digital eigenvalues of the comment was "(90, 3.33, 0.02), (70, 3.33, 0.02), (50, 3.33, 0.02), (30, 3.33,0.02), (10, 3.33, 0.02)". The cloud uncertainty reasoning was used to calculate the measurement value of the evaluation index. As an example, we showed the second-level index QRPQ to establish the following inference rules: If QRPQ is "Very High", the score is "Very High"; If QRPQ is "High", the score is "High"; If QRPQ is "Middle", the score is "Center"; If QRPQ is "Low", the score is "Low"; If QRPQ is "Very Low", the score is "Very Low." According to the relevant steps of the single condition single rule generator, the score of QRPQ is 85.98 through cloud uncertainty inference. Similarly, the quantitative scores of other quantitative indicators can be obtained, as shown in Table 4. (2) Determining the value of qualitative indicator value Taking the indicator of "the level of project safety management" as an example, we calculated the reverse cloud generator and positive cloud generator, respectively, based on the scores of 10 experts. Through three rounds of expert scoring, three digital eigenvalues of the indicator (88.03, 1.57, 0.14) were obtained, and the expected value (88.03) was taken as the measured value of "the level of project safety management." The process was performed as follows: 1 In the first round, due to the influence of personal experience and knowledge, experts had different views of "the level of project safety management" and needed to meet a second time. The specific performance of the first round was that the expert scores were relatively scattered, the digital eigenvalues of En and He were larger, and the cloud chart showed a fog shape, as shown in Figure 6a; 2 After summarizing the scores, we gave feedback to the experts and invited the experts to conduct the second round of scoring. The digital eigenvalues showed that the En and He decreased, and the cloud began to be condensed from the fog to the standard normal cloud, as shown in Figure 6b; 3 After feedback was provided, the experts scored in the third round, and the entropy and Hyper entropy further decreased. Consequently, the cohesion of the cloud diagram was significantly enhanced. Figure 6c shows that the standard normal cloud representing the expert score was gradually formed.
(1) Construction of the comment sets of qualitative indicators In this paper, the expert evaluating method was used to determine the value of qualitative indicators. The comment of indicator was "Very High, High, Center, Low, Very Low" and the corresponding score range was "[0 20), [21 40), [41 60), [61-80), [81 100]". A group of experts was formed to evaluate the qualitative indicators, including two government department staff, three university academics, and five experienced management personnel in construction, operation, and maintenance.
(2) Determining the value of qualitative indicator value Taking the indicator of "the level of project safety management" as an example, we calculated the reverse cloud generator and positive cloud generator, respectively, based on the scores of 10 experts. Through three rounds of expert scoring, three digital eigenvalues of the indicator (88.03, 1.57, 0.14) were obtained, and the expected value (88.03) was taken as the measured value of "the level of project safety management." The process was performed as follows: ① In the first round, due to the influence of personal experience and knowledge, experts had different views of "the level of project safety management" and needed to meet a second time. The specific performance of the first round was that the expert scores were relatively scattered, the digital eigenvalues of En and He were larger, and the cloud chart showed a fog shape, as shown in Figure 6a; ② After summarizing the scores, we gave feedback to the experts and invited the experts to conduct the second round of scoring. The digital eigenvalues showed that the En and He decreased, and the cloud began to be condensed from the fog to the standard normal cloud, as shown in Figure 6b; ③ After feedback was provided, the experts scored in the third round, and the entropy and Hyper entropy further decreased. Consequently, the cohesion of the cloud diagram was significantly enhanced. Figure 6c shows that the standard normal cloud representing the expert score was gradually formed.

Project Performance Evaluation Based on Normal Cloud
(1) Building the comment domain and digital eigenvalues The comment domain of its performance level was determined as "Very Low, Low, Middle, High, Very High", while its standard normal cloud digital feature was " (

Project Performance Evaluation Based on Normal Cloud
(1) Building the comment domain and digital eigenvalues The comment domain of its performance level was determined as "Very Low, Low, Middle, High, Very High", while its standard normal cloud digital feature was " (10, 8.49, 2.01), (30, 8.49, 2.01), (50, 8.49, 2.  (2) Performance evaluation of the project According to each index value of the item to be evaluated, a positive cloud generator is used to determine the membership matrix corresponding to each index for each level. We repeatedly ran the forward cloud generator 2000 times to improve the credibility of the assessment. After calculation, first, we obtained the performance level of the first-level indicators of the project (Table 4). Then, we obtained the overall performance level of the project as "2.02 × 10 5 , 0.0139, 0.1185, 0.6734, 0.2630", corresponding to the performance levels "Very Low, Low, Middle, High, Very High", as shown in Table 5. (2) Performance evaluation of the project According to each index value of the item to be evaluated, a positive cloud generator is used to determine the membership matrix corresponding to each index for each level. We repeatedly ran the forward cloud generator 2000 times to improve the credibility of the assessment. After calculation, first, we obtained the performance level of the first-level indicators of the project (Table 4). Then, we obtained the overall performance level of the project as "2.02 × 10 5 , 0.0139, 0.1185, 0.6734, 0.2630", corresponding to the performance levels "Very Low, Low, Middle, High, Very High", as shown in Table 5. (3) Analysis of the evaluation results Table 5 shows that the project's performance evaluation result isφ = 0.6734. The result shows that the overall performance level is "High" and the performance level of the project is on an upward trend, i.e., it tends to be "Very High." 1 The performance evaluation result of the first-level indicator F1 is ϕ F1 = 0.0457, and the performance level is "Very High"; 2 The evaluation result of indicator F2 is ϕ F2 = 0.2176, and the performance rating is "High," which shows an upward trend, i.e., it tends to be "Very High"; 3 The evaluation results of indicators F3 and F4 are, respectively, ϕ F3 = 0.1098 and ϕ F4 = 0.2426. Thus, the overall performance level is "High" and the performance level of the project is on a downward trend, i.e., it tends to be "Middle." According to the above evaluation, the project's overall performance level is "High." In the first-level indicators, the level of the efficiency index F1 is "Very High," the level of performance of the "fairness" index F2, the "economic" index F3, and the "effectiveness" index F4 are all "High." However, the level of performance of the "economic" index F3, and the "effectiveness" index F4 are on a downward trend, i.e., it tends to be "Middle." After the analysis of the second-level indicators, its performance level "Operating cost change rate" in the "economic" index is "Low," and the performance level of the second-level indicator "waterproofing and maintenance of roofs" and "the renovation of parking lots" are "Low." By identifying and analyzing the problems in the project, we propose the following improvement measures: follow-up projects should strengthen cost management in energy consumption, material procurement, and equipment maintenance, and control non-essential cost expenditures. In order to improve the roof waterproofing quality after the completion of the project, the follow-up projects should pay attention to the supervision of the construction process and material quality, and build a suitable quality supervision and management system. Relevant departments should rationally plan parking lots and strengthen parking management to avoid limited parking.

Conclusions
The performance evaluation of 4P2R can identify the shortcomings in the project, and provide means of analysis for the causes of the problems and related improvement measures. Our key conclusions can be stated as follows: (1) We built a performance evaluation index system for 4P2R based on the 4E (Efficiency, Economic, Effectiveness, Equity) theory. (2) The performance evaluation model was established based on OWA and the cloud models.
We used OWA to analyze the evaluation index's weight. To reasonably determine the index value, we used a positive reverse cloud to determine the qualitative index value and cloud uncertainty reasoning to obtain the quantitative index value. We also used the normal cloud to evaluate the project performance. (3) The case study showed that the overall performance level of the project was determined as "High", whereas the performance level of "operating cost change rate", "waterproof maintenance of roofs", and "the renovation of parking lots" were "Low". Our work shows that the index system is suitable for the performance evaluation of 4P2R. The performance evaluation model can reasonably determine the indicator weights and index values and comprehensively measure the project performance. It can identify the deficiencies of past projects, propose improvement measures, and guide similar projects in the future. (4) The performance evaluation index system and performance evaluation model can improve the performance evaluation of 4P2R and potentially be applied to a great extent. It can provide reference for the government to formulate performance evaluation indicators for different types of PPP projects, so as to identify the problems of PPP projects and improve the comprehensive benefits of PPP projects. Because the performance evaluation model integrates an OWA operator and cloud model, the calculation process may be complicated. However, it can be programmed with MATLAB software (MATLAB 7.0 (7.0.0.19920 (R14), Math Works, Natick, MA, USA)) that has strong operability. Concurrently, introducing sustainability as an essential game-changer in the performance evaluation of 4P2R proves itself to be a significant research direction, which is worth studying further.