Site Selection of Digital Signage in Beijing: A Combination of Machine Learning and an Empirical Approach

: With the extensive use of digital signage, precise site selection is an urgent issue for digital signage enterprises and management agencies. This research aims to provide an accurate digital signage site-selection model that integrates the spatial characteristics of geographical location and multisource factor data and combines empirical location models with machine learning methods to recommend locations for digital signage. The outdoor commercial digital signage within the Sixth Ring Road area in Beijing was selected as an example and was combined with population census, average house prices, social network check-in data, the centrality of tra ﬃ c networks, and point of interest (POI) facilities data as research data. The data were divided into 100–1000 m grids for digital signage site-selection modelling. The empirical approach of the improved Hu ﬀ model was used to calculate the spatial accessibility of digital signage, and machine learning approaches such as back propagation neural network (BP neural networks) were used to calculate the potential location of digital signage. The site of digital signage to be deployed was obtained by overlay analysis. The result shows that the proposed method has a higher true positive rate and a lower false positive rate than the other three site selection models, which indicates that this method has higher accuracy for site selection. The site results show that areas suitable for digital signage are mainly distributed in Sanlitun, Wangfujing, Financial Street, Beijing West Railway Station, and along the main road network within the Sixth Ring Road. The research provides a reference for integrating geographical features and content data into the site-selection algorithm. It can e ﬀ ectively improve the accuracy and scientiﬁc nature of digital signage layouts and the e ﬃ ciency of digital signage to a certain extent.


Introduction
Digital signage is a multimedia audio-visual system that releases business, financial, and entertainment information through terminal display devices in public places [1,2]. Compared with traditional TV and newspaper advertisements, digital signage can be used for personalized and customized advertising based on different audiences [3,4]. Currently, digital signage advertising has gradually become the main trend in the development of outdoor advertising, supporting the overall

Data Source
As a nonlinear complex system, the distribution of digital signage needs to comprehensively consider the interaction between various factors. Demographics, transportation, competition, and facilities are the main factors affecting the layout and location of digital signage [20,37,38]. Therefore, we select 19 indicators as shown in Table 1, including the spatial information and broadcast price of digital signage, the number of POI facilities, Sina Weibo check-in data, census data, and transportation network centrality index data (calculated based on the street network) [41,42]. The digital signage assumed in this study included outdoor commercial digital signage with broadcasting video, animation, pictures or text, as well as all of the outdoor commercial large digital signage distributed within the Sixth Ring Road in Beijing based on our database established by field

Data Source
As a nonlinear complex system, the distribution of digital signage needs to comprehensively consider the interaction between various factors. Demographics, transportation, competition, and facilities are the main factors affecting the layout and location of digital signage [20,37,38]. Therefore, we select 19 indicators as shown in Table 1, including the spatial information and broadcast price of digital signage, the number of POI facilities, Sina Weibo check-in data, census data, and transportation network centrality index data (calculated based on the street network) [41,42]. The digital signage assumed in this study included outdoor commercial digital signage with broadcasting video, animation, pictures or text, as well as all of the outdoor commercial large digital signage distributed within the Sixth Ring Road in Beijing based on our database established by field investigation. The information displayed on the digital signage should have at least 100 m visibility, that is, it can be seen clearly 100 meters away from the audience.

Methods
The site selection of digital signage is the process of identifying new places to deploy digital signage by combining digital signage data and influential factors. The improved Huff model is an empirical approach that can calculate the distance between the audience and the digital signage and then select grids with low spatial accessibility. The machine learning methods analyse the number of commercial outlets, POI facilities, and other influential factors in the grid to obtain the layout potential of digital signage and then select grids with a high deployment potential as the place to be deployed. Compared with previous studies, by combining the two methods, the spatial characteristics and the commercial potential of the location are integrated into the content-based site-selection method. This not only improves computing efficiency but also improves the interpretability of the model. This process mainly includes three parts: data spatialization, site selection, and model verification ( Figure 2). ISPRS Int. J. Geo-Inf. 2020, 9, 217 5 of 21 investigation. The information displayed on the digital signage should have at least 100 m visibility, that is, it can be seen clearly 100 meters away from the audience.

Methods
The site selection of digital signage is the process of identifying new places to deploy digital signage by combining digital signage data and influential factors. The improved Huff model is an empirical approach that can calculate the distance between the audience and the digital signage and then select grids with low spatial accessibility. The machine learning methods analyse the number of commercial outlets, POI facilities, and other influential factors in the grid to obtain the layout potential of digital signage and then select grids with a high deployment potential as the place to be deployed. Compared with previous studies, by combining the two methods, the spatial characteristics and the commercial potential of the location are integrated into the content-based siteselection method. This not only improves computing efficiency but also improves the interpretability of the model. This process mainly includes three parts: data spatialization, site selection, and model verification ( Figure 2).
First, the data are processed and spatialized at multiple scales by ESRI ® ArcGIS TM 10.3. Then, the improved Huff model is used to calculate the spatial accessibility of digital signage, and machine learning methods such as the BP neural network are used to calculate the digital signage layout potential within the Sixth Ring Road in Beijing. Finally, the MCI model, BP neural network model, and Huff model are used to contrast with our model. The validity and accuracy of the digital signage site-selection model proposed in this paper are verified by cross validation and ROC curves.   First, the data are processed and spatialized at multiple scales by ESRI ® ArcGIS TM 10.3. Then, the improved Huff model is used to calculate the spatial accessibility of digital signage, and machine learning methods such as the BP neural network are used to calculate the digital signage layout potential within the Sixth Ring Road in Beijing. Finally, the MCI model, BP neural network model, and Huff model are used to contrast with our model. The validity and accuracy of the digital signage site-selection model proposed in this paper are verified by cross validation and ROC curves.

Data Processing
Due to the multisource heterogeneity of digital signage influential factors, to obtain the spatial characteristics of digital signage and influential factors, it is necessary to build a standard grid and suitable scales to obtain digital signage modelling factors. Grid-based digital signage modelling factors can not only reflect spatial feature information more realistically and intuitively but can also provide a unified spatial reference for data fusion and subsequent analysis.
In the process of spatialization, it is considered that different experimental results will occur due to different spatial scales (i.e., there is a modifiable areal unit problem [43][44][45][46][47][48][49] (MAUP)). According to Wang et al. [38], dividing the study area into a 100 m to 1000 m grid scale will meet the needs of site selection experiments. Therefore, to accurately evaluate the potential of digital signage within the Sixth Ring Road in Beijing and to find a more suitable spatial scale, this paper selects 100 m to 1000 m for spatial scale division experiments.
The specific experimental process is shown in Figure 3. First, data processing is performed on digital signage influential factor data, and then multiscale areal interpolation is performed, that is, 100 m to 1000 m standard grids are used for the features using ESRI ® ArcGIS TM 10.3. The number of standard grids at 10 different scales is shown in

Data Processing
Due to the multisource heterogeneity of digital signage influential factors, to obtain the spatial characteristics of digital signage and influential factors, it is necessary to build a standard grid and suitable scales to obtain digital signage modelling factors. Grid-based digital signage modelling factors can not only reflect spatial feature information more realistically and intuitively but can also provide a unified spatial reference for data fusion and subsequent analysis.
In the process of spatialization, it is considered that different experimental results will occur due to different spatial scales (i.e., there is a modifiable areal unit problem [43][44][45][46][47][48][49] (MAUP)). According to Wang et al. [38], dividing the study area into a 100 m to 1,000 m grid scale will meet the needs of site selection experiments. Therefore, to accurately evaluate the potential of digital signage within the Sixth Ring Road in Beijing and to find a more suitable spatial scale, this paper selects 100 m to 1,000 m for spatial scale division experiments.
The specific experimental process is shown in Figure 3. First, data processing is performed on digital signage influential factor data, and then multiscale areal interpolation is performed, that is, 100 m to 1,000 m standard grids are used for the features using ESRI ® ArcGIS TM 10.3. The number of standard grids at 10 different scales is shown in

Location Model
The site-selection process is mainly divided into two steps. The first step is to use the modified Huff model to calculate the spatial accessibility of digital signage and use the K-means clustering algorithm to classify the calculation results into 3 levels, high, medium and low, to obtain the spatial accessibility of digital signage at ten scales (from 100 m to 1,000 m). The second step is to calculate the layout potential of the digital signage in the area by the BP neural network, random forest, and

Location Model
The site-selection process is mainly divided into two steps. The first step is to use the modified Huff model to calculate the spatial accessibility of digital signage and use the K-means clustering algorithm to classify the calculation results into 3 levels, high, medium and low, to obtain the spatial accessibility of digital signage at ten scales (from 100 m to 1000 m). The second step is to calculate the layout potential of the digital signage in the area by the BP neural network, random forest, and support vector machine regression algorithms and use the clustering algorithm to classify the calculation results into high, medium, and low. Finally, overlay analysis is used to obtain the site-selection results.

Preliminary Site Selection
The preliminary site selection refers to the calculation of the spatial accessibility of digital signage in the area, and the preliminary selection results are based on the broadcast price, number of check-ins (data type indicating the audience), and the distance between the digital signage and the audience. The Huff model, as an empirical commercial site-selection model, takes the distance between the retail store and the customer and the area of the retail store itself into account. Therefore, a modified Huff model centred on the Weibo check-in data is used to calculate the spatial accessibility of digital signage. Then, the K-means clustering algorithm is used to classify the spatial accessibility calculation results, and clusters of different levels of digital signage spatial accessibility are obtained.

Huff Model
The Huff model is an empirical business location model [50]. The model uses distance and the area of the sales area to calculate the attractiveness of a retail store to consumers, that is, the probability that a customer chooses a store. Variable P ij represents the probability that a consumer at location i will spend at store j; the formula can be written as where T ij represents the time to reach the store (this refers to the calculation of the distance to the store); S ij represents the area of the sales area of the store; and β is a parameter estimated from experience that indicates the impact of the time required for consumers to form different behaviours, generally set as 2.

Modified Huff Model
Based on the calculation framework of the Huff model, combined with the experimental needs of digital signage site selection, the model was improved to concentrate on the Weibo check-in pointin the standard grid to calculate the attractiveness of digital signage to the audiences in the unit area ( Figure 4) and to characterize the spatial accessibility of digital signage, as shown in Formula (2): where i denotes Weibo check-in point data; j denotes digital signage point data; num i denotes the number of check-in times of the check-in point; p j denotes the broadcast price of the digital signage; d ij denotes the distance between the check-in point and the digital signage; and a i represents the sum of one of the check-in points (indicates the audience) in a grid and all the digital signage data points ( , indicating the ability of the check-in point to reach the digital signage. A G characterizes the average ability of a grid to reach a digital signage, and n is the number of check-in points in a grid.

K-means clustering
The K-means [51] model is a clustering method based on distance division. The clustering process is as follows: First, it is necessary to determine K centroids, that is, the number of clusters that are desired to be aggregated; second, the Euclidean distance of each index is calculated, and the distances are integrated between the calculated objects; finally, the program is run continuously, and each group randomly selects the centroid until the sum of the squared errors is the smallest [52].
It is necessary to distinguish regions with different spatial accessibility and layout potentials. Since the data are numerical data, different levels of regions need to be divided by Euclidean distance. Therefore, the K-means algorithm is needed to obtain clusters of different levels.

Calinski-Harabasz index (CH index)
The CH index is a statistical measurement index to detect the distribution effect. The specific formula is shown in formula (3) are the distances between categories and within categories, respectively. The CH index indicates that the difference between measurement categories is greater than the difference within categories; that is, when the clustering result is optimal, CH has the maximum value [53].

1
( where k B is the intraclass divergence matrix, k W is the interclass divergence matrix, and the calculation of k W and k B is as follows: where n is the number of points in the data, q C is the set of points in cluster q, c is the centre of the sample point, and q n is the number of points in class q.

K-Means Clustering
The K-means [51] model is a clustering method based on distance division. The clustering process is as follows: First, it is necessary to determine K centroids, that is, the number of clusters that are desired to be aggregated; second, the Euclidean distance of each index is calculated, and the distances are integrated between the calculated objects; finally, the program is run continuously, and each group randomly selects the centroid until the sum of the squared errors is the smallest [52].
It is necessary to distinguish regions with different spatial accessibility and layout potentials. Since the data are numerical data, different levels of regions need to be divided by Euclidean distance. Therefore, the K-means algorithm is needed to obtain clusters of different levels.

Calinski-Harabasz Index (CH Index)
The CH index is a statistical measurement index to detect the distribution effect. The specific formula is shown in Formula (3), where Tr(B k ) and Tr(W k ) are the distances between categories and within categories, respectively. The CH index indicates that the difference between measurement categories is greater than the difference within categories; that is, when the clustering result is optimal, CH has the maximum value [53].
where B k is the intraclass divergence matrix, W k is the interclass divergence matrix, and the calculation of W k and B k is as follows: where n is the number of points in the data, C q is the set of points in cluster q, c is the centre of the sample point, and n q is the number of points in class q.
The CH index is used to evaluate the clustering effect of different parameters in the above algorithm. The higher the index value, the better the clustering effect, and then the parameters in the algorithm are determined.

Calculation of Digital Signage Layout Potential
The layout potential of digital signage means selecting multiscale modelling factors as independent variables and digital signage advertising prices as dependent variables to predict the potential within a unit area. Typical prediction algorithms include the BP neural network, support vector machine regression, and random forest. Due to the difference between the dataset and the data characteristics, there is no universal prediction method applicable to all datasets; therefore, we conduct comparative experiments, taking the root mean square error (RMSE) as the evaluation index and selecting a suitable algorithm and the appropriate scale to predict the layout potential of digital signage in the region.

BP Neural Network
The BP neural networks [54] consist of input, hidden, and output layers. The training process is as follows: first, the synaptic weight and threshold matrix of the network are initialized, and the training samples are presented; second, the forward propagation and the error back propagation are calculated, then the weight is updated; and last, iteration is performed, using the new samples to perform forward propagation calculations and error back propagation calculations until the stopping criterion is met.

Support Vector Machine Regression (SVR)
SVR regression, in simple terms, finds a regression plane that makes the distance of all the data of a set to the plane the closest [55]. Different from the empirical regression model, support vector regression assumes that as long as f(x) and y do not deviate too much, the prediction can be considered correct, and the loss is not calculated. Specifically, the threshold α is set, and only |f(x)−y|>α is calculated as the loss value of the data point. The commonly used kernel functions are RBF (radial basis function), linear, and poly.

Random Forest (RF)
Random forest is a structured supervised learning method. The calculation process using the regression prediction is as follows: first, extract N sample units from the original data randomly and generate a regression tree; second, randomly extract m at each node; finally, integrate the result of each regression tree and generate predicted values [56].
In the above prediction models, the grids without digital signage are used as the training set of the model, and the grids with digital signage are used as the test set of the model.

Root Mean Square Error (RMSE)
The root mean square error is used to measure the deviation between the predicted value and the true value. The smaller the RMSE value, the smaller the error of the algorithm. The formula is To exclude the possible correlation between multidimensional features, the principal component analysis method was introduced to reduce the dimensions of multidimensional feature data, and the results of dimensionality reduction were used as model inputs.

Principal Component Analysis (PCA)
PCA is a data-compression algorithm [57]. When dealing with multidimensional data, PCA is used to filter out data attributes with higher similarity, thereby achieving the purpose of dimensionality reduction and accelerating the speed of data processing. The specific calculation process is removing the average value, calculating the covariance matrix, calculating the eigenvalues and eigenvectors of the covariance matrix, sorting the eigenvalues, retaining the eigenvectors corresponding to the first N largest eigenvalues, and transforming the data into a new space constructed by the N feature vectors obtained above.

Model Verification
In the model verification, the digital signage site-selection method combined with the modified Huff model and BP neural network model is compared with the empirical Huff model, BP neural network model, and MCI location model. The receiver operating characteristic curve (ROC curve) is used as the evaluation standard to verify the effectiveness and accuracy of the digital signage site-selection method proposed in this paper.
The ROC curve can reflect the sensitivity and accuracy of the model when selecting different thresholds. When the distribution of positive and negative samples changes, the shape of curve can be kept basically unchanged, so this evaluation index can reduce the interference caused by different test sets and more objectively measure the performance of the model itself [58].
The horizontal axis of the curve is the false positive rate (FPR). The false positive rate indicates how many negative samples in the sample are predicted to be positive samples. There are two possibilities for this case. One is to change the original negative value-the class is predicted as a positive class (false positive, FP)-and the other is to predict the original negative class as a negative class (true negative, TN), that is, The vertical axis of the curve is the true positive rate (TPR). The true positive rate represents the proportion of samples that are predicted to be positive relative to the total number of positive samples. There are two possibilities for predicting positive results. One is the positive class (true positive, TP), and the other is to predict the negative class as the positive class (false positive, FP), that is, In this paper, the false positive rate (FPR) indicates the proportion of samples without digital signage selected as samples to be deployed, and the true positive rate (TPR) means the proportion of samples with digital signage selected as samples to be deployed. The closer the ROC curve is to the upper left, the higher the true class rate, the lower the negative class rate, and the better the model effect.
In the Huff model, multiscale digital signage modelling factors are used as input data to obtain the attractiveness of the computing centre to the surrounding grids, which characterizes the possibility of digital signage in this grid. After calculating the accuracy of the model through the ROC curve, the probability is divided into multiple thresholds of 0-1 with a step size of 0.1, and finally, the accuracy of the Huff model is obtained.
In the BP neural network, multiscale digital signage modelling factors are used as input data, and the normalized digital signage broadcast price is used as output to predict the digital signage layout potential in the sixth ring of Beijing. The grid where the potential is high will be selected as the unit to be deployed. After calculating the accuracy of the model through the ROC curve, the layout potential is divided into multiple thresholds of 0-1 with a step size of 0.1, and finally, the accuracy of the BP neural network model is obtained.
In the MCI site-selection model, multiscale digital signage modelling factors are used as input data, the grid digital signage layout probability is output, and the probability value is set as the threshold of the ROC curve with a step size of 0.1. Finally, a threshold with a high accuracy rate is selected. Grids with a probability of layout greater than the selected threshold are used as units to be deployed.

Cross Validation
The cross-validation method is used to improve the accuracy of the three regression prediction models, such as the BP neural network, through multiple sampling and training processes. In the model, the training and validation datasets are randomly selected, so the quality of the training and validation data may be uneven. To reduce the accuracy loss during data segmentation, this experiment uses ten-fold cross validation and randomly divides the dataset into four. In this process, data division was repeated 10 times. Therefore, the entire training-verification process was performed 40 times. The final error is the average error of the 40 iterations.

Preliminary Site-Selection Results
The method described in Section 3.2.1 is used to calculate the spatial accessibility of digital signage in 100-1000 m standard grid units. Moreover, the K-means clustering algorithm is used to cluster the spatial accessibility results and adjust the parameters (K value) of the algorithm for multiple experiments to obtain accurate clustering results. The CH index is a statistical measurement index to detect clustering effects, usually used to evaluate the effect of the clustering algorithm. Therefore, we use the CH index described in Section 3.2.1 as an evaluation index. The clustering quality of the algorithm with different parameters is shown in Figure 5. As K increases, the CH index continues to fall; when the K value is 3, the CH index is relatively high. Therefore, the K-means algorithm is used to divide accessibility into three categories: low, medium, and high ( Figure 5). ISPRS Int. J. Geo-Inf. 2020, 9, 217 11 of 21 model, the training and validation datasets are randomly selected, so the quality of the training and validation data may be uneven. To reduce the accuracy loss during data segmentation, this experiment uses ten-fold cross validation and randomly divides the dataset into four. In this process, data division was repeated 10 times. Therefore, the entire training-verification process was performed 40 times. The final error is the average error of the 40 iterations.

Preliminary Site-Selection Results
The method described in Section 3.2.1 is used to calculate the spatial accessibility of digital signage in 100-1,000 m standard grid units. Moreover, the K-means clustering algorithm is used to cluster the spatial accessibility results and adjust the parameters (K value) of the algorithm for multiple experiments to obtain accurate clustering results. The CH index is a statistical measurement index to detect clustering effects, usually used to evaluate the effect of the clustering algorithm. Therefore, we use the CH index described in Section 3.2.1 as an evaluation index. The clustering quality of the algorithm with different parameters is shown in Figure 5. As K increases, the CH index continues to fall; when the K value is 3, the CH index is relatively high. Therefore, the K-means algorithm is used to divide accessibility into three categories: low, medium, and high ( Figure 5). The experiments are performed on 10 scales from 100 m to 1,000 m, so the spatial accessibility results are also obtained at each scale. At the same time, we use ESRI ® ArcGIS TM 10.3 to conduct a spatial query to obtain the number of Weibo check-ins in the standard grid within the Sixth Ring Road of Beijing. The number of check-ins indicates the visibility of digital signage. The highly visible grid unit is equipped with digital signage, which is more likely to be seen by the audience.
Taking a grid unit of 100 m as an example, Figure 6 (a) shows the result of spatial accessibility. The yellow grid in the figure indicates that the accessibility is low, and digital signage is urgently needed. The green grid indicates that the accessibility is medium, and digital signage can still be deployed; the blue grid indicates that the accessibility is high, and there is no need to deploy digital signage. Figure 6 (b) shows the distribution of the number of Weibo check-ins in the grid. The yellow grid in Figure 6 (b) indicates that the area has a low number of check-ins, which indicates that the area has a relatively small number of people. If digital signage is placed here, the visibility is relatively low, and the blue grid indicates that the number of check-ins is high, meaning a wider audience here. If digital signage is installed here, the visibility is high, and it has a very high deployment value. The experiments are performed on 10 scales from 100 m to 1000 m, so the spatial accessibility results are also obtained at each scale. At the same time, we use ESRI ® ArcGIS TM 10.3 to conduct a spatial query to obtain the number of Weibo check-ins in the standard grid within the Sixth Ring Road of Beijing. The number of check-ins indicates the visibility of digital signage. The highly visible grid unit is equipped with digital signage, which is more likely to be seen by the audience.
Taking a grid unit of 100 m as an example, Figure 6a shows the result of spatial accessibility. The yellow grid in the figure indicates that the accessibility is low, and digital signage is urgently needed. The green grid indicates that the accessibility is medium, and digital signage can still be deployed; the blue grid indicates that the accessibility is high, and there is no need to deploy digital signage. Figure 6b shows the distribution of the number of Weibo check-ins in the grid. The yellow grid in Figure 6b indicates that the area has a low number of check-ins, which indicates that the area has a relatively small number of people. If digital signage is placed here, the visibility is relatively low, and the blue grid indicates that the number of check-ins is high, meaning a wider audience here. If digital signage is installed here, the visibility is high, and it has a very high deployment value. Overlaying the two layers of Figure 6a,b, the grid units with medium and low spatial accessibility and a high number of Weibo check-ins are the preliminary results of the positions where the digital signage will be deployed (Figure 6c). Therefore, the primary results were mainly distributed near Weigong Village, Wangfujing, Xidan, and the main transportation lines. In general, various commercial districts and tourist attractions have greater appeal to an audience. Because of the higher number of check-ins, more visibility and relatively lower spatial accessibility, digital signage is required. As shown in Figure 6 (c), the areas around Weigongcun, Sihui Building Materials Market, Wangfujing, Xidan, and Huilongguan Wholesale Market basically cover the culture, franchise, shopping centre, and comprehensive business district. The attractiveness and influential scope of these areas are extensive and suitable for layout digital signage. With the government and universities relocating, audiences in some areas, such as Fangshan, Daxing, and Tongzhou, have gradually increased, and digital signage here will also have greater benefits.

Layout Potential Results
The layout potential of digital signage is calculated with the method described in Section 3.2.2. The RMSE described in Section 3.2.2 is generally used as an evaluation index for prediction algorithms. This paper uses it to evaluate the prediction effects of five regression prediction algorithms. The results are shown in Table 3 and Figure 7. With the increase of the grid scale, the RMSE values of five algorithms generally increased gradually. When the scales are 100 m, the errors are relatively small, and the RMSE values of the five algorithms are 0.277, 0.265, 0.268, 0.271, and 0.268. The relatively small RMSE value is 0.265, and the algorithm with a relatively small error is the BP neural network (Figure 7). Therefore, the digital signage modelling factors at 100 m grid scale and the BP neural network algorithm are selected to calculate the digital signage layout potential within the Sixth Ring Road in Beijing. The spatial accessibility of digital signage is mainly affected by the distance between the signage and the audience, the number of check-ins, and the price and quantity of the digital signage. Therefore, the primary results were mainly distributed near Weigong Village, Wangfujing, Xidan, and the main transportation lines. In general, various commercial districts and tourist attractions have greater appeal to an audience. Because of the higher number of check-ins, more visibility and relatively lower spatial accessibility, digital signage is required. As shown in Figure 6c, the areas around Weigongcun, Sihui Building Materials Market, Wangfujing, Xidan, and Huilongguan Wholesale Market basically cover the culture, franchise, shopping centre, and comprehensive business district. The attractiveness and influential scope of these areas are extensive and suitable for layout digital signage. With the government and universities relocating, audiences in some areas, such as Fangshan, Daxing, and Tongzhou, have gradually increased, and digital signage here will also have greater benefits.

Layout Potential Results
The layout potential of digital signage is calculated with the method described in Section 3.2.2. The RMSE described in Section 3.2.2 is generally used as an evaluation index for prediction algorithms. This paper uses it to evaluate the prediction effects of five regression prediction algorithms. The results are shown in Table 3 and Figure 7. With the increase of the grid scale, the RMSE values of five algorithms generally increased gradually. When the scales are 100 m, the errors are relatively small, and the RMSE values of the five algorithms are 0.277, 0.265, 0.268, 0.271, and 0.268. The relatively small RMSE value is 0.265, and the algorithm with a relatively small error is the BP neural network (Figure 7). Therefore, the digital signage modelling factors at 100 m grid scale and the BP neural network algorithm are selected to calculate the digital signage layout potential within the Sixth Ring Road in Beijing.
To exclude the correlation between 19 types of digital signage modelling factors from the experiment, PCA is introduced to perform analysis of the above features to achieve the purpose of feature dimension reduction, thereby reducing the RMSE value and improving the accuracy of the regression analysis of digital signage potential. It can be seen from the results after using PCA (Table 4, Figure 8) that the minimum value of RMSE at this time is 0.264, which is relatively smaller than that before dimension reduction. After extracting the principal components of the modelling factor sequence, the calculation error of the digital signage potential is relatively reduced.   To exclude the correlation between 19 types of digital signage modelling factors from the experiment, PCA is introduced to perform analysis of the above features to achieve the purpose of feature dimension reduction, thereby reducing the RMSE value and improving the accuracy of the regression analysis of digital signage potential. It can be seen from the results after using PCA (Table 4, Figure 8) that the minimum value of RMSE at this time is 0.264, which is relatively smaller than that before dimension reduction. After extracting the principal components of the modelling factor sequence, the calculation error of the digital signage potential is relatively reduced.   We take the digital signage modelling factor at a 100 m grid scale after dimensionality reduction as input and the normalized digital signage broadcast price as the output. The model is trained for the digital signage layout potential. The K-means clustering algorithm divides the result into three levels, low, medium, and high, as shown in Figure 9. Among the grids, the blue area is the area with high potential for digital signage deployment, mainly distributed in the CBD, Olympic Park, and other commercial areas and tourism areas. The result is consistent with the digital signage that should be deployed in a large number of check-in locations, likely to be seen by the audience with a higher probability. The green area shown in the figure is a medium-level area of layout potential, mainly distributed in Fangshan Changyang, Tongzhou Baliqiao, Daxing Zaoyuan, and other places. These developing business districts are characterized by large populations and fewer commercial facilities, so more digital signage needs to be deployed. The yellow areas shown in the figure are the areas with a low potential for deployment. There are basically no large-scale shopping malls or business districts, and the areas are less attractive to nearby residents. The possibility of digital signage being noticed by the audience is low, which leads to the low layout potential of digital signage in this place, and deployment is not recommended. We take the digital signage modelling factor at a 100 m grid scale after dimensionality reduction as input and the normalized digital signage broadcast price as the output. The model is trained for the digital signage layout potential. The K-means clustering algorithm divides the result into three levels, low, medium, and high, as shown in Figure 9. Among the grids, the blue area is the area with high potential for digital signage deployment, mainly distributed in the CBD, Olympic Park, and other commercial areas and tourism areas. The result is consistent with the digital signage that should be deployed in a large number of check-in locations, likely to be seen by the audience with a higher probability. The green area shown in the figure is a medium-level area of layout potential, mainly distributed in Fangshan Changyang, Tongzhou Baliqiao, Daxing Zaoyuan, and other places. These developing business districts are characterized by large populations and fewer commercial facilities, so more digital signage needs to be deployed. The yellow areas shown in the figure are the areas with a low potential for deployment. There are basically no large-scale shopping malls or business districts, and the areas are less attractive to nearby residents. The possibility of digital signage being noticed by the audience is low, which leads to the low layout potential of digital signage in this place, and deployment is not recommended.

Results and Analysis of Digital Signage Location
To verify the accuracy of the abovementioned digital signage site-selection method, we compare our method with the BP neural network model in the machine learning method and the Huff and MCI empirical model of commercial geography. The ROC curve (described in Section 3.3) characterizes the accuracy of the site-selection results. A lower false positive rate and higher true

Results and Analysis of Digital Signage Location
To verify the accuracy of the abovementioned digital signage site-selection method, we compare our method with the BP neural network model in the machine learning method and the Huff and MCI empirical model of commercial geography. The ROC curve (described in Section 3.3) characterizes the accuracy of the site-selection results. A lower false positive rate and higher true positive rate, that is, the closer the ROC curve is to the upper left, indicate the model has higher accuracy and better site selection effect.
The location results of each model are shown in Figure 10. The result of the BP neural network model is shown in Figure 10a. Using 100-1000 m modelling factors as input of the BP neural network model, as the modelling scale increases, the ROC curve gradually approaches the upper left of the coordinates; at 700 m, the experimental effect is the best and gradually decreases after 800 m. Therefore, using the BP neural network model to predict the location of digital signage, the prediction effect is the best at the 700 m scale. is relatively high and its false positive rate is relatively low-the curve is closest to the top left, which shows the location effect of this experiment is relatively good, followed by the BP neural network model. Both the Huff model and the MCI model are probabilistic models, and the location effects of the two are similar, thereby verifying the accuracy and reliability of this experiment. In summary, the current methods and parameters and the results of the site selection are relatively good, and the results may be more satisfactory if we change the methodology. The abovementioned preliminary signage site-selection results and digital signage layout potential results are overlaid to analyse areas that simultaneously satisfy low spatial accessibility, high check-in numbers, and high layout potential (areas for digital signage to be selected), as shown in Figure 11.
As shown in Figure 11, the sites are mainly located in Sanlitun, 798 Art District, Beijing West Railway Station, and other places. These locations are divided into three categories: (1) Olympic Park, Sanlitun, 798 Art District, and other places in the figure that are convenient for transportation, have complete catering and entertainment facilities, and are part of the well-known cultural and entertainment industry zone in Beijing. This area has a large number of check-ins and high potential for deployment. Deploying more digital signage is recommended. (2) Fangshan Liangxiang, Daxing Huangcun, Tongzhou Beiyuan, etc., have gradually increased their commercial development due to the construction of Beijing's subcentres and the relocation of some universities. This area has high The Huff model is one of the main methods of commercial site selection, and its variables are the store area and the distance of the store from the neighbourhood. The Huff model was modified to adapt to the digital signage site-selection experiment. When the modelling scale is 100 m, the model effect is better. As the modelling scale increases, the model effect gradually decreases and reaches a certain convergence (Figure 10b).
The MCI model characterizes the probability of the audience choosing this region for consumption under the influence of the population, economy, and other geographical environments [29]. The mentioned multiscale factors are brought into the model, and the locations with higher probability are recommended as the new locations of digital signage. At each modelling scale, as the threshold increases, the model effect gradually converges. When the modelling scale is 100 m, the model location results are better. As the modelling scale increases, the MCI model location effect gradually decreases (Figure 10c).
The ROC curve of the Huff-BP model proposed in this paper is the blue curve shown in Figure 10d. Based on the selection of the ROC curves of the above site selection models (BP neural network, Huff model, and MCI model) with better scales, it can be seen that the true positive rate of Huff-BP is relatively high and its false positive rate is relatively low-the curve is closest to the top left, which shows the location effect of this experiment is relatively good, followed by the BP neural network model. Both the Huff model and the MCI model are probabilistic models, and the location effects of the two are similar, thereby verifying the accuracy and reliability of this experiment. In summary, the current methods and parameters and the results of the site selection are relatively good, and the results may be more satisfactory if we change the methodology.
The abovementioned preliminary signage site-selection results and digital signage layout potential results are overlaid to analyse areas that simultaneously satisfy low spatial accessibility, high check-in numbers, and high layout potential (areas for digital signage to be selected), as shown in Figure 11. When we trained the model, we used the digital signage point data and digital signage modelling factors within the Six Ring Road of Beijing and adjusted the parameters based on the training results, so the model is more suitable for the digital signage site selection experiment within the Six Ring Road of Beijing, that is, the results are adaptable to this specific study area.
In summary, we selected some grids from a relatively macro scale that are suitable for digital signage layout within the Six Ring road of Beijing and provide a digital signage site selection method. Moreover, we provided a research idea that combines empirical site selection methods with machine learning methods to improve the interpretability, computing efficiency. Our study area is in Beijing, training model with data of Beijing, so it is likely more suitable for cosmopolitan cities with a large number of check-ins and high prosperity. For other commercial facility location issues, the site selection method proposed in this paper can also be used as a reference.

Conclusions
With the wide use of digital signage, the precise location of digital signage is an urgent issue for digital signage companies and management agencies. We proposed a hybrid Huff and BP neural network model that comprehensively considers the population, economy, transportation, and other factors that affect the location of digital signage and performs standard grid processing to form multiscale modelling factors. We modified the Huff model to calculate the spatial accessibility of digital signage and comprehensively calculate the layout potential of digital signage using multiple machine learning methods such as BP neural networks. The Huff-BP, MCI, Huff, and BP neural network models were compared. The conclusions are as follows: As shown in Figure 11, the sites are mainly located in Sanlitun, 798 Art District, Beijing West Railway Station, and other places. These locations are divided into three categories: (1) Olympic Park, Sanlitun, 798 Art District, and other places in the figure that are convenient for transportation, have complete catering and entertainment facilities, and are part of the well-known cultural and entertainment industry zone in Beijing. This area has a large number of check-ins and high potential for deployment. Deploying more digital signage is recommended. (2) Fangshan Liangxiang, Daxing Huangcun, Tongzhou Beiyuan, etc., have gradually increased their commercial development due to the construction of Beijing's subcentres and the relocation of some universities. This area has high commercial value and layout potential, and digital signage is recommended. (3) Beijing Railway Station, Beijing West Railway Station, Beijing South Railway Station, Siyuanqiao, etc., are passenger logistics gathering places. These locations have high population mobility throughout the year, and there are many commercial outlets nearby, so digital signage is also recommended.
When we trained the model, we used the digital signage point data and digital signage modelling factors within the Six Ring Road of Beijing and adjusted the parameters based on the training results, so the model is more suitable for the digital signage site selection experiment within the Six Ring Road of Beijing, that is, the results are adaptable to this specific study area.
In summary, we selected some grids from a relatively macro scale that are suitable for digital signage layout within the Six Ring road of Beijing and provide a digital signage site selection method. Moreover, we provided a research idea that combines empirical site selection methods with machine learning methods to improve the interpretability, computing efficiency. Our study area is in Beijing, training model with data of Beijing, so it is likely more suitable for cosmopolitan cities with a large number of check-ins and high prosperity. For other commercial facility location issues, the site selection method proposed in this paper can also be used as a reference.

Conclusions
With the wide use of digital signage, the precise location of digital signage is an urgent issue for digital signage companies and management agencies. We proposed a hybrid Huff and BP neural network model that comprehensively considers the population, economy, transportation, and other factors that affect the location of digital signage and performs standard grid processing to form multiscale modelling factors. We modified the Huff model to calculate the spatial accessibility of digital signage and comprehensively calculate the layout potential of digital signage using multiple machine learning methods such as BP neural networks. The Huff-BP, MCI, Huff, and BP neural network models were compared. The conclusions are as follows: (1) A set of multifactor and multiscale digital signage modelling factors was constructed. Based on the influential factors of digital signage, such as broadcasting price, POI facilities, census, transportation network centrality index, and social network check-ins, we unified the spatial scale with multiscale grid processing of 100 m to 1000 m.
(2) A digital signage site-selection method that combines a machine learning algorithm with a modified Huff model was proposed. Based on the location and price of digital signage, the Huff model was improved to calculate the spatial accessibility of digital signage, and preliminary selection was made for the unit. By using different prediction algorithms to calculate the digital signage deployment potential, the results show that the 100 m scale and BP neural network algorithm had better performance than RF and SVR algorithm under the other scales. The results of site selection were obtained by overlaying the preliminary selection results and the layout potential results.
(3) The site-selection results indicated that the areas suitable for digital signage are mainly distributed in Sanlitun, Wangfujing, Financial Street, Beijing West Railway Station, and areas along the main road network. These areas are mostly well-known cultural and entertainment industrial districts and passenger logistics gathering places in Beijing. They have relatively better commercial outlets, a higher population, and convenient transportation, which can maximize the benefits of digital signage.
Our research provides a reference for integrating geographical features and content data into the site-selection algorithm for digital signage. Furthermore, we provide some grids from a relatively macro scale that are suitable for digital signage layout within the Six Ring road of Beijing and provide a digital signage site selection method. The method can effectively improve the accuracy and scientific nature of digital signage deployment, maximize the deployment benefits, and optimize the allocation of digital signage resources. In addition, this paper provides a research idea that combines empirical methods with machine learning methods to improve the model' s interpretability and computing efficiency. For other commercial facility location issues, the location method proposed in this paper can also be used as a reference. However, one of the most important factors for the site location of digital signage is visibility from streets. Therefore, our future research will aim at the location of digital signage along the street network. Based on the micro-location theory, introducing factors, such as the position and orientation of digital signage into more precise location experiments, and incorporating constraints into the model, such as user needs and deployment costs, will achieve an optimal layout of digital signage.