A Survey of Mobile Laser Scanning Applications and Key Techniques over Urban Areas

: Urban planning and management need accurate three-dimensional (3D) data such as light detection and ranging (LiDAR) point clouds. The mobile laser scanning (MLS) data, with up to millimeter-level accuracy and point density of a few thousand points/m 2 , have gained increasing attention in urban applications. Substantial research has been conducted in the past decade. This paper conducted a comprehensive survey of urban applications and key techniques based on MLS point clouds. We first introduce the key characteristics of MLS systems and the corresponding point clouds, and present the challenges and opportunities of using the data. Next, we summarize the current applications of using MLS over urban areas, including transportation infrastructure mapping, building information modeling, utility surveying and mapping, vegetation inventory, and autonomous vehicle driving. Then, we review common key issues for processing and analyzing MLS point clouds, including classification methods, object recognition, data registration, data fusion, and 3D city modeling. Finally, we discuss the future prospects for MLS technology and urban applications.


Introduction
Accurate three-dimensional (3D) point cloud data have been an important data source for 3D urban models, which are an integral part of urban planning, simulation, mapping and visualization, emergency response training, and so on [1]. Mobile laser scanning (MLS) is an emerging technology for generating highly accurate 3D point clouds and thus have many potential urban applications [2]. A MLS system consists of 3D laser scanner, GNSS (global navigation satellite system), IMU (inertial measurement unit), and camera. They are usually mounted on moving platforms (e.g., vehicle, boat, backpack, robot), and can collect 3D surface information along the driving paths. Due to the short measure range and flexibility of data acquisition, a MLS system can acquire very accurate (millimeter-level) point clouds of high point density (up to a few thousand points/m 2 ) [3][4][5].
Given those advantages, MLS data have been used in recent years in a wide range of urban applications, including urban land cover analysis [6][7][8][9], digital 3D city modeling [10,11], urban environment monitoring [12][13][14][15][16], and autonomous vehicle driving [17][18][19]. The use of these data has involved techniques such as data fusion and classification (e.g., using machine learning approaches) developed from the remote sensing, computer vision and robotics communities [20,21]. We searched the database of Web of Science core collection with the topics "mobile laser scanning", "mobile LiDAR and urban", and "mobile LiDAR and classification" from 2008 to 2018 (Figure 1). This showed that the numbers of research papers on mobile LiDAR has been accelerating, while those on MLS urban applications and data classification have been steadily increasing.  Zhang and Lin (2016) focused on the data fusion methodology of optical images and LiDAR point cloud, and reviewed its various applications, including registration, classification, 3D reconstruction and change detection [25]. Wang et al. (2018a) reviewed the existing 3D urban reconstruction techniques, such as computer graphics, computer vision and photogrammetry disciplines, focusing on the architectural modeling and generic mesh reconstruction [1]. Puente et al. (2013) introduced the mobile mapping and surveying technologies, reviewed the MMSs and their main categories [3]. Guan et al. (2016) informed the advancements of MLS technologies and their applications in road information inventory [26]. However, these studies mostly focused on the ALS point clouds, remote sensing images, and certain types of data processing methods. No reviews have been dedicated to various MLS applications over urban areas. This article aims to fill in this gap by summarizing the recent developments of MLS systems, introducing their typical applications in urban areas, and discussing the key issues of data processing and analysis.
MLS has been developing rapidly, and a review of all aspects related to its technologies and applications would be a challenge and is thus beyond the scope of this review. Instead, we focus on the applications of MLS technologies over urban areas, where MLS has been most frequently used. The review is organized as follows: Section 2 introduces the characteristics of MLS systems and their point clouds and presents the relevant opportunities and challenges. Section 3 summarizes MLS applications in urban areas. Section 4 discusses the key issues for MLS data processing and analysis. The future developments of MLS are discussed in Section 5, and conclusions are drawn in Section 6.

Characteristics of MLS Point Clouds
The main advantage of the MLS point cloud is its ultra-high point density, which could be up to a few thousand points per square meter with centimeter-level point spacing. In contrast, the point density of ALS data is usually <10 point m -2 and typical point spacing is 30-50 cm. The point density of TLS (Terrestrial Laser Scanning) data could reach the same level as MLS, but TLS point clouds are often captured in a stop-and-go mode, often without the IMU system, which makes it not easy to register directly the point clouds collected at different scanning positions. Moreover, TLS data has a very inhomogeneous point distribution and much lower productivity than MLS. The continuous collection of MLS point clouds of high point density allows a capture of detailed road features such as curbs and surface road marking ( Figure 2).
However, MLS data have some challenges for their boarder application in urban areas: (1) the ultra-high point density of mobile LiDAR data comes with the expense of large file size (on the order of Gigabytes for every mile of data), which makes it relatively slow to process and render [27]; (2) automatic methods are still not efficient for detecting finer and more connected features such as road curbs and markings, which are crucial for autonomous vehicle driving and intelligent transportation; and (3) despite its high sampling density over roadways, MLS data are not able to map out-of-the-sight terrain beyond the roadways due to occlusions caused by buildings and trees. In contrast, ALS data from a high vantage point or high-resolution imagery can more easily have a complete coverage over a large geographical extent. Therefore, MLS data are complementary with other data sources for large-area urban applications.

MLS Applications over Urban Areas
Urban areas have a mix of both built-up and natural objects, including bare ground, grass cover, trees, roads, and buildings. The analysis of urban land cover and land use has conventionally used optical imagery and, in the 21st century, ALS point clouds [7]. In recent years, the advantages of MLS for extracting finer scale objects with detailed information have been recognized [28]; see Figure 3 for a sample of features that can be extracted from MLS point cloud. Many urban applications based on MLS have emerged, including road environment reconstruction [5,29,30], single tree stem modeling and pole-like furniture detection [31-34], building roof segmentation [35], and other utilities reconstruction [1,[36][37][38]. Here, we organize the urban applications of MLS data into several categories: transportation infrastructure mapping, vegetation mapping and inventory, building information modeling, urban utility surveying and mapping, and autonomous vehicle driving. Figure 3. Urban scene mapping and modeling based on MLS data. The objects are roughly detected from the more detailed MLS point clouds. The A represents road markings, the B represents road curbs, the C represents zebra stripes, the D represents trees, the E depicts street poles, the F depicts traffic light or advertising boards, and the G depicts buildings.

Transportation Infrastructure Mapping
MLS data have considerable use for road surface object classification, road environment modeling and 3D road asset inventory [5,14, [46], cars [47], and traffic sign [48,49] have been investigated. The detected information provides basis support for transportation infrastructure management.
Early studies reported that the road markings, zebra crossings, centerline, and curbstones can be identified with mean accuracies of about 80% [50][51][52]. Road surface objects, including urban road manholes, and left and right road edges, were also extracted from MLS point clouds [38,40,42]. These studies mostly applied image-processing algorithms and supervised classification methods. Some recent studies have shown that the accuracy of extracting roads was up to 90% by tracking and refining the detected curb points [39, [53][54][55]. The finer vertical profiles, including linear parameters (slope and vertical curves) and cross-sections of roads, could be modeled [56]. This automatic processing approach used a hierarchical strategy, and the error of experimental results was under 3.5%. Some studies mapped detailed road surface features and road marking types with high accuracy [14,41,57], in which the average correctness of road surfaces could be up to 98.3. By validating with real test sites and a fully automatic approach, these proposed methods have shown satisfactory and feasible performance. Such extracted transportation infrastructure information is useful for urban facility maintenance, autonomous vehicle assistance, and navigation.

Building Information Modeling
MLS data with high point density allows the extraction and 3D reconstructions of buildings for building information modeling (BIM) [58][59][60][61][62]. MLS data have been used to identify building shape, structure, boundary, façade, roof, and outdoor/indoor environments [12,14,30,[63][64][65][66] and street-level building change detection [32,67], and thus have gained attention in the fields of architecture, engineering, and construction [68,69]. The automatic reconstruction method of permanent structures, such as walls, floors and ceilings, combined with feature primitive extraction and graph-cut formulation. Therefore, different wall directions could be extracted accurately and the Hausdorff distance with ground truth was under 2.6 cm. For evaluating the robustness of these methods, the sparse sampling with down-sampled instances were produced with 50%, 20%, 10% and 5% of the original point cloud. With adjusted parameters, the experiments showed that the reconstruction of the 50% data set provided the same amount of details as the original one. Some wearable devices and autonomous robots mounted 3D laser sensors have been used for navigation in outdoor environments [70] or simultaneous localization and mapping (SLAM)-driven robotic mapping in indoor environments [71][72][73][74][75]. Conventionally, MLS have been used for mapping buildings and roofs [58,[76][77][78]. In recent years, new applications such as generating building models with multiple levels of detail for accessibility diagnosis have begun to emerge [79,80].

Utility Surveying and Mapping
The extraction of street-scene objects from mobile LiDAR point clouds is a crucial task for urban utility surveying and mapping [39,53]. A few studies have tried to detect and classify pole-like objects (such as light pole and utility poles) and street-side car parks from MLS point clouds [33, 81,82]. Moreover, some studies have considered the pillar structure and geometric index feature for detecting vertical elements in complex road environments [83,84], and their classification rates could be up to 90%. However, misclassification often occurred over man-made poles adjacent to trees or low-height traffic signs. By considering the spatial relationship, the finer detection of pole-like road furniture was approximately 95% [85], and the optimized decomposition improved the correctness by 7.3% and 18.4% in the test areas. Furthermore, studies have investigated heuristic segmentation, various classifiers, and the decomposition of road furniture into different components [85][86][87], including traffic signs, traffic lights, advertisement panels, and trees. The overall accuracy of the classified poles achieved 96%, and advertisement panels might be classified as traffic sign due to their reduced panel size and shape similarity.

Vegetation Mapping and Inventory
Vegetation mapping and inventory is another important application field. MLS has been used to derive information related to vegetation structure or species. For example, 3D individual trees could be segmented from MLS data using imagery processing techniques [53]. Puttonen et al. (2011) combined MLS and hyperspectral data for tree species classification and found that the combined dataset outperformed single-sensor data, and separated the coniferous and deciduous trees with overall classification accuracy of 83% [88]. To classify tree species, the point clouds were usually first converted to 3D voxels, followed by segmentation or classification of these voxels [89,90], and the overall accuracy of urban tree species classification achieved 86.1% . By applying a deep learning model, the average classification accuracy could be over 93.1%, while the number of projection images for rasterization had impact on the training and testing stages. Therefore, it should consider more effective ways to express 3D objects.

Autonomous Vehicle Driving
MLS data can be used for perceiving street environments [82] and detecting the precise position, orientation and geometric features of vehicles [91]. Therefore, MLS is a crucial component for visual perception system in autonomous vehicle [17]. For autonomous vehicle driving, three MLS applications are particularly important: vehicle and pedestrian detection, lane detection, and drivable surface detection [18,19,92]. By experimenting with parked/moving autonomous vehicle platform within real urban environment, the true positive rates of pedestrian recognition were over 96% and 93%, respectively. Based on convolutional neural network and image upsampling theory, the car, cyclist, pedestrian, and truck were classified with average accuracy of 97%. Compared to traditional RGB images, the four-channel RGB-LiDAR image data was able to reach approximately 5% higher accuracy over different scales of training/testing sets. MLS is often combined with cameras (optical sensor) and RADARs (electromagnetic sensor) for timely prediction of pedestrian motions, and even constructing a visual perception system [19, [93][94][95][96][97][98][99]. With MLS, it is possible to determine the real-time navigation paths of autonomous mobile robots in unknown environments [3,100], while detecting obstacles and reconstructing and mapping the environment [36,95,101].

Key Issues for MLS Application and Technology
For the applications described in Section 3, many key technologies of MLS data processing and analysis are involved ( Figure 4). First, the raw MLS point cloud often needs to be classified to identify the class for each point. Second, individual objects can be identified from MLS raw or classified point clouds. Third, MLS data are often fused with other data sources such as ALS point clouds and optical imagery to improve the classification and detection accuracy. Therefore, the registration of MLS with ALS point clouds and/or imagery is important. Finally, the detected and classified objects from MLS point cloud can be used for 3D city modeling.

Classification Methods
The semantic classification of MLS point clouds is a crucial and prerequisite task for various applications, including transportation infrastructure mapping, vegetation mapping and inventory, building information modeling, utility surveying and so on. The classification methods can be grouped as follows: (1) Unsupervised classification or segmentation with prior knowledge. This group of methods classify points based on multiple thresholds, which can be predetermined based on the characteristics of point cloud data or prior knowledge (e.g., related to height, density, or intensity). The procedures usually involved transforming point clouds to 2D images or triangular irregular network (TIN) and then voting in the parameter space. The specific methods varied from simple statistical analysis [33,102], HT (Hough transform) [ [53][54][55] used the partition strategy of consecutive "scanning lines" and moving window filtering methods to extract roads. In contrast, others are to classify points from different objects. For example, the mean-shift algorithms with different input variables (such as range, smoothed normal, texture information, and RGB of image color) were utilized for urban scene segmentation [24, 105,106].
(2) Supervised classification. Compared to unsupervised methods, supervised methods classify points using the rules learnt from the training data. The choice of classifier and input features are important factors for supervised classification methods [107,108]. Before classification, raster images were often generated by interpolation from height and intensity of MLS data. Then, supervised methods such as RF (Random Forests), SVM (Support Vector Machine), Bayesian network, NN (Neural Network, including deep learning) were applied for urban object classification [38]. The extraction of distinct features from LiDAR point cloud is a crucial step for many classifiers. These features often are computed from points and their neighbors, representing the spatial structures of local context. Different neighborhood types and scales were proposed for computing the spatial structure for each considering point, including the spherical, cylindrical, and k nearest neighborhoods, single-scale and multi-scale neighborhoods [109]. In particular, an optimal eigenvalue-based and slant cylindrical neighborhoods were proved to increase the distinctiveness of features and more suitable for certain object classification [110][111][112]. The extraction of local spatial features is computation-intensive, so a key challenge is to balance the accuracy and computation needs of the procedures.

Object Recognition
Object recognition is the task for extracting or detecting geographical objects of particular types from point clouds, which might involve 3D reconstruction and shape restoring. For successful object detection and recognition, it is critical to extract affine-invariant descriptors of geometric structure [113]. In this regard, researchers have introduced covariance-based feature descriptors [114][115][116]. Content-based shape descriptors have also been proposed to identify 3D objects such as street light poles, road markings, zebra crossings, and tunnel luminaires [26, 51,113,117,118]. However, current research of the object recognition is mainly focused on one type of object or improvement after point cloud classification. Further studies are needed to develop frameworks for semantically classifying point clouds, extracting 3D shape descriptors, and detecting objects simultaneously.

Data Registration
Optical imagery and ALS data can be combined with MLS to provide complementary information for urban environmental mapping. However, accurate geometric registration of different data sources is a crucial issue for "geospatial big data" applications [119,120]. The GNSS/IMU system commonly onboard a mobile mapping system makes it possible to directly georeference the laser scanner and optical camera data into a common map coordinate system. However, the georeferencing quality is often challenged by the GNSS signal perturbation or even denial due to tall buildings, trees, and tunnels. A synergistic use of GNSS, IMU, and odometer information is often proposed to improve the estimation of position and orientation information of the platform. To achieve the best accuracy when registering different sensor data via direct georeferencing, the misalignment among sensors needs to carefully calibrated (through either indirect or direct sensor orientation), and their time needs to be rigorously synchronized. In addition, the data registration among different sensors can be done as discussed in the following studies: (1) Registration of MLS with other point clouds. The point clouds from different sources can be aligned in a common, global coordinate system by a pairwise registration and individual transformations, using the overlap of point clouds to establish a link [121][122][123]. Some studies have introduced artificial targets (such as spheres) in the registration of multi-view point clouds [124,125], in which the centroids of the detected targets were used for rough registration, and the position, orientation, and geometric shapes were used for refining the results. Cheng et al. (2015Cheng et al. ( , 2018 proposed a hierarchical, coarse-to-fine registration strategy for registering laser scanning point clouds [122,123]. The road networks and 3D building contours were used for the initial coarse and fine registration, respectively. Often, local feature descriptors and geometric key point descriptors have been designed for efficient 3D point cloud registration, involving a comprehensive description of local shape geometries about local depth, point density, and angles between normal [114,126]. An automatic non-rigid registration method for multi-strip MLS point clouds was also proposed by Yan et al. (2018) [125]. This method was verified with MLS data scanned in city road and highway environment, and the root mean square errors were about 4-5 cm.
(2) Registration of MLS with images. The combination of MLS point clouds with optical images is another important way of data registration [127]. Some recent studies have introduced semantic features-based, sensor-constellation-based and image-to-geometry registration methods [127][128][129]. the first method applied the semantic features extracted from panoramic images and point clouds. The consumer-grade GPS and IMU aided structure from motion was used to estimate the rotation parameters, and the final errors were less than three pixels with two challenging urban scenes; the sensor constellation method first applied the intersection feature points, which lie in the connecting line between global positioning system antenna and the panoramic camera, to separate the point clouds into blocks. Then, the accurate corresponding pixels in relevant images could be found by the relationship of position and orientation between different sensors [128]; the image-to-geometry method consisted of generating a wide-angle synthetic image, translation in the image plane, and fine registration by a new image metric [128,129].

Data Integration
Due to the differences in sensor scanning angle, time and pose, different data sources (MLS, ALS, and imagery data) can be combined to complement each other for geospatial applications [7,[130][131][132][133][134][135]. For example, optical imagery, captured using an unmanned aerial vehicles (UAV) system or aerospace sensor, and LiDAR point clouds can be combined for DTM generation, land use and land cover classification, building extraction, and 3D construction [6,25,136,137]. Conventionally, aerial images and ALS are commonly utilized for urban land-use and land cover mapping [6,130,138]. Research showed that the usual synergistic use of LiDAR point clouds, multispectral and normalized difference vegetation index (NDVI) data provided significant advantages based on several channels, and thus could achieve more accurate urban land cover classification results than single-sensor data. However, these aerial datasets might not provide sufficient details for analysis over complex urban areas [139]. MLS can be combined with other data information to address this issue. The fusion of 3D point clouds and 2D images can be applied for the semantic segmentation of large-scale urban scenes, obstacle recognition, road detection, and autonomous vehicle driving [19,97,[140][141][142]. There are different levels of data integration for MLS point clouds and other data sources in Table 2. The processing procedures are simple and easy. However, it should develop independent methods for different data sources. The accuracy of results is usually not very high.
[32, 76,88,97,139,141,143] Medium First, processing one data source to obtain useful features or background knowledge. Then, based on this computed information, analyzing the other data to classify or extract final results.
The referenced information improves the accuracy of results. It is difficult to design and execute the serial procedures. It also needs the procedures to be much related. [5,29,48,127,129,140,144] High First, computing and extracting features from MLS point clouds and other data sources, directly. Then, analyzing and classifying based on the concentrated features as one type of data source.
The processing could utilize adequately all the data sources and obtain high accuracy. It needs complex methods and professional knowledge.

3D City Modeling
The research of 3D digital city modeling is becoming popular [145]. With MLS data of high point density, not only the different types of objects (e.g., ground, roads, trees, buildings, utilities) can be classified and detected, but also the detailed city model could be 3D regularized and reconstructed geometrically. MLS 3D point clouds can overcome the limitations of very low productivity and low geometric accuracy in real-world high-quality 3D city modeling [146]. The images and point clouds acquired by a MLS system are usually combined for texture mapping and semantical labelling for constructing 3D city mesh [143,144]. Moreover, MLS data provide an efficient solution for automatic geometry generation and shape detection for BIM and city models [32, 69,147]. Recently, multiple cooperative robots with 3D lasers have also been applied for automatic large-scale 3D modeling [148,149].

Emerging of New MLS Systems and Applications
MLS data have been increasingly used in the urban sensing recognition applications, such as transportation infrastructure mapping and autonomous vehicle driving. However, their applications have been largely limited to roads where vehicles can drive. To access other areas in complex urban environment, there is a growing need to develop and use MLS systems on non-vehicle platforms, such as backpacked MLS, to collect data over areas where vehicle MLS cannot reach. Another trend is the integration of MLS with other instruments such as robot probe units, road disease detection robots, guiding robots, and UAVs for in-depth diagnosis and more detailed measurements of urban environments (e.g., detect surface health of roads and monitor micro-level shape changes). The applications of MLS have also been expanding from conventional street environmental mapping and perception to other areas such as inverse engineering for industrial control, SLAM, risky environment exploration. MLS data can also be integrated with high resolution satellite imagery for urban analysis over relatively large areas.

Data Processing Framework Tailored to MLS
Compared to ALS, the much higher point density of MLS could provide more detailed 3D spatial information for object classification and environment modeling, such as road surface [50,54,85,150], building [53,76,77], power line [151,152]. However, MLS has the challenges of large data processing. Few analysis methods have been tailored for processing massive MLS point clouds while taking full advantage of their high point density [153]. The MLS data provides more profile information along the scanning paths, while ALS provides vertically earth surface representation. The laser beam scatters along the horizontal scanning direction. The registration between MLS and ALS data also needs efficient methods and further study. Moreover, the MLS has been increasingly applied for indoor/underground real-time navigation and SLAM. This means that accurate real-time processing algorithms are needed.

Machine Learning Based Classification Methods
Classifying 3D LiDAR point clouds is often a critical step for many urban applications. To address this issue, machine learning methods are increasingly applied [90,[154][155][156][157][158]. Although these methods have been successfully used for high-resolution 2D imagery classification [159][160][161][162][163][164], extending their applications to 3D MLS data need to consider the informative yet irregular 3D distribution of point clouds. Some relevant key issues include: First, it is important to select suitable classifiers and features. Some classifiers, such as SVM, RF, and NN, have been used for road, building, pole-like object and power line extraction from MLS [82,86,87,151,165]. The semantic constraints, roughness parameter, topological structural information and clustering algorithms were considered for unsupervised classification [166][167][168]. However, identifying useful features and classifiers for a wide range of applications deserves more investigation.
Second, deep learning, especially convolutional neural network, have experienced explosive growth in computer vision for image perception and thus is a promising technique for automatic classification of point cloud. However, most existing deep learning frameworks were developed for processing 2D images with pixels over a regular grid, not for 3D point clouds that are irregularly distributed in a 3D space. Therefore, it is difficult to directly apply the commonly used neural network architecture for semantic classification from MLS data. Some studies proposed the data transformation methods from 3D point clouds to 2D images, before using them as inputs to multi-layer neural network. Another alternative method was the data dimensionality reduction. However, such processing resulted in information loss, larger classification errors, and more computation. Therefore, fast, accurate, and efficient deep learning frameworks tailored for processing massive MLS point cloud are urgently needed.
Third, automation and efficiency is another concern for classification methods. For many supervised or semi-supervised classification methods, it is very costly and time consuming to produce ground truth datasets, especially when labeling the point-wise class manually. The selection strategy of training, validation and test datasets is also crucial for the results. Therefore, it needs to develop the fully automated, efficient and unsupervised classification method in future.

Conclusions
With the recent developments in sensing technology, MLS point clouds have attracted more attention, and have been increasingly used in various urban management and planning applications. The data processing, data mining and knowledge discovery from MLS data have become an active area of research. This paper conducted a comprehensive review of MLS applications and key techniques in urban areas. As a state-of-the-art technology of obtaining detailed 3D geographical surface information, MLS provides fine-scale and highly accurate point clouds for urban environment surveying and mapping. Therefore, we introduced the characteristics of MLS systems and data, and also investigated the emerging urban applications based on MLS data, such as transportation infrastructure mapping, vegetation mapping and inventory, building information modeling, utility surveying and mapping, and autonomous vehicle driving. To achieve these applications, several key issues should be addressed, including classification methods, object recognition, data registration, data fusion, and 3D city modeling. In large-scale, complex and changing urban areas, new MLS systems will emerge for the potential applications. Moreover, it is also necessary to develop rapid, automated and intelligent techniques, such as special processing framework and machine learning-based methods.