Complexity Reduction of MLSE and MAP Equalizers Using Modiﬁed Prolate Basis Expansion

: Maximum likelihood sequence estimation (MLSE) and maximum a posteriori probability (MAP) equalizers are optimum receivers for dealing with intersymbol interference (ISI) in time-dispersive channels. However, their high complexity and latency limit their widespread implementation; therefore, research into reducing their complexity is an open topic. This paper proposes a novel modiﬁcation to reduce the computational complexity of the aforementioned algorithms, which exploits the representation of the communication channels in a time-delay-domain basis expansion model (BEM). It is shown that an appropriate basis is a set of modiﬁed prolate functions, in which the transmitter and receiver ﬁlters are considered in the kernel construction. Simulation results show that a reduction in sums and multiplications on the order of 55% can be obtained, maintaining the same bit error rate performance as in the traditional implementation.


Introduction
Reflection, refraction, and diffraction are propagation phenomena that cause many delayed replicas of the transmitted signal to reach the receiver. If the maximum delay of these replicas τ max is close to or greater than the symbol period T, then each received symbol possesses energy from its neighbors. This impairment is known as intersymbol interference (ISI), and it greatly degrades system performance. A common approach to counteracting ISI is the use of orthogonal frequency division modulation (OFDM), which permits low-complexity frequency-domain equalization in low-mobility environments. However, OFDM has two major drawbacks: high sensitivity to Doppler spread, dependent on the system's mobility, and a high peak-to-average power ratio (PAPR), proportional to the length of the Fourier transform. In high carrier frequency bands (e.g., 60-70 GHz), even a small movement in the transceivers can cause a high Doppler spread, provoking intercarrier interference and long transmission latencies [1]. To avoid excessive PAPR while complexity equalization remains low in the frequency domain, the discrete-Fourier-transform (DFT)-Spread OFDM has been employed. Unfortunately, as noted in [2,3], frequency-domain equalization is not a computationally advantageous strategy when the channel is rapidly time-varying. Hence, single carrier transmission remains as an alternative considered in standards such as IEEE 802.11ad wireless local area networks (WLAN) [4], IEEE 802.15.3 high data rate wireless multi-media networks [5], IEEE 802. 16 air interface for broadband wireless access systems [6], and the proposed standard IEEE 802.11ay [7].
Focusing on single carrier transmission, a common approach for dealing with ISI is a channel equalizer, and they can be classified according to linearity. Linear equalizers, such as the minimum mean square error (MMSE) and zero-forcing (ZF), present low computational complexity, but they perform poorly when the channel presents deep notches in frequency. In contrast, non-linear equalizers, such as the decision feedback equalizer (DFE), the maximum likelihood sequence estimation (MLSE), and the maximum a posteriori probability (MAP), are computationally intensive algorithms; however, they significantly outperform the linear equalizers [8]. The MLSE equalizer finds the sequence of data symbols with the highest probability of being transmitted given the received signal, usually using the Viterbi algorithm. The MAP equalizer, meanwhile, is a symbol-by-symbol estimator, usually based on the Bahl, Cocke, Jelinek, and Raviv algorithm (BCJR) [9]. Although MLSE and MAP are the most effective equalizers, their implementation becomes infeasible due to excessive computational complexity when the channel impulse response (CIR) lasts several symbol periods or the data symbol alphabet is large. Recent works that use variations of these equalization techniques can be found in [10][11][12]. In [10] the authors compared the performance of detecting quadrature phase shift keying (QPSK) in a super-Nyquist wavelength division multiplexing system as a QPSK and quadrature duobinary (QDB) signal using MLSE and MAP equalizers with different detection schemes, and they concluded that QDB had the better performance. The authors in [11] reduced the complexity of faster-than-Nyquist (FTN) system receivers designing an MMSE-based channel shortening before the MLSE detection. Exact closed-form asymptotic expressions for symbol and bit error probability for arbitrary constellations and input distributions are presented in [12].
In order to reduce the computational complexity, sub-optimal algorithms [8] have been used, such as reduced state sequence estimation (RSSE), delayed decision feedback sequence estimation (DDFSE), the M-algorithm, Max-Log-MAP, Log-MAP, and other recent ideas including the reduced-state soft-output Viterbi equalizer [13] and the estimation of channel sparsity to assist in the detection of the transmitted sequence [14]. In [15], the authors presented an MLSE and a MAP equalizer using the discrete-Fourier basis expansion model (BEM) of the channel. Despite that reduction of the effective channel impulse response decreases the number of states in the equalizer, the accuracy of estimation is closely affected. Hence, these solutions create an error floor because of the modeling error.
In this tenor, this paper presents a modification of MLSE and MAP algorithms using a basis expansion model, which allows translating part of the computational complexity into memory storage. In this way, the computational complexity is reduced without error floor, notably when the channel is varying during the transmission of a block of N symbols using read-only memories.
There are two main contributions of this paper. Firstly, the modified versions of the prolate functions (MPFs) have never been used to reduce the computational complexity of MLSE and MAP algorithms, to the best of the authors' knowledge. Thus, for a given approximation error, it can be observed that the number of basis functions is fewer when MPFs are used compared to the suggested basis in the state of the art implementations. Secondly, contrary to intuition, the use of MPFs in the time-delay domain gather better performance than the Fourier base using the same number of functions.
The paper is organized as follows: The mathematical model of the communication system and the representation of the channel using a BEM are presented in Section 2.1. The MAP and MLSE algorithms are outlined in Section 2.2. The modification of the MAP and MLSE algorithms using BEM and their computational complexity are established in Section 2.3 and Section 2.4, respectively. In Section 3, the corroboration of the proposed methods through simulation is presented. Finally, the conclusions are presented in Section 4.

Materials and Methods
This work is intended to highlight the use of the modified version of the Prolate basis expansion [16][17][18], in MLSE and MAP equalization processes [13,15,19,20]. The application of MPF in the time-delay domain conveys a more accurate channel representation, which leads to a reduction in the error floor. Additionally, the computation complexity is lower, compared with the MLSE and MAP algorithms found in the literature [15,19,20].
The descriptive statistics of the signals related to the system model are presented in Section 2.1. Furthermore, the equalizer assumes to know the channel impulse response, which is decomposed in K basis functions to compute their relative weights. Employing a limited number of basis functions affects the variance of the modeling error. Accordingly, Monte-Carlo simulations were used in Section 3 to compare the benefits of this method, assuming perfect channel state information (CSI) and estimated channel impulse response. This paper does not contain studies involving human participants, data, or tissue, or animals, performed by any of the authors.

System Model
Consider the discrete-time baseband-equivalent model of a digital single carrier (SC) communication system, as shown in Figure 1.
with zero mean and variance of σ 2 b = E(|b l | 2 ), whose elements come from an equally probable finite alphabet X (b l ∈ X ), and E(·) denotes the expected value. Perfect synchronization between the transmitter and receiver with T as the symbol period is assumed. The symbols are propagated through the communication channel h l , which is the discrete representation of the continuous impulse response of the baseband channel model h c (t) in a given moment, where t represents the time variable and l is its equivalent discrete index. The aforementioned continuous model is composed of the convolution of f (t), v(t), and w(t). The propagation medium f (t), hereinafter assumed to be wide-sense stationary uncorrelated scattering (WSSUS) with a maximum time span of τ max seconds. v(t) and w(t) are the causal impulse response of the transmitter and receiver filters with a span of τ v and τ w , respectively, a bandwidth W and a roll-off factor β. Considering h c (t) to be almost invariant during the transmission of data symbols with period T and time duration of τ v + τ w + τ max , it can be written as follows: where 3 , the convolution of the impulse responses of the transmitter and receiver filters and h c (t, τ) results in a wide-sense stationary correlated scattering.
During the transmission of the sequence b, the CIR h l could be visualized as a variant finite impulse response (FIR) filter, with P coefficients; i.e., h l = [h l,0 , h l,1 , . . . , h l,P−1 ] T , where P = τ v +τ w +τ max T and · denotes the ceiling operation. The propagated signal is contaminated with a zero mean additive white Gaussian noise (AWGN) n l with a variance of σ 2 n . Thus, the received signal is expressed as y l =ỹ l + n l , wherẽ is the noiseless output of the channel.
If the channel is approximated using BEM, then the number of required coefficients for representing the channel could be less than P and consequently easier to implement. However, this reduction in complexity is firmly attached to the selection of the base, the optimal base being the Karnuhen-Loève expansion (KLE). Nevertheless, to compute the KLE, it is necessary to know the autocorrelation of the channel, which is difficult, as it changes from one scenario to another. For overcoming these impairments, it is proposed in [17,18] to take the impulse response of the transmitter and the receiver filters into account together with the maximum path delay for obtaining the MPFs. The channel BEM can be represented as where K < P is the number of required functions, and e l,p is the modeling error in the p-th tap index, in the l-th time index. The number of MPF basis functions which grant a negligible variance of the modeling error is close to the KLE case. The orthogonal sequences h (k) are obtained by solving the equation where λ k is the eigenvalue associated with the eigenvector h (k) . C h is the kernel matrix of the basis, constructed as described in [17,18]: and τ 0 = 0 is some starting value. If it is assumed r v w (τ, τ 1 , is the rectangular function, then: With these definitions, it can be shown that: The use of MPF offers the advantage of expanding the filtered stochastic process [17,18] with fewer basis functions. Thus, instead of requiring K P = (τ v + τ w + τ max )(W + β) + 1 prolate functions, it is necessary only to consider K = ( (W + β)τ max + 1) < P MPFs.

MLSE and MAP Equalizers
In this section, the traditional maximum a posteriori probability (MAP) and the maximum likelihood sequence estimator (MLSE) are introduced.

MAP Equalization
The MAP equalizer obtains an estimate of the transmitted symbol b l , by maximizing the a posteriori probability given the received signal sequence y = [y 0 , y 1 , . . . , y L−1 ] T [20][21][22]. The latter is expressed as follows:b where b is chosen from the finite input alphabet X with cardinality |X | = M. The MAP equalizer can be implemented using the Bahl, Cocke, Jelinek, and Raviv algorithm (BCJR) [20] as: where s l ∈ S is the channel state in the instant l, with S = {a m |m = 0, . . . , M P−1 − 1} being the set of all possible combinations of the previous P − 1 symbols a m = [b l−1 , . . . , b l−P+1 ], γ(s l → s l+1 ) is the error probability in the transition from the state s l to s l+1 , and α(s l ) and β(s l ) are the forward and backward coefficients in the state s l , respectively, which are obtained through the following recursions: with α(s −1 = a m ) = 1 and β(s L = a m ) = 1, ∀a m ∈ S. Note that γ(s l → s l+1 ) depends on the sequence b l = [b l , s l ], composed of the l-th transmitted symbol and the previous P − 1; thus, γ(s l → s l+1 ) = Pr(s l+1 |s l )Pr(y l |b l ).
Assuming an AWGN with a zero mean value and a variance of σ 2 n , then To reduce the complexity of the BCJR, two algorithms were derived in the log-domain: the Max-Log-MAP [21] and the Log-MAP [22].

MLSE Equalization
The goal of an MLSE equalizer is to determine the sequence that will maximize the likelihood functionb which is equivalent tob where is the path metric of sequence b.
The most common and efficient way of obtaining V(b) is utilizing of the Viterbi algorithm (VA) [19], where the metric is computed a branch by a by branch in the trellis representation of the received symbols.

Proposed MLSE and MAP Equalizers
From the description presented of the MLSE and MAP algorithms, it can be seen that both require calculating the parameterỹ l |b l in order to obtain the estimated data. Such a parameter corresponds to the noiseless received signal when a specific b l is transmitted through the channel h. Note thatỹ l |b l is highly dependent on the current communication channel. Moreover, the iterative computation of the objective functions Equations (13) and (16) leads to a highly demanding computational problem, which increases with the cardinality M of the transmitted alphabet and the length of the CIR. In order to deal with the latter, the use of BEM is proposed for representing h employing MPFs. Thus, using Equations (2) and (3),ỹ|b l is approximated bỹ where g (k) p b l−p is the response of the k-th sub-channel h (k) when the sequence b l is transmitted.
Equation (17) shows thatỹ l |b l is the weighted sum of the data signal b l filtered by each function of the basis {h (k) }. It is important to note that such a basis can expand any channel realization that satisfies the conditions of maximum spread delay and the particular shape of the filters, while the weights w l,k are the dependent values of each CIR. Therefore, it is possible to precompute and store g (k) l |b l for each function and each combination of data symbols. The desiredỹ l can be obtained by weighting g (k) l |b l with the w l,k that corresponds to the current channel realization. For that reason, all the M P possible outputs must be saved in a read-only memory for each of the K functions in the basis, resulting in KM P memory locations.
The equalizers based on orthogonal functions, hereafter referred to as orthogonal MLSE and orthogonal MAP, can be implemented using a methodology split into two stages. The first stage consists of computing a priori information, which will be calculated only once, throughout the following steps: • Obtain the kernel of the required expansion C h according to [17,18]. • Solve Equation (4) to find the eigenfunctions h (k) and eigenvalues λ k associated with the kernel.

•
Sort the eigenvalues in descending order.

•
Select the K eigenfunctions corresponding to the eigenvalues with higher energy while maintaining a determined modeling error energy, Equation (7).
l |b l for all possible transmitted sequences and store the results in memory. The second stage is the runtime implementation of the algorithms, which is carried out as follows: • Obtain the weights w l,k .
• Use the weights for calculatingỹ l |b l as needed in Equation (13) for the MAP case or Equation (16) for the MLSE case.

Complexity Analysis
The number of multiplications and sums in each state of the trellis for different estimators is summarized in Table 1, requiring O LKM P multiplications. The capacity of MPFs for modeling a filtered stochastic process with the minimum number of basis functions after KLE [17,18] makes this proposal the orthogonal equalizer with optimal performance and reduced complexity. This is the main achievement of this work.  Table 2 compares the number of multiply-add complex operations, the number of memory locations, when required, and their update period in the described channel model for the classical and orthogonal equalizers mentioned above. The number of basis functions employed in orthogonal Fourier ( [15]) and MPF equalizers are the values K 1 and K 2 , respectively. From Table 1, given that each node has M P−1 states inside the trellis, it is possible to observe that the number of multiply-add operations required to compute {ỹ l |b l } L−1 l=0 in Equation (13) and Equation (16) is M P−1 K 2 M, L times. In [15], the number of multiply-add operations is LK 1 M P . Following different strategies, the equation to obtain the memory locations used by the MPF equalizers, K 2 M P , is similar to K 1 M P in [15]. The same happens with the number of multiplications to fill the memories that store the values g (k) l |b l in Equation (17), but the memories in the proposed equalizers do not need updates. Another marked difference respects the work in [15] is that MPFs do not have the observation period of L 1 ≥ L to compute the channel coefficients. Every time K 2 < P, the proposed methods lead to a simplified hardware implementation, which can result in less power consumption and lower latency. The trade-off in the hardware implementation is reflected in more memory locations, due to the need of storing all the M P possible noiseless channel outputs g (k) l |b l .

Results and Discussion
In this section, the performance evaluation of the proposed equalizer for an SC communication system is presented. Transmission of data blocks composed of 800 symbols at a symbol period of T = 10 −7 s was assumed. These are considered QPSK and 16 quadrature amplitude modulation (QAM) modulations. The pulse shaping and matched filter were square-root raised cosine with a roll-off factor of 0.5 and a length of six symbol periods.
The orthogonal equalizers were simulated for a communication system with a power delay profile defined by the following normalized path's power: [0, −1.5, −1.4, −3.6, −0.6, −9.1, −7, −12, −16.9] dB, and corresponding path delay: [0, 3,15,31,37,71,109,173,251] ns, each path with Jakes Doppler profile and with a maximum normalized Doppler spread f max T = 0.001. The latter corresponds to a CIR of size P = 9. Given the former system values, the number of MPFs needed to expand the channels results in K = 5. In Figures 2 and 3, there are depicted the bit error rate (BER) performance of the equalizers versus the signal to noise ratio (SNR) under the assumption of perfect knowledge of the CSI and over the pilot-assisted channel estimation as described in [23]. The simulation results confirm that the performance of the orthogonal equalizers is quite similar to that of the MLSE. Moreover, the prediction in the chosen number of eigenfunctions K is valid: The performance is not affected when the number of computations is reduced.
The performance of the equalizer proposed in [15], based on the Fourier basis, is also presented in Figures 2 and 3. In that work, the channel BEM is developed in the time domain, in contrast to this work, which considers a BEM in the time-delay domain. The BEM in time, as developed in [15], causes the necessity of computing the K 1 BEM coefficients and updating the memory locations every L samples. Furthermore, it does not provide an expression to define L and L 1 . For the simulations in this work, we chose the parameters K 1 = 5, L = 800 and L 1 = 1600, used in [15] to expand the channel for the given normalized Doppler spread. Due to the lack of accuracy in the representation through the Fourier basis, the simulation results for [15] in Figures 2 and 3 exhibit the presence of an error floor, which is worst when the modulation order is increased. A better resolution in Fourier basis can reduce the modeling error and leads to the absence of error floor, but this is achieved either by extending the observation window and introducing lags in the estimated data and extra memory or by an iterative process of joint channel estimation and equalization.  Table 3 shows the number of multiply-add complex operations, memory locations, and their update period in the simulated scenario for the classical and orthogonal equalizers for QPSK and 16-QAM. It is possible to observe that Fourier and MPF algorithms have the same computational complexity, achieving both a reduction in the number of multiplications of 55.6% concerning the classical MLSE. The latter because we are considering the same number of functions (K 1 = K 2 ). However, as it was shown, MPFs outperforms the Fourier implementation when the same number of functions is used (Figures 2 and 3). Thus, it is possible to say that for an equal computational effort, our proposal presents a better performance or conversely, for achieving a predefined BER, our proposal requires fewer operations and memory locations than Fourier proposals. These reductions lead to a simplified hardware implementation, which can result in less power consumption and lower latency. The trade-off in the hardware implementation is reflected in extra memory locations, due to the need for storing all the M P possible noiseless channel outputs g  Performance of the equalizers [15] and the proposed 5-MPFs with 16-QAM modulation scheme. Table 3. Complexity and memory-requirement comparison to obtain {ỹ l |b l } L−1 l=0 with L = 800, L 1 = 1600, P = 9, K 1 = K 2 = 5.

Conclusions
A near-optimum orthogonal MLSE and orthogonal MAP equalizers based on BEM in the time-delay domain and using the modified version of prolate functions were first time introduced in this work. They show the same performance as the classical algorithms, but with lower computational effort. Whenever the length of the channel impulse response is greater than the number of basis functions needed in the basis expansion model, the sums and multiplications of the MLSE and MAP algorithms, as well as the execution time, are substantially reduced. The use of the MPFs as the basis expansion model offers the benefit of being robust to the particular channel realization. For an equal computational effort, our proposal presents better performance than state of the art alternatives, or conversely, for achieving a predefined BER, our proposal requires fewer operations and memory locations.