Design of Sparse FIR Decision Feedback Equalizers in MIMO Systems Using Hybrid l1/l2 Norm Minimization and the OMP Algorithm

In this paper, a novel scheme using hybrid l1/l2 norm minimization and the orthogonal matching pursuit (OMP) algorithm is proposed to design the sparse finite impulse response (FIR) decision feedback equalizers (DFE) in multiple input multiple output (MIMO) systems. To reduce the number of nonzero taps for the FIR DFE while ensuring its design accuracy, the problem of designing a sparse FIR DFE is transformed into an l0 norm minimization problem, and then the proposed scheme is used to obtain the sparse solution. In the proposed scheme, a sequence of minimum weighted l2 norm problems is solved using the OMP algorithm. The nonzero taps positions can be corrected with the different weights in the diagonal weighting matrix which is computed through the hybrid l1/l2 norm minimization. The simulation results verify that the sparse FIR MIMO DFEs designed by the proposed scheme get a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse optimum DFE under the minimum mean square error (MMSE) criterion. In addition, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.


Introduction
In high-speed wireless communication systems, the multipath effect in channels will cause severe intersymbol interference (ISI). The orthogonal frequency division multiplexing (OFDM) system [1][2][3], with the presence of the orthogonal subcarriers and the cyclic prefix, has a strong ability to combat the ISI and generally requires simple equalization. However, compared with OFDM, the single-carrier communication has the advantage of low peak-to-average power ratio of the transmitted signal and insensitivity to carrier frequency offsets [4]. Therefore, the single-carrier communication is widely applied in many communication systems, such as in underwater acoustic communication [5][6][7] and vehicular communication systems [8]. One of the widely used methods to reduce the ISI in single-carrier systems is employing the finite impulse response (FIR) decision feedback equalizers (DFE) with a large number of taps to filter the received signals. However, the design complexity of FIR DFEs grows proportional to the square of the number of nonzero taps. Especially in multiple input multiple output (MIMO) systems, the design complexity can be much higher because the number of DFE taps grows proportional to the number of multiple antennas [9]. Therefore, to reduce the design complexity, it is necessary to design the FIR DFE with only a few nonzero taps (i.e., sparse FIR DFE). In [10], the number of nonzero taps is reduced by choosing only a few significant taps of the minimum mean square error (MMSE) equalizer which is not sparse in general. Nevertheless, the non-sparse MMSE solution must be calculated first which increases the design complexity. Therefore, how to determine the positions and values of nonzero taps directly is one of the main difficulties to design the sparse FIR DFE.
In recent years, some new ways of designing sparse FIR DFE have arisen with the research and application of sparse representation and compressed sensing theory [11][12][13][14][15][16]. Among them, the compressed sensing framework for sparse signal processing is proposed in [11]. A greedy search method to determine the nonzero taps positions of a single input single output (SISO) system is presented in [12]. The l 1 norm minimization method to design sparse filters is proposed in [13], but the modeling without wireless channel is quite different from the sparse equalizer design. In [14], a sparse adaptive equalizer design approach using the stochastic gradient pursuit algorithm based on compressed sensing is proposed, but the channel impulse response in the approach is assumed to be sparse. The new design frameworks for sparse FIR MIMO equalizers based on the orthogonal matching pursuit algorithm (OMP) and sparse approximation of a vector using different dictionaries are exploited in [15,16], respectively. They achieve better performance than the method in [10] with the same number of nonzero taps. Furthermore, for the pursuit of both sparse promotion and improved accuracy, the sparse signals reconstruction algorithms inspired by l 1 and weighted l 1 regularization schemes are proposed in [17,18]; the joint smoothed l 0 norm algorithm for direction-of-arrival estimation in MIMO radar is proposed in [19].
We aim to reduce the number of nonzero taps for the FIR DFE while ensuring its design accuracy by applying the OMP algorithm to hybrid l 1 /l 2 norm minimization. The OMP algorithm is a popular greedy algorithm that determines a nonzero tap position for each iteration [20]. However, there is no correction for the selected nonzero taps positions in the previous iterations, which will reduce the design accuracy for the sparse FIR DFE. The hybrid l 1 /l 2 norm minimization is proposed in [21] to make the solution for data fitting less sensitive to large residuals and more robust, and it has been applied in many fields such as geophysics, image processing and machine learning. Within this context, we propose a scheme using hybrid l 1 /l 2 norm minimization and OMP algorithm. In the proposed scheme, a minimum weighted l 2 norm problems is solved using the OMP algorithm in each iteration, and the nonzero taps positions can be corrected with the iteratively updated weights in the diagonal weighting matrix, then the weighting matrix is updated through the hybrid l 1 /l 2 norm minimization.
In this paper, we transform the problem of designing a sparse FIR MIMO DFE into an l 0 norm minimization problem, then use the proposed scheme to solve the l 0 norm minimization problem and obtain the sparse solution. It is verified through simulations that, when compared with the non-sparse optimum MMSE DFE, the sparse FIR MIMO DFEs designed by the proposed scheme obtain a significant reduction in the number of nonzero taps with a small performance loss. In addition, the proposed scheme provides better design accuracy than the OMP algorithm with the same nonzero taps percentage.

System Model
Consider a baseband equivalent discrete signal model in the MIMO systems with n i transmit (input) antennas and n o receive (output) antennas. Assuming that the channel is a linear time-invariant Rayleigh fading channel with an oversampling factor of l, the output of the MIMO channel at time k, i.e., the received samples from the n o receive antennas which are grouped into a ln o × 1 vector can be formed as: where y is the l × 1 output vector of the j-th receive antennas, H m (0 ≤ m ≤ ν) is the ln o × n i m-th MIMO channel matrix, x k−m is the input vector of the transmit antennas at time (k − m). The parameter ν is the maximum length of all the subchannel impulse response. n (j) k is an additive white Gaussian noise vector of the j-th receive antennas with mean 0 and variance σ 2 . Considering a block of N f received vector y k , (1) can be rewritten as [9] where Assuming that the transmit signals are independent with the average power ε x , the input autocorrelation matrix can be expressed as where I is the identity matrix, and the noise autocorrelation matrix can be expressed as Furthermore, the output-input crosscorrelation and the output autocorrelation matrices are as follows

Design of Sparse FIR DFE Using Hybrid l 1 /l 2 Norm Minimization and OMP Algorithm
In the FIR MIMO DFE model, in order to eliminate ISI and inter-antenna interference, then recover the input signal of the i-th transmit antenna, the feedforward filter w i with ln o N f taps processes the received signals y k , and the feedback filter b i with n i N b taps convolves with the previous decision signals, then the result combined both together is applied to the decision device to obtain current decision signal. If the previous decision signals before time k are correct, the DFE error sample at time k for the i-th input antenna is given by where k−∆ is the transmit signal at time (k − ∆) by the i-th input antenna, and ∆ is an integer representing the decision delay, usually 0 ≤ ∆ ≤ N f + ν − 1. Using N = ln o N f + n i N b to represent the number of DFE taps for simplicity, so w i is an N × 1 vector. Then the equalizer mean square error (MSE) for the i-th input antenna is expressed as where R y y E y y H , . Performing the Cholesky decomposition R y y = L L H , where L is an N × N lower triangular matrix, (9) can be rewritten as From (10) it shows that the different choice of w i can only change ξ excess,i , and ξ min,i remains unchanged. Obviously, ξ excess,i ≥ 0. Therefore, when ξ excess,i = 0, w i achieves the optimum solution under the MMSE criterion as w opt,i is usually non-sparse with too much complexity to compute directly. Therefore, in order to reduce the computational complexity, considering to reduce the number of the DFE nonzero taps at the cost of a small increase in the equalizer MSE, i.e., finding the sparse solution w s,i for w i when ξ excess,i ≤ θ (θ > 0), the sparse FIR MIMO DFE design problem can be transformed into the l 0 norm minimization problem as follows [15] w s,i = argmin In general, finding the optimal solution of the above l 0 norm minimization problem is NP-hard. One of the approaches to computing the suboptimal solution for the approximation of (12) is greedy algorithms. To design the sparse solution for the FIR MIMO DFE, in this paper, we propose the scheme based on the OMP algorithm which is a widely-used greedy algorithm.
The OMP algorithm selects an atom from the matrix L H (i.e., the column vector of the matrix L H ) that is most correlated with the current residual, then solves a constraint least squares problem to obtain all the selected atoms coefficients and updates the residual in each iteration. There is no correction for the selected nonzero taps positions in OMP algorithm, which will reduce the design accuracy for the sparse FIR DFE. To solve the problem, in this paper, we propose the scheme using hybrid l 1 /l 2 norm minimization and OMP algorithm. According to the hybrid l 1 /l 2 norm minimization, at the (k + 1)-th iteration, the proposed scheme will reweight the residual vector weighting matrix Q (k) , and minimize the weighted l 2 norm Then solve the above problem using the OMP algorithm to find a sparse solution. Q (k) is updated from the current residual through the hybrid l 1 /l 2 norm minimization. The diagonal entries of Q (k) are as follows q where r (k) n is the n-th entry of the current residual r (k) and µ is determined by the sparse solution. Substituting (14) into (13), minimizing (13) is equivalent to minimizing For an arbitrary value of r (k) n , the approximation of j(r which suggests J( w i ) behaves like an l 2 norm for small residuals and like an l 1 norm for large residuals. Therefore, the hybrid l 1 /l 2 norm minimization will make the sparse solution more robust and accurate by decreasing the sensitivity of the solution to large residuals. The procedures of the proposed scheme for finding the sparse solution of the FIR MIMO DFE are as follows: Initialization: Set the initial residual r (0) = L −1 r ∆,i , define the index set Ω = {1, 2, · · · , N}, and initialize the diagonal weighting matrix Q (0) = diag(1, 1, · · · , 1) for the first iteration.
Main iteration: Set k = 1 and perform the following steps: Step 1: Reweight the residual vector r = L H w i − L −1 r ∆,i and formulate the problem Set A (k) = Q (k−1) L H and normalize the column vectors {a of the matrix A (k) as follows where then (14) becomes Step 2: Solve (16) with the OMP algorithm. Set the equalizer nonzero taps initial index set I 0 = ∅, initialize the residual r 0 = β (k) . In the t-th (1 ≤ t ≤ k) iteration, the process is as follows: (1) Calculate the correlation between the residual r t−1 and the column vector of the matrix D (k) , and find the index λ t to the vector d (2) Update the nonzero taps index set I t = I t−1 ∪ {λ t }, and then update the set of column vectors Φ 2 , and update the residual (4) Check the value of t. If t = k, update the sparse solutionẑ (k) =ẑ (k) t and residual r (k) = (Q (k−1) ) −1 r t , then proceed to Step 3, else set t = t + 1 and go to (1).
Step 3: Check the stopping rule whether r (k) 2 2 ≤ θ. If met, get the sparse solution w s,i = Λ (k)ẑ(k) for the FIR DFE, and exit the algorithm; else update the diagonal weighting matrix Q (k) whose diagonal entries are determined by (14). From [22], set µ = max( ẑ (k) )/100. Then set k = k + 1 and go to Step 1.

Simulation Results
In the following, the simulation results are provided to verify the performance of the proposed scheme for sparse FIR MIMO DFE design in two different channels. As a comparison, the performance of the optimum MMSE DFE and the OMP method in [15] are presented respectively.
Two different multipath Rayleigh fading channels, namely ITU (International Telecommunication Union) Vehicular A channel [23] and an exponential decay channel, are simulated. ITU Vehicular A channel has a sparse power delay profile (PDP) with 6-paths spanning about 13 symbol periods. The exponential decay channel has a non-sparse PDP as ρ(n) = 10 −n/10 , 0 ≤ n ≤ 12 with 13-paths. The simulation system uses binary phase shift keying (BPSK) modulation. The decision delay is set to be ∆ = N f -1, which is optimum when N b = ν for DFEs [24]. The small increase θ in the equalizer MSE of the sparse DFE compared with the MMSE DFE is determined by the maximum tolerable performance loss η max as θ = (10 η max /10 − 1)ξ min , where ξ min is the MSE for the MMSE DFE. We define the signal-to-noise (SNR) as SNR 10lg(ε x /σ 2 ), set the oversampling factor l = 2 and MIMO DFEs with n i = 2 and n o = 2. Figure 1 plots the nonzero taps percentage for the sparse SISO DFEs and MIMO DFEs versus the performance loss η max using the proposed scheme respectively. It is shown that the number of nonzero taps for DFE is reduced with the increase of η max . When η max = 0.4 dB, the nonzero taps for both the sparse SISO DFEs and MIMO DFEs are reduced by more than 70% and 60% in the two different channels respectively. Therefore, comparing the ITU Vehicular A channel with sparse PDP, the FIR DFEs needs more nonzero taps to eliminate the ISI caused by the multipath effect in the exponential decay channel with more paths. Furthermore, in MIMO systems, the DFEs needs more nonzero taps to eliminate both ISI and inter-antenna interference, as shown in Figure 1, MIMO DFEs has a larger percentage of nonzero taps when compared with SISO DFEs.
Step 3: Check the stopping rule whether . If met, get the sparse solution for the FIR DFE, and exit the algorithm; else update the diagonal weighting matrix ( ) k Q whose diagonal entries are determined by (14). From [22], set   ( ) max 100 k μ  z . Then set k = k + 1 and go to Step 1.

Simulation Results
In the following, the simulation results are provided to verify the performance of the proposed scheme for sparse FIR MIMO DFE design in two different channels. As a comparison, the performance of the optimum MMSE DFE and the OMP method in [15] are presented respectively.
Two different multipath Rayleigh fading channels, namely ITU (International Telecommunication Union) Vehicular A channel [23] and an exponential decay channel, are simulated. ITU Vehicular A channel has a sparse power delay profile (PDP) with 6-paths spanning about 13 symbol periods. The exponential decay channel has a non-sparse PDP as  both the sparse SISO DFEs and MIMO DFEs are reduced by more than 70% and 60% in the two different channels respectively. Therefore, comparing the ITU Vehicular A channel with sparse PDP, the FIR DFEs needs more nonzero taps to eliminate the ISI caused by the multipath effect in the exponential decay channel with more paths. Furthermore, in MIMO systems, the DFEs needs more nonzero taps to eliminate both ISI and inter-antenna interference, as shown in Figure 1, MIMO DFEs has a larger percentage of nonzero taps when compared with SISO DFEs.   Figure 2 gives a single example of the SISO DFE by the three design methods in ITU Vehicular A channel. Note that only the nonzero taps are marked, and the tap indices below 25 are insignificant to show. It is shown that there are many taps whose values are close to zero in the non-sparse MMSE DFE. Compared with the MMSE DFE, most of the taps for sparse FIR DFEs designed by the OMP method and the proposed scheme are zero, and the tap positions selected by the two methods are not identical.
To verify the performance of the proposed scheme, we compare the bit error rate (BER) and the normalized mean square error (NMSE) for the three design methods. Figures 3 and 4 compare the BER performance and the equalizer NMSE respectively in ITU Vehicular A channel. The sparse DFEs designed by the proposed scheme and the OMP algorithm are constrained to have the same sparsity level in the simulation. The simulation results show that the maximum SNR loss for the SISO DFEs and MIMO DFEs designed by the proposed scheme are only 0.4 dB and 0.6 dB when the number of nonzero taps is reduced by more than 83% and 80%, respectively. That is, the BER performance and the NMSE for the sparse DFEs designed by the proposed scheme are similar to the MMSE DFE performance. As shown in Figure 3, the BER performance for the sparse DFEs designed by the proposed scheme outperform the OMP algorithm with the same sparsity level. Besides, by comparing the NMSE of the two methods, it is known that the design accuracy of the proposed scheme is superior to the OMP method. Considering that the MIMO system has a certain diversity gain, the simulation results in Figure 3 show that the BER performance of the MIMO DFEs have a significant improvement than the SISO DFEs.  To verify the performance of the proposed scheme, we compare the bit error rate (BER) and the normalized mean square error (NMSE) for the three design methods. Figures 3 and 4 compare the BER performance and the equalizer NMSE respectively in ITU Vehicular A channel. The sparse DFEs designed by the proposed scheme and the OMP algorithm are constrained to have the same sparsity level in the simulation. The simulation results show that the maximum SNR loss for the SISO DFEs and MIMO DFEs designed by the proposed scheme are only 0.4 dB and 0.6 dB when the number of nonzero taps is reduced by more than 83% and 80%, respectively. That is, the BER performance and the NMSE for the sparse DFEs designed by the proposed scheme are similar to the MMSE DFE performance. As shown in Figure 3, the BER performance for the sparse DFEs designed by the proposed scheme outperform the OMP algorithm with the same sparsity level. Besides, by comparing the NMSE of the two methods, it is known that the design accuracy of the proposed scheme is superior to the OMP method. Considering that the MIMO system has a certain diversity gain, the simulation results in Figure 3 show that the BER performance of the MIMO DFEs have a significant improvement than the SISO DFEs.    To verify the performance of the proposed scheme in different channels, Figures 5 and 6 show the simulation results in the exponential decay channel. Comparing with the OMP method, the results also show that the proposed scheme has better BER performance and smaller NMSE with the same sparsity level. Therefore, the proposed scheme provides excellent performance in different channel conditions. To verify the performance of the proposed scheme in different channels, Figures 5 and 6 show the simulation results in the exponential decay channel. Comparing with the OMP method, the results also show that the proposed scheme has better BER performance and smaller NMSE with the same sparsity level. Therefore, the proposed scheme provides excellent performance in different channel conditions.

Conclusions
In this paper, a novel scheme using hybrid l1/l2 norm minimization and OMP algorithm has been proposed to design the sparse FIR MIMO DFE. The proposed scheme solves a of minimum weighted l2 norm problem using OMP algorithm in each iteration, then the weights weighting matrix is updated through the hybrid l1/l2 norm minimization. The simulation results demonstrate that the sparse FIR MIMO DFEs designed by the proposed scheme achieve a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse MMSE DFE. Meanwhile, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.

Conclusions
In this paper, a novel scheme using hybrid l 1 /l 2 norm minimization and OMP algorithm has been proposed to design the sparse FIR MIMO DFE. The proposed scheme solves a of minimum weighted l 2 norm problem using OMP algorithm in each iteration, then the weights weighting matrix is updated through the hybrid l 1 /l 2 norm minimization. The simulation results demonstrate that the sparse FIR MIMO DFEs designed by the proposed scheme achieve a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse MMSE DFE. Meanwhile, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.

Conflicts of Interest:
The authors declare no conflict of interest.