Adaptive Direction of Arrival Estimation Using Sparsity Constrained Complex NLMS Algorithm With Variable Penalty Factor

Direction of arrival (DOA) estimation using sensor arrays is a challenging task in noisy environments, especially when rapid convergence and efficient utilization of array elements are required. Addressing these challenges, we propose an adaptive filtering framework for DOA estimation that effectively tackles convergence speed, sparsity promotion, and array thinning. Typically, the complex Least Mean Square (LMS) algorithm is employed for error minimization of the adaptive process. In this work, we propose a sparsity constrained complex normalized LMS (NLMS) method for DOA estimation, which introduces faster convergence compared to the conventional complex LMS and NLMS adaptive methods. The sparsity constrained formulation of the proposed method is converted into an unconstrained problem by using a variable penalty factor (VPF). The update rules of the DOA filter weight and the VPF of the proposed complex VPF NLMS algorithm are derived. Further, we introduce a complex convex VPF NLMS algorithm that uses a convex combination of two complex VPF NLMS filters to further enhance the DOA estimation performance. We also present a theoretical derivation of the condition for mean square convergence of the proposed complex VPF NLMS adaptive algorithm. The performances of the proposed algorithms have been assessed through different simulations that highlight their effectiveness. Since the proposed algorithm promotes sparsity in the DOA weight vector, we also examine its performance in scenarios of array thinning, where not all sensors are utilized for DOA estimation.