Abstract:This paper proposes a Modified MUSIC (MMUSIC) algorithm to address the performance degradation of the conventional Multiple Signal Classification (MUSIC) algorithm in direction-of-arrival (DOA) estimation with acoustic vector sensor (AVS) arrays. Conventional MUSIC relies heavily on the strict orthogonality between the signal and noise subspaces. However, in AVS arrays, the statistical inconsistency between pressure and particle velocity channel noise leads to an expansion of the signal subspace and contamination of the noise subspace, thereby violating the expected orthogonality. This results in spurious spectral peaks and even degraded resolution. To overcome this issue, a detailed analysis of the AVS reception model is conducted, which reveals the mechanism by which noise power imbalance between pressure and velocity channels introduces “virtual sources.” Based on this analysis, the proposed algorithm reconstructs the criterion for noise subspace partitioning and formulates a new spectrum function by exploiting the orthogonality between eigenvectors and the steering matrix, enabling accurate identification and elimination of virtual source effects. Simulation results demonstrate that the proposed MMUSIC algorithm can effectively resolve coherent signals under challenging conditions such as low signal-to-noise ratio (SNR) and small angular separations, thereby significantly enhancing the resolution and robustness of AVS-based DOA estimation.