Audio-visual based non-line-of-sight sound source localization: A feasibility study

This paper examines the feasibility of using an audio-visual methodology for sound source localization of acoustic sources hidden from direct view. A four channel microphone array is used in conjunction with LiDAR and 2D/3D mapping to merge estimated angles of arrival with room parameters for sound source localization. The time difference of arrival (TDOA) approach is used to estimate the angle(s) of arrival from an unknown sound source and a reverse ray-tracing approach is used to identify a possible source location behind a barrier/obstacle. Using a single-source zone approach based TDOA algorithm, a variety of estimated source directions are identified, each arising from multiple sound sources due to reflections in the room. Results are combined with a three-dimensional map of the space acquired from a ground robot equipped with a 2D SICK LMS LiDAR, and a reverse ray tracing approach is used to triangulate the likely position of the source. Tests were performed in both controlled and uncontrolled environments and the method was capable of finding the hidden source to within 0.5 m accuracy, which is approximately 10% of the length of an automobile. It is proposed that a methodology like this could be used to add sound source localization capabilities on autonomous vehicles to detect the position of emergency/warning sounds in traffic that may be shielded from the direct field of view.