page created : August 11th, 2009       author : Claude Baumann

Robot name: R_3_XT  (former "Nic_3")

Abstract: This LEGO™ Mindstorms® robot R_3_XT uses a NXT and an RCX brick that communicate with each other through a 2-wire connection. The 3 degrees of freedom (DOFs) robot samples 1024 data-points of a continous sound wave on each channel of the stereo audio sensor. The sampling frequency is 36kHz. Then it performs a rapid cross-correlation (zero-crossing method) and determines the time-lag between both signals. Applying a novel algorithm that is based on interaural time difference (ITD) measurements and head movements (Kneip/Baumann 2008), the robot is able to localize a sound source in space with an accuracy of 15° both in azimuth and elevation. (Notice that the new robot is a further development of Nic_3 that has been presented on this site one year ago.)
Photo:

Previous work:
2D sound localizing based on interaural level differences
Sensor based on the precedence effect
3D precedence effect sensor
Detail of a robot using the previous sensor
binaural sensor
2D sound localizing based on interaural time differences (Nic_1)
Article: C. BAUMANN, L. KNEIP, Stereo robot ears, Elektor, July/August 2007, p. 13-17 (presents a digital ITD sensor)
L. KNEIP, C. BAUMANN, Binaural model for spatial sound localization based on interaural time delay cues ..., unpublished work that comprises the 3 DOFs LEGO™ RCX based robot Nic_2, Feb. 2007 (Notice: With 10° Nic_2 has better accuracy than Nic_3, but it is more difficult to manipulate.)
Article: L. KNEIP, C. BAUMANN, Binaural model for artificial spatial sound localization based on interaural time delays and movements of the interaural axis, The Journal of the Acoustical Society of America -- November 2008 -- Volume 124, Issue 5, pp. 3108-3119 (link)
Phase detection
Spatial sound localization
Bill of material:
LEGO™ NXT: (tasks: 2-wire connection to LEGO™ RCX; application of Kneip/Baumann algorithm; motor control)
LEGO™ RCX: (tasks: 2-channel ADC at 36kHz; cross-correlation through zero-crossing method; 2-wire connection to LEGO™ NXT; power to laser pointer)
3 LEGO™ NXT servo-motors
NXT touch-sensor (purpose: reactivate robot after one cue)
NXT light-sensor (purpose: the red light is switched on each time the robot is "listening")
2-wire serial connection between RCX and NXT
RCX compatible 2-channel audio amplifier (connected to RCX input ports 1,2 and 3, where port 1 is used to power amplifier)
RCX compatible laser pointer
About 100 standard LEGO™ pieces
Function summary:
    • A small music playing radio, representing an immobile audio source, is emitting sound waves that arrive at the microphones with different delays depending on its position in the robot head referential. 
    • The stereo amplifier boosts the signals in order to make them measurable by the RCX 10bit analog-to-digital converters (ADCs).
    • The RCX is programmed with ULTIMATE ROBOLAB®  to sample 1024 data values on each channel at a sampling frequency of 36kHz.
    • The RCX cross-correlates both signals and yields the interaural time difference (ITD). The program applies a zero-crossing method in the time domain (see: improvements).
    • The RCX sends the ITD data to the NXT using a 2-wire serial connection. (Baud-rate: 15b/s)
    • The robot first tries to horizontally center the head into the sound direction by rotating in direct function of the measured ITD.
    • Then it rotates about the z-axis for a given angle and mesures the ITD.
    • Next it rotates the head about the y-axis and repeats the ITD measurement. Applying the Kneip/Baumann 2008 method, which essentially is a mathematically exact determination of the spatial direction of sound, the robot calculates the azimuth and the elevation of the sound vector.
    • Finally the robot rotates the head about the x-axis, in order to have the laser point into the direction of sound. The laser pointer is powered from the RCX.

 

Video:

 

Software:
The RCX is programmed using the ULTIMATE ROBOLAB® (UR) environment. Each UR-program is a real firmware for the RCX. Therefore it is possible to graphically program the device at lowest level. This unlocks unexpected features of the LEGO™ brick, such as high speed ADC sampling.
The NXT is programmed using the LabVIEW® NXT toolkit.
Notes:
    • The current UR-program does not use Blauert's degree of coherence in order to prune bad cues. No error analysis has been applied either.
    • The mechanical errors are about 5° for the x- and z- rotations and 3° for the y-rotations.
    • Accuracy for ITD estimations: ~30ms.
    • The tests have been made in an ordinary reverberant room of 40m3 volume.
    • The distance to the sound source was 1.5m.
    • Actual NXT program : NXT switches off after 15 minutes. The power-down function is not disabled.
    • The motor control is more precise and smooth.
Literature: