Steve Nguyen

Whiskers – or vibrissae – are very important for rats. Indeed, usually living in narrow and dark environments they highly rely on this very rich tactile sensor. Using them, rats can recognize shapes, discriminate fine textures, estimate gap length or aperture size. They are often compared to human fingertips in terms of haptic abilities. This sensory modality is also largely studied in biology as the pathway from individual vibrissa to the somatosensory cortex (the "Barrel cortex") is known quite precisely.

However, quite few work has been done in robotics to try to exploit such a rich sensor. So I developed a new tactile sensor based on conductive elastomer reproducing the stereotyped organization of rat's whisker pad.

Rat's whiskers and their artificial counterparts

I also developed a bio-inspired model for texture feature extraction. This model allowed for texture recognition with very high performance. With this system I was able to explore the effect of whiskers organization (in particular the size gradient) on texture recognition. The model I proposed may help to reconcile two opposing theories in biology about the coding of textures, the frequency coding (resonance hypothesis) and the temporal coding (kinetic signature hypothesis).

See my papers on this subject: N'Guyen et al. 2009, N'Guyen et al. 2010, N'Guyen et al. 2011

The auditory system was developed in collaboration with Mathieu Bernard.

We proposed a bio-inspired model that allowed to relate audition and tactile. This system comprises a classical cochlear model (Lyon's model) in order to extract multi-channel "spikes" from sound. Then, inter-aural time difference (ITD) and inter-aural level difference (ILD) were implemented for each of these channels. Using this model and by taking advantage of the mobile pinnea of the robot, we were able efficiently separate and locate sound sources.

Close view of the artificial ear

Moreover, based on the idea that tactile and auditory information are in fact quite homogeneous (at least for "texture") we were able to use the same texture recognition model for audition and tactile. This idea may in fact have some biological relevance as cross-sensory interference between tactile and audition have been observed in humans suggesting a common cortical area responsible for both.

See our papers: Bernard et al. 2010b, Bernard et al. 2010a

The vision system was based on a real-time hardware processing chip developed by the company BVS (which funded my PhD). This module mimics the human visual system with several filtering layers (color, edges, movements…) along with tracking units. A low-level visual layer was developed using this chip, allowing for the detection, tracking and "features" extraction of objects in the environment.

Psikharpax in front of a screen during a saccade experiment

Then I developed a saccadic system able to select and learn visual targets. This system integrated a Superior Colliculus model (responsible of the visual to motor transformation) along with a Basal Ganglia model (responsible for selection and reinforcement learning).

See our papers: N'guyen et al. 2010a, N'Guyen et al. 2013a (about a more complete model of saccadic circuitry, coming soon)

Finally all these sensory modalities were integrated by the projections to the Superior Colliculus. Indeed this structure is known to be a primary center for multi-sensory integration. It contains not only visual information but also retino-centric maps for other modalities including somatosensory (tactile), audition, smell… For each modality, perceived objects features were selected and learned by parallel Basal Ganglia loops. The resulting multi-sensory model was then able to trigger orienting behavior depending on conjunction of rewarded multi-sensory cues.

Psikharpax in a visuo-auditive orienting task

Psikharpax in a tactile-visuo-auditive discrimination task

Unfortunately I still didn't have time to write a paper about the multi-sensory integration, but several papers were done about using the whole system for navigation: Caluwaerts et al. 2011, Caluwaerts et al. 2012b, Caluwaerts et al. 2012a

A.k.a. the artificial rat, my PhD thesis project (see below).

Psikharpax v1

At a the VES parisian exposition

Live tactile texture recognition at the VES in front of iCub

Psikharpax v2 with lift up and grasping capability (design by Mr.QQ)

The well known Nao from Aldebaran Robotics but with a bit of customization (3 DoF head).

Nao during a "push" experiment

Nao during "slide" experiment

Developed by the Italian Biorobotics Institute (SSSA), based on the WABIAN robot from Waseda University. I had opportunity to make some experiments on this robot in collaboration with SSSA in the RoboSoM project.

SABIAN

Poppy is a new custom small sized and low cost humanoid robot developed by Matthieu Lapeyre from the INRIA Flowers team. It is still in development and we are implementing probabilistic control on it. I have modified it from the original design in order to add another degree of freedom, different feet and various sensors.

Jack is another small low cost humanoid based on Robotis Dynamixel motors. It is inspired by the Acroban robot from INRIA and is intended for experimentations on compliance.

Poppy

My Poppy after some modifications

Jack

Jack again

SigmaBan is the Rhoban's team small humanoid robot used for the Robocup.

SigmaBan

GrosBan is the "experimental" (and usually bigger) version of SigmaBan.

GrosBan

This is a small side project I made in order to test some unusual spherical holonomous wheels. The robot moves very smoothly, and of course I added a tilt controlled camera and a hacked airsoft gun for a better remote control experience…

Hologun with its turret

Hologun