Neuromorphic Semantic Segmentation

About the Project

Image segmentation is a popular task in the field of computer vision in which specific regions of an image must be labelled according to their semantic contents. The objective of NeSS project is to design a real-time, low-energy, bio-inspired semantic scene segmentation system using a neuromorphic approach.

More specifically, the project combines a new kind of vision sensor, event cameras (or silicon retinas), with a brain-inspired machine learning paradigm, spiking neural networks. These two complementary neuromorphic models are both based on binary discrete events, sparse and asynchronous, allowing for low-latency and low-energy implementations, thus making them suitable for real-time embedded applications such as self-driving vehicles or robots.

This interdisciplinary project, that involves researchers from computer vision, computational neurosciences, hardware, and circuits, targets frugal systems to tackle some important challenges in embedded computer vision.

Principal Investigator
Project's partner(s)
  • Dalia Hareb, i3S
  • Benoît Miramond, LEAT
  • Amélie Gruel, IMS Bordeaux
  • Laurent Perrinet, INT Marseille
  • Teresa Serrano Gotarredona, IMSE Espagne
Duration
  • November 2022 – March 2025
Total Amount
  • 30 000 euros
Publications
  • Dalia Hareb, Jean Martinet, Benoît Miramond. Enhanced neuromorphic semantic segmentation latency through stream event. International Conference on Neural Information Processing (ICONIP). Poster. Auckland, New Zealand, December 2024
  • Huyen Trang Nguyen, Laurent Sparrow, Jean Martinet. Top-down and bottom-up attentions in event data: a survey. International Conference on Neural Information Processing (ICONIP). Poster. Auckland, New Zealand, December 2024
  • Dalia Hareb, Jean Martinet. EvSegSNN: Neuromorphic Semantic Segmentation for Event Data. IJCNN 2024, Yokohama, Japan. July 2024
  • Amélie Gruel, Dalia Hareb, Antoine Grimaldi, Jean Martinet, Laurent Perrinet, Bernabé Linares-Barranco and Teresa Serrano-Gotarredona. Stakes of Neuromorphic Foveation: a promising future for embedded event cameras. Biological Cybernetics Special Issue: What can Computer Vision learn from Visual Neuroscience? 2023
  • Dalia Hareb, Jean Martinet. EvSegSNN : Segmentation sémantique neuromorphique pour la vision événementielle. ORASIS 2023, Carqueiranne, France. May 2023


Publication additionnelle juste avant le démarrage du projet :

Amélie Gruel, Dalia Hareb, Jean Martinet, Bernabé Linares-Barranco, Teresa Serrano-Gotarredona. Neuromorphic foveation applied to semantic segmentation. CVPR 2022 workshop "NeuroVision: What can computer vision learn from visual neuroscience?", New Orleans, Louisiana, June 2022
Leveraged projects