Brain-Inspired Light Sensor Could Speed AI Image Processing

2026-07-07 · Source: IEEE Spectrum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Advanced, short

Summary

A new brain-inspired imaging sensor, a phototransistor, can simultaneously detect light and store data, with the unique ability to gradually forget unneeded information. Developed by Larry Cheng and his team at Oregon State University, this device contrasts with conventional CMOS or CCD cameras that require data transfer to separate memory for processing. The prototype, a four-by-four-pixel array about the size of a USB stick, uses an organic light-absorbing layer and an Indium Gallium Zinc Oxide (IGZO) transistor channel. It stores a recent history of light intensity by trapping holes in organic semiconductor aggregates, which electrostatically modulate the transistor channel. This memory duration is tunable; applying positive voltage accelerates forgetting, while negative voltage prolongs retention for hours or more. This on-sensor processing capability promises significant energy demand reductions and improved speed for AI vision tasks, adapting its temporal response for applications like fast-moving drones or lingering doorbell camera surveillance.

Key takeaway

For AI Hardware Engineers designing embedded vision systems, this brain-inspired sensor architecture presents a critical shift. You should explore integrating on-sensor memory with tunable forgetting to significantly reduce energy consumption and accelerate real-time AI image processing. This approach allows your systems to adapt temporal responses for diverse applications, from high-speed drone navigation to persistent security monitoring, without constant data shuffling. Prioritize sensor designs that decouple light-sensing from electrical transport for optimal performance.

Key insights

A novel phototransistor integrates light detection, data storage, and tunable forgetting directly on-sensor, mimicking brain function.

Principles

• Integrating memory on-sensor significantly reduces energy consumption.
• Tunable memory timescales enable sensor adaptation for varied AI vision tasks.
• Brain-inspired forgetting mechanisms enhance data efficiency.

Method

Photons generate electrons and holes; trapped holes in organic semiconductor aggregates electrostatically modulate an IGZO transistor channel, retaining memory whose decay rate is voltage-tunable.

In practice

• Configure memory for high-speed object tracking (e.g., 250 kilometers per hour drones).
• Adjust memory for extended surveillance (e.g., doorbell cameras).

Topics

• Brain-Inspired Sensors
• On-Sensor Processing
• Phototransistors
• Tunable Memory
• Robotic Vision
• Energy Efficiency

Best for: Computer Vision Engineer, AI Scientist, AI Hardware Engineer, Robotics Engineer

Related on AIssential

• Brain-inspired phototransistor could cut AI energy use by sensing and storing data — Brain-inspired phototransistor integrates sensing and memory to reduce AI energy consumption.
• 😺 🎙️ Watch: Is Brain-like Computing What's Next? — Brain-inspired hardware using light and integrated memory-compute can overcome traditional AI bottlenecks.
• Energy-Aware Spike Budgeting for Continual Learning in Spiking Neural Networks for Neuromorphic Vision — An energy-aware spike budgeting framework improves SNN continual learning by adaptively balancing accuracy and energy across vision modalities.
• Running hardware-aware neural architecture search on embedded devices under 512MB of RAM — A novel HW NAS runs on embedded devices with <512MB RAM, producing tiny CNNs for IoT/robotics while ensuring privacy.
• Otters++: A Time-to-first-spike Based Energy Efficient Optical Spiking Transformer — Otters++ employs physical hardware signal decay for energy-efficient time-to-first-spike computation in optical spiking Transformers.
• Fixing AI’s Bottlenecks: Memory, Scale, and Sparsity — AI's future hinges on addressing memory, connectivity, and sparsity through engineered, biologically-inspired hardware and algorithms.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by IEEE Spectrum.