Integrating RGB and eventsensors improves object detection accuracy, especially during the night, due to the high-dynamic range of event camera. However, introducing an eventsensor leads to an increase in computatio...
详细信息
Integrating RGB and eventsensors improves object detection accuracy, especially during the night, due to the high-dynamic range of event camera. However, introducing an eventsensor leads to an increase in computational resources, which makes the implementation of RGB-event fusion multi-modal AI to CiM difficult. To tackle this issue, this paper proposes RGB-event fusion Multi-modal analog Computation-in-Memory (CiM), called REM-CiM, for multi-modal edge object detection AI. In REM-CiM, two proposals about multi-modal AI algorithms and circuit implementation are co-designed. First, Memory capacity-Efficient Attentional Feature Pyramid Network (MEA-FPN), the model architecture for RGBevent fusion analog CiM, is proposed for parameter-efficient RGB-event fusion. Convolution-less bi-directional calibration (C-BDC) in MEA-FPN extracts important features of each modality with attention modules, while reducing the number of weight parameters by removing large convolutional operations from conventional BDC. Proposed MEA-FPN w/ C-BDC achieves a 76% reduction of parameters while maintaining mean Average Precision (mAP) degradation to < 2 . 3 % during both day and night, compared with Attentional FPN fusion (A-FPN), a conventional BDC-adopted FPN fusion. Second, the low-bit quantization with clipping (LQC) is proposed to reduce area/energy. Proposed REM-CiM with MEA-FPN and LQC achieves almost the same memory cells, 21% less ADC area, 24% less ADC energy and 0.17% higher mAP than conventional FPN fusion CiM without LQC.
For edge AI applications, this paper overviews neuromorphic computing with CiM, Computation-in-Memory with non-volatile memories. AI accelerators like CiM will be heterogeneously integrated with traditional processors...
详细信息
ISBN:
(纸本)9781665456722
For edge AI applications, this paper overviews neuromorphic computing with CiM, Computation-in-Memory with non-volatile memories. AI accelerators like CiM will be heterogeneously integrated with traditional processors such as CPUs. To extremely suppress energy of edge AI, the heterogeneous integration of sensors like event-basedsensors and CiM is promising. Approximate Computing for a wide range of fields such as system-level, circuit-level and device-level resolves the memory trade-off. By tolerating some degree of device errors, the performance, energy and cost of CiM are improved. This paper covers neural networks such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) as well as event driven Spiking Neural Network (SNN) and Reservoir Computing.
Various robots, rovers, drones, and other agents of mass-produced products are expected to encounter scenes where they intersect and collaborate in the near future. In such multi-agent systems, individual identificati...
详细信息
ISBN:
(纸本)9798400704864
Various robots, rovers, drones, and other agents of mass-produced products are expected to encounter scenes where they intersect and collaborate in the near future. In such multi-agent systems, individual identification and communication play crucial roles. In this paper, we explore camera-based visible light communication using event cameras to tackle this problem. An event camera captures the events occurring in regions with changes in brightness and can be utilized as a receiver for visible light communication, leveraging its high temporal resolution. Generally, agents with identical appearances in mass-produced products are visually indistinguishable when using conventional CMOS cameras. Therefore, linking visual information with information acquired through conventional radio communication is challenging. We empirically demonstrate the advantages of a visible light communication system employing event cameras and LEDs for visual individual identification over conventional CMOS cameras with ArUco marker recognition. In the simulation, we also verified scenarios where our event camera-based visible light communication outperforms conventional radio communication in situations with visually indistinguishable multi-agents. Finally, our newly implemented multi-agent system verifies its functionality through physical robot experiments.
This paper discusses co-designing integrated in-sensor and in-memory computing based on the analysis of event data and gives a system-level solution. By integrating an event-based vision sensor (EVS) as a sensor and e...
详细信息
This paper discusses co-designing integrated in-sensor and in-memory computing based on the analysis of event data and gives a system-level solution. By integrating an event-based vision sensor (EVS) as a sensor and event-driven computation-in-memory (CiM) as a processor, event data taken by EVS are processed in CiM. In this work, EVS is used to acquire the scenery from a driving car and the event data are analyzed. based on the EVS data characteristics of temporally dense and spatially sparse, event-driven SRAM-CiM is proposed for extremely energy-efficient edge computing. In the event-driven SRAM-CiM, a set of 8T-SRAMs stores multiple-bit synaptic weights of spiking neural networks. Multiply-accumulate operation with the multiple-bit synaptic weights is demonstrated by pulse amplitude modulation and pulse width modulation. By considering future EVS of high image resolution and high time resolution, the configuration of event-driven CiM for EVS is discussed.
Imaging flow cytometry (FC) is a powerful analytic tool that combines the principles of conventional FC with rich spatial information, allowing more profound insight into single-cell analysis. However, offering such h...
详细信息
Imaging flow cytometry (FC) is a powerful analytic tool that combines the principles of conventional FC with rich spatial information, allowing more profound insight into single-cell analysis. However, offering such high-resolution, full-frame feedback can restrain processing speed and has become a significant trade-off during development. In addition, the dynamic range (DR) offered by conventional photosensors can only capture limited fluorescence signals, which compromises the detection of high-velocity fluorescent objects. Neuromorphic photo-sensing focuses on the events of interest via individual-firing pixels to reduce data redundancy and latency. With its inherent high DR, this architecture has the potential to drastically elevate the performance in throughput and sensitivity to fluorescent targets. Herein, we presented an early demonstration of neuromorphic cytometry, demonstrating the feasibility of adopting an event-based resolution in describing spatiotemporal feedback on microscale objects and for the first time, including cytometric-like functions in object counting and size estimation to measure 8 mu m, 15 mu m microparticles and human monocytic cell line (THP-1). Our work has achieved highly consistent outputs with a widely adopted flow cytometer (CytoFLEX) in detecting microparticles. Moreover, the capacity of an event-based photosensor in registering fluorescent signals was evaluated by recording 6 mu m Fluorescein isothiocyanate-marked particles in different lighting conditions, revealing superior performance compared to a standard photosensor. Although the current platform cannot deliver multiparametric measurements on cells, future endeavours will include further functionalities and increase the measurement parameters (granularity, cell condition, fluorescence analysis) to enrich cell interpretation.
暂无评论