检索结果-内蒙古大学图书馆

CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Baumann, Alexander Ayala, Leonardo Seidlitz, Silvia Sellner, Jan Studier-Fischer, Alexander Özdemir, Berkin Maier-Hein, Lena Ilic, Slobodan Siemens AG Munich Germany Heidelberg Germany Medical Faculty Faculty of Mathematics and Computer Science Heidelberg University Germany NCT Heidelberg Germany HIDSS4Health Heidelberg Germany Department of Urology and Urosurgery University Medical Center Mannheim Germany Department of General Visceral and Transplantation Surgery Heidelberg University Hospital Germany Division of Intelligent Systems and Robotics in Urology DKFZ Heidelberg Germany DKFZ Hector Cancer Institute University Medical Center Mannheim Germany

Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding, and is already established as a critical modality in remote sensing. However, variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies, leading to camera-specific models with limited generalizability and inadequate cross-camera applicability. To address this bottleneck, we introduce CARL, a model for Camera-Agnostic Representation Learning across RGB, multispectral, and hyperspectral imaging modalities. To enable the conversion of a spectral image with any channel dimensionality to a camera-agnostic embedding, we introduce wavelength positional encoding and a self-attention-cross-attention mechanism to compress spectral information into learned query representations. Spectral-spatial pre-training is achieved with a novel spectral self-supervised JEPA-inspired strategy tailored to CARL. Large-scale experiments across the domains of medical imaging, autonomous driving, and satellite imaging demonstrate our model’s unique robustness to spectral heterogeneity, outperforming on datasets with simulated and real-world cross-camera spectral variations. The scalability and versatility of the proposed approach position our model as a backbone for future spectral foundation models. © 2025, CC BY.

关键词： Image coding

Hardware-in-the-loop simulation of a UAV autonomous landing algorithm implemented in SoC FPGA

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Szolc, Hubert Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Science and Technology Krakow Poland

This paper presents a system for hardware-in-the-loop (HiL) simulation of unmanned aerial vehicle (UAV) control algorithms implemented on a heterogeneous SoC FPGA computing platforms. The AirSim simulator running on a PC and an Arty Z7 development board with a Zynq SoC chip from AMD Xilinx were used. Communication was carried out via a serial USB link. An application for autonomous landing on a specially marked landing strip was selected as a case study. A landing site detection algorithm was implemented on the Zynq SoC platform. This allowed processing a 1280 × 720 @ 60 fps video stream in real time. Performed tests showed that the system works correctly and there are no delays that could negatively affect the stability of the control. The proposed concept is characterised by relative simplicity and low implementation cost. At the same time, it can be applied to test various types of high-level perception and control algorithms for UAV implemented on embedded platforms. We provide the code developed on GitHub, which includes both Python scripts running on the PC and C code running on Arty Z7. © 2022, CC BY.

关键词： System-on-chip

Energy Efficient Hardware Acceleration of Neural Networks with Power-of-Two Quantisation

学校读者我要写书评

暂无评论

TechRxiv 2022年

作者： Przewlocka-Rus, Dominika Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Science and Technology Krakow Poland

Deep neural networks virtually dominate the domain of most modern vision systems, providing high performance at a cost of increased computational complexity. Since for those systems it is often required to operate both in real-time and with minimal energy consumption (e.g., for wearable devices or autonomous vehicles, edge Internet of Things (IoT), sensor networks), various network optimisation techniques are used, e.g., quantisation, pruning, or dedicated lightweight architectures. Due to the logarithmic distribution of weights in neural network layers, a method providing high performance with significant reduction in computational precision (for 4-bit weights and less) is the Power-of-Two (PoT) quantisation (and therefore also with a logarithmic distribution). This method introduces additional possibilities of replacing the typical for neural networks Multiply and ACcumulate (MAC – performing, e.g., convolution operations) units, with more energy-efficient Bitshift and ACcumulate (BAC). In this paper, we show that a hardware neural network accelerator with PoT weights implemented on the Zynq UltraScale + MPSoC ZCU104 SoC FPGA can be at least 1.4x more energy efficient than the uniform quantisation version. To further reduce the actual power requirement by omitting part of the computation for zero weights, we also propose a new pruning method adapted to logarithmic quantisation. © 2022, CC BY.

关键词： System-on-chip

Towards real-time and energy efficient Siamese tracking - a hardware-software approach

学校读者我要写书评

暂无评论

TechRxiv 2022年

Siamese trackers have been among the state-of-the-art solutions in each Visual Object Tracking (VOT) challenge over the past few years. However, with great accuracy comes great computational complexity: to achieve real-time processing, these trackers have to be massively parallelised and are usually run on high-end GPUs. Easy to implement, this approach is energy consuming, and thus cannot be used in many low-power applications. To overcome this, one can use energy-efficient embedded devices, such as heterogeneous platforms joining the ARM processor system with programmable logic (FPGA). In this work, we propose a hardware-software implementation of the well-known fully connected Siamese tracker (SiamFC). We have developed a quantised Siamese network for the FINN accelerator, using algorithm-accelerator co-design, and performed design space exploration to achieve the best efficiency-to-energy ratio (determined by FPS and used resources). For our network, running in the programmable logic part of the Zynq UltraScale+ MPSoC ZCU104, we achieved the processing of almost 50 frames-per-second with tracker accuracy on par with its floating point counterpart, as well as the original SiamFC network. The complete tracking system, implemented in ARM with the network accelerated on FPGA, achieves up to 17 fps. These results bring us towards bridging the gap between the highly accurate but energy-demanding algorithms and energy-efficient solutions ready to be used in low-power, edge systems. © 2022, CC BY.

关键词： Field programmable gate arrays (FPGA)

Real-time FPGA implementation of the Semi-Global Matching stereo vision algorithm for a 4K/UHD video stream

学校读者我要写书评

暂无评论

TechRxiv 2023年

作者： Grabowski, Mariusz Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Science and Technology Krakow Poland

In this paper, we propose a real-time FPGA implementation of the Semi-Global Matching (SGM) stereo vision algorithm. The designed module supports a 4K/Ultra HD (3840×2160 pixels @ 30 frames per second) video stream in a 4 pixel per clock (ppc) format and a 64-pixel disparity range. The baseline SGM implementation had to be modified to process pixels in the 4ppc format and meet the timing constrains, however, our version provides results comparable to the original design. The solution has been positively evaluated on the Xilinx VC707 development board with a Virtex-7 FPGA device. © 2023, CC BY.

关键词： Field programmable gate arrays (FPGA)

EfficientNet-SAM: A Novel EffecientNet with Spatial Attention Mechanism for COVID-19 Detection in Pulmonary CT Scans

学校读者我要写书评

暂无评论

EfficientNet-SAM: A Novel EffecientNet with Spatial Attentio...

IEEE computer Society Conference on computer vision and Pattern Recognition Workshops (CVPRW)

作者： Ramy Farag Parth Upadhay Jacket Demby’s Yixiang Gao Katherin Garces Montoya Seyed Mohamad Ali Tousi Gbenga Omotara Guilherme DeSouza Department of Electrical Engineering and Computer Science Vision-Guided and Intelligent Robotics Lab - ViGIR Lab University of Missouri-Columbia

ISBN: (数字)9798350365474

ISBN: (纸本)9798350365481

Manual analysis and diagnosis of COVID-19 through the examination of Computed Tomography (CT) images of the lungs can be time-consuming and result in errors, especially given high volume of patients and numerous images per patient. So, we address the need for automation of this task by developing a new deep learning-based pipeline. Our motivation was sparked by the CVPR Workshop on "Domain Adaptation, Explainability and Fairness in AI for Medical Image Analysis", more specifically, the "COVID-19 Diagnosis Competition (DEF-AI-MIA COV19D)" under the same Workshop. This challenge provides an opportunity to assess our proposed pipeline for COVID-19 detection from CT scan images. The same pipeline incorporates one of the architectures in the EfficientNet "family", but with an added Spatial Attention Mechanism: EfficientNet-SAM. Also, unlike the traditional/past pipelines, which relied on a preprocessing step, our pipeline takes the raw selected input images without any such step, except for an image-selection step to simply reduce the number of CT images required for training and/or testing. Moreover, our pipeline is computationally efficient, as, for example, it does not incorporate a decoder for segmenting the lungs. It also does not combine different models nor combine RNN with a backbone, as other pipelines in the past did. Nevertheless, our pipeline outperformed all approaches presented by other teams in last year’s instance of the same challenge using the validation subset. It also placed 5th in this year’s competition, ranking less than 1.3% below the 1st place and close to 3.5% above the 6th place based on the macro-F1 score.

关键词： COVID-19 Training Visualization Attention mechanisms Computed tomography Conferences Pipelines

Optimisation of a siamese neural network for real-time energy efficient object tracking

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Przewlocka, Dominika Wasala, Mateusz Szolc, Hubert Blachut, Krzysztof Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Science and Technology Krakow Poland

In this paper the research on optimisation of visual object tracking using a Siamese neural network for embedded vision systems is presented. It was assumed that the solution shall operate in real-time, preferably for a high resolution video stream, with the lowest possible energy consumption. To meet these requirements, techniques such as the reduction of computational precision and pruning were considered. Brevitas, a tool dedicated for optimisation and quantisation of neural networks for FPGA implementation, was used. A number of training scenarios were tested with varying levels of optimisations - from integer uniform quantisation with 16 bits to ternary and binary networks. Next, the influence of these optimisations on the tracking performance was evaluated. It was possible to reduce the size of the convolutional filters up to 10 times in relation to the original network. The obtained results indicate that using quantisation can significantly reduce the memory and computational complexity of the proposed network while still enabling precise tracking, thus allow to use it in embedded vision systems. Moreover, quantisation of weights positively affects the network training by decreasing overfitting. Copyright © 2020, The Authors. All rights reserved.

关键词： Field programmable gate arrays (FPGA)

Optimisation of a siamese neural network for real-time energy efficient object tracking

学校读者我要写书评

暂无评论

TechRxiv 2020年

In this paper the research on optimisation of visual object tracking using a Siamese neural network for embedded vision systems is presented. It was assumed that the solution shall operate in real-time, preferably for a high resolution video stream, with the lowest possible energy consumption. To meet these requirements, techniques such as the reduction of computational precision and pruning were considered. Brevitas, a tool dedicated for optimisation and quantisation of neural networks for FPGA implementation, was used. A number of training scenarios were tested with varying levels of optimisations – from integer uniform quantisation with 16 bits to ternary and binary networks. Next, the influence of these optimisations on the tracking performance was evaluated. It was possible to reduce the size of the convolutional filters up to 10 times in relation to the original network. The obtained results indicate that using quantisation can significantly reduce the memory and computational complexity of the proposed network while still enabling precise tracking, thus allow to use it in embedded vision systems. Moreover, quantisation of weights positively affects the network training by decreasing overfitting. © 2020, CC BY-NC-SA.

关键词： Field programmable gate arrays (FPGA)

Real-time FPGA implementation of the Semi-Global Matching stereo vision algorithm for a 4K/UHD video stream

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Grabowski, Mariusz Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics Agh University of Science and Technology Krakow Poland

In this paper, we propose a real-time FPGA implementation of the Semi-Global Matching (SGM) stereo vision algorithm. The designed module supports a 4K/Ultra HD (3840 ×2160 pixels @ 30 frames per second) video stream in a 4 pixel per clock (ppc) format and a 64-pixel disparity range. The baseline SGM implementation had to be modified to process pixels in the 4ppc format and meet the timing constrains, however, our version provides results comparable to the original design. The solution has been positively evaluated on the Xilinx VC707 development board with a Virtex-7 FPGA device. © 2023, CC BY.

关键词： Field programmable gate arrays (FPGA)