检索结果-内蒙古大学图书馆

Deep learning-based magnetic resonance image super-resolution: a survey

学校读者我要写书评

暂无评论

Neural Computing and Applications 2024年第21期36卷 12725-12752页

作者： Ji, Zexin Zou, Beiji Kui, Xiaoyan Liu, Jun Zhao, Wei Zhu, Chengzhang Dai, Peishan Dai, Yulan School of Computer Science and Engineering Central South University Changsha410083 China Hunan Engineering Research Center of Machine Vision and Intelligent Medicine Central South University Changsha410083 China Department of Radiology The Second Xiangya Hospital Central South University Changsha410011 China The College of Literature and Journalism Central South University Changsha410083 China

Magnetic resonance imaging (MRI) is a medical imaging technique used to show anatomical structures and physiological processes of the human body. Due to limitations like image acquisition time, hardware capabilities, or uncooperative patients, the resolution of MR images is insufficient. Super-resolution (SR) is a crucial method to enhance the resolution of images without expensive scanning equipment. Recent years have witnessed significant progress in MR image super-resolution. Therefore, this survey presents a thorough overview of current developments in deep learning-based MR image super-resolution methods. In general, we can roughly divide the MRI super-resolution methods into single-contrast MR image SR methods and multi-contrast MR image SR methods. Additionally, we introduce the multi-task learning approaches about the MR image super-resolution. We also summarize other crucial topics, such as the degradation model, the definition of the super-resolution problem, the dataset, loss functions, and image quality assessment. Lastly, we indicate the challenges in the field of super-resolution and draw a conclusion to our survey. © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024.

关键词： Magnetic resonance imaging

A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation 37

学校读者我要写书评

暂无评论

A Single 2D Pose with Context is Worth Hundreds for 3D Human...

37th Conference on Neural Information Processing Systems, NeurIPS 2023

作者： Zhao, Qitao Zheng, Ce Liu, Mengyuan Chen, Chen Robotics Institute Carnegie Mellon University United States Center for Research in Computer Vision University of Central Florida United States Key Laboratory of Machine Perception Peking University Shenzhen Graduate School China

ISBN: (纸本)9781713899921

The dominant paradigm in 3D human pose estimation that lifts a 2D pose sequence to 3D heavily relies on long-term temporal clues (i.e., using a daunting number of video frames) for improved accuracy, which incurs performance saturation, intractable computation and the non-causal problem. This can be attributed to their inherent inability to perceive spatial context as plain 2D joint coordinates carry no visual cues. To address this issue, we propose a straightforward yet powerful solution: leveraging the readily available intermediate visual representations produced by off-the-shelf (pre-trained) 2D pose detectors - no finetuning on the 3D task is even needed. The key observation is that, while the pose detector learns to localize 2D joints, such representations (e.g., feature maps) implicitly encode the joint-centric spatial context thanks to the regional operations in backbone networks. We design a simple baseline named Context-Aware PoseFormer to showcase its effectiveness. Without access to any temporal information, the proposed method significantly outperforms its context-agnostic counterpart, PoseFormer [74], and other state-of-the-art methods using up to hundreds of video frames regarding both speed and precision. Project page: ***/ContextAware-PoseFormer. © 2023 Neural information processing systems foundation. All rights reserved.

关键词：

Action Recognition Based on Multi-perspective Feature Excitation 21st

学校读者我要写书评

暂无评论

Action Recognition Based on Multi-perspective Feature Excit...

21st Pacific Rim International Conference on Artificial Intelligence, PRICAI 2024

作者： Li, Xiaoyang Yang, Wenzhu Cui, Zhenchao School of Cyber Security and Computer Hebei University Qiyi East Road Lian Chi Hebei Province Baoding071000 China Machine Vision Engineering Research Center Hebei University Qiyi East Road Lian Chi Hebei Province Baoding071000 China

ISBN: (纸本)9789819601219

Some traditional action recognition methods mainly rely on capturing appearance information to recognize human activities, which in the real world often take place in complex environments, so it becomes a challenge to identify human activities in complex environments accurately. A good way to address this challenge is to excite valuable features from multiple angles (e.g., appearance information, temporal relations and channel relations) for action recognition. Based on this idea, we proposed a Group Excitation (GE) block that excites features from different perspectives along different channel groups in parallel. The GE block enhances the ability to capture complementary information that includes temporal and spatial context, maintaining relatively low computational costs. In particular, we design a set of excitation paths whose axial contexts are dynamically aggregated from other axes to contextualize the feature channel groups. We equip ResNet-50 with the GE block to form a simple but effective GENet with limited extra computational cost. The GENet can capture contextual information from different perspectives, making the network more resilient in recognizing complex human activities. We conducted extensive experiments on Something-Something V1, V2, and UCF101, and GENet has achieved competitive performance. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Complex networks

CDFSL-V: Cross-Domain Few-Shot Learning for Videos

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Samarasinghe, Sarinda Rizve, Mamshad Nayeem Kardan, Navid Shah, Mubarak Center for Research in Computer Vision University of Central Florida OrlandoFL United States

Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples, thereby reducing the challenges associated with collecting and annotating large-scale video datasets. Existing methods in video action recognition rely on large labeled datasets from the same domain. However, this setup is not realistic as novel categories may come from different data domains that may have different spatial and temporal characteristics. This dissimilarity between the source and target domains can pose a significant challenge, rendering traditional few-shot action recognition techniques ineffective. To address this issue, in this work, we propose a novel cross-domain few-shot video action recognition method that leverages self-supervised learning and curriculum learning to balance the information from the source and target domains. To be particular, our method employs a masked autoencoder-based self-supervised training objective to learn from both source and target data in a self-supervised manner. Then a progressive curriculum balances learning the discriminative information from the source dataset with the generic information learned from the target domain. Initially, our curriculum utilizes supervised learning to learn class discriminative features from the source data. As the training progresses, we transition to learning target-domain-specific features. We propose a progressive curriculum to encourage the emergence of rich features in the target domain based on class discriminative supervised features in the source domain. We evaluate our method on several challenging benchmark datasets and demonstrate that our approach outperforms existing cross-domain few-shot learning techniques. Our code is available at https://***/Sarinda251/CDFSL-V © 2023, CC BY-NC-SA.

关键词： Curricula

No More Shortcuts: Realizing the Potential of Temporal Self-Supervision

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dave, Ishan Rajendrakumar Jenni, Simon Shah, Mubarak Center for Research in Computer Vision University of Central Florida United States Adobe Research United States

Self-supervised approaches for video have shown impressive results in video understanding tasks. However, unlike early works that leverage temporal self-supervision, current state-of-the-art methods primarily rely on tasks from the image domain (e.g., contrastive learning) that do not explicitly promote the learning of temporal features. We identify two factors that limit existing temporal self-supervision: 1) tasks are too simple, resulting in saturated training performance, and 2) we uncover shortcuts based on local appearance statistics that hinder the learning of high-level features. To address these issues, we propose 1) a more challenging reformulation of temporal self-supervision as frame-level (rather than clip-level) recognition tasks and 2) an effective augmentation strategy to mitigate shortcuts. Our model extends a representation of single video frames, pre-trained through contrastive learning, with a transformer that we train through temporal self-supervision. We demonstrate experimentally that our more challenging frame-level task formulations and the removal of shortcuts drastically improve the quality of features learned through temporal self-supervision. The generalization capability of our self-supervised video method is evidenced by its state-of-the-art performance in a wide range of high-level semantic tasks, including video retrieval, action classification, and video attribute recognition (such as object and scene identification), as well as low-level temporal correspondence tasks like video object segmentation and pose tracking. Additionally, we show that the video representations learned through our method exhibit increased robustness to the input perturbations. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Images

学校读者我要写书评

暂无评论

Blowing in the Wind: CycleNet for Human Cinemagraphs from St...

Conference on computer vision and Pattern Recognition (CVPR)

作者： Hugo Bertiche Niloy J. Mitra Kuldeep Kulkarni Chun-Hao Paul Huang Tuanfeng Y. Wang Meysam Madadi Sergio Escalera Duygu Ceylan Universitat de Barcelona Computer Vision Center University College London Adobe Research

Cinemagraphs are short looping videos created by adding subtle motions to a static image. This kind of media is popular and engaging. However, automatic generation of cinemagraphs is an underexplored area and current solutions require tedious low-level manual authoring by artists. In this paper, we present an automatic method that allows generating human cinemagraphs from single RGB images. We investigate the problem in the context of dressed humans under the wind. At the core of our method is a novel cyclic neural network that produces looping cinemagraphs for the target loop duration. To circumvent the problem of collecting real data, we demonstrate that it is possible, by working in the image normal space, to learn garment motion dynamics on synthetic data and generalize to real data. We evaluate our method on both synthetic and real data and demonstrate that it is possible to create compelling and plausible cinemagraphs from single RGB images.

关键词：

3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds

学校读者我要写书评

暂无评论

3DMODT: Attention-Guided Affinities for Joint Detection & Tr...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Jyoti Kini Ajmal Mian Mubarak Shah Center for Research in Computer Vision University of Central Florida USA University of Western Australia

We propose a method for joint detection and tracking of multiple objects in 3D point clouds, a task conventionally treated as a two-step process comprising object detection followed by data association. Our method embeds both steps into a single end-to-end trainable network eliminating the dependency on external object detectors. Our model exploits temporal information employing multiple frames to detect objects and track them in a single network, thereby making it a utilitarian formulation for real-world scenarios. Computing affinity matrix by employing features similarity across consecutive point cloud scans forms an integral part of visual tracking. We propose an attention-based refinement module to refine the affinity matrix by suppressing erroneous correspondences. The module is designed to capture the global context in affinity matrix by employing self-attention within each affinity matrix and cross-attention across a pair of affinity matrices. Unlike competing approaches, our network does not require complex post-processing algorithms, and directly processes raw LiDAR frames to output tracking results. We demonstrate the effectiveness of our method on three tracking benchmarks: JRDB, Waymo, and KITTI. Experimental evaluations indicate the ability of our model to generalize well across datasets.

关键词：

R2Former: Unified Retrieval and Reranking Transformer for Place Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhu, Sijie Yang, Linjie Chen, Chen Shah, Mubarak Shen, Xiaohui Wang, Heng ByteDance China Center for Research in Computer Vision University of Central Florida United States

Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database. Conventional methods generally adopt aggregated CNN features for global retrieval and RANSAC-based geometric verification for reranking. However, RANSAC only employs geometric information but ignores other possible information that could be useful for reranking, e.g. local feature correlations, and attention values. In this paper, we propose a unified place recognition framework that handles both retrieval and reranking with a novel transformer model, named R2Former. The proposed reranking module takes feature correlation, attention value, and xy coordinates into account, and learns to determine whether the image pair is from the same location. The whole pipeline is end-to-end trainable and the reranking module alone can also be adopted on other CNN or transformer backbones as a generic component. Remarkably, R2Former significantly outperforms state-of-the-art methods on major VPR datasets with much less inference time and memory consumption. It also achieves the state-of-the-art on the hold-out MSLS challenge set and could serve as a simple yet strong solution for real-world large-scale applications. Experiments also show vision transformer tokens are comparable and sometimes better than CNN local features on local matching. The code is released at https://***/Jeff-Zilence/R2Former. Copyright © 2023, The Authors. All rights reserved.

关键词： Query processing

EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Velazquez, Diego López, Pau Rodriguez Alonso, Sergio Gonfaus, Josep M. Gonzalez, Jordi Richarte, Gerardo Marin, Javier Bengio, Yoshua Lacoste, Alexandre Computer Vision Center Satellogic Uruguay Mila Université de Montréal Canada ServiceNow Research United States Apple Research United States

This paper presents EarthView, a comprehensive dataset specifically designed for self-supervision on remote sensing data, intended to enhance deep learning applications on Earth monitoring tasks. The dataset spans 15 tera pixels of global remote-sensing data, combining imagery from a diverse range of sources, including NEON, Sentinel, and a novel release of 1m spatial resolution data from Satellogic. Our dataset provides a wide spectrum of image data with varying resolutions, harnessed from different sensors and organized coherently into an accessible HuggingFace dataset1 in parquet format. This data spans five years, from 2017 to 2022. Accompanying the dataset, we introduce EarthMAE, a tailored Masked Autoencoder, developed to tackle the distinct challenges of remote sensing data. Trained in a self-supervised fashion, EarthMAE effectively processes different data modalities such as hyperspectral, multispectral, topographical data, segmentation maps, and temporal structure. This model helps us show that pre-training on Satellogic data improves performance on downstream tasks. While there is still a gap to fill in MAE for heterogeneous data, we regard this innovative combination of an expansive, diverse dataset and a versatile model adapted for self-supervised learning as a stride forward in deep learning for Earth monitoring. © 2025, CC BY.

关键词： Self-supervised learning