检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Chen, Tao Wang, Chenhui Chen, Zhihao Lei, Yiming Shan, Hongming Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China

Medical image segmentation has been significantly advanced with the rapid development of deep learning (DL) techniques. Existing DL-based segmentation models are typically discriminative;i.e., they aim to learn a mapping from the input image to segmentation masks. However, these discriminative methods neglect the underlying data distribution and intrinsic class characteristics, suffering from unstable feature space. In this work, we propose to complement discriminative segmentation methods with the knowledge of underlying data distribution from generative models. To that end, we propose a novel hybrid diffusion framework for medical image segmentation, termed HiDiff, which can synergize the strengths of existing discriminative segmentation models and new generative diffusion models. HiDiff comprises two key components: discriminative segmentor and diffusion refiner. First, we utilize any conventional trained segmentation models as discriminative segmentor, which can provide a segmentation mask prior for diffusion refiner. Second, we propose a novel binary Bernoulli diffusion model (BBDM) as the diffusion refiner, which can effectively, efficiently, and interactively refine the segmentation mask by modeling the underlying data distribution. Third, we train the segmentor and BBDM in an alternate-collaborative manner to mutually boost each other. Extensive experimental results on abdomen organ, brain tumor, polyps, and retinal vessels segmentation datasets, covering four widely-used modalities, demonstrate the superior performance of HiDiff over existing medical segmentation algorithms, including the state-of-the-art transformer- and diffusion-based ones. In addition, HiDiff excels at segmenting small objects and generalizing to new datasets. Source codes are made available at https://***/takimailto/HiDiff. Copyright © 2024, The Authors. All rights reserved.

关键词： Diffusion

来源：评论

学校读者我要写书评

暂无评论

HighlightRemover: Spatially Valid Pixel Learning for Image Specular Highlight Removal 24

HighlightRemover: Spatially Valid Pixel Learning for Image S...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Zhang, Ling Ma, Yidong Jiang, Zhi He, Weilei Bao, Zhongyun Fu, Gang Xu, Wenju Xiao, Chunxia School of Computer Science and Technology Hubei Key Laboratory of Intelligent Information Processing and Realtime Industrial Systems Wuhan University of Science and Technology Wuhan China School of Computer Science Wuhan University Wuhan China Department of Computing Hong Kong Polytechnic University Hong Kong Amazon Palo Alto United States

ISBN: (纸本)9798400706868

Recently, learning-based methods have made significant progress for image specular highlight removal. However, many of these approaches treat all the image pixels uniformly, overlooking the negative impact of invalid pixels on feature reconstruction. This oversight often leads to undesirable outcomes, such as color distortion or residual highlights. In this paper, we propose a novel image specular highlight removal network called HighlightRNet, which utilizes valid pixels as references to reconstruct the highlight-free image. To achieve this, we introduce a context-aware fusion block (CFBlock) that aggregates information in four directions, effectively capturing global contextual information. Additionally, we introduce a location-aware feature transformation module (LFTModule) to adaptively learn the valid pixels for feature reconstruction, thereby avoiding information errors caused by invalid pixels. With these modules, our method can produce high-quality highlight-free results without color distortion and highlight residual. Furthermore, we develop a multiple light image-capturing system to construct a large-scale highlight dataset called NSH, which exhibits minimal misalignment in image pairs and minimal brightness variation in non-highlight regions. Experimental results on various datasets demonstrate the superiority of our method over state-of-the-art methods, both qualitatively and quantitatively. © 2024 ACM.

关键词： Pixels

来源：评论

学校读者我要写书评

暂无评论

Sketch-based Point Cloud Generation with Diffusion Model and Pre-training Enhancement

Sketch-based Point Cloud Generation with Diffusion Model and...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Yangdong Chen Mohan Chen Yuejie Zhang Rui Feng Tao Zhang Shang Gao School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China School of Information Management and Engineering Shanghai Key Laboratory of Financial Information Technology Shanghai University of Finance and Economics Shanghai China School of Information Technology Deakin University Victoria Australia

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Diffusion models, known for their success in various generative tasks like image generation and super-resolution, are applied in this study for point cloud generation, a field that has not been extensively explored due to the complexity of point clouds. We propose a novel method using a diffusion model to generate high-quality 3D point clouds from 2D sketches. This method employs a self-supervised contrastive learning scheme to align sketch and point cloud modalities. Additionally, it incorporates a specific partition mixing strategy to integrate edge information during pre-training. Evaluated on two benchmark datasets, our method outperforms existing state-of-the-art approaches, showcasing the potential of diffusion models in point cloud generation and setting a new direction for future research.

关键词： Point cloud compression Three-dimensional displays Image synthesis Superresolution Contrastive learning Benchmark testing Signal processing Diffusion models Stability analysis Speech processing

来源：评论

学校读者我要写书评

暂无评论

Global Cognition and Local Perception Network for Blind Image Deblurring 27th

Global Cognition and Local Perception Network for Blind Imag...

引用

27th International Conference on MultiMedia Modeling, MMM 2021

作者： Zhang, Chuanfa Zhang, Wei Chen, Feiyu Cheng, Yiting Gao, Shuyong Zhang, Wenqiang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

ISBN: (纸本)9783030678319

Nowadays, people are more and more used to taking pictures with handheld devices. However, the photos taken by handheld devices are easy to blur. Recent state-of-the-art methods for image deblurring often restore image through multiple stages, which leads to excessive network parameters and slow running speed, and makes it difficult to apply to mobile devices. While one-stage image deblurring methods have fewer parameters and faster running speed, the performance of their models is not so satisfactory. To solve these problems, we propose a lightweight one-stage image deblurring network called Global Cognition and Local Perception Network (GCLPN) in this paper. We design the Global Cognition Module (GCM) to obtain global feature descriptors and present the Local Perception Module (LPM) to help image reconstruction in local regions. Furthermore, we introduce a gradient loss that focuses on subtle image texture information during network training. Experimental results illustrate that our GCLPN surpasses the state-of-the-art methods in standard metrics of PSNR and SSIM with the least number of parameters (about 1.7M) and real-time running speed (about 0.37 s per 720 × 1280 image). © 2021, Springer Nature Switzerland AG.

关键词： Image enhancement

来源：评论

学校读者我要写书评

暂无评论

SliceProp: A Slice-Wise Bidirectional Propagation Model for Interactive 3D Medical Image Segmentation 1

SliceProp: A Slice-Wise Bidirectional Propagation Model for ...

引用

1st IEEE International Conference on Medical Artificial Intelligence, MedAI 2023

作者： Xu, Xin Lu, Wenjing Lei, Jiahao Qiu, Peng Shen, Hong-Bin Yang, Yang Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Shanghai Ninth People's Hospital Shanghai Jiao Tong University School of Medicine Department of Vascular Surgery China Shanghai Jiao Tong University Institute of Image Processing and Pattern Recognition Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China

ISBN: (纸本)9798350358780

Interactive medical image segmentation methods have become increasingly popular in recent years. These methods combine manual labeling and automatic segmentation, reducing the workload of annotation while maintaining high accuracy. However, most current interactive segmentation frameworks are limited to 2D image data, and are not suitable for 3D image data due to the large size and high complexity of 3D data, as well as the challenges posed by information asymmetry and sparse annotation. In this paper, we propose SliceProp, an interactive segmentation framework that implements slice-wise label Bidirectional Propagation (LBP) for 3D medical image segmentation. SliceProp extends the interactive 2D image segmentation algorithm to 3D image segmentation, and can handle 3D data with large size and high complexity. Moreover, equipped with a Backtracking Feedback Check (BFC) module, SliceProp effectively addresses the issues of information asymmetry and spatial sparse annotation in 3D medical image segmentation. Additionally, we adopt an uncertainty-based criterion to pri-oritize the slices to be refined interactively, which enhances the efficiency of the interaction process by enabling the model to focus on the regions with the most unreliable predictions. SliceProp is evaluated on two datasets and achieves promising results compared to state-of-the-art methods. © 2023 IEEE.

关键词： Medical imaging

来源：评论

学校读者我要写书评

暂无评论

X-Prompt: Multi-modal Visual Prompt for Video Object Segmentation

arXiv

引用

arXiv 2024年

作者： Guo, Pinxue Li, Wanyun Huang, Hao Hong, Lingyi Zhou, Xinyu Chen, Zhaoyu Li, Jinglun Jiang, Kaixun Zhang, Wei Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China

Multi-modal Video Object Segmentation (VOS), including RGB-Thermal, RGB-Depth, and RGB-Event, has garnered attention due to its capability to address challenging scenarios where traditional VOS methods struggle, such as extreme illumination, rapid motion, and background distraction. Existing approaches often involve designing specific additional branches and performing full-parameter fine-tuning for fusion in each task. However, this paradigm not only duplicates research efforts and hardware costs but also risks model collapse with the limited multi-modal annotated data. In this paper, we propose a universal framework named X-Prompt for all multi-modal video object segmentation tasks, designated as RGB+X. The X-Prompt framework first pretrains a video object segmentation foundation model using RGB data, and then utilize the additional modality of the prompt to adapt it to downstream multi-modal tasks with limited data. Within the X-Prompt framework, we introduce the Multi-modal Visual Prompter (MVP), which allows prompting foundation model with the various modalities to segment objects precisely. We further propose the Multi-modal Adaptation Experts (MAEs) to adapt the foundation model with pluggable modality-specific knowledge without compromising the generalization capacity. To evaluate the effectiveness of the X-Prompt framework, we conduct extensive experiments on 3 tasks across 4 benchmarks. The proposed universal X-Prompt framework consistently outperforms the full fine-tuning paradigm and achieves state-of-the-art performance. Code: https://***/PinxueGuo/***. Copyright © 2024, The Authors. All rights reserved.

关键词： Image segmentation

来源：评论

学校读者我要写书评

暂无评论

EC-SLAM: Effectively Constrained Neural RGB-D SLAM with Sparse TSDF Encoding and Global Bundle Adjustment

arXiv

引用

arXiv 2024年

作者： Li, Guanghao Chen, Qi Yan, Yuxiang Pu, Jian Institute of Science and Technology for Brain-Inspired Intelligence Fudan University Shanghai200433 China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China

We introduce EC-SLAM, a real-time dense RGB-D simultaneous localization and mapping (SLAM) system leveraging Neural Radiance Fields (NeRF). While recent NeRF-based SLAM systems have shown promising results, they have yet to fully exploit NeRF’s potential to constrain pose optimization. EC-SLAM addresses this by using sparse parametric encodings and Truncated Signed Distance Fields (TSDF) to represent the map, enabling efficient fusion, reducing model parameters, and accelerating convergence. Our system also employs a globally constrained Bundle Adjustment (BA) strategy that capitalizes on NeRF’s implicit loop closure correction capability, improving tracking accuracy by reinforcing constraints on keyframes most relevant to the current optimized frame. Furthermore, by integrating a feature-based and uniform sampling strategy that minimizes ineffective constraint points for pose optimization, we reduce the impact of random sampling in NeRF. Extensive evaluations on the Replica, ScanNet, and TUM datasets demonstrate state-of-the-art performance, with precise tracking and reconstruction accuracy achieved alongside real-time operation at up to 21 Hz. The source code is available at https://***/Lightingooo/EC-SLAM. Copyright © 2024, The Authors. All rights reserved.

关键词： Mapping

来源：评论

学校读者我要写书评

暂无评论

Boosting Fine-Grained Sketch-Based Image Retrieval with Self-Supervised Learning

Boosting Fine-Grained Sketch-Based Image Retrieval with Self...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Zhaolong Zhang Yangdong Chen Yuejie Zhang Rui Feng Tao Zhang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China School of Information Management and Engineering Shanghai University of Finance and Economics Shanghai China

Fine-grained sketch-based image retrieval (FG-SBIR) aims at aligning images and sketches at the instance level. It is a challenging task as there are significant differences between sketch and image. Existing methods usually produce less desired performance due to the lack of large-scale fine-grained image-sketch datasets and the strong dependence on the classification models pretrained on ImageNet. In this paper, we propose a better self-supervised pre-trained FG-SBIR model which does not depend on large-scale annotated datasets. Only images and their corresponding edge maps are used at the pre-training stage. Mixed modal transformation is designed to generate different mixed-up views. The FG-SBIR model is pre-trained by minimizing the distance between the views of the same instance and then fine-tuned by a simple triplet loss. With a plain downstream network, it achieves generally better performance than state-of-the-art models on three widely used FG-SBIR datasets.

关键词： Image edge detection Image retrieval Self-supervised learning Signal processing Boosting Acoustics Task analysis

来源：评论

学校读者我要写书评

暂无评论

Minimizing Disparities between Real and Pseudo Queries for Unsupervised Visual Grounding

Minimizing Disparities between Real and Pseudo Queries for U...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Hui Jiang Changkai Ji Jilan Xu Yanhao Zhu Yuejie Zhang Rui Feng Tao Zhang Shang Gao School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China School of Information Management and Engineering Shanghai Key Laboratory of Financial Information Technology Shanghai University of Finance and Economics Shanghai China School of Information Technology Deakin University Victoria Australia

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Visual grounding involves the identification and localization of image regions given textual descriptions. To reduce the manual labeling effort on region-text pairs, unsupervised visual grounding aims to generate pseudo bounding box and query pairs for training grounding models. However, there exists significant disparities between real and pseudo queries in terms of object, attribute distributions, and textual formats, limiting the generalization performance of unsupervised grounding methods. To address this challenge, we propose a novel unsupervised visual grounding framework. During training, we prompt Multimodal Large Language Models to generate pseudo queries, in which the entities are beyond the object detector’s pre-defined limited categories, and are associated with richer attributes. We further devise a Modifier Tree structure to bridge the gap of textual format between real and pseudo queries. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art unsupervised approaches on public benchmark datasets, particularly when dealing with complex queries.

关键词： Training Location awareness Visualization Limiting Grounding Large language models Manuals labeling Speech processing Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Read Extensively, Focus Smartly: A Cross-document Semantic Enhancement Method for Visual Documents NER 29

Read Extensively, Focus Smartly: A Cross-document Semantic E...

引用

29th International Conference on Computational Linguistics, COLING 2022

作者： Zhao, Jun Zhao, Xin Zhan, Wenyu Gui, Tao Zhang, Qi Qiao, Liang Cheng, Zhanzhan Pu, Shiliang School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China Institute of Modern Languages and Linguistics Fudan University China Hikvision Research Institute Shanghai China

The introduction of multimodal information and pretraining technique significantly improves entity recognition from visually-rich documents. However, most of the existing methods pay unnecessary attention to irrelevant regions of the current document while ignoring the potentially valuable information in related documents. To deal with this problem, this work proposes a cross-document semantic enhancement method, which consists of two modules: 1) To prevent distractions from irrelevant regions in the current document, we design a learnable attention mask mechanism, which is used to adaptively filter redundant information in the current document. 2) To further enrich the entity-related context, we propose a cross-document information awareness technique, which enables the model to collect more evidence across documents to assist in prediction. The experimental results on two documents understanding benchmarks covering eight languages demonstrate that our method outperforms the SOTA methods. © 2022 Proceedings - International Conference on Computational Linguistics, COLING. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：