检索结果-内蒙古大学图书馆

3D computational modeling and perceptual analysis of kinetic depth effects

学校读者我要写书评

暂无评论

Computational Visual Media 2020年第3期6卷 265-277页

作者： Meng-Yao Cui Shao-Ping Lu Miao Wang Yong-Liang Yang Yu-Kun Lai Paul L.Rosin TKLNDST CSNankai UniversityTianjinChina State Key Laboratory of Virtual Reality Technology and Systems Beihang UniversityBeijingChina Department of Computer Science University of BathUK School of Computer Science and Informatics Cardiff UniversityWalesUK

Humans have the ability to perceive kinetic depth effects, i.e., to perceived 3 D shapes from 2 D projections of rotating 3 D objects. This process is based on a variety of visual cues such as lighting and shading effects. However, when such cues are weak or missing, perception can become faulty, as demonstrated by the famous silhouette illusion example of the spinning dancer. Inspired by this, we establish objective and subjective evaluation models of rotated3 D objects by taking their projected 2 D images as input. We investigate five different cues: ambient luminance, shading, rotation speed, perspective, and color difference between the objects and *** the objective evaluation model, we first apply3 D reconstruction algorithms to obtain an objective reconstruction quality metric, and then use quadratic stepwise regression analysis to determine weights of depth cues to represent the reconstruction quality. In the subjective evaluation model, we use a comprehensive user study to reveal correlations with reaction time and accuracy, rotation speed, and perspective. The two evaluation models are generally consistent, and potentially of benefit to inter-disciplinary research into visual perception and 3 D reconstruction.

关键词： rotation spinning rotating

Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jin, Yueming Yu, Yang Chen, Cheng Zhao, Zixu Heng, Pheng-Ann Stoyanov, Danail The Department of Computer Science University College London United Kingdom The Department of Computer Science and Engineering The Chinese University of Hong Kong HKSAR China Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen China

Automatic surgical scene segmentation is fundamental for facilitating cognitive intelligence in the modern operating theatre. Previous works rely on conventional aggregation modules (e.g., dilated convolution, convolutional LSTM), which only make use of the local context. In this paper, we propose a novel framework STswinCL that explores the complementary intra- and inter-video relations to boost segmentation performance, by progressively capturing the global context. We firstly develop a hierarchy Transformer to capture intra-video relation that includes richer spatial and temporal cues from neighbor pixels and previous frames. A joint space-time window shift scheme is proposed to efficiently aggregate these two cues into each pixel embedding. Then, we explore inter-video relation via pixel-to-pixel contrastive learning, which well structures the global embedding space. A multi-source contrast training objective is developed to group the pixel embeddings across videos with the ground-truth guidance, which is crucial for learning the global property of the whole data. We extensively validate our approach on two public surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset. Experimental results demonstrate the promising performance of our method, which consistently exceeds previous state-of-the-art approaches. Code is available at https://***/YuemingJin/STswinCL. Copyright © 2022, The Authors. All rights reserved.

关键词： Pixels

Mmaa: An Attribute-Aware Collaborative Learning Network for Radiology Report Generation

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Guo, Yuting Li, Shuai Song, Wenfeng Hao, Aimin Qin, Hong State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China School of Computer Science Beijing Information Science and Technology University Beijing100192 China The Department of Computer Science Stony Brook University State University of New York at Stony Brook Stony BrookNY11794-2424 United States

Background and objectives: To date, deep learning-based intelligent diagnosis directly on clinical images has achieved remarkable success. The objective of this research is to leverage artificial intelligence methods to automatically generate intelligent medical reports. This innovation is expected to significantly reduce the workload of senior doctors while enhancing the accuracy of inexperienced radiologists. However, AI-driven automatic approaches still face challenges in generating comprehensive, accurate, and semantically consistent medical reports. One of the challenges is that the generated text descriptions, although mimicking the writing and structural styles of clinical reports, often fail to precisely identify diagnostic attributes and their semantics. Another challenge is the difficulty in capturing patient-specific symptoms due to the imbalanced distribution of training ***: This paper proposes an intelligent medical report generation system that employs collaborative learning of image features and clinical symptom attributes. This approach aims to effectively combine spatial visual features extracted from clinical images with underlying personalized symptom attributes. The system includes an encoder that learns spatial features from clinical images. To incorporate medical knowledge of symptoms, attribute labels are embedded as bottom-up features and fused with the encoded spatial features (acting as top-down features). Our novel feature fusion strategy involves two important aligning modules: the image-attribute memory and the text-attribute memory. These modules are designed to capture the relationships among attributes and their distributions, as well as guide the attention of the decoder during report ***: Extensive experiments and comprehensive evaluations on two public X-ray image datasets validate the effectiveness and advantages of our novel approach. Especially, the BLEU-2 metric increased by 1.8% on the IU-Xray datas

关键词： Semantics

RepMode: Learning to Re-parameterize Diverse Experts for Subcellular Structure Prediction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhou, Donghao Gu, Chunbin Xu, Junde Liu, Furui Wang, Qiong Chen, Guangyong Heng, Pheng-Ann Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China The Chinese University of Hong Kong Hong Kong Zhejiang Lab China

In biological research, fluorescence staining is a key technique to reveal the locations and morphology of subcellular structures. However, it is slow, expensive, and harmful to cells. In this paper, we model it as a deep learning task termed subcellular structure prediction (SSP), aiming to predict the 3D fluorescent images of multiple subcellular structures from a 3D transmitted-light image. Unfortunately, due to the limitations of current biotechnology, each image is partially labeled in SSP. Besides, naturally, subcellular structures vary considerably in size, which causes the multi-scale issue of SSP. To overcome these challenges, we propose Re-parameterizing Mixture-of-Diverse-Experts (RepMode), a network that dynamically organizes its parameters with task-aware priors to handle specified single-label prediction tasks. In RepMode, the Mixture-of-Diverse-Experts (MoDE) block is designed to learn the generalized parameters for all tasks, and gating re-parameterization (GatRep) is performed to generate the specialized parameters for each task, by which RepMode can maintain a compact practical topology exactly like a plain network, and meanwhile achieves a powerful theoretical topology. Comprehensive experiments show that RepMode can achieve state-of-the-art overall performance in SSP. Copyright © 2022, The Authors. All rights reserved.

关键词： Forecasting

Distribution-Aware Calibration for Object Detection with Noisy Bounding Boxes

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Donghao Li, Jialin Li, Jinpeng Huang, Jiancheng Nie, Qiang Liu, Yong Gao, Bin-Bin Wang, Qiong Heng, Pheng-Ann Chen, Guangyong The Chinese University of Hong Kong Hong Kong Tencent YouTu Lab Shenzhen China Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China Guangzhou China Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China Zhejiang Lab Hangzhou China

Large-scale well-annotated datasets are of great importance for training an effective object detector. However, obtaining accurate bounding box annotations is laborious and demanding. Unfortunately, the resultant noisy bounding boxes could cause corrupt supervision signals and thus diminish detection performance. Motivated by the observation that the real ground-truth is usually situated in the aggregation region of the proposals assigned to a noisy ground-truth, we propose DIStribution-aware CalibratiOn (DISCO) to model the spatial distribution of proposals for calibrating supervision signals. In DISCO, spatial distribution modeling is performed to statistically extract the potential locations of objects. Based on the modeled distribution, three distribution-aware techniques, i.e., distribution-aware proposal augmentation (DA-Aug), distribution-aware box refinement (DA-Ref), and distribution-aware confidence estimation (DA-Est), are developed to improve classification, localization, and interpretability, respectively. Extensive experiments demonstrate that DISCO can achieve SOTA performance in this task, especially at high noise levels. Code is available at https://***/Correr-Zhou/DISCO. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Mingye Xu, Mutian He, Tong Ouyang, Wanli Wang, Yali Han, Xiaoguang Qiao, Yu The Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China SSE CUHKSZ China University of Chinese Academy of Sciences China Shanghai Artificial Intelligence Laboratory China FNii CUHKSZ China

Masked Modeling (MM) has demonstrated widespread success in various vision challenges, by reconstructing masked visual patches. Yet, applying MM for large-scale 3D scenes remains an open problem due to the data sparsity and scene complexity. The conventional random masking paradigm used in 2D images often causes a high risk of ambiguity when recovering the masked region of 3D scenes. To this end, we propose a novel informative-preserved reconstruction, which explores local statistics to discover and preserve the representative structured points, effectively enhancing the pretext masking task for 3D scene understanding. Integrated with a progressive reconstruction manner, our method can concentrate on modeling regional geometry and enjoy less ambiguity for masked reconstruction. Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas. By elegantly combining informative-preserved reconstruction on masked areas and consistency self-distillation from unmasked areas, a unified framework called MM-3DScene is yielded. We conduct comprehensive experiments on a host of downstream tasks. The consistent improvement (e.g., +6.1% mAP@0.5 on object detection and +2.2% mIoU on semantic segmentation) demonstrates the superiority of our approach. © 2022, CC BY-NC-SA.

关键词： Distillation

Layout-Bridging Text-to-Image Synthesis

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liang, Jiadong Pei, Wenjie Lu, Feng The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Department of Computer Science Harbin Institute of Technology at Shenzhen Shenzhen518057 China

The crux of text-to-image synthesis stems from the difficulty of preserving the cross-modality semantic consistency between the input text and the synthesized image. Typical methods, which seek to model the text-to-image mapping directly, could only capture keywords in the text that indicates common objects or actions but fail to learn their spatial distribution patterns. An effective way to circumvent this limitation is to generate an image layout as guidance, which is attempted by a few methods. Nevertheless, these methods fail to generate practically effective layouts due to the diversity of input text and object location. In this paper we push for effective modeling in both text-to-layout generation and layout-to-image synthesis. Specifically, we formulate the text-to-layout generation as a sequence-to-sequence modeling task, and build our model upon Transformer to learn the spatial relationships between objects by modeling the sequential dependencies between them. In the stage of layout-to-image synthesis, we focus on learning the textual-visual semantic alignment per object in the layout to precisely incorporate the input text into the layout-to-image synthesizing process. To evaluate the quality of generated layout, we design a new metric specifically, dubbed Layout Quality Score, which considers both the absolute distribution errors of bounding boxes in the layout and the mutual spatial relationships between them. Extensive experiments on three datasets demonstrate the superior performance of our method over state-of-the-art methods on both predicting the layout and synthesizing the image from the given text. © 2022, CC BY.

关键词： Image processing

Shifting More Attention to Breast Lesion Segmentation in Ultrasound Videos

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lin, Junhao Dai, Qian Zhu, Lei Fu, Huazhu Wang, Qiong Li, Weibin Rao, Wenhao Huang, Xiaoyang Wang, Liansheng School of Informatics Xiamen University Xiamen China Guangzhou China Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology Hong Kong Institute of High Performance Computing Agency for Science Technology and Research Singapore Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China School of Medicine Xiamen University Xiamen China

Breast lesion segmentation in ultrasound (US) videos is essential for diagnosing and treating axillary lymph node metastasis. However, the lack of a well-established and large-scale ultrasound video dataset with high-quality annotations has posed a persistent challenge for the research community. To overcome this issue, we meticulously curated a US video breast lesion segmentation dataset comprising 572 videos and 34,300 annotated frames, covering a wide range of realistic clinical scenarios. Furthermore, we propose a novel frequency and localization feature aggregation network (FLA-Net) that learns temporal features from the frequency domain and predicts additional lesion location positions to assist with breast lesion segmentation. We also devise a localization-based contrastive loss to reduce the lesion location distance between neighboring video frames within the same video and enlarge the location distances between frames from different ultrasound videos. Our experiments on our annotated dataset and two public video polyp segmentation datasets demonstrate that our proposed FLA-Net achieves state-of-the-art performance in breast lesion segmentation in US videos and video polyp segmentation while significantly reducing time and space complexity. Our model and dataset are available at https://***/jhlDet/FLA-Net. Copyright © 2023, The Authors. All rights reserved.

关键词： Location

Part-guided Relational Transformers for Fine-grained Visual Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Yifan Li, Jia Chen, Xiaowu Tian, Yonghong The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Department of Computer Science and Technology Peking University Beijing100871 China The Pengcheng Laboratory Shenzhen518055 China

Fine-grained visual recognition is to classify objects with visually similar appearances into subcategories, which has made great progress with the development of deep CNNs. However, handling subtle differences between different subcategories still remains a challenge. In this paper, we propose to solve this issue in one unified framework from two aspects, i.e., constructing feature-level interrelationships, and capturing part-level discriminative features. This framework, namely PArt-guided Relational Transformers (PART), is proposed to learn the discriminative part features with an automatic part discovery module, and to explore the intrinsic correlations with a feature transformation module by adapting the Transformer models from the field of natural language processing. The part discovery module efficiently discovers the discriminative regions which are highly-corresponded to the gradient descent procedure. Then the second feature transformation module builds correlations within the global embedding and multiple part embedding, enhancing spatial interactions among semantic pixels. Moreover, our proposed approach does not rely on additional part branches in the inference time and reaches state-of-the-art performance on 3 widely-used fine-grained object recognition benchmarks. Experimental results and explainable visualizations demonstrate the effectiveness of our proposed approach. Code can be found at https://***/iCVTEAM/PART. Copyright © 2022, The Authors. All rights reserved.

关键词： Embeddings