检索结果-内蒙古大学图书馆

View-aware Salient Object Detection for 360° Omnidirectional Image

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wu, Junjie Xia, Changqun Yu, Tianshu Li, Jia The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China Peng Cheng Laboratory Shenzhen518000 China

Image-based salient object detection (ISOD) in 360◦ scenarios is significant for understanding and applying panoramic information. However, research on 360◦ ISOD has not been widely explored due to the lack of large, complex, high-resolution, and well-labeled datasets. Towards this end, we construct a large scale 360◦ ISOD dataset with object-level pixel-wise annotation on equirectangular projection (ERP), which contains rich panoramic scenes with not less than 2K resolution and is the largest dataset for 360◦ ISOD by far to our best knowledge. By observing the data, we find current methods face three significant challenges in panoramic scenarios: diverse distortion degrees, discontinuous edge effects and changeable object scales. Inspired by humans’ observing process, we propose a view-aware salient object detection method based on a Sample Adaptive View Transformer (SAVT) module with two sub-modules to mitigate these issues. Specifically, the sub-module View Transformer (VT) contains three transform branches based on different kinds of transformations to learn various features under different views and heighten the model’s feature toleration of distortion, edge effects and object scales. Moreover, the sub-module Sample Adaptive Fusion (SAF) is to adjust the weights of different transform branches based on various sample features and make transformed enhanced features fuse more appropriately. The benchmark results of 20 state-of-the-art ISOD methods reveal the constructed dataset is very challenging. Moreover, exhaustive experiments verify the proposed approach is practical and outperforms the state-of-the-art methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Large dataset

Reading Chinese in Natural Scenes with a Bag-of-Radicals Prior

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Ongbin Liu, Ingjie Chen, Iaxin Wang, Unhong School of Computer Science and Engineering Beihang University Beijing China State Key Laboratory of Virtual Reality Technology System Beihang University Beijing China

Scene text recognition (STR) on Latin datasets has been extensively studied in recent years, and state-of-the-art (SOTA) models often reach high accuracy. However, the performance on non-Latin transcripts, such as Chinese, is not satisfactory. In this paper, we collect six open-source Chinese STR datasets and evaluate a series of classic methods performing well on Latin datasets, finding a significant performance drop. To improve the performance on Chinese datasets, we propose a novel radical-embedding (RE) representation to utilize the ideographic descriptions of Chinese characters. The ideographic descriptions of Chinese characters are firstly converted to bags of radicals and then fused with learnable character embeddings by a character-vector-fusion-module (CVFM). In addition, we utilize a bag of radicals as supervision signals for multi-task training to improve the ideographic structure perception of our model. Experiments show performance of the model with RE + CVFM + multi-task training is superior compared with the baseline on six Chinese STR datasets. Copyright © 2022, The Authors. All rights reserved.

关键词： Embeddings

mvEchoSeg: One-shot In-context Learning for Multi-view Echocardiography Segmentation

学校读者我要写书评

暂无评论

mvEchoSeg: One-shot In-context Learning for Multi-view Echoc...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Yuting Guo Ya Duan Wenfeng Song Nannan Li Aili Li Shuai Li State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China State Key Laboratory of Robotics Shenyang Institute of Automation Chinese Academy of Sciences University of Chinese Academy of Sciences Shenyang China School of Computer Science Beijing Information Science and Technology University Beijing China School of Information Science and Technology Dalian Maritime University Dalian China China-Japan Friendship Hospital Beijing China

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

Echocardiography is the clinical standard for evaluation of cardiac morphology, and function, and providing hemodynamic parameters in patients with known or suspected heart disease. Due to the diversity and complexity of diagnostic tasks, comprehensive interpretation of echocardiography often requires multi-view imaging for integrated metric analysis. Deep learning methods have become the mainstream approach for echocardiography segmentation. However, achieving segmentation of multi-view echocardiography still requires a substantial amount of annotation. To remedy this, we propose a one-shot in-context learning network mvEchoSeg for multi-view echocardiography segmentation. This network requires only one annotated image for each view. Specifically, we propose a Task Prompt Identifier (TPI) module to identify the task type of the image and allocate the most precise task prompt for it, with minimal adaptation of CLIP and few-shot strategies. Additionally, we leverage a unified In-Context Model (ICM) capable of performing a diverse set of echocardiography segmentation tasks automatically. Using fine-tuning and low-rank adapters improved the performance of the pre-trained model, achieving significant results with minimal training cost. Furthermore, we collect a multi-view echocardiography dataset (MVECD) with 8 views to evaluate our method. The results show an improvement of more than 10% in the DICE score compared to the SOTA foundational medical image segmentation models. To our knowledge, this is the first exploration of a one-shot model for multi-view echocardiography segmentation. Our codes and models are available at https://***/stellating/mvEchoSeg.

关键词： Training Measurement Image segmentation Adaptation models Echocardiography Biological system modeling Morphology Hemodynamics Standards Medical diagnostic imaging

Dual Adaptive Representation Alignment for Cross-domain Few-shot Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhao, Yifan Zhang, Tong Li, Jia Tian, Yonghong The School of Computer Science Peking University Beijing100871 China The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Pengcheng Laboratory Shenzhen518055 China The School of Electronic and Computer Engineering Peking University Shenzhen Granduate School Shenzhen518055 China

Few-shot learning aims to recognize novel queries with limited support samples by learning from base knowledge. Recent progress in this setting assumes that the base knowledge and novel query samples are distributed in the same domains, which are usually infeasible for realistic applications. Toward this issue, we propose to address the cross-domain few-shot learning problem where only extremely few samples are available in target domains. Under this realistic setting, we focus on the fast adaptation capability of meta-learners by proposing an effective dual adaptive representation alignment approach. In our approach, a prototypical feature alignment is first proposed to recalibrate support instances as prototypes and reproject these prototypes with a differentiable closed-form solution. Therefore feature spaces of learned knowledge can be adaptively transformed to query spaces by the cross-instance and cross-prototype relations. Besides the feature alignment, we further present a normalized distribution alignment module, which exploits prior statistics of query samples for solving the covariant shifts among the support and query samples. With these two modules, a progressive meta-learning framework is constructed to perform the fast adaptation with extremely few-shot samples while maintaining its generalization capabilities. Experimental evidence demonstrates our approach achieves new state-of-the-art results on 4 CDFSL benchmarks and 4 fine-grained cross-domain benchmarks. Copyright © 2023, The Authors. All rights reserved.

关键词： Alignment

Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Liu, Chenyang Chen, keyan Zhao, Rui Zou, Zhengxia Shi, Zhenwei The Department of Aerospace Intelligent Science and Technology School of Astronautics The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The Key Laboratory of Spacecraft Design Optimization and Dynamic Simulation Technologies Ministry of Education China Shanghai Artificial Intelligence Laboratory Shanghai200232 China Shen Yuan Honors College of Beihang University Beijing100191 China The Department of Electrical and Computer Engineering National University of Singapore Singapore117583 Singapore

Recently, generative foundation models have significantly advanced large-scale text-driven natural image generation and have become a prominent research trend across various vertical domains. However, in the remote sensing field, there is still a lack of research on large-scale text-to-image (text2image) generation technology. Existing remote sensing image-text datasets are small in scale and confined to specific geographic areas and scene types. Besides, existing text2image methods have struggled to achieve global-scale, multi-resolution controllable, and unbounded image generation. To address these challenges, this paper presents two key contributions: the Git-10M dataset and the Text2Earth foundation model. Git-10M is a global-scale image-text dataset comprising 10.5 million image-text pairs, 5 times larger than the previous largest one. The dataset contains essential resolution information and covers a wide range of geographic scenes and contains essential geospatial metadata, significantly surpassing existing datasets in both size and diversity. Building on Git-10M, we propose Text2Earth, a 1.3 billion parameter generative foundation model based on the diffusion framework to model global-scale remote sensing scenes. Text2Earth integrates a resolution guidance mechanism, enabling users to specify image resolutions. A dynamic condition adaptation strategy is proposed for training and inference to improve image generation quality. Text2Earth not only excels in zero-shot text2image generation but also demonstrates robust generalization and flexibility across multiple tasks, including unbounded scene construction, image editing, and cross-modal image generation. This robust capability surpasses previous models restricted to the basic fixed size and limited scene types. On the previous text2image benchmark dataset, Text2Earth outperfoms previous models with a significant improvement of +26.23 FID and +20.95% Zero-shot Cls-OA metric. Our project page is https://chen-y

关键词： Optical remote sensing

BAT: Mining Binary-API Topic for Multi-service Application Development

学校读者我要写书评

暂无评论

BAT: Mining Binary-API Topic for Multi-service Application D...

International Conference on computer Supported Cooperative Work in Design

作者： Pengfei He Linlin Liu Dianlong You Limin Shen Zhen Chen School of Information Science and Engineering Yanshan University Qinhuangdao China National Science Libraries Chinese Academy of Sciences Beijing China The Key Laboratory for Computer Virtual Technology and System Integration of Hebei Province Qinhuangdao China

With the growing popularity of API-driven multiservice application (mashup) development, the burgeoning web APIs have left developers drowning in the sea of web API selections. Matching developers with the most appropriate APIs is the key to improving user satisfaction and promoting more popular web applications. As a result, more and more researchers pay attention to web API recommender systems based on collaborative filtering. However, employing collaborative filtering to recommend APIs is challenging due to the severe sparsity of mashup and API interactions. To address this problem, we propose a probabilistic generative model, called the Binary-API Topic model (BAT), to parameterize mashups and APIs. Technically, BAT is equipped with a mechanism to extract binary-APIs and predict unknown pairwise interactions. To improve generality and capture more relevance from a limited number of interactions, we learn binary-API topics by directly modeling the generation of API co-occurrence patterns across the repository (all mashup collections from ***). The main advantage of BAT is that it preserves API co-occurrence patterns in model learning and exploits the rich global relevance. Finally, through extensive experiments, we demonstrate that BAT can achieve the highest performance on the sparse real-world data set.

关键词：

Layout-Bridging Text-to-Image Synthesis

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liang, Jiadong Pei, Wenjie Lu, Feng The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Department of Computer Science Harbin Institute of Technology at Shenzhen Shenzhen518057 China

The crux of text-to-image synthesis stems from the difficulty of preserving the cross-modality semantic consistency between the input text and the synthesized image. Typical methods, which seek to model the text-to-image mapping directly, could only capture keywords in the text that indicates common objects or actions but fail to learn their spatial distribution patterns. An effective way to circumvent this limitation is to generate an image layout as guidance, which is attempted by a few methods. Nevertheless, these methods fail to generate practically effective layouts due to the diversity of input text and object location. In this paper we push for effective modeling in both text-to-layout generation and layout-to-image synthesis. Specifically, we formulate the text-to-layout generation as a sequence-to-sequence modeling task, and build our model upon Transformer to learn the spatial relationships between objects by modeling the sequential dependencies between them. In the stage of layout-to-image synthesis, we focus on learning the textual-visual semantic alignment per object in the layout to precisely incorporate the input text into the layout-to-image synthesizing process. To evaluate the quality of generated layout, we design a new metric specifically, dubbed Layout Quality Score, which considers both the absolute distribution errors of bounding boxes in the layout and the mutual spatial relationships between them. Extensive experiments on three datasets demonstrate the superior performance of our method over state-of-the-art methods on both predicting the layout and synthesizing the image from the given text. © 2022, CC BY.

关键词： Image processing

An Effective Two-stage Training Paradigm Detector for Small Dataset

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Zheng Xie, Dong Wang, Hanzhi Tian, Jiang AI Lab Lenovo Research Beijing China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China Department of Computer Science Yale University New HavenCT United States

Learning from the limited amount of labeled data to the pre-train model has always been viewed as a challenging task. In this report, an effective and robust solution, the two-stage training paradigm YOLOv8 detector (TP-YOLOv8), is designed for the object detection track in VIPriors Challenge 2023. First, the backbone of YOLOv8 is pre-trained as the encoder using the masked image modeling technique. Then the detector is fine-tuned with elaborate augmentations. During the test stage, test-time augmentation (TTA) is used to enhance each model, and weighted box fusion (WBF) is implemented to further boost the performance. With the well-designed structure, our approach has achieved 30.4% average precision from 0.50 to 0.95 on the DelftBikes test set, ranking 4th on the leaderboard. © 2023, CC BY.

关键词： Object detection

Part-guided Relational Transformers for Fine-grained Visual Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Yifan Li, Jia Chen, Xiaowu Tian, Yonghong The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Department of Computer Science and Technology Peking University Beijing100871 China The Pengcheng Laboratory Shenzhen518055 China

Fine-grained visual recognition is to classify objects with visually similar appearances into subcategories, which has made great progress with the development of deep CNNs. However, handling subtle differences between different subcategories still remains a challenge. In this paper, we propose to solve this issue in one unified framework from two aspects, i.e., constructing feature-level interrelationships, and capturing part-level discriminative features. This framework, namely PArt-guided Relational Transformers (PART), is proposed to learn the discriminative part features with an automatic part discovery module, and to explore the intrinsic correlations with a feature transformation module by adapting the Transformer models from the field of natural language processing. The part discovery module efficiently discovers the discriminative regions which are highly-corresponded to the gradient descent procedure. Then the second feature transformation module builds correlations within the global embedding and multiple part embedding, enhancing spatial interactions among semantic pixels. Moreover, our proposed approach does not rely on additional part branches in the inference time and reaches state-of-the-art performance on 3 widely-used fine-grained object recognition benchmarks. Experimental results and explainable visualizations demonstrate the effectiveness of our proposed approach. Code can be found at https://***/iCVTEAM/PART. Copyright © 2022, The Authors. All rights reserved.

关键词： Embeddings