检索结果-内蒙古大学图书馆

SGLoc: Scene Geometry Encoding for Outdoor LiDAR Localization

学校读者我要写书评

暂无评论

SGLoc: Scene Geometry Encoding for Outdoor LiDAR Localizatio...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Wen Li Shangshu Yu Cheng Wang Guosheng Hu Siqi Shen Chenglu Wen Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University China Oosto Belfast UK

LiDAR-based absolute pose regression estimates the global pose through a deep network in an end-to-end manner, achieving impressive results in learning-based localization. However, the accuracy of existing methods still has room to improve due to the difficulty of effectively encoding the scene geometry and the unsatisfactory quality of the data. In this work, we propose a novel LiDAR localization frame-work, SGLoc, which decouples the pose estimation to point cloud correspondence regression and pose estimation via this correspondence. This decoupling effectively encodes the scene geometry because the decoupled correspondence regression step greatly preserves the scene geometry, leading to significant performance improvement. Apart from this decoupling, we also design a tri-scale spatial feature aggregation module and inter-geometric consistency constraint loss to effectively capture scene geometry. Moreover, we empirically find that the ground truth might be noisy due to GPS/INS measuring errors, greatly reducing the pose estimation performance. Thus, we propose a pose quality evaluation and enhancement method to measure and correct the ground truth pose. Extensive experiments on the Oxford Radar RobotCar and NCLT datasets demonstrate the effectiveness of SGLoc, which outperforms state-of-the-art regression-based localization methods by 68.5% and 67.6% on position accuracy, respectively.

关键词：

Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Luo, Gen Zhou, Yiyi Zhang, Yuxin Zheng, Xiawu Sun, Xiaoshuai Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University 361005 China Peng Cheng Laboratory Shenzhen518000 China Institute of Artificial Intelligence Xiamen University 361005 China

Despite remarkable progress, existing multimodal large language models (MLLMs) are still inferior in granular visual recognition. Contrary to previous works, we study this problem from the perspective of image resolution, and reveal that a combination of low- and high-resolution visual features can effectively mitigate this shortcoming. Based on this observation, we propose a novel and efficient method for MLLMs, termed Mixture-of-Resolution Adaptation (MRA). In particular, MRA adopts two visual pathways for images with different resolutions, where high-resolution visual information is embedded into the low-resolution pathway via the novel mixture-of-resolution adapters (MR-Adapters). This design also greatly reduces the input sequence length of MLLMs. To validate MRA, we apply it to a recent MLLM called LLaVA, and term the new model LLaVA-HR. We conduct extensive experiments on 11 vision-language (VL) tasks, which show that LLaVA-HR outperforms existing MLLMs on 8 VL tasks, e.g., +9.4% on TextVQA. More importantly, both training and inference of LLaVA-HR remain efficient with MRA, e.g., 20 training hours and 3× inference speed than LLaVA-1.5. Source codes are released at: https://***/luogen1996/LLaVA-HR. Copyright © 2024, The Authors. All rights reserved.

关键词： Image resolution

Energy-Limited UAV Visiting Planning for Age-Aware Wireless-Powered Sensor Networks

学校读者我要写书评

暂无评论

Energy-Limited UAV Visiting Planning for Age-Aware Wireless-...

IEEE Conference on Vehicular Technology (VTC)

作者： Hanbin Hong Yi Zhang Yajing Xie Department of Information and Communication Engineering School of Informatics Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China

Unmanned aerial vehicle (UAV) has revealed its great advantage to provide efficient data collection and wireless charging services to wireless-power sensor networks (WPSNs). Note that the UAV usually is equipped with limited battery power and therefore may not have enough energy to visit all sensor nodes (SNs) during a flight. However, the energy consumption of the UAV is rarely discussed in UAV-assisted WPSNs. In this paper, we study visiting planning problem in the UAV-assisted WPSN by considering both the energy limitation of the UAV and the age of information (AoI) of data collection. Specifically, we introduce a mixed data collection strategy to reduce the AoI of the collected data and improve the energy efficiency of the UAV during each flight. The formulated AoI-aware problem, which aims to minimize the average AoI of the collected data and meanwhile maximize the number of visiting SNs as well as the amount of data, is further tackled by utilizing the reformulation-linearization-technique (RLT) and alternating direction method of multipliers (ADMM). The results show the proposed ADMM-based algorithm can outperform other approaches in terms of system objective and energy efficiency.

关键词：

AnySR: Realizing Image Super-Resolution as Any-Scale, Any-Resource

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhan, Wengyi Lin, Mingbao Lin, Chia-Wen Ji, Rongrong The Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China The Skywork AI Singapore The Department of Electrical Engineering Institute of Communications Engineering National Tsing Hua University Taiwan

In an effort to improve the efficiency and scalability of single-image super-resolution (SISR) applications, we introduce AnySR, to rebuild existing arbitrary-scale SR methods into any-scale, any-resource implementation. As a contrast to off-the-shelf methods that solve SR tasks across various scales with the same computing costs, our AnySR innovates in: 1) building arbitrary-scale tasks as any-resource implementation, reducing resource requirements for smaller scales without additional parameters;2) enhancing any-scale performance in a feature-interweaving fashion, inserting scale pairs into features at regular intervals and ensuring correct feature/scale processing. The efficacy of our AnySR is fully demonstrated by rebuilding most existing arbitrary-scale SISR methods and validating on five popular SISR test datasets. The results show that our AnySR implements SISR tasks in a computing-more-efficient fashion, and performs on par with existing arbitrary-scale SISR methods. For the first time, we realize SISR tasks as not only any-scale in literature, but also as any-resource. Our code is available at https://***/CrispyFeSo4/AnySR. Copyright © 2024, The Authors. All rights reserved.

关键词： HTTP

MMAPS: End-to-End Multi-Grained Multi-Modal Attribute-Aware Product Summarization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Tao Lin, Ze Li, Hui Ji, Jiayi Zhou, Yiyi Li, Guanbin Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China School of Computer Science and Engineering Sun Yat-sen University China

Given the long textual product information and the product image, Multi-modal Product Summarization (MPS) aims to increase customers' desire to purchase by highlighting product characteristics with a short textual summary. Existing MPS methods can produce promising results. Nevertheless, they still 1) lack end-to-end product summarization, 2) lack multi-grained multi-modal modeling, and 3) lack multi-modal attribute modeling. To improve MPS, we propose an end-to-end multi-grained multi-modal attribute-aware product summarization method (MMAPS) for generating high-quality product summaries in e-commerce. MMAPS jointly models product attributes and generates product summaries. We design several multi-grained multi-modal tasks to better guide the multi-modal learning of MMAPS. Furthermore, we model product attributes based on both text and image modalities so that multi-modal product characteristics can be manifested in the generated summaries. Extensive experiments on a real large-scale Chinese e-commence dataset demonstrate that our model outperforms state-of-the-art product summarization methods w.r.t. several summarization metrics. Our code is publicly available at: https://***/KDEGroup/MMAPS. Copyright © 2023, The Authors. All rights reserved.

关键词： Large datasets

CLIP2UDA: Making Frozen CLIP Reward Unsupervised Domain Adaptation in 3D Semantic Segmentation 24

学校读者我要写书评

暂无评论

CLIP2UDA: Making Frozen CLIP Reward Unsupervised Domain Adap...

32nd ACM International Conference on multimedia, MM 2024

作者： Wu, Yao Xing, Mingwei Zhang, Yachao Xie, Yuan Qu, Yanyun School of Informatics Xiamen University Xiamen China Institute of Artificial Intelligence Xiamen University Xiamen China Tsinghua Shenzhen International Graduate School Tsinghua University Shenzhen China East China Normal University Shanghai China Chongqing Institute of East China Normal University Chongqing China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University Xiamen China

ISBN: (纸本)9798400706868

Multi-modal Unsupervised Domain Adaptation (MM-UDA) for large-scale 3D semantic segmentation involves adapting 2D and 3D models to a target domain without labels, which significantly reduces the labor-intensive annotations. Existing MM-UDA methods have often attempted to mitigate the domain discrepancy by aligning features between the source and target data. However, this implementation falls short when applied to image perception due to the susceptibility of images to environmental changes compared to point clouds. To mitigate this limitation, in this work, we explore the potentials of an off-the-shelf Contrastive Language-Image Pre-training (CLIP) model with rich whilst heterogeneous knowledge. To make CLIP task-specific, we propose a top-performing method, dubbed CLIP2UDA, which makes frozen CLIP reward unsupervised domain adaptation in 3D semantic segmentation. Specifically, CLIP2UDA alternates between two steps during adaptation: (a) Learning task-specific prompt. 2D features response from the visual encoder are employed to initiate the learning of adaptive text prompt of each domain, and (b) Learning multi-modal domain-invariant representations. These representations interact hierarchically in the shared decoder to obtain unified 2D visual predictions. This enhancement allows for effective alignment between the modality-specific 3D and unified feature space via cross-modal mutual learning. Extensive experimental results demonstrate that our method outperforms state-of-the-art competitors in several widely-recognized adaptation scenarios. Code is available at: https://***/Barcaaaa/CLIP2UDA. © 2024 ACM.

关键词： Semantic Segmentation

Federated Learning with Domain Shift Eraser

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Zheng Wang, Zihui Wang, Zheng Fan, Xiaoliang Wang, Cheng Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University China Shanghai Innovation Institution China School of Informatic Xiamen University China Peng Cheng Laboratory Shenzhen China

Federated learning (FL) is emerging as a promising technique for collaborative learning without local data leaving their devices. However, clients’ data originating from diverse domains may degrade model performance due to domain shifts, preventing the model from learning consistent representation space. In this paper, we propose a novel FL framework, Federated Domain Shift Eraser (FDSE), to improve model performance by differently erasing each client’s domain skew and enhancing their consensus. First, we formulate the model forward passing as an iterative deskewing process that extracts and then deskews features alternatively. This is efficiently achieved by decomposing each original layer in the neural network into a Domain-agnostic Feature Extractor (DFE) and a Domain-specific Skew Eraser (DSE). Then, a regularization term is applied to promise the effectiveness of feature deskewing by pulling local statistics of DSE’s outputs close to the globally consistent ones. Finally, DFE modules are fairly aggregated and broadcast to all the clients to maximize their consensus, and DSE modules are personalized for each client via similarity-aware aggregation to erase their domain skew differently. Comprehensive experiments were conducted on three datasets to confirm the advantages of our method in terms of accuracy, efficiency, and generalizability. © 2025, CC BY.

关键词： Federated learning

DiffRate : Differentiable Compression Rate for efficient Vision Transformers

学校读者我要写书评

暂无评论

DiffRate : Differentiable Compression Rate for Efficient Vis...

International Conference on Computer Vision (ICCV)

作者： Mengzhao Chen Wenqi Shao Peng Xu Mingbao Lin Kaipeng Zhang Fei Chao Rongrong Ji Yu Qiao Ping Luo Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University OpenGVLab Shanghai AI Laboratory The University of Hong Kong Tencent Holdings Ltd

Token compression aims to speed up large-scale vision transformers (e.g. ViTs) by pruning (dropping) or merging tokens. It is an important but challenging task. Although recent advanced approaches achieved great success, they need to carefully handcraft a compression rate (i.e. number of tokens to remove), which is tedious and leads to sub-optimal performance. To tackle this problem, we propose Differentiable Compression Rate (DiffRate), a novel token compression method that has several appealing properties prior arts do not have. First, DiffRate enables propagating the loss function’s gradient onto the compression ratio, which is considered as a non-differentiable hyperparameter in previous work. In this case, different layers can automatically learn different compression rates layer-wisely without extra overhead. Second, token pruning and merging can be naturally performed simultaneously in DiffRate, while they were isolated in previous works. Third, extensive experiments demonstrate that DiffRate achieves state-of-the-art performance. For example, by applying the learned layer-wise compression rates to an off-the-shelf ViT-H (MAE) model, we achieve a 40% FLOPs reduction and a 1.5× throughput improvement, with a minor accuracy drop of 0.16% on ImageNet without fine-tuning, even outperforming previous methods with fine-tuning. Codes and models are available at https://***/OpenGVLab/DiffRate.

关键词：

EOV-Seg: efficient Open-Vocabulary Panoptic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Niu, Hongwei Hu, Jie Lin, Jianghang Jiang, Guannan Zhang, Shengchuan Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University 361005 China Institute of Artificial Intelligence Xiamen University Fujian China National University of Singapore Singapore Fujian China

Open-vocabulary panoptic segmentation aims to segment and classify everything in diverse scenes across an unbounded vocabulary. Existing methods typically employ two-stage or single-stage framework. The two-stage framework involves cropping the image multiple times using masks generated by a mask generator, followed by feature extraction, while the single-stage framework relies on a heavyweight mask decoder to make up for the lack of spatial position information through self-attention and cross-attention in multiple stacked Transformer blocks. Both methods incur substantial computational overhead, thereby hindering the efficiency of model inference. To fill the gap in efficiency, we propose EOV-Seg, a novel single-stage, shared, efficient, and spatial-aware framework designed for open-vocabulary panoptic segmentation. Specifically, EOV-Seg innovates in two aspects. First, a Vocabulary-Aware Selection (VAS) module is proposed to improve the semantic comprehension of visual aggregated features and alleviate the feature interaction burden on the mask decoder. Second, we introduce a Two-way Dynamic Embedding Experts (TDEE), which efficiently utilizes the spatial awareness capabilities of ViT-based CLIP backbone. To the best of our knowledge, EOV-Seg is the first open-vocabulary panoptic segmentation framework towards efficiency, which runs faster and achieves competitive performance compared with state-of-the-art methods. Specifically, with COCO training only, EOV-Seg achieves 24.5 PQ, 32.1 mIoU, and 11.6 FPS on the ADE20K dataset and the inference time of EOV-Seg is 4-19 times faster than state-of-the-art methods. Especially, equipped with ResNet50 backbone, EOV-Seg runs 23.8 FPS with only 71M parameters on a single RTX 3090 GPU. Code is available at https://***/nhw649/EOV-Seg. Copyright © 2024, The Authors. All rights reserved.

关键词： Decoding