检索结果-内蒙古大学图书馆

arXiv 2021年

作者： Pavoni, Gaia Ponchio, Federico Corsini, Massimiliano Muntoni, Alessandro Cignoni, Paolo Visual Computing Lab ISTI-CNR Pisa Italy

Fully automatic semantic segmentation of highly specific semantic classes and complex shapes may not meet the accuracy standards demanded by scientists. In such cases, human-centered AI solutions, able to assist operators while preserving human control over complex tasks, are a good trade-off to speed up image labeling while maintaining high accuracy levels. Taglab is an open-source AI-assisted software for annotating large orthoimages which takes advantage of different degrees of automation;it speeds up image annotation from scratch through assisted tools, creates custom fully automatic semantic segmentation models, and, finally, allows the quick edits of automatic predictions. Since the orthoimages analysis applies to several scientific disciplines, Taglab has been designed with a flexible labeling pipeline. We report our results in two different scenarios, marine ecology, and architectural heritage. © 2021, CC BY.

关键词： Open source software

来源：评论

学校读者我要写书评

暂无评论

visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Visual Scan-Path based Data-Augmentation for CNN-based 360-d...

引用

London Imaging Meeting, LIM 2021

作者： Sendjasni, Abderrezzaq Larabi, Mohamed-Chaker Cheikh, Faouzi Alaya CNRS Xlim UMR 7252 Université de Poitiers France NTNU Norwegian Colour and Visual Computing Lab Gjøvik Norway

ISBN: (纸本)9780892083558

360-degree Image quality assessment (IQA) is facing the major challenge of lack of ground-truth databases. This prob lem is accentuated for deep learning based approaches where the performances are as good as the available data. In this context, only two databases are used to train and validate deep learning-based IQA models. To compensate this lack, a data augmentation technique is investigated in this paper. We use vi sual scan-path to increase the learning examples from existing training data. Multiple scan-paths are predicted to account for the diversity of human observers. These scan-paths are then used to select viewports from the spherical representation. The results of the data-augmentation training scheme showed an improve ment over not using it. We also try to answer the question of using the MOS obtained for the 360-degree image as the quality anchor for the whole set of extracted viewports in comparison to 2Dblind quality metrics. The comparison showed the superiority of using the MOS when adopting a patch-based learning. © 2021 Society for Imaging Science and Technology.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Struct2Hair: A hair shape descriptor for hairstyle modeling

Struct2Hair: A hair shape descriptor for hairstyle modeling

引用

作者： Zhang, Wenshu Nie, Yinyu Guo, Shihui Chang, Jian Zhang, Jianjun Tong, Ruofeng Faculty of Arts and Media Norwich University of the Arts Norfolk United Kingdom The Visual Computing & Artificial Intelligence Lab Technical University of Munich München Germany School of Informatics Xiamen University Fujian China National Centre for Computer Animation Bournemouth University Dorset United Kingdom College of Computer Science and Technology Zhejiang University Zhejiang China

In recent years, it becomes possible to extract hair information for hair reconstruction from multiple cameras or monocular camera. Using a single image as the input avoids the high cost setups and complex calibration compared to multiviewed reconstruction. Taking advantage of an extendible hairstyle database, this paper introduced Struct2Hair, a novel single-viewed hair modelling approach by extracting hair shape descriptor (HSD). The HSD is defined as the fundamental structure-aware feature, which is a combination of critical shapes in a hairstyle. A complete dataset of critical hair shapes is constructed from a known database of three-dimensional (3D) hair models. We first analyze the input two-dimensional (2D) image to extract the orientation information and 2D hair sketch automatically. The extracted information is then used to retrieve the corresponding critical shapes with optimization to build the robust HSD. Finally, the HSD constructs a weighted 3D hair orientation field to guide full-head hair model generation. Our method can preserve local geometric features of hair and retain the whole shape of the hairstyle globally owing to the HSD, which will benefit further hair editing and stylization. © 2022 John Wiley & Sons Ltd.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

(Image Present) ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection

arXiv

引用

arXiv 2024年

作者： Sun, Zhihao Jiang, Haoran Chen, Haoran Cao, Yixin Qiu, Xipeng Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China School of Computer Science Fudan University China

(Image Present) Multimodal large language models have unlocked new possibilities for various multimodal tasks. However, their potential in image manipulation detection remains unexplored. When directly applied to the IMD task, M-LLMs often produce reasoning texts that suffer from hallucinations and overthinking. To address this, in this work, we propose ForgerySleuth, which leverages M-LLMs to perform comprehensive clue fusion and generate segmentation outputs indicating specific regions that are tampered with. Moreover, we construct the ForgeryAnalysis dataset through the Chain-of-Clues prompt, which includes analysis and reasoning text to upgrade the image manipulation detection task. A data engine is also introduced to build a larger-scale dataset for the pre-training phase. Our extensive experiments demonstrate the effectiveness of ForgeryAnalysis and show that ForgerySleuth significantly outperforms existing methods in generalization, robustness, and explainability. The resource is available at https://***/sunzhihao18/ForgerySleuth. Copyright © 2024, The Authors. All rights reserved.

关键词： Large datasets

来源：评论

学校读者我要写书评

暂无评论

Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning

Masked Video Distillation: Rethinking Masked Feature Modelin...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Mengchen Liu Lu Yuan Yu–Gang Jiang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing Microsoft Cloud + AI

Benefiting from masked visual modeling, self-supervised video representation learning has achieved remarkable progress. However, existing methods focus on learning representations from scratch through reconstructing low-level features like raw pixel values. In this paper, we propose masked video distillation (MVD), a simple yet effective two-stage masked feature modeling framework for video representation learning: firstly we pretrain an image (or video) model by recovering low-level features of masked patches, then we use the resulting features as targets for masked feature modeling. For the choice of teacher models, we observe that students taught by video teachers perform better on temporally-heavy video tasks, while image teachers transfer stronger spatial representations for spatially-heavy video tasks. visualization analysis also indicates different teachers produce different learned patterns for students. To leverage the advantage of different teachers, we design a spatial-temporal co-teaching method for MVD. Specifically, we distill student models from both video teachers and image teachers by masked feature modeling. Extensive experimental results demonstrate that video transformers pre-trained with spatial-temporal co-teaching outperform models distilled with a single teacher on a multitude of video datasets. Our MVD with vanilla ViT achieves state-of-the-art performance compared with previous methods on several challenging video downstream tasks. For example, with the ViT-Large model, our MVD achieves 86.4% and 76.7% Top-1 accuracy on Kinetics-400 and Something-Something-v2, outperforming VideoMAE by 1.2% and 2.4% respectively. When a larger ViT-Huge model is adopted, MVD achieves the state-of-the-art performance with 77.3% Top-1 accuracy on Something-Something-v2. Code will be available at https://***/ruiwang2021/mvd.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Language models represent beliefs of self and others 24

Language models represent beliefs of self and others

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Wentao Zhu Zhining Zhang Yizhou Wang Center on Frontiers of Computing Studies School of Compter Science Peking University Center on Frontiers of Computing Studies School of Compter Science and Inst. for Artificial Intelligence and Nat'l Eng. Research Center of Visual Technology and Nat'l Key Lab of General Artificial Intelligence Peking University

Understanding and attributing mental states, known as Theory of Mind (ToM), emerges as a fundamental capability for human social reasoning. While Large Language Models (LLMs) appear to possess certain ToM abilities, the mechanisms underlying these capabilities remain elusive. In this study, we discover that it is possible to linearly decode the belief status from the perspectives of various agents through neural activations of language models, indicating the existence of internal representations of self and others' beliefs. By manipulating these representations, we observe dramatic changes in the models' ToM performance, underscoring their pivotal role in the social reasoning process. Additionally, our findings extend to diverse social reasoning tasks that involve different causal inference patterns, suggesting the potential generalizability of these representations.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Survey on Video Diffusion Models

arXiv

引用

arXiv 2023年

作者： Xing, Zhen Feng, Qijun Chen, Haoran Dai, Qi Hu, Han Xu, Hang Wu, Zuxuan Jiang, Yu-Gang Shanghai Collaborative Innovation Center of Intelligent Visual Computing School of Computer Science Fudan University China Microsoft Research Asia China Noah's Ark Lab China

The recent wave of AI-generated content (AIGC) has witnessed substantial success in computer vision, with the diffusion model playing a crucial role in this achievement. Due to their impressive generative capabilities, diffusion models are gradually superseding methods based on GANs and auto-regressive Transformers, demonstrating exceptional performance not only in image generation and editing, but also in the realm of video-related research. However, existing surveys mainly focus on diffusion models in the context of image generation, with few up-to-date reviews on their application in the video domain. To address this gap, this paper presents a comprehensive review of video diffusion models in the AIGC era. Specifically, we begin with a concise introduction to the fundamentals and evolution of diffusion models. Subsequently, we present an overview of research on diffusion models in the video domain, categorizing the work into three key areas: video generation, video editing, and other video understanding tasks. We conduct a thorough review of the literature in these three key areas, including further categorization and practical contributions in the field. Finally, we discuss the challenges faced by research in this domain and outline potential future developmental trends. A comprehensive list of video diffusion models studied in this survey is available at https://***/ChenHsing/Awesome-Video-Diffusion-Models. Copyright © 2023, The Authors. All rights reserved.

关键词： Video analysis

来源：评论

学校读者我要写书评

暂无评论

SVFormer: Semi-supervised Video Transformer for Action Recognition

SVFormer: Semi-supervised Video Transformer for Action Recog...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Zhen Xing Qi Dai Han Hu Jingjing Chen Zuxuan Wu Yu-Gang Jiang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing Microsoft Research Asia

Semi-supervised action recognition is a challenging but critical task due to the high cost of video annotations. Existing approaches mainly use convolutional neural networks, yet current revolutionary vision transformer models have been less explored. In this paper, we investigate the use of transformer models under the SSL setting for action recognition. To this end, we introduce SVFormer, which adopts a steady pseudo-labeling framework (i.e., EMA-Teacher) to cope with unlabeled video samples. While a wide range of data augmentations have been shown effective for semi-supervised image classification, they generally produce limited results for video recognition. We therefore introduce a novel augmentation strategy, Tube Token-Mix, tailored for video data where video clips are mixed via a mask with consistent masked tokens over the temporal axis. In addition, we propose a temporal warping augmentation to cover the complex temporal variation in videos, which stretches selected frames to various temporal durations in the clip. Extensive experiments on three datasets Kinetics-400, UCF-101, and HMDB-51 verify the advantage of SVFormer. In particular, SVFormer outperforms the state-of-the-art by 31.5% with fewer training epochs under the 1% labeling rate of Kinetics-400. Our method can hopefully serve as a strong benchmark and encourage future search on semi-supervised action recognition with Transformer networks. Code is released at https://***/ChenHsing/SVFormer.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Unlocking Textual and visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Image

arXiv

引用

arXiv 2024年

作者： Jiao, Pengkun Zhao, Na Chen, Jingjing Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Singapore University of Technology and Design Singapore

Open-vocabulary 3D object detection (OV-3DDet) aims to localize and recognize both seen and previously unseen object categories within any new 3D scene. While language and vision foundation models have achieved success in handling various open-vocabulary tasks with abundant training data, OV-3DDet faces a significant challenge due to the limited availability of training data. Although some pioneering efforts have integrated vision-language models (VLM) knowledge into OV-3DDet learning, the full potential of these foundational models has yet to be fully exploited. In this paper, we unlock the textual and visual wisdom to tackle the open-vocabulary 3D detection task by leveraging the language and vision foundation models. We leverage a vision foundation model to provide image-wise guidance for discovering novel classes in 3D scenes. Specifically, we utilize a object detection vision foundation model to enable the zero-shot discovery of objects in images, which serves as the initial seeds and filtering guidance to identify novel 3D objects. Additionally, to align the 3D space with the powerful vision-language space, we introduce a hierarchical alignment approach, where the 3D feature space is aligned with the vision-language feature space using a pre-trained VLM at the instance, category, and scene levels. Through extensive experimentation, we demonstrate significant improvements in accuracy and generalization, highlighting the potential of foundation models in advancing open-vocabulary 3D object detection in real-world scenarios. © 2024, CC BY.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

Recognizing Handwritten Mathematical Expressions of Vertical Addition and Subtraction

Recognizing Handwritten Mathematical Expressions of Vertical...

引用

Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI)

作者： Daniel Rosa Filipe R. Cordeiro Ruan Carvalho Everton Souza Sergio Chevtchenko Luiz Rodrigues Marcelo Marinho Thales Vieira Valmir Macario Department of Computing Visual Computing Lab Universidade Federal Rural de Pernambuco (UFRPE) Brazil Centro de Informatica - CIn Universidade Federal de Pernambuco (UFPE) Brazil Center for Excellence in Social Technologies (NEES) Federal University of Alagoas Brazil

Handwritten Mathematical Expression Recognition (HMER) is a challenging task with many educational applications. Recent methods for HMER have been developed for complex mathematical expressions in standard horizontal format. However, solutions for elementary mathematical expression, such as vertical addition and subtraction, have not been explored in the literature. This work proposes a new handwritten elementary mathematical expression dataset composed of addition and sub-traction expressions in a vertical format. We also extended the MNIST dataset to generate artificial images with this structure. Furthermore, we proposed a solution for offline HMER, able to recognize vertical addition and subtraction expressions. Our analysis evaluated the object detection algorithms YOLO_v7, YOLO_v8, YOLO-NAS, NanoDet and FCOS for identifying the mathematical symbols. We also proposed a transcription method to map the bounding boxes from the object detection stage to a mathematical expression in the LaTeX markup sequence. Results show that our approach is efficient, achieving a high expression recognition rate. The code and dataset are available at https://***/Danielgol/HME-VAS.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：