检索结果-内蒙古大学图书馆

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Wang, Junke Jiang, Yi Yuan, Zehuan Peng, Binyue Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Bytedance Inc. China

Tokenizer, serving as a translator to map the intricate visual data into a compact latent space, lies at the core of visual generative models. Based on the finding that existing tokenizers are tailored to image or video inputs, this paper presents OmniTokenizer, a transformer-based tokenizer for joint image and video tokenization. OmniTokenizer is designed with a spatial-temporal decoupled architecture, which integrates window and causal attention for spatial and temporal modeling. To exploit the complementary nature of image and video data, we further propose a progressive training strategy, where OmniTokenizer is first trained on image data on a fixed resolution to develop the spatial encoding capacity and then jointly trained on image and video data on multiple resolutions to learn the temporal dynamics. OmniTokenizer, for the first time, handles both image and video inputs within a unified framework and proves the possibility of realizing their synergy. Extensive experiments demonstrate that OmniTokenizer achieves state-of-the-art (SOTA) reconstruction performance on various image and video datasets, e.g., 1.11 reconstruction FID on ImageNet and 42 reconstruction FVD on UCF-101, beating the previous SOTA methods by 13% and 26%, respectively. Additionally, we also show that when integrated with OmniTokenizer, both language model-based approaches and diffusion models can realize advanced visual synthesis performance, underscoring the superiority and versatility of our method. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

HDNet: Physics-Inspired Neural Network for Flow Estimation based on Helmholtz Decomposition

arXiv

引用

arXiv 2024年

作者： Qi, Miao Idoughi, Ramzi Heidrich, Wolfgang Visual Computing Center King Abdullah University of Science and Technology Thuwal Saudi Arabia

Flow estimation problems are ubiquitous in scientific imaging. Often, the underlying flows are subject to physical constraints that can be exploited in the flow estimation;for example, incompressible (divergence-free) flows are expected for many fluid experiments, while irrotational (curl-free) flows arise in the analysis of optical distortions and wavefront sensing. In this work, we propose a Physics-Inspired Neural Network (PINN) named HDNet, which performs a Helmholtz decomposition of an arbitrary flow field, i.e., it decomposes the input flow into a divergence-only and a curl-only component. HDNet can be trained exclusively on synthetic data generated by reverse Helmholtz decomposition, which we call Helmholtz synthesis. As a PINN, HDNet is fully differentiable and can easily be integrated into arbitrary flow estimation problems. © 2024, CC BY.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Urbanflow: Designing Comfortable Outdoor Areas

Urbanflow: Designing Comfortable Outdoor Areas

引用

Annual Modeling and Simulation Conference (ANNSIM)

作者： Daoming Liu Florian Rist Dominik L. Michels KAUST Visual Computing Center Thuwal Kingdom Of Saudi Arabia

Design decisions in urban planning have to be made with particular carefulness as the resulting constraints are binding for the whole architectural design that follows. In this context, investigating and optimizing the airflow in urban environments is critical to design comfortable outdoor areas as unwanted effects such as windy areas and the formation of heat pockets have to be avoided. Our UrbanFlow framework enables interactive architectural design allowing for decision making based on simulating urban flow. Compared to real-time fluid flow simulation, enabling interactive architecture design poses an even higher computational efficiency challenge as evaluating a design by simulation usually requires hundreds of time steps. This is addressed based on a highly efficient Eulerian fluid simulator in which we incorporate a unified porosity model which is devised to encode digital urban models containing objects such as buildings and trees.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Towards Interpretable Defense Against Adversarial Attacks via Causal Inference

引用

Machine Intelligence Research 2022年第3期19卷 209-226页

作者： Min Ren Yun-Long Wang Zhao-Feng He University of Chinese Academy of Sciences Beijing 100190China Center for Research on Intelligent Perception and Computing National Laboratory of Pattern RecognitionInstitute of AutomationChinese Academy of SciencesBeijing 100190China Laboratory of Visual Computing and Intelligent System Beijing University of Posts and TelecommunicationsBeijing 100876China

Deep learning-based models are vulnerable to adversarial attacks. Defense against adversarial attacks is essential for sensitive and safety-critical scenarios. However, deep learning methods still lack effective and efficient defense mechanisms against adversarial attacks. Most of the existing methods are just stopgaps for specific adversarial samples. The main obstacle is that how adversarial samples fool the deep learning models is still unclear. The underlying working mechanism of adversarial samples has not been well explored, and it is the bottleneck of adversarial attack defense. In this paper, we build a causal model to interpret the generation and performance of adversarial samples. The self-attention/transformer is adopted as a powerful tool in this causal model. Compared to existing methods, causality enables us to analyze adversarial samples more naturally and intrinsically. Based on this causal model, the working mechanism of adversarial samples is revealed, and instructive analysis is provided. Then, we propose simple and effective adversarial sample detection and recognition methods according to the revealed working mechanism. The causal insights enable us to detect and recognize adversarial samples without any extra model or training. Extensive experiments are conducted to demonstrate the effectiveness of the proposed methods. Our methods outperform the state-of-the-art defense methods under various adversarial attacks.

关键词： Adversarial sample adversarial defense causal inference interpretable machine learning transformers

来源：评论

学校读者我要写书评

暂无评论

Retrieval Augmented Recipe Generation

Retrieval Augmented Recipe Generation

引用

2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025

作者： Liu, Guoshan Yin, Hailong Zhu, Bin Chen, Jingjing Ngo, Chong-Wah Jiang, Yu-Gang School of Computer Science Fudan University Shanghai Key Lab of Intelligent Information Processing China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Singapore Management University Singapore

ISBN: (纸本)9798331510831

The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method - first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs), which have achieved notable success across a variety of vision and language tasks, shed light on generating both ingredients and instructions directly from images. Nevertheless, LMMs still face the common issue of hallu-cinations during recipe generation, leading to suboptimal performance. To tackle this issue, we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement, integrating them into the prompt to add diverse and rich context to the input image. Additionally, Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates, which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method, which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset. © 2025 IEEE.

关键词： Food ingredients

来源：评论

学校读者我要写书评

暂无评论

SCTAP: Supporting Scenario-Centric Trigger-Action Programming based on Software-Defined Physical Environments 23

SCTAP: Supporting Scenario-Centric Trigger-Action Programmin...

引用

2023 World Wide Web Conference, WWW 2023

作者： Sun, Bingkun Shen, Liwei Peng, Xin Wang, Ziming Fudan University China School of Computer Science Shanghai Key Laboratory of Data Science Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China

ISBN: (纸本)9781450394161

The physical world we live in is accelerating digitalization with the vigorous development of Internet of Things (IoT). Following this trend, Web of Things (WoT) further enables fast and efficient creation of various applications that perceive and act on the physical world using standard Web technologies. A popular way for creating WoT applications is Trigger-Action Programming (TAP), which allows users to orchestrate the capabilities of IoT devices in the form of "if trigger, then action". However, existing TAP approaches don't support scenario-centric WoT applications which involve abstract modeling of physical environments and complex spatio-temporal dependencies between events and actions. In this paper, we propose an approach called SCTAP which supports Scenario-Centric Trigger-Action Programming based on software-defined physical environments. SCTAP defines a structured and conceptual representation for physical environments, which provides the required programming abstractions for WoT applications. Based on the representation, SCTAP defines a grammar for specifying scenario-centric WoT applications with spatio-temporal dependencies. Furthermore, we design a service-based architecture for SCTAP which supports the integration of device access, event perception, environment representation, and rule execution in a loosely-coupled and extensible way. We implement SCTAP as a WoT infrastructure and evaluate it with two case studies including a smart laboratory and a smart coffee house. The results confirm the usability, feasibility and efficiency of SCTAP and its implementation. © 2023 ACM.

关键词： Internet of things

来源：评论

学校读者我要写书评

暂无评论

TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition 32

TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text R...

引用

32nd International Joint Conference on Artificial Intelligence, IJCAI 2023

作者： Zheng, Tianlun Chen, Zhineng Bai, Jinfeng Xie, Hongtao Jiang, Yu-Gang Shanghai Collaborative Innovation Center of Intelligent Visual Computing School of Computer Science Fudan University China Tomorrow Advance Life China University of Science and Technology of China China

ISBN: (纸本)9781956792034

Text irregularities pose significant challenges to scene text recognizers. Thin-Plate Spline (TPS)based rectification is widely regarded as an effective means to deal with them. Currently, the calculation of TPS transformation parameters purely depends on the quality of regressed text borders. It ignores the text content and often leads to unsatisfactory rectified results for severely distorted text. In this work, we introduce TPS++, an attention-enhanced TPS transformation that incorporates the attention mechanism to text rectification for the first time. TPS++ formulates the parameter calculation as a joint process of foreground control point regression and content-based attention score estimation, which is computed by a dedicated designed gated-attention block. TPS++ builds a more flexible content-aware rectifier, generating a natural text correction that is easier to read by the subsequent recognizer. Moreover, TPS++ shares the feature backbone with the recognizer in part and implements the rectification at feature-level rather than image-level, incurring only a small overhead in terms of parameters and inference time. Experiments on public benchmarks show that TPS++ consistently improves the recognition and achieves state-of-the-art accuracy. Meanwhile, it generalizes well on different backbones and recognizers. Code is at https://***/simplify23/TPS PP. © 2023 International Joint Conferences on Artificial Intelligence. All rights reserved.

关键词： Character recognition

来源：评论

学校读者我要写书评

暂无评论

Making Large Language Models Better Reasoners with Orchestrated Streaming Experiences

arXiv

引用

arXiv 2025年

作者： Liu, Xiangyang He, Junliang Qiu, Xipeng School of Computer Science Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing China

Large language models (LLMs) can perform complex reasoning by generating intermediate thoughts under zero-shot or few-shot settings. However, zero-shot prompting always encounters low performance, and the superior performance of few-shot prompting hinges on the manual-crafted demonstrations. In this paper, we present RoSE (Reasoning with Orchestrated Streaming Experiences), a general framework for solving reasoning tasks that can self-improve without complex external efforts. To enable RoSE, we describe an architecture that extends an LLM to store all answered questions and their thoughts in a streaming experience pool then orchestrates helpful questions from the pool to assist in answering new questions. To set up a question-aware orchestration mechanism, RoSE first calculates the similarity of each question in the pool with a new test question. Since the solution to each answered question is not always correct, RoSE will sort the questions according to their similarity with the new question, and then uniformly divide them into multiple buckets. It finally extracts one question from each bucket to make these extracted questions more diverse. To make these extracted questions help RoSE answer new questions as much as possible, we introduce two other attributes of uncertainty and complexity for each question. RoSE will preferentially select the questions with low uncertainty and high complexity from each bucket. We evaluate the versatility of RoSE in various reasoning tasks, LLMs, and CoT methods. Copyright © 2025, The Authors. All rights reserved.

关键词： Complex networks

来源：评论

学校读者我要写书评

暂无评论

Sparse graphical linear dynamical systems

The Journal of Machine Learning Research

引用

The Journal of Machine Learning Research 2024年第1期25卷 10882-10934页

作者： Emilie Chouzenoux Víctor Elvira Center for Visual Computing Inria University Paris Saclay Gif-sur-Yvette France School of Mathematics University of Edinburgh Edinburgh UK

Time-series datasets are central in machine learning with applications in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Learning the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on various synthetic data showcases the effectiveness of the proposed model and inference algorithm. This work will significantly contribute to the understanding and utilization of time-series data in diverse scientific and engineering applications where incorporating a graphical approach is essential to perform the i

关键词： state-space models graph inference sparsity graphical lasso majorization-minimization proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

GenRec: Unifying Video Generation and Recognition with Diffusion Models 38

GenRec: Unifying Video Generation and Recognition with Diffu...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Weng, Zejia Yang, Xitong Xing, Zhen Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Department of Computer Science University of Maryland United States

Video diffusion models are able to generate high-quality videos by learning strong spatial-temporal priors on large-scale datasets. In this paper, we aim to investigate whether such priors derived from a generative process are suitable for video recognition, and eventually joint optimization of generation and recognition. Building upon Stable Video Diffusion, we introduce GenRec, the first unified framework trained with a random-frame conditioning process so as to learn generalized spatial-temporal representations. The resulting framework can naturally supports generation and recognition, and more importantly is robust even when visual inputs contain limited information. Extensive experiments demonstrate the efficacy of GenRec for both recognition and generation. In particular, GenRec achieves competitive recognition performance, offering 75.8% and 87.2% accuracy on SSV2 and K400, respectively. GenRec also performs the best on class-conditioned image-to-video generation, achieving 46.5 and 49.3 FVD scores on SSV2 and EK-100 datasets. Furthermore, GenRec demonstrates extraordinary robustness in scenarios that only limited frames can be observed. Code will be available at https://***/wengzejia1/GenRec. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：