检索结果-内蒙古大学图书馆

Synthesizing Knowledge-enhanced Features for Real-world Zero-shot Food Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Pengfei Min, Weiqing Song, Jiajun Zhang, Yang Jiang, Shuqiang The Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China The University of Chinese Academy of Sciences Beijing100049 China The Institute of Intelligent Computing Technology Chinese Academy of Sciences Suzhou215124 China

Food computing brings various perspectives to computer vision like vision-based food analysis for nutrition and health. As a fundamental task in food computing, food detection needs Zero-Shot Detection (ZSD) on novel unseen food objects to support real-world scenarios, such as intelligent kitchens and smart restaurants. Therefore, we first benchmark the task of Zero-Shot Food Detection (ZSFD) by introducing FOWA dataset with rich attribute annotations. Unlike ZSD, fine-grained problems in ZSFD like inter-class similarity make synthesized features inseparable. The complexity of food semantic attributes further makes it more difficult for current ZSD methods to distinguish various food categories. To address these problems, we propose a novel framework ZSFDet to tackle fine-grained problems by exploiting the interaction between complex attributes. Specifically, we model the correlation between food categories and attributes in ZSFDet by multi-source graphs to provide prior knowledge for distinguishing fine-grained features. Within ZSFDet, Knowledge-Enhanced Feature Synthesizer (KEFS) learns knowledge representation from multiple sources (e.g., ingredients correlation from knowledge graph) via the multi-source graph fusion. Conditioned on the fusion of semantic knowledge representation, the region feature diffusion model in KEFS can generate fine-grained features for training the effective zero-shot detector. Extensive evaluations demonstrate the superior performance of our method ZSFDet on FOWA and the widely-used food dataset UECFOOD-256, with significant improvements by 1.8% and 3.7% ZSD mAP compared with the strong baseline RRFS. Further experiments on PASCAL VOC and MS COCO prove that enhancement of the semantic knowledge can also improve the performance on general ZSD. Code and dataset are availab.e at https://***/LanceZPF/KEFS. Copyright © 2024, The Authors. All rights reserved.

关键词： Object detection

Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhou, Hanyu Wang, Haonan Liu, Haoyue Duan, Yuxing Chang, Yi Yan, Luxin National Key Lab of Multispectral Information Intelligent Processing Technology School of Artificial Intelligence and Automation Huazhong University of Science and Technology China School of Computing National University of Singapore Singapore

High-dynamic scene optical flow is a challenging task, which suffers spatial blur and temporal discontinuous motion due to large displacement in frame imaging, thus deteriorating the spatiotemporal feature of optical flow. Typically, existing methods mainly introduce event camera to directly fuse the spatiotemporal features between the two modalities. However, this direct fusion is ineffective, since there exists a large gap due to the heterogeneous data representation between frame and event modalities. To address this issue, we explore a common-latent space as an intermediate bridge to mitigate the modality gap. In this work, we propose a novel common spatiotemporal fusion between frame and event modalities for high-dynamic scene optical flow, including visual boundary localization and motion correlation fusion. Specifically, in visual boundary localization, we figure out that frame and event share the similar spatiotemporal gradients, whose similarity distribution is consistent with the extracted boundary distribution. This motivates us to design the common spatiotemporal gradient to constrain the reference boundary localization. In motion correlation fusion, we discover that the frame-based motion possesses spatially dense but temporally discontinuous correlation, while the event-based motion has spatially sparse but temporally continuous correlation. This inspires us to use the reference boundary to guide the complementary motion knowledge fusion between the two modalities. Moreover, common spatiotemporal fusion can not only relieve the cross-modal feature discrepancy, but also make the fusion process interpretable for dense and continuous optical flow. Extensive experiments have been performed to verify the superiority of the proposed method. Copyright © 2025, The Authors. All rights reserved.

关键词： Optical flows

Fan-Net: Fourier-Based Adaptive Normalization for Cross-Domain Stroke Lesion Segmentation 48

学校读者我要写书评

暂无评论

Fan-Net: Fourier-Based Adaptive Normalization for Cross-Doma...

48th IEEE International Conference on Acoustics, Speech and Signal processing, ICASSP 2023

作者： Yu, Weiyi Lei, Yiming Shan, Hongming Fudan University School of Computer Science Shanghai Key Lab of Intelligent Information Processing Shanghai200433 China Institute of Science and Technology for Brain-Inspired Intelligence China Shanghai Center for Brain Science and Brain-Inspired Technology Shanghai201210 China

ISBN: (纸本)9781728163277

Since stroke is the main cause of various cerebrovascular diseases, deep learning-based stroke lesion segmentation on magnetic resonance (MR) images has attracted considerable attention. However, the existing methods often neglect the domain shift among MR images collected from different sites, which has limited performance improvement. To address this problem, we intend to change style information without affecting high-level semantics via adaptively changing the low-frequency amplitude components of the Fourier transform so as to enhance model robustness to varying domains. Thus, we propose a novel FAN-Net, a U-Net-based segmentation network incorporated with a Fourier-based adaptive normalization (FAN) and a domain classifier with a gradient reversal layer. The FAN module is tailored for learning adaptive affine parameters for the amplitude components of different domains, which can dynamically normalize the style information of source images. Then, the domain classifier provides domain-agnostic knowledge to endow FAN with strong domain generalizability. The experimental results on the ATLAS dataset, which consists of MR images from 9 sites, show the superior performance of the proposed FAN-Net compared with baseline methods. © 2023 IEEE.

关键词： Magnetic resonance

Combating Online Misinformation Videos: Characterization, Detection, and Future Directions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Bu, Yuyan Sheng, Qiang Cao, Juan Qi, Peng Wang, Danding Li, Jintao Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences China Institute of Computing Technology Chinese Academy of Sciences China Key Lab of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China

With information consumption via online video streaming becoming increasingly popular, misinformation video poses a new threat to the health of the online information ecosystem. Though previous studies have made much progress in detecting misinformation in text and image formats, video-based misinformation brings new and unique challenges to automatic detection systems: 1) high information heterogeneity brought by various modalities, 2) blurred distinction between misleading video manipulation and nonmalicious artistic video editing, and 3) new patterns of misinformation propagation due to the dominant role of recommendation systems on online video platforms. To facilitate research on this challenging task, we conduct this survey to present advances in misinformation video detection. We first analyze and characterize the misinformation video from three levels including signals, semantics, and intents. Based on the characterization, we systematically review existing works for detection from features of various modalities to techniques for clue integration. We also introduce existing resources including representative datasets and useful tools. Besides summarizing existing studies, we discuss related areas and outline open issues and future directions to encourage and guide more research on misinformation video detection. The corresponding repository is at https://***/ICTMCG/Awesome-Misinfo-Video-Detection. © 2023, CC BY-NC-SA.

关键词： Semantics

HD-KT: Advancing Robust Knowledge Tracing via Anomalous Learning Interaction Detection 24

学校读者我要写书评

暂无评论

HD-KT: Advancing Robust Knowledge Tracing via Anomalous Lear...

33rd ACM Web Conference, WWW 2024

作者： Ma, Haiping Yang, Yong Qin, Chuan Yu, Xiaoshan Yang, Shangshang Zhang, Xingyi Zhu, Hengshu Information Materials and Intelligent Sensing Laboratory of Anhui Province Institutes of Physical Science and Information Technology Anhui University Anhui Hefei China Institutes of Physical Science and Information Technology Anhui University Anhui Hefei China Career Science Lab BOSS Zhipin PBC School of Finance Tsinghua University Beijing China School of Artificial Intelligence Anhui University Anhui Hefei China The Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education School of Computer Science and Technology Anhui University Anhui Hefei China Career Science Lab BOSS Zhipin Beijing China

ISBN: (纸本)9798400701719

Knowledge tracing (KT) is a crucial task in online learning, aimed at tracing and predicting each student's knowledge states throughout their learning process. Over the past decade, it has garnered widespread attention due to it provides the potential for more tailored and adaptive online learning experiences. Although most current KT methodologies emphasize optimizing network structures to enhance predictive accuracy for future student performance, they often neglect anomalous interactions in students' learning processes, which may arise from low data quality (i.e., inferior question quality) and abnormal student behaviors (i.e., guessing and mistakes). To this end, in this paper, we propose a novel framework, termed HD-KT, designed to enhance the robustness of existing KT methodologies with Hybrid learning interactions Denoising approach. Specifically, we introduce two detectors for anomalous learning interactions, namely knowledge state-guided anomaly detector and student profile-guided anomaly detector. In the first detection module, we design a sequential autoencoder to identify anomalous learning interactions by detecting atypical student knowledge states. In the second module, we incorporate an attention mechanism by modeling a student's long-term profile to capture irregular interactions. Extensive experiments on four real-world benchmark datasets have decisively shown our HD-KT markedly boosts the robustness of numerous prevailing KT models, consequently increasing the accuracy of future student performance predictions. Additionally, our case studies highlight the versatility of HD-KT in addressing diverse downstream tasks, such as exercise quality analysis and learning behavior-based student clustering. © 2024 ACM.

关键词： Students

Rich-text document styling restoration via reinforcement learning

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2021年第4期15卷 93-103页

作者： Hongwei LI Yingpeng HU Yixuan CAO Ganbin ZHOU Ping LUO Key Lab of Intelligent Information Processing of Chinese Academy of Sciences(CAS) Institute of Computing TechnologyCASBeijing 100190China University of Chinese Academy of Sciences Beijing 100049China Search Product Center WeChat Search Application DepartmentTencentBeijing 100080China

Richly formatted documents,such as financial disclosures,scientific articles,government regulations,widely exist on ***,since most of these documents are only for public reading,the styling information inside them is usually missing,making them improper or even burdensome to be displayed and edited in different formats and *** this study we formulate the task of document styling restoration as an optimization problem,which aims to identify the styling settings on the document elements,e.g.,lines,table cells,text,so that rendering with the output styling settings results in a document,where each element inside it holds the(closely)exact position with the one in the original *** that each styling setting is a decision,this problem can be transformed as a multi-step decision-making task over all the document elements,and then be solved by reinforcement ***,Monte-Carlo Tree Search(MCTS)is leveraged to explore the different styling settings,and the policy function is learnt under the supervision of the delayed *** a case study,we restore the styling information inside tables,where structural and functional data in the documents are usually *** shows that,our best reinforcement method successfully restores the stylings in 87.65%of the tables,with 25.75%absolute improvement over the *** also discuss the tradeoff between the inference time and restoration success rate,and argue that although the reinforcement methods cannot be used in real-time scenarios,it is suitable for the offline tasks with high-quality ***,this model has been applied in a PDF parser to support cross-format display.

关键词： styling restoration monte-carlo tree search reinforcement learning richly formatted documents tables

Layout-based Causal Inference for Object Navigation

学校读者我要写书评

暂无评论

Layout-based Causal Inference for Object Navigation

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Sixian Zhang Xinhang Song Weijie Li Yubing Bai Xinyao Yu Shuqiang Jiang Key Lab of Intelligent Information Processing Laboratory of the Chinese Academy of Sciences (CAS) Institute of Computing Technology Beijing University of Chinese Academy of Sciences Beijing

Previous works for ObjectNav task attempt to learn the association (e.g. relation graph) between the visual inputs and the goal during training. Such association contains the prior knowledge of navigating in training environments, which is denoted as the experience. The experience performs a positive effect on helping the agent infer the likely location of the goal when the layout gap between the unseen environments of the test and the prior knowledge obtained in training is minor. However, when the layout gap is significant, the experience exerts a negative effect on navigation. Motivated by keeping the positive effect and removing the negative effect of the experience, we propose the layout-based soft Total Direct Effect (L-sTDE) framework based on the causal inference to adjust the prediction of the navigation policy. In particular, we propose to calculate the layout gap which is defined as the KL divergence between the posterior and the prior distribution of the object layout. Then the sTDE is proposed to appropriately control the effect of the experience based on the layout gap. Experimental results on AI2THOR, RoboTHOR, and Habitat demonstrate the effectiveness of our method. The code is availab.e at https://***/sx-zhang/***.

关键词：

Bi-Level Meta-Learning for Few-Shot Domain Generalization

学校读者我要写书评

暂无评论

Bi-Level Meta-Learning for Few-Shot Domain Generalization

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Xiaorong Qin Xinhang Song Shuqiang Jiang Key Lab of Intelligent Information Processing Laboratory of the Chinese Academy of Sciences (CAS) Institute of Computing Technology Beijing University of Chinese Academy of Sciences Beijing

The goal of few-shot learning is to learn the generalization from seen to unseen data with only a few samples. Most previous few-shot learning methods focus on learning the generalization within particular domains. However, the more practical scenarios may also require the generalization ability across domains. In this paper, we study the problem of few-shot domain generalization (FSDG), which is a more challenging variant of few-shot classification. FSDG requires additional generalization with larger gap from seen domains to unseen domains. We address FSDG problem by meta-learning two levels of meta-knowledge, where the lower-level meta-knowledge is domain-specific embedding spaces as subspaces of a base space for intra-domain generalization, and the upper-level meta-knowledge is the base space and a prior subspace over domain-specific spaces for inter-domain generalization. We formulate the two levels of meta-knowledge learning problem with bi-level optimization, and further develop an optimization algorithm without higher-order derivative information to solve it. We demonstrate our method is significantly superior to the previous works by evaluating it on the widely used benchmark Meta-Dataset.

关键词：

Enhancing Neural Machine Translation with Semantic Units

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Langlin Gu, Shuhao Zhang, Zhuocheng Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences China

Conventional neural machine translation (NMT) models typically use subwords and words as the basic units for model input and comprehension. However, complete words and phrases composed of several tokens are often the fundamental units for expressing semantics, referred to as semantic units. To address this issue, we propose a method Semantic Units for Machine Translation (SU4MT) which models the integral meanings of semantic units within a sentence, and then leverages them to provide a new perspective for understanding the sentence. Specifically, we first propose Word Pair Encoding (WPE), a phrase extraction method to help identify the boundaries of semantic units. Next, we design an Attentive Semantic Fusion (ASF) layer to integrate the semantics of multiple subwords into a single vector: the semantic unit representation. Lastly, the semantic-unit-level sentence representation is concatenated to the token-level one, and they are combined as the input of encoder. Experimental results demonstrate that our method effectively models and leverages semantic-unit-level information and outperforms the strong baselines. The code is availab.e at https://***/ictnlp/SU4MT. © 2023, CC BY-NC-ND.

关键词： Neural machine translation