检索结果-内蒙古大学图书馆

A Modular Conditional Diffusion Framework for Image Reconstruction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhussip, Magauiya Koshelev, Iaroslav Lefkimmiatis, Stamatis MTS AI AI Foundation and Algorithm Lab

Diffusion Probabilistic Models (DPMs) have been recently utilized to deal with various blind image restoration (IR) tasks, where they have demonstrated outstanding performance in terms of perceptual quality. However, the task-specific nature of existing solutions and the excessive computational costs related to their training, make such models impractical and challenging to use for different IR tasks than those that were initially trained for. This hinders their wider adoption, especially by those who lack access to powerful computational resources and vast amount of training data. In this work we aim to address the above issues and enable the successful adoption of DPMs in practical IR-related applications. Towards this goal, we propose a modular diffusion probabilistic IR framework (DP-IR), which allows us to combine the performance benefits of existing pre-trained state-of-the-art IR networks and generative DPMs, while it requires only the additional training of a relatively small module (0.7M params) related to the particular IR task of interest. Moreover, the architecture of the proposed framework allows for a sampling strategy that leads to at least four times reduction of neural function evaluations without suffering any performance loss, while it can also be combined with existing acceleration techniques such as DDIM. We evaluate our model on four benchmarks for the tasks of burst JDD-SR, dynamic scene deblurring, and superresolution. Our method outperforms existing approaches in terms of perceptual quality while it retains a competitive performance with respect to fidelity metrics. Copyright © 2024, The Authors. All rights reserved.

关键词： Benchmarking

Robust Two-View Geometry Estimation with Implicit Differentiation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Pyatov, Vladislav Koshelev, Iaroslav Lefkimmiatis, Stamatios Center for AI Technology Russia AI Foundation and Algorithm Lab Russia MTS AI Group Russia

We present a novel two-view geometry estimation framework which is based on a differentiable robust loss function fitting. We propose to treat the robust fundamental matrix estimation as an implicit layer, which allows us to avoid backpropagation through time and significantly improves the numerical stability. To take full advantage of the information from the feature matching stage we incorporate learnable weights that depend on the matching confidences. In this way our solution brings together feature extraction, matching and two-view geometry estimation in a unified end-to-end trainable pipeline. We evaluate our approach on the camera pose estimation task in both outdoor and indoor scenarios. The experiments on several datasets show that the proposed method outperforms both classic and learning-based state-of-the-art methods by a large margin. The project webpage is available at: https://***/VladPyatov/ihls © 2024, CC BY-NC-ND.

关键词： Geometry

A3D: DOES DIFFUSION DREAM ABOUT 3D ALIGNMENT?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ignatyev, Savva Konovalova, Nina Selikhanovych, Daniil Voynov, Oleg Patakin, Nikolay Olkov, Ilya Senushkin, Dmitry Artemov, Alexey Konushin, Anton Filippov, Alexander Wonka, Peter Burnaev, Evgeny Skoltech Russia AIRI Russia Medida AI Israel AI Foundation and Algorithm Lab Russia KAUST Saudi Arabia

We tackle the problem of text-driven 3D generation from a geometry alignment perspective. Given a set of text prompts, we aim to generate a collection of objects with semantically corresponding parts aligned across them. Recent methods based on Score Distillation have succeeded in distilling the knowledge from 2D diffusion models to high-quality representations of the 3D objects. These methods handle multiple text queries separately, and therefore the resulting objects have a high variability in object pose and structure. However, in some applications, such as 3D asset design, it may be desirable to obtain a set of objects aligned with each other. In order to achieve the alignment of the corresponding parts of the generated objects, we propose to embed these objects into a common latent space and optimize the continuous transitions between these objects. We enforce two kinds of properties of these transitions: smoothness of the transition and plausibility of the intermediate objects along the transition. We demonstrate that both of these properties are essential for good alignment. We provide several practical scenarios that benefit from alignment between the objects, including 3D editing and object hybridization, and experimentally demonstrate the effectiveness of our method. ***/a3d © 2024, CC BY-NC-SA.

关键词： Semantics

Robust Two-View Geometry Estimation with Implicit Differentiation

学校读者我要写书评

暂无评论

Robust Two-View Geometry Estimation with Implicit Differenti...

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Vladislav Pyatov Iaroslav Koshelev Stamatios Lefkimmiatis Skolkovo Institute of Science and Technology (Skoltech) Center for AI Technology Russia AI Foundation and Algorithm Lab Russia MTS AI Group Russia

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

We present a novel two-view geometry estimation framework which is based on a differentiable robust loss function fitting. We propose to treat the robust fundamental matrix estimation as an implicit layer, which allows us to avoid backpropagation through time and significantly improves the numerical stability. To take full advantage of the information from the feature matching stage we incorporate learnable weights that depend on the matching confidences. In this way our solution brings together feature extraction, matching and two-view geometry estimation in a unified end-to-end trainable pipeline. We evaluate our approach on the camera pose estimation task in both outdoor and indoor scenarios. The experiments on several datasets show that the proposed method outperforms both classic and learning-based state-of- the-art methods by a large margin. The project webpage is available at: https://***/VladPyatov/ihls

关键词： Geometry Backpropagation Pose estimation Robot vision systems Pipelines Fitting Cameras Feature extraction Numerical stability Intelligent robots

RTD-Lite: Scalable Topological Analysis for Comparing Weighted Graphs in Learning Tasks

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Tulchinskii, Eduard Voronkova, Daria Trofimov, Ilya Burnaev, Evgeny Barannikov, Serguei Skoltech AI Foundation and Algorithm Lab Russia Skoltech AIRI Russia Skoltech Russia Skoltech CNRS Russia

Topological methods for comparing weighted graphs are valuable in various learning tasks but often suffer from computational inefficiency on large datasets. We introduce RTD-Lite, a scalable algorithm that efficiently compares topological features, specifically connectivity or cluster structures at arbitrary scales, of two weighted graphs with one-to-one correspondence between vertices. Using minimal spanning trees in auxiliary graphs, RTD-Lite captures topological discrepancies with O(n2) time and memory complexity. This efficiency enables its application in tasks like dimensionality reduction and neural network training. Experiments on synthetic and real-world datasets demonstrate that RTD-Lite effectively identifies topological differences while significantly reducing computation time compared to existing methods. Moreover, integrating RTD-Lite into neural network training as a loss function component enhances the preservation of topological structures in learned representations. Our code is publicly available at https://***/ArGintum/RTD-Lite. © 2025, CC BY.

关键词： Dimensionality reduction

Quantifying Logical Consistency in Transformers via Query-Key Alignment

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Tulchinskii, Eduard Voznyuk, Anastasia Kushnareva, Laida Andriiainen, Andrei Piontkovskaya, Irina Burnaev, Evgeny Barannikov, Serguei Skolkovo Institute of Science and Technology Russia AI Foundation Algorithm Lab Moscow Institute of Physics and Technology Russia CNRS Université Paris Cité France

Large language models (LLMs) have demonstrated impressive performance in various natural language processing tasks, yet their ability to perform multi-step logical reasoning remains an open challenge. Although Chain-of-Thought prompting has improved logical reasoning by enabling models to generate intermediate steps, it lacks mechanisms to assess the coherence of these logical transitions. In this paper, we propose a novel, lightweight evaluation strategy for logical reasoning that uses query–key alignments inside transformer attention heads. By computing a single forward pass and extracting a "QK-score" from carefully chosen heads, our method reveals latent representations that reliably separate valid from invalid inferences, offering a scalable alternative to traditional ablation-based techniques. We also provide an empirical validation on multiple logical reasoning benchmarks, demonstrating improved robustness of our evaluation method against distractors and increased reasoning depth. The experiments were conducted on a diverse set of models, ranging from 1.5B to 70B parameters. © 2025, CC BY.

关键词： Structured Query Language

NeuSD: Surface Completion with Multi-View Text-to-Image Diffusion

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ignatyev, Savva Selikhanovych, Daniil Voynov, Oleg Wang, Yiqun Wonka, Peter Lefkimmiatis, Stamatios Burnaev, Evgeny Skoltech Russia AIRI Russia Chongqing University China KAUST Saudi Arabia AI Foundation and Algorithm Lab Russia

We present a novel method for 3D surface reconstruction from multiple images where only a part of the object of interest is captured. Our approach builds on two recent developments: surface reconstruction using neural radiance fields for the reconstruction of the visible parts of the surface, and guidance of pre-trained 2D diffusion models in the form of Score Distillation Sampling (SDS) to complete the shape in unobserved regions in a plausible manner. We introduce three components. First, we suggest employing normal maps as a pure geometric representation for SDS instead of color renderings which are entangled with the appearance information. Second, we introduce the freezing of the SDS noise during training which results in more coherent gradients and better convergence. Third, we propose Multi-View SDS as a way to condition the generation of the non-observable part of the surface without fine-tuning or making changes to the underlying 2D Stable Diffusion model. We evaluate our approach on the BlendedMVS dataset demonstrating significant qualitative and quantitative improvements over competing methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Robust ai-Generated Text Detection by Restricted Embeddings

学校读者我要写书评

暂无评论

Robust AI-Generated Text Detection by Restricted Embeddings

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Kuznetsov, Kristian Tulchinskii, Eduard Kushnareva, Laida Magai, German Barannikov, Serguei Nikolenko, Sergey Piontkovskaya, Irina AI Foundation and Algorithm Lab Russia HSE University Russia Noeon Research Japan Skolkovo Institute of Science and Technology Russia CNRS Université Paris Cité France ISP RAS Research Center for Trusted Artificial Intelligence Moscow Russia St. Petersburg Department The Steklov Institute of Mathematics Russia

ISBN: (纸本)9798891761681

Growing amount and quality of ai-generated texts makes detecting such content more difficult. In most real-world scenarios, the domain (style and topic) of generated data and the generator model are not known in advance. In this work, we focus on the robustness of classifier-based detectors of ai-generated text, namely their ability to transfer to unseen generators or semantic domains. We investigate the geometry of the embedding space of Transformer-based text encoders and show that clearing out harmful linear subspaces helps to train a robust classifier, ignoring domain-specific spurious features. We investigate several subspace decomposition and feature selection strategies and achieve significant improvements over state of the art methods in cross-domain and cross-generator transfer. Our best approaches for head-wise and coordinate-based subspace removal increase the mean out-of-distribution (OOD) classification score by up to 9% and 14% in particular setups for RoBERTa and BERT embeddings respectively. We release our code and data. © 2024 Association for Computational Linguistics.

关键词： Embeddings

LISTENING TO THE WISE FEW: SELECT-AND-COPY ATTENTION HEADS FOR MULTIPLE-CHOICE QA

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tulchinskii, Eduard Kushnareva, Laida Kuznetsov, Kristian Voznyuk, Anastasia Andriiainen, Andrei Piontkovskaya, Irina Burnaev, Evgeny Barannikov, Serguei Skolkovo Institute of Science and Technology Russia AI Foundation and Algorithm Lab Moscow Institute of Physics and Technology Russia CNRS Université Paris Cité France

A standard way to evaluate the abilities of LLM involves presenting a multiple-choice question and selecting the option with the highest logit as the model’s predicted answer. However, such a format for evaluating LLMs has limitations, since even if the model knows the correct answer, it may struggle to select the corresponding letter simply due to difficulties in following this rigid format. To address this, we introduce new scores that better capture and reveal model’s underlying knowledge: the Query-Key Score (QK-score), derived from the interaction between query and key representations in attention heads, and the Attention Score, based on attention weights. These scores are extracted from specific select- and-copy heads, which show consistent performance across popular Multi-Choice Question Answering (MCQA) datasets. Based on these scores, our method improves knowledge extraction, yielding up to 16% gain for LLaMA2-7B and up to 10% for larger models on popular MCQA benchmarks. At the same time, the accuracy on a simple synthetic dataset, where the model explicitly knows the right answer, increases by almost 60%, achieving nearly perfect accuracy, therefore demonstrating the method’s efficiency in mitigating MCQA format limitations. To support our claims, we conduct experiments on models ranging from 7 billion to 70 billion parameters in both zero- and few-shot setups. © 2024, CC BY.

关键词： Question answering