检索结果-内蒙古大学图书馆

OpenSlot: Mixed Open-Set Recognition with Object-Centric Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yin, Xu Pan, Fei An, Guoyuan Huo, Yuchi Xie, Zixuan Yoon, Sung-Eui School of Computing Korea Advanced Institute of Science and Technology Daejeon34141 Korea Republic of School of Computer Science Engineering University of Michigan United States State Key Lab of CAD and CG Zhejiang University China Zhejiang Lab 310058 China Institute of Computing Technology University of Chinese Academy of Sciences Beijing100190 China Faculty of School of Computing Korea Advanced Institute of Science and Technology Deajeon34141 Korea Republic of

Existing open-set recognition (OSR) studies typically assume that each image contains only one class label, with the unknown test set (negative) having a disjoint label space from the known test set (positive), a scenario referred to as full-label shift. This paper introduces the mixed OSR problem, where test images contain multiple class semantics, with both known and unknown classes co-occurring in the negatives, leading to a more complex super-label shift that better reflects real-world scenarios. To tackle this challenge, we propose the OpenSlot framework, based on object-centric learning, which uses slot features to represent diverse class semantics and generate class predictions. The proposed anti-noise slot (ANS) technique helps mitigate the impact of noise (invalid or background) slots during classification training, addressing the semantic misalignment between class predictions and ground truth. We evaluate OpenSlot on both mixed and conventional OSR benchmarks. Without elaborate designs, our method not only excels existing approaches in detecting super-label shifts across OSR tasks, but also achieves state-of-the-art performance on conventional benchmarks. Meanwhile, OpenSlot can localize class objects without using bounding boxes during training, demonstrating competitive performance in open-set object detection and potential for generalization. © 2024, CC BY-NC-ND.

关键词： Object detection

ViSpeak: Visual Instruction Feedback in Streaming Videos

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fu, Shenghao Yang, Qize Li, Yuan-Ming Peng, Yi-Xing Lin, Kun-Yu Wei, Xihan Hu, Jian-Fang Xie, Xiaohua Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Tongyi Lab Alibaba Group China Peng Cheng Laboratory China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Guangdong Province Key Laboratory of Information Security Technology China China

Recent advances in Large Multi-modal Models (LMMs) are primarily focused on offline video understanding. Instead, streaming video understanding poses great challenges to recent models due to its time-sensitive, omni-modal and interactive characteristics. In this work, we aim to extend the streaming video understanding from a new perspective and propose a novel task named Visual Instruction Feedback in which models should be aware of visual contents and learn to extract instructions from them. For example, when users wave their hands to agents, agents should recognize the gesture and start conversations with welcome information. Thus, following instructions in visual modality greatly enhances user-agent interactions. To facilitate research, we define seven key subtasks highly relevant to visual modality and collect the ViSpeak-Instruct dataset for training and the ViSpeak-Bench for evaluation. Further, we propose the ViSpeak model, which is a SOTA streaming video understanding LMM with GPT-4o-level performance on various streaming video understanding benchmarks. After finetuning on our ViSpeak-Instruct dataset, ViSpeak is equipped with basic visual instruction feedback ability, serving as a solid baseline for future research. Copyright © 2025, The Authors. All rights reserved.

关键词： Video streaming

Towards HDR and HFR Video from Rolling-Mixed-Bit Spikings

学校读者我要写书评

暂无评论

Towards HDR and HFR Video from Rolling-Mixed-Bit Spikings

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Yakun Chang Yeliduosi Xiaokaiti Yujia Liu Bin Fan Zhaojun Huang Tiejun Huang Boxin Shi Institute of Information Science School of Computer Science Beijing Jiaotong University Beijing Key Laboratory of Advanced Information Science and Network Technology National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University National Engineering Research Center of Visual Technology School of Computer Science Peking University National Key Lab of General AI School of Intelligence Science and Technology Peking University

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

The spiking cameras offer the benefits of high dynamic range (HDR), high temporal resolution, and low data redundancy. However, reconstructing HDR videos in high-speed conditions using single-bit spikings presents challenges due to the limited bit depth. Increasing the bit depth of the spikings is advantageous for boosting HDR performance, but the readout efficiency will be decreased, which is unfavorable for achieving a high frame rate (HFR) video. To address these challenges, we propose a readout mechanism to obtain rolling-mixed-bit (RMB) spikings, which involves inter-leaving multi-bit spikings within the single-bit spikings in a rolling manner, thereby combining the characteristics of high bit depth and efficient readout. Furthermore, we introduce RMB-Net for reconstructing HDR and HFR videos. RMB-Net comprises a cross-bit attention block for fusing mixed-bit spikings and a cross-time attention block for achieving temporal fusion. Extensive experiments conducted on synthetic and real-synthetic data demonstrate the superiority of our method. For instance, pure 3 -bit spikings result in 3 times of data volume, whereas our method achieves comparable performance with less than 2% increase in data volume.

关键词： computer vision Computational modeling Redundancy Cameras Boosting Data models Pattern recognition

LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fu, Shenghao Yang, Qize Mo, Qijie Yan, Junkai Wei, Xihan Meng, Jingke Xie, Xiaohua Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Tongyi Lab Alibaba Group China Peng Cheng Laboratory China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Guangdong Province Key Laboratory of Information Security Technology China China

Recent open-vocabulary detectors achieve promising performance with abundant region-level annotated data. In this work, we show that an open-vocabulary detector co-training with a large language model by generating image-level detailed captions for each image can further improve performance. To achieve the goal, we first collect a dataset, GroundingCap-1M, wherein each image is accompanied by associated grounding labels and an image-level detailed caption. With this dataset, we finetune an open-vocabulary detector with training objectives including a standard grounding loss and a caption generation loss. We take advantage of a large language model to generate both region-level short captions for each region of interest and image-level long captions for the whole image. Under the supervision of the large language model, the resulting detector, LLMDet, outperforms the baseline by a clear margin, enjoying superior open-vocabulary ability. Further, we show that the improved LLMDet can in turn build a stronger large multi-modal model, achieving mutual benefits. The code, model, and dataset is available at https://***/iSEE-laboratory/LLMDet. Copyright © 2025, The Authors. All rights reserved.

关键词： Object detection

DCET-Net: Dual-Stream Convolution Expanded Transformer for Breast Cancer Histopathological Image Classification

学校读者我要写书评

暂无评论

DCET-Net: Dual-Stream Convolution Expanded Transformer for B...

2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021

作者： Zou, Ying Chen, Shannan Sun, Qiule Liu, Bin Zhang, Jianxin Dalian University Key Lab of Advanced Design and Intelligent Computing Ministry of Education Dalian China Dalian University of Technology School of Information and Communication Engineering Dalian China Dalian China Dalian Minzu University School of Computer Science and Engineering Dalian China

ISBN: (纸本)9781665401265

Researches on breast cancer histopathological image classification have achieved a great breakthrough using deep backbones of Convolutional Neural Networks (CNNs) in recent years. However, due to the inductive bias of locality, CNNs are unable to effectively extract the global feature information of breast cancer histopathological images, limiting the improvement of the classification results. To overcome this shortcoming, this paper reasonably introduces an extra backbone stream of a pure transformer, which consists of a self-attention mechanism to capture global receptive fields of histopathological images, thereby compensating the locality characteristic of CNNs backbone. Based on two backbone streams of CNN and transformer, a dual-stream network called DCET-Net is proposed, which considers local features and global ones simultaneously, and progressively combines them from these two streams to form the final representations for classification. DCET-Net is extensively evaluated on the representative BreakHis histopathological image dataset, and experimental results demonstrate that it is highly competitive with the state-of-the-art CNN methods in breast cancer histopathological image classification task. © 2021 IEEE.

关键词： Convolutional neural networks

ASAG: Building Strong One-Decoder-Layer Sparse Detectors via Adaptive Sparse Anchor Generation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fu, Shenghao Yan, Junkai Gao, Yipeng Xie, Xiaohua Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Pengcheng Lab China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Recent sparse detectors with multiple, e.g. six, decoder layers achieve promising performance but much inference time due to complex heads. Previous works have explored using dense priors as initialization and built one-decoder-layer detectors. Although they gain remarkable acceleration, their performance still lags behind their six-decoder-layer counterparts by a large margin. In this work, we aim to bridge this performance gap while retaining fast speed. We find that the architecture discrepancy between dense and sparse detectors leads to feature conflict, hampering the performance of one-decoder-layer detectors. Thus we propose Adaptive Sparse Anchor Generator (ASAG) which predicts dynamic anchors on patches rather than grids in a sparse way so that it alleviates the feature conflict problem. For each image, ASAG dynamically selects which feature maps and which locations to predict, forming a fully adaptive way to generate image-specific anchors. Further, a simple and effective Query Weighting method eases the training instability from adaptiveness. Extensive experiments show that our method outperforms dense-initialized ones and achieves a better speed-accuracy tradeoff. The code is available at https://***/iSEE-laboratory/ASAG. Copyright © 2023, The Authors. All rights reserved.

关键词： Decoding

Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Xiao-Ming Zheng, Dian Liu, Zuhao Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Pengcheng Lab China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Binarization of neural networks is a dominant paradigm in neural networks compression. The pioneering work BinaryConnect uses Straight Through Estimator (STE) to mimic the gradients of the sign function, but it also causes the crucial inconsistency problem. Most of the previous methods design different estimators instead of STE to mitigate it. However, they ignore the fact that when reducing the estimating error, the gradient stability will decrease concomitantly. These highly divergent gradients will harm the model training and increase the risk of gradient vanishing and gradient exploding. To fully take the gradient stability into consideration, we present a new perspective to the BNNs training, regarding it as the equilibrium between the estimating error and the gradient stability. In this view, we firstly design two indicators to quantitatively demonstrate the equilibrium phenomenon. In addition, in order to balance the estimating error and the gradient stability well, we revise the original straight through estimator and propose a power function based estimator, Rectified Straight Through Estimator (ReSTE for short). Comparing to other estimators, ReSTE is rational and capable of flexibly balancing the estimating error with the gradient stability. Extensive experiments on CIFAR-10 and ImageNet datasets show that ReSTE has excellent performance and surpasses the state-of-the-art methods without any auxiliary modules or losses. Copyright © 2023, The Authors. All rights reserved.

关键词： Stability

Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training

学校读者我要写书评

暂无评论

Estimator Meets Equilibrium Perspective: A Rectified Straigh...

International Conference on computer Vision (ICCV)

作者： Xiao-Ming Wu Dian Zheng Zuhao Liu Wei-Shi Zheng School of Computer Science and Engineering Sun Yat-sen University China Pengcheng Lab China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

关键词：

ASAG: Building Strong One-Decoder-Layer Sparse Detectors via Adaptive Sparse Anchor Generation

学校读者我要写书评

暂无评论

ASAG: Building Strong One-Decoder-Layer Sparse Detectors via...

International Conference on computer Vision (ICCV)

作者： Shenghao Fu Junkai Yan Yipeng Gao Xiaohua Xie Wei-Shi Zheng School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Pengcheng Lab China

Recent sparse detectors with multiple, e.g. six, decoder layers achieve promising performance but much inference time due to complex heads. Previous works have explored using dense priors as initialization and built one-decoder-layer detectors. Although they gain remarkable acceleration, their performance still lags behind their six-decoder-layer counterparts by a large margin. In this work, we aim to bridge this performance gap while retaining fast speed. We find that the architecture discrepancy between dense and sparse detectors leads to feature conflict, hampering the performance of one-decoder-layer detectors. Thus we propose Adaptive Sparse Anchor Generator (ASAG) which predicts dynamic anchors on patches rather than grids in a sparse way so that it alleviates the feature conflict problem. For each image, ASAG dynamically selects which feature maps and which locations to predict, forming a fully adaptive way to generate image-specific anchors. Further, a simple and effective Query Weighting method eases the training instability from adaptiveness. Extensive experiments show that our method outperforms dense-initialized ones and achieves a better speed-accuracy trade-off. The code is available at https://***/iSEE-laboratory/ASAG.

关键词：