检索结果-内蒙古大学图书馆

SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Yunchen Yu, Zhou He, Gaoqi Shen, Yunhang Li, Ke Sun, Xing Lin, Shaohui East China Normal University China Key Laboratory of Advanced Theory and Application in Statistics and Data Science Ministry of Education China Youtu Lab Tencent Shanghai China

Symmetric positive definite (SPD) matrices have shown important value and applications in statistics and machine learning, such as FMRI analysis and traffic prediction. Previous works on SPD matrices mostly focus on discriminative models, where predictions are made directly on E(X|y), where y is a vector and X is an SPD matrix. However, these methods are challenging to handle for large-scale data, as they need to access and process the whole data. In this paper, inspired by denoising diffusion probabilistic model (DDPM), we propose a novel generative model, termed SPD-DDPM, by introducing Gaussian distribution in the SPD space to estimate E(X|y). Moreover, our model is able to estimate p(X) unconditionally and flexibly without giving y. On the one hand, the model conditionally learns p(X|y) and utilizes the mean of samples to obtain E(X|y) as a prediction. On the other hand, the model unconditionally learns the probability distribution of the data p(X) and generates samples that conform to this distribution. Furthermore, we propose a new SPD net which is much deeper than the previous networks and allows for the inclusion of conditional factors. Experiment results on toy data and real taxi data demonstrate that our models effectively fit the data distribution both unconditionally and unconditionally and provide accurate predictions. Copyright © 2023, The Authors. All rights reserved.

关键词： Forecasting

Style Transformer for Image Inversion and Editing

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hu, Xueqi Huang, Qiusheng Shi, Zhengyi Li, Siyuan Gao, Changxin Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing China Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai China Huazhong University of Science and Technology Wuhan China

Existing GAN inversion methods fail to provide latent codes for reliable reconstruction and flexible editing simultaneously. This paper presents a transformer-based image inversion and editing model for pretrained StyleGAN which is not only with less distortions, but also of high quality and flexibility for editing. The proposed model employs a CNN encoder to provide multi-scale image features as keys and values. Meanwhile it regards the style code to be determined for different layers of the generator as queries. It first initializes query tokens as learnable parameters and maps them into W+ space. Then the multi-stage alternate selfand cross-attention are utilized, updating queries with the purpose of inverting the input by the generator. Moreover, based on the inverted code, we investigate the referenceand label-based attribute editing through a pretrained latent classifier, and achieve flexible image-to-image translation with high quality results. Extensive experiments are carried out, showing better performances on both inversion and editing tasks within StyleGAN. Codes are available at https://***/sapphire497/style-transformer. Copyright © 2022, The Authors. All rights reserved.

关键词： Generative adversarial networks

Assessment of Effects of Age and Gender on the Incubation Period of COVID-19 with a Mixture Regression Model

学校读者我要写书评

暂无评论

Journal of data science 2021年第1期19卷 p1-16页

作者： SIMING ZHENG JING QIN YONG ZHOU Academy of Mathematics and Systems Science University of Chinese Academy of Sciences Beijing China National Institute of Allergy and Infectious Diseases National Institutes of Health Bethesda Maryland USA Key Laboratory of Advanced Theory and Application in Statistics and Data Science MOE Academy of Statistics and Interdisciplinary Sciences Faculty of Economics and Management East China Normal University Shanghai China

Following the outbreak of COVID-19, various containment measures have been taken, including the use of quarantine. At present, the quarantine period is the same for everyone, since it is implicitly assumed that the incubation period distribution of COVID-19 is the same regardless of age or gender. For testing the effects of age and gender on the incubation period of COVID-19, a novel two-component mixture regression model is proposed. An expectation-maximization (EM) algorithm is adopted to obtain estimates of the parameters of interest, and the simulation results show that the proposed method outperforms the simple regression method and has robustness. The proposed method is applied to a Zhejiang COVID-19 dataset, and it is found that age and gender statistically have no effect on the incubation period of COVID-19, which indicates that the quarantine measure currently in operation is reasonable. [ABSTRACT FROM AUTHOR]

关键词： EM algorithm incubation period length-biased data mixture model

A General and Efficient Training for Transformer via Token Expansion

学校读者我要写书评

暂无评论

A General and Efficient Training for Transformer via Token E...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Wenxuan Huang Yunhang Shen Jiao Xie Baochang Zhang Gaoqi He Ke Li Xing Sun Shaohui Lin School of Computer Science and Technology East China Normal University Shanghai China Tencent Youtu Lab China Xiamen University China Beihang University China Key Laboratory of Advanced Theory and Application in Statistics and Data Science - MOE China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

The remarkable performance of Vision Transformers (ViTs) typically requires an extremely large training cost. Existing methods have attempted to accelerate the training of ViTs, yet typically disregard method universality with accuracy dropping. Meanwhile, they break the training consistency of the original transformers, including the consistency of hyperparameters, architecture, and strategy, which prevents them from being widely applied to different Transformer networks. In this paper, we propose a novel token growth scheme Token Expansion (termed ToE) to achieve consistent training acceleration for ViTs. We introduce an “initialization-expansion-merging” pipeline to maintain the integrity of the intermediate feature distribution of original transformers, preventing the loss of crucial learnable information in the training process. ToE can not only be seamlessly integrated into the training and fine-tuning process of transformers (e.g., DeiT and LV-ViT), but also effective for efficient training frameworks (e.g., EfficientTrain), without twisting the original training hyperparameters, architecture, and introducing additional training strategies. Extensive experiments demonstrate that ToE achieves about 1.3× faster for the training of ViTs in a lossless manner, or even with performance gains over the full-token training baselines. Code is available at https://***/Osilly/TokenExpansion.

关键词： Training Computer vision Accuracy Costs Codes Pipelines Computer architecture

A General and Efficient Training for Transformer via Token Expansion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Wenxuan Shen, Yunhang Xie, Jiao Zhang, Baochang He, Gaoqi Li, Ke Sun, Xing Lin, Shaohui School of Computer Science and Technology East China Normal University Shanghai China Tencent Youtu Lab China Xiamen University China Beihang University China Key Laboratory of Advanced Theory and Application in Statistics and Data Science - MOE China

The remarkable performance of Vision Transformers (ViTs) typically requires an extremely large training cost. Existing methods have attempted to accelerate the training of ViTs, yet typically disregard method universality with accuracy dropping. Meanwhile, they break the training consistency of the original transformers, including the consistency of hyper-parameters, architecture, and strategy, which prevents them from being widely applied to different Transformer networks. In this paper, we propose a novel token growth scheme Token Expansion (termed ToE) to achieve consistent training acceleration for ViTs. We introduce an "initialization-expansion-merging" pipeline to maintain the integrity of the intermediate feature distribution of original transformers, preventing the loss of crucial learnable information in the training process. ToE can not only be seamlessly integrated into the training and fine-tuning process of transformers (e.g., DeiT and LV-ViT), but also effective for efficient training frameworks (e.g., EfficientTrain), without twisting the original training hyper-parameters, architecture, and introducing additional training strategies. Extensive experiments demonstrate that ToE achieves about 1.3× faster for the training of ViTs in a lossless manner, or even with performance gains over the full-token training baselines. Code is available at https://***/Osilly/TokenExpansion. Copyright © 2024, The Authors. All rights reserved.

关键词： Machine learning

A review of distributed statistical inference

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gao, Yuan Liu, Weidong Wang, Hansheng Wang, Xiaozhou Yan, Yibo Zhang, Riquan School of Statistics Key Laboratory of Advanced Theory and Application in Statistics and Data Science - MOE East China Normal University Shanghai China School of Mathematical Sciences Key Lab of Artificial Intelligence - MOE Shanghai Jiao Tong University Shanghai China Guanghua School of Management Peking University Beijing China

The rapid emergence of massive datasets in various fields poses a serious challenge to traditional statistical methods. Meanwhile, it provides opportunities for researchers to develop novel algorithms. Inspired by the idea of divide-and-conquer, various distributed frameworks for statistical estimation and inference have been proposed. They were developed to deal with large-scale statistical optimization problems. This paper aims to provide a comprehensive review for related literature. It includes parametric models, nonparametric models, and other frequently used models. Their key ideas and theoretical properties are summarized. The trade-off between communication cost and estimate precision together with other concerns are discussed. Copyright © 2023, The Authors. All rights reserved.

关键词： Principal component analysis

RecursiveDet: End-to-End Region-based Recursive Object Detection

学校读者我要写书评

暂无评论

RecursiveDet: End-to-End Region-based Recursive Object Detec...

International Conference on Computer Vision (ICCV)

作者： Jing Zhao Li Sun Qingli Li Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University Shanghai China Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai China

End-to-end region-based object detectors like Sparse R-CNN usually have multiple cascade bounding box decoding stages, which refine the current predictions according to their previous results. Model parameters within each stage are independent, evolving a huge cost. In this paper, we find the general setting of decoding stages is actually redundant. By simply sharing parameters and making a recursive decoder, the detector already obtains a significant improvement. The recursive decoder can be further enhanced by positional encoding (PE) of the proposal box, which makes it aware of the exact locations and sizes of input bounding boxes, thus becoming adaptive to proposals from different stages during the recursion. Moreover, we also design centerness-based PE to distinguish the RoI feature element and dynamic convolution kernels at different positions within the bounding box. To validate the effectiveness of the proposed method, we conduct intensive ablations and build the full model on three recent mainstream region-based detectors. The RecusiveDet is able to achieve obvious performance boosts with even fewer model parameters and slightly increased computation cost. Codes are available at https://***/bravezzzzzz/RecursiveDet.

关键词：

RecursiveDet: End-to-End Region-based Recursive Object Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhao, Jing Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University Shanghai China Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai China

关键词： Object detection

IoU-Enhanced Attention for End-to-End Task Specific Object Detection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Jing Wu, Shengjian Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University Shanghai China Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai China

Without densely tiled anchor boxes or grid points in the image, sparse R-CNN achieves promising results through a set of object queries and proposal boxes updated in the cascaded training manner. However, due to the sparse nature and the one-to-one relation between the query and its attending region, it heavily depends on the self attention, which is usually inaccurate in the early training stage. Moreover, in a scene of dense objects, the object query interacts with many irrelevant ones, reducing its uniqueness and harming the performance. This paper proposes to use IoU between different boxes as a prior for the value routing in self attention. The original attention matrix multiplies the same size matrix computed from the IoU of proposal boxes, and they determine the routing scheme so that the irrelevant features can be suppressed. Furthermore, to accurately extract features for both classification and regression, we add two lightweight projection heads to provide the dynamic channel masks based on object query, and they multiply with the output from dynamic convs, making the results suitable for the two different tasks. We validate the proposed scheme on different datasets, including MS-COCO and CrowdHuman, showing that it significantly improves the performance and increases the model convergence speed. Codes are available at https://***/bravezzzzzz/IoU-Enhanced-Attention. Copyright © 2022, The Authors. All rights reserved.

关键词： Object detection