检索结果-内蒙古大学图书馆

CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Siyuan Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing China Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai China

Pre-trained vision-language models like CLIP have recently shown superior performances on various downstream tasks, including image classification and segmentation. However, in fine-grained image re-identification (ReID), the labels are indexes, lacking concrete text descriptions. Therefore, it remains to be determined how such models could be applied to these tasks. This paper first finds out that simply fine-tuning the visual model initialized by the image encoder in CLIP, has already obtained competitive performances in various ReID tasks. Then we propose a two-stage strategy to facilitate a better visual representation. The key idea is to fully exploit the cross-modal description ability in CLIP through a set of learnable text tokens for each ID and give them to the text encoder to form ambiguous descriptions. In the first training stage, image and text encoders from CLIP keep fixed, and only the text tokens are optimized from scratch by the contrastive loss computed within a batch. In the second stage, the ID-specific text tokens and their encoder become static, providing constraints for fine-tuning the image encoder. With the help of the designed loss in the downstream task, the image encoder is able to represent data as vectors in the feature embedding accurately. The effectiveness of the proposed strategy is validated on several datasets for the person or vehicle ReID tasks. Code is available at https://***/Syliz517/CLIP-ReID. Copyright © 2022, The Authors. All rights reserved.

关键词： Image segmentation

Disentangling the spatial structure and style in conditional vae

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Ziye Sun, Li Zheng, Zhilin Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai200241 China

This paper aims to disentangle the latent space in cVAE into the spatial structure and the style code, which are complementary to each other, with one of them zs being label relevant and the other zu irrelevant. The generator is built by a connected encoder-decoder and a label condition mapping network. Depending on whether the label is related with the spatial structure, the output zs from the condition mapping network is used either as a style code or a spatial structure code. The encoder provides the label irrelevant posterior from which zu is sampled. The decoder employs zs and zu in each layer by adaptive normalization like SPADE or AdaIN. Extensive experiments on two datasets with different types of labels show the effectiveness of our method. Copyright © 2019, The Authors. All rights reserved.

关键词： Mapping

QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hu, Xueqi Zhou, Xinyue Huang, Qiusheng Shi, Zhengyi Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing Shanghai China Key Laboratory of Advanced Theory and Application in Statistics and Data Science East China Normal University Shanghai China

Unpaired image-to-image (I2I) translation often requires to maximize the mutual information between the source and the translated images across different domains, which is critical for the generator to keep the source content and prevent it from unnecessary modifications. The self-supervised contrastive learning has already been successfully applied in the I2I. By constraining features from the same location to be closer than those from different ones, it implicitly ensures the result to take content from the source. However, previous work uses the features from random locations to impose the constraint, which may not be appropriate since some locations contain less information of source domain. Moreover, the feature itself does not reflect the relation with others. This paper deals with these problems by intentionally selecting significant anchor points for contrastive learning. We design a query-selected attention (QS-Attn) module, which compares feature distances in the source domain, giving an attention matrix with a probability distribution in each row. Then we select queries according to their measurement of significance, computed from the distribution. The selected ones are regarded as anchors for contrastive loss. At the same time, the reduced attention matrix is employed to route features in both domains, so that source relations maintain in the synthesis. We validate our proposed method in three different I2I datasets, showing that it increases the image quality without adding learnable parameters. Codes are available at https://***/sapphire497/query-selected-attention. Copyright © 2022, The Authors. All rights reserved.

关键词： Location

Novel View Synthesis on Unpaired data by Conditional Deformable Variational Auto-Encoder

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yin, Mingyu Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing China Key Laboratory of Advanced Theory and Application in Statistics & Data Science East China Normal University Shanghai200241 China

Novel view synthesis often needs the paired data from both the source and target views. This paper proposes a view translation model under cVAE-GAN framework without requiring the paired data. We design a conditional deformable module (CDM) which uses the view condition vectors as the filters to convolve the feature maps of the main branch in VAE. It generates several pairs of displacement maps to deform the features, like the 2D optical flows. The results are fed into the deformed feature based normalization module (DFNM), which scales and offsets the main branch feature, given its deformed one as the input from the side branch. Taking the advantage of the CDM and DFNM, the encoder outputs a view-irrelevant posterior, while the decoder takes the code drawn from it to synthesize the reconstructed and the view-translated images. To further ensure the disentanglement between the views and other factors, we add adversarial training on the code. The results and ablation studies on MultiPIE and 3D chair datasets validate the effectiveness of the framework in cVAE and the designed module. https://***/MingyuY/deformable-view-synthesis Copyright © 2020, The Authors. All rights reserved.

关键词： Signal encoding

Modelling Correlation Matrices in Multivariate Dyadic data: Latent Variable Models for Intergenerational Exchanges of Family Support

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Siliang Kuha, Jouni Steele, Fiona Key Laboratory of Advanced Theory and Application in Statistics and Data Science-MOE School of Statistics East China Normal University China Department of Statistics London School of Economics & Political Science United Kingdom

We define a model for the joint distribution of multiple continuous latent variables which includes a model for how their correlations depend on explanatory variables. This is motivated by and applied to social scientific research questions in the analysis of intergenerational help and support within families, where the correlations describe reciprocity of help between generations and complementarity of different kinds of help. We propose an MCMC procedure for estimating the model which maintains the positive definiteness of the implied correlation matrices, and describe theoretical results which justify this approach and facilitate efficient implementation of it. The model is applied to data from the UK Household Longitudinal Study to analyse exchanges of practical and financial support between adult individuals and their non-coresident parents. Copyright © 2022, The Authors. All rights reserved.

关键词： Covariance matrix

Rejoinder on‘A review of distributed statistical inference’

学校读者我要写书评

暂无评论

Statistical theory and Related Fields 2022年第2期6卷 111-113页

作者： Yuan Gao Weidong Liu Hansheng Wang Xiaozhou Wang Yibo Yana Riquan Zhang School of Statistics and Key Laboratory of Advanced Theory and Application in Statistics and Data Science–MOE East China Normal UniversityShanghaiPeople’s Republic of China School of Mathematical Sciences–School of Life Sciences and Biotechnology–MOE Key Lab of Artifcial Intelligence Shanghai Jiao Tong UniversityShanghaiPeople’s Republic of China Guanghua School of Management Peking UniversityBeijingPeople’s Republic of China

We thank the editor,Professor Jun Shao,for orga-nizing this stimulating *** are grateful to all discussants for their insightful comments on our review article on the distributed statistical *** to the urgent need to process the datasets with massive sizes,various distributed computing methods have been proposed for the large-scale statistical ***,some important theoretical results were *** we want to give a relatively comprehensive overview on this hot topic,there are still some important works that have been missed in our review written over a year ***,we are glad to see the discussants provide reviews of some new works and *** hope that these discussions and our review would serve as a stimulus for further studies in this rapidly developing area.

关键词： editor discussions sizes

Distributed algorithms for u-statistics-based empirical risk minimization

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2023年第1期24卷 12301-12343页

作者： Lanjue Chen Alan T.K. Wan Shuyi Zhang Yong Zhou Key Laboratory of Advanced Theory and Application in Statistics and Data Science Ministry of Education Academy of Statistics and Interdisciplinary Sciences and School of Statistics East China Normal University Shanghai China Department of Management Sciences School of Data Science and Department of Biostatistics City University of Hong Kong Kowloon Hong Kong

Empirical risk minimization, where the underlying loss function depends on a pair of data points, covers a wide range of application areas in statistics including pairwise ranking and survival analysis. The common empirical risk estimator obtained by averaging values of a loss function over all possible pairs of observations is essentially a U-statistic. One well-known problem with minimizing U-statistic type empirical risks, is that the computational complexity of U-statistics increases quadratically with the sample size. When faced with big data, this poses computational challenges as the colossal number of observation pairs virtually prohibits centralized computing to be performed on a single machine. This paper addresses this problem by developing two computationally and statistically efficient methods based on the divide-and-conquer strategy on a decentralized computing system, whereby the data are distributed among machines to perform the tasks. One of these methods is based on a surrogate of the empirical risk, while the other method extends the one-step updating scheme in classical M-estimation to the case of pairwise loss. We show that the proposed estimators are as asymptotically efficient as the benchmark global U-estimator obtained under centralized computing. As well, we introduce two distributed iterative algorithms to facilitate the implementation of the proposed methods, and conduct extensive numerical experiments to demonstrate their merit.

关键词：

ID-Unet: Iterative soft and hard deformation for view synthesis

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yin, Mingyu Sun, Li Li, Qingli Shanghai Key Laboratory of Multidimensional Information Processing Shanghi China Key Laboratory of Advanced Theory and Application in Statistics & Data Science East China Normal University Shanghai200241 China

View synthesis is usually done by an autoencoder, in which the encoder maps a source view image into a latent content code, and the decoder transforms it into a target view image according to the condition. However, the source contents are often not well kept in this setting, which leads to unnecessary changes during the view translation. Although adding skipped connections, like Unet, alleviates the problem, but it often causes the failure on the view conformity. This paper proposes a new architecture by performing the source-to-target deformation in an iterative way. Instead of simply incorporating the features from multiple layers of the encoder, we design soft and hard deformation modules, which warp the encoder features to the target view at different resolutions, and give results to the decoder to complement the details. Particularly, the current warping flow is not only used to align the feature of the same resolution, but also as an approximation to coarsely deform the high resolution feature. Then the residual flow is estimated and applied in the high resolution, so that the deformation is built up in the coarse-to-fine fashion. To better constrain the model, we synthesize a rough target view image based on the intermediate flows and their warped features. The extensive ablation studies and the final results on two different data sets show the effectiveness of the proposed model. https://***/MingyuY/Iterative-view-synthesis © 2021, CC BY.

关键词： Signal encoding

Regional consistency evaluation and sample size calculation under two MRCTs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qing, Kunhai Ren, Xinru Xu, Jin School of Statistics East China Normal University Shanghai China Key Laboratory of Advanced Theory and Application in Statistics and Data Science - MOE East China Normal University Shanghai China

Multi-regional clinical trial (MRCT) has been common practice for drug development and global registration. The FDA guidance "Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products Guidance for Industry" (FDA, 2019) requires that substantial evidence of effectiveness of a drug/biologic product to be demonstrated for market approval. In the situations where two pivotal MRCTs are needed to establish effectiveness of a specific indication for a drug or biological product, a systematic approach of consistency evaluation for regional effect is crucial. In this paper, we first present some existing regional consistency evaluations in a unified way that facilitates regional sample size calculation under the simple fixed effect model. Second, we extend the two commonly used consistency assessment criteria of MHLW (2007) in the context of two MRCTs and provide their evaluation and regional sample size calculation. Numerical studies demonstrate the proposed regional sample size attains the desired probability of showing regional consistency. A hypothetical example is provided for illustration of application. We provide an R package for implementation. Copyright © 2024, The Authors. All rights reserved.

关键词：