Self-supervised learning (SSL) aims to learn feature representation without human-annotated data. Existing methods approach this goal by encouraging the feature representations to be invariant under a set of task-irre...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
Self-supervised learning (SSL) aims to learn feature representation without human-annotated data. Existing methods approach this goal by encouraging the feature representations to be invariant under a set of task-irrelevant transformations and distortions defined a priori. However, multiple studies have shown that such an assumption often limits the expressive power of the representations and model would perform poorly when downstream tasks violate this assumption. For example, being invariant to rotations would prevent features from retaining enough information to estimate object rotation angles. This suggests additional manual work and domain knowledge are required for selecting augmentation types during SSL. In this work, we relax the transformation-invariance assumption by introducing a SSL framework that encourages the feature representations to preserve the order of transformation scale in embedding space for some transformations while maintaining invariance to other transformations. This allows the learned feature representations to retain information about task-relevant transformations. In addition, this framework gives rise to a handy mechanism to determine the augmentation types to which the features representations should be invariant and equivariant during SSL. We demonstrate the effectiveness of our method on various datasets such as Fruits 360, Caltech-UCSD Birds 200, and Blood cells dataset.
This paper presents an approach to assess the perfusion of visible human tissue from RGB video files. We propose metrics derived from remote photoplethysmography (rPPG) signals to detect whether a tissue is adequately...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
This paper presents an approach to assess the perfusion of visible human tissue from RGB video files. We propose metrics derived from remote photoplethysmography (rPPG) signals to detect whether a tissue is adequately supplied with blood. The perfusion analysis is done in three different scales, offering a flexible approach for different applications. We perform a plane-orthogonal-to-skin rPPG independently for locally defined regions of interest on each scale. From the extracted signals, we derive the signal-to-noise ratio, magnitude in the frequency domain, heart rate, perfusion index as well as correlation between specific rPPG signals in order to locally assess the perfusion of a specific region of human tissue. We show that locally resolved rPPG has a broad range of applications. As exemplary applications, we present results in intraoperative perfusion analysis and visualization during skin and organ transplantation as well as an application for liveliness assessment for the detection of presentation attacks to authentication systems.
Yield forecasting has been a central task in computational agriculture because of its impact on agricultural management from the individual farmer to the government level. With advances in remote sensing technology, c...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
Yield forecasting has been a central task in computational agriculture because of its impact on agricultural management from the individual farmer to the government level. With advances in remote sensing technology, computational processing power, and machine learning, the ability to forecast yield has improved substantially over the past years. However, most previous work has been done leveraging low-resolution satellite imagery and forecasting yield at the region, county, or occasionally farm-level. In this work, we use high-resolution aerial imagery and output from high-precision harvesters to predict in-field harvest values for corn-raising farms in the US Midwest. By using the harvester information, we are able to cast the problem of yield-forecasting as a density estimation problem and predict a harvest rate, in bushels/acre, at every pixel in the field image. This approach provides the farmer with a detailed view of which areas of the farm may be performing poorly so he can make the appropriate management decisions in addition to providing an improved prediction of total yield. We evaluate both traditional machine learning approaches with hand-crafted features alongside deep learning methods. We demonstrate the superiority of our pixel-level approach based on an encoder-decoder framework which produces a 5.41% MAPE at the field-level.
Unsupervised Domain Adaptation (UDA) deals with transferring knowledge from labeled source domains to an unlabeled target domain under domain shift. However, this does not reflect the breadth of scenarios that arise i...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
Unsupervised Domain Adaptation (UDA) deals with transferring knowledge from labeled source domains to an unlabeled target domain under domain shift. However, this does not reflect the breadth of scenarios that arise in real-world applications since source domains could increase. A plausible conjecture is: can we train a life-long learning model learned on continuous source domains given the target without the presence of labels? We formalize this task as the Continuous Domain Adaptation (CDA) and empirically show that conventional domain adaptation methods may suffer severe generalization deterioration due to the limited incremental transferability and negative transfer. To tackle this issue, we propose a novel sample-to-sample framework-Consolidationand-Exploration Network (CENet) to facilitate incremental transferring. This method underscores both the qualitative and quantitative relationship between samples. Moreover, we conduct comprehensive experiments to evaluate the effectiveness of each component in our pair-based method. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.
Social media holds the power to influence a significant change in the population. Through social media, people all around the world can connect and share their views. However, this social space is now infected due to ...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
Social media holds the power to influence a significant change in the population. Through social media, people all around the world can connect and share their views. However, this social space is now infected due to the infiltration of fraudulent, obscene, fake and possibly, influential media. According to a UNESCO report, prevalence of fake news and deepfake content possess the potential of spreading fake propaganda and can lead to political and social unrest. Trust on social media is an emerging problem and there is an urgent need to address the same. There has been some research around approaches that detect fake news and deepfakes, however, identification of the source of these deepfakes posted on social media platforms is an equally important but relatively unexplored challenge. This paper proposes a novel Deepfake Source Identification (DeSI) algorithm that identifies the sources of deepfakes posted on Twitter. The proposed DeSI algorithm allows for two input modalities - text and images. We rigorously test our algorithm in both constrained and unconstrained experimental setups and report the observed results. In the constrained setting, the algorithm correctly identifies all the deepfake tweets as well their sources. The complete framework is further encased in a web portal to facilitate intuitive use and analysis of the results.
Consistency regularization on label predictions becomes a fundamental technique in semi-supervised learning, but it still requires a large number of training iterations for high performance. In this study, we analyze ...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
Consistency regularization on label predictions becomes a fundamental technique in semi-supervised learning, but it still requires a large number of training iterations for high performance. In this study, we analyze that the consistency regularization restricts the propagation of labeling information due to the exclusion of samples with unconfident pseudo-labels in the model updates. Then, we propose contrastive regularization to improve both efficiency and accuracy of the consistency regularization by well-clustered features of unlabeled data. In specific, after strongly augmented samples are assigned to clusters by their pseudolabels, our contrastive regularization updates the model so that the features with confident pseudo-labels aggregate the features in the same cluster, while pushing away features in different clusters. As a result, the information of confident pseudo-labels can be effectively propagated into more unlabeled samples during training by the well-clustered features. On benchmarks of semi-supervised learning tasks, our contrastive regularization improves the previous consistency-based methods and achieves state-of-the-art results, especially with fewer training iterations. Our method also shows robust performance on open-set semi-supervised learning where unlabeled data includes out-of-distribution samples.
Remote sensing data is plentiful, but downloading, organizing, and transforming large amounts of data into a format readily usable by modern machine learning methods is a challenging and labor-intensive task. We prese...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
Remote sensing data is plentiful, but downloading, organizing, and transforming large amounts of data into a format readily usable by modern machine learning methods is a challenging and labor-intensive task. We present the OpenSentinelMap dataset, which consists of 137,045 unique 3.7 km(2) spatial cells, each containing multiple multispectral Sentinel-2 images captured over a 4 year time period and a set of corresponding per-pixel semantic labels derived from OpenStreetMap data. The labels are not necessarily mutually exclusive, and contain information about roads, buildings, water, and 12 land-use categories. The spatial cells are selected randomly on a global scale over areas of human activity, without regard to OpenStreetMap data availability or quality, making the dataset ideal for both supervised, semi-supervised, and unsupervised experimentation. To demonstrate the effectiveness of the dataset, we a) train an off-the-shelf convolutional neural network with minimal modification to predict land-use and building and road location from multispectral Sentinel-2 imagery and b) show that the learned embeddings are useful for downstream fine-grained classification tasks without any fine-tuning. The dataset is publicly available at https://***/open-sentinel-map/.
In Unsupervised Domain Adaptation (UDA), a network is trained on a source domain and adapted on a target domain where no labeled data is available. Existing UDA techniques consider having the entire target domain avai...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
In Unsupervised Domain Adaptation (UDA), a network is trained on a source domain and adapted on a target domain where no labeled data is available. Existing UDA techniques consider having the entire target domain available at once, which may not be feasible during deployment in realistic settings where batches of target data are acquired over time. Continual Learning (CL) has been dealing with data constrained paradigms in a supervised manner, where batches of labeled samples are sequentially presented to the network and the network continually learns from the new data without forgetting what was previously learned. Our method for unsupervised continual learning serves as a bridge between the UDA and CL paradigms. This research addresses a gradually evolving target domain fragmented into multiple sequential batches where the model continually adapts to the gradually varying stream of data in an unsupervised manner. To tackle this challenge, we propose a source free method based on episodic memory replay with buffer management. A contrastive loss is incorporated for better alignment of the buffer samples and the continual stream of batches. Our experiments on the rotating MNIST and CORe50 datasets confirm the benefits of our unsupervised continual learning method for gradually varying domains.
The evaluation of human epidermal growth factor receptor 2 (HER2) expression is essential to formulate a precise treatment for breast cancer. The routine evaluation of HER2 is conducted with immunohistochemical techni...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
The evaluation of human epidermal growth factor receptor 2 (HER2) expression is essential to formulate a precise treatment for breast cancer. The routine evaluation of HER2 is conducted with immunohistochemical techniques (IHC), which is very expensive. Therefore, for the first time, we propose a breast cancer immunohistochemical (BCI) benchmark attempting to synthesize IHC data directly with the paired hematoxylin and eosin (HE) stained images. The dataset contains 4870 registered image pairs, covering a variety of HER2 expression levels. Based on BCI, as a minor contribution, we further build a pyramid pix2pix image generation method, which achieves better HE to IHC translation results than the other current popular algorithms. Extensive experiments demonstrate that BCI poses new challenges to the existing image translation research. Besides, BCI also opens the door for future pathology studies in HER2 expression evaluation based on the synthesized IHC images. BCI dataset can be downloaded from https://***/BCI.
In pathology, tissue samples are assessed using multiple staining techniques to enhance contrast in unique histologic features. In this paper, we introduce a multimodal CNN-GNN based graph fusion approach that leverag...
详细信息
ISBN:
(数字)9781665487399
ISBN:
(纸本)9781665487399
In pathology, tissue samples are assessed using multiple staining techniques to enhance contrast in unique histologic features. In this paper, we introduce a multimodal CNN-GNN based graph fusion approach that leverages complementary information from multiple non-registered histopathology images to predict pathologic scores. We demonstrate this approach in nonalcoholic steatohepatitis (NASH) by predicting CRN fibrosis stage and NAFLD Activity Score (NAS). Primary assessment of NASH typically requires liver biopsy evaluation on two histological stains: Trichrome (TC) and hematoxylin and eosin (H&E). Our multimodal approach learns to extract complementary information from TC and H&E graphs corresponding to each stain while simultaneously learning an optimal policy to combine this information. We report up to 20% improvement in predicting fibrosis stage and NAS component grades over single-stain modeling approaches, measured by computing linearly weighted Cohen's kappa between machinederived vs. pathologist consensus scores. Broadly, this paper demonstrates the value of leveraging diverse pathology images for improved ML-powered histologic assessment.
暂无评论