In this paper we consider fully online learning algorithms for classification generated from Tikhonov regularization schemes associated with general convex loss functions and reproducing kernel Hilbert spaces. For suc...
详细信息
In this paper we consider fully online learning algorithms for classification generated from Tikhonov regularization schemes associated with general convex loss functions and reproducing kernel Hilbert spaces. For such a fully online algorithm, the regularization parameter in each learning step changes. This is the essential difference from the partially online algorithm which uses a fixed regularization parameter. We first present a novel approach to the drift error incurred by the change of the regularization parameter. Then we estimate the error of the learning process for the strong approximation in the reproducing kernel Hilbert space. Finally, learning rates are derived from decays of the regularization error. The convexity of the loss function plays an important role in our analysis. Concrete learning rates are given for the hinge loss and the support vector machine q-norm loss. (c) 2007 Elsevier Inc. All rights reserved.
We introduce a new model addressing feature selection from a large dictionary of variables that can be computed from a signal or an image. Features are extracted according to an efficiency criterion, on the basis of s...
详细信息
We introduce a new model addressing feature selection from a large dictionary of variables that can be computed from a signal or an image. Features are extracted according to an efficiency criterion, on the basis of specified classification or recognition tasks. This is done by estimating a probability distribution P on the complete dictionary, which distributes its mass over the more efficient, or informative, components. We implement a stochastic gradient descent algorithm, using the probability as a state variable and optimizing a multi-task goodness of fit criterion for classifiers based on variable randomly chosen according to P. We then generate classifiers from the optimal distribution of weights learned on the training set. The method is first tested on several pattern recognition problems including face detection, handwritten digit recognition, spam classification and micro-array analysis. We then compare our approach with other step-wise algorithms like random forests or recursive feature elimination.
A family of classification algorithms generated from Tikhonov regularization schemes are considered. They involve mufti-kernel spaces and general convex loss functions. Our main purpose is to provide satisfactory esti...
详细信息
A family of classification algorithms generated from Tikhonov regularization schemes are considered. They involve mufti-kernel spaces and general convex loss functions. Our main purpose is to provide satisfactory estimates for the excess misclassification error of these mufti-kernel regularized classifiers when the loss functions achieve the zero value. The error analysis consists of two parts: regularization error and sample error. Allowing mufti-kernels in the algorithm improves the regularization error and approximation error, which is one advantage of the mufti-kernel setting. For a general loss function, we show how to bound the regularization error by the approximation in some weighted L-q spaces. For the sample error, we use a projection operator. The projection in connection with the decay of the regularization error enables us to improve convergence rates in the literature even for the one-kernel schemes and special loss functions: least-square loss and hinge loss for support vector machine soft margin classifiers. Existence of the optimization problem for the regularization scheme associated with multi-kernels is verified when the kernel functions are continuous with respect to the index set. Concrete examples, including Gaussian kernels with flexible variances and probability distributions with some noise conditions, are used to illustrate the general theory. (c) 2006 Elsevier Inc. All rights reserved.
This paper explores the problem of the construction of imputation classes using the score method, sometimes called predictive mean stratification or response propensity stratification, depending on the context. This m...
详细信息
This paper explores the problem of the construction of imputation classes using the score method, sometimes called predictive mean stratification or response propensity stratification, depending on the context. This method was studied in Thomsen (1973), Little (1986) and Eltinge & Yansaneh (1997). We use a different framework to evaluate the properties of the resulting imputed estimator of a population mean. In our framework, we condition on the realized sample. This enables us to considerably simplify our theoretical developments in the frequent situation where the boundaries and the number of classes are sample-dependent. We find that the key factor for reducing the non-response bias is to form classes homogeneous with respect to the response probabilities and/or the conditional expectation of the variable of interest. In the latter case, the non-response/imputation variance is also reduced. Finally, we performed a simulation study to fully evaluate various versions of the score method and to compare them with a cross-classification method, which is frequently used in practice. The results showed the superiority of the score method in general.
Skin cancer is the most common form of cancer in the United States. Although melanoma accounts for just 11% of all types of skin cancer, it is responsible for most of the deaths, claiming more than 7910 lives annually...
详细信息
ISBN:
(纸本)9780819466327
Skin cancer is the most common form of cancer in the United States. Although melanoma accounts for just 11% of all types of skin cancer, it is responsible for most of the deaths, claiming more than 7910 lives annually. Melanoma is visually difficult for clinicians to differentiate from Clark nevus lesions which are benign. The application of pattern recognition techniques to these lesions may be useful as an educational tool for teaching physicians to differentiate lesions, as well as for contributing information about the essential optical characteristics that identify them. Purpose: This study sought to find the most effective features to extract from melanoma, melanoma in situ and Clark nevus lesions, and to find the most effective pattern-classification criteria and algorithms for differentiating those lesions, using the Computer Vision and Image Processing Tools (CVIPtools) software package. Methods: Due to changes in ambient lighting during the photographic process, color differences between images can occur. These differences were minimized by capturing dermoscopic images instead of photographic images. Differences in skin color between patients were minimized via image color normalization, by converting original color images to relative-color images. Relative-color images also helped minimize changes in color that occur due to changes in the photographic and digitization processes. Tumors in the relative-color images were segmented and morphologically filtered. Filtered, relative-color, tumor features were then extracted and various pattern-classification schemes were applied. Results: Experimentation resulted in four useful pattern classification methods, the best of which was an overall classification rate of 100% for melanoma and melanoma in situ (grouped) and 60% for Clark nevus. Conclusion: Melanoma and melanoma in situ have feature parameters and feature values that are similar enough to be considered one class of tumor that significantly differs from
This paper considers online classification learning algorithms based on regularization schemes in reproducing kernel Hilbert spaces associated with general convex loss functions. A novel capacity independent approach ...
详细信息
This paper considers online classification learning algorithms based on regularization schemes in reproducing kernel Hilbert spaces associated with general convex loss functions. A novel capacity independent approach is presented. It verifies the strong convergence of the algorithm under a very weak assumption of the step sizes and yields satisfactory convergence rates for polynomially decaying step sizes. Explicit learning rates with respect to the misclassification error are given in terms of the choice of step sizes and the regularization parameter (depending on the sample size). Error bounds associated with the hinge loss, the least square loss, and the support vector machine q-norm loss are presented to illustrate our method.
The quick diagnosis of Burkitt lymphoma (BL) and its clear-cut differentiation from diffuse large B-cell lymphoma (DLBCL) is of great clinical importance because treatment strategies for these two disease entities dif...
详细信息
The quick diagnosis of Burkitt lymphoma (BL) and its clear-cut differentiation from diffuse large B-cell lymphoma (DLBCL) is of great clinical importance because treatment strategies for these two disease entities differ markedly. As these two lymphomas are difficult to distinguish using the current World Health Organization classification, we studied 39 cases of highly proliferative peripheral blastic B-cell lymphoma (HPBCL) to establish a practical differential-diagnostic algorithm. Characteristics set for BL were a typical morphology, a mature B-cell phenotype of CD10(+), Bcl-6(+) and Bcl-2(-) tumour cells, a proliferation rate of > 95%, and the presence of C-MYC rearrangements in the absence of t(14;18)(q32;q21). Altogether, these characteristics were found in only five of 39 cases, whereas the majority of tumours revealed mosaic features. We then followed a pragmatic stepwise approach for a classification algorithm that included the assessment of C-MYC status to stratify HPBCL into four predefined diagnostic categories (DC), namely DC I (5/39, 12.8%): 'classical BL', DC II (11/39, 28.2%): 'atypical BL', DC III (9/39, 23.1%): 'C-MYC+ DLBCL' and DC IV (14/39, 35.9%): 'C-MYC- HPBCL'. This proposal may serve as a robust and objective operational basis for therapeutic decisions for HPBCL within 1 week and is applicable to be evaluated for its prognostic relevance in clinical trials with uniformly treated patients.
Recently, new machine learning classifiers for the prediction of linear B-cell epitopes were presented. Here we show the application of Receiver Operator Characteristics (ROC) convex hulls to select optimal classifier...
详细信息
Recently, new machine learning classifiers for the prediction of linear B-cell epitopes were presented. Here we show the application of Receiver Operator Characteristics (ROC) convex hulls to select optimal classifiers as well as possibilities to improve the post test probability (PTP) to meet real world requirements such as high throughput epitope screening of whole proteomes. The major finding is that ROC convex hulls present an easy to use way to rank classifiers based on their prediction conservativity as well as to select candidates for ensemble classifiers when validating against the antigenicity profile of 10 HIV-1 proteins. We also show that linear models are at least equally efficient to model the available data when compared to multi-layer feed-forward neural networks. Copyright (c) 2006 John Wiley & Sons, Ltd.
Identification and characterization of antigenic determinants on proteins has received considerable attention utilizing both, experimental as well as computational methods. For computational routines mostly structural...
详细信息
Identification and characterization of antigenic determinants on proteins has received considerable attention utilizing both, experimental as well as computational methods. For computational routines mostly structural as well as physicochemical parameters have been utilized for predicting the antigenic propensity of protein sites. However, the performance of computational routines has been low when compared to experimental alternatives. Here we describe the construction of machine learning based classifiers to enhance the prediction quality for identifying linear B-cell epitopes on proteins. Our approach combines several parameters previously associated with antigenicity, and includes novel parameters based on frequencies of amino acids and amino acid neighborhood propensities. We utilized machine learning algorithms for deriving antigenicity classification functions assigning antigenic propensities to each amino acid of a given protein sequence. We compared the prediction quality of the novel classifiers with respect to established routines for epitope scoring, and tested prediction accuracy on experimental data available for HIV proteins. The major finding is that machine learning classifiers clearly outperform the reference classification systems on the HIV epitope validation set. Copyright (c) 2006 John Wiley & Sons, Ltd.
暂无评论