Even though pre-trained language models like BERT and XLNet have produced significant consequences on a variety of tasks of natural language processing, they are difficult to deploy in practical applications due to th...
详细信息
To fuse vocabulary features into the pre-training model is the mainstream data feature processing method for sequence labelling tasks. In general, the feature fusion methods that have been proposed at present are dire...
To fuse vocabulary features into the pre-training model is the mainstream data feature processing method for sequence labelling tasks. In general, the feature fusion methods that have been proposed at present are direct fusion outside the pre-training model or fusion of lexical features using attention mechanism. However, the study found that this way of vocabulary enhancement does not conform to the word formation rules of modern Chinese. In the Chinese language, it is easy to fuse irrelevant or even incorrect lexical features into the sequence using the above feature processing methods, which is bad for the experimental results of the Chinese sequence labelling task. To solve these problems, we propose to use Cosine Similarity Adapter to process lexical features in Chinese sequence labelling tasks. CSBERT is a hybrid model using this structure based on BERT, which conforms to the word formation rules of modern Chinese to a certain extent. It can fuse the features of the word into the character or eliminate the features of the word in the character according to the cosine similarity between the character vector and a word vector. The experimental results show that CSBERT has better ability to label Chinese sequences than the benchmark model. CSBERT has achieved the best experimental results such as F1-Score on 7 open datasets and the best ability of multi-label classification, which proves that the model has good practical value.
Graph similarity search is an important research problem in many applications, such as finding result graphs that have a similar structure to a given entity in biochemistry, data mining, and pattern recognition. Top-k...
详细信息
This paper addresses the limitations of the Contrastive Language-Image Pre-training (CLIP) model’s image encoder and proposes a segmentation model WSSS-ECFE with enhanced CLIP feature extraction, aiming to improve th...
详细信息
ISBN:
(数字)9798350368741
ISBN:
(纸本)9798350368758
This paper addresses the limitations of the Contrastive Language-Image Pre-training (CLIP) model’s image encoder and proposes a segmentation model WSSS-ECFE with enhanced CLIP feature extraction, aiming to improve the performance of the Weakly Supervised Semantic Segmentation (WSSS) task. WSSS-ECFE employs the Enhanced Bottleneck module proposed in this paper and adds dynamic residual connection to improve the model’s processing effect on complex scenes. In terms of implementation, the Enhanced Bottleneck module employs the Swish activation function and the Depthwise Separable Convolution to enhance the feature extraction and segmentation capability of the model, and uses multiple attention mechanisms to further optimize the feature representation and segmentation accuracy. The WSSS task on the public datasets PASCAL VOC 2012 and MS COCO 2014 achieves 82.6% and 56.3% mean intersection over union (mIoU), achieving state-of-the-art performance in models with low resource requirements.
Community search is to explore valuable target community structure from a large social network. In real community, every point has a geographic location information, and many edges are uncertain. Such a network is cal...
详细信息
With the prevalence of deep learning, people use multi-modality information for interpretation and reasoning. In this paper, a cross-modality encoder CMEEA (cross-modality encoder representation based on external atte...
详细信息
With the prevalence of deep learning, people use multi-modality information for interpretation and reasoning. In this paper, a cross-modality encoder CMEEA (cross-modality encoder representation based on external attention mechanism) is adopted to improve the accuracy of the model. The two external storage units of CMEEA can be regarded as a dictionary of the whole dataset to improve the performance of the network, while learning more representative features of the input and reducing the computational cost. This paper uses five pretraining tasks that help the model learn internal modal and cross-modal relationships. It also verifies the generalizability of the model by applying the pretraining cross-modal model to the visual inference task NLVR 2 , and improves the previous results by 0.1%. By fine-tuning the pretrain parameters, the model improves by 1.3% on visual question answering (VQA).
The proposed method in this paper proposes an end-to-end unsupervised semantic segmentation architecture DMSA based on four loss functions. The framework uses Atrous Spatial Pyramid Pooling (ASPP) module to enhance fe...
详细信息
The proposed method in this paper proposes an end-to-end unsupervised semantic segmentation architecture DMSA based on four loss functions. The framework uses Atrous Spatial Pyramid Pooling (ASPP) module to enhance feature extraction. At the same time, a dynamic dilation strategy is designed to better capture multi-scale context information. Secondly, a Pixel-Adaptive Refinement (PAR) module is introduced, which can adaptively refine the initial pseudo labels after feature fusion to obtain high quality pseudo labels. Experiments show that the proposed DSMA framework is superior to the existing methods on the saliency dataset. On the COCO 80 dataset, the MIoU is improved by 2.0, and the accuracy is improved by 5.39. On the Pascal VOC 2012 Augmented dataset, the MIoU is improved by 4.9, and the accuracy is improved by 3.4. In addition, the convergence speed of the model is also greatly improved after the introduction of the PAR module.
The phenomenon that user interests change dynamically over time is called user interest drift, which makes it difficult for recommendation systems to obtain users’ real interests. In order to better provide users wit...
The phenomenon that user interests change dynamically over time is called user interest drift, which makes it difficult for recommendation systems to obtain users’ real interests. In order to better provide users with personalized services, it is very important to study recommendation methods that can be applied to user interest drift. However, the new interests of users are often very different from the interests that users have interacted with historically. It makes the correlation between the original user’s historical interaction and the user’s new interest very weak. Meanwhile, the information contained in user’s sessions is not enough to support the inference of the new interest, resulting in a serious decrease in model performance. To alleviate the above problem, we propose a bidirectional representation recommendation based on relevance information enhancement to solve user interest drift (RA-Bert). The model is based on Bert and uses bidirectional self-attention mechanism to represent user sessions. In this model, we design a relevance information enhancement module to adaptively capture the information associated with the user’s new interest from neighbors’ sessions. Then, we integrate the relevance information into user’s history sessions, which can effectively enhance the correlation between user’s history sessions and the new interest. Further, in order to effectively supplement the missing relevance information in users’ sessions, we design a new attention unit, which explicitly models the difference information between users and neighbors. Extensive experiments have demonstrated the effectiveness of our approach. Our code is publicly available at the link: https://***/LucasZen/paper_code.
Different from the traditional recommendation system which regards a single item as an atomic unit, the purpose of bundle recommendation is to recommend a set of items to users. In recent years, some work often uses t...
Different from the traditional recommendation system which regards a single item as an atomic unit, the purpose of bundle recommendation is to recommend a set of items to users. In recent years, some work often uses the model based on GNN to solve the bundle recommendation problem. However, compared with the traditional recommendation models, the GNN-based bundle recommendation has a more serious data sparse problem, which significantly reduces the performance of the model. Meanwhile, due to the lack of modeling ability of complex interactive information and dependent information, it is still far from enough to learn the representation of users and bundles. In view of the advantages of contrast learning in mining its own supervisory signals, we propose a model named Cross-level relational graph Contrastive Learning for Bundle Recommendation (CL 2 BRec). Different from previous contrastive learning methods, our method learns representations of users and bundles from multiple relational graphs of different levels through a crossview contrastive learning paradigm. Specifically, we regard the user-item/bundle graph as a local interaction relation view, the bundle-item graph as a local dependency relation view, and the user-bundle-item tripartite graph as a global structure view to build a cross-level bundle recommendation model. Extensive experiments conducted on two benchmark datasets demonstrate that CL 2 BRec outperforms the state-of-the-art methods by over 12.56% and 8.6%, respectively.
作者：
Yang, KunLu, JunCollege of Computer Science and Technology
Heilongjiang University 150080 Harbin China Jiaxiang Industrial Technology Research Institute of HLJU Jining Shandong Province China Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin China
The proposed method in this paper proposes an end-to-end unsupervised semantic segmentation architecture DMSA based on four loss functions. The framework uses Atrous Spatial Pyramid Pooling (ASPP) module to enhance fe...
详细信息
暂无评论