检索结果-内蒙古大学图书馆

Data-driven safety management of worker-equipment interactions using visual relationship detection and semantic analysis

引用

AUTOMATION IN CONSTRUCTION 2025年 175卷

作者： Liu, Yipeng Wang, Junwu Torbaghan, Mehran Eskandari Wuhan Univ Technol Sanya Sci & Educ Innovat Pk Sanya 572025 Peoples R China Wuhan Univ Technol Sch Civil Engn & Architecture Wuhan 430070 Peoples R China Univ Birmingham Dept Civil Engn Birmingham B15 2TT England

Existing technologies struggle to accurately identify interactions between workers and equipment, as well as the deep semantics of complex construction scenes. To address these limitations, this paper proposes an automated construction site safety management system designed to enhance scene understanding and identify safety hazards while focusing on hazard-area and personal protective equipment (PPE) interaction. The system transforms image information into worker-centric triplets and generates precise textual descriptions through semantic enhancement, enabling effective scene analysis. By comparing the generated descriptions with predefined hazard statements, the system identifies potential risks. Experimental results demonstrate a 9.6 % improvement in recall for Ng-mR@K metrics (K = 20, 50, 100). Additionally, the system successfully filters over 90 % of invalid relationships, achieving 83.7 % accuracy in semantic similarity matching, significantly enhancing detection precision and semantic understanding. By advancing from object detection to a structured image-to-triplet-to-text framework, this paper offers an efficient and reliable solution for automated construction site safety management.

关键词： Construction site safety Multi-source data integration visual relationship detection Semantic triplets Image-to-text transformation Semantic enhancement

来源：评论

学校读者我要写书评

暂无评论

Approach to Deep-learning-based visual relationship detection for Scene Analysis

引用

SENSORS AND MATERIALS 2025年第3期37卷 1193-1209页

作者： Shieh, Ming-Yuan Wu, Po-Kuan Pai, Neng-Sheng Natl Chin Yi Univ Technol Dept Elect Engn Taichung 41170 Taiwan

The paper is focused on the implementation of a deep-learning-based visual relationship detection system for scene analysis. Initially, the system employs convolutional neural networks (CNNs) for precise object detection and localization, accurately capturing detailed information about the objects in the scene. Following this, the system applies the vision transformer model to infer relationships between objects, enabling detailed analysis, interpretation of spatial layouts, and understanding of behavioral interactions among objects within the scene. This process allows the system to gain a profound understanding of the relationships and interaction patterns among the objects. Furthermore, the system integrates an intuitive and feature-rich user interface to display detailed relationships among objects and the model's inference outcomes. We not only conduct scene analysis but also provide functionality for users to select two distinct subjects for visual relationship detection. Users can choose two subjects of interest to explore their relationships with other objects in the same scene. Through this selection capability, the system analyzes the selected subjects while excluding results that do not meet specified criteria. This functionality enables users to precisely control the analysis direction, ensuring that the results align with their expectations and requirements effectively.

关键词： visual relationship detection scene analysis convolutional neural networks vision transformer

来源：评论

学校读者我要写书评

暂无评论

visual relationship detection With visual-Linguistic Knowledge From Multimodal Representations

引用

IEEE ACCESS 2021年 9卷 50441-50451页

作者： Chiou, Meng-Jiun Zimmermann, Roger Feng, Jiashi Natl Univ Singapore Dept Comp Sci Singapore 117417 Singapore Natl Univ Singapore Dept Elect & Comp Engn Singapore 117583 Singapore

visual relationship detection aims to reason over relationships among salient objects in images, which has drawn increasing attention over the past few years. Inspired by human reasoning mechanisms, it is believed that external visual commonsense knowledge is beneficial for reasoning visual relationships of objects in images, which is however rarely considered in existing methods. In this paper, we propose a novel approach named Relational visual-Linguistic Bidirectional Encoder Representations from Transformers (RVL-BERT), which performs relational reasoning with both visual and language commonsense knowledge learned via self-supervised pre-training with multimodal representations. RVL-BERT also uses an effective spatial module and a novel mask attention module to explicitly capture spatial information among the objects. Moreover, our model decouples object detection from visual relationship recognition by taking in object names directly, enabling it to be used on top of any object detection system. We show through quantitative and qualitative experiments that, with the transferred knowledge and novel modules, RVL-BERT achieves competitive results on two challenging visual relationship detection datasets. The source code is available at https://***/coldmanck/RVL-BERT.

关键词： visualization Linguistics Bit error rate Task analysis Cognition Predictive models Object detection Computer vision image analysis visual relationship detection scene graph generation multimodal representation

来源：评论

学校读者我要写书评

暂无评论

Knowledge Enhanced Zero-Shot visual relationship detection 1

引用

17th International Conference on Knowledge Science, Engineering and Management (KSEM)

作者： Ding, Nan Lai, Yong Liu, Jie Jilin Univ Coll Comp Sci & Technol Changchun 130012 Peoples R China Jilin Univ Key Lab Symbol Computat & Knowledge Engn Minist Educ Changchun 130012 Peoples R China

ISBN: (数字)9789819754984

ISBN: (纸本)9789819754977;9789819754984

In visual relationship detection (VRD), the diversity of relationships often results in many unseen (i.e. zero-shot) relationships in the test set. Predicting zero-shot relationships poses a significant challenge. Traditional methods often incorporate semantic knowledge and spatial features from images, which still rely on priors in language and annotations in the dataset. Therefore, we propose encoding spatial structure information in the knowledge graph and incorporating spatial relationships from commonsense to guide predictions. The model comprises two modules: logic tensor networks that encoded the negative domain of semantic and spatial knowledge, and a commonsense knowledge graph module updated by local spatial structure as positive domain semantic knowledge. Predictions are further constrained by region connection calculus (RCC). Experimental results demonstrate competitive performance on the visual relationship Datasets under the zero-shot setting and the entire subset of visual Genome. In predicate detection, it achieves comparable results to benchmarks, while significantly outperforming benchmarks in relationship and phrase detection.

关键词： visual relationship detection Zero-shot Commonsense knowledge graph Region connection calculus Logic tensor networks

来源：评论

学校读者我要写书评

暂无评论

Optimizing Continuous Prompts for visual relationship detection by Affix-Tuning

引用

IEEE ACCESS 2022年 10卷 70104-70112页

作者： Xiao, Shouguan Fu, Weiping Xian Univ Technol Sch Mech & Precis Instrument Engn Xian 710048 Peoples R China Xian Int Univ Sch Engn Xian 710077 Peoples R China

visual relationship detection is crucial for understanding visual scenes and is widely used in many areas, including visual navigation, visual question answering, and machine trouble detection. Traditional detection methods often fuse multiple region modules, which takes considerable time and resources to train every module with extensive samples. As every module is independent, the computation process has difficulty achieving unity and lacks a higher level of logical reasonability. In response to the above problems, we propose a novel method of affix-tuning transformers for visual relationship detection tasks, which keeps transformer model parameters frozen and optimizes a small continuous task-specific vector. It not only makes the model unified and reduces the training cost but also maintains the common-sense reasonability without multiscale training. In addition, we design a vision-and-language sentence expression prompt template and train a few transformer model parameters for downstream tasks. Our method, Prompt Template and Affix-Tuning Transformers (PTAT), is evaluated on visual relationship detection and visual Genome datasets. Finally, the results of the proposed method are close to or even higher than those of the state-of-the-art methods on some evaluation metrics.

关键词： visualization Task analysis Licenses Transformers Training Training data Semantics visual relationship detection prompt template affix-tuning transformers

来源：评论

学校读者我要写书评

暂无评论

Fixed-Size Objects Encoding for visual relationship detection

引用

NEURAL PROCESSING LETTERS 2022年第4期54卷 3249-3261页

作者： Pan, Hengyue Niu, Xin Shen, Siqi Chen, Yixin Qiao, Peng Huang, Zhen Li, Dongsheng Natl Univ Def Technol 109 Deya St Changsha Hunan Peoples R China Xiamen Univ 422 Siming South Rd Xiamen Fujian Peoples R China

In this paper, we propose a fixed-size object encoding method called FOE-VRD to improve performance of visual relationship detection tasks. For each relationship triplet in a given image, FOE-VRD not only considers the subject and object, but also uses one fixed-size vector to encoding all background objects of the image. In this way, we introduce more background knowledge to assist the relationship detector for better performance. We firstly use a regular convolution neural network as a feature extractor to generate high-level features of input images. Then, for each relationship triplet, we apply ROI-pooling as the feature generator on the bounding boxes of subject and object to get two corresponding feature vectors. Moreover, we propose a novel method to encode all background objects in each image by using one fixed-size vector (i.e., FBE vector). By concatenating the 3 generated feature vectors, we successfully encode the relationship using one fixed-size vector. The generated feature vector is then feed into a fully connected neural network to get the predicate classification result. Experimental results on VRD and visual Genome databases show that the proposed method works well on both predicate classification and relationship detection tasks, especially on the situation of zero-shot detection.

关键词： Objects encoding visual relationship detection Deep learning

来源：评论

学校读者我要写书评

暂无评论

Naive Scene Graphs : How visual is Modern visual relationship detection? 20

Naive Scene Graphs : How Visual is Modern Visual Relationshi...

引用

20th Conference on Robots and Vision (CRV)

作者： Abou Chacra, David Zelek, John Univ Waterloo Dept Syst Design Engn Waterloo ON Canada

ISBN: (纸本)9798350341393

Modern approaches to scene graph generation still struggle with their performance, with even state of the art approaches hovering under a 15% mean recall on certain evaluation modes. This poor performance is partially a result of networks heavily relying and fixating on non-visual data, such as class statistics, instead of the pixel-level signals present in the images. We demonstrate this by examining the 'visual-ness' of visual relationship detection approaches. We first describe and implement a new Naive Bayes-based statistical baseline for scene graph generation. Most notably, this basic classifier does not utilize the image pixels, but relies on the properties of the bounding boxes (class labels, topological configuration,... etc.) to predict the relationship labels. We demonstrate that our classical machine learning approach, one as simple as a categorical Naive Bayes classifier, can perform relationship detection in a manner that achieves relatively competitive performance to that of modern scene graph generators. This is an alarming finding regarding scene graph generation that implies that visual data in images may not be utilized in modern visual relationship detection past the point of object detection. We finally discuss how more visual modern approaches to scene graph generation appear to remedy some of these shortcomings.

关键词： Scene Graph Generation visual relationship detection Explainability Statistical Approaches

来源：评论

学校读者我要写书评

暂无评论

visual relationship detection with Multimodal Fusion and Reasoning

引用

SENSORS 2022年第20期22卷 7918-7918页

作者： Xiao, Shouguan Fu, Weiping Xian Univ Technol Sch Mech & Precis Instrument Engn Xian 710048 Peoples R China Xian Int Univ Sch Engn Xian 710077 Peoples R China

visual relationship detection aims to completely understand visual scenes and has recently received increasing attention. However, current methods only use the visual features of images to train the semantic network, which does not match human habits in which we know obvious features of scenes and infer covert states using common sense. Therefore, these methods cannot predict some hidden relationships of object-pairs from complex scenes. To address this problem, we propose unifying vision-language fusion and knowledge graph reasoning to combine visual feature embedding with external common sense knowledge to determine the visual relationships of objects. In addition, before training the relationship detection network, we devise an object-pair proposal module to solve the combination explosion problem. Extensive experiments show that our proposed method outperforms the state-of-the-art methods on the visual Genome and visual relationship detection datasets.

关键词： visual relationship detection vision-language fusion knowledge graph reasoning

来源：评论

学校读者我要写书评

暂无评论

Contextual Translation Embedding for visual relationship detection and Scene Graph Generation

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2021年第11期43卷 3820-3832页

作者： Hung, Zih-Siou Mallya, Arun Lazebnik, Svetlana Univ Illinois Comp Sci Dept Urbana IL 61801 USA Nvidia Res Santa Clara CA 95051 USA

Relations amongst entities play a central role in image understanding. Due to the complexity of modeling (subject, predicate, object) relation triplets, it is crucial to develop a method that can not only recognize seen relations, but also generalize to unseen cases. Inspired by a previously proposed visual translation embedding model, or VTransE [1], we propose a context-augmented translation embedding model that can capture both common and rare relations. The previous VTransE model maps entities and predicates into a low-dimensional embedding vector space where the predicate is interpreted as a translation vector between the embedded features of the bounding box regions of the subject and the object. Our model additionally incorporates the contextual information captured by the bounding box of the union of the subject and the object, and learns the embeddings guided by the constraint predicate union (subject, object) - subject - object. In a comprehensive evaluation on multiple challenging benchmarks, our approach outperforms previous translation-based models and comes close to or exceeds the state of the art across a range of settings, from small-scale to large-scale datasets, from common to previously unseen relations. It also achieves promising results for the recently introduced task of scene graph generation.

关键词： visual relationship detection scene graph generation scene understanding

来源：评论

学校读者我要写书评

暂无评论

Adaptively Clustering-Driven Learning for visual relationship detection

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2021年 23卷 4515-4525页

作者： Liu, An-An Wang, Yanhui Xu, Ning Nie, Weizhi Nie, Jie Zhang, Yongdong Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Ocean Univ China Coll Informat Sci & Engn Qingdao 266100 Peoples R China Univ Sci & Technol China Sch Informat Sci & Technol Hefei 230027 Peoples R China

visual relationship detection aims to describe the interactions between pairs of objects, such as person-ride-bike and bike-next to-car triplets. In reality, it is often the case that there exist some groups of strongly correlated relationships, while others are weakly related. Intuitively, the common relationships can be roughly categorized into several types such as geometric (e.g., next to), action (e.g., ride), and so on. However, previous studies ignore the relatedness discovery among multiple relationships, which only lie on a unified space to leverage visual features or statistical dependencies into categories. To tackle this problem, we propose an adaptively clustering-driven network for visual relationship detection, which can implicitly divide the unified relationship space into several subspaces with specific characteristics. Particularly, we propose two novel modules to discover the common distribution space and latent relationship association, respectively, which map pairs of object features into translation subspaces to induce the discriminative relationship clustering. Then, a fused inference is designed to integrate the group-induced representations with the language prior to facilitate the predicate inference. Especially, we design the Frobenius-norm regularization to boost the clustering. To the best of our knowledge, the proposed method is the first supervised framework to realize subject-predicate-object relationship-aware clustering for visual relationship detection. Extensive experiments show that the proposed method can achieve competing performances against the state-of-the-art methods on the visual Genome dataset. Additional ablation studies further validate its effectiveness.

关键词： visualization Task analysis Semantics Proposals Object detection Feature extraction Portable computers Adaptively clustering-driven learning translation embedding unit visual relationship detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：