检索结果-内蒙古大学图书馆

AffectiveNet: Affective-motion feature learning for micro expression recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Verma, Monu Vipparthi, Santosh Kumar Singh, Girdhari Vision Intelligence Lab Department of Computer Science and Engineering Malaviya National Institute of Technology Jaipur India

Micro-expressions are hard to spot due to fleeting and involuntary moments of facial muscles. Interpretation of micro emotions from video clips is a challenging task. In this paper we propose an affective-motion imaging that cumulates rapid and short-lived variational information of micro expressions into a single response. Moreover, we have proposed an AffectiveNet: affective-motion feature learning network that can perceive subtle changes and learns the most discriminative dynamic features to describe the emotion classes. The AffectiveNet holds two blocks: MICRoFeat and MFL block. MICRoFeat block conserves the scale-invariant features, which allows network to capture both coarse and tiny edge variations. While MFL block learns micro-level dynamic variations from two different intermediate convolutional layers. Effectiveness of the proposed network is tested over four datasets by using two experimental setups: person independent (PI) and cross dataset (CD) validation. The experimental results of the proposed network outperforms the state-of-the-art approaches with significant margin for MER approaches. © 2021, CC BY.

关键词： Machine learning

An empirical review of deep learning frameworks for change detection: Model design, experimental frameworks, challenges and research needs

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Mandal, Murari Vipparthi, Santosh Kumar Vision Intelligence Lab Department of Computer Science and Engineering Malaviya National Instituteof Technology Jaipur302017 India

Visual change detection, aiming at segmentation of video frames into foreground and background regions, is one of the elementary tasks in computer vision and video analytics. The applications of change detection include anomaly detection, object tracking, traffic monitoring, human machine interaction, behavior analysis, action recognition, and visual surveillance. Some of the challenges in change detection include background fluctuations, illumination variation, weather changes, intermittent object motion, shadow, fast/slow object motion, camera motion, heterogeneous object shapes and real-time processing. Traditionally, this problem has been solved using hand-crafted features and background modelling techniques. In recent years, deep learning frameworks have been successfully adopted for robust change detection. This article aims to provide an empirical review of the state-of-the-art deep learning methods for change detection. More specifically, we present a detailed analysis of the technical characteristics of different model designs and experimental frameworks. We provide model design based categorization of the existing approaches, including the 2D-CNN, 3D-CNN, ConvLSTM, multi-scale features, residual connections, autoencoders and GAN based methods. Moreover, an empirical analysis of the evaluation settings adopted by the existing deep learning methods is presented. To the best of our knowledge, this is a first attempt to comparatively analyze the different evaluation frameworks used in the existing deep change detection methods. Finally, we point out the research needs, future directions and draw our own conclusions. © 2021, CC BY-NC-ND.

关键词： Anomaly detection

Automatic Polyp Segmentation with Multiple Kernel Dilated Convolution Network

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Tomar, Nikhil Kumar Srivastava, Abhishek Bagci, Ulas Jha, Debesh School of Informatics and Computer Science Indira Gandhi National Open University India Computer Vision and Pattern Recognition Unit Indian Statistical Institute India Machine and Hybrid Intelligence Lab Department of Radiology Feinberg School of Medicine Northwestern University United States

The detection and removal of precancerous polyps through colonoscopy is the primary technique for the prevention of colorectal cancer worldwide. However, the miss rate of colorectal polyp varies significantly among the endoscopists. It is well known that a computer-aided diagnosis (CAD) system can assist endoscopists in detecting colon polyps and minimize the variation among endoscopists. In this study, we introduce a novel deep learning architecture, named MKDCNet, for automatic polyp segmentation robust to significant changes in polyp data distribution. MKDCNet is simply an encoder-decoder neural network that uses the pre-trained ResNet50 as the encoder and novel multiple kernel dilated convolution (MKDC) block that expands the field of view to learn more robust and heterogeneous representation. Extensive experiments on four publicly available polyp datasets and cell nuclei dataset show that the proposed MKDCNet outperforms the state-of-the-art methods when trained and tested on the same dataset as well when tested on unseen polyp datasets from different distributions. With rich results, we demonstrated the robustness of the proposed architecture. From an efficiency perspective, our algorithm can process at (≈ 45) frames per second on RTX 3090 GPU. MKDCNet can be a strong benchmark for building real-time systems for clinical colonoscopies. The code of the proposed MKDCNet is available at https://***/nikhilroxtomar/MKDCNet. © 2022, CC BY.

关键词： Convolution

Domain-independent video summarization based on transfer learning using convolutional neural network 1st

学校读者我要写书评

暂无评论

Domain-independent video summarization based on transfer lea...

1st International Conference on Advances in Electrical and computer Technologies, ICAECT 2019

作者： Mohan, Jesna Nair, Madhu S. Department of Computer Science University of Kerala Kariavattom ThiruvananthapuramKerala695581 India Department of Computer Science and Engineering Mar Baselios College of Engineering and Technology Nalanchira ThiruvananthapuramKerala695015 India Artificial Intelligence & Computer Vision Lab Department of Computer Science Cochin University of Science and Technology KochiKerala682022 India

ISBN: (纸本)9789811555572

Video summarization methods generate a compact representation of the original video preserving the essential content of the input video. Video summaries utilize less storage space compared to the original video. The summaries also facilitate efficient browsing and retrieval of video data. This paper presents a novel method for static video summarization based on deep features extracted using convolutional neural network (CNN) by means of transfer learning. To detect the frames with content change, the Chebyshev distance scores between the feature vectors of consecutive frames are thresholded. Uniform pre-sampling and redundancy elimination are done before summarization to reduce the complexity of the summarization step. Uniform pre-sampling is done by selecting two frames in one second. Redundant frames are eliminated based on flow vectors between consecutive frames using the SIFT flow algorithm. The experimental results on two well-known datasets show that our approach significantly improves the performance of the baseline methods and achieves competitive results compared with other state-of-the-art methods. © Springer Nature Singapore Pte Ltd 2020.

关键词： Convolutional neural networks

NTIRE 2023 Image Shadow Removal Challenge Report

学校读者我要写书评

暂无评论

NTIRE 2023 Image Shadow Removal Challenge Report

2023 IEEE/CVF Conference on computer vision and Pattern Recognition Workshops, CVPRW 2023

作者： Vasluianu, Florin-Alexandru Seizinger, Tim Timofte, Radu Cui, Shuhao Huang, Junshi Tian, Shuman Fan, Mingyuan Zhang, Jiaqi Zhu, Li Wei, Xiaoming Wei, Xiaolin Luo, Ziwei Gustafsson, Fredrik K. Zhao, Zheng Sjölund, Jens Schön, Thomas B. Dong, Xiaoyi Zhang, Xi Sheryl Li, Chenghua Leng, Cong Yeo, Woon-Ha Oh, Wang-Taek Lee, Yeo-Reum Ryu, Han-Cheol Luo, Jinting Jiang, Chengzhi Han, Mingyan Wu, Qi Lin, Wenjie Yu, Lei Li, Xinpeng Jiang, Ting Fan, Haoqiang Liu, Shuaicheng Xu, Shuning Song, Binbin Chen, Xiangyu Zhang, Shile Zhou, Jiantao Zhang, Zhao Zhao, Suiyi Zheng, Huan Gao, Yangcheng Wei, Yanyan Wang, Bo Ren, Jiahuan Luo, Yan Kondo, Yuki Miyata, Riku Yasue, Fuma Naruki, Taito Ukita, Norimichi Chang, Hua-En Yang, Hao-Hsiang Chen, Yi-Chung Chiang, Yuan-Chun Huang, Zhi-Kai Chen, Wei-Ting Chen, I-Hsiang Hsieh, Chia-Hsuan Kuo, Sy-Yen Xianwei, Li Fu, Huiyuan Liu, Chunlin Ma, Huadong Fu, Binglan He, Huiming Wang, Mengjia She, Wenxuan Liu, Yu Nathan, Sabari Kansal, Priya Zhang, Zhongjian Yang, Huabin Wang, Yan Zhang, Yanru Phutke, Shruti S. Kulkarni, Ashutosh Khan, Md Raqib Murala, Subrahmanyam Vipparthi, Santosh Kumar Ye, Heng Liu, Zixi Yang, Xingyi Liu, Songhua Wu, Yinwei Jing, Yongcheng Yu, Qianhao Zheng, Naishan Huang, Jie Long, Yuhang Yao, Mingde Zhao, Feng Zhao, Bowen Ye, Nan Shen, Ning Cao, Yanpeng Xiong, Tong Xia, Weiran Li, Dingwen Xia, Shuchen Computer Vision Lab Ifi Caidas University of Würzburg Germany Computer Vision Lab Eth Zürich Switzerland Meituan Group China Department of Information Technology Uppsala University Sweden Institute of Automation Chinese Academy of Sciences Beijing China Nanjing China Maicro Nanjing China Department of Artificial Intelligence Convergence Sahmyook University Seoul Korea Republic of Megvii Technology China University of Electronic Science and Technology of China China University of Macau China China Toyota Technological Institute Japan Graduate Institute of Electronics Engineering National Taiwan University Taiwan Department of Electrical Engineering National Taiwan University Taiwan Graduate Institute of Communication Engineering National Taiwan University Taiwan ServiceNow United States Beijing University of Post and Teleconmunication Beijing China Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education China Couger Inc. Computer Vision and Pattern Recognition Lab Indian Institute of Technology Ropar Punjab Rupnagar India Research Institute Singapore National University of Singapore Singapore Research Institute Singapore University of Sydney Australia Brain-Inspired Vision Laboratory Information Science and Technology Institution University of Science and Technology of China China State Key Laboratory of Fluid Power and Mechatronic Systems School of Mechanical Engineering Zhejiang University Hangzhou310027 China Key Laboratory of Advanced Manufacturing Technology of Zhejiang Province School of Mechanical Engineering Zhejiang University Hangzhou310027 China South China University of Technology China

ISBN: (纸本)9798350302493

This work reviews the results of the NTIRE 2023 Challenge on Image Shadow Removal. The described set of solutions were proposed for a novel dataset, which captures a wide range of object-light interactions. It consists of 1200 roughly pixel aligned pairs of real shadow free and shadow affected images, captured in a controlled environment. The data was captured in a white-box setup, using professional equipment for lights and data acquisition sensors. The challenge had a number of 144 participants registered, out of which 19 teams were compared in the final ranking. The proposed solutions extend the work on shadow removal, improving over the performance level describing state-of-the-art methods. © 2023 IEEE.

关键词： Data acquisition

Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Wangtao Zhang, Chenxiang Zhang, Xueyou Yu, Xuanqing Huang, Ziyang Xu, Haotian Chen, Pei He, Shizhu Zhao, Jun Liu, Kang The Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China CAS Engineering Laboratory for Intelligent Industrial Vision Institute of Automation Chinese Academy of Sciences Beijing China Department of Computer Science and Engineering Texas A&M University United States Shanghai Artificial Intelligence Laboratory China Xiaohongshu Inc China AI Lab AIGility Cloud Innovation Beijing China

Although Large Language Models (LLMs) have demonstrated strong instruction-following ability, they are further supposed to be controlled and guided by inferential rules in real-world scenarios to be safe, accurate, and intelligent. This demands the possession of inferential rule-following capability of LLMs. However, no prior work has made a clear evaluation of the inferential rule-following capability of LLMs. Previous studies that try to evaluate the inferential rule-following capability of LLMs fail to distinguish the inferential rule-following scenarios from the instruction-following scenarios. Therefore, this paper first clarifies the concept of inferential rule-following and proposes a comprehensive benchmark, RuleBench, to evaluate a diversified range of inferential rule-following abilities. Our experimental results on a variety of LLMs show that they are still limited in following rules. Our analysis based on the evaluation results provides insights into the improvements for LLMs toward a better inferential rule-following intelligent agent. We further propose Inferential Rule-Following Tuning (IRFT). The experimental results show that through IRFT, LLMs can learn abstract inferential rule-following abilities from purely synthetic data and then generalize to RuleBench. The data and code can be found at: https://***/r/llm-rulefollowing-B3E3/ Copyright © 2024, The Authors. All rights reserved.

关键词： Intelligent agents

NavCoT: Boosting LLM-Based vision-and-Language Navigation via Learning Disentangled Reasoning

学校读者我要写书评

暂无评论

IEEE Transactions on Pattern Analysis and Machine intelligence 2025年第7期47卷 5945-5957页

作者： Bingqian Lin Yunshuang Nie Ziming Wei Jiaqi Chen Shikui Ma Jianhua Han Hang Xu Xiaojun Chang Xiaodan Liang Shenzhen Campus Sun Yat-sen University Shenzhen China Shanghai Jiao Tong University Shanghai China University of Hong Kong Pok Fu Lam Hong Kong Dataa Robotics Company Beijing China Huawei Noah's Ark Lab Shanghai China School of Information Science and Technology University of Science and Technology of China Hefei China Department of Computer Vision Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) Abu Dhabi UAE Peng Cheng Laboratory Shenzhen China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China

vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions. Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability. However, their predominant use in an offline manner usually suffers from substantial domain gap between the VLN task and the LLM training corpus. This paper proposes a novel strategy called Nav igational C hain- o f- T hought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision, leading to a significant mitigation of the domain gap in a cost-effective manner. Specifically, at each timestep, the LLM is prompted to forecast the navigational chain-of-thought by: 1) acting as a world model to imagine the next observation according to the instruction, 2) selecting the candidate observation that best aligns with the imagination, and 3) determining the action based on the reasoning from the prior steps. In this way, the action prediction can be effectively simplified benefiting from the disentangled reasoning. Through constructing formalized labels for training, the LLM can learn to generate desired and reasonable chain-of-thought outputs for improving the action decision. Experimental results across various training settings and popular VLN benchmarks (e.g., Room-to-Room (R2R), Room-across-Room (RxR), Room-for-Room (R4R)) show the significant superiority of NavCoT over the direct action prediction variants. Through simple parameter-efficient finetuning, our NavCoT outperforms a recent GPT4-based approach with $\sim$ 7% relative improvement on the R2R dataset. We believe that NavCoT will help unlock more task-adaptive and scalable LLM-based embodied agents, which are helpful for developing real-world robotics applications.

关键词： Navigation Cognition Training Scalability Glass Planning Large language models History Electronic mail Visualization

On the analysis of HEVC Intra Prediction Mode Decision Variants

学校读者我要写书评

暂无评论

Procedia computer science 2020年 171卷 1887-1897页

作者： Preethi S. Nair Madhu S. Nair Department of Computer Science University of Kerala Kariavattom Thiruvananthapuram-695581 Kerala India Artificial Intelligence & Computer Vision Lab Department of Computer Science Cochin University of Science and Technology Kochi -682022 Kerala India

This paper presents a comparative analysis of some major High Efficiency Video Coding (HEVC) intra prediction mode decision methods reported in the literature. Intra coding in HEVC is based on spatial sample prediction and can be described as a collection of many elements including quad tree partitioning, 35 intra prediction modes, reference sample smoothing, boundary sample filtering process, intra mode coding and so on. Each of these elements results in an improved coding efficiency in HEVC intra coding. In this review, we provide an extensive and up-to-date discussion of the research proposals regarding a variety of HEVC intra prediction mode decision strategies. Moreover, some most significant research contributions on fast intra mode decision methods in HEVC to improvise the encoder efficiency are reviewed in detail along with a comparative performance analysis of the reviewed proposals in terms of some standard evaluation metrics such as bit rate, PSNR and time savings. In addition, this paper point out some research directions towards further developments in the intra prediction for the betterment of the encoder efficacy.

关键词： HEVC Intra prediction Mode decision RMD RDO

A Survey of the Self Supervised Learning Mechanisms for vision Transformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Khan, Asifullah Sohail, Anabia Fiaz, Mustansar Hassan, Mehdi Afridi, Tariq Habib Marwat, Sibghat Ullah Munir, Farzeen Ali, Safdar Naseem, Hannan Zaheer, Muhammad Zaigham Ali, Kamran Sultana, Tangina Tanoli, Ziaurrehman Akhter, Naeem Pattern Recognition Lab DCIS PIEAS Nilore Islamabad45650 Pakistan PIEAS Nilore Islamabad45650 Pakistan Deep Learning Lab Center for Mathematical Sciences PIEAS Nilore Islamabad45650 Pakistan Center of Secure Cyber-Physical Security Systems Khalifa University Abu Dhabi United Arab Emirates IBM Research United States Department of Computer Science Air University Islamabad Pakistan Department of Computer Science and Engineering Kyung Hee University Global Campus 1732 Gyeonggi-do Yongin17104 Korea Republic of Department of Electrical Engineering and Automation Aalto University Finland Finnish Center of Artificial Center Finland Faculty of Engineering and Green Technology Universiti Tunku Abdul Rahman Malaysia Computer Vision Department Mohamed Bin Zayed University of Artificial Intelligence United Arab Emirates Karachi Pakistan Department of Electronics and Communication Engineering Hajee Mohammad Danesh Science and Technology University Bangladesh HiLIFE University of Helsinki Finland

vision Transformers (ViTs) have recently demonstrated remarkable performance in computer vision tasks. However, their parameter-intensive nature and reliance on large amounts of data for effective performance have shifted the focus from traditional human-annotated labels to unsupervised learning and pretraining strategies that uncover hidden structures within the data. In response to this challenge, self-supervised learning (SSL) has emerged as a promising paradigm. SSL leverages inherent relationships within the data itself as a form of supervision, eliminating the need for manual labeling and offering a more scalable and resource-efficient alternative for model training. Given these advantages, it is imperative to explore the integration of SSL techniques with ViTs, particularly in scenarios with limited labeled data. Inspired by this evolving trend, this survey aims to systematically review SSL mechanisms tailored for ViTs. We propose a comprehensive taxonomy to classify SSL techniques based on their representations and pre-training tasks. Additionally, we discuss the motivations behind SSL, review prominent pre-training tasks, and highlight advancements and challenges in this field. Furthermore, we conduct a comparative analysis of various SSL methods designed for ViTs, evaluating their strengths, limitations, and applicability to different scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning