检索结果-内蒙古大学图书馆

Seeing beyond the crop: using language priors for out-of-bounding box keypoint prediction 24

学校读者我要写书评

暂无评论

Seeing beyond the crop: using language priors for out-of-bou...

Proceedings of the 38th International Conference on Neural Information processing Systems

作者： Bavesh Balaji Jerrin Bright Yuhao Chen Sirisha Rambhatla John S. Zelek David Anthony Clausi Critical ML Lab and Vision and Image Processing Lab and University of Waterloo Vision and Image Processing Lab and University of Waterloo Critical ML Lab and University of Waterloo

ISBN: (纸本)9798331314385

Accurate estimation of human pose and the pose of interacting objects, like a hockey stick, is crucial for action recognition and performance analysis, particu larly in sports. Existing methods capture the object along with the human in the bounding boxes, assuming all keypoints are visible within the bounding box. This necessitates larger bounding boxes to capture the object, introducing unnecessary visual features and hindering performance in real-world cluttered environments. We propose a simple image and text-based multimodal solution TokenCLIPose that addresses this limitation. Our approach focuses solely on human keypoints within the bounding box, treating objects as unseen. TokenCLIPose leverages the rich semantic representations endowed by language for inducing keypoint-specific con text, even for occluded keypoints. We evaluate the performance of TokenCLIPose on a real-world ice hockey dataset, and demonstrate its generalizability through zero-shot transfer to a smaller Lacrosse dataset. Additionally, we showcase its flexibility on CrowdPose, a popular occlusion benchmark with keypoints within the bounding box. Our method significantly improves over state-of-the-art approaches on ice hockey, Lacrosse, and CrowdPose datasets, with gains of 4.36%, 2.35%, and 3.8%, respectively.

关键词：

Domain-Guided Masked Autoencoders for Unique Player Identification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Balaji, Bavesh Bright, Jerrin Rambhatla, Sirisha Chen, Yuhao Wong, Alexander Zelek, John Clausi, David A. Vision and Image Processing Lab University of Waterloo Canada Critical ML Lab University of Waterloo Canada

Unique player identification is a fundamental module in vision-driven sports analytics. Identifying players from broadcast videos can aid with various downstream tasks such as player assessment, in-game analysis, and broadcast production. However, automatic detection of jersey numbers using deep features is challenging primarily due to: a) motion blur, b) low resolution video feed, and c) occlusions. With their recent success in various vision tasks, masked autoencoders (MAEs) have emerged as a superior alternative to conventional feature extractors. However, most MAEs simply zero-out image patches either randomly or focus on where to mask rather than how to mask. Motivated by human vision, we devise a novel domain-guided masking policy for MAEs termed dMAE to facilitate robust feature extraction in the presence of motion blur for player identification. We further introduce a new spatio-temporal network leveraging our novel d-MAE for unique player identification. We conduct experiments on three large-scale sports datasets, including a curated baseball dataset, the SoccerNet dataset, and an in-house ice hockey dataset. We preprocess the datasets using an upgraded keyframe identification (KfID) module by focusing on frames containing jersey numbers. Additionally, we propose a keyframe-fusion technique to augment keyframes, preserving spatial and temporal context. Our spatio-temporal network showcases significant improvements, surpassing the current state-of-the-art by 8.58%, 4.29%, and 1.20% in the test set accuracies, respectively. Rigorous ablations highlight the effectiveness of our domain-guided masking approach and the refined KfID module, resulting in performance enhancements of 1.48% and 1.84% respectively, compared to original architectures. © 2024, CC BY.

关键词： image enhancement

Zero-Shot Monocular Motion Segmentation in the Wild by Combining Deep Learning with Geometric Motion Model Fusion

学校读者我要写书评

暂无评论

Zero-Shot Monocular Motion Segmentation in the Wild by Combi...

IEEE Computer Society Conference on Computer vision and Pattern Recognition Workshops (CVPRW)

作者： Yuxiang Huang Yuhao Chen John Zelek Vision and Image Processing Lab University of Waterloo Waterloo Canada

ISBN: (数字)9798350365474

ISBN: (纸本)9798350365481

Detecting and segmenting moving objects from a moving monocular camera is challenging in the presence of unknown camera motion, diverse object motions and complex scene structures. Most existing methods rely on a single motion cue to perform motion segmentation, which is usually insufficient when facing different complex environments. While a few recent deep learning based methods are able to combine multiple motion cues to achieve improved accuracy, they depend heavily on vast datasets and extensive annotations, making them less adaptable to new scenarios. To address these limitations, we propose a novel monocular dense segmentation method that achieves state-of-the-art motion segmentation results in a zero-shot manner. The proposed method synergestically combines the strengths of deep learning and geometric model fusion methods by performing geometric model fusion on object proposals. Experiments show that our method achieves competitive results on several motion segmentation datasets and even surpasses some state-of-the-art supervised methods on certain benchmarks, while not being trained on any data. We also present an ablation study to show the effectiveness of combining different geometric models together for motion segmentation, highlighting the value of our geometric model fusion strategy.

关键词： Deep learning Computer vision Motion segmentation Conferences Computational modeling Geometric modeling Benchmark testing

Autocalibration for Creating 2D/3D Displays on Complex Geometric Surfaces

学校读者我要写书评

暂无评论

Information Display 2022年第6期38卷 31-37页

作者： Arnold, Katherine Fieguth, Paul Dennison, Andrew Vision and Image Processing Lab University of Waterloo Canada Systems Design Engineering Department University of Waterloo Canada Software Solutions Development Department Christie Digital Systems

Geometric calibration methods tend to rely on precalibrated cameras or real calibration targets. Here we explore the chal-lenge of achieving a sufficiently accurate projector system calibration for complex projection mapping tasks that can rely on neither precalibrated cameras nor calibration targets. Virtual scene models can take the place of a real calibration target for the purpose of providing scene information that is used to refine geometric estimates. Refinement of the display parameters aims to reduce the reprojection error, or the error between the theoretical coordinate (generated from the projection of the scene estimate coordinate by the estimated parameters) and the actual coordinate (obtained from pixel cor-respondences). For Christie installations, the final reprojection error of each pixel should be less than 1 pixel. Mystique was not always so capable. Christie’s research department, Christie labs, has a history of creating solutions for new projection concepts. Part of that strategy has been a long partnership with the university of waterloo, particularly the vision and image processing (VIP) lab in Systems Design Engineering. Graduate students study computer vision and machine learning—topics related to problems often at the heart of techniques—which can be used to bring new ideas, meth-ods, and solutions to Mystique and, by extension, to Christie’s customers. Years of investment in research and development, including partnership with the VIP lab and others, have enabled Mystique to handle challenges beyond any other projec-tor-alignment solution. ID. © 2022, John Wiley and Sons Inc. All rights reserved.

关键词： Pixels

Zero-Shot Monocular Motion Segmentation in the Wild by Combining Deep Learning with Geometric Motion Model Fusion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Yuxiang Chen, Yuhao Zelek, John Vision and Image Processing Lab University of Waterloo WaterlooON Canada

关键词： Cameras

Distribution and Depth-Aware Transformers for 3D Human Mesh Recovery

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bright, Jerrin Balaji, Bavesh Prakash, Harish Chen, Yuhao Clausi, David A. Zelek, John Vision and Image Processing Lab University of Waterloo Canada

Precise Human Mesh Recovery (HMR) with in-the-wild data is a formidable challenge and is often hindered by depth ambiguities and reduced precision. Existing works resort to either pose priors or multi-modal data such as multi-view or point cloud information, though their methods often overlook the valuable scene-depth information inherently present in a single image. Moreover, achieving robust HMR for out-of-distribution (OOD) data is exceedingly challenging due to inherent variations in pose, shape and depth. Consequently, understanding the underlying distribution becomes a vital subproblem in modeling human forms. Motivated by the need for unambiguous and robust human modeling, we introduce Distribution and depth-aware human mesh recovery (D2AHMR), an end-to-end transformer architecture meticulously designed to minimize the disparity between distributions and incorporate scene-depth leveraging prior depth information. Our approach demonstrates superior performance in handling OOD data in certain scenarios while consistently achieving competitive results against state-of-the-art HMR methods on controlled datasets. © 2024, CC BY.

关键词： Recovery

A Moving Camera and Synthetic Calibration Target Solution for Non-Planar Scene Estimation and Projector Calibration 59th

学校读者我要写书评

暂无评论

A Moving Camera and Synthetic Calibration Target Solution fo...

59th International Symposium, Seminar and Exhibition, Display Week 2022

作者： Arnold, Katherine Fieguth, Paul Lamm, Mark Vision and Image Processing Lab University of Waterloo WaterlooON Canada Christie Digital Systems Canada Inc KitchenerON Canada

The automatic calibration of projector systems improves the possible environments available to this technology which relies on exact scene calibration. This paper explores the challenge of projector calibration and non-planar scene estimation. This formulation assumes no prior information on the moving camera or fixed projector. Limited scene understanding provides the basis for constructing synthetic calibration targets to perform a geometric recovery through bundle adjustment. Synthetic data are generated from ground-truth camera and projector parameters to explore the method’s performance over varying properties of the synthetic calibration targets. © 2022 SID.

关键词： Cameras

SAMJAM: Zero-Shot Video Scene Graph Generation for Egocentric Kitchen Videos

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Li, Joshua Cantu, Fernando Jose Pena Yu, Emily Wong, Alexander Cui, Yuchen Chen, Yuhao Vision and Image Processing Lab University of Waterloo Canada Robot Intelligence Lab UCLA United States

Video Scene Graph Generation (VidSGG) is an important topic in understanding dynamic kitchen environments. Current models for VidSGG require extensive training to produce scene graphs. Recently, vision Language Models (VLM) and vision Foundation Models (VFM) have demonstrated impressive zero-shot capabilities in a variety of tasks. However, VLMs like Gemini struggle with the dynamics for VidSGG, failing to maintain stable object identities across frames. To overcome this limitation, we propose SAMJAM, a zero-shot pipeline that combines SAM2’s temporal tracking with Gemini’s semantic understanding. SAM2 also improves upon Gemini’s object grounding by producing more accurate bounding boxes. In our method, we first prompt Gemini to generate a frame-level scene graph. Then, we employ a matching algorithm to map each object in the scene graph with a SAM2-generated or SAM2-propagated mask, producing a temporally-consistent scene graph in dynamic environments. Finally, we repeat this process again in each of the following frames. We empirically demonstrate that SAMJAM outperforms Gemini by 8.33% in mean recall on the EPIC-KITCHENS and EPIC-KITCHENS-100 datasets. © 2025, CC BY.

关键词：

How Much You Ate? Food Portion Estimation on Spoons

学校读者我要写书评

暂无评论

How Much You Ate? Food Portion Estimation on Spoons

IEEE Computer Society Conference on Computer vision and Pattern Recognition Workshops (CVPRW)

作者： Aaryam Sharma Chris Czarnecki Yuhao Chen Pengcheng Xi Linlin Xu Alexander Wong Vision and Image Processing Lab University of Waterloo National Research Council Canada

ISBN: (数字)9798350365474

ISBN: (纸本)9798350365481

Monitoring dietary intake is a crucial aspect of promoting healthy living. In recent years, advances in computer vision technology have facilitated dietary intake monitoring through the use of images and depth cameras. However, the current state-of-the-art image-based food portion estimation algorithms assume that users take images of their meals one or two times, which can be inconvenient and fail to capture food items that are not visible from a top-down perspective, such as ingredients submerged in a stew. To address these limitations, we introduce an innovative solution that utilizes stationary user-facing cameras to track food items on utensils, not requiring any change of camera perspective after installation. The shallow depth of utensils provides a more favorable angle for capturing food items, and tracking them on the utensil’s surface offers a significantly more accurate estimation of dietary intake without the need for post-meal image capture. The system is reliable for estimation of nutritional content of liquid-solid heterogeneous mixtures such as soups and stews. Through a series of experiments, we demonstrate the exceptional potential of our method as a non-invasive, user-friendly, and highly accurate dietary intake monitoring tool.

关键词： Computer vision image segmentation Accuracy Pipelines Estimation image capture Cameras