We present a method for decomposing the 3D scene flow observed from a moving stereo rig into stationary scene elements and dynamic object motion. Our unsupervised learning framework jointly reasons about the camera mo...
We present a method for decomposing the 3D scene flow observed from a moving stereo rig into stationary scene elements and dynamic object motion. Our unsupervised learning framework jointly reasons about the camera motion, optical flow, and 3D motion of moving objects. Three cooperating networks predict stereo matching, camera motion, and residual flow, which represents the flow component due to object motion and not from camera motion. Based on rigid projective geometry, the estimated stereo depth is used to guide the camera motion estimation, and the depth and camera motion are used to guide the residual flow estimation. We also explicitly estimate the 3D scene flow of dynamic objects based on the residual flow and scene depth. Experiments on the KITTI dataset demonstrate the effectiveness of our approach and show that our method outperforms other state-of-the-art algorithms on the optical flow and visual odometry tasks.
Awareness of what surrounds a vehicle directly affects the safe driving and maneuvering of an automobile. Surround information or maps can help in ethnographic studies of driver behavior as well as provide a critical ...
详细信息
Awareness of what surrounds a vehicle directly affects the safe driving and maneuvering of an automobile. Surround information or maps can help in ethnographic studies of driver behavior as well as provide a critical input in the development of effective driver assistance system. In this paper, we introduce the concept of Dynamic Panoramic Surround (DPS) map that shows the nearby surroundings of the vehicle, and detects the objects of importance on the road. Omnidirectional cameras which give a panoramic view of the surroundings can be useful for visualizing and analyzing the nearby surroundings of the vehicle. A novel approach for synthesizing the DPS using stereo and motion analysis of video images from a pair of omni-directional cameras on the vehicle is developed. Successful generation of DPS in experimental runs on an instrumented vehicle testbed is demonstrated. These experiments prove the basic feasibility and show promise of omni video based DPS capture algorithm to provide useful semantic descriptors of the state of moving vehicles and obstacles in the vicinity of a vehicle.
Awareness to a vehicle's surrounding is necessary for safe driving. Current surround technologies focus on the detection of obstacles in hard-to-view places but may neglect temporal information. This paper seeks t...
详细信息
Awareness to a vehicle's surrounding is necessary for safe driving. Current surround technologies focus on the detection of obstacles in hard-to-view places but may neglect temporal information. This paper seeks the causes of dangerous situations by examining surround behavior. A general hierarchical learning framework is introduced to automatically learn surround behaviors. By observing motion trajectories during natural driving, models of rear vehicle behaviors are obtained in an unsupervised fashion. The extracted behaviors are shown to correspond to typical driving scenarios, vehicle overtake and surround overtake, demonstrating the effectiveness of the learning framework.
This paper presents a real-time highway monitoring system for tracking and classification of vehicles with the computation of traffic flow parameters from live video streams. The proposed system robustly detects and t...
详细信息
This paper presents a real-time highway monitoring system for tracking and classification of vehicles with the computation of traffic flow parameters from live video streams. The proposed system robustly detects and tracks vehicles during daylight hours and accurately classifies them into 8 different types by leveraging tracking information. The system is able to process video continuously over long time periods, accumulating large volumes of tracking data to build daily highway models consisting of the traffic flow parameters, density, flow, and speed. These daily models are used to categorize the speed profile of live traffic.
We propose a set of features derived from skeleton tracking of the human body and depth maps for the purpose of action recognition. The descriptors proposed are easy to implement, produce relatively small-sized featur...
详细信息
We propose a set of features derived from skeleton tracking of the human body and depth maps for the purpose of action recognition. The descriptors proposed are easy to implement, produce relatively small-sized feature sets, and the multi-class classification scheme is fast and suitable for real-time applications. We intuitively characterize actions using pairwise affinities between view-invariant joint angles features over the performance of an action. Additionally, a new descriptor for spatio-temporal feature extraction from color and depth images is introduced. This descriptor involves an application of a modified histogram of oriented gradients (HOG) algorithm. The application produces a feature set at every frame, and these features are collected into a 2D array which then the same algorithm is applied to again (the approach is termed HOG 2 ). Both feature sets are evaluated in a bag-of-words scheme using a linear SVM, showing state-of-the-art results on public datasets from different domains of human-computer interaction.
Orientation selection is the inference of orientation information out of images. It is one of the foundations on which other visual structures are built, since it must precede the formation of contours out of pointill...
详细信息
Orientation selection is the inference of orientation information out of images. It is one of the foundations on which other visual structures are built, since it must precede the formation of contours out of pointillist data and surfaces out of surface markings. We take a differential geometric view in defining orientation selection and develop algorithms for actually doing it. The goal of these algorithms is formulated in mathematical terms as the inference of a vector field of tangents (to the contours), and the algorithms are studied in both abstract and computational forms. They are formulated as matching problems, and algorithms for solving them are reduced to biologically plausible terms. We show that two different matching problems are necessary, the first for 1-dimensional contours (which we refer to as Type I processes) and the second for 2-dimensional flows (or Type II processes).
This paper proposes a concept of panoramic appearance map to perform reidentification of a people who leave the scene and reappear after some time. The map is a compact signature of appearance information of a person ...
详细信息
This paper proposes a concept of panoramic appearance map to perform reidentification of a people who leave the scene and reappear after some time. The map is a compact signature of appearance information of a person extracted from multiple cameras. The person is detected and tracked in multiple cameras and triangulation is used to accurately localize the person in 3-D. A virtual cylinder is formed around the person's location and mapped onto an image with the horizontal axis representing the azimuth angle and vertical axis representing the height. Each bin in the map image gets the appearance information from all the cameras which can observe it. The maps between different tracks are matched using a weighted metric. Experimental results showing person matching and reidentification show the effectiveness of the approach.
In order for an autonomous unmanned ground vehicle (UGV) to drive in off-road terrain at high speeds, it must analyze and understand its surrounding terrain in realtime: it must know where it intends to go, where are ...
详细信息
In order for an autonomous unmanned ground vehicle (UGV) to drive in off-road terrain at high speeds, it must analyze and understand its surrounding terrain in realtime: it must know where it intends to go, where are the hazards, and many details of the topography of the terrain. Much research has been done in the way of obstacle avoidance, terrain classification, and path planning, but still so few UGV systems can accurately traverse off-road environments at high speeds autonomously. One of the most dangerous hazards found off-road are negative obstacles, mainly because they are so difficult to detect. We present algorithms that analyze the terrain using a point cloud produced by a 3D laser range finder, then attempt to classify the negative obstacles using both a geometry-based method we call the Negative Obstacle DetectoR (NODR) as well as a support vector machine (SVM) algorithm. The terrain is analyzed with respect to a large UGV with the sensor mounted up high as well as a small UGV with the sensor mounted low to the ground.
A system for face model adaptation combining active tracking is presented. Input from an active camera is used for MPEG4 model based coding. First, the background is compensated considering a moving camera (tilt or pa...
详细信息
A system for face model adaptation combining active tracking is presented. Input from an active camera is used for MPEG4 model based coding. First, the background is compensated considering a moving camera (tilt or pan). Second, the talking face is segmented from the compensated background using fusion of frame differences. A morphological filter is then applied to make the system less sensitive to noise. Third, Hough transform and deformable template coupled with color information are exploited to detect the facial features, e.g., eyes, mouth. Fourth, a wireframe model is adapted to the extracted face by an extended dynamic mesh. The feasibility of the proposed system is demonstrated using several real active video sequences.
Detecting vehicles from a moving vehicle is an important task. In this paper a new vehicle detector is introduced. The new vehicle detector employs the use of the ubiquitous wheel. Every car has wheels; this wheel det...
详细信息
Detecting vehicles from a moving vehicle is an important task. In this paper a new vehicle detector is introduced. The new vehicle detector employs the use of the ubiquitous wheel. Every car has wheels; this wheel detector finds wheels and infers vehicle location from the wheel detection. Views from an omnidirectional camera are used to generate side view images. These images are processed using a difference of Gaussian filter bank. The responses from the filter bank are applied to a precomputed set of principle components. The principle component responses are compared against a Gaussian mixture model of wheels and Gaussian model roadbed. Wheel candidates are chosen and tracked. Initial experimental results along with analysis are shown.
暂无评论