Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Tony LindebergSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works237 from public data
- 4,230
- On Space-Time Interest Points4,065
This paper builds on the idea of the Harris and Förstner interest point operators and detects local structures in space-time where the image values have significant local variations in both space and time and illustrates how a video representation in terms of local space- time features allows for detection of walking people in scenes with occlusions and dynamic cluttered backgrounds.
- 3,819
- 3,316
- Segmenter: Transformer for Semantic Segmentation2,051
This paper introduces Segmenter, a transformer model for semantic segmentation that outperforms the state of the art on both ADE20K and Pascal Context datasets and is competitive on Cityscapes.
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips1,545
It is demonstrated that a text-video embedding trained on this data leads to state-of-the-art results for text-to-video retrieval and action localization on instructional video datasets such as YouCook2 or CrossTask.
- Evaluation of local spatio-temporal features for action recognition1,508
It is demonstrated that regular sampling of space-time features consistently outperforms all testedspace-time interest point detectors for human actions in realistic settings and is a consistent ranking for the majority of methods over different datasets.
- Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding1,455
This work proposes a novel Hollywood in Homes approach to collect data, collecting a new dataset, Charades, with hundreds of people recording videos in their own homes, acting out casual everyday activities, and evaluates and provides baseline results for several tasks including action recognition and automatic description generation.
- 1,407
- Learning from Synthetic Humans1,076
This work presents SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data and shows that CNNs trained on this synthetic dataset allow for accurate human depth estimation and human part segmentation in real RGB images.
- Long-Term Temporal Convolutions for Action Recognition954
It is demonstrated that LTC-CNN models with increased temporal extents improve the accuracy of action recognition and the impact of different low-level representations, such as raw values of video pixels and optical flow vector fields, and the importance of high-quality optical flow estimation for learning accurate action models.
- 938
- The THUMOS challenge on action recognition for videos “in the wild”918
The THUMOS benchmark is described in detail, the evaluation protocols used to quantify results in the two THUMOS tasks of action classification and temporal action detection are presented, and how well methods trained on trimmed videos generalize to untrimmed videos is evaluated.
- End-to-End Learning of Visual Representations From Uncurated Instructional Videos813
This work proposes a new learning approach, MIL-NCE, capable of addressing mis- alignments inherent in narrated videos and outperforms all published self-supervised approaches for these tasks as well as several fully supervised baselines.
- Learning Joint Reconstruction of Hands and Manipulated Objects683
This work presents an end-to-end learnable model that exploits a novel contact loss that favors phys- ically plausible hand-object constellations, and improves grasp quality metrics over baselines, using RGB images as input.
- P-CNN: Pose-Based CNN Features for Action Recognition641
A new Pose-based Convolutional Neural Network descriptor (P-CNN) for action recognition that aggregates motion and appearance information along tracks of human body parts is proposed.
- 511
- BodyNet: Volumetric Inference of 3D Human Body Shapes457
BodyNet is an end-to-end trainable network that benefits from a volumetric 3D loss, a multi-view re-projection loss, and intermediate supervision of 2D pose, 2D body part segmentation, and 3D pose and achieves state-of-the-art performance.
- Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning427
Vid2Seq is introduced, a multi-modal single-stage dense event captioning model pretrained on narrated videos which are readily-available at scale and improves the state of the art on a variety of dense video captioning benchmarks including YouCook2, ViTT and ActivityNet Captions.
- 392
- 381
- 378
- Just Ask: Learning to Answer Questions from Millions of Narrated Videos360
This work proposes to avoid manual annotation and generate a large-scale training dataset for video question answering making use of automatic cross-modal supervision, and introduces iVQA, a new VideoQA dataset with reduced language biases and high-quality redundant manual annotations.
- Cross-Task Weakly Supervised Learning From Instructional Videos330
The experiments demonstrate that sharing across tasks improves performance, especially when done at the component level and that the component model can parse previously unseen tasks by virtue of its compositionality.
- Unsupervised Learning from Narrated Instruction Videos329
A new unsupervised learning approach that takes advantage of the complementary nature of the input video and the associated narration to solve two clustering problems, one in text and one in video, and can automatically discover the main steps to achieve the task and locate the steps in the input videos.
- Video copy detection322
Local methods demonstrate their superior performance over the global ones, when detecting video copies subjected to various transformations, as well as for different lengths of video segments.
- Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation321
This work proposes a dual-scale graph transformer (DUET) for joint long-term action planning and fine-grained cross-modal understanding, and builds a topological map on-the-fly to enable efficient exploration in global action space.
- 317
- ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization315
This work introduces two types of context-aware guidance models, additive and contrastive models, that leverage their surrounding context regions to improve localization in objects in images using image-level supervision only.
- Recognizing human actions in still images: a study of bag-of-features and part-based representations290
The role of background scene context is investigated and it is demonstrated that improved action recognition performance can be achieved by combining the statistical and part-based representations, and integrating person-centric description with the Background scene context.
- Cross-View Action Recognition from Temporal Self-similarities263
An action descriptor is developed that captures the structure of temporal similarities and dissimilarities within an action sequence that relies on weak geometric properties and combines them with machine learning for efficient cross-view action recognition.
- 253
- Weakly Supervised Action Labeling in Videos under Ordering Constraints250
It is shown that the action label assignment can be determined together with learning a classifier for each action in a discriminative manner and evaluated on a new and challenging dataset of 937 video clips.
- 239
- Local Descriptors for Spatio-temporal Recognition233
An experimental evaluation of a set of local space-time descriptors for representing and recognizing motion patterns in video shows that high classification performance can be achieved, and that there is a clear advantage of using local position dependent histograms, consistent with previously reported findings regarding spatial recognition.
- Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction230
This work presents a method to leverage photometric consistency across time when annotations are only available for a sparse subset of frames in a video and demonstrates that the approach allows us to improve the pose estimation accuracy by leveraging information from neighboring frames in low-data regimes.
- Object Detection Using Strongly-Supervised Deformable Part Models229
This method is able to deal with sub-optimal and incomplete annotations of object parts and is shown to benefit from semi-supervised learning setups where part-level annotation is provided for a fraction of positive examples only.
- 228
- 227
- 218
- Weakly-Supervised Learning of Visual Relations202
A novel approach for modeling visual relations between pairs of objects where the predicate is typically a preposition or a verb that links a pair of objects, and proposes a weakly-supervised discriminative clustering model to learn relations from image-level labels only.
- 199
- Airbert: In-domain Pretraining for Vision-and-Language Navigation195
In this work, BnB1 is introduced, a large-scale and diverse in-domain VLN dataset that is used to pretrain the Airbert2 model that can be adapted to discriminative and generative settings and outperforms state of the art for Room-to-Room (R2R) navigation and Remote Referring Expression (REVERIE) benchmarks.
- Improvements of Object Detection Using Boosted Histograms195
A method for object detection that combines AdaBoost learning with local histogram features that outperforms all methods reported in [5] for 7 out of 8 detection tasks and four object classes.
- Local velocity-adapted motion events for spatio-temporal recognition190
A systematic experimental evaluation on a large video database with human actions demonstrates that local spatio-temporal image descriptors can be defined to carry important information of space-time events for subsequent recognition and that local velocity adaptation is an important mechanism in situations when the relative motion between the camera and the interesting events in the scene is unknown.
- People Watching: Human Actions as a Cue for Single View Geometry175
It is demonstrated that observing people performing different actions can significantly improve estimates of 3D scene geometry and is validated on monocular time-lapse sequences from YouTube and still images of indoor scenes gathered from the Internet.
- 161
- Learning person-object interactions for action recognition in still images158
A discriminatively trained model of person-object interactions for recognizing common human actions in still images that bypasses the difficult problem of estimating the complete human body pose configuration is investigated.
- 152
- TubeDETR: Spatio-Temporal Video Grounding with Transformers148
TubeDETR is proposed, a transformer-based architecture inspired by the recent success of such models for text-conditioned object detection that includes an efficient video and text encoder that models spatial multi-modal interactions over sparsely sampled frames and a space-time decoder that jointly performs spatio-temporal localization.
- Context-Aware CNNs for Person Head Detection148
This work leverage person-scene relations and propose a global CNN model trained to predict positions and scales of heads directly from the full image via energy-based model where the potentials are computed with a CNN framework.
- Weakly-Supervised Alignment of Video with Text144
This paper proposes a method for aligning the two modalities of video and text, i.e., automatically providing a time (frame) stamp for every sentence, and formulate this problem as an integer quadratic program, and solve its continuous convex relaxation using an efficient conditional gradient algorithm.
- 142
- 137
- Improving object detection with boosted histograms132
This work shows how histogram-based image descriptors can be combined with a boosting classifier to provide a state of the art object detector and introduces a weak learner for multi-valued histogram features and shows how to overcome problems of limited training sets.
- Improving bag-of-features action recognition with non-local cues131
This work decompose video into region classes and augment local features with corresponding region-class labels and demonstrates how this information can be integrated with BoF representations in a kernel-combination framework.
- Unsupervised Object Discovery and Tracking in Video Collections128
This paper addresses the problem of automatically localizing dominant objects as spatio-temporal tubes in a noisy collection of videos with minimal or even no supervision as a combination of two complementary processes: discovery and tracking.
- 118
- Synthetic Humans for Action Recognition from Unseen Viewpoints115
This work makes use of the recent advances in monocular 3D human body reconstruction from real action sequences to automatically render synthetic training videos for the action labels, and introduces a new data generation methodology that allows training of spatio-temporal CNNs for action classification.
- Scene Semantics from Long-Term Observation of People115
This paper constructs a functional object description with the aim to recognize objects by the way people interact with them, and describes scene objects (sofas, tables, chairs) by associated human poses and object appearance.
- 107
- Deep Metric Learning Beyond Binary Supervision106
A new triplet loss is proposed that allows distance ratios in the label space to be preserved in the learned metric space and enables the model to learn the degree of similarity rather than just the order.
- Tracking of Multi-state Hand Models Using Particle Filtering and a Hierarchy of Multi-scale Image Features⋆95
A scale-invariant dissimilarity measure is proposed for comparing scale-space features at different positions and scales, and the likelihood of hierarchical, parameterized models can be evaluated in such a way that maximization of the measure over different models and their parameters allows for both model selection and parameter estimation.
- Weakly supervised object recognition with convolutional neural networks94
A weakly supervised convolutional neural network for object recognition that does not rely on detailed object annotation and yet returns 86.3% mAP on the Pascal VOC classification task, outperforming previous fully-supervised systems by a sizable margin.
- Estimating 3D Motion and Forces of Person-Object Interactions From Monocular Video90
A method is introduced to automatically reconstruct the 3D motion of a person interacting with an object from a single RGB video, and develops a method to automatically recognize from the input video the position and timing of contacts between the person and the object or the ground.
- Joint Discovery of Object States and Manipulation Actions90
This work proposes a joint model that learns to identify object states and to localize state-modifying actions and demonstrates successful discovery of seven manipulation actions and corresponding object states on a new dataset of videos depicting real-life object manipulations.
- Towards Unconstrained Joint Hand-Object Reconstruction From RGB Videos89
This work proposes a learning-free fitting approach for hand-object reconstruction which can seamlessly handle two-hand object interactions and shows that it can be applied to datasets with varying levels of difficulty for which training data is unavailable.
- 85
- Multi-scale and snakes for automatic road extraction82
An approach for automatic road extraction in aerial imagery which exploits the scale-space behavior of roads in combination with geometric constrained snake-based edge extraction is proposed, which for the first time allows for a bridging of shadows and partially occluded areas using the heavily disturbed evidence in the image.
- MALT: Improving Reasoning with Multi-Agent LLM Training77
MALT (Multi-Agent LLM Training), a novel post-training strategy that divides the reasoning process into generation, verification, and refinement steps using a sequential pipeline of heterogeneous agents, and allows each agent to specialize by learning from correct and incorrect trajectories, ultimately improving the end-to-end reasoning chain
- gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction75
This work ex-ploit the hand structure and uses it as guidance for SDF-based shape reconstruction of hands and manipulated objects from monocular RGB images, and predicts kinematic chains of pose transformations and align SDFs with highly-articulated hand poses.
- Contact Models in Robotics: A Comparative Analysis74
This article surveys the main contact models and the associated numerical methods commonly used in robotics for simulating advanced robot motions involving contact interactions, and proposes theoretically grounded quantitative criteria on which to build benchmarks assessing both the physical and computational aspects of simulation.
- 74
- AUTOMATIC ROAD EXTRACTION BASED ON MULTI-SCALE MODELING, CONTEXT, AND SNAKES71
So-called ribbon snakes are used not only to extract roads in a robust manner in fine scale, but they can be also used to bridge gaps in the extracted roads due to occlusions or shadows cast by buildings and trees.
- 70
- Predicting Actions from Static Scenes67
This work discovers action-scene correlation for a large number of scene categories and proposes a new application of geo-localized action prediction and demonstrates ability of the method to automatically answer queries such as “Where is a good place for a picnic?” or “Can I cycle along this path?"
- 65
- Learning Interactions and Relationships Between Movie Characters65
This work proposes neural models to learn and jointly predict interactions, relationships, and the pair of characters that are involved, and notes that interactions are informed by a mixture of visual and dialog cues, and presents a multimodal architecture to extract meaningful information from them.
- GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos59
This work addresses the task of generating temporally consistent and physically plausible images of actions and object state transformations and develops and train a conditioned diffusion model dubbed GenHowTo, which achieves superior performance compared to existing methods.
- Learning to Augment Synthetic Images for Sim2Real Policy Transfer58
The main contribution is to optimize the augmentation strategy for sim2real transfer and to enable domain-independent policy learning, and to design an efficient search for depth image augmentations using object localization as a proxy task.
- 54
- Learning from Video and Text via Large-Scale Discriminative Clustering46
This work applies the proposed method to the problem of weakly supervised learning of actions and actors from movies together with corresponding movie scripts and proposes an online optimization algorithm based on the Block-Coordinate Frank-Wolfe algorithm.
- A Prototype System for Computer Vision Based Human Computer Interaction44
Perceptual interfaces based on speech have already started to nd a number of commercial and technical applications, and the ability to interact with computerized equipment without need for special external equipment is attractive.
- Action Modifiers: Learning From Adverbs in Instructional Videos43
A method to learn a representation for adverbs from instructional videos using weak supervision from the accompanying narrations, which outperforms all baselines for video-to-adverb retrieval with a performance of 0.719 mAP.
- 42
- Local spatio-temporal image features for motion interpretation41
Visual motion carries information about the dynamics of ascene, and automatic interpretation of this information is important when designing computer systems for visual navigation, surveillance, human-com ...
- 37
- 35
- Will person detection help bag-of-features action recognition?35
It is found that given locations of people in video, background suppression leads only to a limited gain in performance and it is shown how spatial locations of people enable to incorporate strong geometrical constraints in BoF models and in this way improve the accuracy of action recognition in some cases.
- RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigation32
RoomTour3D, a video-instruction dataset derived from web-based room tour videos that capture real-world indoor spaces and human walking demonstrations, facilitates the development of trainable zero-shot VLN agents, showcasing the potential and challenges of advancing towards open-world navigation.
- Towards reliable object detection in noisy images31
This work addresses object detection in noisy images and proposes a novel low-cost method for image denoising that shows improvements of object detection under varying levels of image noise.
- Much Ado About Time: Exhaustive Annotation of Temporal Data31
This work investigates and determines the most cost-effective way of obtaining high-quality multi-label annotations for temporal data such as videos, and concludes that the optimal strategy is to ask as many questions as possible in a HIT.
- Velocity adaptation of spatio-temporal receptive fields for direct recognition of activities: an experimental study31
This article presents an experimental study of the influence of velocity adaptation when recognizing spatio-temporal patterns using a histogram-based statistical framework, and shows the advantages of using velocity-adapted spatio/temporal receptive fields, compared to directional derivatives or regular partial derivatives for which the filter kernels have not been adapted to the local image motion.
- 30
- 28
- 27
- 27
- 25
- 25
- 24
- 24
- 22
- 20
- 20
- 20
- 19
- 19
- 18
- 17
- 17
- 17
- 17
- 16
- 16
- 15
- 15
- 15
- ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructions14
This work introduces an automatic approach for collecting large step-by-step visual instruction training data from instructional videos and develops and trains ShowHowTo, a video diffusion model capable of generating step-by-step visual instructions consistent with the provided input image.
- 14
- 14
- 13
- 13
- 13
- 13
- 13
- 12
- 12
- 12
- 12
- 12
- GLaD: Geometric Latent Distillation for Vision-Language-Action Models11
This work introduces GLaD, a geometry-aware VLA framework that incorporates 3D geometric priors during pretraining through knowledge distillation, validate that geometry-aware pretraining enhances spatial reasoning and policy generalization without requiring explicit depth sensors or 3D annotations.
- 11
- 11
- 11
- 10
- 10
- 10
- 10
- 10
- 10
- 9
- 9
- 8
- 8
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 6
- 6
- 6
- 5
- 5
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 1
- 1
- Computer Vision – ACCV 20241
This paper attempts to integrate an efficient clustering module into the principled framework for learning structured representation, in which the clustering module is used to provide partition information to guide the cluster-wise compression and the learned embeddings is aligned to desired geometric structures in turn to help for yielding more accurate partitions.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation–
ActCam is presented, a zero-shot method for video generation that jointly transfers character motion from a driving video into a new scene and enables per-frame control of intrinsic and extrinsic camera parameters.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.