Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks375 from public data
- 2,128
- Early versus late fusion in semantic video analysis1,010
It is shown by experiment on 184 hours of broadcast video data and for 20 semantic concepts, that late fusion tends to give slightly better performance for most concepts, however, for those concepts where early fusion performs better the difference is more significant.
- The challenge problem for automated detection of 101 semantic concepts in multimedia691
The challenge problem for generic video indexing is introduced to gain insight in intermediate steps that affect performance of multimedia analysis methods, while at the same time fostering repeatability of experiments.
- Multimodal Video Indexing: A Review of the State-of-the-art574
A unifying and multimodal framework is put forward, which views a video document from the perspective of its author, which forms the guiding principle for identifying index types, for which automatic methods are found in literature.
- VideoLSTM convolves, attends and flows for action recognition489
A new architecture for end-to-end sequence learning of actions in video, called VideoLSTM, which introduces motion-based attention and demonstrates how the attention from VideoLSTM can be used for action localization by relying on just the action class label.
- Concept-Based Video Retrieval452
This paper presents a component-wise decomposition of such an interdisciplinary multimedia system, covering influences from information retrieval, computer vision, machine learning, and human–computer interaction and lays down the anatomy of a concept-based video search engine.
- 401
- 378
- 305
- 258
- 249
- The MediaMill TRECVID 2004 Semantic Video Search Engine240
The 2008 edition of the TRECVID bench- mark has been the most successful MediaMill participation to date, resulting in the top ranking for both concept de- tection and interactive search, and a runner-up ranking for automatic retrieval.
- 235
- 225
- 210
- Learning to Learn with Variational Information Bottleneck for Domain Generalization206
A probabilistic meta-learning model for domain generalization is introduced, in which classifier parameters shared across domains are modeled as distributions, which enables better handling of prediction uncertainty on unseen domains.
- Learning tag relevance by neighbor voting for social image retrieval202
A novel algorithm that scalably and reliably learns tag relevance by accumulating votes from visually similar neighbors is proposed, and treated as tag frequency, learned tag relevance is seamlessly embedded into current tag-based social image retrieval paradigms.
- PointMixup: Augmentation for Point Clouds201
It is proved that the PointMixup finds the shortest path between two point clouds and that the interpolation is assignment invariant and linear, which allows to introduce strong interpolation-based regularizers such as mixup and manifold mixup to the point cloud domain.
- Online Action Detection200
A realistic dataset composed of 27 episodes from 6 popular TV series and an evaluation protocol for fair comparison is introduced, showing this is a challenging problem for which none of the methods provides a good solution.
- 200
- 190
- Skeleton-Contrastive 3D Action Representation Learning189
This paper proposes inter-skeleton contrastive learning, which learns from multiple different input skeleton representations in a cross-contrastive manner, and contributes several skeleton-specific spatial and temporal augmentations which further encourage the model to learn the spatio-temporal dynamics of skeleton data.
- 178
- Socializing the Semantic Gap173
A two-dimensional taxonomy is introduced to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations.
- Objects2action: Classifying and Localizing Actions without Any Video Example172
Objects2action is a semantic word embedding that is spanned by a skip-gram model of thousands of object categories that proposes a mechanism to exploit multiple-word descriptions of actions and objects and demonstrates how to extend the zero-shot approach to the spatio-temporal localization of actions in video.
- APT: Action localization proposals from dense trajectories169
This paper proposes bypassing the segmentation step of existing proposals completely by generating proposals directly from the dense trajectories used to represent videos during classification, using an efficient proposal generation algorithm to handle the high number of trajectories in a video.
- Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation165
An Attentionbased Multi-Context Guiding (A-MCG) network is proposed, which consists of three branches: the support branch, the query branch, the feature fusion branch, and adopts a spatial attention along the fusion branch to highlight context information from several scales, enhancing self-supervision in one-shot learning.
- 143
- Spot On: Action Localization from Pointly-Supervised Proposals129
An overlap measure between action proposals and points is introduced and incorporated into the objective of a non-convex Multiple Instance Learning optimization and shows that the approach is competitive to the state-of-the-art.
- 128
- The ImageNet Shuffle125
This paper introduces a bottom-up and top-down approach for reorganization of the ImageNet hierarchy based on all its 21,814 classes and more than 14 million images to deal with the problems of over-specific classes and classes with few images.
- TubeR: Tubelet Transformer for Video Action Detection120
TubeR is a simple solution to directly detect an action tubelet in a video by simultaneously performing action localization and recognition from a single representation and outperforms the previous state-of-the-art on commonly used action detection datasets AVA, UCF101-24 and JHMDB51-21.
- VideoStory116
This paper proposes a new video representation for few-example event recognition and translation called VideoStory, which outperforms an embedding without joint-objective and alternatives without any embedding, and translates a previously unseen video to its most likely description from visual content only.
- 115
- A comparison of color features for visual concept classification113
It can be derived that invariance to light intensity changes and light color changes affects concept classification and that the usefulness of invariance is concept-specific.
- 109
- 109
- Comparing compact codebooks for visual categorization105
This paper compares four approaches to achieve a compact codebook vocabulary while retaining categorization performance and investigates the trade-off between codebook compactness and categorizationperformance.
- On Measuring and Controlling the Spectral Bias of the Deep Image Prior98
A frequency-band correspondence measure is introduced to characterize the spectral bias of the deep image prior, where low-frequency image signals are learned faster and better than high-frequency counterparts.
- Dance With Flow: Two-In-One Stream Action Detection97
This paper proposes to embed RGB and optical-flow into a single two-in-one stream network with new layers, and demonstrates that leveraging the motion condition to modulate RGB features improves detection accuracy.
- Unsupervised multi-feature tag relevance learning for social image retrieval97
Two simple and effective unsupervised methods based on the classical Borda Count, comparable to a state-of-the-art supervised alternative, but without the need of any training data are introduced.
- Pixelated Semantic Colorization91
A network that jointly optimizes a color embedding loss, a semantic segmentation loss and a color generation loss, in an end-to-end fashion and produces more realistic and finer results compared to the colorization state-of-the-art.
- Local Alignments for Fine-Grained Categorization81
By using object detectors as voters to generate object confidence saliency maps, the proposed local alignments set a new state-of-the-art on both the fine-grained birds and dogs datasets, even without any human intervention.
- Composite Concept Discovery for Zero-Shot Video Event Detection81
It is demonstrated that by combining concepts into composite concepts, one can train more accurate classifiers for the concept vocabulary, which leads to improved zero-shot event detection.
- 80
- Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on N-Spheres78
By introducing a spherical exponential mapping on n-spheres at the regression output, this work obtains well-behaved gradients, leading to stable training and shows how the spherical regression can be utilized for several computer vision challenges, specifically viewpoint estimation, surface normal estimation and 3D rotation estimation.
- 77
- Early Embedding and Late Reranking for Video Captioning77
The proposed solution for the MSR Video to Language Challenge is ranked at the 4th place in terms of overall performance, while scoring the best CIDEr-D, which measures the human-likeness of generated captions.
- 76
- 73
- Bayesian Prompt Learning for Image-Language Model Generalization72
This work frames prompt learning from the Bayesian perspective and formulate it as a variational inference problem, and demonstrates empirically on 15 benchmarks that Bayesian prompt learning provides an appropriate coverage of the prompt space, prevents learning spurious features, and exploits transferable invariant features.
- 72
- Active Transfer Learning with Zero-Shot Priors: Reusing Past Datasets for Future Tasks72
This work uses zero-shot classifiers to guide the learning process by linking the new task to the the existing classifiers, and proposes an effective active learning algorithm which learns the best possible target classification model with minimum human labeling effort.
- 71
- Recommendations for video event recognition using concept vocabularies70
The recommendation that for effective event recognition the concept vocabulary should contain more than 200 concepts and be diverse by covering object, action, scene, people, animal and attribute concepts is considered.
- New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval67
This paper presents a large-scale data set of real-world emoji usage collected from Twitter, and presents baseline results on the challenge of predicting emoji from both text and images, using state-of-the-art neural networks.
- 66
- 65
- 63
- 62
- Image2Emoji61
It is proposed that the widespread adoption of emoji suggests a semantic universality which is well-suited for interaction with visual media, and the value of visual, textual and multi-modal prediction of emoji is demonstrated.
- Latent Factors of Visual Popularity Prediction61
This work proposes a latent ranking approach that takes into account not only the distinctive visual cues in popular images, but also those in unpopular images, and allows for insight into the semantics of image popularity on social networks.
- 60
- Audio-Adaptive Activity Recognition Across Video Domains59
An audio-adaptive encoder and associated learning methods that discriminatively adjust the visual feature representation as well as addressing shifts in the semantic distribution are proposed, which effectively models the cross-modal interaction across domains.
- 59
- 58
- Bag-of-Fragments57
The bag-of-fragments forms an effective encoding for event detection and is able to provide a precise temporally localized event recounting, and it is concluded that fragments matter for video event Detection and recounting.
- 55
- MediaMill54
This technical demonstration shows how the lexicon of concepts can be exploited for novel applications using advanced semantic visualizations and how users can obtain highly relevant retrieval results using query-by-concept.
- University of Amsterdam at THUMOS Challenge 201452
This notebook paper describes the approach for the action classification task of the THUMOS Challenge 2014, investigating and exploiting the action-object relationship by capturing both motion and related objects.
- 52
- Motion Flow Matching for Human Motion Synthesis and Editing50
A novel generative model designed for human motion generation featuring efficient sampling and effectiveness in motion editing applications and establishes a new state-of-the-art Fr\'echet Inception Distance on the KIT-ML dataset.
- 49
- 49
- 48
- 47
- [Invited Paper] TRECVid Semantic Indexing of Video: A 6-Year Retrospective45
The data, protocol and metrics used for the main and the secondary tasks, the results obtained and the main approaches used by participants are described.
- Learning rich semantics from news video archives by style analysis45
A generic and robust framework for news video indexing which is founded on a broadcast news production model and demonstrates that the accuracy of the proposed style analysis framework for classification of several rich semantic concepts is state-of-the-art.
- BoxeR: Box-Attention for 2D and 3D Transformers44
BoxeR, short for Box Transformer, which attends to a set of boxes by predicting their transformation from a reference window on an input feature map, and computes attention weights on these boxes by considering its grid structure, which improves the learning capability of transformers for several vision tasks.
- Classifying tag relevance with relevant positive and negative examples44
This work argues the importance of relevant positive and relevant negative examples for tag relevance estimation, and proposes a system that selects positive and negative examples, deemed most relevant with respect to the given tag from crowd-annotated images.
- Searching informative concept banks for video event detection43
This paper proposes in this paper an algorithm that learns from examples what concepts in a bank are most informative per event, and proposes an approximate solution that finds the optimal concept bank using a cross-entropy optimization.
- Query on demand video browsing43
A novel method for browsing a large collection of news video by linking various forms of related video fragments together as threads, where in general the query result and the timeline are the most important threads.
- 42
- Learned Lexicon-Driven Interactive Video Retrieval42
This paper combines automatic learning of a large lexicon of semantic concepts with traditional video retrieval methods into a novel approach to narrow the semantic gap.
- TagBook: A Semantic Video Representation Without Supervision for Event Detection41
Despite its simple nature, the proposed TagBook video representation is remarkably effective for few-example and zero-example event detection, even outperforming very recent state-of-the-art alternatives building on supervised representations.
- 39
- MediaMill at TRECVID 2014: Searching Concepts, Objects, Instances and Events in Video38
The 2014 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the best result for concept detection and object localization.
- Attributes Make Sense on Segmented Objects38
This paper proposes joint learning for simultaneous object classification and segment proposal ranking, solely on the basis of attributes, and concludes that attributes make sense on segmented objects.
- 38
- Personalizing automated image annotation using cross-entropy37
A cross-entropy based learning algorithm is proposed which personalizes a generic annotation model by learning from a user's multimedia tagging history and optimizes the personalization process in terms of a performance measurement which can be flexibly chosen.
- 36
- 36
- Guess Where? Actor-Supervision for Spatiotemporal Video Action Localization35
Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-super supervised alternatives.
- Guess where? Actor-supervision for spatiotemporal action localization35
Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-super supervised alternatives.
- 31
- 30
- 29
- 28
- 28
- 28
- 28
- 28
- 27
- 27
- 26
- 26
- 24
- 23
- 23
- 23
- 23
- 22
- 22
- 22
- 22
- 22
- 21
- 21
- 21
- 21
- 20
- 20
- 20
- 20
- 19
- 18
- 18
- 18
- 18
- 17
- 17
- 17
- 17
- 17
- 16
- 16
- 16
- 16
- 15
- 15
- 15
- Deep Learning for Visual Understanding: Part 2 [From the Guest Editors]15
Computer vision aims to make computers understand the seen data in the same way that human vision does, and incorporates several scientific disciplines such as signal processing, machine learning, applied mathematics, sensing, geometry, optimization, statistics, and data sciences.
- 15
- 15
- 15
- 14
- CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation14
Inspired by human cooperation schemes, CaPo improves cooperation efficiency with two phases: 1) meta-plan generation, and 2) progress-adaptive meta-plan and execution, which eliminates redundant actions, improving the overall cooperation efficiency of agents.
- 14
- 14
- 14
- 14
- 14
- 13
- 13
- 13
- 13
- 13
- Video Indexing, Search, Detection, and Description with Focus on TRECVID12
This tutorial presents and discusses some of the most important and fundamental content-based video retrieval problems such as recognizing predefined visual concepts, searching in videos for complex ad-hoc user queries, searching by image/video examples in a video dataset to retrieve specific objects, persons, or locations, detecting events, and finally bridging the gap between vision and language.
- 12
- 12
- 12
- 12
- 12
- 12
- 12
- 11
- 11
- 11
- 11
- 11
- 11
- 10
- 10
- 10
- 10
- 10
- 10
- 9
- 9
- 9
- 9
- 9
- 8
- 8
- 8
- 8
- 8
- 8
- 8
- 7
- 7
- Deep Learning for Visual Understanding [From the Guest Editors]7
In the past decade, there has been a transformative and permanent revolution in computer vision cultivated by the reinvigorated adoption of deep learning for visual understanding tasks, driven by the increasing availability of large annotated data sets, efficient training techniques, and faster computational platforms.
- 7
- 7
- 7
- 6
- 6
- 6
- Special issue on weakly supervised learning6
This dissertation aims to provide a history of web exceptionalism from 1989 to 2002, a period chosen in order to explore its roots as well as specific cases up to and including the year in which descriptions of “Web 2.0” began to circulate.
- 6
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- Advances in Multimedia Information Processing – PCM 20145
Some of the challenges raised by large-scale geo-referenced applications, some approaches and solutions, and future opportunities are presented, especially because real-world sensor data is noisy and sometimes inaccurate.
- 5
- 5
- 5
- 5
- 5
- 4
- SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning4
No method generalizes consistently across all factors, video-only transformers perform better under domain shifts, CNNs outperform for fine-grained tasks, and video-text models often underperform despite large scale pretraining.
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- Structured-Noise Masked Modeling for Video, Audio and Beyond1
This work introduces a structured noise-based masking approach that generates structured masks that capture modality-specific patterns without requiring handcrafted heuristics or access to the data.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Best practices for learning video concept detectors from social media examples (Author's Manuscript)–
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.