Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Andrew ZissermanSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works130 from public data
- VGGFace2: A Dataset for Recognising Faces across Pose and Age3,189
A new large-scale face dataset named VGGFace2 is introduced, which contains 3.31 million images of 9131 subjects, with an average of 362.6 images for each subject, and the automated and manual filtering stages to ensure a high accuracy for the images of each identity are described.
- Vggsound: A Large-Scale Audio-Visual Dataset992
The goal is to collect a large-scale audio-visual dataset with low label noise from videos ‘in the wild’ using computer vision techniques and investigates various Convolutional Neural Network architectures and aggregation approaches to establish audio recognition baselines for this new dataset.
- Voxceleb: Large-scale speaker verification in the wild887
A very large-scale audio-visual dataset collected from open source media using a fully automated pipeline and developed and compared different CNN architectures with various aggregation methods and training loss functions that can effectively recognise identities from voice under various conditions are introduced.
- Prompting Visual-Language Models for Efficient Video Understanding539
A simple but strong baseline is presented to efficiently adapt the pre-trained I-VL model, and exploit its powerful ability for resource-hungry video understanding tasks, with minimal training, to optimise a few random vectors that convert video-related tasks into the same format as thePre-training objectives.
- Microscopy cell counting and detection with fully convolutional regression networks505
A new state-of-the-art performance for cell count on standard synthetic image benchmarks is set and it is shown that the FCRNs trained entirely with synthetic data can generalise well to real microscopy images both for cell counting and detections for the case of overlapping cells.
- PMC-LLaMA: toward building open-source language models for medicine474
The proposed PMC-LLaMA model can promote further development of foundation models in medicine, serving as a medical trainable basic generative language backbone, and contribute a large-scale, comprehensive dataset for instruction tuning.
- PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering408
This study reframe the problem of MedVQA as a generation task that naturally follows the human-machine interaction and proposes a generative-based model for medical visual understanding by aligning visual information from a pre-trained vision encoder with a large language model.
- Self-supervised Co-Training for Video Representation Learning405
This paper investigates the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation (InfoNCE) training, and proposes a novel self-supervised co-training scheme to improve the popular infoNCE loss.
- Video Representation Learning by Dense Predictive Coding403
With single stream (RGB only), DPC pretrained representations achieve state-of-the-art self-supervised performance on both UCF101 and HMDB51, outperforming all previous learning methods by a significant margin, and approaching the performance of a baseline pre-trained on ImageNet.
- Utterance-level Aggregation for Speaker Recognition in the Wild375
This paper proposes a powerful speaker recognition deep network, using a ‘thin-ResNet’ trunk architecture, and a dictionary-based NetVLAD or GhostVLAD layer to aggregate features across time, that can be trained end-to-end.
- Memory-Augmented Dense Predictive Coding for Video Representation Learning266
A new architecture and learning framework Memory-augmented Dense Predictive Coding (MemDPC) is proposed for the self-supervised learning from video, in particular for representations for action recognition, trained with a predictive attention mechanism over the set of compressed memories.
- MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training262
This paper adopts a novel triplet extraction module to extract the medical-related information, avoiding unnecessary complexity from language grammar and enhancing the supervision signals, and proposes a novel triplet encoding module with entity translation by querying a knowledge base.
- A Sanity Check for AI-generated Image Detection256
A sanity check on whether the task of AI-generated image detection has been solved is conducted, and AIDE (AI-generated Image DEtector with Hybrid Features), which leverages multiple experts to simultaneously extract visual artifacts and noise patterns is proposed.
- Localizing Visual Sounds the Hard Way252
The key technical contribution is to show that, by training the network to explicitly discriminate challenging image fragments, even for images that do contain the object emitting the sound, the authors can significantly boost the localization performance by introducing a mechanism to mine hard samples and add them to a contrastive learning formulation automatically.
- PromptDet: Towards Open-Vocabulary Detection Using Uncurated Images245
The goal of this work is to establish a scalable pipeline for expanding an object detector towards novel/unseen categories, using zero manual annotations, and the proposed detector, termed as PromptDet, shows superior performance over existing approaches with fewer additional training images andzero manual annotations whatsoever.
- 245
- MAST: A Memory-Augmented Self-Supervised Tracker207
A dense tracking model trained on videos without any annotations is proposed that surpasses previous self-supervised methods on existing benchmarks by a significant margin, and achieves performance comparable to supervised methods.
- Self-supervised Video Object Segmentation by Motion Grouping202
A simple variant of the Transformer is introduced to segment optical flow frames into primary objects and the background, which can be trained in a self-supervised manner, i.e. without using any manual annotations, and achieves superior results compared to previous state-of-the-art self- supervised methods on public benchmarks.
- Class-Agnostic Counting201
The model achieves competitive performance on cell and crowd counting datasets, and surpasses the state-of-the-art on the car dataset using only three training images, when training on the entire dataset, and outperforms all previous methods by a large margin.
- Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval196
Smooth-AP is a plug-and-play objective function that allows for end-to-end training of deep networks with a simple and elegant implementation and improves the performance over the state-of-the-art, especially for larger-scale datasets, thus demonstrating the effectiveness and scalability of Smooth-AP to real-world scenarios.
- 186
- PMC-LLaMA: Towards Building Open-source Language Models for Medicine178
Preliminary evaluations are conducted on three biomedical QA datasets, showing that the PMC-LLaMA model after netuning demonstrates better understanding of biomedical domain-specific concepts, thus achieving high performance on QA benchmarks.
- WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs170
The first benchmark to assess the multi-modal video understanding, that simultaneously encompasses visual, audio, and text inputs, is introduced and the experimental results indicate that existing models face significant challenges in understanding real-world scenarios.
- Collaboration Helps Camera Overtake LiDAR in 3D Detection159
The proposed collaborative camera-only 3D detection (CoCa3D) enables agents to share complementary information with each other through communication, and shows a potential that with sufficient collaboration, the camera might overtake LiDAR in some practical scenarios.
- CounTR: Transformer-based Generalised Visual Counting151
This paper introduces a novel transformer-based architecture for generalised visual object counting, termed as Counting Transformer (CounTR), which explicitly capture the similarity between image patches or with given"exemplars" with the attention mechanism.
- Learning Open-Vocabulary Semantic Segmentation Models From Natural Language Supervision148
A transformer-based model for OVS is proposed, termed as OVSegmentor, which only exploits web-crawled imagetext pairs for pre-training without using any mask annotations, and achieves superior results over state-of-the-art approaches on PASCAL VOC using only 3% data forPre-training.
- Ω-Net (Omega-Net): Fully automatic, multi-view cardiac MR detection, orientation, and segmentation with deep neural networks141
This architecture represents a substantive advancement over prior approaches, with implications for biomedical image segmentation more generally, as well as for convolutional neural network architecture in general.
- Label, Verify, Correct: A Simple Few Shot Object Detection Method128
A simple pseudo-labelling method is introduced to source high-quality pseudo-annotations from the training set, for each new category, vastly increasing the number of training instances and reducing class imbalance; this method finds previously unlabelled instances.
- Synchformer: Efficient Synchronization From Sparse Cues123
This work proposes a novel audio-visual synchronization model, and training that decouples feature extraction from synchronization modelling through multi-modal segment-level contrastive pre-training, which achieves state-of-the-art performance in both dense and sparse settings.
- Temporal Alignment Networks for Long-term Video123
A temporal alignment network that ingests long term video sequences, and associated text sentences, in order to determine if a sentence is alignable with the video, and if it is alignedable, then determine its alignment is proposed.
- Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosis119
The observation shows that, while GPT-4V demonstrates proficiency in distinguishing between medical image modalities and anatomy, it faces significant challenges in disease diagnosis and generating comprehensive reports, highlighting that while large multimodal models have made significant advancements in computer vision and natural language processing, it remains far from being used to effectively support real-world medical applications and clinical decision-making.
- 115
- Large-vocabulary segmentation for medical images with text prompts106
A model that can Segment Anything in 3D medical images, driven by medical terminologies as Text prompts, termed as SAT, consistently outperforms across all 7 human body regions with +7.1% average Dice Similarity Coefficient (DSC) improvement, while showing enhanced scalability and robustness.
- 106
- Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentation105
A novel architecture that consists of two essential components for breaking camouflage, namely, a differentiable registration module to align consecutive frames based on the background, and a motion segmentation module with memory that discovers the moving objects, while maintaining the object permanence even when motion is absent at some point.
- Fully-automated alignment of 3D fetal brain ultrasound to a canonical reference space using multi-task learning96
A multi‐task fully convolutional neural network architecture is proposed to address the problem of 3D fetal brain localization, structural segmentation, and alignment to a referential coordinate system and shows good structural correspondence between fetal anatomies.
- Unsupervised Salient Object Detection with Spectral Cluster Voting93
This paper proposes a simple but effective winner-takes-all voting mechanism for selecting the salient masks, leveraging object priors based on framing and distinctiveness and trains a salient object detector, termed SELF-MASK, which outperforms prior approaches on three unsupervised SOD benchmarks.
- EchoSight: Advancing Visual-Language Models with Wiki Knowledge80
EchoSight is introduced, a novel multimodal Retrieval-Augmented Generation (RAG) framework that enables large language models (LLMs) to answer visual questions requiring fine-grained encyclopedic knowledge, leading to enhanced retrieval outcomes and more accurate VQA responses.
- Layered neural rendering for retiming people in video79
A key property of this model is that it not only disentangles the direct motions of each person in the input video, but also correlates each person automatically with the scene changes they generate---e.g., shadows, reflections, and motion of loose clothing.
- Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks79
This paper introduces a novel neural network architecture that can sample anatomically accurate images conditionally on spatial position of the (real or mock) freehand ultrasound probe, based on conditional generative adversarial networks (GANs), and presents what it believes are state-of-the-art visually realistic ultrasound images simulated by the proposed GAN architecture.
- Comparator Networks77
A Deep Comparator Network is proposed that can ingest a pair of sets, and compute a similarity between the pair, and is Inspired by image retrieval, a novel hard sample mining regime is proposed to control the sampling process, such that the DCN is complementary to the standard image classification models.
- Microscopy cell counting with fully convolutional regression networks77
A new state-of-the-art performance for cell counting on the standard synthetic image benchmarks is set and the potential of the FCRNs for providing cell detections for overlapping cells is shown.
- 76
- 75
- Towards Open-Vocabulary Video Instance Segmentation68
A Large-Vocabulary Video Instance Segmentation dataset (LV-VIS), that contains well-annotated objects from 1,196 diverse categories, significantly surpassing the category size of existing datasets by more than one order of magnitude is collected.
- AutoAD: Movie Description in Context67
An automatic Audio Description model that ingests movies and outputs AD in text form and only train a mapping network that bridges the two models for visually-conditioned text generation is proposed.
- AutoAD II: The Sequel – Who, When, and What in Movie Audio Description66
A new model for automatically generating movie AD, given CLIP visual features of the frames, the cast list, and the temporal locations of the speech is developed, and a new vision-language model is implemented that can ingest the proposals from the character bank whilst conditioning on the visual features using cross-attention.
- Amodal Ground Truth and Completion in the Wild65
This paper uses 3D data to establish an automatic pipeline to determine authentic ground truth amodal masks for partially occluded objects in real images and achieves a new state-of-the-art performance on Amodal segmentation datasets that cover a large variety of objects, in-cluding COCOA and the authors' new MP3D-Amodal dataset.
- OvarNet: Towards Open-Vocabulary Object Attribute Recognition65
This paper considers the problem of simultaneously detecting objects and inferring their visual attributes in an image, even for those with no manual annotations provided at the training stage, resembling an open-vocabulary scenario, and concludes that recognition of semantic category and attributes is complementary for visual scene understanding.
- Annotation-free Audio-Visual Segmentation64
A novel pipeline for generating artificial data for the AVS task without extra manual annotations is initiated and a lightweight model SAMA-AVS which adapts the pre-trained segment anything model (SAM) to the AVS task is introduced.
- Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models62
This paper introduces Fusioner, with a lightweight, transformer-based fusion module, that pairs the frozen visual representation with language concept through a handful of image segmentation data, and demonstrates superior performance over previous models.
- 60
- Audio-Visual Synchronisation in the wild58
The proposed model outperforms the previous state-of-the-art by a significant margin on standard lip reading speech benchmarks, LRS2 and LRS3, and sets the first benchmark for general audio-visual synchronisation with over 160 diverse classes in the new VGG-Sound Sync video dataset.
- Learning to map 2D ultrasound images into 3D space with minimal human annotation58
A convolutional neural network is proposed that predicts the position of 2D ultrasound fetal brain scans in 3D atlas space by sampling 2D slices from 3D fetal brain volumes, and target the model to predict the inverse of the sampling process, resembling the idea of self-supervised learning.
- All you need are a few pixels: semantic segmentation with PixelPick57
This work shows that in order to achieve a good level of segmentation performance, all you need are a few well-chosen pixel labels and demonstrates that deep neural networks can use a handful of such labels to good effect.
- Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisation56
The proposed framework learns strong multi-modal representations that are beneficial to sound localisation and generalization to further applications, and systematically investigates the effects of data augmentations to understand what enables to learn useful representations.
- AutoAD III: The Prequel - Back to the Pixels50
A Q-former-based architecture which ingests raw video and generates AD, using frozen pre-trained visual encoders and large language models is developed, and new evaluation metrics to benchmark AD quality that are well matched to human performance are provided.
- 46
- VP-Nets : Efficient automatic localization of key brain structures in 3D fetal neurosonography46
It is demonstrated that the model can pinpoint the structure in 3D space by visualizing the trained VP‐Nets, despite only 2D supervision being provided for a single stream during training, and consistently outperformed other methods on localization.
- Aerial Monocular 3D Object Detection45
This work proposes a dual-view detection system named DVDET to achieve aerial monocular object detection in both the 2D image space and the 3D physical space and proposes a novel trainable geo-deformable transformation module that can properly warp information from the drone's perspective to the birds' eye view (BEV).
- 45
- 44
- Zero-shot Composed Text-Image Retrieval41
This paper introduces a transformer-based adaptive aggregation model, TransAgg, which employs a simple yet efficient fusion mechanism, to adaptively combine information from diverse modalities, and initiates a scalable pipeline to automatically construct datasets for training CIR model.
- 38
- A General Protocol to Probe Large Vision Models for 3D Physical Understanding37
A general and lightweight protocol to evaluate whether features of an off-the-shelf large vision model encode a number of physical 'properties' of the 3D scene, by training discriminative classifiers on the features for these properties is introduced.
- 36
- Low-Memory CNNs Enabling Real-Time Ultrasound Segmentation Towards Mobile Deployment33
This article demonstrates the power of ‘thin’ CNNs (with very few feature channels) for fast medical image segmentation, and proposes three approaches to training efficient CNNs that can operate in real-time on a CPU, with a low memory footprint, for minimal compromise in accuracy.
- Sensorless volumetric reconstruction of fetal brain freehand ultrasound scans with deep implicit representation31
3D volumes reconstructed by ImplicitVol show significantly better visual and semantic quality than the existing interpolation-based reconstruction approaches, while providing richer information for diagnosis and evaluation of the developing brain.
- Multi-modal Prompting for Low-Shot Temporal Action Localization29
This paper adopts a Transformer-based two-stage action localization architecture with class-agnostic action proposal, followed by open-vocabulary classification, and proposes to prompt the pre-trained CLIP text encoder either with detailed action descriptions, or visually-conditioned instance-specific prompt vectors.
- Joint-Relation Transformer for Multi-Person Motion Prediction29
This paper proposes the Joint-Relation Transformer, which utilizes relation information to enhance interaction modeling and improve future motion prediction, and designs a novel joint-relation fusion layer with relation-aware attention to update both features.
- It's About Time: Analog Clock Reading in the Wild27
This paper creates a scalable pipeline for generating synthetic clocks, significantly reducing the requirements for the labour-intensive annotations, and introduces a clock recognition architecture based on spatial transformer networks (STN), which is trained end-to-end for clock alignment and recognition.
- Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning25
This work proposes EgoVideo, a model with a new lightweight motion adapter to capture fine-grained hand-object motion information and introduces HOD, a novel pipeline employing a hand-object detector and a large language model to generate high-quality narrations with detailed descriptions of hand-object dynamics.
- NamedMask: Distilling Segmenters from Complementary Foundation Models24
It is shown that a standard segmentation architecture trained on category-specific archives of images with appropriate data augmentation achieves impressive semantic segmentation abilities for both single-object and multi-object images.
- Multi-Agent System for Comprehensive Soccer Understanding23
This paper constructs SoccerWiki, the first large-scale multimodal soccer knowledge base, integrating rich domain knowledge about players, teams, referees, and venues to enable knowledge-driven reasoning and introduces SoccerAgent, a novel multi-agent system that decomposes complex soccer questions via collaborative reasoning.
- 23
- The Making and Breaking of Camouflage21
It is shown that camouflage can be measured by the similarity between background and foreground features and boundary visibility, and incorporated into a generative model as an auxiliary loss and shown that effective camouflage images or videos can be synthesised in a scalable manner.
- Robust Regression of Brain Maturation from 3D Fetal Neurosonography Using CRNs21
This work proposes a fully three-dimensional Convolutional Regression Network (CRN) for the task of predicting fetal brain maturation from 3D ultrasound (US) data, and outperforms the current state-of-art for automated BM estimation from3D US images.
- Object-Centric Video Question Answering with Visual Grounding and Referring19
This paper introduces a VideoLLM model, capable of performing both object referring for input and grounding for output in video reasoning tasks, and proposes STOM (Spatial-Temporal Overlay Module), a novel approach that propagates arbitrary visual prompts input to the remaining frames within a video.
- Diagnosing Human-Object Interaction Detectors19
A diagnosis toolbox is introduced to offer a detailed quantitative breakdown of HOI detection models, revealing that the state-of-the-art model RLIPv2 outperforms others primarily due to its significant improvement in multi-label interaction classification accuracy.
- 14
- 14
- 13
- Feature Tracking Cardiac Magnetic Resonance via Deep Learning and Spline Optimization13
A fully automated technique whereby a CMR image sequence is first segmented with a deep, fully convolutional neural network (CNN) architecture, and quadratic basis splines are fitted simultaneously across all cardiac frames using least squares optimization is proposed.
- 11
- Adaptive 3D Localization of 2D Freehand Ultrasound Brain Images10
AdLocUI, a framework that Adaptively Localizes 2D Ultrasound Images in the 3D anatomical atlas without using any external tracking sensor, is proposed and can be used for sensorless 2D freehand ultrasound guidance by the bedside.
- 9
- Zero-shot Unsupervised Transfer Instance Segmentation9
The proposed Zero-shot Unsupervised Transfer Instance Segmentation (ZUTIS) framework is a unified framework for semantic and instance segmentations with solid performance on both tasks compared to state-or-the art unsupervised methods.
- 8
- 8
- 8
- 7
- 7
- 7
- 7
- 6
- 6
- 5
- 5
- 5
- 4
- On-the-Fly Improving Segment Anything for Medical Image Segmentation Using Auxiliary Online Learning4
- 4
- Boost Video Frame Interpolation via Motion Adaptation4
This paper proposes a novel optimization-based VFI method that can adapt to unseen motions at test time, based on a cycle-consistency adaptation strategy that leverages the motion characteristics among video frames.
- 4
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- 2
- 1
- 1
- Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation1
A novel approach that leverages the advantages of online machine learning to enhance Segment Anything (SA) during test time and validates a new, practical, and effective approach for enhancing SA on downstream segmentation tasks (e.g., medical image segmentation).
- 1
- 1
- 1
- 1
- LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models–
A novel framework named Low-Rank Knowledge Decomposition (LoRKD), which explicitly separates gradients from different tasks by incorporating low-rank expert modules and efficient knowledge separation convolution, and significantly improves algorithm efficiency by achieving knowledge separation within a single forward propagation.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.