Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Ahmed ElgammalSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works165 from public data
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models3,392
MiniGPT-4 is presented, which aligns a frozen visual encoder with a frozen advanced LLM, Vicuna, using one projection layer to uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by G PT-4.
- Memory Aware Synapses: Learning What (not) to Forget2,207
This paper argues that, given the limited model capacity and the unlimited new information to be learned, knowledge has to be preserved or erased selectively and proposes a novel approach for lifelong learning, coined Memory Aware Synapses (MAS), which computes the importance of the parameters of a neural network in an unsupervised and online manner.
- Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction1,090
The Social Spatio-Temporal Graph Convolutional Neural Network (Social-STGCNN), which substitutes the need of aggregation methods by modeling the interactions as a graph, and is data efficient, and exceeds previous state of the art on the ADE metric with only 20% of the training data.
- MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning759
This work introduces MiniGPT-v2, a model that can be treated as a unified interface for better handling various vision-language tasks and proposes using unique identifiers for different tasks when training the model to better distinguish each task instruction effortlessly.
- 605
- ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes527
This work designs an approach for combining linguistic and geometric information (in the form of 3D point clouds) and creating multi-modal (3D) neural listeners that can comprehend object-centric natural language and identify the referred object directly in a 3D scene.
- Continual learning with tiny episodic memories504
It is observed that a very simple baseline, which jointly trains on both examples from the current task as well as examples stored in the memory, outperforms state-of-the-art CL approaches with and without episodic memory.
- StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2405
This work rethink the traditional image + video discriminators pair and design a holistic dis-criminator that aggregates temporal information by simply concatenating frames' features, which decreases the training cost and provides richer learning signal to the generator, making it possible to train directly on 10242 videos for the first time.
- A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts403
This work proposes a simple yet effective generative model that takes as input noisy text descriptions about an unseen class and generates synthesized visual features for this class and shows that this method consistently outperforms the state of the art on the largest available benchmarks on Text-based Zero-shot Learning.
- VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning310
This work proposes VisualGPT, which employs a novel self-resurrecting encoder-decoder attention mechanism to quickly adapt the PLM with a small amount of in-domain image-text data and achieves the state-of-the-art result on IU X-ray, a medical report generation dataset.
- 298
- 218
- ArtEmis: Affective Language for Visual Art206
A novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language are presented.
- Adversarial Generation of Continuous Images203
This paper proposes two novel architectural techniques for building INR-based image decoders: factorized multiplicative modulation and multi-scale INRs, and uses them to build a state-of-the-art continuous image GAN.
- 197
- Large-Scale Visual Relationship Understanding176
A new relationship detection model is developed that embeds objects and relations into two vector spaces where both discriminative capability and semantic affinity are preserved and can achieve superior performance even when the visual entity categories scale up to more than 80,000, with extremely skewed class distribution.
- The Shape of Art History in the Eyes of the Machine148
A comprehensive study of several of the state-of-the-art convolutional neural networks applied to the task of style classification on 67K images of paintings and analyzed the learned representation through correlation analysis with concepts derived from art history found that there are a few underlying factors that explain the visual variations of style in art.
- Link the Head to the "Beak": Zero Shot Learning from Noisy Text Description at Part Precision145
A learning framework that is able to connect text terms to its relevant parts and suppress connections to non- visual text terms without any part-text annotations is proposed, which enables terms like beak to be sparsely linked to the visual representation of parts like head, while reduces the effect of non-visual terms like migrate on classifier prediction.
- DesIGN: Design Inspiration from Generative Networks139
This study designs and investigates different image generation models associated with different loss functions to boost creativity in fashion generation, and shows that the proposed creativity criterion yield better overall appreciation than the one employed in Creative Adversarial Networks.
- Learning Hypergraph-regularized Attribute Predictors136
A novel attribute learning framework named Hypergraph-based Attribute Predictor, which is casted as a regularized hypergraph cut problem, in which a collection of attribute projections is jointly learnt from the feature space to a hypergraph embedding space aligned with the attributes.
- ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions135
This paper introduces ChatCaptioner, a novel automatic-questioning method deployed in image captioning that identifies 53% more objects within the image than BLIP-2 alone measured by WordNet synset matching.
- HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models128
HRS-Bench is introduced, a concrete evaluation benchmark for T2I models that is Holistic, Reliable, and Scalable and measures 13 skills that can be categorized into five major categories: accuracy, robustness, generalization, fairness, and bias.
- Aligning Latent and Image Spaces to Connect the Unconnectable103
This work develops a method to generate infinite high-resolution images with diverse and complex content based on a perfectly equivariant patch-wise generator with synchronous interpolations in the image and latent spaces and modify the AdaIN mechanism to work in such a setup.
- Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting88
A novel teacher-student learning paradigm to teach robots about their surrounding environment is proposed, which outperforms the state-of-theart on DAVIS and FBMS with 6.8% and 1.2% in F-measure respectively and improves over the baseline on IVOS dataset.
- 84
- MammalNet: A Large-Scale Video Benchmark for Mammal Recognition and Behavior Understanding77
MammalNet is proposed, a new large-scale animal behavior dataset with taxonomy-guided annotations of mammals and their common behaviors and contains over 18K videos totaling 539 hours, which is ~10 times larger than the largest existing animal behavior datasets.
- Creativity Inspired Zero-Shot Learning77
A learning signal inspired by creativity literature is introduced that explores the unseen space with hallucinated class-descriptions and encourages careful deviation of their visual feature generations from seen classes while allowing knowledge transfer from seen to unseen classes.
- GDPP: Learning Diverse Generations using Determinantal Point Processes76
The Generative DPP approach shows a consistent resistance to mode-collapse on a wide variety of synthetic data and natural image datasets while outperforming state-of-the-art methods for data-efficiency, generation quality, and convergence-time whereas being 5.8x faster than its closest competitor.
- 73
- A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning68
It is argued that the pretrained feature extractor itself can be strong enough to achieve a competitive or even better continual learning performance on Split-CIFAR100 and CoRe 50 benchmarks and a simple baseline is designed that achieves 88.53% on 10-Split-CifAR-100, surpassing most state-of-the-art continual learning methods that are all initialized using the same pretrained transformer model.
- 66
- It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collection59
This work proposes a contrastive data collection approach to balance ArtEmis with a new complementary dataset such that a pair of similar images have contrasting emotions (one positive and one negative).
- LLM as A Robotic Brain: Unifying Egocentric Memory and Control53
A novel and generalizable framework called LLM-Brain: using Large-scale Language Model as a robotic brain to unify egocentric memory and control, utilizing a zero-shot learning approach.
- Choose Your Neuron: Incorporating Domain Knowledge Through Neuron-Importance42
This work learns to map domain knowledge about novel “unseen” classes onto this dictionary of learned concepts and optimizes for network parameters that can effectively combine these concepts – essentially learning classifiers by discovering and composing learned semantic concepts in deep networks.
- 39
- Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models34
A supervised learning framework called Bi-Factorial Preference Optimization is proposed, which re-parameterizes a joint RLHF objective of both safety and helpfulness into a single supervised learning objective and significantly outperforms existing approaches in both safety and helpfulness.
- Overcoming Generic Knowledge Loss with Selective Parameter Update30
This work proposes a novel approach that, instead of updating all parame-ters equally, localizes the updates to a sparse set of parame-ters relevant to the task being learned, striking a balance between efficiency and new task performance, while main-taining the transferability and generalizability of foundation models.
- 29
- No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languages27
ArtELingo-28 is presented, a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations (140 annotations per image) and finds that cross-lingual transfer is more successful for culturally-related languages.
- 27
- 26
- 24
- 23
- The Second Challenge on Cross-Domain Few-Shot Object Detection at NTIRE 2026: Methods and Results21
This report presents a detailed overview of the NTIRE 2026 CD-FSOD Challenge, including a summary of the submitted approaches and an analysis of the final results across all participating teams.
- Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents20
V-RAG is proposed, a novel, vision-centric retrieval-augmented generation (RAG) framework that leverages a suite of multimodal vision encoders, each optimized for specific strengths, and a dedicated question-document relevance module that sets a new standard on document haystack benchmarks.
- OxfordTVG-HIC: Can Machine Make Humorous Captions from Images?20
This paper presents OxfordTVG-HIC (Humorous Image Captions), a large-scale dataset for humour generation and understanding and identifies the visual and linguistic cues influential for evoking humour prediction and generation and observes qualitatively that these cues are aligned with the benign violation theory of humour in cognitive psychology.
- 20
- 20
- 18
- 17
- 17
- 17
- How Well Can Vision Language Models See Image Details?16
This research reveals that incorporating pixel value prediction as one of the VLM pre-training tasks and vision encoder adaptation markedly boosts VLM performance on downstream image-language understanding tasks requiring detailed image perception and video game decision making.
- 15
- 14
- Label Delay in Online Continual Learning13
A new continual learning framework with explicit modeling of the label delay between data and label streams over time steps is introduced, showing experimentally that the method is the least affected by the labeldelay factor and in some cases successfully recovers the accuracy of the non-delayed counterpart.
- 13
- Diffusion-Based Imaginative Coordination for Bimanual Manipulation12
A multi-frame latent prediction strategy that encodes future states in a compressed latent space, preserving task-relevant features and a unidirectional attention mechanism where video prediction is conditioned on the action, while action prediction remains independent of video prediction is introduced.
- Exploring Open-Vocabulary Semantic Segmentation without Human Labels12
This work presents ZeroSeg, a novel method that leverages the existing pretrained vision-language (VL) model to train open-vocabulary zero-shot semantic segmentation models, achieving state-of-the-art performance when compared to other zero- shot segmentation methods under the same training data, while also performing competitively compared to strongly supervised methods.
- 12
- 12
- 12
- 11
- 10
- 10
- 10
- 9
- 9
- 4D-Bench: Benchmarking Multi-Modal Large Language Models for 4D Object Understanding8
4D-Bench is introduced, the first benchmark to evaluate the capabilities of MLLMs in 4D object understanding, featuring tasks in 4D object Question Answering and 4D object captioning, and indicates that MLLMs generally exhibit weaker temporal understanding compared to their appearance understanding.
- Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation8
DietCL outperforms, by a large margin, all existing supervised CL algorithms as well as more recent continual semi-supervised methods, and meticulously allocates computational budget for both unlabeled and labeled data jointly.
- 8
- 8
- 8
- 7
- 7
- 7
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- Continual Zero-Shot Learning through Semantically Guided Generative Random Walks5
This paper introduces generalization-bound tools and provides the first theoretical explanation for the benefits of generative modeling to CZSL tasks, and proposes a learning algorithm that employs a novel semantically guided Generative Random Walk (GRW) loss.
- 5
- 5
- 5
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.