Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks78 from public data
- CoCa: Contrastive Captioners are Image-Text Foundation Models1,864
Contrastive Captioner (CoCa), a minimalist design to pretrain an image-text encoder-decoder foundation model jointly with contrastive loss and captioning loss, thereby subsuming model capabilities from contrastive approaches like CLIP and generative methods like SimVLM.
- Scaling Autoregressive Models for Content-Rich Text-to-Image Generation1,583
The Pathways Autoregressive Text-to-Image (Parti) model is presented, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge and explores and highlights limitations of the models.
- Ferret: Refer and Ground Anything Anywhere at Any Granularity625
The introduction of Ferret, a new Multimodal Large Language Model capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions, and a novel and powerful hybrid region representation that integrates discrete coordinates and continuous features jointly to represent a region in the image.
- NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior378
Experiments on real-world indoor and outdoor scenes show that this method can handle challenging camera trajectories and outperforms existing methods in terms of novel view rendering quality and pose estimation accuracy.
- 327
- 245
- Reveal: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory194
An end-to-end Retrieval-Augmented Visual Language Model that learns to encode world knowledge into a large-scale memory, and to retrieve from it to answer knowledge-intensive queries, and achieves state-of-the-art results on visual question answering and image captioning.
- FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation109
This work presents FlowNet3D++, a deep scene flow estimation network that incorporates geometric constraints in the form of point-toplane distance and angular alignment between individual vectors in the flow field, into Flow net3D, and proposes a benchmark for flow estimation on the task of dynamic 3D reconstruction.
- 103
- Direct-PoseNet: Absolute Pose Regression with Photometric Consistency93
This work designs a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering and shows that the method can easily cope with additional unlabeled data without the need for external supervision.
- Efficient Meta Lifelong-Learning with Limited Memory84
This paper identifies three common principles of lifelong learning methods and proposes an efficient meta-lifelong framework that combines them in a synergistic fashion and alleviates both catastrophic forgetting and negative transfer at the same time.
- On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment77
The results show that negative interference is more common than previously known, suggesting new directions for improving multilingual representations and presenting a meta-learning algorithm that obtains better cross-lingual transferability and alleviates negative interference.
- 66
- 63
- Generative models for protein sequence modeling: recent advances and future directions56
This study intends to provide structured guidance and robust framework for delivering a prospective outlook in the ML-driven protein engineering field by providing a comprehensive survey of model details, novel architecture developments, comparisons of model applications, and current challenges.
- NTIRE 2021 Challenge on Perceptual Image Quality Assessment55
The NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021, achieved much better results than existing IQA methods, while the winning method can demonstrate state-of-the-art performance.
- TokenCompose: Text-to-Image Diffusion with Token-Level Supervision52
TokenCompose aims to improve multi-category instance composition by introducing the token-wise consistency terms between the image content and object segmentation maps in the finetuning stage of the Latent Diffusion Model.
- 48
- VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners48
VideoCoCa is presented, that reuses a pretrained image-text contrastive captioner (CoCa) model and adapt it to video-text tasks with minimal extra training, and achieves strong results on video question-answering and video captioning, and establishes a simple and effective video- text baseline for future research.
- 43
- DifIISR: A Diffusion Model with Gradient Guidance for Infrared Image Super-Resolution39
The proposed DifIISR is an infrared image super-resolution diffusion model optimized for visual quality and perceptual performance that achieves task-based guidance for diffusion by injecting gradients derived from visual and perceptual priors into the noise during the reverse process.
- 36
- OmniControlNet: Dual-stage Integration for Conditional Image Generation30
This work provides a two-way integration for the widely adopted ControlNet by integrating external condition generation algorithms into a single dense prediction method and incorporating its individually trained image generation processes into a single model.
- Language Models as Science Tutors25
This work introduces TutorEval and TutorChat, and uses TutorChat to fine-tune Llemma models with 7B and 34B parameters, and creates TutorChat, a dataset of 80,000 long synthetic dialogues about textbooks.
- Understanding Alignment in Multimodal LLMs: A Comprehensive Study21
A novel way of creating multimodal preference data called Bias-Driven Hallucination Sampling (BDHS) is introduced that needs neither additional annotation nor external models, and can achieve competitive performance to previously published alignment work for multimodal models across a range of benchmarks.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.