Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks125 from public data
- ImageNet Large Scale Visual Recognition Challenge43,796
The creation of this benchmark dataset and the advances in object recognition that have been possible as a result are described, and the state-of-the-art computer vision accuracy with human accuracy is compared.
- What’s the Point: Semantic Segmentation with Point Supervision1,128
This work takes a natural step from image-level annotation towards stronger supervision: it asks annotators to point to an object if one exists, and incorporates this point supervision along with a novel objectness potential in the training loss function of a CNN model.
- Art and the science of generative AI759
The generative capabilities of generative AI are likely to fundamentally alter the creative processes by which creators formulate ideas and put them into production, and may be many sectors of society may be reimagined.
- End-to-End Learning of Action Detection from Frame Glimpses in Videos630
A fully end-to-end approach for action detection in videos that learns to directly predict the temporal bounds of actions and uses REINFORCE to learn the agent's decision policy.
- Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos479
A novel variant of long short-term memory deep networks is defined for modeling these temporal relations via multiple input and output connections and it is shown that this model improves action labeling accuracy and further enables deeper understanding tasks ranging from structured retrieval to action prediction.
- Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation434
A simple but surprisingly effective visual recognition benchmark for studying bias mitigation, and a simple but similarly effective alternative to the inference-time Reducing Bias Amplification method of Zhao et al., and design a domain-independent training technique that outperforms all other methods.
- Human Uncertainty Makes Classification More Robust411
It is shown that, while contemporary classifiers fail to exhibit human-like uncertainty on their own, explicit training on this dataset closes this gap, supports improved generalization to increasingly out-of-training-distribution test datasets, and confers robustness to adversarial attacks.
- Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy364
This paper examines ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods, and considers three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology.
- REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets230
The key aim of the work is to tackle the machine learning bias problem early in the pipeline, and enables the preemptive analysis of large-scale datasets.
- ViBE: A Tool for Measuring and Mitigating Bias in Image Datasets230
The key aim of the work is to tackle the machine learning bias problem early in the pipeline, and enables the preemptive analysis of large-scale datasets.
- Attribute Learning in Large-Scale Datasets223
This work learns 20 visual attributes and uses them in a zero-shot transfer learning experiment as well as to make visual connections between semantically unrelated object categories.
- 220
- Object-Centric Spatial Pooling for Image Classification213
A framework that learns object detectors using only image-level class labels, or so-called weak labels is proposed, comparable in accuracy with state-of-the-art weakly supervised detection methods and significantly outperforms SPM-based pooling in image classification.
- "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction210
A mixed-methods study with 20 end-users of a real-world AI application, the Merlin bird identification app, found that participants desire practically useful information that can improve their collaboration with the AI, more so than technical system details.
- Fair Attribute Classification through Latent Space De-biasing195
This work uses GANs to generate realistic-looking images, and perturb these images in the underlying latent space to generate training data that is balanced for each protected attribute, and empirically demonstrates that target classifiers trained on the augmented dataset exhibit a number of both quantitative and qualitative benefits.
- CornerNet-Lite: Efficient Keypoint based Object Detection187
CornerNet-Lite is a combination of two efficient variants of CornerNet: Corner net-Saccade, which uses an attention mechanism to eliminate the need for exhaustively processing all pixels of the image, and CornerNet-Squeeze, which introduces a new compact backbone architecture that addresses the two critical use cases in efficient object detection.
- Understanding and Evaluating Racial Biases in Image Captioning181
Differences in caption performance, sentiment, and word choice between images of lighter versus darker-skinned people are found to be greater in modern captioning systems compared to older ones, thus leading to concerns that without proper consideration and mitigation these differences will only become increasingly prevalent.
- Scalable multi-label annotation176
An algorithm that exploits correlation, hierarchy, and sparsity of the label distribution is proposed that results in up to 6x reduction in human computation time compared to the naive method of querying a human annotator for the presence of every object in every image.
- HIVE: Evaluating the Human Interpretability of Visual Explanations149
HIVE (Human Interpretability of Visual Explanations), a novel human evaluation framework that assesses the utility of explanations to human users in AI-assisted decision making scenarios, and enables falsifiable hypothesis testing, cross-method comparison, and human-centered evaluation of visual interpretability methods is introduced.
- What Actions are Needed for Understanding Human Actions in Videos?139
This work analyzes datasets, evaluation metrics, algorithms, and potential future directions to discover that fine-grained understanding of objects and pose when combined with temporal reasoning is likely to yield substantial improvements in algorithmic accuracy.
- Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation134
This work grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: which demographic attributes to include as dataset labels, how to handle the progressively smaller size of subgroups during model training, and how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups.
- Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies121
It is found that the presence of explanations increases reliance on both correct and incorrect responses, however, users observe less reliance on incorrect responses when sources are provided or when explanations exhibit inconsistencies.
- SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition106
SpatialSense is a dataset specializing in spatial relation recognition which captures a broad spectrum of such challenges, allowing for proper benchmarking of computer vision techniques, and provides a path forward to advancing the spatial reasoning capabilities ofComputer vision systems.
- Toward More Gender Diversity in CS through an Artificial Intelligence Summer Program for High School Girls106
The curriculum designed to achieve goals of the pilot program are to increase interest in AI, contextualize technically rigorous AI concepts through societal impact, and address barriers that could discourage 10th grade girls from pursuing computer science.
- 92
- 73
- GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition66
GeoDE is introduced, a geographically diverse dataset with 61,940 images from 40 classes and 6 world regions, with no personally identifiable information, collected by soliciting images from people around the world, to highlight and begin to mitigate the shortcomings in current models, despite its relatively small size.
- 65
- Overlooked Factors in Concept-Based Explanations: Dataset Choice, Concept Learnability, and Human Capability63
It is argued that only easily learnable concepts should be used in concept-based explanations, and a much stricter upper bound of 32 concepts or less is revealed, beyond which the explanations are much less practically useful.
- Predictive-Corrective Networks for Action Detection55
This work rethink both the underlying network architecture and the stochastic learning paradigm for temporal data, and derives a series of novel recurrent neural networks that sequentially make top-down predictions about the future and then correct those predictions with bottom-up observations.
- Humans, AI, and Context: Understanding End-Users’ Trust in a Real-World Computer Vision Application54
A holistic and nuanced understanding of trust in AI is provided through a qualitative case study of a real-world computer vision application, finding domain knowledge and context are important factors for trust-related assessment and decision-making.
- Overwriting Pretrained Bias with Finetuning Data54
It is found that models finetuned on top of pretrained models can indeed inherit their biases, but this bias can be corrected for through relatively minor interventions to the finetuning dataset, and often with a negligible impact to performance.
- 53
- 52
- Gender Artifacts in Visual Datasets44
It is claimed that attempts to remove these artifacts from such datasets are largely infeasible as certain removed artifacts may be necessary for the downstream task of object recognition and the responsibility lies with researchers and practitioners to be aware that the distribution of images within datasets is highly gendered and develop fairness-aware methods which are robust to these distributional shifts across groups.
- 41
- Boundary Guided Learning-Free Semantic Control with Diffusion Models41
It is proved that the critical latent space the authors found already exhibits semantic subspace boundaries at the generic level in unconditional DDMs, which allows us to do controllable manipulation by guiding the denoising trajectory towards the targeted boundary via a single-step operation.
- Towards Unique and Informative Captioning of Images39
A new metric is designed (SPICE-U) that is better correlated with human judgements compared to SPICE, and effectively captures notions of diversity and descriptiveness, and improves three different state-of-the-art models on SPICE- U as well as average score over existing metrics.
- Training Conditional Random Fields for Maximum Labelwise Accuracy37
This work gives a gradient-based procedure for minimizing an arbitrarily accurate approximation of the empirical risk under a Hamming loss function.
- ICONS: Influence Consensus for Vision-Language Data Selection35
This work introduces ICONS, a gradient-based Influence CONsensus approach for vision-language data Selection, which leverages first-order training dynamics to estimate each example's influence on validation performance, then aggregates these estimates across tasks via majority voting.
- Learning to Learn from Noisy Web Videos32
This work proposes a reinforcement learning-based formulation for selecting the right examples for training a classifier from noisy web search results and uses Q-learning to learn a data labeling policy on a small labeled training dataset, and then uses this to automatically label noisy web data for new visual concepts.
- 32
- ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms31
Through comprehensive experiments, it is shown that OOD detectors are more sensitive to covariate shift than to semantic shift, and the benefits of recent OOD detection algorithms on semantic shift detection is minimal.
- Much Ado About Time: Exhaustive Annotation of Temporal Data31
This work investigates and determines the most cost-effective way of obtaining high-quality multi-label annotations for temporal data such as videos, and concludes that the optimal strategy is to ask as many questions as possible in a HIT.
- Benchmark suites instead of leaderboards for evaluating AI fairness29
The advocate for collecting benchmarks into carefully curated “benchmark suites” is advocated, which can provide researchers and practitioners with tools for understanding the wide range of potential harms and trade-offs among different aspects of fairness.
- Multi-query Video Retrieval25
This paper shows that multi-query retrieval task effectively mitigates the dataset noise introduced by imperfect annotations and better correlates with human judgement on evaluating retrieval abilities of current models, and investigates several methods which leverage multiple queries at training time.
- Enabling Detailed Action Recognition Evaluation Through Video Dataset Augmentation23
The Human-centric Analysis Toolkit (HAT) is introduced, which enables evaluation of learned background bias without the need for new manual video annotation and is open-source to enable the community to design more robust and generalizable human action recognition models.
- 22
- 20
- 19
- Take the Scenic Route: Improving Generalization in Vision-and-Language Navigation19
This work finds that shortest path sampling, which is used by both the R2R benchmark and existing augmentation methods, encode biases in the action space of the agent which they dub as action priors, and proposes a path sampling method based on random walks to augment the data.
- 17
- The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generation16
NoiseQuery is introduced as a novel method for enhanced noise initialization in versatile goal-driven text-to-image (T2I) generation to leverage an aligned Gaussian noise as implicit guidance to complement explicit user-defined inputs, such as text prompts, for better generation quality and controllability.
- What is Dataset Distillation Learning?16
Light is shed on the intricate nature of distilled data, providing a better understanding on how they can be effectively utilized, and it is revealed that individual distilled data points contain meaningful semantic information.
- Vision-Language Dataset Distillation16
This work takes the first steps towards expanding on the idea of trajectory matching to create a distillation method for vision-language datasets by jointly distill the images and their corresponding language descriptions in a contrastive formulation.
- ELUDE: Generating interpretable explanations via a decomposition into labelled and unlabelled features16
A novel explanation framework ELUDE (Expla-nation via Labelled and Unlabelled DEcomposition) is developed that decomposes a model’s prediction into two parts: one that is explainable through a linear combination of the semantic attributes, and another that is dependent on the set of uninterpretable features.
- What's in a Question: Using Visual Questions as a Form of Supervision16
This work proposes two simple but surprisingly effective modifications to the standard visual question answering models that allow them to make use of weak supervision in the form of unanswered questions associated with images and demonstrates that a simple data augmentation strategy inspired by the insights results in a 7.1% improvement on the standard VQA benchmark.
- 14
- 14
- Portraying Large Language Models as Machines, Tools, or Companions Affects What Mental Capacities Humans Attribute to Them13
It is found that the intentional portrayal of LLMs increases the attribution of mental capacities to LLMs, and that individuals tend to attribute mind-related capacities the most, followed by heart- then body-related capacities.
- Joint calibration of Ensemble of Exemplar SVMs12
Results show that the joint calibration procedure outperforms independent calibration on the task of classifying windows as belonging to an object class or not; and this improved window classifier leads to better performance on the object detection task.
- Unifying Specialized Visual Encoders for Video Language Models11
The method, MERV, Multi-Encoder Representation of Videos, leverages multiple frozen visual encoders to create a unified representation of a video, providing the VideoLLM with a comprehensive set of specialized visual knowledge.
- 10
- Video Models Reason Early: Exploiting Plan Commitment for Maze Solving8
Analysis of the internal planning dynamics of video models using 2D maze solving as a controlled testbed reveals that current video models possess deeper reasoning capabilities than previously recognized, which can be elicited more reliably with better inference-time scaling.
- 7
- Analyzing the Roles of Language and Vision in Learning from Limited Data6
It is found that a language model leveraging all components recovers a majority of a VLM's performance, despite its lack of visual input, and that language seems to allow this by providing access to prior knowledge and reasoning.
- 5
- Motion Attribution for Video Generation4
To the authors' knowledge, this is the first framework to attribute motion rather than visual appearance in video generative models and to use it to curate fine-tuning data.
- Machine understanding4
A framework for asking more precise questions and making more precise claims about machine understanding is developed, conceptualizing understanding as a relation between a system and a target of understanding (T), and how to specify the relation, the system, and the target.
- Interactivity x Explainability: Toward Understanding How Interactivity Can Improve Computer Vision Explanations4
It is found that while interactivity enhances user control, facilitates rapid convergence to relevant information, and allows users to expand their understanding of the model and explanation, it also introduces new challenges.
- SOWing Information: Cultivating Contextual Coherence with MLLMs in Image Generation4
This work reframing disordered diffusion as a powerful tool for text-vision-to-image generation (TV2I) tasks, achieving pixel-level condition fidelity while maintaining visual and semantic coherence throughout the image.
- DETER: Detecting Edited Regions for Deterring Generative Manipulations4
DETER is introduced, a large-scale dataset for DETEcting edited image Regions and deterring modern advanced generative manipulations, and it is confirmed that human deep fake detection rate on DETER is 20.4% lower than on other fake datasets.
- 3
- 3
- 2
- Seeing Beyond the Scene: Analyzing and Mitigating Background Bias in Action Recognition2
This work presents a systematic analysis of background bias across classification models, contrastive text-image pretrained models, and Video Large Language Models and finds that all exhibit a strong tendency to default to background reasoning.
- ICON$^2$: Reliably Benchmarking Predictive Inequity in Object Detection2
This work introduces ICON$^2, a framework for robustly answering the question of fairness in real-world vision systems, which leverages prior knowledge on the deficiencies of object detection systems to identify performance discrepancies across sub-populations, compute correlations between these potential confounders and a given sensitive attribute, and control for the most likely confounder to obtain a more reliable estimate of model bias.
- Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning2
It is shown that a combination of well-engineered reconstruction losses and inductive priors can help coordinate pose learning alongside reconstruction in a self-supervised paradigm.
- [Re] Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias2
It is found that both proposed methods in the original paper help mitigate contextual bias, although for some methods, one could not completely replicate the quantitative results in the paper even after completing an extensive hyperparameter search.
- 2
- 2
- Bias at the End of the Score1
A large scale audit of RM robustness with respect to demographic biases during T2I model training and generation provides quantitative and qualitative evidence that RMs encode demographic biases, which cause reward-guided optimization to disproportionately sexualize female image subjects reinforce gender/racial stereotypes, and collapse demographic diversity.
- GASS: Geometry-Aware Spherical Sampling for Disentangled Diversity Enhancement in Text-to-Image Generation1
This work introduces Geometry-Aware Spherical Sampling (GASS) to enhance diversity by explicitly controlling both prompt-dependent and prompt-independent sources of variation and decomposes the diversity measure in CLIP embeddings using two orthogonal directions.
- CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation1
Experiments show that CoMet consistently improves uncertainty estimation over existing baselines while remaining efficient in practice, and a lightweight post-hoc uncertainty module is trained to estimate these quantities, which enables efficient uncertainty estimation without autoregressive answer generation or repeated sampling.
- Personalized Generative Models for Contextual Debiasing1
Decoupling Contextual Patterns with Generations (DecoupleGen), a method that personalizes text-to-image diffusion models to facilitate coherent synthesis of images with rare contexts while preserving original visual details, is introduced.
- Reinforced Fast Weights with Next-Sequence Prediction1
This work introduces REFINE (Reinforced Fast weIghts with Next sEquence prediction), a reinforcement learning framework that trains fast weight models under the next-sequence prediction (NSP) objective and provides an effective and versatile framework for improving long-context modeling in fast weight architectures.
- D2D: Detector-to-Differentiable Critic for Improved Numeracy in Text-to-Image Generation1
Detector-to-Differentiable (D2D), a novel framework that transforms non-differentiable detection models into differentiable critics, thereby leveraging their superior counting ability to guide numeracy generation, is proposed.
- 1
- UFO: A unified method for controlling Understandability and Faithfulness Objectives in concept-based explanations for CNNs1
UFO formalizes understandability and faithfulness as mathematical objectives and unifies most existing concept-based explanations methods for CNNs, and systematically investigates how explanations change as they turn the knobs of faithfulness and understandability.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Learning actionness from action/background discrimination–
This study proposes to use auxiliary representation formed from baseline video to step label assignment scores to reinforce the discrimination of video clips and presents a method to learn an actionness score for each video clip to be used for post-processing baselineVideo clip to step labels assignment scores.
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.