Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Trevor J. DarrellSuggested from co-authorship
- Piotr IndykSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works125 from public data
- Deep Back-Projection Networks for Super-Resolution1,599
It is shown that extending the idea to allow concatenation of features across up- and downsampling stages (Dense DBPN) allows us to reconstruct further improve super-resolution, yielding superior results and in particular establishing new state of the art results for large scaling factors such as 8× across multiple data sets.
- FractalNet: Ultra-Deep Neural Networks without Residuals.1,078
In experiments, fractal networks match the excellent performance of standard residual networks on both CIFAR and ImageNet classification tasks, thereby demonstrating that residual representations may not be fundamental to the success of extremely deep convolutional neural networks.
- Learning Representations for Automatic Colorization1,070
A fully automatic image colorization system that leverages recent advances in deep networks, exploiting both low-level and semantic representations, and explores colorization as a vehicle for self-supervised visual representation learning.
- Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation709
This work proposes to apply chain rule on the learned gradients, and back-propagate the score of a diffusion model through the Jacobian of a differentiable renderer, which is instantiate to be a voxel radiance field.
- Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (Neural Information Processing)592
This volume presents theoretical and practical discussions of nearest-neighbor (NN) methods in machine learning and examines computer vision as an application domain in which the benefit of these advanced methods is often dramatic.
- Nearest-neighbor methods in learning and vision : theory and practice592
This volume presents theoretical and practical discussions of nearest-neighbor (NN) methods in machine learning and examines computer vision as an application domain in which the benefit of these advanced methods is often dramatic.
- Recurrent Back-Projection Network for Video Super-Resolution553
A novel architecture for the problem of video super-resolution that integrates spatial and temporal contexts from continuous video frames using a recurrent encoder-decoder module, that fuses multi-frame information with the more traditional, single frame super- resolution path for the target frame.
- Colorization as a Proxy Task for Visual Understanding524
This work investigates and improves self-supervision as a drop-in replacement for ImageNet pretraining, focusing on automatic colorization as the proxy task, and presents the first in-depth analysis of self- supervision via colorization, concluding that formulation of the loss, training details and network architecture play important roles in its effectiveness.
- Feedforward semantic segmentation with zoom-out features484
This work introduces a purely feed-forward architecture for semantic segmentation that exploits statistical structure in the image and in the label space without setting up explicit structured prediction mechanisms, and thus avoids complex and expensive inference.
- Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model456
This paper proposes Instant3D, a novel method that generates high-quality and diverse 3D assets from text prompts in a feed-forward manner and can generate diverse3D assets of high visual quality within 20 seconds, which is two orders of magnitude faster than previous optimization-based methods.
- 360
- 339
- Style Transfer by Relaxed Optimal Transport and Self-Similarity331
The results indicate that for any desired level of content preservation, the proposed Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS), a new optimization-based style transfer algorithm, provides higher quality stylization than prior work.
- Face Recognition from Long-Term Observations315
This work addresses the problem of face recognition from a large set of images obtained over time - a task arising in many surveillance and authentication applications and proposes an information-theoretic algorithm that classifies sets of images using the relative entropy between the estimated density of the input set and that of stored collections of images for each class.
- 282
- Decoding Complete Reach and Grasp Actions from Local Primary Motor Cortex Populations278
The results suggest that achieving high-dimensional reach and grasp actions with neuroprosthetic devices may be possible using small intracortical arrays like those already being tested in human pilot clinical trials.
- 257
- 254
- Nearest-Neighbor Searching and Metric Space Dimensions247
Several measures of dimension can be estimated using nearest-neighbor searching, while others can be used to estimate the cost of that searching.
- Task-Driven Super Resolution: Object Detection in Low-Resolution Images226
A novel framework to train a deep neural network where the SR sub-network explicitly incorporates a detection loss in its training objective, via a tradeoff with a traditional detection loss is proposed.
- Discriminability Objective for Training Descriptive Captions213
By incorporating into the captioning training objective a loss component directly related to ability (by a machine) to disambiguate image/caption matches, this work obtains systems that produce much more discriminative caption, according to human evaluation.
- Face Recognition in Subspaces209
This chapter describes in roughly chronologic order techniques that identify, parameterize, and analyze linear and nonlinear subspaces, from the original Eigenfaces technique to the recently introduced Bayesian method for probabilistic similarity analysis.
- Diverse M-Best Solutions in Markov Random Fields200
This paper proposes an algorithm for the Diverse M-Best problem, which involves finding a diverse set of highly probable solutions under a discrete probabilistic model and shows that for certain families of dissimilarity functions the authors can guarantee that these solutions can be found as easily as the MAP solution.
- Comprehension-Guided Referring Expressions187
A comprehension module trained on human-generated expressions serves as a differentiable proxy of human evaluation, providing training signal to the generation module, and it is shown that both approaches lead to improved referring expression generation on multiple benchmark datasets.
- Learning task-specific similarity159
An algorithmic approach to learning similarity from examples of what objects are deemed similar according to the task-specific notion of similarity at hand, as well as optional negative examples, which allows to predict when two previously unseen examples are similar and to efficiently search a very large database for examples similar to a query.
- 139
- Space-Time-Aware Multi-Resolution Video Enhancement126
The components of the model that generate latent low- and high-resolution representations during ST-SR can be used to finetune a specialized mechanism for just spatial or just temporal super-resolution.
- 124
- 110
- 109
- 106
- A Systematic Exploration of Diversity in Machine Translation99
It is found that diversity can improve performance on these tasks, especially for sentences that are difficult for MT.
- 97
- Deep Back-ProjectiNetworks for Single Image Super-Resolution92
This work proposes Deep Back-Projection Networks (DBPN), the winner of two image super-resolution challenges, that exploit iterative up- and down-sampling layers that represent different types of low- and high-resolution components.
- 89
- Fingerspelling Recognition in the Wild With Iterative Visual Attention83
This work proposes an end-to-end model based on an iterative attention mechanism, without explicit hand detection or segmentation, that out-performs prior work by a large margin on recognition of fingerspelling sequences in ASL videos collected in the wild.
- American Sign Language Fingerspelling Recognition in the Wild80
This work introduces the largest data set available so far for the problem of fingerspelling recognition, and the first using naturally occurring video data, and presents the first attempt to recognize fingerspelling sequences in this challenging setting.
- 78
- Conditional Random People: Tracking Humans with CRFs and Grid Filters76
It is shown how a state temporal prior in the grid-filter can be computed in a manner similar to a sparse HMM, resulting in real-time system performance and is used for human pose tracking in video sequences.
- 76
- 71
- Full Surround Monodepth From Multiple Cameras68
This work learns a single network generating dense, consistent, and scale-aware point clouds that cover the same full surround as a typical LiDAR scanner, and proposes a new scale-consistent evaluation metric more suitable to multi-camera settings.
- Visually Grounded Learning of Keyword Prediction from Untranscribed Speech65
This work uses an image-to-words multi-label visual classifier to tag images with soft textual labels, and then trains a neural network to map from the speech to these soft targets, and shows that the resulting speech system is able to predict which words occur in an utterance without seeing any parallel speech and text.
- Deformable Style Transfer63
De deformable style transfer (DST), an optimization-based approach that jointly stylizes the texture and geometry of a content image to better match a style image, is proposed.
- 61
- Lexicon-free fingerspelling recognition from video: Data, models, and signer adaptation60
This work collects and annotates a new data set of continuous fingerspelling videos, compares several types of recognizers, and explores the problem of signer variation.
- 60
- Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech56
This work studies how a visually grounded speech model, trained on images of scenes paired with spoken captions, captures aspects of semantics, and introduces a newly collected data set of human semantic relevance judgements and an associated task, semantic speech retrieval, where the goal is to search for spoken utterances that are semantically relevant to a given text query.
- Semantic keyword spotting by learning from images and speech.56
This work studies how a visually grounded speech model, trained on images of scenes paired with spoken captions, captures aspects of semantics, and introduces a newly collected data set of human semantic relevance judgements and an associated task, semantic speech retrieval, where the goal is to search for spoken utterances that are semantically relevant to a given text query.
- Generative Models: What Do They Know? Do They Know Things? Let's Find Out!54
The findings indicate that a small Low-Rank Adaptators (LoRA) can recover intrinsic images-depth, normals, albedo and shading-across different generators (Autoregressive, GANs and Diffusion) while using the same decoder head that generates the image.
- Discovery of phosphorylation motif mixtures in phosphoproteomics data51
An algorithm that simultaneously discovers multiple motifs in a set of peptides that were phosphorylated by several different kinases is described, which provides a new approach for uncovering the sequence specificities of uncharacterized kinases or phosphatases.
- 49
- 48
- Viewpoint-aware object detection and continuous pose estimation48
An approach to category-level detection and viewpoint estimation for rigid 3D objects from single 2D images is described, which relies on a nonparametric representation of a joint distribution of shape and appearance of the object class, allowing efficient accumulation of evidence.
- 45
- Pixel Consensus Voting for Panoptic Segmentation40
The discretization of centroid voting reduces the training of instance segmentation to pixel labeling, analogous and complementary to FCN-style semantic segmentation, leading to an efficient and unified architecture that jointly models things and stuff.
- 40
- Sparse Coding for Learning Interpretable Spatio-Temporal Primitives39
This paper formulated the problem as a tensor factorization problem with tensor group norm constraints over the primitives, diagonal constraints on the activations that provide interpretability as well as smoothness constraints that are inherent to human motion.
- 38
- Information-Theoretic Segmentation by Inpainting Error Maximization37
This work proposes a novel adversarial method that performs unsupervised segmentation by partitioning images into maximally independent sets, with the goal of minimizing predictability of one set from the other.
- Fingerspelling Detection in American Sign Language34
A new model is proposed that learns to detect fingerspelling via multi-task training, incorporating pose estimation and fingerspelling recognition along with detection, and this model outperforms all alternative approaches across all metrics, establishing a state of the art on the benchmark.
- Self-Supervised Camera Self-Calibration from Video33
This paper proposes a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models, and achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods.
- 33
- Neural Neighbor Style Transfer32
This work proposes Neural Neighbor Style Transfer (NNST), a pipeline that offers state-of-the-art quality, generalization, and competitive efficiency for artistic style transfer by explicitly replacing neural features extracted from the content input with those from a style exemplar.
- Regularizing Deep Networks by Modeling and Predicting Label Structure32
This work constructs custom regularization functions for use in supervised training of deep neural networks by building an abstract model of the label space, and then training the network to understand this abstraction alongside the original task.
- Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion31
It is shown that self-supervision can be used to learn accurate depth and ego-motion estimation without prior knowledge of the camera model, and Neural Ray Surfaces (NRS) are introduced, convolutional networks that represent pixel-wise projection rays, approximating a wide range of cameras.
- 31
- Locally Weighted Regression29
Two " global " approaches to regression: Parametric: assume parametric form y = f (x; θ), fit the parameters to the training set θ * = argmin θ n i=1 L(y i , f (x i ; θ) and then estimatê y 0 = f (x 0 ; θ *) • Pros: Once trained, cheap to apply on any new data points.
- 29
- Adapting CLIP For Phrase Localization Without Further Training28
This work adapts CLIP to generate high-resolution spatial feature maps and can extract feature maps from both ViT and ResNet CLIP model while maintaining the semantic properties of an image embedding, which provides a natural framework for phrase localization.
- 28
- An investigation of computational and informational limits in Gaussian mixture clustering28
It is shown that using principal component projection greatly aids in recovering the clustering using EM, and empirical evidence is presented that even using such a projection, there is still a large gap between the number of samples needed to recover the clusters using EM.
- 27
- A consistent estimator of the expected gradient outerproduct26
It is shown that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks and it is proved that, despite its simplicity, this rough estimators remains statistically consistent under mild conditions.
- Boosted Dyadic Kernel Discriminants25
A novel learning algorithm for binary classification with hyperplane discriminants based on pairs of training points from opposite classes (dyadic hypercuts) is introduced and this algorithm is further extended to nonlinear discriminant using kernel functions satisfying Mercer's conditions.
- EditDuet: A Multi-Agent System for Video Non-Linear Editing21
This work proposes to automate the core task of video editing, formulating it as sequential decision making process with a multi-agent approach, and introduces a learning-based approach for enabling effective communication across specialized agents to address the language-driven video editing task.
- 20
- 18
- SPLART: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting16
SplArt is introduced, a self-supervised, category-agnostic framework that uses 3D Gaussian Splatting (3DGS) to reconstruct and infer the kinematics of articulated objects from two sets of posed RGB images captured at different articulation states, enabling real-time photorealistic rendering for novel viewpoints and articulations.
- HyperFields: Towards Zero-Shot Generation of NeRFs from Text15
This work introduces HyperFields, a method for generating text-conditioned Neural Radiance Fields with a single forward pass and (optionally) some fine-tuning that benefits from accelerated convergence thanks to the learned general map, and is capable of synthesizing novel scenes 5 to 10 times faster than existing neural optimization-based methods.
- Knowing a Good HOG Filter When You See It: Efficient Selection of Filters for Detection15
It is shown that one can learn a universal model of part “goodness” based on properties that can be computed from the filter itself, which will improve its detection performance on the PASCAL VOC data sets, while speeding up training by an order of magnitude.
- Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion14
MVGD is introduced, a diffusion-based architecture capable of direct pixel-level generation of images and depth maps from novel viewpoints, given an arbitrary number of input views, and a novel strategy that enables the efficient training of larger models by incrementally fine-tuning smaller ones, with promising scaling behavior.
- 14
- 14
- Part Annotations via Pairwise Correspondence.13
The use of an interface to mark pairs of points on two images which are in “correspondence” with one another, as a way of collecting part annotations is explored.
- Transcrib3D: 3D Referring Expression Resolution through Large Language Models12
This work introduces Transcrib3D, an approach that brings together 3D detection methods and the emergent reasoning capabilities of large language models (LLMs), and shows that the method enables a real robot to perform pick-and-place tasks given queries that contain challenging referring expressions.
- Self-Supervised Video Transformers for Isolated Sign Language Recognition12
The results on WLASL2000 highlight the power of masked reconstruction pre-training, and the linear probing results demonstrate the importance of hierarchical vision transformers for sign language representation.
- 12
- Discriminative Metric Learning by Neighborhood Gerrymandering12
This work describes an efficient algorithm for exact loss augmented inference, and a fast gradient descent algorithm for learning in this model of metric learning for k nearest neighbor classification, reminiscent of gerrymandering.
- Smoothed Bootstrap and Statistical Data Cloning for Classifier Evaluation11
A number of novel bootstrap estimators, based on kernel smoothing, are presented that consistently show superior performance on both synthetic and real data, with respect to other established methods, enabling better classifier evaluation over limited real data repositories such as the UCI repository.
- 10
- NeRFuser: Large-Scale Scene Representation by NeRF Fusion9
A novel architecture for NeRF registration and blending that assumes only access to pre-generated NeRFs, and not the potentially large sets of images used to generate them, is proposed, and a sample-based inverse distance weighting to blend visual information at the ray-sample level is proposed.
- Self-Supervised Depth Learning for Urban Scene Understanding9
This work trains a deep network, using fully automatic supervision, to predict relative scene depth from single images, and shows that this pretext task of predicting depth from a single image induces features in the network that result in large improvements in a set of downstream tasks.
- Part and Attribute Discovery from Relative Annotations9
An annotation framework based on comparisons between pairs of instances within a set, which aims to reduce the overhead in manually specifying the set of part and attribute labels, which is applicable to a wide range of categories.
- Virtual Visual Hulls: Example-Based 3D Shape Inference from Silhouettes9
A method for estimating the 3D visual hull of an object from a known class given a single silhouette or sequence of silhouettes observed from an unknown viewpoint and results are shown on real and synthetic images of people.
- 8
- 8
- 8
- FastMap: Revisiting Structure from Motion Through First-Order Optimization7
Through extensive experiments, this work shows that FastMap is up to 10 × faster than COLMAP and GLOMAP with GPU acceleration and achieves comparable pose accuracy, and identifies and eliminates two key performance bottlenecks: computational complexity and the kernel implementation of each optimization step.
- 7
- 6
- Nonlinear physically-based models for decoding motor-cortical population activity6
It is found that the decoded spring coefficients produced accurate hand trajectories compared with state-of-the-art methods for direct decoding of hand kinematics and that their frequency spectrum more closely matched that of natural hand movements.
- 6-DoF Stability Field via Diffusion Models5
6-DoFusion is presented, a generative model capable of generating 3D poses of an object that produces a stable configuration of a given scene that is able to construct stable scenes that involve novel object classes as well as to improve the accuracy of state-of-the-art 3D pose estimation methods.
- Training Deep Networks to be Spatially Sensitive5
By incorporating spatial information into the objective, this work can use a simpler model than competing methods without sacrificing accuracy, resulting in faster inference speeds and alleviating the need for pre/post-processing.
- 4
- 4
- Approximate Nearest Neighbor Regression in Very High Dimensions4
This chapter contains sections titled: Introduction, Evidence for Low-Dimensional Distributions, Local Dimensionality Reduction, Locally Weighted Projection Regression, Empirical Evaluation, Bayesian Backfitting, Discussion.
- Alpha Invariance: On Inverse Scaling Between Distance and Volume Density in Neural Radiance Fields3
This work revisits a few popular radiance field models and finds that these systems use various heuristics to deal with issues arising from scene scaling, and recommends a discretization-agnostic initialization strategy to guarantee high ray transmittance.
- Boosting Barely Robust Learners: A New Perspective on Adversarial Robustness3
This work presents an oracle-efficient algorithm for boosting the adversarial robustness of barely robust learners and reveals a qualitative and quantitative equivalence between two seemingly unrelated problems: strongly robust learning and barely robust learning.
- LoopDraw: a Loop-Based Autoregressive Model for Shape Synthesis and Editing3
This work presents a new, compelling alternative for representing shapes using a sequence of cross-sectional closed loops, and demonstrates that loops are intuitive and natural primitive for analyzing and editing shapes, both computationally and for users.
- OmniShape: Zero-Shot Multi-Hypothesis Shape and Pose Estimation in the Real World2
This work proposes OmniShape, the first method of its kind to enable probabilistic pose and shape estimation, based on the key insight that shape completion can be decoupled into two multi-modal distributions.
- Text-Free Learning of a Natural Language Interface for Pretrained Face Generators2
Fast text2StyleGAN is a natural language interface that adapts pre-trained GANs for text-guided human face synthesis and offers faster and more accurate generation of images from natural language descriptions with varying levels of detail compared to prior work.
- 2
- 2
- Notes on the Shannon Entropy of the Neural Response1
In these notes, the concept of Shannon entropy is focused on in an attempt to provide a systematic way of assessing the discrimination properties of the neural response, and quantifying the role played by the number of layers and theNumber of templates.
- A Slicing-Based Coherence Measure for Clusters of DTI Integral Curves1
A slicing-based coherence measure based on the spatial relations between the projections of the curve points in individual slices and their change across the slices is presented and used in refining agglomerative hierarchical clustering results of DTI curves that correspond to neural pathways.
- 1
- Virtual Visual Hulls: Example-Based 3D Shape Estimation from a Single Silhouette1
This work presents a method for generating a “virtual visual hull”– an estimate of the 3D shape of an object from a known class, given a single silhouette observed from an unknown viewpoint.
- –
- Autoregressive Modeling of Film with Applications in Video Montage–
This work introduces FilmGPT, an autoregressive transformer designed to address the challenge of video montage – turning a collection of raw, “unwatchable” footage into coherent cinematic sequences, and introduces a footage-constrained decoding algorithm to select the best next shot from the input raw footage according to the statistical patterns learned from films.
- Targeted Linguistic Analysis of Sign Language Models with Minimal Translation Pairs–
A targeted analysis of the state-of-the-art ASL-to-English translation model is conducted, showing that, while the model performs above chance level on most of the phenomena, it relies strongly on manual cues while often missing crucial non-manual cues.
- Generative AI Meets Computer Graphics–
This workshop will bring together researchers and practitioners working on text-/image-conditioned 3D generation, learning-based and hybrid simulation methods, and generative tools for geometric editing, retopology, and asset refinement, as well as generative approaches for design and physically plausible motion and dynamics generation.
- –
- –
- Performance of Approximate Nearest Neighbor Classification–
Convergence of RN to R∞ can be arbitrarily slow [2, 4]; no distribution-independent results for RN are known.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.