Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks457 from public data
- 3,070
- Ego4D: Around the World in 3,000 Hours of Egocentric Video2,124
Ego4D, a massive-scale egocentric video dataset and benchmark suite, is introduced and a host of new benchmark challenges centered around understanding the first-person visual experience in the past, present, and future are presented.
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society1,998
This paper proposes a novel communicative agent framework named role-playing, using inception prompting to guide chat agents toward task completion while maintaining consistency with human intentions, and conducts comprehensive studies on instruction-following cooperation in multi-agent settings.
- A Benchmark and Simulator for UAV Tracking1,912
A new aerial video dataset and benchmark for low altitude UAV target tracking, as well as, a photo-realistic UAV simulator that can be coupled with tracking methods to easily extend existing real-world datasets.
- DeepGCNs: Can GCNs Go As Deep As CNNs?1,662
This work presents new ways to successfully train very deep GCNs by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures, and building a very deep 56-layer GCN.
- Can GCNs Go as Deep as CNNs1,662
This work presents new ways to successfully train very deep GCNs by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures, and building a very deep 56-layer GCN.
- 1,403
- PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies1,280
This work revisits the classical PointNet++ through a systematic study of model training and scaling strategies, and proposes PointNeXt, the next version of PointNets, which can be flexibly scaled up and outperforms state-of-the-art methods on both 3D classification and segmentation tasks.
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild1,171
This work presents TrackingNet, the first large-scale dataset and benchmark for object tracking in the wild, which covers a wide selection of object classes in broad and diverse context and provides an extensive benchmark on TrackingNet by evaluating more than 20 trackers.
- The Visual Object Tracking VOT2016 Challenge Results776
The Visual Object Tracking challenge VOT2016 goes beyond its predecessors by introducing a new semi-automatic ground truth bounding box annotation methodology and extending the evaluation system with the no-reset experiment.
- 725
- 665
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives640
To push the frontier of first-person video understanding of skilled human activity, a suite of benchmark tasks and their annotations are presented, including fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand/body pose.
- G-TAD: Sub-Graph Localization for Temporal Action Detection516
This work proposes a graph convolutional network (GCN) model to adaptively incorporate multi-level semantic context into video features and cast temporal action detection as a sub-graph localization problem.
- Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors493
A two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both 2D and 3D priors, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.
- Self-Supervised Learning by Cross-Modal Audio-Video Clustering484
Cross-Modal Deep Clustering (XDC), a novel self-supervised method that leverages unsupervised clustering in one modality as a supervisory signal for the other modality, is proposed, which is the first self- supervised learning method that outperforms large-scale fully- Supervised pretraining for action recognition on the same architecture.
- 478
- SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network473
Extensive experiments on the challenging COCO dataset demonstrate the effectiveness of the proposed MTGAN method in restoring a clear super-resolved image from a blurred small one, and show that the detection performance, especially for small sized objects, improves over state-of-the-art methods.
- DAPs: Deep Action Proposals for Action Understanding442
Deep Action Proposals (DAPs), an effective and efficient algorithm for generating temporal action proposals from long videos, is introduced, which outperforms previous work on a large scale action benchmark, runs at 134 FPS making it practical for large-scale scenarios, and exhibits an appealing ability to generalize.
- Robust Visual Tracking via Structured Multi-Task Sparse Learning433
The results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity.
- FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model381
This work uses off-the-shelf pretrained networks, such as a face detection model, to construct time-independent energy functions, which guide the generation process without requiring training, and has a broader range of applications than existing training-free methods.
- MVTN: Multi-View Transformation Network for 3D Shape Recognition313
The Multi-View Transformation Network (MVTN) is introduced that regresses optimal view-points for 3D shape recognition, building upon advances in differentiable rendering and can provide network robustness against rotation and occlusion in the 3D domain.
- SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos311
This paper introduces SoccerNet, a benchmark for action spotting in soccer videos, and shows that the best model for classifying temporal segments of length one minute reaches a mean Average Precision (mAP) of 67.8%.
- Egocentric Video-Language Pretraining306
The recently released Ego4D dataset is exploited to pioneer Egocentric VLP along three directions, and a novel pretraining objective is proposed, dubbed EgoNCE, which adapts video-text contrastive learning to the egocentric domain by mining egOCentric-aware positive and negative samples.
- 305
- Towards Open Vocabulary Learning: A Survey285
This paper thoroughly reviews open vocabulary learning, summarizing and analyzing recent developments in the field, and juxtaposing open vocabulary learning with analogous concepts such as zero-shot learning, open-set recognition, and out-of-distribution detection.
- PU-GCN: Point Cloud Upsampling using Graph Convolutional Networks278
A novel model called NodeShuffle is proposed, which uses a Graph Convolutional Network (GCN) to better encode local point information from point neighborhoods to improve state-of-the-art upsampling methods.
- Robust Visual Tracking Via Consistent Low-Rank Sparse Learning261
A consistent low-rank sparse tracker that builds upon the particle filter framework for tracking that adaptively prunes and selects candidate particles by using linear sparse combinations of dictionary templates is proposed.
- SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos258
This work proposes SoccerNet-v2, a novel large-scale corpus of manual annotations for the SoccerNet video dataset, along with open challenges to encourage more research in soccer understanding and broadcast production, and extends current tasks in the realm of soccer to include action spotting, camera shot segmentation with boundary detection, and a novel replay grounding task.
- Low-Rank Sparse Learning for Robust Visual Tracking254
By representing particles jointly, LRST not only outperforms the state-of-the-art in tracking accuracy but also significantly improves the time complexity of methods that use a similar sparse linear representation model for particles.
- End-to-End, Single-Stream Temporal Action Detection in Untrimmed Videos253
This work introduces the new architecture for Single-Stream Temporal Action Detection (SS-TAD), which effectively integrates joint action detection with its semantic sub-tasks in a single unifying end-to-end framework.
- 252
- Face Super-Resolution Guided by Facial Component Heatmaps248
This paper proposes a method that explicitly incorporates structural information of faces into the face super-resolution process by using a multi-task convolutional neural network (CNN) and achieves superior face hallucination results and outperforms the state-of-the-art.
- 228
- DeepGCNs: Making GCNs Go as Deep as CNNs223
This work transfers concepts such as residual/dense connections and dilated convolutions from CNNs to GCNs in order to successfully train very deep GCNs, achieving very promising performance in part segmentation and semantic segmentation on point clouds and in node classification of protein functions across biological protein-protein interaction (PPI) graphs.
- 212
- Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications207
A photo-realistic training and evaluation simulator with extensive applications across various fields of computer vision built on top of the Unreal Engine, which provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.
- 3D Instance Segmentation via Multi-Task Metric Learning193
This work proposes a novel method for instance label segmentation of dense 3D voxel grids that achieves state-of-the-art performance on the ScanNet 3D instance segmentation benchmark.
- 193
- Leveraging Shape Completion for 3D Siamese Tracking191
A Siamese tracker is designed that encodes model and candidate shapes into a compact latent representation, regularize the encoding by enforcing the latent representation to decode into an object model shape, observing that 3D object tracking and 3D shape completion complement each other.
- Video Self-Stitching Graph Network for Temporal Action Localization188
This paper proposes a multi-level cross-scale solution dubbed as video self-stitching graph network (VSGN), which improves the localization performance of short actions as well as achieving the state-of-the-art overall performance on THUMOS-14 and ActivityNet-v1.3.
- 184
- 184
- AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds179
A new point cloud attack is developed (dubbed AdvPC) that exploits the input data distribution by adding an adversarial loss, after Auto-Encoder reconstruction, to the objective it optimizes, and leads to perturbations that are resilient against current defenses, while remaining highly transferable compared to state-of-the-art attacks.
- 178
- A Unified Continual Learning Framework with General Parameter-Efficient Tuning173
This paper positions prompting as one instantiation of PET, and proposes a unified CL framework with general PET, dubbed as Learning-Accumulation-Ensemble (LAE), which is compatible with a battery of PET methods and gains strong CL capability.
- Target Response Adaptation for Correlation Filter Tracking164
This work proposes a generic framework that can adaptively change the target response from frame to frame, so that the tracker is less sensitive to the cases where circular shifts do not reliably approximate translations.
- Maximum Margin Distance Learning for Dynamic Texture Recognition162
This paper proposes an efficient maximum margin distance learning (MMDL) method, called DL-PEGASOS, which outperforms state-of-the-art recognition methods on the UCLA benchmark DT dataset and shows that, for certain classes of DTs, spatial texture features are dominantly "salient", while for other classes, this "saliency" lies in their temporal features.
- OASIS: Open Agent Social Interaction Simulations with One Million Agents161
OASIS is a generalizable and scalable social media simulator that replicates various social phenomena, including information spreading, group polarization, and herd effects across X and Reddit platforms, and demonstrates OASIS's potential as a powerful tool for studying complex systems in digital environments.
- Spatio-temporal Relation Modeling for Few-shot Action Recognition158
A novel few-shot action recognition framework, STRM, which enhances class-specific feature discrim-inability while simultaneously learning higher-order temporal representations and introduces a query-class similarity classifier on the patch-level enriched features to enhance class- specific feature dis-criminability.
- MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions156
MAD (Movie Audio Descriptions), a novel benchmark that departs from the paradigm of augmenting existing video datasets with text annotations and focuses on crawling and aligning available audio descriptions of mainstream movies, exhibits a significant reduction in the currently diagnosed biases for video-language grounding datasets.
- SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos154
This work proposes a novel dataset for multiple object tracking composed of 200 sequences of 30s each, representative of challenging soccer scenarios, and a complete 45-minutes half-time for long-term tracking.
- TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks153
Extensive experiments show that using features trained with the novel pretraining strategy significantly improves the performance of recent state-of-the-art methods on three tasks: Temporal Action Localization, Action Proposal Generation, and Dense Video Captioning.
- 144
- 138
- GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering133
GES (Generalized Exponential Splatting), a novel representation that employs Generalized Exponential Function (GEF) to model 3D scenes, requiring far fewer parti-cles to represent a scene and thus significantly outperforming Gaussian Splatting methods in efficiency with a plug-and-play replacement ability for Gaussian-based utilities.
- FLAG: Adversarial Data Augmentation for Graph Neural Networks133
This work proposes a simple but effective solution, FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training, and boosts performance at test time.
- 133
- Diagnosing Error in Temporal Action Detectors130
A new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric is introduced, finding the lack of agreement among annotator is not a major roadblock to attain progress in the field.
- p-Box ADMM: A Versatile Framework for Integer Programming125
This paper revisits the integer programming (IP) problem, and proposes a novel and versatile framework called Alternating Direction Method of Multipliers (ADMM), which significantly outperforms existing generic IP solvers both in runtime and objective.
- 125
- Mindstorms in Natural Language-Based Societies of Mind124
This work assemble and experiment with several of NLSOMs, leveraging mindstorms in them to solve some practical AI tasks: visual question answering, image captioning, text-to-image synthesis, 3D generation, egocentric retrieval, embodied AI, and general language-based task solving.
- Robust Optimization as Data Augmentation for Large-scale Graphs118
This work proposes FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training, which universally works in node classification, link prediction, and graph classification tasks.
- Temporal Localization of Moments in Video Collections with Natural Language.117
The CAL model outperforms the recently proposed Moment Context Network on all criteria across all datasets on the proposed task, obtaining an 8%-85% and 11%-47% boost for average recall and median rank, respectively, and achieves 5x faster retrieval and 8x smaller index size with a 500K video corpus.
- A Context-Aware Loss Function for Action Spotting in Soccer Videos113
This paper proposes a novel loss function that specifically considers the temporal context naturally present around each action, rather than focusing on the single annotated frame to spot, and demonstrates the generalization capability of this loss for generic activity proposals and detection on ActivityNet.
- 111
- 108
- 105
- Scaling up SoccerNet with multi-view spatial localization and re-identification102
This work releases SoccerNet-v3, a major extension of the SoccerNet dataset, providing a wide variety of spatial annotations and cross-view correspondences, making SoccerNet-v3 the largest dataset for multi-view soccer analysis.
- BOLT: Boost Large Vision-Language Model Without Training for Long-Form Video Understanding96
BOLT is introduced, a method to BOost Large VLMs without additional Training through a comprehensive study of frame selection strategies based on query-frame similarity, and shows that inverse transform sampling yields the most significant performance improvement.
- Computationally Budgeted Continual Learning: What Does Matter?95
It is found that, under compute-constrained setting, traditional CL approaches, with no exception, fail to outperform a simple minimal baseline that samples uniformly from memory, suggesting that most existing CL methods are particularly too computationally expensive for realistic budgeted deployment.
- Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization95
This work proposes Action Search, a novel Recurrent Neural Network approach that mimics the way humans spot actions in video, and puts forward the Human Searches dataset, which compiles the search sequences employed by human annotators spotting actions in the AVA and THUMOS14 datasets.
- Multi-label Learning with Missing Labels Using Mixed Dependency Graphs95
This work proposes two convex transductive formulations of the MLML problem, denoted as MLMG-CO and ML MG-SL respectively, and proposes two important applications, including image annotation and tag based image retrieval, which can be jointly handled using these proposed methods.
- VLG-Net: Video-Language Graph Matching Network for Video Grounding94
A novel Video-Language Graph Matching Network (VLG-Net) is designed to enable the mutual exchange of information across the modalities, and superior performance over state-of-the-art grounding methods on three widely used datasets for temporal localization of moments in videos with language queries.
- 94
- 93
- SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries89
A novel task of dense video captioning focusing on the generation of textual commentaries anchored with single times-tamps that has the potential to enhance the accessibility and understanding of soccer content for a wider audience and bring the excitement of the game to more people.
- Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting85
This work distill a powerful commercial calibration tool in a recent neural network architecture on the largescale SoccerNet dataset, composed of untrimmed broadcast videos of 500 soccer games, and leverages it to provide 3 ways of representing the calibration results along with player localization.
- Active Speakers in Context85
The Active Speaker Context is introduced, a novel representation that models relationships between multiple speakers over long time horizons that improves the state-of-the-art on the AVA-ActiveSpeaker dataset achieving an mAP of 87.1%.
- End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames81
This paper reduces the memory consumption for end-to-end training, and manages to scale up the TAD backbone to 1 billion parameters and the input video to 1,536 frames, leading to significant detection performance.
- Characterization of a thermostable Cas13 enzyme for one-pot detection of SARS-CoV-281
A thermostable Cas13 orthologue from Thermoclostridium caenicola (TccCas13a) is identified and characterized, which exhibits robust cis and trans catalytic activities over a broad range of temperatures from 37 to 70 °C, and was clinically validated and is suitable for multiplexed detection.
- Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training79
The resulting method, called SGDiff, allows for the semantic manipulation of generated images by modifying scene graph nodes and connections, and outperforms state-of-the-art methods, as measured by both the Inception Score and Fr\'echet Inception Distance metrics.
- Real-Time Evaluation in Online Continual Learning: A New Hope77
It is shown that a simple baseline outperforms state-of-the-art CL methods under this evaluation, questioning the applicability of existing methods in realistic settings and suggesting that the majority of existing CL literature is tailored to a specific class of streams that is not practical.
- 76
- 76
- SPAD: Spatially Aware Multi-View Diffusers75
SPAD offers full camera control and achieves state-of-the-art results in novel view synthesis on unseen objects from the Objaverse and Google Scanned Objects datasets and demon-strate that text-to-3D generation using SPAD prevents the multi-face Janus issue.
- AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation75
A novel GAN-based anime-face translator, called AniGAN, is proposed to synthesize high-quality anime-faces which are style-consistent with a given reference anime- Face, while preserving the global structure of the source photo-face.
- 75
- Fish identification from videos captured in uncontrolled underwater environments74
A new paradigm of image set classification is presented that can be used to achieve improved recognition rates for a number of fish species and it is demonstrated that these algorithms have the potential of solving the automatic fish species identification problem in underwater videos captured within unconstrained environments.
- SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?73
This work provides an analysis on CLIP models trained on synthetic data, providing insights on the data generation strategy, number of samples required, scaling trends, and resulting properties.
- Large-Capacity and Flexible Video Steganography via Invertible Neural Network73
A Large-capacity and Flexible Video Steganography Network (LF-VSN) that has high security, large hiding capacity, and flexibility, and a scalable strategy in multiple videos hiding, which can hide variable numbers of secret videos in a cover video with a single model and a single training session.
- MAAS: Multi-modal Assignation for Active Speaker Detection72
This work presents a novel approach to active speaker detection that directly addresses the multi-modal nature of the problem and provides a straightforward strategy, where independent visual features in the scene are assigned to a previously detected speech event.
- End-to-end Learned, Optically Coded Super-resolution SPAD Camera69
This work investigates a simple, low-cost, and compact optical coding camera design that supports high-resolution image reconstructions from raw measurements with low pixel counts, and uses an end-to-end framework to simultaneously optimize the optical design and a reconstruction network for obtaining super-resolved images from raw measures.
- SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap67
This work formalizes the task of Game State Reconstruction and introduces SoccerNet-GSR, a novel Game State Reconstruction dataset focusing on football videos, and introduces GS-HOTA, a novel metric to evaluate game state reconstruction methods.
- 66
- Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts66
A novel feature pooling method based on NetVLAD, dubbed netVLAD++, that embeds temporally-aware knowledge that split the context before and after an action occurs, and argues that considering the contextual information around the action spot as a single entity leads to a sub-optimal learning for the pooling module.
- Boundary-sensitive Pre-training for Temporal Localization in Videos66
This paper proposes to synthesize temporal boundaries in existing video action classification datasets and introduces a novel boundary-sensitive pretext (BSP) task, which is superior and complementary to the existing action classification-based pre-training counterpart and achieves new state-of-the-art performance on several temporal localization tasks.
- 3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes65
A novel method, named 3D Convex Splatting (3DCS), which leverages 3D smooth convexes as primitives for modeling geometrically-meaningful radiance fields from multi-view images, highlighting the potential of 3D Convex Splatting to become the new standard for high-quality scene reconstruction and novel view synthesis.
- Relation-aware Video Reading Comprehension for Temporal Language Grounding65
This paper will formulate temporal language grounding into video reading comprehension and propose a Relation-aware Network (RaNet) to address it, which aims to select a video moment choice from the predefined answer set with the aid of coarse-and-fine choice-query interaction and choice-choice relation construction.
- PIVOT: Prompting for Video Continual Learning64
PIVOT is a novel method that leverages extensive knowledge in pre-trained models from the image domain, thereby reducing the number of trainable parameters and the associated forgetting in continual learning without any in-domain pre-training.
- VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views63
To validate VARS, SoccerNet-MVFoul is introduced, a novel video dataset of soccer fouls from multiple camera views, annotated with extensive foul descriptions by a professional soccer referee, and VARS is benchmarked to automatically recognize the characteristics of these fouls.
- Representation learning with deep extreme learning machines for efficient image set classification62
This paper proposes an efficient image set representation that does not make any prior assumptions about the structure of the underlying data and learns the nonlinear structure of image sets with deep extreme learning machines that are very efficient and generalize well even on a limited number of training samples.
- Constrained Submodular Minimization for Missing Labels and Class Imbalance in Multi-label Learning62
This work forms the problem as a constrained submodular minimization that is composed of a sub modular objective function that encourages label consistency and smoothness, as well as, class cardinality bound constraints to handle class imbalance.
- Missing Labels in Object Detection60
This paper study the effect of missing annotations on FSOD methods and analyze approaches to train an object detector from a hybrid dataset, where both instancelevel and image-level labels are employed, to demonstrate the effectiveness of the method.
- Triangle Splatting for Real-Time Radiance Field Rendering59
A differentiable renderer that directly optimizes triangles via end-to-end gradients is developed, rendering each triangle as differentiable splats, combining the efficiency of triangles with the adaptive density of representations based on independent primitives.
- RefineLoc: Iterative Refinement for Weakly-Supervised Action Localization59
This paper proposes RefineLoc, a novel weaklysupervised temporal action localization method that uses an iterative refinement approach by estimating and training on snippet-level pseudo ground truth at every iteration, and shows competitive results with the state-of-the-art in weakly- Supervised temporal localization.
- 59
- 57
- Tagging Like Humans: Diverse and Distinct Image Annotation55
This work generates a relevant and distinct tag subset, in which the tags are relevant to the image contents and semantically distinct to each other, using sequential sampling from a determinantal point process (DPP) model.
- An Exact Penalty Method for Binary Optimization Based on MPEC Formulation54
This work reformulates the binary program as an equivalent augmented biconvex optimization problem with a bilinear equality constraint, then proposes an exact penalty method to solve it, and proves that the penalty function, induced by adding the complementarity constraint to the objective, is exact.
- LLM as A Robotic Brain: Unifying Egocentric Memory and Control53
A novel and generalizable framework called LLM-Brain: using Large-scale Language Model as a robotic brain to unify egocentric memory and control, utilizing a zero-shot learning approach.
- 53
- 53
- 52
- 52
- 51
- SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation50
This work proposes a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer, and shows that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
- Multi-task Generative Adversarial Network for Detecting Small Objects in the Wild50
The proposed end-to-end multi-task generative adversarial network (MTGAN), which is a general framework, is proposed, which can up-sample small blurred images into fine-scale ones and recover detailed information for more accurate detection.
- SeedQuant: a deep learning-based tool for assessing stimulant and inhibitor activity on root parasitic seeds48
Software that efficiently and automatically assesses the germination rate of parasitic seeds in in vitro bioassays enables high-throughput screening for germination stimulants and inhibitors.
- 48
- Starfish-inspired wearable bioelectronic systems for physiological signal monitoring during motion and real-time heart disease diagnosis47
Inspired by starfish’s pentaradial symmetry, a starfish-like wearable bioelectronic system designed for high-fidelity signal monitoring during movement is introduced, leveraging bioinspired device concepts combined with cutting-edge data science to boost bioelectronic performance and diagnostic precision.
- 46
- OIL: Observational Imitation Learning46
Observational Imitation Learning is proposed, a novel imitation learning variant that supports online training and automatic selection of optimal behavior by observing multiple imperfect teachers and outperforms conventional imitation learning (IL) and reinforcement learning (RL) baselines and even humans in simulation.
- What Do I Annotate Next? An Empirical Study of Active Learning for Action Localization46
This paper introduces a novel active learning framework for temporal localization that aims to mitigate data dependency issue, and equips this framework with active selection functions that can reuse knowledge from previously annotated datasets.
- Enhancing Adversarial Robustness via Test-time Transformation Ensembling45
It is shown that TTE consistently improves model robustness against a variety of powerful attacks without any need for re-training, and that this improvement comes at virtually no trade-off with accuracy on clean samples.
- Combating Adversaries with Anti-adversaries43
The anti-adversary layer is proposed, which generates an input perturbation in the opposite direction of the adversarial one and feeds the classifier a perturbed version of the input.
- Quantitative and Real‐Time Evaluation of Human Respiration Signals with a Shape‐Conformal Wireless Sensing System43
A soft, wireless, and non‐invasive device for quantitative and real‐time evaluation of human respiration, which simultaneously captures respiration and temperature signatures using customized capacitive and resistive sensors, encapsulated by a breathable layer, and does not limit the user's daily life.
- 43
- SoccerNet 2022 Challenges Results42
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team, and tasks (3-6) were novel, including their underlying data and annotations.
- 41
- 41
- 41
- 40
- 40
- 40
- Facial action unit recognition under incomplete data based on multi-label learning with missing labels40
Facial action unit recognition under incomplete data as a multi-label learning with missing labels (MLML) problem is formulated and the superiority of the proposed method in comparison with several state-of-the-art works is demonstrated.
- 40
- 40
- 39
- Weakly-supervised object detection via mining pseudo ground truth bounding-boxes39
This work proposes a weakly-supervised to fully- Supervised framework (W2F), where aWeakly- supervised detector is implemented using multiple instance learning, and a pseudo ground-truth excavation (PGE) algorithm to find the accurate pseudo ground truth bounding-box for each instance.
- Template Assembly for Detailed Urban Reconstruction39
A new framework to reconstruct building details by automatically assembling 3D templates on coarse textured building models that can enrich the details of coarse models using various data sets is proposed.
- On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective38
The main finding is that the decision boundaries of a simple network of the form (Affine, ReLU, Affine) are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes.
- Robust video registration applied to field-sports video analysis38
Here, robust video registration (RVR) becomes equivalent to solving a sequence of � 1 minimization problems, each of which can be solved using the Inexact Augmented Lagrangian Method (IALM).
- X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Models37
This paper introduces the EXplainable Video Assistant Referee System, X-VARS, a multi-modal large language model designed for understanding football videos from the point of view of a referee, and highlights the potential of X-VARS to reach human performance and support football referees in the future.
- ISTA-Net: Iterative Shrinkage-Thresholding Algorithm Inspired Deep Network for Image Compressive Sensing37
This paper proposes a novel structured deep network, dubbed ISTA-Net, which is inspired by the Iterative Shrinkage-Thresholding Algorithm (ISTA) for optimizing a general $l_1$ norm CS reconstruction model, and proposes an accelerated version of ISTA -Net, dubbed FISTA-Net , which isinspired by the fast iterative shrinkage-thresholding algorithm (FISTA).
- MovieCuts: A New Dataset and Benchmark for Cut Type Recognition36
The Cut type recognition task, which requires modeling multi-modal information, is introduced and a large-scale dataset called MovieCuts is constructed, which contains 173,967 video clips labeled with ten cut types defined by professionals in the movie industry.
- A Novel Framework for Robustness Analysis of Visual QA Models36
This work proposes a new framework that uses semantically relevant questions, dubbed basic questions, acting as noise to evaluate the robustness of VQA models, and analyzes the robusts of several state-of-the-art VqA models to show that attention-based V QA models are more robust than other methods in general.
- Localizing Moments in Long Video Via Multimodal Guidance35
A guided grounding framework consisting of a Guidance Model and a base grounding model that outperforms state-of-the-art models by 4.1% in MAD and 4.52% in Ego4D (NLQ), respectively.
- Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline35
This work comprehensively study the effects of pipelines on the mixture problem of learning-based DN, DM, and SR, in both sequential and joint solutions, and proposes an end-to-end Trinity Pixel Enhancement NETwork (TENet) that achieves the state-of-the-art performance for the mixture Problem.
- Guess Where? Actor-Supervision for Spatiotemporal Video Action Localization35
Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-super supervised alternatives.
- Guess where? Actor-supervision for spatiotemporal action localization35
Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-super supervised alternatives.
- 35
- SoccerNet 2023 challenges results34
This report indicates performance trends across tasks: Action spotting is nearing saturation, while ball action spotting improved significantly with advanced end-to-end models and dense video captioning also saw substantial enhancements aligned with Large Language Models advancements.
- Re2TAL: Rewiring Pretrained Video Backbones for Reversible Temporal Action Localization34
This work proposes a novel end-to-end method Re2TAL, which rewires pretrained video backbones for reversible TAL, and proposes a network rewiring mechanism, to transform any module with a residual connection to a reversible module without changing any parameters.
- Real-time Hyperspectral Imaging in Hardware via Trained Metasurface Encoders34
This work designs and implements a prototype version of Hyplex™, a CMOS-compatible, fast hyperspectral camera that replaces bulk optics with nanoscale metasurfaces inversely designed through artificial intelligence, and presents what is, to the best of the authors' knowledge, the largest publicly available labeled hyperspectrals dataset for semantic segmentation.
- 34
- Detecting small faces in the wild based on generative adversarial network and contextual information34
An algorithm to directly generate a clear high-resolution face from a small blurry one by adopting a generative adversarial network (GAN) is proposed, and the achieved performance outperforms previous state-of-the-art methods by a large margin.
- Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation34
A deep neural network is trained to predict UAV controls from raw image data for the task of autonomous UAV racing in a photo-realistic simulation, demonstrating that the trained network outperforms state-of-the-art methods and flies more consistently than many human pilots.
- 34
- 34
- 34
- 33
- 32
- SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications31
This work presents a general framework for adversarial attacks on trained agents, which covers semantic perturbations to the environment of the agent performing the task as well as pixel-level attacks.
- EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queries30
This work formalizes a pipeline that better entangles 3D multiview geometry with 2D object retrieval from egocentric videos and achieves an overall success rate of up to 87.12%, which sets a new state-of-the-art result in the VQ3D task.
- 29
- 29
- 29
- Re-ReND: Real-time Rendering of NeRFs across Devices28
Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines, and can be directly integrated with standard rasterization frameworks.
- Gabor Layers Enhance Network Robustness28
It is observed that architectures enhanced with Gabor layers gain a consistent boost in robustness over regular models and preserve high generalizing test performance, even though these layers come at a negligible increase in the number of parameters.
- 28
- Behind the Magic, MERLIM: Multi-Modal Evaluation Benchmark for Large Image-Language Models26
A Multi-modal Evaluation Benchmark named MERLIM is introduced, a scalable test-bed to assess the capabilities of IT-LVLMs on fundamental computer vision tasks and suggests that these models have weak visual grounding, but manage to make adequate guesses from global visual patterns or language biases contained in the LLM component.
- Towards Active Learning for Action Spotting in Association Football Videos26
This paper proposes an active learning framework that selects the most informative video samples to be annotated next, thus drastically reducing the annotation effort and accelerating the training of action spotting models to reach the highest accuracy at a faster pace.
- 26
- Learning to Cut by Watching Movies26
This paper devise a model that learns to discriminate between real and artificial cuts via contrastive learning, and sets up a new task and a set of baselines to benchmark video cut generation.
- Efficient Image Pre-training with Siamese Cropped Masked Autoencoders25
This work proposes CropMAE, an alternative approach to the Siamese pre-training introduced by SiamMAE, and demonstrates that CropMAE learns similar object-centric representations without explicit motion, showing that current self-supervised learning methods do not learn such representations from explicit object motion, but rather thanks to the implicit image transformations between the two views.
- SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images25
This work presents SPARF, a large-scale ShapeNet-based synthetic dataset for novel view synthesis consisting of ~17 million images rendered from nearly 40,000 shapes at high resolution (400 × 400 pixels), and proposes a novel pipeline (SuRFNet) that learns to generate sparse voxel radiance fields from only few views.
- Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization25
The proposed adaptations improve egocentric query detection, leading to a better visual query localization system in both 2D and 3D configurations, and a novel transformer-based module that allows for object-proposal set context to be considered while incorporating query information is proposed.
- Assessing the Robustness of Visual Question Answering25
This work proposes a new method that uses semantically related questions, dubbed basic questions, acting as noise to evaluate the robustness of VQA models, and hypothesizes that as the similarity of a basic question to the main question decreases, the level of noise increases.
- 25
- 25
- Train Long, Think Short: Curriculum Learning for Efficient Reasoning24
This work proposes a curriculum learning strategy for length-controlled reasoning using Group Relative Policy Optimization (GRPO), and starts with generous token budgets and gradually tightens them over training, encouraging models to first discover effective solution strategies and then distill them into more concise reasoning traces.
- 24
- Using deep neural networks to diagnose engine pre-ignition24
Results indicate that the first model with reduced input dimensions, and correspondingly smaller size of the network, shows better performance in detecting pre-ignition cycles with an F1 score of 79%.
- Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?23
This work proposes a novel metric for measuring adaptation of Online Continual Learning algorithms based on the accuracy on the near-future samples, where spurious correlations are removed, and finds that better generalization can be achieved by retaining and reusing past seen information.
- Contrastive Language-Action Pre-training for Temporal Localization23
A novel post-pre-training approach without freezing the video encoder which leverages language is proposed to capture visio-linguistic relations between activities, background video clips and language in the form of captions and improves the state-of-the-art on temporal action localization, few-shot temporalaction localization, and video language grounding tasks.
- DeformRS: Certifying Input Deformations with Randomized Smoothing23
This work reformulate certification in randomized smoothing setting for both general vector field and parameterized deformations and proposes DeformRS-VF and Deform RS-Par, respectively, which scales to large networks on large input datasets.
- BAOD: Budget-Aware Object Detection23
By combining an optimal image/annotation selection scheme with hybrid supervised learning to solve the BAOD problem, it is shown that one can achieve the performance of a strongly supervised detector on PASCAL-VOC 2007 while saving 12.8% of its original annotation budget.
- Efficient Tracking Proposals using 2D-3D Siamese Networks on LIDAR22
This paper uses the dense and structured Bird Eye View (BEV) representation of LIDAR point clouds to efficiently search for objects of interest and shows that the Region Proposal in the BEV outperforms Bayesian methods such as Kalman and Particle Filters in providing proposal by a significant margin and such candidates are suitable for the 3D Siamese network.
- 22
- Dinkelbach NCUT: An Efficient Framework for Solving Normalized Cuts Problems with Priors and Convex Constraints22
A novel framework, called Dinkelbach NCUT (DNCUT), is proposed, which efficiently solves the normalized graph cut (NCUT) problem under general, convex constraints, as well as, under given priors on the nodes of the graph.
- 21
- AToM: Amortized Text-to-Mesh using 2D Diffusion20
Amortized Text-to-Mesh (AToM) demonstrates strong generalizability, offering finegrained 3D assets for unseen interpolated prompts without further optimization during inference, unlike per-prompt solutions.
- Learning to Read Analog Gauges from Synthetic Data20
A two-stage CNN pipeline that identifies the key structural components of an analog gauge and outputs an angular reading is proposed that identifies the key structural components of an analog gauge and outputs an angular reading.
- 20
- Boundary-Denoising for Video Activity Localization20
An encoder-decoder model named DenoiseLoc is proposed, which achieves state-of-the-art performance on TACoS and MAD datasets, but with much fewer predictions compared to other current methods.
- Combating Mode Collapse via Offline Manifold Entropy Estimation20
A novel training pipeline to address the mode collapse issue of GANs is proposed to generalize the discriminator as feature embedding and maximize the entropy of distributions in the embedding space learned by the discriminators.
- 20
- Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think19
This paper challenges the reliance on the final answer by posing the following two questions: Does the final answer reliably represent the model's optimal conclusion?
- SMILE: Infusing Spatial and Motion Semantics in Masked Video Learning19
A new self-supervised video learning paradigm capable of learning strong video representations without requiring any natural video data is established, called SMILE, which surpasses current state-of-the-art SSL methods.
- 19
- An Embarrassingly Simple Defense Against LLM Abliteration Attacks18
Comprehensive evaluations of safety and utility demonstrate that extended-refusal fine-tuning effectively neutralizes abliteration attacks while preserving general model performance and enhancing robustness across multiple alignment scenarios.
- Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants18
A unified solution to the misalignment dilemma in the three tasks of CIL, LTCIL and FSCIL is offered andoretical analysis indicates that the method holds the neural collapse optimality in an incremental fashion regardless of data imbalance or data scarcity.
- ETAD: Training Action Detection End to End on a Laptop18
This work proposes an efficient temporal action detector that can train directly from video frames with extremely low GPU memory consumption and dynamically sample only a small subset of proposals during training to alleviate the computational redundancy in training.
- Towards Analyzing Semantic Robustness of Deep Neural Networks18
This work formalizes the problem of finding robust semantic regions of the network as optimizing integral bounds and develops expressions for update directions of the region bounds to quantitatively evaluate the semantic robustness of different popular network architectures.
- 18
- Rethinking Safety in LLM Fine-tuning: An Optimization Perspective17
This work proposes a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance by creating a stable optimization path and retains the original pre-trained model's safety properties, and proposes practical guidelines for maintaining both model performance and safety during adaptation.
- Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers17
The Loong Project is introduced: an open-source framework for scalable synthetic data generation and verification across a diverse range of reasoning-intensive domains and benchmarking LoongBench on a broad suite of both open-source and proprietary LLMs to evaluate domain coverage and reveal performance bottlenecks.
- A data-driven approach to interfacial polymerization exploiting machine learning for predicting thin-film composite membrane formation17
This work constructed a dataset using 18 organic- and 73 water-phase monomers, conducting 1246 interfacial reactions and analyzing membranes via AFM and optical microscopy, indicating that film formation can be predicted directly from monomers, facilitating the potential of data-driven membrane development.
- MVTN: Learning Multi-view Transformations for 3D Understanding17
This work introduces the Multi-View Transformation Network (MVTN), which uses differentiable rendering to determine optimal view-points for 3D shape recognition and integrates MVTN into a novel adaptive multi-view pipeline that is capable of rendering both 3D meshes and point clouds.
- Privacy-Preserving Optics for Enhancing Protection in Face De-Identification17
This paper first learns an optical encoder along with a regression model to obtain a face heatmap while hiding the face identity from the source image, and proposes an anonymization framework that generates a new face using the privacy-preserving image, face heatmap, and a reference face image from a public dataset as input.
- Pix4Point: Image Pretrained Standard Transformers for 3D Point Cloud Understanding17
This work presents Progressive Point Patch Embedding and presents a new point cloud Transformer model namely PViT, which shares the same backbone as Transformer but is shown to be less hungry for data, enabling Transformer to achieve performance comparable to the state-of-the-art.
- Automatic Animation of Hair Blowing in Still Portrait Photos17
This work innovatively treats hair wisp extraction as an instance segmentation problem, where a hair wisp is referred to as an instance, and proposes a wisp-aware animation module that animates hair wisps with pleasing motions without noticeable artifacts.
- A Stochastic Derivative-Free Optimization Method with Importance Sampling: Theory and Learning to Control17
This paper proposes the first derivative free optimization method with importance sampling and derive new improved complexity results on non-convex, convex and strongly convex functions and test the method on a collection of continuous control tasks on MuJoCo environments with varying difficulty.
- A Stochastic Derivative-Free Optimization Method with Importance Sampling17
This paper proposes the first derivative free optimization method with importance sampling and derive new improved complexity results on non-convex, convex and strongly convex functions and test the method on a collection of continuous control tasks on MuJoCo environments with varying difficulty.
- 17
- Improving cost estimation in market-based coordination of a distributed sensing task17
This paper provides a first analysis of the cost estimation process in the TraderBots approach applied to a distributed sensing task and shows increased team efficiency when cost estimates reflect different environmental and mission characteristics.
- Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learning16
This paper considers the fine-tuning-as-a-service setup where the user uploads their data to a service provider to get a customized model that excels on the user's selected task.
- Opentad: a Unified Framework and Comprehensive Study of Temporal Action Detection16
OpenTAD, a unified TAD framework consolidating 16 different TAD methods and 9 standard datasets into a modular codebase is proposed, leading to a new state-of-the-art TAD method built upon existing techniques for each component.
- Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognition16
This work revisits the traditional backdoor threat model and incorporates additional video-related aspects to that model, and shows that poisoned-label image backdoor attacks could be extended temporally in two ways, statically and dynamically, leading to highly effective attacks in the video domain.
- Towards AI-Powered Video Assistant Referee System (VARS) for Association Football16
The semi-automated Video Assistant Referee System (VARS) is presented that leverages the latest findings in multi-view video analysis and achieves a new state-of-the-art on the SoccerNet-MVFoul dataset.
- Negative Frames Matter in Egocentric Visual Query 2D Localization16
A more efficient and effective solution to the VQ2D challenge, which brings the training loop from ~15 days to less than 24 hours, and achieves 0.17% spatial-temporal AP, which is 31% higher than the baseline.
- 16
- Harnessing Temporal Causality for Advanced Temporal Action Detection15
This work introduces CausalTAD, which combines causal attention and causal Mamba to achieve state-of-the-art performance on multiple benchmarks, and ranked 1st in the Action Recognition, Action Detection, and Audio-Based Interaction Detection tracks at the EPIC-Kitchens Challenge 2024.
- Towards Automated Movie Trailer Generation15
This work introduces Trailer Generation Transformer (TGT), a deep-learning framework utilizing an encoder-decoder architecture that generates plausible trailers from a full movie by automating shot selection and composition.
- Ego4D: Around the World in 3,600 Hours of Egocentric Video15
Ego4D, a massive-scale egocentric video dataset and benchmark suite, introduced, dramatically expands the volume of diverse egocentric video footage publicly available to the research community and presents a host of new benchmark challenges centered around understanding the first-person visual experience.
- Don’t FREAK Out: A Frequency-Inspired Approach to Detecting Backdoor Poisoned Samples in DNNs15
This work investigates the frequency sensitivity of Deep Neural Networks when presented with clean samples versus poisoned samples and proposes FREAK, a frequency-based poisoned sample detection algorithm that is simple yet effective against frequency backdoor attacks.
- Learning to Identify Critical States for Reinforcement Learning from Videos15
This work proposes a new method called Deep State Identifier, which learns to predict returns from episodes encoded as videos by using a kind of mask-based sensitivity analysis to extract/identify important critical states.
- 15
- 15
- 15
- 15
- 14
- 3DeformRS: Certifying Spatial Deformations on Point Clouds14
3DeformRS, a method to certify the robustness of point cloud Deep Neural Networks (DNNs) against real-world deformations, is proposed and is fast, scales well with point cloud size, and provides comparable-to-better certificates.
- Generalizability of Adversarial Robustness Under Distribution Shifts14
This work trains robust models on multiple domains and evaluates their accuracy and robustness on an unseen domain, and observes that both empirical and certified robustness generalize to unseen domains, and the level of generalizability does not correlate well with input visual similarity.
- ThumbNet: One Thumbnail Image Contains All You Need for Recognition14
ThumbNet is proposed, a unified framework to simultaneously accelerate and compress CNN models by enabling them to infer on one thumbnail image, and the thumbnail-input inference network learned by ThumbNet can adequately retain the accuracy of the original-input network even when the input images are downscaled 16 times.
- Learning a Controller Fusion Network by Online Trajectory Filtering for Vision-Based UAV Racing14
This paper proposes learning an optimized controller using a DNN that fuses multiple controllers and develops a network that beats all baselines in extensive experiments and approaches the performance of a professional human pilot.
- SynFER: Towards Boosting Facial Expression Recognition With Synthetic Data13
This work introduces SynFER (Synthesis of Facial Expressions with Refined Control), a novel synthetic framework for synthesizing facial expression image data based on high-level textual descriptions as well as more fine-grained and precise control through facial action units.
- Action Anticipation from Soccernet Football Video Broadcasts13
The task of action anticipation for football broadcast videos is introduced, which consists in predicting future actions in unobserved future frames, within a fiveor ten-second anticipation window, and a baseline method is proposed that adapts FUTR, a state-of-the-art action anticipation model, to predict ball-related actions.
- SegNeRF: 3D Part Segmentation with Neural Radiance Fields13
This work proposes SegNeRF: a neural field representation that integrates a semantic field along with the usual radiance field, and inherits from previous works the ability to perform novel view synthesis and 3D reconstruction, and enables 3D part segmentation from a few images.
- 13
- Exploring Missing Modality in Multimodal Egocentric Datasets12
This study delves into the impact of missing modalities on egocentric action recognition, particularly within transformer-based models, and introduces a novel concept-Missing Modality Token (MMT) to maintain performance even when modalities are absent, a strategy that proves effective in the Ego4D, Epic-Kitchens, and Epic-Sounds datasets.
- Diffusion-Based Imaginative Coordination for Bimanual Manipulation12
A multi-frame latent prediction strategy that encodes future states in a compressed latent space, preserving task-relevant features and a unidirectional attention mechanism where video prediction is conditioned on the action, while action prediction remains independent of video prediction is introduced.
- Hardware-accelerated integrated optoelectronic platform towards real-time high-resolution hyperspectral video understanding12
A hardware-accelerated integrated optoelectronic platform specifically designed for the real-time analysis of multidimensional video analysis, achieving data processing speeds of 1.2 Tb/s, shows promise in real-time AI video understanding, with potential implications for enhancing human-machine interactions and advancing cognitive processing technologies.
- RMP-SAM: Towards Real-Time Multi-Purpose Segment Anything12
This work explores a new real-time segmentation setting, named all-purpose segmentation in real-time, to transfer VFMs in real-time deployment and presents Real-Time All Purpose SAM (RAP-SAM), which contains an efficient encoder and an efficient decoupled decoder to perform prompt-driven decoding.
- Online Distillation with Continual Learning for Cyclic Domain Shifts12
This work integrates several state-of-the-art continual learning methods in the context of online distillation and demonstrates their effectiveness in reducing catastrophic forgetting and provides a detailed analysis of the proposed solution in the case of cyclic domain shifts.
- Just a Glimpse: Rethinking Temporal Information for Video Continual Learning12
This paper proposes SMILE a novel replay mechanism for effective video continual learning based on individual/single frames, and shows that under extreme memory constraints, video diversity plays a more significant role than temporal information.
- Exploring Open-Vocabulary Semantic Segmentation without Human Labels12
This work presents ZeroSeg, a novel method that leverages the existing pretrained vision-language (VL) model to train open-vocabulary zero-shot semantic segmentation models, achieving state-of-the-art performance when compared to other zero- shot segmentation methods under the same training data, while also performing competitively compared to strongly supervised methods.
- On the Robustness of Quality Measures for GANs12
This work evaluates the robustness of quality measures of generative models such as Inception Score (IS) and Fr\'echet Inception Distance (FID) and shows that such metrics can also be manipulated by additive pixel perturbations.
- Large Scale Asset Extraction for Urban Images12
A novel adaptive pipeline for interleaving object proposals with object classification is proposed and used as a formulation for asset detection and it is demonstrated that other methods can be improved by incorporating the authors' interleaved proposals.
- Camera Motion and Surrounding Scene Appearance as Context for Action Recognition12
A weak foreground-background segmentation approach to robustly extract not only foreground features that are focused on the actors, but also global camera motion and contextual scene information that interestingly is shown to be discriminative for certain action classes.
- 11
- 11
- Combating Mode Collapse in GANs via Manifold Entropy Estimation11
A novel training pipeline to address the mode collapse issue of GANs is proposed, which outperforms the GAN baseline, MaF-GAN on CelebA, and surpasses the recent state-of-the-art energy-based model on the ANIME-FACE dataset.
- Adaptive Learning of the Optimal Mini-Batch Size of SGD.11
This paper designs a practical SGD method capable of learning the optimal mini-batch size adaptively throughout its iterations, and in its experiments with synthetic and real data robustly exhibits nearly optimal behaviour.
- 11
- Designing Camera Networks by Convex Quadratic Programming11
It is shown that the general camera placement problem can be formulated mathematically as a convex binary quadratic program (BQP) under linear constraints and an optimization strategy with a favorable trade‐off between speed and solution quality is proposed.
- 11
- ShadowNet for Data-Centric Quantum System Learning10
This work presents ShadowNet, implemented under both convolutional and attention mechanisms, to efficiently and faithfully tackle two pivotal QSL tasks: quantum state tomography (QST) and direct fidelity estimation (DFE).
- Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization10
A plug-and-play module, S6MOD, which can be integrated into most existing methods and directly improve adaptability, and design a class-conditional routing algorithm for dynamic, uncertainty-based adjustment and implement a contrastive discretization loss to optimize it.
- Can Video Diffusion Model Reconstruct 4D Geometry?10
Sora3R is a novel framework that taps into the rich spatiotemporal priors of large-scale video diffusion models to directly infer 4D pointmaps from casual videos, achieving performance on par with state-of-the-art methods for dynamic 4D reconstruction across diverse scenarios.
- Physics-Informed Deep Neural Network for Backward-in-Time Prediction: Application to Rayleigh–Bénard Convection10
A physics-informed deep neural network (PI-DNN) is presented to predict past states of a dissipative dynamical system from snapshots of the subsequent evolution of the system state using a non-positive diffusion coefficient when integrating backward.
- LC-NAS: Latency Constrained Neural Architecture Search for Point Cloud Networks10
This paper introduces a new NAS framework, dubbed LC-NAS, that searches for point cloud architectures constrained to a target latency, and implements a novel latency constraint formulation for the trade-off between accuracy and latency in the architecture search.
- Certified Robustness in Federated Learning10
This work deploys randomized smoothing, a widely-used and scalable certification method, to certify deep networks trained on a federated setup against input perturbations and transformations and finds that the simple federated averaging technique is effective in building more accurate, but also more certifiably-robust models, compared to training solely on local data.
- 10
- Learning a strong detector for action localization in videos10
This paper proposes a strong object detector for better action localization in videos, based on the single shot multibox detector (SSD) framework, and outperform the previous state-of-the-art methods by a large margin in terms of frame-m AP and video-mAP, especially at a higher overlap threshold.
- Teaching UAVs to Race With Observational Imitation Learning10
A modular network architecture that decouples perception from control, and is trained using Observational Imitation Learning (OIL), a novel imitation learning variant that supports online training and automatic selection of optimal behavior from observing multiple teachers is proposed.
- Teaching UAVs to Race Using UE4Sim10
A photo-realistic simulator is developed that can afford the generation of large amounts of training data to teach a UAV to autonomously race through challenging tracks and outperforms state-of-the-art methods and flies more consistently than many human pilots.
- A Proximal Alternating Direction Method for Semi-Definite Rank Minimization10
This paper proposes a proximal Alternating Direction Method (ADM) for the well-known semi-definite rank regularized minimization problem and proves that the proposed method always converges to a KKT stationary point under mild conditions.
- 3D Aware Correction and Completion of Depth Maps in Piecewise Planar Scenes10
This paper proposes a framework that uses the RGB-D sensor’s noise profile to adaptively and robustly fit plane segments and iteratively complete the depth map, when possible, and exploits 3D and appearance cues that encourage pixels to take on depth values that will be compatible in 3D to the piecewise planar assumption.
- 10
- 10
- Deep Learning for Action Spotting in Association Football Videos9
The history of action spotting in sports is traced, from the creation of the task back in 2018, to the role it plays today in research and the sports industry, as well as improving on the first baselines and making action spotting a viable option for the sports industry.
- EAGLE: Enhanced Visual Grounding Minimizes Hallucinations in Instructional Multimodal Models9
This paper shows that a straightforward reformulation of the original contrastive pre-training task results in an improved visual encoder that can be incorporated into the instructional multi-modal architecture without additional instructional training.
- Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation9
This paper theoretically study non-greedy layer-wise training and shows that the convergence cannot be assured when the local gradient in a module w.r.t. its input is not reconciled with the local gradient in the previous module w.r.t. its output.
- Dr2Net: Dynamic Reversible Dual-Residual Networks for Memory-Efficient Finetuning9
The proposed Dynamic Reversible Dual-Residual Networks, or Dr2Net, a novel family of network architectures that acts as a surrogate net-work to finetune a pretrained model with substantially re-duced memory consumption and introduces a dynamic training strategy that seam-lessly transitions the pretrained model to a reversible net-work with much higher numerical precision.
- Constrained Clustering: General Pairwise and Cardinality Constraints9
The experimental results show that the proposed method exploits the constraints to achieve perfect clustering performance with improved clustering to $2-5$ % in classical clustering metrics, e.g., Adjusted Random Index (ARI), Mirkin's Index (MI), and Huber’s Index (HI), outerperfomring all compared-againts methods across the board.
- When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search9
A new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy, is proposed.
- SAR: Stroke Authorship Recognition9
The Stroke Authorship Recognition approach is proposed, a novel method that distinguishes the authorship of 2D digitized drawings and its usefulness is demonstrated in several applications including the detection of fraudulent sketches, the training and monitoring of artists in learning a particular new style and the first quantitative way to measure the quality of automatic sketch synthesis tools.
- Recognizing Team Formation in American Football9
This chapter proposes a novel framework where given an American football play clip, the video frame in which the offensive team lines in formation, the line of scrimmage for that play, and the type of player formation the Offensive team takes on is automatically identified.
- 4D-Bench: Benchmarking Multi-Modal Large Language Models for 4D Object Understanding8
4D-Bench is introduced, the first benchmark to evaluate the capabilities of MLLMs in 4D object understanding, featuring tasks in 4D object Question Answering and 4D object captioning, and indicates that MLLMs generally exhibit weaker temporal understanding compared to their appearance understanding.
- Shh, don't say that! Domain Certification in LLMs8
This work introduces domain certification; a guarantee that accurately characterizes the out-of-domain behavior of language models that provides adversarial bounds as a certificate and proposes a simple yet effective approach, which is called VALID, that provides adversarial bounds as a certificate.
- 8
- Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation8
DietCL outperforms, by a large margin, all existing supervised CL algorithms as well as more recent continual semi-supervised methods, and meticulously allocates computational budget for both unlabeled and labeled data jointly.
- Improving GAN Training via Feature Space Shrinkage8
This paper proposes a simple yet effective module, namely AdaptiveMix, for GANs, which shrinks the regions of training data in the image representation space of the discriminator, and proposes to construct hard samples and narrow down the feature distance between hard and easy samples.
- OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos8
OWL introduces a simple-yet-effective approach via Observing, Watching, and Listening (OWL) that leverages audiovisual information and context for egocentric Temporal Action Localization (TAL) and boosts the localization performance over visual-only models.
- Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions8
This work proposes a new method that utilizes semantically related questions, referred to as basic questions, acting as noise to evaluate the robustness of VQA models, and proposes a novel robustness measure, R_score, and two basic question datasets to standardize the analysis of V QA model robustness.
- 8
- 8
- 8
- Egocentric Video-Language Pretraining @ Ego4D Challenge 20228
A pretrained video-language model that is able to transfer its egocentric video-text representation or video-only representation to several video downstream tasks and pioneer Egocentric VLP from pretraining dataset, pretraining objective, and development set is developed.
- Multi-Modal Few-Shot Temporal Action Detection8
A novel MUlti-modality PromPt mETa-learning (MUPPET) method that bridges pretrained vision and language models whilst maximally reusing already learned capacity to tackle few-shot object detection problem and achieves the state-of-the-art performance on MS-COCO dataset.
- 8
- 8
- 8
- 8
- Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection7
Rank-One Safety Injection (ROSI), a white-box method that amplifies a model's safety alignment by permanently steering its activations toward the refusal-mediating subspace, is proposed, suggesting that targeted, interpretable weight steering is a cheap and potent mechanism to improve LLM safety, complementing more resource-intensive fine-tuning paradigms.
- 7
- 7
- 7
- SimCS: Simulation for Domain Incremental Online Continual Segmentation7
This work explores the problem of Online Domain-Incremental Continual Segmentation (ODICS), where the model is continually trained over batches of densely labeled images from different domains, with limited computation and no information about the task boundaries.
- APES: Audiovisual Person Search in Untrimmed Video7
This paper presents the Audiovisual Person Search dataset (APES), a new dataset composed of untrimmed videos whose audio (voices) and visual streams are densely annotated and shows that modeling audiovisUAL cues benefits the recognition of people’s identities.
- 7
- Integration of Absolute Orientation Measurements in the KinectFusion Reconstruction Pipeline7
This paper shows how absolute orientation measurements provided by low-cost but high-fidelity IMU sensors can be integrated into the KinectFusion pipeline and uses this orientation data to seed and regularize the ICP registration technique.
- Action Search: Learning to Search for Human Activities in Untrimmed Videos7
A novel approach that mimics the way people annotate activities in video sequences is proposed, using a Recurrent Neural Network, that can efficiently explore a video and determine the time boundaries during which an action occurs.
- A Topic Model Approach to Representing and Classifying Football Plays7
This paper addresses the combined problem of automatic representation and classification of football plays as summarized in Fig. 1 and will allow broadcasters to analyze large collections of videos, index and retrieve them more efficiently and help coaches to infer patterns and tendencies of opponents more efficiently, resulting in better strategy planning in a game.
- SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues6
Experiments across multiple VLMs and an additional state-of-the-art benchmark show that safety decisions are highly sensitive to semantic cues, indicating reliance on learned visual-linguistic associations rather than grounded visual understanding.
- $β$-CLIP: Text-Conditioned Contrastive Learning for Multi-Granular Vision-Language Alignment6
This work proposes $\beta$-CLIP, a multi-granular text-conditioned contrastive learning framework designed to achieve hierarchical alignment between multiple textual granularities-from full captions to sentences and phrases-and their corresponding visual regions, and introduces the $\beta$-Contextualized Contrastive Alignment Loss ($\beta$-CAL).
- Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence6
Context-oriented decomposition adaptation (CorDA), a novel method that initializes adapters in a task-aware manner, and develops CorDA++ in KPM, which not only achieves better fine-tuning performance than LoRA, but also mitigates the forgetting of pre-trained knowledge in both large language models and vision language models.
- Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation6
This work introduces an automated pipeline that constructs image pairs with annotated semantic and visual correspondences based on existing subject-driven image generation datasets, and proposes a new metric, Visual Semantic Matching (VSM), that quantifies visual inconsistencies in subject-driven image generation.
- 6
- MatchDiffusion: Training-Free Generation of Match-Cuts6
This work presents the first training-free method for match-cut generation using text-to-video diffusion models, and leverages a key property of diffusion models: early denoising steps define the scene's broad structure, while later steps add details.
- 6
- 6
- 6
- 6
- Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges6
A multi-class margin generalization bound is derived, which provides a theoretical explanation for why continuing training can still lead to accuracy improvement on test set, even after the train accuracy has reached 100% and is referred to as "non-conservative generalization".
- Semantic Part RCNN for Real-World Pedestrian Detection.6
This work introduces the semantic part information for learning the pedestrian detector by detecting key points of each pedestrian proposal and extracting six semantic parts according to the predicted key points.
- 6
- 6
- Dynamic textures: models and applications6
This dissertation believes that no single spatiotemporal model can be proposed to handle dynamic textures sampled from this three-dimensional space and proposes three models, each of which applies to a certain “range” of dynamic textures in this space.
- QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation5
This work introduces QuanBench+, a unified benchmark spanning Qiskit, PennyLane, and Cirq, with 42 aligned tasks covering quantum algorithms, gate decomposition, and state preparation, which shows clear progress, but also that reliable multi-framework quantum code generation remains unsolved and still depends strongly on framework-specific knowledge.
- Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic5
This work shows how reasoning capabilities, typically developed through expensive training, can be extracted from existing open-source models and reused through simple tensor arithmetic, offering a practical way to enhance models by recycling prior computational investments.
- Compressed-Language Models for Understanding Compressed File Formats: a JPEG Exploration5
This study investigates whether Compressed-Language Models, i.e. language models operating on raw byte streams from Compressed File Formats~(CFFs), can understand files compressed by CFFs, and tests if CLMs understand the JPEG format.
- Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable5
This paper proposes a novel arbitrary-in-arbitrary-out (AIAO) strategy that makes watermarks resilient to fine-tuning-based removal and embeds the backdoor into the feature space of sampled subpaths, where a mask-controlled trigger function is proposed to preserve the generation performance and ensure the invisibility of the embedded backdoor.
- Generative Timelines for Instructed Visual Assembly5
A large multimodal language model is developed, which is designed to process visual content, compactly represent timelines and accurately interpret timeline editing instructions, which substantially outperforms established baseline models in accurately executing complex assembly instructions across various real-world inspired scenarios.
- 5
- Multi-Stream Cellular Test-Time Adaptation of Real-Time Models Evolving in Dynamic Environments5
A novel Multi-Stream Cellular Test-Time Adaptation (MSC-TTA) setup where models adapt on the fly to a dynamic environment divided into cells, and a real-time adaptive student-teacher method that leverages the multiple streams available in each cell to quickly adapt to changing data distributions.
- Estimating more camera poses for ego-centric videos is essential for VQ3D5
A new pipeline is designed for the challenging egocentric video camera pose estimation problem and the current VQ3D framework is revisited to revisit it in terms of performance and efficiency.
- 5
- MAP Inference Via $$\ell _2$$-Sphere Linear Program Reformulation5
A perturbed alternating direction method of multipliers (ADMM) algorithm is proposed to optimize the LS–LP problem, by adding a sufficiently small perturbation of the original MAP inference problem, and it is proved that the perturbed ADMM algorithm globally converges to the ϵ\documentclass[12pt]{minimal} \usepackage{amsmath} \use package{wasysym} ℓ2.
- Constrained K-means with General Pairwise and Cardinality Constraints5
This paper enforces pairwise and cardinality constraints into a unified clustering model starting with the integer program formulation of the standard K-means and derives an equivalent continuous reformulation with simple constraints, which can be efficiently solved by Alternating Direction Method of Multipliers (ADMM).
- 5
- 4
- SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning4
No method generalizes consistently across all factors, video-only transformers perform better under domain shifts, CNNs outperform for fine-grained tasks, and video-text models often underperform despite large scale pretraining.
- 4
- Evaluation of Test-Time Adaptation Under Computational Time Constraints4
This paper proposes a more realistic evaluation protocol for Test Time Adaptation methods, where data is received in an online fashion from a constant-speed data stream, thereby accounting for the method’s adaptation speed.
- Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels4
A novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations and outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground- Truth flows.
- 4
- 4
- 4
- 4
- 4
- Modeling dynamic swarms4
This paper presents a probabilistic model that learns both the spatial layout of swarm elements and their joint dynamics that are modeled as linear transformations using ICM, which iterates between estimating these transformations and updating their distribution in the spatiotemporal neighborhoods.
- Low-rank quadratic semidefinite programming4
This paper addresses a special class of nonconvex quadratic matrix optimization problems, which require a low rank positive semidefinite solution, and exploits the structure of these problems to derive an efficient solver that converges to their local optima.
- NAKBA NLP 2026: Shared Task on Arabic Handwritten Manuscript Understanding (Palestine Memory–Omar Al-Saleh Memoir)3
The NAKBA NLP 2026 shared task on Arabic manuscript understanding is introduced, comprising two complementary tracks: a manual transcription track, in which participating teams annotate unlabelled handwritten line images, and an automatic system track for handwritten text recognition (HTR).
- Spiideo SoccerNet SynLoc: Single Frame World Coordinate Athlete Detection and Localization with Synthetic Data3
The Spiideo SoccerNet SynLoc dataset consists of synthetic athletes rendered on top of images from real world installation of such cameras and a new task of detecting the players in the world pitch coordinate system and a new metric based solely on real world physical properties where the representation in the image is irrelevant.
- Pixels or Positions? Benchmarking Modalities in Group Activity Recognition3
A novel role-aware graph architecture for tracking-based GAR directly encodes tactical structure through positional edges connecting players by their on-pitch roles, providing new insights into the relative strengths of pixels and positions for group activity recognition in sports.
- 3
- UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields3
UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling joint hyperspectral novel view synthesis and unsupervised material segmentation, is introduced, demonstrating superior spectral reconstruction and material segmentation to existing methods.
- 3
- 3
- 3
- 3
- 3
- 3
- From Categories to Classifiers: Name-Only Continual Learning by Exploring the Web3
Overall, this paper underscores the potential of using uncurated webly-supervised data to mitigate the challenges associated with manual data labeling in continual learning.
- 3
- 3
- ClusTR: Clustering Training for Robustness.3
ClusTR (Clustering Training for Robustness), a clustering-based and adversary-free training framework to learn robust models, is proposed and can be equipped with simple and fast adversarial training to improve the current state-of-the-art in robustness.
- Robust Gabor Networks3
The effect of replacing the first layers of neural network architectures with convolutional layers that are based on Gabor filters with learnable parameters is explored, and architectures enhanced with Gabor layers gain advantages in terms of robustness when compared to the regular models.
- 3
- 3
- 3
- 3
- A Matrix Splitting Method for Composite Function Minimization3
Extensive experiments show that the proposed and analyzes a new Matrix Splitting Method (MSM) for minimizing composite functions outperforms existing composite function minimization techniques in term of both efficiency and efficacy.
- Improving Saliency Models by Predicting Human Fixation Patches3
A fixation patch detector that predicts image patches that contain human fixations with high probability and allows state-of-the-art saliency methods to achieve better prediction performance on benchmark datasets is proposed.
- DeGMix: Efficient Multi-Task Dense Prediction With Deformable and Gating Mixer2
DeGMix is proposed, a unified encoder–decoder framework for multi-task dense prediction that combines deformable convolution and attention-based task interaction that consistently outperforms competitive CNN-based and Transformer-based multi-task methods across multiple dense prediction tasks.
- MeXtract: Light-Weight Metadata Extraction from Scientific Papers2
MeXtract, a family of lightweight language models designed for metadata extraction from scientific papers, achieves state-of-the-art performance on metadata extraction on the MOLE benchmark and extends the MOLE benchmark to incorporate model-specific metadata, providing an out-of-domain challenging subset.
- ResidualViT for Efficient Temporally Dense Video Encoding2
A vision transformer (ViT) architecture is introduced that leverages the large temporal redundancy in videos to efficiently compute temporally dense frame-level features and a lightweight distillation strategy is proposed to approximate the frame-level features of the original foundation model.
- Investigating Event-Based Cameras for Video Frame Interpolation in Sports2
This work designs and implements a bi-camera recording setup, including an RGB and an event-based camera to capture sports videos, to temporally align and spatially register both cameras, and demonstrates that TimeLens, an off-the-shelf event-based VFI model, can effectively generate slow-motion footage for sports videos.
- Hybrid Structure-from-Motion and Camera Relocalization for Enhanced Egocentric Localization2
A model ensemble strategy to improve the camera pose estimation part of the VQ3D task, which has been proven to be essential in previous work, and achieves the best performance regarding the most important metric, the overall success rate.
- 2
- 2
- OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos2
OSL-ActionSpotting is introduced, a Python library that unifies different action spotting algorithms to streamline research and applications in sports video analytics and significantly contributes to the field of sports video analysis, fostering enhanced analytical capabilities and collaborative research opportunities.
- Dynamically Masked Discriminator for Generative Adversarial Networks2
A novel method for GANs from the viewpoint of online continual learning is proposed, which automatically detects its retardation and then dynamically masks its features, such that the discriminator can adaptively learn the temporally-vary distribution of generated data.
- Learning Semantic Segmentation with Query Points Supervision on Aerial Images2
This work presents a weakly supervised learning algorithm to train semantic segmentation algorithms that only rely on query point annotations instead of full mask labels, and performs accurate semantic segmentation and improves efficiency by significantly reducing the cost and time required for manual annotation.
- Decoupled Mixup for Generalized Visual Recognition2
This work proposes a novel "Decoupled-Mixup" method to train CNN models for OOD visual recognition, which enhances features from discriminative regions and suppress noise-prone ones when combining an image pair to improve the generalization ability of trained models.
- MAIN: Multi-Attention Instance Network for video segmentation2
This work designs MAIN to segment multiple instances in a single forward pass, and optimize it with a novel loss function that favors class agnostic predictions and assigns instance-specific penalties.
- 2
- 2
- 2
- 2
- 2
- Stroke Style Transfer2
A novel method to transfer sketch style at the stroke level from one free-hand line drawing to another, whereby these drawings can be from different artists, and the quality of style transfer qualitatively is showcased by applying the proposed technique on several input-target combinations.
- BILGO: Bilateral greedy optimization for large scale semidefinite programming2
It is proved that BILGO converges to the global optimal solution at a rate of O(1/k), where k is the iteration counter, and successfully combines the efficiency of conventional rank-1 update algorithms and the effectiveness of gradient descent.
- Reduction of lymph tissue false positives in pulmonary embolism detection2
A novel approach is proposed that specifically aims at reducing the lymph FPs of PE, exploiting the anatomical context configuration of PE and lymph tissue in the lungs: a lymph FP connects to the airway and is located outside the artery, while a true PE should not connect to theAirway and must be inside the artery.
- Structured-Noise Masked Modeling for Video, Audio and Beyond1
This work introduces a structured noise-based masking approach that generates structured masks that capture modality-specific patterns without requiring handcrafted heuristics or access to the data.
- EgoPlay: Event-Triggered Video Editing for Egocentric Streams1
An event-aware evaluation protocol that separately measures post-trigger editing quality, pre-trigger preservation, and false-trigger robustness is introduced.
- 1
- ReactMotion: Generating Reactive Listener Motions from Speaker Utterance1
Reactive Listener Motion Generation from Speaker Utterance is introduced, a new task which aims to generate naturalistic listener body motions that appropriately respond to a speaker's utterance, and ReactMotion, a unified generative framework that jointly models text, audio, emotion, and motion.
- CounterCount: A Diagnostic Framework for Counting Bias in Vision Language Models1
A unified inference-time attention modulation strategy is introduced that reweights selected visual tokens, improving counterfactual counting accuracy by up to 8% across multiple VLMs and provides diagnostic insights for designing future VLMs.
- 1
- On the Importance of Pretraining Data Alignment for Atomic Property Prediction1
The Chemical Similarity Index (CSI), a novel metric inspired by computer vision's Fr\'echet Inception Distance, for molecular graphs which quantifies the alignment between upstream pretraining datasets and downstream tasks, is introduced.
- ADVMEM: Adversarial Memory Initialization for Realistic Test-Time Adaptation via Tracklet-Based Benchmarking1
This work uses ITD to conduct a thorough experimental analysis of current TTA methods, and proposes a novel adversarial memory initialization strategy to improve memory-based TTA methods, which substantially boosts the performance of various methods on the authors' challenging benchmark.
- Towards Faster and More Compact Foundation Models for Molecular Property Prediction1
This work analyzes the layer contributions of JMP and finds that later interaction blocks provide diminishing returns, suggesting an opportunity for model compression, and investigates block reduction strategies by pruning the pre-trained model and evaluating its impact on efficiency and accuracy during fine-tuning.
- 1
- 1
- Enhancing Neural Rendering Methods with Image Augmentations1
It is found that introducing image augmentations during training presents challenges such as geometric and photometric inconsistencies for learning NRMs from images, which can be alleviated by focusing on color manipulations and introducing learnable appearance embeddings that allow NRMs to explain away photometric variations.
- Artificial intelligence optical hardware empowers high-resolution hyperspectral video understanding at 1.2 Tb/s1
This work introduces a hardware-accelerated integrated optoelectronic platform for multidimensional video understanding in real-time, resulting in a data processing speed of 1.2 Tb/s with hundreds of frequency bands and megapixel spatial resolution at video rates.
- 1
- 1
- Rethinking Clustering for Robustness1
This paper studies how encouraging semantically-aligned features during deep neural network training can increase network robustness, and proposes ClusTR (Clustering Training for Robustness), a clustering-based and adversary-free training framework to learn robust models.
- 1
- 1
- 1
- 1
- 1
- Object Oriented Structure from Motion: Can a Scribble Help?1
This thesis aims to demonstrate the efforts towards in-situ applicability of EMMARM, which aims to provide real-time information about the physical properties of E-modulus and its applications in the construction of buildings and infrastructure.
- Physical activity among a cohort of hemodialysed Lebanese patients1
Most patients on chronic hemodialysis patients do not work and have no physical activity and sport and it would be imperative to enable multidisciplinary management and emphasize the role of rehabilitation and exercise in improving the handicap.
- Occlusion Detection via Structured Sparse Learning for Robust Object Tracking1
A novel tracking algorithm that models and detects occlusion through structured sparse learning is proposed and extensive experimental results show that the proposed tracker consistently outperforms the state-of-the-art trackers.
- 1
- –
- –
- –
- –
- –
- Sparse Attention for Dense Open-Vocabulary Prediction in CLIP–
This work studies an inference-time substitution of the row-wise softmax in the final visual self-attention layers with the $\alpha$-entmax transform, applied across both the standard query-key attention and self-correlation variants.
- –
- –
- Towards SDG 4: What if Every Student Had Their own Tutor?–
How education connects to the SDGs, and how new projects like TALEEM can strengthen skills, build confidence, and support a world where every child has the chance to learn is explored.
- SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy–
This work introduces SoccerLens, a benchmark for grounded soccer video understanding, and extends the attribution method of Chefer to jointly model spatial and temporal attention, and introduces evaluation metrics that measure whether model attention aligns with annotated cues or drifts toward spurious regions.
- –
- Hala Technical Report Building Arabic-Centric Instruction & Translation Models at Scale–
H ALA is presented, a family of Arabic-centric instruction and translation models built with the translate-and-tune pipeline that achieves state-of-the-art results within both the “nano” and “small” categories, outperforming their bases.
- –
- Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning–
This work forms a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods, which is based exclusively on the expected label distribution, thus making it class-agnostic and directly integrated into various rehearsal-based continual learning methods, reducing forgetting and favoring faster convergence.
- –
- Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient–
The Action Collapse Policy Gradient (ACPG) method is proposed, which accordingly affixes a synthetic ETF as the authors' action selection layer and induces the policy DNN to produce such an ideal configuration in the action selection layer while remaining optimal.
- –
- MoDA: Modulation Adapter for Fine-Grained Visual Grounding in Instructional MLLMs–
MoDA (Modulation Adapter), a lightweight module that enhances visual grounding through instruction-guided channel-wise modulation through multiplicative modulation on already-aligned features, enabling fine-grained control over which embedding dimensions are relevant for each instruction.
- CAPTAIN: Semantic Feature Injection for Memorization Mitigation in Text-to-Image Diffusion Models–
CAPTAIN is introduced, a training-free framework that mitigates memorization by directly modifying latent features during denoising by applies frequency-based noise initialization and injects semantically aligned features from non-memorized reference images into localized latent regions, suppressing memorization while preserving prompt fidelity and visual quality.
- Deep Learning at the Intersection: Certified Robustness as a Tool for 3D Vision–
This paper presents preliminary work on a novel connection between certified robustness in machine learning and the modeling of 3D objects, and proposes an algorithm to efficiently run RS in low-dimensional applications, such as 3D space, by expressing RS' fundamental operations as Gaussian smoothing on pre-computed voxel grids.
- –
- –
- –
- –
- On Robust Learning from Noisy Labels: A Permutation Layer Approach–
A permutation layer learning approach termed PermLL to dynamically calibrate the training process of the DNN subject to instance-dependent and instance-independent label noise and achieves state-of-the-art performance on both real and synthetic datasets.
- UnrealNAS: Can We Search Neural Architectures with Unreal Data?–
This work constructs three types of unreal datasets using randomly labeled real images; generated images and labels; and generated Gaussian noise with random labels to explore if NAS needs real data, suggesting the feasibility of performing NAS with unreal data.
- A thermophilic Cas13 enzyme for sensitive and specific one-pot detection of SARS-CoV-2–
The OPTIMA-dx detection module exhibits key features for point-of-care SARS-CoV-2 screening and pathogen detection in general and developed a mobile phone application to facilitate data reading, collection, and sharing.
- Customized Summarizations of Visual Data Collections–
The results show that the problem formulation enables a wide variety of customized summarizations, and that the solver is both significantly faster than state‐of‐the‐art commercial integer programming solvers and produces better solutions than fast relaxation‐based solvers.
- –
- –
- –
- –
- –
- –
- Expected Tight Bounds for Robust Deep Neural Network Training–
This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research.
- Probabilistically True and Tight Bounds for Robust Deep Neural Network Training.–
With such tight bounds, it is demonstrated that a simple standard training procedure can achieve the best robustness-accuracy trade-off across several architectures on both MNIST and CIFAR10.
- –
- –
- –
- –
- –
- –
- –
- –
- Exploring the visual components that make an image memorable–
Results indicate the first direct evidence from subject data that memorability of specific image regions is consistent across observers (i.e. individual regions remembered by one viewer are remembered by others as well), and that certain regions within a particular image are more memorable than others.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.