Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Xiaoou TangSuggested from co-authorship
- John W. FisherSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works435 from public data
- Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition5,594
A novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data.
- 4,387
- Unsupervised Feature Learning via Non-parametric Instance Discrimination4,044
This work forms this intuition as a non-parametric classification problem at the instance-level, and uses noise-contrastive estimation to tackle the computational challenges imposed by the large number of instance classes.
- VBench: Comprehensive Benchmark Suite for Video Generative Models2,062
VBench is a comprehensive benchmark suite that dissects “video generation quality” into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods, and investi-gate the gaps between video and image generation models.
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models1,936
InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure.
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling1,813
This work explores the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations and hopes this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems.
- Libra R-CNN: Towards Balanced Learning for Object Detection1,762
Libra R-CNN is proposed, a simple but effective framework towards balanced learning for object detection that integrates three novel components: IoU-balanced sampling, balanced feature pyramid, and balanced L1 loss, respectively for reducing the imbalance at sample, feature, and objective level.
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning1,729
The proposed AnimateDiff is a practical framework for animating personalized T2I models without requiring model-specific tuning and MotionLoRA is a lightweight fine-tuning technique for AnimateDiff that enables a pre-trained motion module to adapt to new motion patterns, such as different shot types, at a low training and data collection cost.
- Hybrid Task Cascade for Instance Segmentation1,565
This work proposes a new framework, Hybrid Task Cascade (HTC), which differs in two important aspects: (1) instead of performing cascaded refinement on these two tasks separately, it interweaves them for a joint multi-stage processing; (2) it adopts a fully convolutional branch to provide spatial context, which can help distinguishing hard foreground from cluttered background.
- 1,456
- PSANet: Point-wise Spatial Attention Network for Scene Parsing1,191
The point-wise spatial attention network (PSANet) is proposed to relax the local neighborhood constraint and achieves top performance on various competitive scene parsing datasets, including ADE20K, PASCAL VOC 2012 and Cityscapes, demonstrating its effectiveness and generality.
- Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering1,011
This work introduces Scaffold-GS, which uses an-chor points to distribute local 3D Gaussians, and predicts their attributes on-the-fly based on viewing direction and distance within the view frustum, and effectively reduces redun-dant Gaussians while delivering high-quality rendering.
- CARAFE: Content-Aware ReAssembly of FEatures1,010
Comprehensive evaluations on standard benchmarks in object detection, instance/semantic segmentation and inpainting show CARAFE shows consistent and substantial gains across all the tasks, and has great potential to serve as a strong building block for future research.
- Temporal Action Detection with Structured Segment Networks992
This paper addresses an important and challenging task, namely detecting the temporal intervals of actions in untrimmed videos by presenting a framework called structured segment network (SSN), built on temporal proposals of actions.
- Temporal Segment Networks for Action Recognition in Videos989
The proposed TSN framework, called temporal segment network (TSN), aims to model long-range temporal structure with a new segment-based sampling and aggregation scheme and won the video classification track at the ActivityNet challenge 2016 among 24 teams.
- FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection854
The solution achieves 1st place out of all the vision-only methods in the nuScenes 3D detection challenge of NeurIPS 2020 and proposes a general framework FCOS3D, getting rid of any 2D detection or 2D-3D correspondence priors.
- Revisiting Skeleton-based Action Recognition831
PoseConv3D is more effective in learning spatiotemporal features, more robust against pose estimation noises, and generalizes better in cross-dataset settings, and achieves the state-of-the-art on all eight multi-modality action recognition benchmarks.
- Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation756
A new framework for the outdoor LiDAR segmentation is proposed, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties.
- Region Proposal by Guided Anchoring685
This paper presents an alternative scheme, named Guided Anchoring, which leverages semantic features to guide the anchoring, and jointly predicts the locations where the center of objects of interest are likely to exist as well as the scales and aspect ratios at different locations.
- 668
- OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation628
OPERA is presented, a novel MLLM decoding method grounded in an Over-trust Penalty and a Retrospection-Allocation strategy, serving as a nearly free lunch to alleviate the hallucination issue without additional data, knowledge, or training.
- VLMEvalKit: An Open-Source ToolKit for Evaluating Large Multi-Modality Models628
The VLMEvalKit toolkit aims to provide a user-friendly and comprehensive framework for researchers and developers to evaluate existing multi-modality models and publish reproducible evaluation results.
- Visual-RFT: Visual Reinforcement Fine-Tuning575
This work introduces Visual Reinforcement Fine-Tuning (Visual-RFT), which further extends the application areas of RFT on visual tasks and represents a paradigm shift in fine-tuning LVLMs, offering a data-efficient, rewarddriven approach that enhances reasoning and adaptability for domain-specific tasks.
- UntrimmedNets for Weakly Supervised Action Recognition and Detection546
This paper presents a new weakly supervised architecture, called UntrimmedNet, which is able to directly learn action recognition models from untrimmed videos without the requirement of temporal annotations of action instances.
- Detecting Visual Relationships with Deep Relational Networks529
The proposed Deep Relational Network is a novel formulation designed specifically for exploiting the statistical dependencies between objects and their relationships and achieves substantial improvement over state-of-the-art on two large data sets.
- Towards Diverse and Natural Image Descriptions via a Conditional GAN482
A new framework based on Conditional Generative Adversarial Networks (CGAN) is proposed, which jointly learns a generator to produce descriptions conditioned on images and an evaluator to assess how well a description fits the visual content.
- OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation469
OmniObject3D, a large vocabulary 3D object dataset with massive high-quality real-scanned 3D objects, is proposed, facilitating the development of 3D perception, reconstruction, and generation in the real world.
- FineGym: A Hierarchical Video Dataset for Fine-Grained Action Understanding467
FineGym is a new dataset built on top of gymnasium videos that provides temporal annotations at both action and sub-action levels with a three-level semantic hierarchy and systematically investigates different methods on this dataset and obtains a number of interesting findings.
- Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation458
This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images and allows the generator to be fine-tuned on-the-fly in a progressive manner regularized by feature distance obtained by the discriminator in GAN.
- InternLM2 Technical Report440
InternLM2 is introduced, an open-source LLM that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks, long-context modeling, and open-ended subjective evaluations through innovative pre-training and optimization techniques.
- 430
- InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model397
Experimental results demonstrate the superiority of InternLM-XComposer2 based on InternLM2-7B in producing high-quality long-text multi-modal content and its exceptional vision-language understanding performance across various benchmarks, where it not only significantly outperforms existing multimodal models but also matches or even surpasses GPT-4V and Gemini Pro in certain assessments.
- 386
- Molecular recognition using corona phase complexes made of synthetic polymers adsorbed on carbon nanotubes366
Two common microscopy techniques are combined into a single instrument that has the capacity to study both nanoparticles and biological molecules simultaneously with spatial and temporal resolution appropriate for nanoscale interactions, which will enable the development of early-onset diagnostic tools at the level of a single cell.
- InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition361
This work proposes InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition that achieves competitive text-image composition scores compared to public solutions, including GPT4-V and GPT3.5.
- Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models349
This study serves as a clarion call for a collective effort to overhaul and fortify the safety of open-source LLMs against malicious attackers.
- MinerU: An Open-Source Solution for Precise Document Content Extraction348
MinerU leverages the sophisticated PDF-Extract-Kit models to extract content from diverse documents effectively and employs finely-tuned preprocessing and postprocessing rules to ensure the accuracy of the final results.
- MovieNet: A Holistic Dataset for Movie Understanding341
MovieNet is the largest dataset with richest annotations for comprehensive movie understanding and it is believed that such a holistic dataset would promote the researches on story-based long video understanding and beyond.
- BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering333
BungeeNeRF is introduced, a progressive neural radiance field that achieves level-of-detail rendering across drastically varied scales and its support for high-quality rendering in different levels of detail is demonstrated.
- PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction332
This work proposes PyramidDrop, a visual redundancy reduction strategy for LVLMs to boost their efficiency in both training and inference with neglectable performance loss, and could also serve as a plug-and-play strategy for inference acceleration without training, with better performance and lower inference cost than counterparts.
- Seesaw Loss for Long-Tailed Instance Segmentation314
This work proposes Seesaw Loss to dynamically re-balance gradients of positive and negative samples for each category, with two complementary factors, i.e., mitigation factor and compensation factor, which obtains significant gains over Cross-Entropy Loss and achieves state-of-the-art performance on LVIS dataset without bells and whistles.
- Distribution-Balanced Loss for Multi-label Classification in Long-Tailed Datasets307
The Distribution-Balanced Loss tackles issues through two key modifications to the standard binary cross-entropy loss: a new way to re-balance the weights that takes into account the impact caused by label co-occurrence, and a negative tolerant regularization to mitigate the over-suppression of negative labels.
- 305
- Scene as Occupancy297
OccNet is proposed, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy embedding to represent 3D physical world and there are evident performance gain across multiple tasks, demonstrating the superiority of this method.
- Be Your Own Prada: Fashion Synthesis with Structural Coherence297
A novel and effective approach for generating new clothing on a wearer through generative adversarial learning, given an input image of a person and a sentence describing a different outfit, which “redresses” the person as desired, while keeping the wearer and her/his pose unchanged.
- AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views287
This work introduces AnySplat, a feed-forward network for novel-view synthesis from uncalibrated image collections that greatly reduces rendering latency compared to optimization-based neural fields, bringing real-time novel-view synthesis within reach for unconstrained capture settings.
- OneLLM: One Framework to Align All Modalities with Language286
This paper presents OneLLM, an MLLM that aligns eight modalities to language using a unified framework through a unified multimodal encoder and a progressive multimodal alignment pipeline.
- PolyNet: A Pursuit of Structural Diversity in Very Deep Networks279
This work presents a new family of modules, namely the PolyInception, which can be flexibly inserted in isolation or in a composition as replacements of different parts of a network, and demonstrates substantial improvements over the state-of-the-art on the ILSVRC 2012 benchmark.
- 278
- SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction267
This paper proposes a random-mask video diffusion model to automatically generate transitions based on textual descriptions that generates transition videos that ensure coherence and visual quality and validate the effectiveness of the approach over existing methods for generative transition and prediction.
- Motion Guided 3D Pose Estimation from Videos256
A new graph convolutional network architecture, U-shaped GCN (UGCN), which captures both short-term and long-term motion information to fully leverage the additional supervision from the motion loss.
- Delving Into the Devils of Bird’s-Eye-View Perception: A Review, Evaluation and Recipe252
A full suite of practical guidebook to improve the performance of BEV perception tasks, including camera, LiDAR and fusion inputs is introduced, and the future research directions in this area are pointed out.
- PYSKL: Towards Good Practices for Skeleton Action Recognition247
PYSKL implements six different algorithms under a unified framework with both the latest and original good practices to ease the comparison of efficacy and efficiency and provides an original GCN-based skeleton action recognition model named ST-GCN++, which achieves competitive recognition performance without any complicated attention schemes.
- Conditional Infomax Learning: An Integrated Framework for Feature Extraction and Fusion247
A new framework for feature learning in classification motivated by information theory is introduced where a novel concept called class-relevant redundancy is introduced and a new algorithm called Conditional Informative Feature Extraction is formulated, which maximizes the jointclass-relevant information by explicitly reducing the class- relevant redundancies among features.
- MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyond246
A large-scale, comprehensive, and high-quality synthetic dataset for city-scale neural rendering researches is built, and a thorough benchmark is conducted, which not only reveals unique challenges of the task of city-scale neural rendering, but also highlights potential improvements for future works.
- Prime Sample Attention in Object Detection243
The notion of Prime Samples, those that play a key role in driving the detection performance are proposed, and a simple yet effective sampling and learning strategy called PrIme Sample Attention (PISA) is developed that directs the focus of the training process towards such samples.
- 230
- 225
- Alpha-CLIP: A CLIP Model Focusing on Wherever you Want223
Alpha-CLIp, an enhanced version of CLIP with an auxiliary alpha channel to suggest attentive regions and fine-tuned with constructed millions of RGBA region-text pairs, is introduced, which not only preserves the visual recognition ability of CLIP but also enables precise control over the emphasis of image contents.
- Lifelong Learning via Progressive Distillation and Retrospection219
A novel approach to lifelong learning is proposed, which tries to seek a better balance between preservation and adaptation via two techniques: Distillation and Retrospection.
- InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output216
A versatile large-vision language model that supports long-contextual input and output, InternLM-XComposer-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend.
- Inter-modality face recognition211
A novel algorithm called Common Discriminant Feature Extraction specially tailored to the inter-modality face recognition problem is proposed and two nonlinear extensions of the algorithm are developed: one is based on kernelization, while the other is a multi-mode framework.
- Single Molecule Detection of Nitric Oxide Enabled by d(AT)15DNA Adsorbed to Near Infrared Fluorescent Single-Walled Carbon Nanotubes209
The ability to detect nitric oxide quantitatively at the single-molecule level may find applications in new cellular assays for the study of nitricoxide carcinogenesis and chemical signaling, as well as medical diagnostics for inflammation.
- EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI206
This work introduces EmbodiedScan, a multi-modal, ego-centric 3D perception dataset and benchmark for holistic 3D scene understanding, and a baseline framework named Embodied Perceptron, capable of processing an arbitrary number of multi-modal inputs and demonstrating remarkable 3D perception capabilities.
- GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation206
An automatic, ver-satile, and human-aligned evaluation metric for text-to-3D generative models is presented and experimental results suggest it strongly aligns with human preference across different evaluation criteria.
- MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence204
This work introduces MMSI-Bench, a VQA benchmark dedicated to multi-image spatial intelligence, and provides an automated error analysis pipeline that diagnoses four dominant failure modes, including grounding errors, overlap-matching and scene-reconstruction errors, and spatial-logic errors, offering insights for advancing spatial intelligence.
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation204
By closing the loop between vision and action, the end-to-end PIDM can be a better scalable action learner, and is initially pre-trained on large-scale robotic datasets, such as DROID, and can be adapted to realworld scenarios with a little fine-tuning data.
- Nonparametric Discriminant Analysis for Face Recognition201
A dual NFA-based multi-classifier fusion framework by employing the over complete Gabor representation to boost the recognition performance and shows the improvements of the developed new algorithms over the traditional subspace methods.
- Characterization of Large Language Model Development in the Datacenter200
An in-depth characterization study of a six-month LLM development workload trace collected from the authors' GPU datacenter Acme finds discrepancies between LLMs and prior task-specific Deep Learning (DL) workloads, explore resource utilization patterns, and identify the impact of various job failures.
- Self-Supervised Scene De-Occlusion194
This paper makes the first attempt to address the problem of scene de-occlusion through a novel and unified framework that recovers hidden scene structures without ordering and amodal annotations as supervisions, via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
- 187
- Low-Latency Video Semantic Segmentation181
A framework for video semantic segmentation is developed, which incorporates two novel components: a feature propagation module that adaptively fuses features over time via spatially variant convolution, thus reducing the cost of per-frame computation and an adaptive scheduler that dynamically allocate computation based on accuracy prediction.
- When NAS Meets Robustness: In Search of Robust Architectures Against Adversarial Attacks180
This work takes an architectural perspective and investigates the patterns of network architectures that are resilient to adversarial attacks, and discovers a family of robust architectures (RobNets) that exhibit superior robustness performance to other widely used architectures.
- SpotServe: Serving Generative Large Language Models on Preemptible Instances178
SpotServe is presented, the first distributed LLM serving system on preemptible instances and stateful inference recovery is introduced, a new inference mechanism that commits inference progress at a much finer granularity and allows SpotServe to cheaply resume inference upon preemption.
- 176
- 175
- SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition167
A new end-to-end scene text spotting framework termed SwinTextSpotter is proposed, using a transformer encoder with dynamic head as the detector and a novel Recognition Conversion mechanism to explicitly guide text localization through recognition loss.
- Side-Aware Boundary Localization for More Precise Object Detection167
This paper proposes an alternative approach, named as Side-Aware Boundary Localization (SABL), where each side of the bounding box is respectively localized with a dedicated network branch, to tackle the difficulty of precise localization in the presence of displacements with large variance.
- Grid-guided Neural Radiance Fields for Large Urban Scenes163
A light-weighted NeRF is sufficient, under the guidance of the feature grid representation, to render photorealistic novel views with fine details; and the jointly optimized ground feature planes, can meanwhile gain further refinements, forming a more accurate and compact feature space and output much more natural rendering results.
- Pose Guided Human Video Generation163
A pose guided method to synthesize human videos in a disentangled way: plausible motion prediction and coherent appearance generation and enforce semantic consistency between the generated and ground-truth poses at a high feature level.
- 163
- 161
- VideoBooth: Diffusion-based Video Generation with Image Prompts159
This paper proposes a feed-forward framework VideoBooth, a generalizable framework where a single model works for a wide range of image prompts with only feed-forward passes, and proposes to embed image prompts in a coarse-to-fine manner.
- Scaling Laws of RoPE-based Extrapolation147
This work proposes a unified framework from the periodic perspective, to describe the relationship between the extrapolation performance and base value as well as tuning context length, and achieves extrapolation up to 1 million context length within only 16K training length on LLaMA2 7B and 13B.
- DSNAS: Direct Neural Architecture Search Without Parameter Retraining147
DSNAS is proposed, an efficient differentiable NAS framework that simultaneously optimizes architecture and parameters with a low-biased Monte Carlo estimate and successfully discovers networks with comparable accuracy on ImageNet.
- Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction146
Experiments prove that Dispider outperforms existing methods not only in its superior understanding of video content in conventional video QA settings, but also in proactive response capability and temporal awareness under the streaming setting.
- Open Compound Domain Adaptation146
This work proposes a new approach based on two technical insights into OCDA: a curriculum domain adaptation strategy to bootstrap generalization across domains in a data-driven self-organizing fashion and a memory module to increase the model's agility towards novel domains.
- Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-Based Perception145
This paper proposes a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties.
- Accelerating Diffusion Models via Early Stop of the Diffusion Process142
This work proposes a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), which obtains promising acceleration effect and performance improvement over representative baseline methods and demonstrates several attractive properties, including being orthogonal to existing acceleration methods, as well as simultaneously enabling both global semantic and local pixel-level control in image generation.
- Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction142
Voxurf is presented, a voxel-based surface reconstruction approach that is both efficient and accurate and achieves higher reconstruction quality with a 20x training speedup compared to previous fully implicit methods.
- SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds142
A novel 3D shape signature is proposed to explore the shape information from point clouds and is not only compact and effective but also robust to the noise, which serves as a soft constraint to improve the feature capability of multi-class discrimination.
- Learning to Cluster Faces on an Affinity Graph141
This work explores a novel approach, namely, learning to cluster instead of relying on hand-crafted criteria, and proposes a framework based on graph convolutional network, which combines a detection and a segmentation module to pinpoint face clusters.
- A Local-to-Global Approach to Multi-Modal Movie Scene Segmentation138
This work builds a large-scale video dataset MovieScenes, which contains 21K annotated scene segments from 150 movies, and proposes a local-to-global scene segmentation framework, which integrates multi-modal information across three levels, i.e. clip, segment, and movie.
- InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning137
This paper introduces and open-source their math reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2 and unify chain-of-thought reasoning, reward modeling, formal reasoning, data augmentation, and code interpreter in a unified seq2seq format and supervise the authors' model to be a versatile math reasoner, verifier, prover, and augmenter.
- Unified Human-Scene Interaction via Prompted Chain-of-Contacts135
A unified HSI framework, UniHSI, which supports unified control of diverse interactions through language commands, built upon the definition of interaction as Chain of Contacts (CoC): steps of human joint-object part pairs, which is inspired by the strong correlation between interaction types and human-object contact regions.
- VR-NeRF: High-Fidelity Virtualized Walkable Spaces132
This work designed and built a custom multi-camera rig to densely capture walkable spaces in high fidelity and with multi-view high dynamic range images in unprecedented quality and density, and extends instant neural graphics primitives with a novel perceptual color space for learning accurate HDR appearance.
- Guided Diffusion Model for Adversarial Purification132
The proposed GDMP is shown to reduce the perturbations raised by adversarial attacks to a shallow range, thereby significantly improving the correctness of classification and achieving 70.94% robustness on the challenging ImageNet dataset.
- 132
- DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering131
This work presents DNA-Rendering, a large-scale, high-fidelity repository of human performance data for neural actor rendering, and provides a large-scale and quantitative benchmark in full-scale, with multiple tasks to evaluate the existing progress of novel view synthesis, novel pose animation synthesis, and novel identity rendering methods.
- 131
- HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting127
An efficient yet effective framework that generates high-quality 3D humans with fine-grained geometry and realistic appearance, based on the insight that 3D Gaussian Splatting is an efficient renderer with peri-odic Gaussian shrinkage or growing, where such adaptive density control can be naturally guided by intrinsic human structures.
- 127
- Temporal ROI Align for Video Object Recognition125
A novel Temporal ROI Align operator is proposed to extract features from other frames feature maps for current frame proposals by utilizing feature similarity, and can extract temporal information from the entire video for proposals.
- Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis123
This paper decomposes the diversity of scene-aware human motions into three aspects, namely interaction diversity, path diversity and motion diversity, and proposes a hierarchical framework, with each sub-module responsible for modeling one aspect.
- Vision Transformer with Progressive Sampling121
An iterative and progressive sampling strategy to locate discriminative regions and when combined with the Vision Transformer, the obtained PS-ViT network can adaptively learn where to look.
- Long Context Tuning for Video Generation119
This work introduces Long Context Tuning (LCT), a training paradigm that expands the context window of pre-trained single-shot video diffusion models to learn scene-level consistency directly from data, enabling both joint and auto-regressive shot generation without additional parameters.
- Move Forward and Tell: A Progressive Generator of Video Descriptions119
This work proposes a new approach, which produces a descriptive paragraph by assembling temporally localized descriptions by selecting a sequence of distinctive clips and generating sentences thereon in a coherent manner given a video.
- 3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion118
3DTopia-XL is introduced, a scalable native 3D generative model designed to overcome limitations of existing methods in generating high-quality 3D assets with fine-grained textures and materials, efficiently bridging the quality gap between generative models and real-world applications.
- Optimizing Video Object Detection via a Scale-Time Lattice117
This paper presents a unified framework that integrates detection, temporal propagation, and across-scale refinement on a Scale-Time Lattice and develops an adaptive scheme with the detector invoked on demand and thus obtain improved tradeoff.
- Adversarial Robustness under Long-Tailed Distribution114
The negative impacts induced by imbalanced data on both recognition performance and adversarial robustness are revealed and a clean yet effective framework, RoBal, is proposed, which consists of two dedicated modules, a scale-invariant classifier and data re-balancing via both margin engineering at training stage and boundary adjustment during inference.
- T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step112
T-Eval provides a more fine-grained analysis of the capabilities of LLMs, providing a new perspective in LLM evaluation on tool-utilization ability, and comprehensively decompose the tool utilization into multiple sub- processes, including instruction following, planning, reasoning, retrieval, understanding, and review.
- 111
- 110
- 109
- 108
- Omni-Sourced Webly-Supervised Learning for Video Recognition107
Experiments show that by utilizing data from multiple sources and formats, OmniSource is more data-efficient in training, and several good practices, including data balancing, resampling, and cross-dataset mixup are adopted in joint training.
- Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation107
This work proposes a novel loss function, i.e., Conservative Loss, which penalizes the extreme good and bad cases while encouraging the moderate examples and enables the network to learn features that are discriminative by gradient descent and are invariant to the change of domains via gradient ascend method.
- V3Det: Vast Vocabulary Visual Detection Dataset104
V3Det, a vast vocabulary visual detection dataset with precisely annotated bounding boxes on massive images, has the potential to serve as a cornerstone dataset for developing more general visual perception systems.
- Scene-aware Generative Network for Human Motion Synthesis103
This paper proposes a new framework, which factorizes the distribution of human motions into a distribution of movement trajectories conditioned on scenes and that of body pose dynamics conditioned on both scenes and trajectories, and derives a GAN-based learning approach.
- Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation103
This work integrates both stereo generation and source separation into a unified framework, Sep-Stereo, by considering source separation as a particular type of audio spatialization, and proposes a novel associative pyramid network architecture carefully designed for audio-visual feature fusion.
- 101
- A Unified Framework for Shot Type Classification Based on Subject Centric Lens100
A learning framework Subject Guidance Network (SGNet) for shot type recognition is proposed, which separates the subject and background of a shot into two streams, serving as separate guidance maps for scale and movement type classification respectively.
- Learning to Cluster Faces via Confidence and Connectivity Estimation100
This paper proposes a fully learnable clustering framework without requiring a large number of overlapped subgraphs, and transforms the clustering problem into two sub-problems, designed to estimate the confidence of vertices and the connectivity of edges, respectively.
- SPTS v2: Single-Point Scene Text Spotting99
The SPTS v2 can outperform previous state-of-the-art single-point text spotters with fewer parameters while achieving 19× faster inference speed and experiments suggest a potential preference for single- point representation in scene text spotting when compared to other representations.
- Construction of Dependent Dirichlet Processes based on Poisson Processes98
A novel method that exploits the intrinsic relationship between Dirichlet and Poisson processes in order to create a Markov chain of Dirichlets suitable for use as a prior over evolving mixture models is presented.
- GPT4Point: A Unified Framework for Point-Language Understanding and Generation96
GPT4Point, an innovative, groundbreaking point-language multimodal model explicitly designed for unified 3D object understanding and generation within the MLLMframework, is introduced and Pyramid-XL, a point-language dataset annotation engine, is developed to support the expansive needs of 3D object-text pairs.
- LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K96
LV-Eval is introduced, a challenging long-context benchmark with five length levels (16k, 32k, 64k, 128k, and 256k) reaching up to 256k words, and the design has incorporated three key techniques, namely confusing facts insertion, keyword and phrase replacement, and keyword-recall-based metric design.
- MMOCR95
MMOCR is an open-source toolbox which provides a comprehensive pipeline for text detection and recognition, as well as their downstream tasks such as named entity recognition and key information extraction.
- Online Hyper-Parameter Learning for Auto-Augmentation Strategy95
This paper proposes Online Hyper-parameter Learning for Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the augmentation policy distribution along with network training that eliminates the need of re-training and dramatically reduces the cost of the overall search process.
- Recursive Visual Sound Separation Using Minus-Plus Net95
This paper proposes a novel framework, referred to as MinusPlus Network (MP-Net), for the task of visual sound separation, which obtains state-of-the-art results on two large scale datasets, across mixtures with different types and numbers of sounds.
- Trajectory Convolution for Action Recognition95
This work proposes a new CNN architecture TrajectoryNet, which incorporates trajectory convolution, a new operation for integrating features along the temporal dimension, to replace the existing temporal convolution.
- 93
- OriGen: Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection91
OriGen is a fully open-source framework that incorporates self-reflection capabilities and a novel dataset augmentation methodology for generating high-quality, large-scale RTL code, and exhibits superior capabilities in self-reflection and error correction.
- 3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors91
A two-stage text-to-3D generation system, namely 3DTopia, which generates high-quality general 3D assets within 5 minutes using hybrid diffusion priors and clean and caption the largest open-source 3D dataset, Objaverse, by combining the power of vision language models and large language models.
- CARAFE++: Unified Content-Aware ReAssembly of FEatures91
This work proposes unified Content-Aware ReAssembly of FEatures (CARAFE++), a universal, lightweight, and highly effective operator to fulfill this goal and shows great potential to serve as a strong building block for modern deep networks.
- Intra- and Inter-Action Understanding via Temporal Action Parsing91
This study shows that a sport activity usually consists of multiple sub-actions and that the awareness of such temporal structures is beneficial to action recognition, and investigates a number of temporal parsing methods, and devise an improved method that is capable of mining sub- actions from training data without knowing the labels of them.
- 90
- 90
- InternLM2.5-StepProver: Advancing Automated Theorem Proving via Critic-Guided Search88
This work proposes to use large-scale LEAN problem datasets Lean-workbook for expert iteration with more than 20,000 CPU days and finds log-linear trends between solved problem amount with proof length and CPU usage.
- Scaling Spatial Intelligence with Multimodal Foundation Models85
This work explores scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models and unified understanding and generation models (i.e., Bagel).
- HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion84
This work proposes a unified framework, HyperHuman, that generates in-the-wild human images of high realism and diverse layouts, and enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network.
- SPTS: Single-Point Text Spotting84
This work demonstrates that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance, and shows that the performance is not very sensitive to the positions of the point annotation, meaning that it can be much easier to be annotated or even be automatically generated than the bounding box that requires precise positions.
- 83
- SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling83
A new synthetic dataset, SynBody, with three appealing features: a clothed parametric human model that can generate a diverse range of subjects; the layered human representation that naturally offers high-quality 3D annotations to support multiple tasks; and a scalable system for producing realistic data to facilitate real-world tasks.
- Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition83
It is shown that unlabeled face data can be as effective as the labeled ones, and Consensus-Driven Propagation (CDP) is proposed to tackle this challenging problem with two modules, the “committee” and the ”mediator”, which select positive face pairs robustly by carefully aggregating multi-view information.
- Find and Focus: Retrieve and Localize Video Events with Natural Language Queries83
This work proposes a new framework called Find and Focus (FIFO), which not only performs top-level matching (paragraph vs. video), but also makes part-level associations, localizing a video clip for each sentence in the query with the help of a focusing guide.
- 83
- From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models81
A new open-vocabulary SGG framework based on sequence generation that leverages vision-language pre-trained models (VLM) by incorporating an image-to-graph generation paradigm and seamlessly integrate explicit relational modeling for enhancing the VL tasks.
- Visually Informed Binaural Audio Generation without Binaural Audios81
This work proposes PseudoBinaural, an effective pipeline that is free of binaural recordings, and shows great stability in cross-dataset evaluation and achieves comparable performance under subjective preference.
- Learning visual flows: A Lie algebraic approach81
A Lie algebraic representation of the transform process is introduced, which maps the transformation group to a vector space, and thus overcomes the difficulties due to the group structure, and therefore the statistical learning techniques based on vector spaces can be readily applied.
- LEGION: Learning to Ground and Explain for Synthetic Image Detection78
This paper introduces SynthScars, a high-quality and diverse dataset consisting of 12,236 fully synthetic images with human-expert annotations and proposes LEGION, a multimodal large language model (MLLM)-based image forgery analysis framework that integrates artifact detection, segmentation, and explanation.
- 3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation76
3DTrajMaster is introduced, a robust controller that regulates multi-entity dynamics in 3D space, given user-desired 6DoF pose (location and rotation) sequences of entities and sets a new state-of-the-art in both accuracy and generalization for controlling multi-entity 3D motions.
- VEnhancer: Generative Space-Time Enhancement for Video Generation74
A generative space-time enhancement framework that improves the existing text-to-video results by adding more details in spatial domain and synthetic detailed motion in temporal domain, and surpasses existing state-of-the-art video super-resolution and space-time super-resolution methods in enhancing AI-generated videos.
- 74
- Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation72
A Bayesian learning algorithm for DP mixture models that recursively transforms a given prior into an approximate posterior through sequential variational approximation, making it particularly suited for applications with massive data.
- VideoRoPE: What Makes for Good Video Rotary Position Embedding?71
VideoRoPE is introduced, with a 3D structure designed to preserve spatio-temporal relationships, which consistently surpasses previous RoPE variants, across diverse downstream tasks such as long video retrieval, video understanding, and video hallucination.
- 71
- VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Grounding71
Experiments show VLM-Grounder outperforms previous zero-shot methods, achieving 51.6% Acc@0.25 on ScanRefer and 48.0% Acc on Nr3D, without relying on 3D geometry or object priors.
- DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition71
A new framework for skeleton-based action recognition, namely Dynamic Group Spatio-Temporal GCN (DG-STGCN), which consists of two modules, DG-GCN and DG-TCN, respectively, for spatial and temporal modeling.
- Fastened CROWN: Tightened Neural Network Robustness Certificates71
This work demonstrates the optimality of deterministic CROWN (Zhang et al. 2018) solutions in a given linear programming problem under mild constraints and proposes an optimization-based approach FROWN (Fastened CROWN): a general algorithm to tighten robustness certificates for neural networks.
- A Graph-Based Framework to Bridge Movies and Synopses71
A dataset called Movie Synopses Associations (MSA) is constructed, which provides a synopsis for each movie, together with annotated associations between synopsis paragraphs and movie segments, and a framework to perform matching between movie segments and synopsis paragraphs is developed.
- Towards Text-guided 3D Scene Composition70
SceneWiz3D is introduced - a novel approach to synthesize high-fidelity 3D scenes from text by introducing a hybrid 3D representation - explicit for objects and implicit for scenes, and achieves superior quality over previous approaches.
- 69
- LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation68
LayerPano3D, a novel framework for full-view, explorable panoramic 3D scene generation from a single text prompt, pioneer the Layered 3D Panorama as underlying representation to manage complex scene hierarchies and lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes with unconstrained viewing paths.
- Person Search in Videos with One Portrait Through Visual and Temporal Links68
A novel framework is proposed, which takes into account the identity invariance along a tracklet, thus allowing person identities to be propagated via both the visual and the temporal links and remarkably outperforms mainstream person re-id methods.
- Efficient training of large language models on distributed infrastructures: a survey65
This survey explores recent advancements in training systems for LLMs, including innovations in training infrastructure with AI accelerators, networking, storage, and scheduling, as well as optimizations for computation, communication, and memory in distributed LLM training.
- 65
- 65
- 62
- Joint People, Event, and Location Recognition in Personal Photo Collections Using Cross-Domain Context62
This work develops a unified approach to jointly tag across multiple domains (specifically people, events, and locations) using a generic probabilistic model of context that couples the domains through a set of cross-domain relations.
- SIM-CoT: Supervised Implicit Chain-of-Thought61
SIM-CoT significantly improves both in-domain accuracy and out-of-domain stability of implicit CoT methods, boosting Coconut by +8.2\% on GPT-2 and CODI by +3.0\% on LLaMA-3.1 8B.
- Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models61
This work presents Vchitect-2.0, a parallel transformer architecture designed to scale up video diffusion models for large-scale text-to-video generation, and proposes a Memory-efficient Training framework that incorporates hybrid parallelism and other memory reduction techniques, enabling efficient training of long video sequences on distributed systems.
- OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text60
This paper introduces OmniCorpus, a 10 billion-scale image-text interleaved dataset that has 15 times larger scales while maintaining good data quality and is more flexible, easily degradable from an image-text interleaved format to pure text corpus and image-text pairs.
- 60
- Generative Adversarial Frontal View to Bird View Synthesis60
The BridgeGAN, a novel generative model for bird view synthesis, is proposed, which is much better than those generated by existing methods, with more consistent global appearance and sharper details.
- 60
- 59
- LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models58
This work introduces LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities, and evaluated 22 open-source LMMs and 6 closed-source models on it, highlighting their potential as synthetic data detectors and also revealing some limitations in the development of LMM capabilities.
- 57
- RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars57
This paper presents RenderMe-360, a comprehensive 4D human head dataset to drive advance in head avatar research, with 16 state-of-the-art methods performed on five main tasks: novel view synthesis, novel expression synthesis, hair rendering, hair editing, and talking head generation.
- Controllable Mesh Generation Through Sparse Latent Point Diffusion Models56
A novel sparse latent point diffusion model is designed for mesh generation, to regard point clouds as an intermediate representation of meshes, and model the distribution of point clouds instead.
- Self-Supervised Learning via Conditional Motion Propagation55
This work designs a new learning-from-motion paradigm that solves the inherent motion ambiguity, and encourages the emergence of kinematically-sound representations that poss greater expressive power, and is successfully extended to several useful applications such as semi-automatic pixel-level annotation.
- Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances54
Parcae is presented, a system that enables cheap, fast, and scalable DNN training on preemptible instances by proactively adjusting the parallelization strategy of a DNN training job to adapt to predicted resource changes before instance preemptions and allocations really happen, which significantly reduces the cost of handling these events.
- Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning53
A token-level reward model is applied to sample important tokens in reasoning trajectories for learning and theoretically proves that behavior cloning on positive trajectories from best-of-N (BoN) sampling is sufficient to learn the KL-regularized optimal policy in binary feedback environments.
- Joint Semantic-geometric Learning for Polygonal Building Segmentation53
This paper designs a multi-task segmentation network for joint semantic and geometric learning via three tasks, i.e., pixel-wise building segmentation, multi-class corner prediction, and edge orientation prediction and proposes a simple but effective vertex generation module for transforming the segmentation contour into high-quality polygon vertices.
- Flames: Benchmarking Value Alignment of LLMs in Chinese52
A value alignment benchmark named Flames, which encompasses both common harmlessness principles and a unique morality dimension that integrates specific Chinese values such as harmony is proposed, which has far exceeded existing benchmarks.
- InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions52
This project draws inspiration from the concept of the Specialized Generalist AI and introduces disentangled streaming perception, reasoning, and memory mechanisms, enabling real-time interaction with streaming video and audio input.
- SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention51
This paper proposes SampleAttention, an adaptive structured and near-lossless sparse attention, which can seamlessly replace vanilla attention in off-the-shelf LLMs with nearly no accuracy loss, and reduces TTFT by up to $2.42\times compared with FlashAttention.
- WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models51
A large-scale multimodal dataset composed of both Chinese and English data, collected from a wide range of web sources, utilized in the training of InternLM, a model that demonstrated significant advantages in multi-dimensional evaluations when compared to models of a similar scale.
- 51
- 50
- MM-IFEngine: Towards Multimodal Instruction Following49
The MMIFEngine pipeline is presented, an effective pipeline to generate high-quality image-instruction pairs and a comprehensive evaluation pipeline incorporating both rule-based assessment and judge model, and introduces MM-IFEval, a challenging and diverse multi-modal instruction-following benchmark.
- Improving Pixel-based MIM by Reducing Wasted Modeling Capability49
This paper is the first to systematically investigate multilevel feature fusion for isotropic architectures like the standard Vision Transformer (ViT) and is the first to systematically investigate multilevel feature fusion for isotropic architectures like the standard Vision Transformer (ViT).
- OpenDataLab: Empowering General Artificial Intelligence with Open Datasets47
OpenDataLab is introduced, a platform designed to bridge the gap between diverse data sources and the need for unified data processing, and will significantly boost artificial general intelligence (AGI) research and facilitate advancements in related AI fields.
- PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling47
A remarkably simple and effective method that entails filtering the high-frequency components from the reconstruction target to de-emphasize the network's focus on texture-rich details and adopting a conservative data transform strategy to alleviate the problem of missing foreground in MIM training is proposed.
- Accelerated Training for Massive Classification via Dynamic Class Selection47
This paper presents a new method that can efficiently and accurately identify a small number of "active classes" for each mini-batch, based on a set of dynamic class hierarchies constructed on the fly, which significantly reduces the training cost and memory demand, while maintaining competitive performance.
- Visual Agentic Reinforcement Fine-Tuning46
The findings suggest that Visual-ARFT offers a promising path toward building robust and generalizable multimodal agents.
- Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study45
DevBench is proposed, a comprehensive benchmark that evaluates LLMs across various stages of the software development lifecycle, including software design, environment setup, implementation, acceptance testing, and unit testing.
- Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning44
Spatial-SSRL is introduced, a self-supervised RL paradigm that derives verifiable signals directly from ordinary RGB or RGB-D images and provides a practical route to stronger spatial intelligence in LVLMs.
- SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models44
This paper presents a strategy called SKVQ, which stands for sliding-window KV cache quantization, to address the issue of extremely low bitwidth KV cache quantization, allowing for quantization of the KV cache to 2-bit keys and 1.5-bit values with minimal loss of accuracy.
- Force-Aware Interface via Electromyography for Natural VR/AR Interaction43
It is shown that lightweight electromyography sensors, resting non-invasively on users' forearm skin, inform and establish a robust understanding of their complex hand activities, and human perception of virtual objects' physical properties, such as stiffness, can be significantly enhanced by the interface.
- IRLAS: Inverse Reinforcement Learning for Architecture Search43
An inverse reinforcement learning method for architecture search (IRLAS), which trains an agent to learn to search network structures that are topologically inspired by human-designed network to extract the abstract topological knowledge of an expert human-design network (ResNeXt).
- MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training42
The Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training and a carefully designed data-efficient 3D object detection benchmark on the Waymo dataset is introduced.
- Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control40
RoboMaster is presented, a novel framework that models inter-object dynamics via a collaborative trajectory formulation and decomposes the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction, and models each phase using the dominant object.
- Position-Guided Point Cloud Panoptic Segmentation Transformer40
This work designs a robust Mixed-parameterized Positional Embedding to guide the segmentation process in LiDAR-based point cloud segmentation and outperforms previous state-of-the-art methods by 2.7% and 1.2% PQ on SemanticKITTI and nuScenes datasets, respectively.
- Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side Images40
A novel pipeline called Tailor3D is proposed, which swiftly creates customized 3D assets from editable dual-side images to emulate a tailor's ability to locally change objects or perform overall style transfer, and demonstrates its effectiveness across various 3D generation and editing tasks, including 3D generative fill and style transfer.
- 40
- Unifying Identification and Context Learning for Person Recognition40
A Region Attention Network is proposed, which is learned to adaptively combine visual cues with instance-dependent weights and a unified formulation is developed, where the social contexts are learned along with the reasoning of people identities.
- 40
- CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling38
- Horizon-Gs: Unified 3D Gaussian Splatting for Large-Scale Aerial-To-Ground Scenes38
Horizon-Gs, a novel approach built upon Gaussian Splatting techniques, tackles the unified reconstruction and rendering for aerial and street views, and addresses the key challenges of combining these perspectives with a new training strategy, overcoming viewpoint discrepancies to generate high-fidelity scenes.
- Learning Diverse Fashion Collocation by Neural Graph Filtering38
A novel fashion collocation framework, Neural Graph Filtering, that models a flexible set of fashion items via a graph neural network that significantly outperforms the state-of-the-art methods with over 10% improvements on the standard AUC metric on the established tasks.
- OmniCity: Omnipotent City Understanding with Multi-Level and Multi-View Images37
OmniCity contains multi-view satellite images as well as street-level panorama and mono-view images, constituting over 100K pixel-wise annotated images that are well-aligned and collected from 25K geo-locations in New York City.
- Modeling and estimating persistent motion with geometric flows36
This work first introduces the concept of geometric flow that describes motion simultaneously over space and time, and derive a vector space representation based on Lie algebra, which is extended to model complex motion by combining multiple flows in a geometrically consistent manner.
- 35
- Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future35
This comprehensive review, conducted in collaboration with esteemed colleagues from both academia and industry, systematically assesses over seventy open-source autonomous driving datasets from domestic and international sources and offers insights into various aspects, such as the principles underlying the creation of high-quality datasets.
- MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligence34
MMSI-Video-Bench is introduced, a fully human-annotated benchmark for video-based spatial intelligence in MLLMs, revealing a striking human--AI gap: many models perform near chance, and the best reasoning model lags humans by nearly 60%.
- MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design34
MxMoE, a mixed-precision optimization framework for MoE models that considers both algorithmic and system perspectives, and automatically generates optimized mixed-precision GroupGEMM kernels, enabling parallel execution of GEMMs with different precisions is introduced.
- Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production34
VDS is presented to allow users storyboarding shots in virtual environments, where the filming staff can easily test the settings of shots before the actual filming, demonstrating its efficiency, effectiveness, and great potential in assisting amateur video production.
- 34
- 33
- MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models33
Multi-Image Augmented Direct Preference Optimization (MIA-DPO) mitigates the scarcity of diverse multi-image training data by extending single-image data with unrelated images arranged in grid collages or pic-in-pic formats, significantly reducing the costs associated with multi-image data annotations.
- RIFormer: Keep Your Vision Backbone Effective But Removing Token Mixer33
This paper develops an RepIdentityFormer base on the re-parameterizing idea, and explores the improved learning paradigm to break the limitation of simple token mixer free backbone, and summarizes the empirical practice into 5 guidelines.
- OCSampler: Compressing Videos to One Clip with Single-step Sampling33
OCSampler designs a new paradigm of learning instance-specific video condensation policies to select frames only in a single step, and extends the proposed method with a frame number budget, enabling the framework to produce correct predictions in high confidence with as few frames as possible.
- SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning32
This work introduces SenseNova-MARS, a novel Multimodal Agentic Reasoning and Search framework that empowers VLMs with interleaved visual reasoning and tool-use capabilities via reinforcement learning (RL).
- RelightVid: Temporal-Consistent Diffusion Model for Video Relighting32
RelightVid is a flexible framework for video relighting that can accept background video, text prompts, or environment maps as relighting conditions and achieves arbitrary video relighting with high temporal consistency without intrinsic decomposition while preserving the illumination priors of its image backbone.
- RAR: Retrieving and Ranking Augmented MLLMs for Visual Recognition31
RAR, a Retrieving And Ranking augmented method for Multimodal Large Language Models, demonstrates a significant improvement in performance on 5 fine-grained visual recognition benchmarks, 11 few-shot image recognition datasets, and the 2 object detection datasets under the zero-shot recognition setting.
- Online Multi-modal Person Search in Videos31
This paper proposes an online person search framework, which can recognize people in a video on the fly and maintains a multimodal memory bank at its heart as the basis for person recognition, and updates it dynamically with a policy obtained by reinforcement learning.
- 31
- Deep Markov Random Field for Image Modeling31
A novel MRF model that uses fully-connected neurons to express the complex interactions among pixels is proposed that combines the expressive power of deep neural networks and the cyclic dependency structure of MRF in a unified model, bringing the modeling capability to a new level.
- 30
- InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint29
This work introduces a novel controllable motion generation method, InterControl, to encourage the synthesized motions maintaining the desired distance between joint pairs, and demonstrates that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model.
- Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases29
An in-depth comparative study of two pioneering models: Google's Gemini and OpenAI's GPT-4V, involving a multi-faceted evaluation of both models across key dimensions such as Vision-Language Capability, Interaction with Humans, Temporal Understanding, and assessments in both Intelligence and Emotional Quotients.
- 29
- Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback28
Linear Alignment is introduced, a novel algorithm that aligns language models with human preferences in one single inference step, eliminating the reliance on data annotation and model training.
- TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition28
TransRank, a unified framework for recognizing Transformations in a Ranking formulation that provides accurate supervision signals by recognizing transformations relatively, consistently outperforming the classification-based formulation, achieves competitive performance on video retrieval and action recognition.
- MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy27
This work proposes MathSmith, a novel framework for synthesizing challenging mathematical problems to enhance LLM reasoning, which exhibits strong scalability, generalization, and transferability, highlighting the promise of high-difficulty synthetic data in advancing LLM reasoning capabilities.
- HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image27
The proposed HyperDreamer holds promise for advancing 3D content creation and finding applications in various domains by modeling region-aware materials with high-resolution textures and enabling user-friendly editing.
- 27
- Learning H-Infinity Locomotion Control26
This paper proposes to model the robust locomotion learning process as an adversarial interaction between the locomotion policy and a learnable disturbance that is conditioned on the robot state to generate appropriate external forces, and imposes a novel constraint to make the joint optimization stable.
- Proteus: Simulating the Performance of Distributed DNN Training26
Proteus is presented, the first standalone simulator to model the performance of complex parallelization strategies through simulation execution and reduces prediction error by up to 133.8% compared to state-of-the-art approaches.
- Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos26
This work proposes a novel semantic-aware masked slot attention that effectively identifies multiple object instances with semantic structure, reaching promising results on unsupervised video object discovery and achieves state-of-the-art performance on dense label propagation tasks, demonstrating the potential for object-centric analysis.
- Towards Evaluating and Training Verifiably Robust Neural Networks26
A relaxed version of CROWN, linear bound propagation (LBP), is proposed that can be used to verify large networks to obtain lower verified errors than IBP, and a new activation function, parameterized ramp function (ParamRamp), which has more diversity of neuron status than ReLU.
- Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals25
Phased DMD is proposed, a multi-step distillation framework that bridges the idea of phase-wise distillation with Mixture-of-Experts (MoE), reducing learning difficulty while enhancing model capacity.
- 25
- 25
- 25
- SwinTextSpotter v2: Towards Better Synergy for Scene Text Spotting24
A new end-to-end scene text spotting framework termed SwinTextSpotter v2 is proposed, which seeks to find a better synergy between text detection and recognition and enhances the relationship between two tasks using novel Recognition Conversion and Recognition Alignment modules.
- VFLowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization24
VFlowOpt is proposed, a token pruning framework that introduces an importance map derivation process and a progressive pruning module with a recycling mechanism that minimizes the discrepancy between token representations in LMMs with and without pruning, thereby enabling superior pruning strategies tailored to different LMMs.
- 24
- IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations24
IDArb is introduced, a diffusion-based model designed to perform intrinsic decomposition on an arbitrary number of images under varying illuminations and outperforms state-of-the-art methods both qualitatively and quantitatively.
- 24
- 24
- Caption-Supervised Face Recognition: Training a State-of-the-Art Face Model Without Manual Annotation24
A simple yet effective method, which trains a state-of-the-art face recognition model by progressively expanding the labeled set via both selective propagation and caption-driven expansion, and shows the great potential of caption-supervised face recognition.
- Rethinking the Form of Latent States in Image Captioning24
This work visually reveals the internal dynamics in the process of caption generation, as well as the connections between input visual domain and output linguistic domain, using two-dimensional maps to encode latent states in RNNs.
- Creation-Mmbench: Assessing Context-Aware Creative Intelligence in Mllms23
CreationMMBench is introduced, a multimodal benchmark specifically designed to evaluate the creative capabilities of MLLMs in realworld, image-based tasks and demonstrates that visual fine-tuning can negatively impact the base LLM's creative abilities.
- 23
- GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography22
This work introduces an auto-regressive model inspired by the expertise of Directors of Photography to generate artistic and expressive camera trajectories based on text guidance and RGBD inputs, named GenDoP, and believes this approach establishes a new standard for learningbased cinematography.
- BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning22
BoostStep is a method that enhances reasoning accuracy through step-aligned ICL, a novel mechanism that carefully aligns retrieved reference steps with the corresponding reasoning steps, and enhances state-of-the-art LLMs to solve challenging math problems using simpler examples.
- Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate22
This paper presents the Modality Integration Rate (MIR), an effective, robust, and generalized metric to indicate the multi-modal pre-training quality of Large Vision Language Models (LVLMs), and proposes evaluating the pre-training quality from the inter-modal distribution distance perspective.
- Discover and Learn New Objects from Documentaries22
A joint probabilistic framework is developed, where individual pieces of information, including video frames and subtitles, are brought together via both visual and linguistic links, and a weakly supervised learning algorithm is derived, where object model learning and training set mining are unified in an optimization procedure.
- 21
- Generating Multi-sentence Natural Language Descriptions of Indoor Scenes21
This paper proposes a novel framework for generating lingual descriptions of indoor scenes, by generating coherent descriptions with multiple sentences, and develops a framework with three major components: a holistic visual parser that couples the inference of objects, attributes, and relations to produce a semantic representation of a 3D scene.
- RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation20
The RoboInter Manipulation Suite is introduced, a unified resource including data, benchmarks, and models of intermediate representations for manipulation including modular and end-to-end VLA variants that bridge high-level planning with low-level execution via intermediate supervision.
- Transcript to Video: Efficient Clip Sequencing from Texts19
This work presents Transcript-to-Video -- a weakly-supervised framework that uses texts as input to automatically create video sequences from an extensive collection of shots, with a Content Retrieval Module and a Temporal Coherent Module to learn visual-language representations and model shot sequencing styles.
- 19
- Coupling Nonparametric Mixtures via Latent Dirichlet Processes19
A new method is developed that jointly estimates mixture models over multiple data sets by exploiting the statistical dependencies between them, and derives a sampling algorithm that jointly infers the model parameters and present experiments on both document analysis and image modeling.
- 19
- A survey of low-bit large language models: Basics, systems, and algorithms18
A systematic overview from basic, system, and algorithm perspectives can offer valuable insights and guidelines for future works to enhance the efficiency and applicability of LLMs through low-bit quantization.
- DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models17
DualFocus's superiority in balancing detailed examination with holistic insight is demonstrated, significantly reducing hallucination instances in MLLMs and improving their performance in various vision-language tasks.
- 17
- OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value16
This work introduces OpenDataArena (ODA), a holistic and open platform designed to benchmark the intrinsic value of post-training data, and envisions a shift from trial-and-error data curation to a principled science of Data-Centric AI.
- SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition16
SongComposer is a music-specialized large language model (LLM) that integrates the capability of simultaneously composing lyrics and melodies into LLMs by leveraging three key innovations: a flexible tuple format for word-level alignment of lyrics and melodies, an extended tokenizer vocabulary for song notes, and a multi-stage pipeline that captures musical structure.
- PointLLM-V2: Empowering Large Language Models to Better Understand Point Clouds16
PointLLM is developed, significantly outperforming 2D and 3D baselines and achieving SOTA performance, with a notable achievement in object captioning tasks where it surpasses human annotators in over 50% of the samples.
- 16
- ReasonPix2Pix: Instruction Reasoning Dataset for Advanced Image Editing15
ReasonPix2Pix, a comprehensive reasoning-attentive instruction editing dataset, is introduced, characterized by 1) reasoning instruction, 2) more realistic images from fine-grained categories, and 3) increased variances between input and edited images.
- X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models15
This work introduces X-Prompt, a purely auto-regressive large-vision language model designed to deliver competitive performance across a wide range of both seen and unseen image generation tasks, all within a unified in-context learning framework.
- Parsing-Conditioned Anime Translation: A New Dataset and Method15
A new anime translation framework is designed by deriving the prior knowledge of a pre-trained StyleGAN model, and disentangled encoders are introduced to separately embed structure and appearance information into the same latent code, governed by four tailored losses.
- DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Tracking15
DORT extracts object-wise local volumes for motion estimation that also alleviates the heavy computational burden by iteratively refining the estimated object motion and location and the preceding features can be precisely aggregated to the current frame to mitigate the aforementioned adverse effects.
- 15
- 15
- 14
- 14
- ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing14
ScaleCap is presented, an inference-time scalable image captioning strategy that generates comprehensive and detailed image captions that continuously enriches and calibrates the caption with increased inference budget and employs sentence-level offline contrastive decoding to effectively identify and eliminate hallucinations caused by linguistic biases.
- Cinematic Behavior Transfer via NeRF-based Differentiable Filming14
A reverse filming behavior estimation technique that optimizes camera trajectories by leveraging NeRF as a differentiable renderer and refining SMPL tracks is introduced, and the incorporation of 3D engine workflow enables superior rendering and control abilities, which also achieves a higher rating in the user study.
- 14
- 14
- 14
- Learn to Propagate Reliably on Noisy Affinity Graphs14
A new framework that allows labels to be propagated reliably on large-scale real-world data is proposed that incorporates a local graph neural network to predict accurately on varying local structures while maintaining high scalability and a confidence-based path scheduler that identifies outliers and moves forward the propagation frontier in a prudent way.
- 14
- Virtualized 3D Gaussians: Flexible Cluster-based Level-of-Detail System for Real-Time Rendering of Composed Scenes13
Virtualized 3D Gaussians (V3DG), a cluster-based LOD solution that constructs hierarchical 3D Gaussian clusters and dynamically selects only the necessary ones to accelerate rendering speed, is proposed.
- Multi-Identity Human Image Animation with Structural Video Diffusion13
This work presents Structural Video Diffusion, a novel framework designed for generating realistic multi-human videos that combines identity-specific embeddings to maintain consistent appearances across individuals and a structural learning mechanism that incorporates depth and surface-normal cues to model human-object interactions.
- 13
- AssetField: Assets Mining and Reconfiguration in Ground Feature Plane Representation13
AssetField is proposed, a novel neural scene representation that learns a set of object-aware ground feature planes to represent the scene, where an asset library storing template feature patches can be constructed in an unsupervised manner.
- 12
- 12
- 12
- Proc-Gs: Procedural Building Generation for City Assembly with 3D Gaussians11
Drawing inspiration from procedural modeling techniques used in the gaming and visual effects industry, Proc-GS integrates procedural code into the 3D Gaussian Splatting framework, offering a flexible, efficient, and scalable framework for city assembly with high-fidelity and precise control.
- ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning11
ARM-Thinker is presented, an Agentic multimodal Reward Model that autonomously invokes external tools to ground judgments in verifiable evidence, replacing static, non-interactive reward scoring.
- Case2Code: Scalable Synthetic Data for Code Generation11
Experimental results show that such induction training benefits not only in distribution Case2Code performance but also enhances various coding abilities of trained LLMs, demonstrating the great potential of learning inductive reasoning.
- Temporal and Contextual Transformer for Multi-Camera Editing of TV Shows11
A new approach temporal and contextual transformer that utilizes clues from historical shots and other views to make shot transition decisions and predict which view to be used outperforms existing methods on the proposed multi-camera editing benchmark.
- 11
- 11
- 11
- 11
- Plenoptic Video Generation10
PlenopticDreamer is a framework that synchronizes generative hallucinations to maintain spatio-temporal memory and achieves state-of-the-art video re-rendering, delivering superior view synchronization, high-fidelity visuals, accurate camera control, and diverse view transformations.
- 10
- 10
- 10
- 9
- 9
- Efficient Sampling from Combinatorial Space via Bridging9
This work proposes an ecient method for sampling from combinatorial spaces that addresses issues via “bridging states” that facilitate the communication between dierent parts of the space, providing more flexibility than methods relying on specific space structures to design jump proposals.
- 9
- MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving8
The key insight behind is to colocate LLMs considering their popularity to multiplex memory resources, and leverage the characteristics of prefill and decoding phases to separate and flexibly colocate them to multiplex computation resources.
- HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE8
The first deep learning based online and adaptive factor model, HireVAE, is proposed, at the core of which is a hierarchical latent space that embeds the underlying relationship between the market situation and stock-wise latent factors, so that H hireVAE can effectively estimate useful latent factors given only historical market information and subsequently predict accurate stock returns.
- Learning Referring Video Object Segmentation from Weak Annotation8
A new annotation scheme is proposed that reduces the annotation effort by 8 times, while providing sufficient supervision for RVOS, and a novel RVOS method is developed that exploits weak annotations effectively and achieves comparable or even superior performance to fully-supervised methods, without requiring dense masks.
- MINI: Mining Implicit Novel Instances for Few-Shot Object Detection8
Mining Implicit Novel Instances (MINI), a novel framework to mine the implicit novel instances as auxiliary training samples, which widely exist in abundant base data but are not annotated, achieves new state-of-the-art performance on any shot and split.
- 8
- Shoot360: Normal View Video Creation from City Panorama Footage8
Shoot360 is a system that efficiently generates multi-shot normal view videos with desired content presentation and various cinematic styles, given a collection of 360 video recordings on different environments through a three-step decision process.
- 8
- 8
- 8
- Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic7
This work proposes Timely Machine, redefining test-time as wall-clock time, where models dynamically adjust strategies based on time budgets, and introduces Timely-Eval, a benchmark spanning high-frequency tool calls, low-frequency tool calls, and time-constrained reasoning.
- SS4D: Native 4D Generative Model via Structured Spacetime Latents7
SS4D is presented, a native 4D generative model that synthesizes dynamic 3D objects directly from monocular video, achieving high fidelity, temporal coherence, and structural consistency, and significantly outperforming state-of-the-art methods on both synthetic and real-world datasets.
- ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way7
ByTheWay is a training-free method to improve the quality of text-to-video generation without introducing additional parameters, augmenting memory or sampling time, and demonstrates that ByTheWay significantly improves the quality of text-to-video generation with negligible additional cost.
- 7
- InternLM-Law: An Open Source Chinese Legal Large Language Model7
This paper introduces InternLM-Law, a specialized LLM tailored for addressing diverse legal queries related to Chinese laws, spanning from responding to standard legal questions to analyzing complex real-world legal situations.
- Cycle-Consistent Learning for Weakly Supervised Semantic Segmentation7
A principle way to accomplish the weakly supervised semantic segmentation, only using scribbles as supervision to accurately propagate the semantic labels from the annotated scribbles to those unlabeled regions so that accurate pseudo masks can be harvested to learn better segmentation models.
- 7
- Placepedia: Comprehensive Place Understanding with Multi-faceted Annotations7
Placepedia, a large-scale place dataset with more than 35M photos from 240K unique places, is contributed and PlaceNet, a unified framework for multi-level place recognition, and a method for city embedding are developed, which can produce a vector representation for a city that captures both visual and multi-faceted side information.
- Compound Domain Adaptation in an Open World.7
A class-confusion loss is proposed to disentangle the domain-dominant factors out of the data and then use them to schedule a curriculum domain adaptation strategy, and a memory-augmented neural network architecture is used to increase the network's capacity for handling previously unseen domains.
- 7
- Bootstrap3D: Improving Multi-View Diffusion Model with Synthetic Data6
Bootstrap3D, a novel framework that automatically generates filtered multi-view images to assist in training multi-view diffusion models, is proposed and a Training Timestep Reschedule (TTR) strategy that leverages the denoising process to learn multi-view consistency while maintaining the original 2D diffusion prior is presented.
- 6
- 6
- FLAVA: Find, Localize, Adjust and Verify to Annotate LiDAR-based Point Clouds6
This work proposes FLAVA, a systematic approach to minimizing human interaction in the annotation process, which divides the annotation pipeline into four parts: find, localize, adjust and verify.
- 6
- More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning5
This study provides actionable guidance for integrating training data to enhance LLM capabilities, supporting both cost-effective data curation and scalable model enhancement, and hopes this work will inspire further research on how to balance "more data" against "better data" for real-world reasoning tasks.
- AICC: Parse HTML Finer, Make Models Better -- A 7.3T AI-Ready Corpus Built by a Model-Based HTML Parser5
MinerU-HTML is introduced, a novel extraction pipeline that reformulates content extraction as a sequence labeling problem solved by a 0.6B-parameter language model, and AICC (AI-ready Common Crawl), a 7.3-trillion token multilingual corpus from two Common Crawl snapshots is constructed.
- OPV: Outcome-based Process Verifier for Efficient Long Chain-of-Thought Verification5
The Outcome-based Process Verifier (OPV) is proposed, which verifies the rationale process of summarized outcomes from long CoTs to achieve both accurate and efficient verification and enable large-scale annotation.
- 5
- 5
- 5
- Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation5
- CIBench: Evaluating Your LLMs with a Code Interpreter Plugin5
An interactive evaluation framework, named CIBench, is proposed to comprehensively assess LLMs' ability to utilize code interpreters for data science tasks and provides valuable insights for future LLMs in code interpreter utilization.
- E2EAI: End-to-End Deep Learning Framework for Active Investing5
This paper is the first to propose an E2E approach that encompasses nearly the entire process of factor investing, including factor selection, combination, stock selection, and portfolio construction, and provides a comprehensive solution for managing multi-task learning conflicts in the context of deep learning-based factor investing.
- Factor Investing with a Deep Multi-Factor Model5
A novel deep multi-factor model is developed that adopts industry neutralization and market neutralization modules with clear financial insights that help to build a dynamic and multi-relational stock graph in a hierarchical structure.
- 5
- 5
- Parallel Multi-Environment Shaping Algorithm for Complex Multi-step Task5
Extensive experiments on the mini-game ‘Build Marines’ of StarCraft II environment show that the proposed algorithm is more effective than Reward Shaping, Curriculum Learning, and PLAID algorithms, which is almost close to the level of human Grandmaster.
- Mining text snippets for images on the web5
An algorithm to mine multiple diverse, relevant, and interesting text snippets for images on the web resulting in the construction of a database of images with their corresponding text snippets is proposed.
- GPT4Point++: Advancing Unified Point-Language Understanding and Generation4
GPT4Point and its enhanced version, GPT4Point++, both of which are pioneering point-language multimodal models designed for 3D object understanding and generation, excel in tasks such as 3D object recognition, 3D point cloud captioning and question answering.
- The Imitation Game: Turing Machine Imitator is Length Generalizable Reasoner4
The experimental results reveal that the key concepts in the Turing Machine, instead of the thinking styles, are indispensable for TAIL for length generalization, through which the model exhibits read-and-write behaviors consistent with the properties of the Turing Machine in their attention layers.
- 4
- ViSAudio: End-to-End Video-Driven Binaural Spatial Audio Generation4
Comprehensive experiments demonstrate that ViSAudio outperforms existing state-of-the-art methods across both objective metrics and subjective evaluations, generating high-quality binaural audio with spatial immersion that adapts effectively to viewpoint changes, sound-source motion, and diverse acoustic environments.
- 4
- 4
- SepRep-Net: Multi-source Free Domain Adaptation via Model Separation And Reparameterization4
A novel framework called SepRep-Net is proposed, which tackles multi-source free domain adaptation via model Separation and Reparameterization, and is characterized by effectiveness: competitive performance on the target domain, efficiency: low computational costs, and generalizability: maintaining more source knowledge than existing solutions.
- CriticEval: Evaluating Large Language Model as Critic4
C RITIC B ENCH is introduced, a novel benchmark designed to comprehensively and reliably evaluate four key critique ability dimensions of LLMs: feedback, comparison, refinement and meta-feedback.
- Data-free Weight Compress and Denoise for Large Language Models4
This paper proposes a novel approach termed Data-free Joint Rank-k Approximation for compressing the parameter matrices of large Language Models without necessitating additional involvement of any corpus, while simultaneously preserving orthogonality in conjunction with pruning and quantization methods.
- 4
- 4
- 4
- 4
- Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering3
KoLasSimpleQA is introduced, the first benchmark evaluating the multilingual factual ability of Large Language Models (LLMs), inspired by existing research, and hopes it will help the research community better identify LLM capability boundaries in multilingual contexts and provide guidance for model optimization.
- Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM3
HBP designs a novel batch-construction method and training recipe to address inefficiencies in data packing, and designs a dynamic training pipeline specifically tailored to HBP, including curriculum learning, adaptive sequential parallelism, and stable loss.
- 3
- 3
- Tropical: Enhancing SLO Attainment in Disaggregated LLM Serving via SLO-Aware Multiplexing3
Tropical introduces an sevice-level objectives (SLO)-aware multiplexing strategy that balances the queuing time and the interference, enabling the LLM serving to achieve high TTFT and TPOT SLOs simultaneously.
- SLIDE: A Unified Mesh and Texture Generation Framework with Enhanced Geometric Control and Multi-view Consistency3
A separate texture generation process is proposed that integrates multi-view priors post-geometry generation, effectively resolving the issue of multi-view texture inconsistency and ensuring the production of coherent and high-quality textures that complement the precisely generated meshes, thereby creating visually appealing and detailed models.
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- Learning Driving Decisions by Imitating Drivers' Control Behaviors3
This work proposes a hybrid framework for learning a decision module, which is agnostic to the mechanisms of perception, planning, and control modules, and demonstrates that the simulation agents with a learned decision module can be generalized to various complex driving scenarios where the rule-based approach fails.
- Probabilistic Ensemble of Collaborative Filters3
An ensemble-based framework to enhance the capability of a recommender in handling diverse data by forming a probabilistic model which integrates the items, the users, as well as the associations between them into a generative process.
- Integrating Specialized Classifiers Based on Continuous Time Markov Chain3
This work first decomposes the predictions into sets of pairwise preferences, treating them as transition channels between classes, and thereon constructs a continuous-time Markov chain, and uses the equilibrium distribution of this chain as the final prediction.
- Learning Deformations with Parallel Transport3
This work aims to provide a generative model that explicitly decouples deformation from appearance, which is fundamentally different from the prior work that focuses on deformation-resilient features or metrics.
- How to make Face Recognition Work: The Power of Modeling Context3
This paper explores how contextual cues can aid and improve face recognition in the wild and incorporates methods to consider what, when, and where, and shows how such contextual can immensely help with the task.
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- EasyView: Enabling and Scheduling Tensor Views in Deep Learning Compilers2
EasyView is proposed, which enables and schedules tensor views in an end-to-end workflow from neural networks onto devices, and categorize various view contexts in high-level language, and lower views in accordance with different scenarios at maximizing memory utilization and reducing data movement.
- 2
- Generative modeling of dynamic visual scenes2
A probabilistic generative model of visual scenes that coherently integrates different aspects, notably appearance, motion, and the interaction between them is derived, enabling various forms of dynamic variations for nonparametric mixture models by harnessing the connections between Poisson and Dirichlet processes.
- 2
- TL-GRPO: Turn-Level RL for Reasoning-Guided Iterative Optimization1
Turn-Level GRPO (TL-GRPO), a lightweight RL algorithm that performs turn-level group sampling for fine-grained optimization, is proposed, and results show that TL-GRPO outperforms standard GRPO and Bayesian optimization methods across various specifications.
- GetMesh: A Controllable Model for High-quality Mesh Generation and Manipulation1
This paper introduces GetMesh, a new generative model designed for mesh generation and manipulation across various categories that significantly outperforms existing methods for both single-category and multi-category generation.
- 1
- 1
- 1
- 1
- 1
- 1
- FoundaBench: Evaluating Chinese Fundamental Knowledge Capabilities of Large Language Models1
An extensive evaluation of 12 state-of-the-art LLMs using FoundaBench, employing both traditional assessment methods and the CircularEval protocol to mitigate potential biases in model responses reveals a significant disparity between models' reasoning and memory recall capabilities.
- V3Det Challenge 2024 on Vast Vocabulary and Open Vocabulary Object Detection: Methods and Results1
The V3Det Challenge 2024 aims to push the boundaries of object detection research and encourage innovation in this field, and aims to inspire future research directions in vast vocabulary and open-vocabulary object detection, driving progress in this field.
- DSDL: Data Set Description Language for Bridging Modalities and Tasks in AI Data1
This article introduces a framework called Dataset Description Language (DSDL) that aims to simplify dataset processing by providing a unified standard for AI datasets, and provides predefined DSDL templates for various tasks, convert mainstream datasets to comply with DSDL specifications, and provide comprehensive documentation and DSDL tools.
- 1
- 1
- 1
- Rethinking Trajectory Prediction via "Team Game"1
A novel formulation for multi-agent trajectory prediction is presented, which explicitly introduces the concept of interactive group consensus via an interactive hierarchical latent space and allows group-level and individual-level interactions to be captured jointly, thus substantially improving the capability of modeling complex dynamics.
- 1
- 1
- –
- ResiHP: Taming LLM Training Failures with Dynamic Hybrid Parallelism–
ResiHP is a resilient system that enables robust failure detection and fine-grained adaptation for hybrid parallel training and employs a workload-aware execution time predictor that disentangles failures from iteration time fluctuations while remaining lightweight for online detection.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Collaborative Deep Embedding via Dual Networks–
This paper presents a new method to tackle the lack of expressive power to capture complex matching relations between items and users, called Collaborative Deep Embedding, which outperforms the state of the art on multiple real-world datasets.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.