Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Shuicheng YanSuggested from co-authorship
- Eric P. XingSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works413 from public data
- Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach900
This work investigates a principle way to progressively mine discriminative object regions using classification networks to address the weakly-supervised semantic segmentation problems and proposes a new adversarial erasing approach for localizing and expanding object regions progressively.
- Perceptual Generative Adversarial Networks for Small Object Detection888
This work addresses the small object detection problem by developing a single architecture that internally lifts representations of small objects to super-resolved ones, achieving similar characteristics as large objects and thus more discriminative for detection.
- Is Faster R-CNN Doing Well for Pedestrian Detection?869
A very simple but effective baseline for pedestrian detection, using an RPN followed by boosted forests on shared, high-resolution convolutional feature maps, presenting competitive accuracy and good speed.
- Scale-aware Fast R-CNN for Pedestrian Detection843
This paper argues that the issue of large variance in instance scales, which results in undesirable large intracategory variance in features, may severely hurt the performance of modern object instance detection methods, can be substantially alleviated by the divide-and-conquer philosophy.
- Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning634
This work extends Faster /Mask R-CNN by proposing meta-learning over RoI (Region-of-Interest) features instead of a full image feature, and introduces a Predictor-head Remodeling Network (PRN) that shares its main backbone with Faster / Mask R- CNN.
- STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation617
A simple to complex (STC) framework in which only image-level annotations are utilized to learn DCNNs for semantic segmentation, which demonstrates the superiority of the proposed STC framework compared with other state-of-the-arts frameworks.
- Voxel Transformer for 3D Object Detection574
Voxel Transformer is presented, a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds that shows consistent improvement over the convolutional baselines while maintaining computational efficiency on the KITTI dataset and the Waymo Open dataset.
- Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning571
This work constructs a new largescale benchmark, Geometry3K, consisting of 3,002 geometry problems with dense annotation in formal language, and proposes a novel geometry solving approach with formal language and symbolic reasoning, called Interpretable Geometry Problem Solver (InterGPS).
- Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing527
A new benchmark Look into Person (LIP) is introduced that makes a significant advance in terms of scalability, diversity and difficulty, and a novel self-supervised structure-sensitive learning approach, which imposes human pose structures into parsing results without resorting to extra supervision.
- Toward Characteristic-Preserving Image-Based Virtual Try-On Network504
A new fully-learnable Characteristic-Preserving Virtual Try-On Network (CP-VTON) for addressing all real-world challenges in this task, and achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.
- Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark385
A new benchmark named “Look into Person (LIP)” is introduced that provides a significant advancement in terms of scalability, diversity, and difficulty, which are crucial for future developments in human-centric analysis.
- Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI384
This survey gives a comprehensive exploration of the latest advancements in Embodied AI, exploring the complexities of MLMs in virtual and real embodied agents, highlighting their significance in facilitating interactions in digital and physical environments.
- Instance-Level Human Parsing via Part Grouping Network383
This work makes the first attempt to explore a detection-free Part Grouping Network (PGN) for efficiently parsing multiple people in an image in a single pass and outperforms all state-of-the-art methods on PASCAL-Person-Part dataset.
- Rethinking Knowledge Graph Propagation for Zero-Shot Learning380
This work proposes a Dense Graph Propagation module with carefully designed direct links among distant nodes to exploit the hierarchical graph structure of the knowledge graph through additional connections and outperforms state-of-the-art zero-shot learning approaches.
- Semantic Object Parsing with Graph LSTM379
The Graph Long Short-Term Memory network is proposed, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data.
- Dual Motion GAN for Future-Flow Embedded Video Prediction377
A dual motion Generative Adversarial Net architecture is developed, which learns to explicitly enforce future-frame predictions to be consistent with the pixel-wise flows in the video through a duallearning mechanism.
- GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning351
This paper proposes a Geometric Question Answering dataset GeoQA, containing 4,998 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems, and introduces a Neural Geometric Solver (NGS) to address geometric problems by comprehensively parsing multimodal information and generating interpretable programs.
- Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation345
Experiments show that the proposed KERP approach generates structured and robust reports supported with accurate abnormality description and explainable attentive regions, achieving the state-of-the-art results on two medical report benchmarks, with the best medical abnormality and disease classification accuracy and improved human evaluation performance.
- Deep Human Parsing with Active Template Regression323
In this work, the human parsing task, namely decomposing a human image into semantic fashion/body regions, is formulated as an active template regression (ATR) problem, where the normalized mask is expressed as the linear combination of the learned mask templates.
- CIRL: Controllable Imitative Reinforcement Learning for Vision-Based Self-driving318
This work presents a general and principled Controllable Imitative Reinforcement Learning (CIRL) approach which successfully makes the driving agent achieve higher success rates based on only vision inputs in a high-fidelity car simulator, which performs better than supervised imitation learning.
- Vision-Language Navigation With Self-Supervised Auxiliary Reasoning Tasks299
AuxRN is introduced, a framework with four self-supervised auxiliary reasoning tasks to exploit the additional training signals derived from these semantic information that help the agent to acquire knowledge of semantic representations in order to reason about its activities and build a thorough perception of environments.
- Peak-Piloted Deep Network for Facial Expression Recognition299
This work presents a novel peak-piloted deep network (PPDN) that uses a sample with peak expression to supervise the intermediate feature responses for a sample of non-peak expression (hard sample) of the same type and from the same subject.
- DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection290
This paper presents DetCLIP, a paralleled visual-concept pre-training method for open-world detection by resorting to knowledge enrichment from a designed concept dictionary, and designs a concept dictionary from various online sources and detection datasets to provide prior knowledge for each concept.
- Adversarial Geometry-Aware Human Motion Prediction287
This work proposes a novel frame-wise geodesic loss as a geometrically meaningful, more precise distance measurement and presents a new learning procedure to simultaneously validate the sequence-level plausibility of the prediction and its coherence with the input sequence by introducing two global recurrent discriminators.
- 270
- Attentive Contexts for Object Detection269
This work develops a novel attention to context convolution neural network (AC-CNN)-based object detection model that effectively incorporates global and local contextual information into the region-based CNN (e.g., fast R-CNN and faster R- CNN) detection framework and provides better object detection performance.
- Dynamic Graph Enhanced Contrastive Learning for Chest X-Ray Report Generation262
This paper proposes a knowledge graph with Dynamic structure and nodes to facilitate chest X-ray report generation with Contrastive Learning, named DCL, and introduces Image-Report Contrastive and Image- report Matching losses to better represent visual features and textual information.
- Deep Variation-Structured Reinforcement Learning for Visual Relationship and Attribute Detection262
A deep Variation-structured Re-inforcement Learning (VRL) framework is proposed to sequentially discover object relationships and attributes in the whole image, and an ambiguity-aware object mining scheme is used to resolve semantic ambiguity among object categories that the object detector fails to distinguish.
- CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending251
A novel lane-sensitive architecture search framework named CurveLane-NAS to automatically capture both long-ranged coherent and accurate short-range curve information while unifying both architecture search and post-processing on curve lane predictions via point blending.
- Proposal-Free Network for Instance-Level Object Segmentation239
A Proposal-Free Network (PFN) is proposed to address the instance-level object segmentation problem, which outputs the numbers of instances of different categories and the pixel-level information on i) the coordinates of the instance bounding box each pixel belongs to, and ii) the confidences ofDifferent categories for each pixel, based on pixel-to-pixel deep convolutional neural network.
- Attention-Aware Face Hallucination via Deep Reinforcement Learning238
A novel Attention-aware Face Hallucination (Attention-FH) framework which resorts to deep reinforcement learning for sequentially discovering attended patches and then performing the facial part enhancement by fully exploiting the global interdependency of the image.
- Towards Multi-Pose Guided Virtual Try-On Network234
This paper makes the first attempt towards a multi-pose guided virtual try-on system, which enables clothes to transfer onto a person with diverse poses, and significantly outperforms all state-of-the-art methods both qualitatively and quantitatively.
- 227
- Recurrent Topic-Transition GAN for Visual Paragraph Generation220
A semi-supervised paragraph generative framework that is able to synthesize diverse and semantically coherent paragraph descriptions by reasoning over local semantic regions and exploiting linguistic knowledge is investigated.
- GP-VTON: Towards General Purpose Virtual Try-On via Collaborative Local-Flow Global-Parsing Learning204
A General-Purpose Virtual Try-ON framework, named GP-VTON, is proposed by developing an innovative Local-Flow Global-Parsing warping module and a Dynamic Gradient Truncation (DGT) training strategy, which effectively avoids the texture squeezing problem.
- Graphonomy: Universal Human Parsing via Graph Transfer Learning204
A new universal human parsing agent, named ``Graphonomy", is proposed, which incorporates hierarchical graph transfer learning upon the conventional parsing network to encode the underlying label semantic structures and propagate relevant semantic information.
- End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis203
An End-to-End Knowledge-routed Relational Dialogue System (KR-DS) that seamlessly incorporates rich medical knowledge graph into the topic transition in dialogue management, and makes it cooperative with natural language understanding and natural language generation is proposed.
- SOON: Scenario Oriented Object Navigation with Graph-based Exploration202
A novel graph-based exploration (GBE) method is proposed that outperforms various state-of-the-arts on both FAO and R2R datasets and the ablation studies on FAO validates the quality of the dataset.
- Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation201
This work proposes to modularize the large search space of NAS into blocks to ensure that the potential candidate architectures are fully trained, and distill the neural architecture (DNA) knowledge from a teacher model to supervise the block-wise architecture search, which significantly improves the effectiveness of NAS.
- Dynamic Slimmable Network195
A dynamic network slimming regime, named Dynamic Slimmable Network (DS-Net), which aims to achieve good hardware-efficiency via dynamically adjusting filter numbers of networks at test time with respect to different inputs, while keeping filters stored statically and contiguously in hardware to prevent the extra burden.
- LSTM-CF: Unifying Context Modeling and Fusion with LSTMs for RGB-D Scene Labeling191
This paper addresses the problem of semantic labeling of RGB-D scenes by developing a novel Long Short-Term Memorized Context Fusion (LSTM-CF) Model that captures and fuses contextual information from multiple channels of photometric and depth data, and incorporating this model into deep convolutional neural networks (CNNs) for end-to-end training.
- Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection188
This work presents a flexible and high-performance framework, named Pyramid R-CNN, for two-stage 3D object detection from point clouds, which outperforms the state-of-the-art 3D detection models by a large margin on both the KITTI dataset and the Waymo Open dataset.
- Semantic Object Parsing with Local-Global Long Short-Term Memory188
A novel deep Local-Global Long Short-Term Memory architecture to seamlessly incorporate short-distance and long-distance spatial dependencies into the feature learning over all pixel positions and demonstrates the significant superiority of this LG-LSTM over other state-of-the-art methods.
- SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-Rays182
The proposed SCAN incorporates a critic network to impose on the convolutional segmentation network the structural regularities inherent in human physiology, and surpasses the current state-of-the-art and generalizes well to CXR images from different patient populations and disease profiles.
- 176
- Dynamic-Structured Semantic Propagation Network170
A Dynamic-Structured Semantic Propagation Network (DSSPN) is proposed that builds a semantic neuron graph by explicitly incorporating the semantic concept hierarchy into network construction and demonstrates the superiority of the DSSPN over state-of-the-art segmentation models.
- Symbolic Graph Reasoning Meets Convolutions169
A new Symbolic Graph Reasoning (SGR) layer is proposed, which performs reasoning over a group of symbolic nodes whose outputs explicitly represent different properties of each semantic in a prior knowledge graph.
- Matching-CNN meets KNN: Quasi-parametric human parsing169
This work aims to develop a new solution with the advantages of both methodologies, namely supervision from annotated data and the flexibility to use newly annotated (possibly uncommon) images, and present a quasi-parametric human parsing model.
- MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigation168
A novel map-guided GPT-based agent is presented, dubbed MapGPT, which introduces an online linguistic-formed map to encourage global exploration and proposes an adaptive planning mechanism to assist the agent in performing multi-step path planning based on a map.
- LEGO-Prover: Neural Theorem Proving with Growing Libraries167
This work presents LEGO-Prover, which employs a growing skill library containing verified lemmas as skills to augment the capability of LLMs used in theorem proving, and advances the state-of-the-art pass rate on miniF2F-valid and miniF 2F-test.
- Auxiliary signal-guided knowledge encoder-decoder for medical report generation158
This work proposes an Auxiliary Signal-Guided Knowledge Encoder-Decoder (ASGK) to mimic radiologists’ working patterns and confirms that auxiliary signals driven Transformer-based models are with solid capabilities to outperform previous approaches on both medical terminology classification and paragraph generation metrics.
- Knowledge Distillation via the Target-aware Transformer158
This work proposes a novel one-to-all spatial matching knowledge distillation approach that surpasses the state-of-the-art methods by a significant margin on various computer vision benchmarks, such as ImageNet, Pascal VOC and COCOStuff10k.
- A Modulation Module for Multi-task Learning with Applications in Image Retrieval157
A general modulation module is proposed, which can be inserted into any convolutional neural network architecture, to encourage the coupling and feature sharing of relevant tasks while disentangling the learning of irrelevant tasks with minor parameters addition.
- 149
- Target-Guided Open-Domain Conversation148
Quantitative and human evaluations show the proposed structured approach to imposing conversational goals on open-domain chat agents can produce meaningful and effective conversations, significantly improving over other approaches.
- 146
- NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning145
Experimental results across various training settings and popular VLN benchmarks show the significant superiority of NavCoT over the direct action prediction variants, and through simple parameter-efficient finetuning, this NavCoT outperforms a recent GPT4-based approach with 7% relative improvement on the R2R dataset.
- Exploring Inter-Channel Correlation for Diversity-preserved Knowledge Distillation144
This work proposes the novel Inter-Channel Correlation for Knowledge Distillation, with which the diversity and homology of the feature space of the student network can align with that of the teacher network, and introduces the grid-level inter-channel correlation, making it capable of dense prediction tasks.
- 139
- DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment137
An efficient and scalable training framework that incorporates large-scale imagetext pairs to achieve open-vocabulary object detection (OVD) and is trained with a hybrid supervision from detection, grounding and image-text pair data under a unified data formulation.
- RCAA: Relational Context-Aware Agents for Person Search130
This paper proposes to use the target person as the query in the query-dependent relational network and incorporates the relational spatial and temporal contexts into the framework to address the problem of search for a target person from a gallery of whole scene images for which the annotations of pedestrian bounders are unavailable.
- Teaching Robots to Predict Human Motion129
A motion GAN model is introduced that achieves superior prediction performance to state-of-the-art approaches when evaluated on the standard H3.6M dataset and demonstrates its ability to replay the predicted motion in a human-like manner when interacting with a person.
- Holistic Autonomous Driving Understanding by Bird'View Injected Multi-Modal Large Models128
This work proposes a novel SQL-based method to generate instruction-response pairs automatically, which is inspired by the driving logical progression of humans, and presents BEV-InMLLM, an end-to-end method for efficiently deriving instruction-aware Bird's-Eye-View (BEV) features, language-aligned for large language models.
- BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search128
This work presents Block-wisely Self-supervised Neural Architecture Search (BossNAS), an unsupervised NAS method that addresses the problem of in-accurate architecture rating caused by large weight-sharing space and biased supervision in previous methods
- Prototypical Graph Contrastive Learning126
This article proposes a prototypical graph contrastive learning approach, which models the underlying semantic structure of the graph data via clustering semantically similar graphs into the same group and simultaneously encourages the clustering consistency for different augmentations of the same graph.
- Nonparametric Variational Auto-Encoders for Hierarchical Representation Learning123
This work proposes hierarchical non-parametric variational autoencoders, which combines tree-structured Bayesian nonparametric priors with VAEs, to enable infinite flexibility of the latent representation space.
- Recurrent 3D Pose Sequence Machines120
A Recurrent 3D Pose Sequence Machine (RPSM) is presented to automatically learn the image-dependent structural constraint and sequence-dependent temporal context by using a multi-stage sequential refinement.
- Vision-Language Navigation with Random Environmental Mixup117
The experimental results on benchmark datasets demonstrate that the augmentation data via REM help the agent reduce its performance gap between the seen and unseen environment and improve the overall performance, making the model the best existing approach on the standard VLN benchmark.
- GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems117
This work proposes a new evaluation metric GRADE, which stands for Graph-enhanced Representations for Automatic Dialogue Evaluation, which incorporates both coarse-grained utterance-level contextualized representations and fine- grained topic-level graph representations to evaluate dialogue coherence.
- CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data116
This work proposes Contrastive Language-Image-Point Cloud Pretraining (CLIP2) to directly learn the transferable 3D point cloud representation in realistic scenarios with a novel proxy alignment mechanism, and exploits naturally-existed correspondences in 2D and 3D scenarios.
- Controllable Text Generation.114
A new neural generative model is proposed which combines variational auto-encoders and holistic attribute discriminators for effective imposition of semantic structures inGeneric generation and manipulation of text.
- 110
- Learning to segment with image-level annotations110
This paper explores a more challenging problem by learning to segment under image-level annotations by incorporating the hypotheses-aware classification and cross-image contextual refinement in a supervised manner, and proposes two network training strategies for achieving good segmentation performance.
- 110
- 106
- Blending-Target Domain Adaptation by Adversarial Meta-Adaptation Networks105
Empirical results show that BTDA is a quite challenging transfer setup for most existing DA algorithms, yet AMEAN significantly outperforms these state-of-the-art baselines and effectively restrains the negative transfer effects in BTDA.
- Interpretable Structure-Evolving LSTM101
The effectiveness of structure-evolving LSTM in the application of semantic object parsing is evaluated and it is demonstrated to have an advantage over state-of-the-art L STM models on standard benchmarks.
- Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs99
A benchmark consisting of a new large-scale webpage-to-code dataset for instruction tuning and an evaluation framework for the webpage understanding and HTML code translation abilities of MLLMs is proposed.
- TransNAS-Bench-101: Improving transferability and Generalizability of Cross-Task Neural Architecture Search98
This work proposes TransNAS-Bench-101, a benchmark dataset containing network performance across seven tasks, covering classification, regression, pixel-level prediction, and self-supervised tasks, and explores two fundamentally different types of search space: cell-level search space and macro- level search space.
- Temporal Dynamic Graph LSTM for Action-Driven Video Object Detection96
This paper proposes a novel temporal dynamic graph Long Short-Term Memory network (TDGraph LSTM), which enables global temporal reasoning by constructing a dynamic graph that is based on temporal correlations of object proposals and spans the entire video.
- 96
- 95
- Unsupervised object-level video summarization with online motion auto-encoder93
A novel online motion Auto-Encoder (online motion-AE) framework that functions on the super-segmented object motion clips that can be distinguished from existing pipelines in two aspects: extracting key motions of participated objects, and learning to summarize in an unsupervised and online manner.
- Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection91
The robust fusion dataset, benchmark, detailed documents and instructions are published and the effectiveness of the toolkit is showcased by establishing two novel robustness benchmarks on widely-adopted datasets, nuScenes and Waymo, then holistically evaluate the state-of-the-art fusion methods.
- M3D-VTON: A Monocular-to-3D Virtual Try-On Network91
Extensive experiments demonstrate that the proposed M3D-VTON can manipulate and reconstruct the 3D human body wearing the given clothing with compelling details and is more efficient than other 3D approaches.
- Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining91
A novel model named Cross-modal contrAstive Product Transformer for instance-level prodUct REtrieval (CAPTURE) is proposed, that excels in capturing the potential synergy between multi- modal inputs via a hybrid-stream transformer in a self-supervised manner.
- SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection91
This paper presents a two-stage coarse-to-fine searching strategy named Structural- to-Modular NAS (SM-NAS) for searching a GPU-friendly design of both an efficient combination of modules and better modular-level architecture for object detection.
- Unsupervised Domain Adaptation for Automatic Estimation of Cardiothoracic Ratio90
This work proposes an unsupervised domain adaptation framework based on adversarial networks and introduces a discriminator that distinguishes segmentation predictions from ground truth masks, and demonstrates the clinical practicability for the diagnosis of cardiomegaly.
- 89
- 88
- 87
- MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation86
This work makes the first attempt to build and release a large-scale high-quality Medical Dialogue dataset related to 12 types of common Gastrointestinal diseases named MedDG, with more than 17K conversations collected from the online health consultation community.
- Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning85
This article proposes to use the medical visual language BERT (Medical-VLBERT) model to identify the abnormality on the COVID-19 scans and generate the medical report automatically based on the detected lesion regions to produce more accurate medical reports.
- 83
- AI on the edge: a comprehensive review81
A comprehensive survey of existing architectures, technologies, frameworks and implementations in edge intelligence and edge inference, the prior training models using local data at the resource-constrained edge devices, and discusses existing challenges, possible solutions and future directions.
- Generative Semantic Manipulation with Mask-Contrasting GAN81
This work introduces a contrasting GAN (contrast-GAN) with a novel adversarial contrasting objective which is able to perform all types of semantic translations with one category-conditional generator, and proposes a novel mask-Conditional contrast-GAN architecture to enable disentangle image background with object semantic changes.
- 81
- ConnNet: A Long-Range Relation-Aware Pixel-Connectivity Network for Salient Segmentation79
This work argues that semantic salient segmentation can be effectively resolved by reformulating it as a simple yet intuitive pixel-pair-based connectivity prediction task, and proposes a pure Connectivity Net (ConnNet), which predicts the connectivity probabilities of each pixel with its neighboring pixels.
- LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields76
The extensive experiments on the scene-level KITTI-360 dataset, and on the object-level NeRF-MVL show that the LiDAR-NeRF surpasses the model-based algorithms significantly, and a structural regularization method to preserve local structural details is introduced.
- Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems75
A simple but efficient method to make the first attempt to represent the equations of various MWPs uniformly, and a semantically-aligned universal tree-structured solver (SAU-Solver) based on an encoder-decoder framework is proposed to resolve multiple types of MWPs in a unified model, benefiting from the UET representation.
- Complex Background Subtraction by Pursuing Dynamic Spatio-Temporal Models75
This work proposes an effective background subtraction method by learning and maintaining an array of dynamic texture models within the spatio-temporal representations, and validate the proposed method in several complex scenarios, and show superior performances over other state-of-the-art approaches of background subtracted.
- DetCLIPv3: Towards Versatile Generative Open-Vocabulary Object Detection74
In this pa-per, DetCLIPv3 is introduced, a high-performing detector that excels not only at both open-vocabulary object detection, but also generating hierarchical labels for detected objects.
- ADAPT: Vision-Language Navigation with Modality-Aligned Action Prompts74
This paper proposes modAlity-aligneD Action PrompTs (ADAPT), which provides the VLN agent with action prompts to enable the explicit learning of action-level modality alignment to pursue successful navigation and shows the superiority of ADAPT over state-of-the-art methods.
- 74
- Making Large Language Models Better Planners with Reasoning-Decision Alignment73
An end-to-end decision-making model based on multimodality-augmented LLM, which simultaneously executes CoT reasoning and carries out planning results, and proposes a reasoning-decision alignment constraint between the paired CoTs and planning results, imposing the correspondence between reasoning and decision-making.
- Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks73
The Neural-Symbolic Solver (NS-Solver) is proposed to explicitly and seamlessly incorporate different levels of symbolic constraints by auxiliary tasks to explicitly incorporate essential math symbolic constraints, leading to unexplainable and unreasonable predictions.
- 71
- DAGN: Discourse-Aware Graph Network for Logical Reasoning71
This work proposes a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts, and encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse- aware features via a graph network for downstream QA tasks.
- Adaptive Temporal Encoding Network for Video Instance-level Human Parsing71
This paper introduces a novel Adaptive Temporal Encoding Network (ATEN) that alternatively performs temporal encoding among key frames and flow-guided feature propagation from other consecutive frames between two key frames to achieve a good balance between frame-level accuracy and time efficiency.
- 70
- Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation67
A Graph-Evolving Meta-Learning (GEML) framework that learns to evolve the commonsense graph for reasoning disease-symptom correlations in a new disease, which effectively alleviates the needs of a large number of dialogues and addresses the real-world challenges that the disease- symptom correlations of each disease may vary or evolve along with more diagnostic cases.
- 67
- DT-Solver: Automated Theorem Proving with Dynamic-Tree Sampling Guided by Proof-level Value Function66
A novel Dynamic-Tree Driven Theorem Solver (DT-Solver) is proposed, which introduces a dynamic-tree Monte-Carlo search algorithm, which dynamically allocates computing budgets for different state confidences, guided by a new proof-level value function to discover proof states that require substantial exploration.
- Bidirectional Graph Reasoning Network for Panoptic Segmentation65
A Bidirectional Graph Reasoning Network (BGRNet), which incorporates graph structure into the conventional panoptic segmentation network to mine the intra-modular and inter- modular relations within and between foreground things and background stuff classes, and proposes a Biddirectional Graph Connection Module to diffuse information across branches in a learnable fashion.
- EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions64
This work proposes the EMOVA (EMotionally Omni-present Voice Assistant), to enable Large Language Models with end-to-end speech abilities while maintaining the leading vision-language performance and surprisingly notices that omni-modal alignment can further enhance vision-language and speech abilities compared with the bi-modal aligned counterparts.
- Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation64
An automatic information extraction scheme based on natural language processing to obtain clinical entities and relations directly from in-domain training reports to endow models with the capability of incorporating expert knowledge.
- StepDeep64
StepDeep proposes a novel problem formulation towards an end-to-end mobility prediction framework, switching mobility events over time in an area into an event video and then posing the mobility prediction problem as a video prediction task, which is generalizable and can be applied to numerous spatial-temporal event prediction scenarios.
- ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency62
The text-to-views consistency modeling benefits the dense assignment of the visual features by encouraging different crops to align with the same text, while the cross-view segmentation consistency modeling provides additional self-supervision, overcoming the limitation of ambiguous text supervision for segmentation masks.
- Multistage Object Detection With Group Recursive Learning61
A new EM-like group recursive learning approach to iteratively refine object proposals by incorporating such context of surrounding proposals and provide an optimal spatial configuration of object detections is proposed.
- Reversible Recursive Instance-Level Object Segmentation61
This work proposes a novel Reversible Recursive Instance-level Object Segmentation (R2-IOS) framework that consists of a reversible proposal refinement sub-network that predicts bounding box offsets for refining the object proposal locations, and an instance-level segmentation sub- network that generates the foreground mask of the dominant object instance in each proposal.
- Real-to-Virtual Domain Unification for End-to-End Autonomous Driving59
DU-drive is presented, an unsupervised real-to-virtual domain unification framework for end- to-end autonomous driving that first transforms real driving data to its less complex counterpart in the virtual domain, and then predicts vehicle control commands from the generated virtual image.
- Interpretable Visual Question Answering by Reasoning on Dependency Trees58
A novel neural network model that performs global reasoning on a dependency tree parsed from the question and is capable of building an interpretable visual question answering (VQA) system that gradually derives image cues following question-driven parse-tree reasoning.
- 57
- MUSTARD: Mastering Uniform Synthesis of Theorem and Proof Data56
This work introduces MUSTARD, a data generation framework that masters uniform synthesis of theorem and proof data of high quality and diversity, and performs extensive analysis and demonstrates that MUSTARD generates validated high-quality step-by-step data.
- Automated Progressive Learning for Efficient Training of Vision Transformers56
This paper proposes automated progressive learning (AutoProg), an efficient training scheme that aims to achieve lossless acceleration by automatically increasing the training overload on-the-fly; this is achieved by adaptively deciding whether, where and how much should the model grow during progressive learning.
- M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining56
Self-harmonized ContrAstive LEarning (SCALE), a novel pretraining framework that integrates the different modalities into a unified model through an adaptive feature fusion mechanism, is proposed, providing insights into the importance of dataset scale and diversity.
- Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation56
This work introduces Texar, an open-source toolkit aiming to support the broad set of text generation tasks that transform any inputs into natural language, such as machine translation, summarization, dialog, content manipulation, and so forth.
- Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving54
A simple yet effective pretrain-adapt-finetune paradigm for general multi-task training, where the off-the-shelf pretrained models can be effectively adapted without increasing the training overhead is proposed.
- Vision-Dialog Navigation by Exploring Cross-Modal Memory54
The Cross-modal Memory Network (CMN) is proposed for remembering and understanding the rich information relevant to historical navigation actions and outperforms the previous state-of-the-art model on both seen and unseen environments.
- DTR-GAN: Dilated Temporal Relational Adversarial Network for Video Summarization54
A Dilated Temporal Relational Adversarial Network (DTR-GAN) is introduced to achieve frame-level video summarization and a discriminator is presented that learns to enhance both the information completeness and compactness via a three-player loss.
- Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models52
This work proposes to reframe the face-swapping task as a self-supervised, train-time inpainting problem, enhancing the identity transfer while blending with the target image, and introduces a mask shuffling technique during inpainting training, which allows for a so-called universal model for swapping.
- CapDet: Unifying Dense Captioning and Open-World Detection Pretraining52
This paper unify the open-world detection and dense caption tasks into a single yet effective framework by introducing an additional dense captioning head to generate the region-grounded captions and proposes a novel method named CapDet to either predict under a given category list or directly generate the category of predicted bounding boxes.
- Beyond Fixation: Dynamic Window Visual Transformer51
The dynamic window strategy proposed by DW- ViT goes beyond the model that employs a fixed single window setting and is the first to use dynamic multi-scale windows to explore the upper limit of the effect of window settings on model performance.
- MedDG: A Large-scale Medical Consultation Dataset for Building Medical Dialogue System.51
This work makes the first attempt to build and release a large-scale high-quality Medical Dialogue dataset related to 12 types of common Gastrointestinal diseases named MedDG, with more than 17K conversations collected from the online health consultation community.
- 51
- Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding50
A novel data set named knowledge-routed visual question reasoning is proposed, which aims to cut off the shortcut learning exploited by the current deep embedding models and push the research boundary of the knowledge-based visual question Reasoning.
- Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism49
Comprehensive experiments show that ROSETTA yields state-of-the-art performance on both task-based and class-based continual object detection.
- Reinforced Auto-Zoom Net: Towards Accurate and Fast Breast Cancer Segmentation in Whole-Slide Images49
This paper proposes a simple yet efficient framework Reinforced Auto-Zoom Net (RAZN), motivated by the zoom-in operation of a pathologist using a digital microscope, which outperforms both single-scale and multi-scale baseline approaches, achieving better accuracy at low inference cost.
- CoupAlign: Coupling Word-Pixel with Sentence-Mask Alignments for Referring Image Segmentation48
CoupAlign is proposed, a simple yet effective multi-level visual-semantic alignment method to couple sentence-mask alignment with word-pixel alignment to enforce object mask constraint for achieving more accurate localization and segmentation.
- 48
- Deep learning-based subdivision approach for large scale macromolecules structure recovery from electron cryo tomograms48
This article proposes a new approach for subdividing subtomograms into smaller but relatively homogeneous subsets that is based on supervised structural feature extraction using deep learning, in combination with unsupervised clustering and reference‐free classification.
- Composable Text Controls in Latent Space with ODEs47
A new efficient approach for composable text operations in the compact latent space of text, based on ordinary differential equations given arbitrary plug-in operators, which manages to generate or edit high-quality text substantially improving over previous methods in terms of generation quality and efficiency.
- CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation46
A new algorithm: compatible momentum contrast with Topology Preservation, dubbed CTP is proposed, which absorbs the knowledge of the current and previous-task models to flexibly update the modal feature and transfers the knowledge of embedding across tasks while preserving the flexibility of feature adjustment.
- 43
- Visual Question Reasoning on General Dependency Tree43
A novel reasoning network called Adversarial Composition Modular Network (ACMN), capable of building an interpretable VQA system that gradually dives the image cues following a question-driven reasoning route and makes global reasoning by incorporating the learned knowledge of all attention modules in a principled manner.
- AutoBERT-Zero: Evolving BERT Backbone from Scratch42
This work makes the first attempt to automatically discover novel pre-trained language model (PLM) backbone on a flexible search space containing the most fundamental operations from scratch and proposes an Operation-Priority Neural Architecture Search (OP-NAS) algorithm, which optimizes both the search algorithm and evaluation of candidate models.
- 42
- Neural Architecture Search for Adversarial Medical Image Segmentation42
This work automates the process of neural architecture design for the discriminator with continuous relaxation and gradient-based optimization and proposes a neural architecture search framework for adversarial medical image segmentation.
- 42
- Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration41
This work introduces Difficulty Adaptive Rollout Sampling (DARS), a method that re-weights difficult problems by using targeted, multi-stage rollouts and demonstrates simultaneous gains in both Pass@K and Pass@1, confirming that depth and breadth are orthogonal and complementary dimensions for unlocking the full power of RLVR.
- 41
- Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition41
This work proposes a novel framework that unifies template retrieval and sentence generation to handle both common and rare abnormality while ensuring the semantic coherency among the detected medical terms, and exploits hybrid-knowledge co-reasoning.
- Layout-Graph Reasoning for Fashion Landmark Detection41
The first Fine-grained Fashion Landmark Dataset (FFLD) containing 200k images annotated with at most 32 key-points for 13 clothes types is contributed and extensive experiments on two public fashion landmark datasets demonstrate the superiority of the model.
- 40
- Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition39
This work develops a novel adversarial meta sampling (AMS) approach to improve MML-ASR, and demonstrates the applicability of AMS to other low-resource speech tasks and transfer learning ASR approaches.
- 39
- OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusion38
A novel unified open-vocabulary detection method called OV-DINO is proposed, which is pre-trained on diverse large-scale datasets with language-aware selective fusion in a unified framework, and a Unified Data Integration pipeline is introduced to enable end-to-end training and eliminate noise from pseudo-label generation by unifying different data sources into detection-centric data format.
- Iterative Graph Self-Distillation38
This work proposes a method called Iterative Graph Self-Distillation (IGSD) which learns graph-level representation in an unsupervised manner through instance discrimination using a self-supervised contrastive learning approach and shows that fine-tuning the IGSD-trained models with self-training can further improve graph representation learning.
- Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning38
This paper proposes a novel inference-embedded multi-task learning framework for predicting human pose from still depth images, which is implemented with a deep architecture of neural networks, and shows that the proposed deep model significantly improves the accuracy of human pose estimation over other several state-of-the-art methods or SDKs.
- 37
- Dressing in the Wild by Watching Dance Videos37
It is found that the pixel flow excels at handling loose gar-ments whereas the vertex flow is preferred for hard poses, and by combining their advantages a novel generative network called wFlow is proposed that can effectively push up garment transfer to in-the-wild context.
- TreeRPO: Tree Relative Policy Optimization36
A novel method that estimates the mathematical expectations of rewards at various reasoning steps using tree sampling using tree sampling, and innovatively computes rewards based on step-level groups generated during tree sampling, significantly enhancing the learning process and overall performance of LLMs.
- Fine-Grained Visual–Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection36
This article proposes a simple but effective fine-grained visual-text prompt-driven self-training paradigm for open-vocabulary detection (VTP-OVD) that introduces a fine-grained visual–text prompt adapting stage to enhance the current self-training paradigm with a more powerful fine-grained alignment.
- ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design36
This work argues that simply aligning texture and garment information is not sufficient to capture the semantics of the visual information and therefore proposes MaskCLIP, a unified cross-modal fashion designer with part-level garment-text alignment that outperforms existing state-of-the-art cross- modal image synthesis approaches.
- Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL36
This study quantifies the agent's behavior difference and builds its relationship with the policy performance via {\bf Role Diversity], a metric to measure the characteristics of MARL tasks, and clearly shows that role diversity can serve as a robust measurement for the characteristic of a multi-agent cooperation task.
- WAS-VTON: Warping Architecture Search for Virtual Try-on Network36
This paper introduces a NAS-Warping Module and elaborately design a bilevel hierarchical search space to identify the optimal network-level and operation-level flow estimation architecture and jointly learns a combination of repeatable warping cells and convolution operations specifically for the clothing-person alignment.
- Dynamic Knowledge Routing Network for Target-Guided Open-Domain Conversation36
Quantitative and human evaluations show the proposed novel dynamic knowledge routing network (DRKN) can produce meaningful and effective target-guided conversations, significantly improving over other state-of-the-art methods by more than 20% in success rate and more than 0.6 in average smoothness score.
- Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning35
Ground-R1 is proposed, a de-biased thinking-with-images framework trained via a novel Scale Relative Policy Optimization (SRPO) objective that replaces standard GRPO, which recalibrates reward learning across evidence regions of different sizes through scale-aware binning and intra-/inter-bin comparisons, enabling balanced credit assignment during training.
- FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model34
FireEdit, an innovative Fine-grained Instruction-based image editing framework that exploits a REgion-aware VLM enhanced with fine-grained region tokens and the time-dependent diffusion model, demonstrates significant advantages in comprehending editing instructions and maintaining high semantic consistency.
- StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration34
StoryAgent is proposed, a versatile framework for video generation tasks, and novel techniques for preserving protagonist consistency are introduced, including a customized Image-to-Video (I2V) method, LoRA-BE, to enhance intra-shot temporal consistency, while a novel storyboard generation pipeline is proposed to maintain subject consistency across shots.
- Heterogeneous graph reasoning for knowledge-grounded medical dialogue system34
This work proposes an explainable Heterogeneous Graph Reasoning (HGR) model to unify the relational dialogue context understanding and entity-correlation reasoning into a heterogeneous graph structure.
- Towards Quantifiable Dialogue Coherence Evaluation34
Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics.
- Correctable Landmark Discovery via Large Models for Vision-Language Navigation33
In CONSOLE, VLN is cast as an open-world sequential landmark discovery problem, by introducing a novel correctable landmark discovery scheme based on two large models ChatGPT and CLIP, and a learnable cooccurrence scoring module, which corrects the importance of each cooccurrence according to actual observations for accurate landmark discovery.
- Contrastive Instruction-Trajectory Learning for Vision-Language Navigation33
A Contrastive Instruction-Trajectory Learning (CITL) framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation and can be easily integrated with VLN backbones to form a new learning paradigm and achieve better generalizability in unseen environments.
- Deep Structured Scene Parsing by Learning with Image Descriptions33
This paper proposes a deep architecture consisting of a convolutional neural network extracting the image representation for pixelwise object labeling and a recursive neural network discovering the hierarchical object structure and the inter-object relations and decomposes each sentence into a semantic tree.
- RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigation32
RoomTour3D, a video-instruction dataset derived from web-based room tour videos that capture real-world indoor spaces and human walking demonstrations, facilitates the development of trainable zero-shot VLN agents, showcasing the potential and challenges of advancing towards open-world navigation.
- CombiBench: Benchmarking LLM Capability for Combinatorial Mathematics32
This work introduces CombiBench, a comprehensive benchmark comprising 100 combinatorial problems, each formalized in Lean~4 and paired with its corresponding informal statement, and provides a comprehensive and standardized evaluation framework, dubbed Fine-Eval, for formal mathematics, which accommodates not only proof-based problems but also, for the first time, the evaluation of fill-in-the-blank questions.
- MLP Can Be a Good Transformer Learner32
This paper introduces a novel strategy that simplifies vision transformers and reduces computational load through the selective removal of non-essential attention layers, guided by entropy considerations.
- 32
- FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving31
This paper proposes FVEL, an interactive Formal Verification Environment with LLMs, which transforms a given code to be verified into Isabelle, and then conducts verification via neural automated theorem proving with an LLM.
- 31
- Structure-Preserving 3D Garment Modeling with Neural Sewing Machines30
A novel Neural Sewing Machine (NSM) is proposed, a learning-based framework for structure-preserving 3D garment modeling, which is capable of learning representations for garments with diverse shapes and topologies and is successfully applied to3D garment reconstruction and controllable manipulation.
- DialogGen: Multi-modal Interactive Dialogue System for Multi-turn Text-to-Image Generation29
The Multi-modal Dialogue Benchmark (DialogBen) is introduced, a comprehensive bilingual benchmark designed to assess the ability of MLLMs to generate accurate and coherent multi-modal content that supports image editing and contains two evaluation metrics to measure the model's ability to switch modalities and the coherence of the output images.
- 29
- 29
- 29
- DiffCloth: Diffusion Based Garment Synthesis and Manipulation via Structural Cross-modal Semantic Alignment28
DiffCloth is introduced, a diffusion-based pipeline for cross-modal garment synthesis and manipulation, which empowers diffusion models with flexible compositionality in the fashion domain by structurally aligning the cross-modal semantics.
- EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation28
This paper has a critical insight that improving the feed-forward network (FFN) in BERT has a higher gain than improving the multi-head attention (MHA) since the computational cost of FFN is 2$\sim$3 times larger than MHA.
- Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation28
A Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target by destroying the most instructive information in instructions at different timesteps, is proposed.
- Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition28
This work introduces Wav-BERT, a cooperative acoustic and linguistic representation learning method to fuse and utilize the contextual information of speech and text and significantly outperforms the existing approaches and achieves state-of-the-art performance on low-resource speech recognition.
- Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN28
This paper presents a novel universal object detector called Universal-RCNN that incorporates graph transfer learning for propagating relevant semantic information across multiple datasets to reach semantic coherency and achieves the state-of-the-art results on multiple object detection benchmarks.
- 28
- Deep Attribute-preserving Metric Learning for Natural Language Object Retrieval28
A deep Attribute-Preserving Metric learning (AP-Metric) framework that jointly generates unique query-sensitive region proposals and conducts novel cross-modal feature learning that explicitly pursues consistency over semantic attribute abstraction within both domains for deep metric learning is proposed.
- HumanRefiner: Benchmarking Abnormal Human Generation and Refining with Coarse-to-fine Pose-Reversible Guidance27
This work introduces AbHuman, the first large-scale synthesized human benchmark focusing on anatomical anomalies, and proposes HumanRefiner, a novel plug-and-play approach for the coarse-to-fine refinement of human anomalies in text-to-image generation.
- Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search27
This work proposes Arch-Graph, a transferable NAS method that predicts task-specific optimal architectures with respect to given task embeddings, and shows its transferability and high sample efficiency across numerous tasks, beating many NAS methods designed for both single-task and multi-task search.
- 27
- 27
- Towards robust partially supervised multi-structure medical image segmentation on small-scale data26
This work proposes Vicinal Labels Under Uncertainty (VLUU), a simple yet efficient framework utilizing the human structure similarity for partially supervised medical image segmentation that can consistently outperform previous partially supervised models in these settings.
- DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance25
A high-fidelity image-to-video generation method by devising a frame retention branch based on a pre-trained video diffusion model, named DreamVideo, which outperforms the state-of-the-art method and can generate high-quality videos of different actions by providing varying prompt texts.
- AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis25
This work conducts comprehensive analyses on the multimodal implicit field of LiDAR-camera joint synthesis, revealing the underlying issue lies in the misalignment of different sensors, and introduces AlignMiF, a geometrically aligned multimodal implicit field with two proposed modules: Geometry-Aware Alignment (GAA) and Shared Geometry Initialization (SGI).
- WarpDiffusion: Efficient Diffusion Model for High-Fidelity Virtual Try-on25
WarpDiffusion is proposed, which bridges the warping-based and diffusion-based paradigms via a novel informative and local garment feature attention mechanism, and surpassing state-of-the-art methods both qualitatively and quantitatively.
- MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation25
MixReorg is proposed, a novel and straightforward pre-training paradigm for semantic segmentation that enhances a model’s ability to reorganize patches mixed across images, exploring both local visual relevance and global semantic coherence.
- LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts25
This paper achieves accurate complex scene generation by proposing a semantically controllable Layout-AWare diffusion model, termed LAW-Diffusion, which introduces a spatial dependency parser to encode the location-aware semantic coherence across objects as a layout embedding and produces a scene with perceptually harmonious object styles and contextual relations.
- 25
- Atom correlation based graph propagation for scene graph generation25
Experimental results on two benchmark datasets show the superiority of the proposed Atom Correlation Based Graph Propagation (AC-GP) over strong baseline methods.
- 25
- BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving24
The BEV-TSR framework is proposed which leverages descriptive text as an input to retrieve corresponding scenes in the Bird’s Eye View (BEV) space and a multi-level retrieval dataset, nuScenes-Retrieval, based on the widely adopted nuScenes dataset is established.
- AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generation24
This work introduces a training-free multi-agent framework called AutoStudio, which can generate a sequence of multi-subject images interactively and consistently and introduces a Parallel-UNet to replace the original UNet in the drawer, which employs two parallel cross-attention modules for exploiting subject-aware features.
- Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Models24
This work proposes a new benchmark for COntinuAl inStruction Tuning on LVLMs (COAST), which encompasses the aforementioned domain-incremental, capability-incremental, and dataset-incremental configurations and proposes Continual LLaVA, a rehearsal-free method tailored for continual instruction tuning in LVLMs.
- 24
- 24
- 24
- Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation24
Experimental results show the novel Edit-Invariant Sequence Loss (EISL) method significantly outperforms the common CE loss and other strong baselines on all the tasks, and is easy to implement and efficient to compute with existing libraries.
- PASTA-GAN++: A Versatile Framework for High-Resolution Unpaired Virtual Try-on24
This work proposes a characteristic-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN++ (PASTA-GAN++), to achieve a versatile system for high-resolution unpaired virtual try-on and incorporates a novel spatially-adaptive residual module to inject the coarse warped garment feature into the generator.
- CATCH: Context-Based Meta Reinforcement Learning for Transferrable Architecture Search24
This work proposes CATCH, a novel Context-bAsed meTa reinforcement learning (RL) algorithm for transferrable arChitecture searcH, which allows CATCH to efficiently adapt to new tasks while being agnostic to search spaces.
- Learning to Segment Human by Watching YouTube24
A very-weakly supervised learning framework for human segmentation task, where only an imperfect human detector is available along with massive weakly-labeled YouTube videos, and reaches a new state-of-the-art performance on the human segmentsation task.
- 23
- InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction23
InfiniteWorld is introduced, a unified and scalable simulator for general vision-language robot interaction built on Nvidia Isaac Sim that integrates a series of improvements in generation-driven 3D asset construction, Real2Sim, automated annotation framework, and unified 3D asset processing.
- Visual Exemplar Driven Task-Prompting for Unified Perception in Autonomous Driving23
An effective multi-task framework, VE-Prompt, is presented, which introduces visual exemplars via task-specific prompting to guide the model toward learning high-quality task- specific representations and bridges transformer-based encoders and convolutional layers for efficient and accurate unified perception in autonomous driving.
- 23
- RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environments22
The ability of some existing models to reason intention-oriented objects in open environments is evaluated and the proposed RIO is evaluated, specifically designed to incorporate diverse real-world scenarios and a wide range of object categories.
- Knowledge Fusion Distillation: Improving Distillation with Multi-scale Attention Mechanisms22
It is found that the feature maps obtained by early feature fusion do not serve as a good teacher to guide the training of the branches of the network, and a new self-distillation method, knowledge fusion distillation, is proposed.
- Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift22
This work addresses the ranking correlation problem to the supernet training consistency shift, including feature shift and parameter shift, using a nontrivial supernet-∏ model, called ∏-NAS, which runs in an unsupervised manner, which can search for more transferable architectures.
- FULLER: Unified Multi-modality Multi-task 3D Perception via Multi-level Gradient Calibration21
A novel yet simple multi-level gradient calibration learning framework across tasks and modalities during optimization, ensuring the downstream tasks pay balanced attention to different modalities.
- Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining21
The proposed MOTOR successfully mimics the human practice of fulfilling a"medical student" to accelerate the process of becoming a"specialist" and believes that this work makes a significant stride in realizing MAGI.
- Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning21
A noise-robust bi-level re-weighting framework which is able to learn the per-sample weights measuring the data quality without requiring any gold data is proposed.
- 21
- 20
- RGB-D Scene Labeling with Long Short-Term Memorized Fusion Model.20
This paper addresses the problem of semantic labeling of RGB-D scenes by developing a novel Long Short-Term Memorized Fusion (LSTM-F) Model that captures and fuses contextual information from multiple channels of photometric and depth data, and incorporating this model into deep convolutional neural networks (CNNs) for end-to-end training.
- HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models19
HiRes-LLaVA is proposed, a novel framework designed to efficiently process high-resolution inputs of any size without altering the original contextual and geometric information, and demonstrates the superiority of HiRes-LLaVA on both existing public benchmarks and EntityGrid-QA.
- EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation19
A universal framework called EasyControl, which enables users to control video generation with a single condition map by propagating and injecting condition features through condition adapters, which demonstrates powerful image retention ability and significantly improves various evaluation metrics across multiple validation datasets.
- GarmentAligner: Text-to-Garment Generation via Retrieval-augmented Multi-level Corrections19
This work proposes GarmentAligner, a text-to-garment diffusion model trained with retrieval-augmented multi-level corrections that achieves superior fidelity and fine-grained semantic alignment when compared to existing competitors.
- 19
- 19
- 19
- VisDiaHalBench: A Visual Dialogue Benchmark For Diagnosing Hallucination in Large Vision-Language Models18
A novel visual dialogue hallucination evaluation benchmark that investigates the hallucination problem of LVLMs when given long-term misleading textual history and differs from previous hallucination benchmarks in the following three points.
- Language-Driven Visual Consensus for Zero-Shot Semantic Segmentation18
Equipped with a vision-language prompting strategy, this approach significantly boosts the generalization capacity of segmentation models for unseen classes and introduces route attention into the transformer decoder to find visual consensus, thereby enhancing semantic consistency within the same object.
- Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning18
3D-aware global correspondences are presented, which are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies and demonstrate the superiority of this method against the SOTA try-on approaches.
- 18
- ElixirNet: Relation-Aware Network Architecture Adaptation for Medical Lesion Detection18
A novel ElixirNet that includes three components, TruncatedRPN balances positive and negative data for false positive reduction, Auto-lesion Block is automatically customized for medical images to incorporates relation-aware operations among region proposals, and leads to more suitable and efficient classification and localization.
- 18
- All Robots in One: A New Standard and Unified Dataset for Versatile, General-Purpose Embodied Agents17
A large-scale unified ARIO dataset, comprising approximately 3 million episodes collected from 258 series and 321,064 tasks, is presented, representing a significant step towards bridging the gaps of existing data resources.
- 17
- 17
- PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos17
This paper proposes PhysVLM as a physical knowledge-enhanced video LLM, a pioneering benchmark to evaluate physical commonsense violations in gameplay videos, and proposes a preference optimization dataset PhysDPO, where the dis-preferred responses are generated conditioned on misleading titles.
- UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model17
This work introduces a new 3D human-body model with a series of decoupled parameters that could freely control the generation of the body, and constructs an ultra dense synthetic benchmark UltraPose, containing around 1.3 billion corresponding points.
- 17
- 17
- Improving Deep Binary Embedding Networks by Order-Aware Reweighting of Triplets17
An order-aware reweighting method is proposed to effectively train the triplet-based deep networks, which up- weights the important triplets and down-weights the uninformative triplets via the rank lists of the binary codes.
- 17
- Deep Learning Based Supervised Semantic Segmentation of Electron Cryo-Subtomograms17
A novel 3D convolutional neural network inspired by Fully Convolutional Network and Encoder-Decoder Architecture is introduced for the supervised segmentation of macromolecules of interest in subtomograms.
- ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research16
ORMind is introduced, a cognitive-inspired framework that enhances optimization through counterfactual reasoning that emulates human cognition, implementing an end-to-end workflow that systematically transforms requirements into mathematical models and executable solver code.
- Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes16
Sitcom-Crafter is introduced, a comprehensive and extendable system for human motion generation in 3D space, which can be guided by extensive plot contexts to enhance workflow efficiency for anime and game designers.
- 16
- 16
- Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer16
This work proposes a graph reasoning and transfer learning framework, named “Graphonomy,” which incorporates human knowledge and label taxonomy into the intermediate graph representation learning beyond local convolutions, and applies it to two relevant but different image understanding research topics.
- RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation15
RoBridge maintains the declarative skill of VLM and unleashes the procedural skill of reinforcement learning, effectively bridging the gap between cognition and execution in robotic systems, offering a new paradigm for general robotic manipulation.
- VITON-DiT: Learning In-the-Wild Video Try-On from Human Dance Videos via Diffusion Transformers15
This work proposes the first DiT-based video try-on framework for practical in-the-wild applications, named VITON-DiT, and demonstrates the superiority of VITON-DiT in generating spatio-temporal consistent try-on results for in-the-wild videos with complicated human poses.
- Dilated temporal relational adversarial network for generic video summarization15
A discriminator that learns to enforce both the information completeness and compactness of summaries via a three-player loss is presented, to ensure that summaries capture enough key video representation from a global perspective rather than a trivial randomly shorten sequence.
- Construction and effect evaluation of prediction model for red blood cell transfusion requirement in cesarean section based on artificial intelligence14
The important predictors of blood transfusion during cesarean section included preoperative RBC, surgical method, the site of surgery, coagulation-related indicators, and other factors and it was found that the overall accuracy of the AI model was higher than actual blood using.
- Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval14
A novel Entity-Graph Enhanced Cross-Modal Pretraining (EGE-CMP) model is proposed for instance-level commodity retrieval, which explicitly injects entity knowledge in both node-based and subgraph-based ways into the multi-modal networks via a self-supervised hybrid-stream transformer.
- Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation14
This work introduces Texar, an open-source toolkit aiming to support the broad set of text generation tasks by abstracting the common patterns underlying the diverse tasks and methodologies, creating a library of highly reusable modules and functionalities, and enabling arbitrary model architectures and various algorithmic paradigms.
- Adaptive context-aware reinforced agent for handwritten text recognition14
This paper addresses handwritten text recognition problems by leveraging an adaptive context-aware reinforced agent which learns the actions to determine the scales of context regions during inference by formulate the approach in a reinforcement learning framework.
- 13
- UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation13
A novel approach for long-context Retrieval-Augmented Generation that utilizes Signal-to-Noise Ratio (SNR)-based span uncertainty to estimate similarity between text chunks, leading to improved generalization and robustness in long-context RAG tasks.
- Towards Causality-Aware Inferring: A Sequential Discriminative Approach for Medical Diagnosis13
A progressive assurance agent is proposed, which includes the dual processes accounting for symptom inquiry and disease diagnosis respectively, which can improve upon the out-of-distribution generalization significantly and possesses the advantage of transportability.
- 13
- Unbiased Math Word Problems Benchmark for Mitigating Solving Bias13
A novel MWP dataset named UnbiasedMWP is introduced which is constructed by varying the grounded expressions in collected data and annotating them with corresponding multiple new questions manually, and a Dynamic Target Selection (DTS) Strategy to dynamically select more suitable target expressions according to the longest pre-match between the current model output and candidate equivalent equations which are obtained by applying commutative law during training.
- 13
- 13
- 12
- 12
- New insights into restoring microbial communities by side-stream supersaturated oxygenation to improve the resilience of rivers affected by combined sewer overflows12
The side-stream supersaturation (SSS) oxygenation was applied to restore microbial function of rivers which are affected by CSOs to improve the self-purification capacity, and showed that apart from the dissolved organic matter inputs from CSO event, the sediment had become an important contributor to pollution in the studied river.
- Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp12
A concise yet powerful method to construct Continuous Transition, which exploits the trajectory information by exploiting the potential transitions along the trajectory and synthesizes new transitions for training by linearly interpolating the consecutive transitions.
- Grammatically Recognizing Images with Tree Convolution12
A simple yet effective tree convolution (TreeConv) operation for deep neural networks that designs a generative image grammar, i.e., tree generation rule, to parse the hierarchy of internal feature maps by generating tree structures and implicitly learning the specific visual grammars for each object category.
- 12
- Embodied Arena: A Comprehensive, Unified, and Evolving Evaluation Platform for Embodied AI11
Embodied Arena is presented, a comprehensive, unified, and evolving evaluation platform for Embodied AI that establishes a systematic embodied capability taxonomy spanning three levels (perception, reasoning, task execution), seven core capabilities, and 25 fine-grained dimensions, enabling unified evaluation with systematic research objectives.
- GLaD: Geometric Latent Distillation for Vision-Language-Action Models11
This work introduces GLaD, a geometry-aware VLA framework that incorporates 3D geometric priors during pretraining through knowledge distillation, validate that geometry-aware pretraining enhances spatial reasoning and policy generalization without requiring explicit depth sensors or 3D annotations.
- Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models?11
This paper proposes a paradigm of Self-structured Chain of Thought (SCoT), which is composed of minimal semantic atomic steps and designs a novel AtomThink framework, which can not only generate cognitive CoT structures for various complex tasks but also mitigates the phenomenon of overthinking.
- DiffDis: Empowering Generative Diffusion Model with Cross-Modal Discrimination Capability11
This paper proposes DiffDis to unify the cross-modal generative and discriminative pretraining into one single framework under the diffusion process, and proposes a novel dual-stream network architecture, which fuses the noisy text embedding with the knowledge of latent images from different scales for image-text discrim inative learning.
- MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library11
This paper proposes Multi-Agent RLlib (MARLlib), a comprehensive MARL algorithm library that facilitates RLlib for solving multi-agent problems and goes beyond current work by integrating diverse environment interfaces and providing flexible parameter sharing strategies.
- 11
- RelCLIP: Adapting Language-Image Pretraining for Visual Relationship Detection via Relational Contrastive Learning11
A novel Relational Contrastive Learning (RCL) approach which explores relation-level visual-semantic alignment via learning to match cross-modal relational embeddings, and achieves significant improvements over strong baselines under full and distant supervision, demonstrating its powerful generalization ability in learning relationship representations.
- 11
- Data-to-Text Generation with Style Imitation11
A new way of stylistic control is studied by using existing sentences as “soft” templates, that is, a model learns to imitate the writing style of any given exemplar sentence, with automatic adaptions to faithfully describe the record.
- Fashion Editing with Multi-scale Attention Normalization.11
A novel Fashion Editing Generative Adversarial Network (FE-GAN), which is capable of manipulating fashion images by free-form sketches and sparse color strokes and significantly outperforms the state-of-the-art methods on image manipulation.
- Unseen From Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation10
A Rewriting-driven AugMentation (RAM) paradigm for VLN is proposed, which directly creates the unseen observation-instruction pairs via rewriting human-annotated training data and develops a mixing-then-focusing training strategy with a random observation cropping scheme, effectively enhancing data distribution diversity while suppressing augmentation data noise during training.
- InstruGen: Automatic Instruction Generation for Vision-and-Language Navigation Via Large Multimodal Models10
Experimental results demonstrate that agents trained with path-instruction pairs generated by InstruGen achieves state-of-the-art performance on the R2R and RxR benchmarks, particularly in unseen environments.
- 10
- 10
- Heterogeneous Excitation-and-Squeeze Network for visual dialog10
The visualization analysis shows that the proposed HESNet alleviates the information mix by collecting the heterogeneous clues parallelly and adaptively, and the superiority of the proposed approach over the state-of-the-art methods is shown.
- Image Comes Dancing With Collaborative Parsing-Flow Video Synthesis10
Results show the method substantially improves over previous approaches and is able to generate appealing and photo-realistic target videos given any input person image and the dedicated designed fusion network ensure the temporal coherence.
- NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models10
A joint Neural Architecture Search and Online Adaption framework named NASOA is proposed towards a faster task-oriented fine-tuning upon the request of users and proposes a novel joint block and macro level search space to enable a flexible and effi-cient search.
- 10
- 10
- 9
- Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMs9
An in-depth analysis reveals that 3D scene-centric VLMs show limited reliance on the 3D scene encoder, and the pre-train stage appears less effective than in 2D VLMs, and that data scaling benefits are less pronounced on larger datasets.
- OmniGen: Unified Multimodal Sensor Generation for Autonomous Driving9
This work proposes OminiGen, which generates aligned multimodal sensor data in a unified framework and designs a novel generalizable multimodal reconstruction method, UAE, to jointly decode LiDAR and multi-view camera data.
- 9
- UniGS: Unified Language-Image-3D Pretraining with Gaussian Splatting9
The effectiveness of UniGS in learning a more general and stronger aligned multi-modal representation is demonstrated, achieving leading results across different 3D tasks with remarkable improvements over previous SOTA, Uni3D, including on zero-shot classification, text-driven retrieval and open-world understanding tasks.
- 9
- S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking9
An Uncertainty-aware Probabilistic Decoder is introduced to capture the uncertainty in object prediction with probabilistic attention and an Uncertainty-guided Query Denoising strategy is proposed to enhance training robustness and convergence against uncertainty.
- CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation9
A novel zero-shot framework called CorNav is introduced, utilizing a large language model for decision-making and comprising two key components: 1) incorporating environmental feedback for refining future plans and adjusting its actions, and 2) multiple domain experts for parsing instructions, scene understanding, and refining predicted actions.
- Discourse-Aware Graph Networks for Textual Logical Reasoning9
This work proposes logic structural-constraint modeling to solve the logical reasoning QA and introduces discourse-aware graph networks (DAGNs), which first construct logic graphs leveraging in-line discourse connectives and generic logic theories, then learn logic representations by end-to-end evolving the logic relations with an edge-reasoning mechanism and updating the graph features.
- Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning9
This paper presents a model-agnostic training paradigm, called Progressive Perturbation-aware Contrastive Learning (PROPER) to enhance the generalization ability of existing VLN agents to the real world, by requiring them to learn towards deviation-robust navigation.
- 9
- REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement9
A recursive erasure memory network (REM-Net) equipped with a module to refine the evidence by recursively erasing the low-quality evidence that does not explain the question answering, which demonstrates the performance of REM- Net and shows that the refined evidence is explainable.
- 9
- 8
- Coordinate Transformer: Achieving Single-stage Multi-person Mesh Recovery from Videos8
The Coordinate transFormer (Coord-Former) is proposed, that directly models multi-person spatial-temporal relations and simultaneously performs multi-mesh recovery in an end-to-end manner and leverages a novel Coordinate-Aware Attention to preserve pixel-level spatial-temporal coordinate information.
- 8
- Video Spatial Reasoning with Object-Centric 3D Rollout7
Object-Centric 3D Rollout (OCR) is proposed, a novel strategy that introduces structured perturbations to the 3D geometry of selected objects during training and confirms OCR’s superiority over prior rollout strategies.
- 7
- Vision Language Navigation with Knowledge-driven Environmental Dreamer7
This paper introduces the Knowledge-driven Environmental Dreamer (KED), a method that leverages the knowledge of the embodied environment and generates unseen scenes for a navigation agent to learn and outperforms the state-of-the-art methods on various VLN benchmarks.
- Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data7
H ELIP is presented, a low-cost strategy for boosting the performance of well-trained CLIP models bynetuning them with hard samples over original training data and consistently boosts existing models to achieve leading performance.
- DNA Family: Boosting Weight-Sharing NAS With Block-Wise Supervisions7
This work modularizes a large search space into blocks with small search spaces and develops a family of models, namely a DNA family, that are capable of resolving multiple dilemmas of the weight-sharing NAS, such as scalability, efficiency, and multi-modal compatibility.
- 7
- 7
- A Data-Centric Framework for Composable NLP Workflows7
A unified open-source framework to support fast development of such sophisticated NLP workflows in a composable manner and introduces a uniform data representation to encode heterogeneous results by a wide range of NLP tasks.
- EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning6
Experimental results under both task-specific and cross-task training paradigms demonstrate the consistent superiority of EvolveNav over previous LLM-based VLN approaches on various popular benchmarks, including R2R, REVERIE, CVDN, and SOON.
- 6
- 6
- 6
- 6
- Guest editorial: Image/video understanding and analysis6
This special issue aims to collect recent state-of-the-art achievement on image/video understanding and analysis, especially the work devoted to several new challenges in the field.
- 6
- AtomThink: Multimodal Slow Thinking With Atomic Step Reasoning5
The core idea is that models can learn to adaptively use different levels of reasoning to tackle questions of varying complexity, and proposes a novel paradigm of Self-structured Chain of Thought (SCoT), which consists of minimal semantic atomic steps.
- Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI5
This survey provides a systematic framework revealing how physical AI advances through four interconnected stages: perception transforms sensory data into structured physical representations, reasoning derives explanations from observed phenomena, modeling enables predictive simulation grounded in physical principles, and embodied interaction closes the loop through physical manipulation and environmental feedback.
- 5
- 5
- Efficient Training of Large Vision Models via Advanced Automated Progressive Learning4
This work proposes AutoProg-One, an automated progressive learning scheme featuring momentum growth (MoGrow) and the one-shot growth schedule search, and introduces AutoProg-Zero, a novel zero-shot automated progressive learning method, eliminating the need for one-shot supernet training.
- DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder4
The proposed DreamFit incorporates a lightweight Anything-Dressing Encoder specifically tailored for the garment-centric human generation, which surpasses all existing methods and leverages pretrained large multi-modal models to enrich the prompt with fine-grained garment descriptions, thereby reducing the prompt gap between training and inference.
- Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling4
MILO is introduced, an Implicit spatio-spatial reasoning modeling paradigm that simulates human-like spatial imagination and proposes RePE (Relative Positional Encoding), a novel encoding scheme that captures relative camera-pose transformations, offering superior performance over absolute coordinate systems.
- 4
- Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs4
An axiomatic system is proposed to define and quantify the precise memorization and in-context reasoning effects used by the large language model (LLM) for language generation and shows that the clear disentanglement of memorization effects and in-context reasoning effects enables a straightforward examination of detailed inference patterns encoded by LLMs.
- 4
- PTUS: Photo-Realistic Talking Upper-Body Synthesis via 3D-Aware Motion Decomposition Warping4
A Photo-realistic Talking Upper-body Synthesis method via 3D-aware motion decomposition warping, named PTUS, to both precisely synthesize the upper body as well as recover the details of the face such as blinking and lip synchronization, is proposed.
- GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training4
GrowCLIP is proposed, a data-driven automatic model growing algorithm for contrastive language-image pre-training with continuous image-text pairs as input that adopts a dynamic growth space and seeks out the optimal architecture at each growth step to adapt to online learning scenarios.
- Fashion Matrix: Editing Photos by Just Talking4
A hierarchical AI system dedicated to editing photos by just talking, which employs LLM as its foundational support and engages in iterative interactions with users to explore the collaborative potential of functionally diverse pre-trained models in the domain of fashion editing.
- 4
- 4
- Establishment of a Convenient System for the Culture and Study of Perineurium Barrier In Vitro4
An efficient procedure for the establishment of perineurial cell lines is developed as a tool for investigating the physiology and pathophysiology of the peripheral nerve barriers and may elucidate the pathogenesis and progression of PNI, and other peripheral nerve disorders.
- 3
- LaVieID: Local Autoregressive Diffusion Transformers for Identity-Preserving Video Creation3
LaVieID is a novel local a utoregressive vi deo diffusion framework designed to tackle the challenging id entity-preserving text-to-video task and can generate high-fidelity personalized videos and achieve state-of-the-art performance.
- 3
- 3
- 3
- AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations3
Alignedcot is introduced, an LLM-acquainted prompting technique that includes proficient ``native-speaking'' in in-context learning for the LLMs and achieves consistent and correct step-wise prompts in zero-shot scenarios by progressively probing, refining, and formatting the LLM chain of thoughts so that free from handcrafted few-shot demonstrations while maintaining the prompt quality.
- 3
- 3
- 3
- UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning3
UniDiff is a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) and effectively learns aligned semantics and mitigates the issue of semantic collapse during fine-tuning on small datasets.
- 3
- 3
- 3
- Learning Reinforced Agents with Counterfactual Simulation for Medical Automatic Diagnosis.3
A propensity-based patient simulator (PBPS), capable of facilitating the training of MAD agents by generating informative counterfactual answers along with the disease diagnosis, and a progressive assurance agent~(P2A) trained with PBPS, which includes two separate yet cooperative branches accounting for the execution of symptom-inquiry and disease-diagnosis actions, respectively.
- 3
- 2
- 2
- GS-CLIP: Gaussian Splatting for Contrastive Language-Image-3D Pretraining from Real-World Data2
GS-CLIP is proposed for the first attempt to introduce 3DGS (3D Gaussian Splatting) into multimodal pre-training to enhance 3D representation and significantly improves the state-of-the-art, outperforming the previously best results.
- 2
- Surfer: Progressive Reasoning with World Models for Robotic Manipulation2
A realistic robotic manipulation simulator is introduced and a Robotic Manipulation with Progressive Reasoning Tasks (RM-PRT) benchmark is built on this basis, which evaluates the robot’s ability to understand natural language instructions in two modes of adsorption and grasping.
- 2
- 2
- 2
- A Medical Diagnostic Expert System with LPSO-based Artificial Neural Network2
The results prove that, LPSO-based NN approach has outperformed the BP-NN approach, and significant features identified: L PSO can obtain higher prediction accuracy of diagnosis.
- On-line Detection Method for Outliers of Dynamic Instability Measurement Data in Geological Exploration Control Process2
Through the experiment and application, it is proven that the anomaly data detection method proposed in this paper is more suitable for the detection data in the process of unstable regulation.
- 1
- 1
- 1
- DirectSwap: Paired, Mask-Free Video Head Swapping with Full-Reference Evaluation1
Di-rectSwap is proposed, a mask-free, direct video head-swapping framework that extends an image U-Net into a video diffusion model with a motion module and conditioning inputs and introduces the Motion-and Expression-Aware Reconstruction (MEAR) loss, which reweights the diffusion loss per pixel using frame-difference magnitudes and facial-landmark proximity, thereby enhancing cross-frame coherence in motion and expressions.
- 1
- 1
- Predicting Genetic Mutation from Whole Slide Images via Biomedical-Linguistic Knowledge Enhanced Multi-label Classification1
A Biological-knowledge enhanced PathGenomic multi-label Transformer to improve genetic mutation prediction performances and demonstrates that BPGT outperforms the state-of-the-art.
- Multi-scale adaptive networks for efficient inference1
A new inference framework, multi-scale adaptive networks (MSAN), is proposed, where several branches are added at different stages of the network, and a scalable attention as well as self-distillation are used to improve the performance of shallow branches.
- 1
- Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers1
This work reduces the search cost of AutoView to nearly zero by learning views and network parameters simultaneously in a single forward-backward step, minimizing and maximizing the mutual information among different augmented views, respectively, and proposes a self-regularized loss term to guarantee the information propagation.
- Chapter 4. Graph Reasoning Networks and Applications1
Through integrating knowledge graphs into neural networks, one can collaborate feature learning and graph reasoning with the same supervised loss function and achieve a more effective and interpretable way to introduce structure constraints.
- 1
- 1
- 1
- 1
- –
- –
- ViTWGAN: An Improved WGAN and Vision Transformer-Based Model for Intrusion Detection–
By improving the discriminator’s loss function, the approach reduces blind spots in the discriminator by explicitly reinforcing the learning of hard negative samples, thereby mitigating the forgetting of negative samples in the generative adversarial network.
- ComposeAnyone: Controllable Layout-to-Human Generation with Decoupled Multimodal Conditions–
This work introduces ComposeAnyone, a controllable layout-to-human generation method with decoupled multimodal conditions, which allows decoupled control of any part in hand-drawn human layouts using text or reference images, seamlessly integrating them during the generation process.
- –
- –
- Critique to Verify: Accurate and Honest Test-Time Scaling with RL-Trained Verifiers–
Mirror-Critique is introduced, a framework that trains a verifier with informative critiques to leverage the rich critique signal by contrasting model-generated solutions with ground-truth solutions and significantly outperforms majority voting in terms of solution accuracy.
- –
- Predicting Public Perception of Information Technology Enhanced Vocational Education Aid Using Random Forest Algorithm: A Case Study of Xinjiang 1+X+Y Model–
The Random Forest methodology demonstrated here offers scalable frameworks applicable to evaluating public perception toward diverse IT-integrated programs across developing regions, and provides actionable insights for policymakers to design targeted stakeholder engagement strategies emphasizing employment outcomes and equity considerations.
- –
- –
- –
- –
- –
- Smart runway management system based on large-scale ultra-weak FBG sensing array and its application–
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Neutral theory based mutation operator for Differential Evolutionary algorithms to enhance population diversity–
Comprehensive experimental results demonstrate that the presented NM operator could improve population diversity to some extent, especially when the population is not divergent at all.
- –
- Structured Deep Learning for Pixel-level Understanding–
This tutorial reviews recent progresses for pixel-level understanding with structured deep learning, including 1) human-centric analysis: human parsing and pose estimation; 2) part-based analysis: part and face parsing; 3) weakly-supervised analysis: object localization and semantic segmentation; 4) depth estimation: stereo matching.
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.