Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks245 from public data
- Graph Retrieval-Augmented Generation: A Survey647
This article provides the first comprehensive overview of GraphRAG methodologies, encompassing Graph-Based Indexing, Graph-Guided Retrieval, and Graph-Enhanced Generation, and outlines the core technologies and training methods at each stage.
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models447
This work proposes NaturalSpeech 3, a TTS system with novel factorized diffusion models to generate natural speech in a zero-shot way that outperforms the state-of-the-art TTS systems on quality, similarity, prosody, and intelligibility, and achieves on-par quality with human recordings.
- AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea261
AnyEdit is presented, a comprehensive multi-modal instruction editing dataset, comprising 2.5 million high-quality editing pairs spanning over 20 editing types and five domains, and shows that AnyEdit consistently boosts the performance of diffusion-based editing models.
- Rethinking the Bottom-Up Framework for Query-Based Video Localization189
It is argued that the performance of bottom-up framework is severely underestimated by current unreasonable designs, including both the backbone and head network, and designed a novel top-up model: Graph-FPN with Dense Predictions (GDP).
- HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data167
This work presents a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm, and uses this framework to identify and eliminate hallucinations in the training data automatically, and indicates that spurious correlations arising from long-tail object cooccurrences contribute to hallucinations.
- Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning156
The proposed Momentor, a Video-LLM capable of accomplishing fine-grained temporal understanding tasks, is designed and trained on Moment-10M, a large-scale video instruction dataset with segment-level instruction data, enabling it to perform segment-level reasoning and localization.
- HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation138
This work presents HealthGPT, a powerful Medical Large Vision-Language Model (Med-LVLM) that integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm through a novel heterogeneous low-rank adaptation technique and a tailored hierarchical visual perception approach.
- 136
- 123
- Fine-Grained Semantically Aligned Vision-Language Pre-Training111
This paper introduces LOUPE, a fine-grained semantically aLigned visiOn-langUage PrE-training framework, which learns fine- grained semantic alignment from the novel perspective of game-theoretic interactions, and proposes an uncertainty-aware neural Shapley interaction learning module.
- NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results106
This paper reviews the NTIRE 2020 challenge on real image denoising with focus on the newly introduced dataset, the proposed methods and their results, based on the SIDD benchmark.
- Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning105
A variational cross-graph reasoning framework that explicitly decomposes video and language into multiple structured hierarchies and learns fine-grained semantic correspondence among them is proposed and empirically finds that they fail to generalize to queries with novel combinations of seen words.
- 105
- A Study on ReLU and Softmax in Transformer94
A full ReLU architecture named ReLUFormer is proposed which performs better than the baseline Transformer on long sequence tasks such as document translation and FFN and key-value memory are equivalent, and thus the Transformer can be viewed as a memory network where FFNs and self-attention networks are both key- value memories.
- 89
- WorldGPT: Empowering LLM as Multimodal World Model80
WorldGPT is introduced, a generalist world model built upon Multimodal Large Language Model (MLLM) that acquires an understanding of world dynamics through analyzing millions of videos across various domains and is integrated with a novel cognitive architecture that combines memory offloading, knowledge retrieval, and context reflection.
- Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation80
This paper proposes a novel unsupervised reinforcement learning approach to learn transferable meta-skills from unannotated environments without any supervisory signals, and shows that this method learns transferable motor primitives for visual navigation.
- GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs79
This work generates and curates large-scale graph-summary pair data with the assistance of LLMs, and introduces a novel graph-summary pretraining method, combined with invariant learning, to enhance graph foundation models with strong cross-domain zero-shot transferability.
- Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction73
This paper proposes to conduct multi-instance learning with a novel Cross-relation Cross-bag Selective Attention (C2SA), which leads to noise-robust training for distant supervised relation extractor and employs the sentence-level selective attention to reduce the effect of noisy or mismatched sentences.
- Learning Dynamic Context Augmentation for Global Entity Linking72
A simple yet effective solution, called Dynamic Context Augmentation (DCA), for collective EL, which requires only one pass through the mentions in a document, and can cope with different local EL models as a plug-and-enhance module.
- Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis68
A sequential-level deep learning model is proposed to effectively discriminate the distinction and subtlety of infectious corneal disease via the classification of clinical images through the preservation of spatial structures and disentangle the informative features for clinical image classification of infectious keratitis.
- Learning Hybrid Behavior Patterns for Multimedia Recommendation65
A Hybrid Clustering Graph Convolutional Network (HCGCN) is proposed for multimedia recommendation that performs high-order graph convolutions inside user- item clusters and item-item clusters to capture various user behavior patterns and designs corresponding clustering losses to enhance user-item preference feedback and multimodal representation learning constraint to adjust the modality importance.
- CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction64
This paper proposes a novel contrastive instance learning (CIL) framework, which regards the initial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance.
- Consensus Graph Representation Learning for Better Grounded Image Captioning64
The Consensus Rraph Representation Learning framework (CGRL) is proposed for GIC that incorporates a consensus representation into the grounded captioning pipeline that can capture both the correct linguistic characteristics and visual relevance, and then grounding appropriate image regions further.
- Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World63
A Cross-modal prediCate boosting (CaCao) framework, where a visually-prompted language model is learned to generate diverse fine-grained predicates in a low-resource way for open-world predicate scene graph generation (Epic).
- 63
- Media and public reactions toward vaccination during the ‘hepatitis B vaccine crisis’ in China61
This study found the Kangtai vaccine crisis raised great public attention and negative sentiment toward vaccinations on the internet in China and the government should provide ways to address emerging public concerns after policy change to avoid misinformation and misunderstanding during such a vaccine crisis.
- EyeMap: a software system for visualizing and analyzing eye movement data in reading58
EyeMap has more advanced features for text stimulus presentation, interest area extraction, eye movement data visualization, and experimental variable calculation, and it is unique in supporting binocular data analysis for unicode, proportional, and nonproportional fonts and spaced and unspaced scripts.
- InstructVid2Vid: Controllable Video Editing with Natural Language Instructions56
Experimental results demonstrate that InstructVid2Vid is capable of generating high-quality, temporally coherent videos and performing diverse edits, including attribute editing, background changes, and style transfer, and underscores the versatility and effectiveness of the proposed method.
- Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment54
This paper considers learning a multi-modal embedding from the perspective of optimizing a pairwise ranking problem while enhancing both local alignment and global alignment and demonstrates the superiority of the proposed C2MLR due to its nature of multi- modal compositional embedding.
- A low rank structural large margin method for cross-modal ranking52
A general cross-modal ranking algorithm to optimize the listwise ranking loss with a low rank embedding, which is called Latent Semantic Cross-Modal Ranking (LSCMR) and shows significant improvements over the state-of-the-art methods.
- Continual Vision-Language Representation Learning with Off-Diagonal Information50
A new continual vision-language representation learning framework Mod-X is proposed: Maintain off-diagonal information-matriX, which improves the capability of the multi-modal model by maintaining the multi -modal representation space alignment on the old data domain during continuously fitting the new training data domain.
- Logistic Tensor Regression for Classification50
The proposed Logistic Tensor Regression not only reserves the underlying structural information embedded in data by tensorial representations, but also avoids overfitting by the introduction of a sparsity regularizer.
- 49
- GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning48
This work introduces an innovative deployment module coined as GraphControl, motivated by ControlNet, to realize better graph domain transfer learning by leveraging universal structural pre-trained models and GraphControl, which align the input space across various graphs and incorporate unique characteristics of target data as conditional inputs.
- Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer43
A Dynamic integrated uncertainty valuation framework (Detective) that comprehensively consider the domain shift between multi-source domains and target domain to detect the informative target samples and mitigating uncertainty miscalibration is proposed.
- Data Shunt: Collaboration of Small and Large Models for Lower Costs and Better Performance42
The proposed Data Shunt (DS), a general paradigm for collaboration of small and large models, which not only substantially reduces the cost associated with deploying large models but also effectively enhances overall performance.
- 42
- Auto-Encoding Morph-Tokens for Multimodal LLM41
This work proposes encoding images into morph-tokens to serve a dual purpose: for comprehension, they act as visual prompts instructing MLLM to generate texts; for generation, they take on a different, non-conflicting role as complete visual-tokens for image reconstruction, where the missing visual cues are recovered by the MLLM.
- 41
- Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models41
A novel Gradient-RegulAted Meta-prompt learning (GRAM) framework that jointly meta-learns an efficient soft prompt initialization for better adaptation and a lightweight gradient regulating function for strong cross-domain generalizability in a meta-learning paradigm using only the unlabeled image-text pre-training data.
- SGL-PT: A Strong Graph Learner with Graph Prompt Tuning41
A novel framework named SGL-PT is proposed which raises a strong and universal pre- training task coined as SGL that acquires the complementary merits of generative and contrastive self-supervised graph learning and unifies pre-training and fine-tuning.
- Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels40
This work introduces a novel task, Partial labeling and Long-Tailed Multi-Label Classification (PLT-MLC), and proposes an end-to-end learning framework, COMIC, which significantly outperforms general MLC, LT-MLC and PL-MLC methods in terms of effectiveness and robustness on newly created PLT-MLC datasets.
- Cross-Media Hashing with Neural Networks40
This work proposes a cross-media hashing approach based on multi-modal neural networks that achieves superior cross- media retrieval performance compared with the state-of-the-art methods.
- Disentangled Motif-aware Graph Learning for Phrase Grounding39
A novel graph learning framework for phrase grounding in the image with special attention to different motifs implied in the context of the scene graph and devise the disentangled graph network to integrate the motif-aware contextual information into representations.
- 38
- 37
- Dest-ResNet35
Dest-ResNet (Deep spatiotemporal Residual Network) is proposed for hotspot traffic speed prediction and shows a 30% relative boost over the state-of-the-art methods on real-world datasets from Baidu Map.
- Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion34
A novel strategy named Degeneration-Tuning (DT) is proposed to shield contents of unwanted concepts from SD weights by utilizing Scrambled Grid to reconstruct the correlation between undesired concepts and their corresponding image domain, and guide SD to generate meaningless content when such textual concepts are provided as input.
- STEP: Enhancing Video-LLMs’ Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training33
STEP, a novel graph-guided self-training method that enables Video-LLMs to generate reasoning-rich fine-tuning data from any raw videos to improve itself, achieves superior performance with a minimal amount of self-generated rationale-enriched training samples in both compositional reasoning and comprehensive understanding benchmarks, highlighting the broad applicability and vast potential.
- Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens33
This paper builds a proper visual language by leveraging diffusion timesteps to learn discrete, recursive visual tokens that recursively compensate for the progressive attribute loss in noisy images as timesteps increase, enabling the diffusion model to reconstruct the original image at any timestep.
- Improving Weakly Supervised Object Localization via Causal Intervention33
The proposed method, dubbed CI-CAM, explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps thus improving the accuracy of object localization.
- Generating Natural Language Adversarial Examples on a Large Scale with Generative Models32
This paper proposes an end to end solution to efficiently generate adversarial texts from scratch using generative models, which are not restricted to perturbing the given texts, and calls it unrestricted adversarial text generation.
- 30
- 30
- Supervised Nonnegative Tensor Factorization with Maximum-Margin Constraint30
The proposed Supervised Non-negative Tensor Factorization with Maximum-Margin Constraint (SNTFM2) formulates the optimal discriminative factorization of non-negative tensorial data as a coupled least-squares optimization problem via a maximum-margin method.
- On Path to Multimodal Generalist: General-Level and General-Bench29
This project introduces General-Level, an evaluation framework that defines 5-scale levels of MLLM performance and generality, offering a methodology to compare MLLMs and gauge the progress of existing systems towards more robust multimodal generalists and, ultimately, towards AGI.
- Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding28
A variational cross-graph reasoning framework is proposed that explicitly decomposes video and language into hierarchical semantic graphs, respectively, and learns fine-grained semantic correspondence between the two graphs, and introduces a novel adaptive structured semantics learning approach to derive the structure-informed and domain-generalizable graph representations.
- Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference28
An adaptive hierarchical graph network is proposed that achieves in-depth understanding of the video over complex interactions andsemantic coherence learning is introduced to explicitly encourage the semantic coherence of the adaptive hierarchicalgraph network from three hierarchies.
- Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration27
UniKE is proposed, a novel multimodal editing method that establishes a unified perspective and paradigm for intrinsic knowledge editing and external knowledge resorting, and further promotes knowledge collaboration by disentangling the knowledge representations into the semantic and truthfulness spaces.
- Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images27
This work designs a depth decoupling convolutional neural network (DDCNN), which contains a depth estimation branch and a saliency detection branch, and introduces a consistency loss on the intermediate attention and saliency maps for the unlabeled data, as well as a supervised depth and Saliency loss for labeled data.
- 27
- Tell and guess: cooperative learning for natural image caption generation with hierarchical refined attention26
This paper designed the “Tell and Guess” Cooperative Learning model with a Hierarchical Refined Attention mechanism (CL-HRA) that bidirectionally improves the performance to generate more informative captions.
- Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training25
This paper proposes collaborative adversarial training to improve the data utilization, which coordinates virtual adversarialTraining (VAT) and adversarial Training (AT) at different levels, and employs the instance-level VAT to recycle instances abandoned by MIL.
- Informative Visual Storytelling with Cross-modal Rules25
This work proposes a method to mine the cross-modal rules to help the model infer these informative concepts given certain visual input, and leverages these concepts in the authors' encoder-decoder framework with the attention mechanism.
- 24
- 24
- 24
- Walking with MIND23
A novel Mental Imagery eNhanceD (MIND) module for the embodied agent, as well as a relevant deep reinforcement framework for training, which proves that the agent with the MIND module is superior to its counterparts not only in EQA performance but in many other aspects such as route planning, behavioral interpretation, and the ability to generalize from a few examples.
- LSTM-in-LSTM for generating long descriptions of images23
This model outperforms state-of-theart methods on several benchmark datasets when used to generate long rich fine-grained descriptions of given images in terms of four different metrics (BLEU, CIDEr, ROUGE-L, and METEOR).
- HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models22
HyperLLaVA is introduced, which involves adaptive tuning of the projector and LLM parameters, in conjunction with a dynamic visual expert and language expert, respectively, derived from HyperNetworks, enabling dynamic projector and LLM modeling in two-stage training.
- Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering22
This work proposes to regularize distantly supervised models with Compact Latent Space Clustering (CLSC) to bypass the problem of dealing with false entity type labels, and effectively utilize noisy data yet.
- Flickr group recommendation with auxiliary information in heterogeneous information networks22
The group recommendation in this paper is formulated as a non-negative matrix factorization (NMF) method regularized with user–user similarity via visual features and heterogeneous information networks, and experiments show that the proposed approach outperforms other counterpart recommendation approaches.
- The classification of multi-modal data with hidden conditional random field22
The experimental results show the effectiveness of the proposed Multi-modal Hidden Conditional Random Field (M-HCRF), which represents the intrinsical structure shared by different modalities as hidden variables in a undirected general graphical model when learning the latent shared structure of the multi- modal data.
- 22
- 21
- Hierarchical Attention Based Spatial-Temporal Graph-to-Sequence Learning for Grounded Video Description21
Two ways to construct a sequence graph that captures spatial-temporal correlations among different objects in each frame are exploited and a novel graph topology refinement technique is presented to discover optimal underlying graph structure.
- MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning20
This work proposes a Model-Agnostic Recipe for Improving OOD generalizability of unsupervised graph contrastive learning methods, which is referred to as MARIO, and introduces two principles aimed at developing distributional-shift-robust graph contrastive methods to overcome the limitations of existing frameworks.
- Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model20
Dancing Avatar is proposed, designed to fabricate human motion videos driven by poses and textual cues, that exhibits the capacity to generate human videos with markedly superior quality, both in terms of human and background fidelity, as well as temporal coherence compared to existing state-of-the-art approaches.
- 19
- Temporality-enhanced knowledgememory network for factoid question answering19
This paper devise an architecture named the temporality-enhanced knowledge memory network (TE-KMN) and apply the model to a factoid question answering dataset from a trivia competition called quiz bowl and demonstrates that the method achieves better performance than several state-of-the-art methods.
- Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization18
SUPMER leverages self-supervised meta-learning with a diverse set of well-designed meta-training tasks to learn a universal prompt initialization for efficient adaptation using only unlabeled data, and jointly meta-learns a gradient regularization function to transform raw gradients into a domain-generalizable direction, thus alleviating the problem of overfitting.
- On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals18
The validated techniques are combined and improve the baseline performances of five small architectures with considerable margins, which indicates that training small self-supervised contrastive models is feasible even without distillation signals.
- Video question answering via grounded cross-attention network learning18
A novel hierarchical cross-attention mechanism of mutual attention learning for video question answering, named as GCANet is proposed, which helps to learn the joint question-video representation based on both rough representation and grounded representation of video for video questioning.
- Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration17
This work extensively study how LLMs communicate with AIGC model to achieve more controllable image generation and makes the first attempt to collaborate them for automatic data augmentation for a variety of downstream tasks.
- Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos17
A cross-modal consensus learning paradigm is introduced, which leverages the inherent semantic consensus between the aligned video and subtitle to achieve weakly-supervised learning on temporal emotion localization.
- 16
- FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL15
This work proposes FocusDiff, which enhances fine-grained text-image semantic alignment by focusing on subtle differences between similar text-image pairs, and achieves state-of-the-art performance on existing text-to-image benchmarks and significantly outperforms prior methods on PairComp.
- Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program15
Experimental results demonstrate that SVIP-Reward improves MLLM performance across training and inference-time scaling, yielding better results on benchmarks while reducing hallucinations and enhancing reasoning ability.
- De-fine: De composing and Re fin ing Visual Programs with Auto-Feedback14
Inspired by benders decomposition, this work introduces De-fine, a training-free framework that automatically decomposes complex tasks into simpler subtasks and refines programs through auto-feedback, which can improve logical reasoning performance by integrating the strengths of multiple models.
- 14
- Hashing with List-Wise learning to rank14
In this paper, the hashing problem is considered from the perspective of optimizing a list-wise learning to rank problem and an approach called List-Wise supervised Hashing (LWH) is proposed, which obtains a significant improvement over the state-of-the-art hashing approaches due to both structural large margin and list- Wise ranking pursuing in a supervised manner.
- Supervised cross-collection topic modeling14
The results suggest that the proposed scLDA can generate meaningful collection-specific topics and achieves better retrieval accuracy than other related topic models.
- 13
- Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile13
Eigen-Reptile (ER) is presented that updates the meta-parameters with the main direction of historical task-specific parameters to alleviate sampling and label noise and Introspective Self-paced Learning (ISPL) is proposed, to obtain a more accurate main direction for Eigen- reptile in the presence of many noisy labels.
- Mastering Collaborative Multi-Modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativeness12
A collaborative framework, DataTailor, is proposed, which leverages three key principles,formativeness, uniqueness, and representativeness, for effective data selection and achieves 101.3% of the performance of full-data fine-tuning with only 15% of the data, significantly reducing computational costs while maintaining superior results.
- Improving Vision Anomaly Detection With the Guidance of Language Modality12
This paper proposes Cross-modal Guidance (CMG), comprising of Cross-modal Entropy Reduction (CMER) and Cross-modal Linear Embedding (CMLE), to address the issues of redundant information and sparse latent space, respectively.
- Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales12
Fact is introduced, a novel paradigm designed to generate multimodal rationales that are faithful, concise, and transferable for teaching MLLMs and filter rationales that can be transferred to end-to-end paradigms from programming paradigms to guarantee transferability.
- Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning11
This paper proposes to enable the collaborative co-evolution of visual comprehension and generation, advancing image generation into an iterative introspective process and introduces a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT for visual generation, while reinforcement learning activates its full potential via an exploration-exploitation trade-off.
- LLAUS: A High-Quality Instruction-Tuned Large Vision Language Assistant for UltraSound11
This study introduces a high-quality instruction-tuned Large vision Language Assistant for UltraSound (LLAUS), designed to answer questions about medical ultrasound images, aiming to assist clinicians in impoverished areas to improve the provision of healthcare services.
- 11
- Multi-modal Sequence to Sequence Learning with Content Attention for Hotspot Traffic Speed Prediction11
This work proposes Multi-Seq2Seq-Att, a multi-modal sequence learning model that deals with two sequences in different modalities, namely, the query sequence and the traffic speed sequence, and addresses the modality gap between queries and the Traffic speed.
- Disambiguating named entities with deep supervised learning via crowd labels11
A crowd model is devised to elicit the underlying features (crowd features) from crowd labels that indicate a matching candidate for each mention, and then use the crowd features to fine-tune a dynamic convolutional neural network (DCNN).
- 11
- What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities10
OmniBench is introduced, a self-generating, cross-platform, graph-based benchmark with an automated pipeline for synthesizing tasks of controllable complexity through subtask composition and OmniEval, a multidimensional evaluation framework that includes subtask-level evaluation, graph-based metrics, and comprehensive tests across 10 capabilities.
- 10
- Alleviate Dataset Shift Problem in Fine-grained Entity Typing with Virtual Adversarial Training10
This work proposes a novel regularization module based on virtual adversarial training (VAT), which first uses a self-paced sample selection function to select suitable samples for VAT, then constructsvirtual adversarial perturbations based on the selected samples, and finally regularizes the model to be robust to such perturbation.
- Sketch the storyline with CHARCOAL: a non-parametric approach10
The adaptation of non-parametric nature in CHARCOAL makes the model can adaptively learn the appropriate number of stories and topics from news corpus and demonstrates both interpretability and superiority of the proposed approach.
- WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing9
WiseEdit is introduced, a knowledge-intensive benchmark for comprehensive evaluation of cognition- and creativity-informed image editing, featuring deep task depth and broad knowledge breadth and incorporating three fundamental types of knowledge: Declarative, Procedural, and Metacognitive knowledge.
- Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms9
This paper aims to bridge the theoretical and practical aspects of GVAs, concluding those that operate in environments closely mirroring the real world are more likely to demonstrate human-like intelligence.
- NITE: A Neural Inductive Teaching Framework for Domain Specific NER9
It is proved that without using any labeled data, NITE can significantly boost the performance of a CNN-bidirectional LSTM-CRF NER neural network nearly over 30% in terms of F1-score.
- Jointly Discovering Fine-grained and Coarse-grained Sentiments via Topic Modeling9
The proposed MgS-LDA not only discovers the topics in social media, but also identifies opinions about a given topic in terms of fine-grained and coarse- grained sentiment.
- Logic Distillation: Learning from Code Function by Function for Decision-making Tasks8
Experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in continuous decision-making tasks, comparable to, or even surpassing, those of L-LLMs.
- Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining8
Iris is introduced, a foundational visual agent that addresses challenges in visual perception through two key innovations: Information-Sensitive Cropping (ISC) and Self-Refining Dual Learning (SRDL).
- 8
- 8
- 8
- 8
- 8
- 8
- Image Ranking via Attribute Boosted Hypergraph8
In this paper, hypergraph is used to integrate the visual attributes with low-level features of images and a ranking algorithm is performed on the hypergraph to boost the result of image ranking.
- WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation7
The first suite for in-context interleaved cross-modality comprehension and generation for multi-modal community is presented, and it is believed WEAVE provides a view and foundation for studying in-context interleaved comprehension and generation for multi-modal community.
- I3: I ntent- I ntrospective Retrieval Conditioned on I nstructions7
I3 is introduced, a unified retrieval system that performs Intent-Introspective retrieval across various tasks, conditioned on Instructions without any task-specific training, embodying two key designs: progressive structure pruning and drawback extrapolation-based data refinement.
- 7
- 7
- Learning Deep Contextual Attention Network for Narrative Photo Stream Captioning7
A novel deep contextual attention network (CAN) is proposed to narratively describe photo streams by jointly exploring the rich context among attended regions and the coherence in sentences by exploiting visual attention and context in the photo stream.
- 7
- Evolving Generalist Virtual Agents with Generative and Associative Memory6
CA3Mem, a framework inspired by the human hippocampus that organizes experiences into a structured memory graph, features two key innovations: a generative memory recombination mechanism that synthesizes novel solutions to drive agent evolution and an associative retrieval algorithm that employs spreading activation to recall a comprehensive and contextually-aware set of experiences.
- 6
- 6
- 6
- 6
- BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrieval6
A novel Bottom-up crOss-modal Semantic compoSition (BOSS) with Hybrid Counterfactual Training framework, which sheds new light on the CIR task by studying it from two previously overlooked perspectives.
- Photo Stream Question Answer6
This paper presents a new visual question answering (VQA) task -- Photo Stream QA, which aims to answer the open-ended questions about a narrative photo stream, and proposes an end-to-end baseline (E-TAA), which provides promising results outperforming all the other baseline methods.
- 6
- Entity mention aware document representation6
Experimental results demonstrate that the proposed approach to learning low-dimensional vector representations of documents is able to significantly improve text clustering, text classification performance and outperform previous studies on the TAC-KBP entity linking benchmark.
- EvidenceMap: Learning evidence analysis to unleash the power of small language models for biomedical question answering5
This study proposes EvidenceMap, which aims to enable a lightweight pre-trained language model to explicitly learn multiple aspects of biomedical evidence, including supportive evaluation, logical correlation and content summarization, thereby latently guiding a generative model (around 3B parameters) to provide textual responses.
- Improving large models with small models: Lower costs and better performance5
DS+ not only substantially reduces the cost associated with querying large models but also effectively improves large models' performance, and experiments prove that the proposed collaborative-based paradigm can better inject specific task knowledge into PLMs compared to fine-tuning.
- 5
- Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference5
A new double-hints guided Graph-to-Sequence learning framework is proposed, which first models them as a dynamic graph and learns the implicit topology end-to-end, and then utilizes a graph-to-sequence model to generate the questions with double hints.
- Attribute-driven streaming edge partitioning with reconciliations for distributed graph neural network training5
A novel distributed graph training approach, attribute-driven streaming edge partitioning with reconciliations (ASEPR), where the local model loads only the subgraph stored on its own machine to make fewer communications.
- KICE: A Knowledge Consolidation and Expansion Framework for Relation Extraction5
KICE, a Knowledge-evolving framework by Iterative Consolidation and Expansion with the guidance of PLMs and rule-based patterns, which solicits human-enlightened understanding and expands the knowledge coverage for better task model training.
- Lformer: Text-to-Image Generation with L-shape Block Parallel Decoding5
Lformer, a semi-autoregressive text-to-image generation model that achieves faster speed than the existing transformer-based methods while keeping good generation quality and can edit images without the requirement for finetuning.
- 4
- Robust Modality-Incomplete Anomaly Detection: A Modality-Instructive Framework with Benchmark4
RADAR, a robust two-stage Robust modAlity-instructive fusing & Detecting frAmewoRk, is proposed, which integrates a Modality-Incomplete Instruction mechanism-guiding the multimodal Transformer to focus more on available modal info, and a Double-Pseudo Hybrid Module to highlight unique modality combinations and reduce overfitting.
- Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts4
A novel Chart Hypothetical Question Answering (HQA) task is introduced, which imposes assumptions on the same question to compel models to engage in counterfactual reasoning based on the chart content, and HAI, a human-AI interactive data synthesis approach that leverages the efficient text-editing capabilities of LLMs alongside human expert knowledge to generate diverse and high-quality HQA data at a low cost.
- 4
- E-CGL: an efficient continual graph learner4
Results indicate that E-CGL not only effectively manages correlations between different graph data during continual training but also enhances efficiency in large-scale CGL, which is a crucial paradigm for learning from sequential data while retaining previous knowledge.
- 4
- DEMON24: ACM MM24 Demonstrative Instruction Following Challenge4
The DEMON Challenge aims to assess the ability of models and systems to comprehend demonstrative instructions consisting of multiple, interleaved, and multimodal context that demonstrate the required information to complete a task.
- 4
- 4
- Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of Semi-Supervised Learning and Active Learning4
An innovative Inconsistency-based virtual aDvErsarial Active Learning (IDEAL) algorithm to further investigate SSL-AL's potential superiority and achieve mutual enhancement of AL and SSL, i.e., SSL propagates label information to unlabeled samples and provides smoothed embeddings for AL, while AL excludes samples with inconsistent predictions and considerable uncertainty for SSL.
- 4
- Hybrid embedding and joint training of stacked encoder for opinion question machine reading comprehension4
A novel framework based on neural networks is proposed to address problem of imbalance of the dataset, irrelevancy of question and passage is used for data augmentation and the proposed method achieves state-of-the-art performance.
- 4
- 3
- Graft: Integrating the Domain Knowledge via Efficient Parameter Synergy for MLLMs3
This work proposes a unified parameter integration framework that enables modular composition of expert capabilities and introduces a domain compatibility scoring mechanism that quantifies inter-expert alignment at the activation level and correlates with downstream task utility.
- EvolvedGRPO: Unlocking Reasoning in LVLMs via Progressive Instruction Evolution3
A progressive instruction evolution framework to gradually generate more complex questions via editing instructions in an adversarial way, progressively aligned with the model’s evolving capabilities, which achieves state-of-the-art performance among open-source RL models on multi-modal reasoning tasks, even approaching the closed-source GPT-4o in reasoning capabilities.
- LASER: Tuning-Free LLM-Driven Attention Control for Efficient Text-conditioned Image-to-Animation3
This paper introduces LASER, a tuning-free LLM-driven attention control framework that follows a progressive process: LLM planning, feature-attention injection, and stable animation generation, and proposes a Text-conditioned Image-to-Animation Benchmark to validate the effectiveness and efficacy of LASER.
- Boosting Private Domain Understanding of Efficient MLLMs: A Tuning-free, Adaptive, Universal Prompt Optimization Framework3
This paper focuses on the efficient adaptation of EMLLMs to private domains, specifically in two areas: how to reduce data requirements, and how to avoid parameter fine-tuning.
- 3
- FedAA: Using Non-sensitive Modalities to Improve Federated Learning while Preserving Image Privacy3
Experiments show that FedAA significantly outperforms recent non-IID federated learning algorithms while preserving image privac, by sharing information from non-sensitive modalities.
- 3
- 3
- ENCORE3
A novel neural framework, named ENCORE (External Neural COnstraints REgularized distant supervision), which allows an integration of other information for standard DS through regularizations under multiple external neural networks.
- The ZHI-EDL System for Entity Discovery and Linking at TAC KBP 2017.3
This report gives a detailed description of ZHI(知)-EDL system of team ‘rise_dcd_zju’, which submitted to the TAC KBP 2017 Trilingual Entity Detection and Linking (EDL) track.
- Flickr group recommendation via heterogeneous information networks3
The group recommendation in this paper is formulated as a non-negative matrix factorization (NMF) method regularized with user-user similarity via heterogeneous information networks, and experiments show that the proposed approach outperforms other counterpart recommendation techniques.
- πLDA3
A model called πLDA is proposed, which introduces a latent variable π into LDA, a traditional topic model, to capture the characteristic of each segment unit to determine whether a segment unit is assigned (or selected) to the topic embedded in its corresponding segment.
- 2
- MedQuery: A Graph-Driven Medical Literature-Enhanced Query Answering System2
This study introduces MedQuery, a multimodal medical knowledge query-answering system that integrates query-based literature retrieval with a response generation module capable of reasoning across multimodal data.
- Arcadia: Toward a Full-Lifecycle Framework for Embodied Lifelong Learning2
Arcadia delivers consistent gains on navigation and manipulation benchmarks and transfers robustly to physical robots, indicating that a tightly coupled lifecycle: continuous real-world data acquisition, generative simulation update, and shared-representation learning, supports lifelong improvement and end-to-end generalization.
- 2
- Fast Thinking for Large Language Models2
GainRouter is proposed, a lightweight routing mechanism that adaptively switches between fast codebook guided inference and slow explicit reasoning, thereby suppressing overthinking and reducing unnecessary token generation in large language models.
- Align2LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation2
A novel instruction curation algorithm, derived from two unique perspectives, human and LLM preference alignment, to compress this vast corpus of machine-generated multimodal instructions to a compact and high-quality form.
- 2
- Language Model is a Branch Predictor for Simultaneous Machine Translation2
This work utilizes a language model as a branch predictor to predict potential branch directions, namely, future source words, and utilizes the predicted source words to decode the output in advance to reduce translation latency.
- 2
- Distilling Task-specific Logical Rules from Large Pre-trained Models2
A novel framework STREAM to distill task-specific logical rules from large pre-trained models is developed based on the formed high-quality instance pool that acts as an intermediary role and keeps teaching the expert to fit the authors' task and learning task- specific logical rules.
- MGD-GAN: Text-to-Pedestrian Generation Through Multi-grained Discrimination2
The Multi-Grained Discrimination enhanced Generative Adversarial Network (MGD-GAN) is proposed, that capitalizes a human-part-based Discriminator (HPD) and a self-cross-attended (SCA) global Discriminators in order to capture the coherence of the complex body structure.
- 2
- 2
- 2
- 2
- Neural-DINF: A Neural Network based Framework for Measuring Document Influence2
This paper uses both frequency changes and word semantic shifts to measure document influence by developing a neural network framework and shows that the model outperforms DIM.
- KCAT: A Knowledge-Constraint Typing Annotation Tool2
An efficient Knowledge Constraint Fine-grained Entity Typing Annotation Tool is proposed, which further improves the entity typing process through entity linking together with some practical functions.
- 2
- Reading Document and Answering Question via Global Attentional Inference2
A novel global attentional inference (GAI) neural network architecture is proposed, which learns useful cues from structural knowledge via a dynamically terminated multi-hop inference mechanism, to answer cloze-style questions.
- Ad Recommendation for Sponsored Search Engine via Composite Long-Short Term Memory2
This paper addresses ad recommendation problem that finding and ranking the most relevant ads with respect to users' search queries by utilizing the long-short term memory (LSTM) network to effectively encode two kinds of sequences: the (user, query) sequence and the query word sequence to represent users' query intention in a continuous vector space and decode them as distributions over ads respectively.
- 2
- 2
- Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs1
The proposed MIND (Meta-cognitive INtegrated Dynamic Knowledge Editing), a framework that constructs a meta-knowledge memory for self-awareness, employs game-theoretic interactions to monitor knowledge activation, and incorporates label refinement for noise-robust updates, significantly outperforms existing cognitive editing approaches.
- 1
- GMFVAD: Using Grained Multi-modal Feature to Improve Video Anomaly Detection1
More grained multi-modal feature based on the video snippet, which summarizes the main content, and text features based on the captions of original video will be introduced to further enhance the visual features of highlighted portions to reduce the redundancy in visual features.
- Counterfactual Evolution of Multimodal Datasets via Visual Programming1
Experiments show that SCOPE improves reasoning performance, exposes model blind spots, and enhances visual dialog capabilities, and is proposed to support training, a curriculum learning strategy that aligns model capacity with sample difficulty.
- 1
- 1
- DuetRAG: Collaborative Retrieval-Augmented Generation1
This work proposes a novel Collaborative Retrieval-Augmented Generation framework, DuetRAG, to simultaneously integrate the domain fintuning and RAG models to improve the knowledge retrieval quality, thereby enhancing generation quality.
- Reasoning Makes Good Annotators : An Automatic Task-specific Rules Distilling Framework for Low-resource Relation Extraction1
This work guides the pre-trained language model to reason rules as experts and compose them into robust compound rules for data labeling, which could continuously enrich the rules set to power the labeling ability by discovering reliable model-labeled data for distinguishable rules generation.
- 1
- 1
- Citation Trajectory Prediction via Publication Influence Representation Using Temporal Knowledge Graph1
CTPIR is a new citation trajectory prediction framework that is able to represent the influence (the momentum of citation) of either new or existing publications using the history information of all their attributes, composed of three modules: difference-preserved graph embedding, fine-grained influence representation, and learning-based trajectory calculation.
- 1
- 1
- 1
- 1
- Visual knowledge: an attempt to explore machine creativity1
This paper discusses the issues of visual knowledge representation and its potential applications to machine creativity, and enumerates related research on imagery-thinking-based reasoning, then focuses on a special type of visualknowledge representation, i.e., visual scene graph, and reviews the problem of visual scenegraph construction and its possible applications in detail.
- 1
- Sentences Embedding for Slot Filling via Convolutional Neural Networks.1
The DCD slot filling system for the TAC Cold Start evaluations 2016 Task mainly uses neural networks methods to obtain the slot filler and applies Stanford Core NLP system for Named Entity Recognize and coreference.
- 1
- The ZJU-EDL System for Entity Discovery and Linking at TAC KBP 2015.1
A fully pipelined EDL system implemented by ZJU-DCD-EDL team for the TAC 2015 EDL (Entity Discovery and Linking) task, which mainly focuses on the linking of English name mentions, is described.
- Combining MIML and Distant Supervision for KBP Slot Filling.1
This paper describes the DCD slot filling system for the TAC Cold Start evaluations 2015 Task and uses the Multi Instances Multi Label method, which is pretty suitable for the nature of the training data generated by distant supervision.
- Multi-modal Retrieval via Deep Textual-Visual Correlation Learning1
The experimental results indicate the effectiveness of the proposed Deep Textual-Visual correlation learning (DTV) when applied to multi-modal retrieval.
- 1
- Research of Military Software Quality Characteristic and Design Attribute1
The definition of SVTM, engineering significance and a capability maturity model are given, to overcome the difficulties of examination, test and service the military users with finalized software products in the process of finalizing the military software products.
- Virtual Battlefield Attack-Defense Countermeasure Simulation on the Grid1
The design of a system supporting execution of HLA-based Parallel and Distributed Simulation (PADS) on the Grid Computing Environment is presented and a model of Virtual Battlefield Attack-Defense Countermeasure Simulation (VBADCS) onThe Grid is presented.
- –
- –
- –
- –
- –
- –
- TRIDENT: Efficient Small-Large Model Collaboration via Heterogeneous Expert Decoupling–
This paper proposes Tri-gate Routing for Inference via Decoupled Efficient Network Technologies (TRIDENT), a highly efficient and robust heterogeneous collaborative inference framework that provides a robust and interpretable pathway for efficient collaborative intelligence.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Posterior-regularized REINFORCE for Instance Selection in Distant Supervision–
This paper adopts posterior regularization (PR) to integrate some domain-specific rules in instance selection using REINFORCE and remarkably improves the performance of the relation classifier trained on cleaned distant supervision dataset as well as the efficiency of the REIN FORCE training.
- –
- Detecting Temporal Proposal for Action Localization with Tree-structured Search Policy–
A reinforcement learning based method, which trains a localizer that learns a search policy that finds an optimal search path to locate a salient proposal based on the currently observing video segment in a tree structure, therefore reduces the number of video segments fed into the proposal detector.
- The Y_dcd_zju Slot Filling System for TAC KBP 2017.–
This report describes the Y dcd zju Slot Filling System for the TAC KBP 2017 Cold Start slot Filling task, which uses text-CNN (Convolutional Neural Network) model to detect sentences that contain pre-defined relations, then the NER tag to identify slot fillers.
- The ijk System for EAL at TAC KBP 2016 Event Track.–
Experimental results indicate that the joint event extraction model using both common features and sparse vectors performs better and also makes use of promising distributed representations of words and transform these representations into sparse vectors through sparse coding.
- ZJU Participation in TAC 2016 EDL task.–
The modifications made to the last year’s system and the neural network approach for entity discovery are introduced and the results of some experiments conducted are demonstrated.
- –
- An informatics system for exploringeye movements in reading–
An informatics system with two major software suites: Experiment Executor and EyeMap designed specifically for reading research to accelerate the life cycle of a reading experiment from design through analysis.
- The study of letter-oriented hashing schemes–
Two minimal perfect hashing schemes suitable for letter-oriented keys are proposed, one based on the Chinese remainder theorem, and the other based on Ghosh's rank-hashing function.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.