Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Suggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works92 from public data
- UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models380
The UnifiedSKG framework is proposed, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset.
- Episodic Memory in Lifelong Language Learning350
This work proposes an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier.
- ZeroGen: Efficient Zero-shot Learning via Dataset Generation317
It is argued that ZeroGen can also provide useful insights from the perspective of data-free model-agnostic knowledge distillation, and unreferenced text generation evaluation.
- 302
- A Dependency Parser for Tweets221
A new dependency parser for English tweets, TWEEBOPARSER, which builds on several contributions: new syntactic annotations for a corpus of tweets, with conventions informed by the domain; adaptations to a statistical parsing algorithm; and a new approach to exploiting out-of-domain Penn Treebank data.
- OS-Copilot: Towards Generalist Computer Agents with Self-Improvement220
OS-Copilot is introduced, a framework to build generalist agents capable of interfacing with comprehensive elements in an operating system (OS), including the web, code terminals, files, multimedia, and various third-party applications, and FRIDAY, a self-improving embodied agent for automating general computer tasks.
- DetGPT: Detect What You Need via Reasoning153
This paper introduces a new paradigm for object detection that is called reasoning-based object detection, and leverages state-of-the-art multi-modal models and open-vocabulary object detectors to perform reasoning within the context of the user's instructions and the visual scene.
- M$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning144
Ying-VLM, a VLM model trained on the M$^3$IT dataset, is developed, showcasing its potential to answer complex questions requiring world knowledge, generalize to unseen video tasks, and comprehend unseen instructions in Chinese.
- 133
- Language Models Can See: Plugging Visual Controls in Text Generation128
A training-free framework for plugging in visual controls in the generation process and enabling LMs to perform multimodal tasks (e.g., image captioning) in a zero-shot manner, which outperforms the state-of-the-art method by notable margins with a nearly 27 times decoding speedup.
- 117
- TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos114
This work introduces TimeChat-Online, a novel online VideoLLM that revolutionizes real-time video interaction and presents TimeChat-Online-139K, a comprehensive streaming video dataset featuring diverse interaction patterns including backward-tracing, current-perception, and future-responding scenarios.
- 110
- Adaptive Semiparametric Language Models107
A language model that combines a large parametric neural network with a non-parametric episodic memory component in an integrated architecture is presented and a gating function to adaptively combine multiple information sources to make a prediction is designed.
- Audio-Visual Segmentation with Semantics104
A new baseline method is proposed that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process, and a regularization loss is designed to encourage audio-visual mapping during training.
- VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models102
VL-RewardBench is introduced, a comprehensive benchmark spanning general multimodal queries, visual hallucination detection, and complex reasoning tasks and uncovered three critical insights for improving VL-GenRMs: models predominantly fail at basic visual perception tasks rather than reasoning tasks.
- 99
- Lemur: Harmonizing Natural Language and Code for Language Agents98
Lemur and Lemur-Chat are introduced, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents, demonstrating balanced proficiencies in both domains, unlike existing open-source models that tend to specialize in either.
- GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning97
Experimental results demonstrate that GIMLET significantly outperforms molecule-text baselines in instruction-based zero-shot learning, even achieving closed results to supervised GNN models on tasks such as toxcast and muv.
- Segmental Recurrent Neural Networks for End-to-End Speech Recognition84
Practical training and decoding issues as well as the method to speed up the training in the context of speech recognition are discussed, and the model is self-contained and can be trained end-to-end.
- 72
- ImgTrojan: Jailbreaking Vision-Language Models with ONE Image69
This paper proposes a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions, and designs two metrics to quantify the success rate and the stealthiness of the attack.
- Why Does the Effective Context Length of LLMs Fall Short?69
Experimental results show that without additional training, STRING dramatically improves the performance of the latest large-scale models, such as Llama3.1 70B with \method, and achieves better performance than GPT-4-128K and clearly surpasses Claude 2 and Kimi-chat.
- 64
- Vicinity Vision Transformer58
Vicinity Attention, which is a type of linear attention that integrates 2D locality that achieves state-of-the-art image classification accuracy with 50% fewer parameters than previous approaches, is proposed.
- Toeplitz Neural Network for Sequence Modeling55
A lightweight sub-network called relative position encoder is proposed to generate relative position coefficients with a fixed budget of parameters, enabling the proposed Toeplitz neural network to deal with varying sequence lengths.
- 55
- In-Context Learning with Many Demonstration Examples48
A long-range language model EVALM based on an efficient transformer mechanism that can achieve higher performance with more demonstrations under many-shot instruction tuning, and further extending the length of instructions can further improve the upper bound of scaling in-context learning.
- Better Document-Level Machine Translation with Bayes’ Rule48
It is shown that Bayes’ rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents a compelling benefit because parallel documents are not always available.
- ABC: Attention with Bounded-memory Control42
This work shows that disparate approaches can be subsumed into one abstraction, attention with bounded-memory control (ABC), and it outperforms previous efficient attention models; compared to the strong transformer baselines, it significantly improves the inference time and space efficiency with no or negligible accuracy loss.
- Temporal Reasoning Transfer from Text to Video41
This diagnostic study reveals that video representations contain sufficient information for even small probing classifiers to achieve perfect accuracy, and introduces the Textual Temporal reasoning Transfer (T3), validating the efficacy of transferring temporal reasoning abilities from text to video domains.
- Syntactic Structure Distillation Pretraining for Bidirectional Encoders38
A knowledge distillation strategy for injecting syntactic biases into BERT pretraining, by distilling the syntactically informative predictions of a hierarchical—albeit harder to scale—syntactic language model.
- CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration35
This paper raises the question: ``Do the MLLMs possess safety-awareness against malicious image inputs?", and introduces a simple yet effective technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution.
- A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios34
This survey systematically review existing research on LLM-based social agents within game-theoretic scenarios and provides insights to advance the development and evaluation of social agents in game-theoretic scenarios.
- Transforming Dependencies into Phrase Structures34
This work presents a new algorithm for transforming dependency parse trees into phrase-structure parse trees that is faster than traditional phrasestructure parsing and achieves near the state of the art on both benchmarks.
- 30
- 30
- Fine-grained Audible Video Description28
An audio-visual-language transformer that extends existing video captioning model with an additional audio branch is proposed, and the masked language modeling and auto-regressive language modeling losses are combined to optimize the model so that it can produce paragraph-level descriptions.
- End-to-End Neural Segmental Models for Speech Recognition27
This work reviews neural segmental models, which can be viewed as consisting of a neural network-based acoustic encoder and a finite-state transducer decoder and explores training approaches, including multistage versus end-to-end training and multitask training that combines segmental and frame-level losses.
- Non-myopic Generation of Language Models for Reasoning and Planning26
This paper revisits LLM reasoning from an optimal-control perspective, proposing a novel method, Predictive-Decoding, that leverages Model Predictive Control to enhance planning accuracy and mitigate early errors and promote non-myopic planning.
- 24
- PromptCoT: Synthesizing Olympiad-level Problems for Mathematical Reasoning in Large Language Models21
This work introduces PromptCoT, a novel approach for automatically generating high-quality Olympiad-level math problems, and demonstrates that PromptCoT exhibits superior data scalability, consistently maintaining high performance as the dataset size increases, outperforming the baselines.
- BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoning21
BioMaze is introduced, a dataset with 5.1K complex pathway problems derived from real research, covering various biological contexts including natural dynamic changes, disturbances, additional intervention conditions, and multi-scale research targets, and multi-scale research targets, and an LLM agent that enhances reasoning through interactive subgraph-based navigation is proposed.
- Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning21
A noise-robust bi-level re-weighting framework which is able to learn the per-sample weights measuring the data quality without requiring any gold data is proposed.
- Dependency Parsing for Weibo: An Efficient Probabilistic Logic Programming Approach21
This work presents a new GFL/FUDG-annotated Chinese treebank with more than 18K tokens from Sina Weibo, and forms the dependency parsing problem as many small and parallelizable arc prediction tasks: for each task, a programmable probabilistic firstorder logic is used to infer the dependency arc of a token in the sentence.
- Linearized Relative Positional Encoding20
This work puts together a variety of existing linear relative positional encoding approaches under a canonical form and further proposes a family of linearrelative positional encoding algorithms via unitary transformation, leading to a principled framework that can be used to develop new relative positional encode methods that preserve linear space-time complexity.
- ParallelComp: Parallel Long-Context Compressor for Length Extrapolation19
This work proposes ParallelComp, a parallel long-context compression method that effectively overcomes the memory bottleneck, enabling 8B-parameter LLMs to extrapolate from 8K to 128K tokens on a single A100 80GB GPU in a training-free setting and designs a KV cache eviction technique to mitigate this phenomenon.
- UNComp: Can Matrix Entropy Uncover Sparsity? — A Compressor Design from an Uncertainty-Aware Perspective18
This work proposes UNComp, an uncertainty-aware framework that leverages truncated matrix entropy to identify areas of low information content, thereby revealing sparsity patterns that can be used for adaptive compression, and shows that sparsity patterns, when derived from uncertainty estimates, can be exploited to reveal special long-range dependencies.
- 18
- TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models16
By conditioning the T2I model on detailed descriptions produced by T2T models, TTIDA is able to generate photo-realistic labeled images in a flexible and controllable manner, and shows consistent improvements over other data augmentation baselines.
- Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling16
With an ensemble setup consisting of GPT-2 and ground-truth graphs from one of 7 different formalisms, it is found that the graph information indeed improves perplexity and other metrics, and this architecture provides a new way to compare different frameworks of linguistic representation.
- 13
- TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions12
Omni Dense Captioning is proposed, a novel task designed to generate continuous, fine-grained, and structured audio-visual narratives with explicit timestamps, and a six-dimensional structural schema is introduced to create "script-like"captions, enabling readers to vividly imagine the video content scene by scene.
- MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards12
Mixture of Shards (MoS) is proposed, incorporating both inter-layer and intra-layer sharing schemes, and integrating four nearly cost-free differentiation strategies, namely subset selection, pair dissociation, vector sharding, and shard privatization, which retains all the advantages of LoRA while offering enhanced parameter efficiency.
- 11
- 10
- LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction9
The Collaborative Emotional Modeling (CoEM) framework is introduced, which integrates Retrieval-Augmented Generation (RAG) and multi-agent collaboration to improve models'EI in long-context scenarios.
- 9
- How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models8
Current Cantonese NLP methods are outlined and new benchmarks designed to evaluate LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonese are introduced, which aim to advance open-source Cantonese LLM technology.
- ATTS: Asynchronous Test-Time Scaling via Conformal Prediction8
Asynchronous Test-Time Scaling (Asynchronous Test-Time Scaling) is introduced, a statistically guaranteed adaptive scaling framework that follows the hypothesis testing process to address challenges of large language models and revisiting arithmetic intensity.
- Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration7
A novel approach, ReverseGen, designed to automatically generate effective training samples that expose the weaknesses of LLMs, with a dedicated proposer trained to produce queries that lead target models to generate unsatisfactory responses.
- 7
- 7
- Self-Infilling Code Generation6
This work introduces self-infilling code generation, a general framework that incorporates infilling operations into auto-regressive decoding, and utilizes this capability to introduce novel interruption and looping mechanisms in conventional decoding, evolving it into a non-monotonic process.
- SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving5
S SwingArena presents a scalable and extensible methodology for evaluating LLMs in realistic, CI-driven software development settings, and introduces a retrieval-augmented code generation (RACG) module that efficiently handles long-context challenges.
- SORTIE: Dependency-Aware Symbolic Reasoning for Logical Data-to-text Generation5
This work proposes a dependency-aware symbolic reasoning framework that reasons out each entity in the table description with the authors' designed table-compatible programming language and devise an entity scheduling mechanism to determine the order of programme synthesis such that the reasoning of an entity only relies on other “re-solved” entities.
- 5
- 4
- 4
- 4
- 4
- ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models3
ImplicitMemBench is introduced, the first systematic benchmark evaluating implicit memory through three cognitively grounded constructs drawn from standard cognitive-science accounts of non-declarative memory: Procedural Memory, Priming, and Classical Conditioning.
- 3
- 2
- Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting2
This paper proposes a knowledge-driven progressive thought prompting method to guide large language models to generate multi-turn psychology-related dialogue, which integrates a progressive thought generator, a psychology knowledge generator, and a multi-turn dialogue generator.
- ACBiMA: Advanced Chinese Bi-Character Word Morphological Analyzer2
A morphological analyzer is developed to classify Chinese bi-character words into the defined categories, which outperforms strong baselines and achieves about 66% macro F-measure for compound words, and effectively covers derived words.
- 2
- 1
- 1
- 1
- 1
- Relative Pixel Prediction For Autoregressive Image Generation1
This paper proposes to predict pixels relatively, by predicting new pixels relative to previously generated pixels (or pixels from the conditioning context, when available), and shows that this form of prediction fare favorably to its absolute counterpart when used independently, but their coordination under an unified probabilistic model yields optimal performance.
- 1
- –
- –
- The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models–
The findings suggest that social role granularity is not merely a stylistic surface feature, but a structured, ordered, and causally manipulable latent direction in role-conditioned language model behavior.
- ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Emergent Adaptation–
Extensive experiments across diverse benchmarks demonstrate that ToolSelf rivals specialized workflows while generalizing to novel tasks, achieving a 24.1% average performance gain and illuminating a path toward truly self-adaptive agents.
- –
- –
- –
- Attentive Multi-Layer Perceptron for Non-autoregressive Generation–
A novel MLP variant, Attentive-Multi-Layer-AMLP, to produce a generation model with linear time and space complexity, which surpasses competitive efficient NAR models, by a significant margin on text-to-speech synthesis and machine translation.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.