Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Suggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works92 from public data
- UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models380
The UnifiedSKG framework is proposed, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset.
- Episodic Memory in Lifelong Language Learning350
This work proposes an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier.
- ZeroGen: Efficient Zero-shot Learning via Dataset Generation317
It is argued that ZeroGen can also provide useful insights from the perspective of data-free model-agnostic knowledge distillation, and unreferenced text generation evaluation.
- 302
- A Dependency Parser for Tweets221
A new dependency parser for English tweets, TWEEBOPARSER, which builds on several contributions: new syntactic annotations for a corpus of tweets, with conventions informed by the domain; adaptations to a statistical parsing algorithm; and a new approach to exploiting out-of-domain Penn Treebank data.
- OS-Copilot: Towards Generalist Computer Agents with Self-Improvement220
OS-Copilot is introduced, a framework to build generalist agents capable of interfacing with comprehensive elements in an operating system (OS), including the web, code terminals, files, multimedia, and various third-party applications, and FRIDAY, a self-improving embodied agent for automating general computer tasks.
- DetGPT: Detect What You Need via Reasoning153
This paper introduces a new paradigm for object detection that is called reasoning-based object detection, and leverages state-of-the-art multi-modal models and open-vocabulary object detectors to perform reasoning within the context of the user's instructions and the visual scene.
- M$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning144
Ying-VLM, a VLM model trained on the M$^3$IT dataset, is developed, showcasing its potential to answer complex questions requiring world knowledge, generalize to unseen video tasks, and comprehend unseen instructions in Chinese.
- 133
- Language Models Can See: Plugging Visual Controls in Text Generation128
A training-free framework for plugging in visual controls in the generation process and enabling LMs to perform multimodal tasks (e.g., image captioning) in a zero-shot manner, which outperforms the state-of-the-art method by notable margins with a nearly 27 times decoding speedup.
- 117
- TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos114
This work introduces TimeChat-Online, a novel online VideoLLM that revolutionizes real-time video interaction and presents TimeChat-Online-139K, a comprehensive streaming video dataset featuring diverse interaction patterns including backward-tracing, current-perception, and future-responding scenarios.
- 110
- Adaptive Semiparametric Language Models107
A language model that combines a large parametric neural network with a non-parametric episodic memory component in an integrated architecture is presented and a gating function to adaptively combine multiple information sources to make a prediction is designed.
- Audio-Visual Segmentation with Semantics104
A new baseline method is proposed that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process, and a regularization loss is designed to encourage audio-visual mapping during training.
- VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models102
VL-RewardBench is introduced, a comprehensive benchmark spanning general multimodal queries, visual hallucination detection, and complex reasoning tasks and uncovered three critical insights for improving VL-GenRMs: models predominantly fail at basic visual perception tasks rather than reasoning tasks.
- 99
- Lemur: Harmonizing Natural Language and Code for Language Agents98
Lemur and Lemur-Chat are introduced, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents, demonstrating balanced proficiencies in both domains, unlike existing open-source models that tend to specialize in either.
- GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning97
Experimental results demonstrate that GIMLET significantly outperforms molecule-text baselines in instruction-based zero-shot learning, even achieving closed results to supervised GNN models on tasks such as toxcast and muv.
- Segmental Recurrent Neural Networks for End-to-End Speech Recognition84
Practical training and decoding issues as well as the method to speed up the training in the context of speech recognition are discussed, and the model is self-contained and can be trained end-to-end.
- 72
- ImgTrojan: Jailbreaking Vision-Language Models with ONE Image69
This paper proposes a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions, and designs two metrics to quantify the success rate and the stealthiness of the attack.
- Why Does the Effective Context Length of LLMs Fall Short?69
Experimental results show that without additional training, STRING dramatically improves the performance of the latest large-scale models, such as Llama3.1 70B with \method, and achieves better performance than GPT-4-128K and clearly surpasses Claude 2 and Kimi-chat.
- 64
- Vicinity Vision Transformer58
Vicinity Attention, which is a type of linear attention that integrates 2D locality that achieves state-of-the-art image classification accuracy with 50% fewer parameters than previous approaches, is proposed.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.