Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Ming ZhouSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works249 from public data
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages4,421
This work develops CodeBERT with Transformer-based neural architecture, and trains it with a hybrid objective function that incorporates the pre-training task of replaced token detection, which is to detect plausible alternatives sampled from generators.
- scGPT: toward building a foundation model for single-cell multi-omics using generative AI1,440
This study has constructed a foundation model for single-cell biology, scGPT, based on a generative pretrained transformer across a repository of over 33 million cells, and illustrates that scGPT effectively distills critical biological insights concerning genes and cells.
- CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioning1,224
A CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner can achieve SOTA results on various video-text retrieval datasets, including MSR-VTT, MSVC, LSMDC, ActivityNet, and DiDeMo.
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing1,013
A framework called CRITIC is introduced that allows LLMs, which are essentially"black boxes" to validate and progressively amend their own outputs in a manner similar to human interaction with tools.
- Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training983
After pretraining on large-scale image-caption pairs, Unicoder-VL is transferred to caption-based image-text retrieval and visual commonsense reasoning, with just one additional output layer, and shows the powerful ability of the cross-modal pre-training.
- Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models908
A system to enable the user to interact with ChatGPT by sending and receiving not only languages but also images and providing complex visual questions or visual editing instructions that require the collaboration of multiple AI models with multi-steps, and opens the door to investigating the visual roles ofChatGPT with the help of Visual Foundation Models.
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters648
K-Adapter is proposed, which remains the original parameters of the pre-trained model fixed and supports continual knowledge infusion and captures richer factual and commonsense knowledge than RoBERTa.
- 635
- ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training512
A new sequence-to-sequence pre-training model called ProphetNet is presented, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism that predicts the next n tokens simultaneously based on previous context tokens at each time step.
- Improving Question Answering by Commonsense-Based Pre-training489
Results show that incorporating commonsense-based function improves the baseline on three question answering tasks that require commonsense reasoning and leverages useful evidence from an external commonsense knowledge base, which is missing in existing neural network models.
- UniViLM: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation.410
Experimental results demonstrate that the UniVL can learn strong video-text representation and achieves state-of-the-art results on five downstream tasks.
- XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation404
A recent cross-lingual pre-trained model Unicoder is extended to cover both understanding and generation tasks, which is evaluated on XGLUE as a strong baseline and the base versions of Multilingual BERT, XLM and XLM-R are evaluated for comparison.
- 400
- NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion373
Compared to several strong baselines, N\"UWA achieves state-of-the-art results on text-to-image generation, text- to-video generation, video prediction, etc, and it also shows surprisingly good zero-shot capabilities onText-guided image and video manipulation tasks.
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving331
This paper proposes ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools, thereby amalgamating the analytical prowess of language and the computational efficiency of tools.
- Automating code review activities by large-scale pre-training315
This research proposes CodeReviewer, a pre-trained model that utilizes four pre-training tasks tailored specifically for the code review scenario, and establishes a high-quality benchmark dataset based on the collected data for these three tasks.
- Question Generation for Question Answering299
Experimental results show that, by using generated questions as an extra signal, significant QA improvement can be achieved.
- Progress in Neural NLP: Modeling, Learning, and Reasoning281
The importance of reasoning is emphasized in this paper because it is important for building interpretable and knowledge-driven neural NLP models to handle complex tasks.
- Compare to The Knowledge: Graph Neural Fake News Detection with External Knowledge266
A novel end-to-end graph neural model called CompareNet is proposed, which compares the news to the knowledge base (KB) through entities for fake news detection and significantly outperforms state-of-the-art methods.
- 258
- Constraint-Based Question Answering with Knowledge Graph247
A novel systematic KBQA approach to solve multi-constraint questions is proposed, which not only obtains comparable results on the two existing benchmark data-sets, but also achieves significant improvements on the ComplexQuestions.
- Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks244
It is found that doing fine-tuning on multiple languages together can bring further improvement in Unicoder, a universal language encoder that is insensitive to different languages.
- Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering232
This work proposes to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence, and achieves the state-of-the-art accuracy on the CommonsenseQA dataset.
- ReCo: Region-Controlled Text-to-Image Generation220
The proposed model, dubbed as ReCo (Region-Controlled T2I), enables the region control for arbitrary objects described by open-ended regional texts rather than by object labels from a constrained category set, and can better control the object count, spatial relationship, and region attributes such as color/size, with the free-form regional description.
- Building Task-Oriented Dialogue Systems for Online Shopping202
This work presents a general solution towards building task-oriented dialogue systems for online shopping, aiming to assist online customers in completing various purchase-related tasks, such as searching products and answering questions, in a natural language conversation manner, using existing NLP techniques, data resources, and crowdsourcing.
- 196
- 192
- Reasoning Over Semantic-Level Graph for Fact Checking191
This work proposes two mechanisms to exploit the structure of evidence while leveraging the advances of pre-trained models like BERT, GPT or XLNet, and is the state-of-the-art system in terms of both official evaluation metrics, namely claim verification accuracy and FEVER score.
- Visual Question Generation as Dual Task of Visual Question Answering181
This paper proposes an end-to-end unified model, the Invertible Question Answering Network (iQAN), to introduce question generation as a dual task of question answering to improve the VQA performance and shows that the proposed dual training framework can consistently improve model performances of many popular V QA architectures.
- 167
- 163
- CoSQA: 20,000+ Web Queries for Code Search and Question Answering162
A contrastive learning method dubbed CoCLR is introduced to enhance text-code matching, which works as a data augmenter to bring more artificially generated training instances to bring better semantic matching between query and code.
- WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach153
This work conducts a thorough examination of pretrained model based unsupervised sentence embeddings and concludes that an easy whitening-based vector normalization strategy with less than 10 lines of code consistently boosts the performance.
- Graph Neural News Recommendation with Unsupervised Preference Disentanglement153
This paper model the user-news interactions as a bipartite graph and proposes a novel Graph Neural News Recommendation model with Unsupervised Preference Disentanglement, named GNUD, which can effectively improve the performance of news recommendation and outperform state-of-the-art news recommendation methods.
- 152
- Knowledge-Based Question Answering as Machine Translation136
A translation-based approach to solve knowledge-based question answering tasks in one unified framework based on CYK parsing, which achieves better results than previous methods.
- Joint Type Inference on Entities and Relations via Graph Convolutional Networks127
A new paradigm for the task of joint entity relation extraction is developed that first identifies entity spans, then performs a joint inference on entity types and relation types and is competitive with the state-of-the-art in relation performance.
- 123
- 123
- DocChat: An Information Retrieval Approach for Chatbot Engines Using Unstructured Documents118
This paper presents DocChat, a novel information retrieval approach for chat-bot engines that can leverage unstructured documents, instead of Q-R pairs, to respond to utterances, to measure the relevance between utterances and responses directly.
- Table-to-Text: Describing Table Region With Natural Language116
A generative model to generate a natural language sentence describing a table region, e.g., a row, and a flexible copying mechanism that selectively replicates contents from the table in the output sequence is presented.
- Dialog-to-action: conversational question answering over a large-scale knowledge base110
An approach to map utterances in conversation to logical forms, which will be executed on a large-scale knowledge base, and shows that the semantic parsing-based approach outperforms a memory network based encoder-decoder model by a huge margin.
- 108
- 106
- Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models103
Synthetic prompting is introduced, a method that leverages a few handcrafted examples to prompt the model to generate more examples by itself, and selects effective demonstrations to elicit better reasoning.
- Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text94
The results show that the proposed logic-driven data augmentation algorithm achieves state-of-the-art performance on both datasets, and even surpasses human performance on the ReClor dataset.
- 92
- Joint Relational Embeddings for Knowledge-based Question Answering89
This paper proposes a novel embedding-based approach that maps NL-questions into LFs for KBQA by leveraging semantic associations between lexical representations and KBproperties in the latent space.
- Asking Clarification Questions in Knowledge-Based Question Answering88
A new clarification dataset, CLAQUA, with nearly 40K open-domain examples is constructed and a coarse-to-fine model for clarification question generation is introduced, showing that the proposed model achieves better performance than strong baselines.
- 87
- Answering Questions with Complex Semantic Constraints on Open Knowledge Bases87
TAQA is presented, a novel KB-QA system that is based on an nOKB and illustrated via experiments how TAQA can effectively answer complex questions with rich semantic constraints.
- XGPT: Cross-modal Generative Pre-Training for Image Captioning86
XGPT is a new method of Cross-modal Generative Pre-Training for Image Captioning that is designed to pre-train text-to-image caption generators through three novel generation tasks, including Image-conditioned Masked Language Modeling (IMLM), Image- Conditioned Denoising Autoencoding (IDA), and Text-conditioning Image Feature Generation (TIFG).
- 85
- Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base85
This work proposes an innovative multi-task learning framework where a pointer-equipped semantic parsing model is designed to resolve coreference in conversations, and naturally empower joint learning with a novel type-aware entity detection model.
- Mask Attention Networks: Rethinking and Strengthen Transformer83
A novel understanding of SAN and FFN are presented as Mask Attention Networks (MANs) and it is shown that they are two special cases of MANs with static mask matrices, which limit the capability for localness modeling in text representation learning.
- Neural Deepfake Detection with Factual Structure of Text83
Results of experiments show that the proposed graph-based model for deepfake detection significantly improves strong base models built with RoBERTa and indicates that the model can distinguish the difference in the factual structure between machine-generated text and human-written text.
- Semantic Parsing with Syntax- and Table-Aware SQL Generation83
The quality of the generated SQL query is significantly improved through learning to replicate content from column names, cells or SQL keywords; and improving the generation of WHERE clause by leveraging the column-cell relation.
- Knowledge Aware Semantic Concept Expansion for Image-Text Matching82
This paper develops a Scene Concept Graph (SCG) by aggregating image scene graphs and extracting frequently co-occurred concept pairs as scene common-sense knowledge and proposes a novel model to incorporate this knowledge to improve image-text matching.
- 79
- 77
- GLGE: A New General Language Generation Evaluation Benchmark75
The General Language Generation Evaluation (GLGE), a new multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks, is presented and a leaderboard with strong baselines including MASS, BART, and ProphetNet is built.
- VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers74
The functionalities of VL-InterpreT are demonstrated through the analysis of KD-VLP, an end-to-end pretraining vision-language multimodal transformer-based model, in the tasks of Visual Commonsense Reasoning (VCR) and WebQA, two visual question answering benchmarks.
- 70
- Dense Procedure Captioning in Narrated Instructional Videos68
A framework to extract procedures by a cross-modality module, which fuses video content with the entire transcript; and generate captions by encoding video frames as well as a snippet of transcripts within each extracted procedure is introduced.
- Question Generation from SQL Queries Improves Neural Semantic Parsing68
This study conducts a study on WikiSQL, the largest hand-annotated semantic parsing dataset to date, and demonstrates that question generation is an effective method that empowers us to learn a state-of-the-art neural network based semantic parser with thirty percent of the supervised training data.
- 64
- LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network63
This work proposes LogicalFactChecker, a neural network approach capable of leveraging logical operations for fact checking, which achieves the state-of-the-art performance on TABFACT, a large-scale, benchmark dataset built for verifying a textual statement with semi-structured tables.
- DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation59
A new dialog pre-training framework called DialogVED is proposed, which introduces continuous latent variables into the enhanced encoder-decoder pre- training framework to increase the relevance and diversity of responses.
- RikiNet: Reading Wikipedia Pages for Natural Question Answering59
This paper introduces a new model, called RikiNet, which reads Wikipedia pages for natural question answering, which is the first single model that outperforms the single human performance.
- Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension57
A novel multi-grained machine reading comprehension framework that focuses on modeling documents at their hierarchical nature, which are different levels of granularity: documents, paragraphs, sentences, and tokens, which significantly outperforms previous systems at both long and short answer criteria.
- Syntax-Enhanced Pre-trained Model56
It is demonstrated that infusing automatically produced syntax of text improves pre-trained models, and global syntactic distances among tokens bring larger performance gains compared to local head relations between contiguous tokens.
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation56
The ProphetNet family pre-training models, named ProphetNet-X, where X can be English, Chinese, Multi-lingual, and so on, are presented, and achieve new state-of-the-art performance on 10 benchmarks.
- GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis52
A cascaded labeling module is developed to enhance the interchange between aspect terms and improve the attention of sentiment tokens when labeling sentiment polarities and the proposed GRACE adopts a post-pretraining BERT as its backbone.
- 52
- Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing51
An approach to incorporate retrieved datapoints as supporting evidence for context-dependent semantic parsing, such as generating source code conditioned on the class environment, and shows that both the context-aware retriever and the meta-learning strategy improve accuracy.
- Learning to Collaborate for Question Answering and Asking51
A training algorithm is presented that generalizes both Generative Adversarial Network (GAN) and Generative Domain-Adaptive Nets (GDAN) under the question answering scenario and finds that the performance of a QG model could be easily improved by a QA model via policy gradient.
- 50
- 42
- 40
- Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space40
A novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension, question generation, and question-answering natural language inference tasks.
- 35
- Control Image Captioning Spatially and Temporally34
Comprehensive experimental results demonstrate that the LoopCAG model learns better correspondence among the three modalities (vision, language, and traces) and achieves SOTA performance on trace-controlled image captioning task and the controllability and explainability ofloopCAG are validated by analyzing spatial and temporal sensitivity during the generation process.
- An Enhanced Knowledge Injection Model for Commonsense Generation32
This work integrates two additional modules into the pretrained encoder-decoder model for prototype modeling to enhance the knowledge injection procedure and shows results that significantly improves the performance on all the metrics.
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search29
The proposed CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training, achieves new state-of-the-art with significant improvement over existing code pre-trained models, on eleven domain/language-specific code search tasks with six programming languages in different code granularity.
- Content-based table retrieval for web queries29
This work presents a ranking based approach to content-based table retrieval, and implements both carefully designed features and neural network architectures to measure the relevance between a query and the content of a table.
- 28
- 27
- 26
- 26
- Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning26
This work aims to learn a neural semantic parser when only prior knowledge about a limited number of simple rules is available, without access to either annotated programs or execution results, and improves the accuracy and stability on examples uncovered by rules.
- 25
- 25
- 25
- 25
- GroundNLQ @ Ego4D Natural Language Queries Challenge 202324
A novel grounding model GroundNLQ is introduced, which employs a multi-modal multi-scale grounding module for effective video and text fusion and various temporal intervals, especially for long videos.
- 24
- 22
- 22
- 22
- GEM: A General Evaluation Benchmark for Multimodal Tasks22
GEM is a large-scale vision-language benchmark, which consists of GEM-I for image-language tasks and Gem-V for video- Language tasks, which is not only the largest vision- language dataset covering image- language tasks and video-language task at the same time, but also labeled in multiple languages.
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation22
A multi-source Transformer decoder, which takes three sources as inputs and generates multiple headlines with keyphrases of user interests to generate multiple keyphrase-relevant headlines, achieves state-of-the-art results in terms of quality and diversity.
- A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos22
The analysis shows that the proposed task is challenging and standard modeling approaches like unsupervised segmentation, semantic role labeling, and visual action detection perform poorly when forced to predict every action of a procedure in a structured form.
- 21
- 20
- 19
- 19
- 18
- 18
- 17
- 17
- 16
- 15
- 15
- 15
- 15
- 15
- 14
- Knowledge-Aware Conversational Semantic Parsing over Web Tables14
A knowledge-aware semantic parser to improve parsing performance by integrating various types of knowledge, including grammar knowledge, expert knowledge, and external resource knowledge, which is based on a decomposable model.
- 14
- 13
- 13
- 12
- Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA12
- 12
- 12
- 12
- 12
- 11
- 11
- 11
- 11
- 11
- 10
- 10
- Hashing based Efficient Inference for Image-Text Matching10
A novel hashing based efficient inference module called HEI is proposed, which can be plugged into the existing framework to speed up the inference step without reducing the retrieval performance.
- 10
- 10
- 9
- 9
- 9
- 9
- 8
- 8
- 8
- 8
- 7
- 7
- 7
- 7
- AdsCVLR: Commercial Visual-Linguistic Representation Modeling in Sponsored Search7
This paper proposes a transformer architecture with Ads data on Commercial Visual-Linguistic Representation (AdsCVLR) with contrastive learning that naturally extends the transformer encoder with the complementary multi-modal inputs, serving as a strong aggregator of image-text features.
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 6
- 6
- 6
- An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 20226
The model CONE dynamically slices the long video into candidate windows via a sliding window approach, an efficient window-centric COarse-to-fiNE alignment framework that learns the inter-window semantic variance through contrastive learning and speeds up inference by pre-filtering the candidate windows relevant to the NL query.
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 4
- HORIZON: High-Resolution Semantically Controlled Panorama Synthesis4
An innovative framework for generating high-resolution panoramas is unveiled, adeptly addressing the issues of spherical distortion and edge discontinuity through sophisticated spherical modeling and empowers users with semantic control, harnessing both image and text inputs.
- 4
- 4
- 4
- Enhancing RDF Verbalization with Descriptive and Relational Knowledge4
Experimental results on the WebNLG and SemEval-2010 datasets show that the two types of knowledge can both enhance the model performance, and their combination is able to obtain further improvements in most cases, providing new state-of-the-art results.
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection3
- Segment-Then-Rank: Non-Factoid Question Answering on Instructional Videos3
A two-stage model for multimodal segmentation of video into span candidates and length-adaptive ranking of the candidates to the question is proposed, dynamically combining the two models with complementary strength for both short and long spans respectively.
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.