Is this you? Claim this profile to correct it, add a bio and choose the work people see first.

Claim this profile

Academic lineage

Possible advisorsa guess from early papers, not confirmed

  • Gang Wang

    Advisor

    Suggested from co-authorship
  • Tsuhan Chen

    Possible advisor · last author on 4 of their early first-author papers, 2016–2018

    Suggested from co-authorship
  • Gang Wang

    Possible advisor · last author on 3 of their early first-author papers, 2018–2019

    Suggested from co-authorship

Is this you? Claim this profile to confirm or dismiss it.

Works110 from public data

TitleCited by
  • Recent advances in convolutional neural networks

    Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, +3 more

    Pattern Recognition · 2017

    This paper details the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation, and introduces various applications of convolutional neural networks in computer vision, speech and natural language processing.

    6,499
  • LRM: Large Reconstruction Model for Single Image to 3D

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Bi, Sai, Yang Min Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, +2 more

    arXiv · 2023

    This work proposes the first Large Reconstruction Model (LRM) that predicts the 3D model of an object from a single input image within just 5 seconds, and adopts a highly scalable transformer-based architecture with 500 million learnable parameters to directly predict a neural radiance field (NeRF) from the input image.

    1,030
  • Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models

    Jiuxiang Gu, Jianfei Cai, Shafiq Joty, Li Niu, Gang Wang

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2018

    This work proposes to incorporate generative processes into the cross-modal feature embedding, through which it is able to learn not only the global abstract features but also the local grounded features of image-text pairs.

    394
  • Scene Graph Generation With External Knowledge and Image Reconstruction

    Jiuxiang Gu, Handong Zhao, Zhe Lin, Sheng Li, Jianfei Cai, Mingyang Ling

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2019

    This paper proposes a novel scene graph generation algorithm with external knowledge and image reconstruction loss to overcome dataset issues, and extracts commonsense knowledge from the external knowledge base to refine object and phrase features for improving generalizability inscene graph generation.

    322
  • Towards Language-Free Training for Text-to-Image Generation

    Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, +1 more

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    The first work to train text-to-image generation models without any text data is proposed, which leverages the well-aligned multi-modal semantic space of the powerful pre-trained CLIP model: the requirement of text-conditioning is seamlessly alleviated via generating text features from image features.

    235
  • Unpaired Image Captioning via Scene Graph Alignments

    Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Handong Zhao, Xu Yang, Gang Wang

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2019

    This paper proposes an unsupervised feature alignment method that maps the scene graph features from the image to the sentence modality and can generate quite promising results without using any image-caption training pairs, outperforming existing methods by a wide margin.

    199
  • Stack-Captioning: Coarse-to-Fine Learning for Image Captioning

    Jiuxiang Gu, Jianfei Cai, Gang Wang, Tsuhan Chen

    Proceedings of the AAAI Conference on Artificial Intelligence · 2018

    A coarse-to-fine multi-stage prediction framework for image captioning, composed of multiple decoders each of which operates on the output of the previous stage, producing increasingly refined image descriptions, which simultaneously solves the well-known exposure bias problem and the loss-evaluation mismatch problem.

    196
  • SelfDoc: Self-Supervised Document Representation Learning

    Peizhao Li, Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Rajiv Jain, Varun Manjunatha, Hongfu Liu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2021

    This work proposes SelfDoc, a task-agnostic pre-training framework for document image understanding that benefits from self-supervised pre- training on documents without requiring annotations by a feature masking training strategy, and proposes a novel modality-adaptive attention mechanism for multimodal feature fusion by adaptively emphasizing language and vision signals.

    187
  • An Empirical Study of Language CNN for Image Captioning

    Jiuxiang Gu, Gang Wang, Jianfei Cai, Tsuhan Chen

    IEEE International Conference on Computer Vision (ICCV) · 2017

    This paper introduces a language CNN model which is suitable for statistical language modeling tasks and shows competitive performance in image captioning, and is competitive with the state-of-the-art methods.

    150
  • Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models

    Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, Xia Hu

    Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · 2021

    This work shows that the words in the NLU training set can be modeled as a long-tailed distribution, and proposes a shortcut mitigation framework LGTR, to suppress the model from making overconfident predictions for samples with large shortcut degree.

    125
  • Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling

    Đạt Tiến Huỳnh, Jason Kuen, Zhe Lin, Jiuxiang Gu, Ehsan Elhamifar

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    This work proposes a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images and designs a robust student model that selectively distills mask knowledge by estimating the mask noise levels, hence mitigating the adverse impact of noisy pseudo masks.

    119
  • High Quality Entity Segmentation

    Lu Qi, Jason Kuen, Tiancheng Shen, Jiuxiang Gu, Wenbo Li, Weidong Guo, Jiaya Jia, Zhe Lin, +1 more

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2023

    CropFormer is the first query-based Transformer architecture that can effectively fuse mask predictions from multiple image views, by learning queries that effectively associate the same entities across the full image and its crop.

    115
  • Open World Entity Segmentation

    Lu Qi, Jason Kuen, Yi Wang, Jiuxiang Gu, Hengshuang Zhao, Philip H. S. Torr, Zhe Lin, Jiaya Jia

    IEEE Transactions on Pattern Analysis and Machine Intelligence · 2022

    This work proposes a CondInst-like fully-convolutional architecture with two novel modules specifically designed to exploit the class-agnostic and non-overlapping requirements of ES, and investigates the feasibility of convolutional center-based representation to segment things and stuffs in a unified manner.

    103
  • Unpaired Image Captioning by Language Pivoting

    Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Gang Wang

    Lecture notes in computer science · 2018

    This work can effectively capture the characteristics of an image captioner from the pivot language (Chinese) and align it to the target language (English) using another pivot-target (Chinese-English) parallel corpus.

    91
  • Exploiting Semantic Embedding and Visual Feature for Facial Action Unit Detection

    Huiyuan Yang, Lijun Yin, Yi Zhou, Jiuxiang Gu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2021

    82
  • Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning

    Xiangxi Shi, Xu Yang, Jiuxiang Gu, Shafiq Joty, Jianfei Cai

    Lecture notes in computer science · 2020

    A novel visual encoder is proposed to explicitly distinguish viewpoint changes from semantic changes in the change captioning task and further simulate the attention preference of humans and proposes a novel reinforcement learning process to fine-tune the attention directly with language evaluation rewards.

    79
  • Multi-Scale Aligned Distillation for Low-Resolution Detection

    Lu Qi, Jason Kuen, Jiuxiang Gu, Zhe Lin, Yi Wang, Yukang Chen, Yanwei Li, Jiaya Jia

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2021

    The idea of spatially aligning feature maps between models of varying input resolutions by shifting feature pyramid position is explored and crossed feature-level fusion is proposed to dynamically fuse teacher’s multi-resolution features to guide the student better.

    74
  • EI-CLIP: Entity-aware Interventional Contrastive Learning for E-commerce Cross-modal Retrieval

    Haoyu Ma, Handong Zhao, Zhe Lin, Ajinkya Kale, Zhangyang Wang, Tong Yu, Jiuxiang Gu, Sunav Choudhary, +1 more

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    63
  • Delving into Out-Of-Distribution Detection with Vision-Language Representations

    Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, Yixuan Li

    neural information processing systems · 2022

    57
  • Resilient Load Restoration in Microgrids Considering Mobile Energy Storage Fleets: A Deep Reinforcement Learning Approach

    Shuhan Yao, Jiuxiang Gu, Huajun Zhang, Peng Wang, Xiaochuan Liu, Tianyang Zhao

    IEEE Power & Energy Society General Meeting (PESGM) · 2020

    A Markov decision process (MDP) formulation for an integrated service restoration strategy that coordinates the scheduling of MESSs and resource dispatching of microgrids and simulation results indicate that mobile and stationary energy resources can be well coordinated to improve system resilience.

    56
  • UniDoc: Unified Pretraining Framework for Document Understanding

    Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Rajiv Jain, Nikolaos Barmpalios, Ani Nenkova, Tong Sun

    neural information processing systems · 2021

    43
  • LAFITE: Towards Language-Free Training for Text-to-Image Generation

    Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Yu, Tong, Jiuxiang Gu, Jinhui Xu, +1 more

    arXiv · 2021

    37
  • Watch It Twice

    Xiangxi Shi, Jianfei Cai, Shafiq Joty, Jiuxiang Gu

    ACM International Conference on Multimedia (ACM MM) · 2019

    A novel recurrent video encoding method and a novel visual spatial feature for the video captioning task that encodes the video twice with a predicted key frame to avoid irrelevant temporal information often occurring at the beginning and at the end of a video.

    29
  • LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding

    Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, Tong Sun

    arXiv · 2023

    27
  • A Survey on Small Language Models

    Chien Van Nguyen, Xuan Shen, Ryan Aponte, Yu Xia, Samyadeep Basu, Zhengmian Hu, Jian Chen, Mihir Parmar, +22 more

    International conference Recent advances in natural language processing · 2025

    22

Show all 110 works

Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.

Report an error

Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.

Sign in to report