Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks38 from public data
- UniViLM: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation.410
Experimental results demonstrate that the UniVL can learn strong video-text representation and achieves state-of-the-art results on five downstream tasks.
- 167
- 112
- 98
- 92
- EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images78
EHRXQA, a novel multi-modal question answering dataset combining structured EHRs and chest X-ray images, is introduced and a NeuralSQL-based strategy equipped with an external VQA API is proposed to address the unique challenges of multi-modal questions within EHRs.
- 77
- Exploring Diffusion Time-steps for Unsupervised Representation Learning39
A theoretical framework is built that connects the diffusion time-steps and the hidden attributes, which serves as an effective inductive bias for unsupervised learning of the modular attributes.
- Control Image Captioning Spatially and Temporally34
Comprehensive experimental results demonstrate that the LoopCAG model learns better correspondence among the three modalities (vision, language, and traces) and achieves SOTA performance on trace-controlled image captioning task and the controllability and explainability ofloopCAG are validated by analyzing spatial and temporal sensitivity during the generation process.
- GroundNLQ @ Ego4D Natural Language Queries Challenge 202324
A novel grounding model GroundNLQ is introduced, which employs a multi-modal multi-scale grounding module for effective video and text fusion and various temporal intervals, especially for long videos.
- GEM: A General Evaluation Benchmark for Multimodal Tasks22
GEM is a large-scale vision-language benchmark, which consists of GEM-I for image-language tasks and Gem-V for video- Language tasks, which is not only the largest vision- language dataset covering image- language tasks and video-language task at the same time, but also labeled in multiple languages.
- A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos22
The analysis shows that the proposed task is challenging and standard modeling approaches like unsupervised segmentation, semantic role labeling, and visual action detection perform poorly when forced to predict every action of a procedure in a structured form.
- 19
- 18
- 18
- 15
- Hierarchical Context-aware Network for Dense Video Event Captioning14
A novel Hierarchical Context-aware Network for dense video event captioning (HCN) to capture context from various aspects is introduced and leverages local and global context with different mechanisms to jointly learn to generate coherent captions.
- 12
- A Supply-Demand Approach for Traffic-Oriented Wireless Resource Virtualization With Testbed Analysis11
- Hashing based Efficient Inference for Image-Text Matching10
A novel hashing based efficient inference module called HEI is proposed, which can be plugged into the existing framework to speed up the inference step without reducing the retrieval performance.
- From PHY to QoE: A Parameterized Framework Design9
In order to establish the relationship between PHY and QoE, this paper models the end-to-end transmission from UE perspective and proposes a five-layer framework based on hierarchical analysis method, which includes system-level model, bitstream model, packet model, service quality model and experience quality model.
- DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections8
DynamixSFT is proposed, a dynamic and automated method for instruction-tuning dataset mixture optimization that effectively optimizes the Tulu-2-mixture and Tulu-3-mixture collections across 10 benchmarks, while introducing minimal computational overhead over naive sampling.
- Generative Prompt Internalization7
This work proposes Generative Prompt Internalization (GenPI), a lightweight method that employs a joint training approach that internalizes complex prompts and introduces a data synthesis technique that autonomously collects conversational datasets by swapping the roles of the agent and environment.
- 7
- An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 20226
The model CONE dynamically slices the long video into candidate windows via a sliding window approach, an efficient window-centric COarse-to-fiNE alignment framework that learns the inter-window semantic variance through contrastive learning and speeds up inference by pre-filtering the candidate windows relevant to the NL query.
- 5
- HORIZON: High-Resolution Semantically Controlled Panorama Synthesis4
An innovative framework for generating high-resolution panoramas is unveiled, adeptly addressing the issues of spherical distortion and edge discontinuity through sophisticated spherical modeling and empowers users with semantic control, harnessing both image and text inputs.
- Segment-Then-Rank: Non-Factoid Question Answering on Instructional Videos3
A two-stage model for multimodal segmentation of video into span candidates and length-adaptive ranking of the candidates to the question is proposed, dynamically combining the two models with complementary strength for both short and long spans respectively.
- 3
- 3
- 2
- 2
- 2
- Functionality Discovery and Prediction of Physical Objects2
This paper mine object-functionality knowledge through pattern-based and model-based methods from text, introduces a novel task on physical object- functionality prediction, and proposes a method to leverage the mined functionality knowledge for the new task.
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.