Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks472 from public data
- Video Question Answering via Gradually Refined Attention over Appearance and Motion840
This paper proposes an end-to-end model which gradually refines its attention over the appearance and motion features of the video using the question as guidance and demonstrates the effectiveness of the model by analyzing the refined attention weights during the question answering procedure.
- ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering793
This work introduces ActivityNet-QA, a fully annotated and large scale VideoQA dataset, which consists of 58,000 QA pairs on 5,800 complex web videos derived from the popular ActivityNet dataset and explores various video representation strategies to improve videoQA performance.
- 576
- 538
- Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models533
This work proposes Make-An- audio with a prompt-enhanced diffusion model that alleviates data scarcity with orders of magnitude concept compositions by using language-free audios, and presents its controllability and generalization for X-to-Audio with "No Modality Left Behind", for the first time unlocking the ability to generate high-definition, high-fidelity audios given a user-defined modality input.
- 503
- 490
- Improving automatic source code summarization via deep reinforcement learning435
An abstract syntax tree structure as well as sequential content of code snippets into a deep reinforcement learning framework (i.e., actor-critic network) which provides the confidence of predicting the next word according to current state and an advantage reward composed of BLEU metric to train both networks.
- Investigating Capsule Networks with Dynamic Routing for Text Classification392
This work proposes three strategies to stabilize the dynamic routing process to alleviate the disturbance of some noise capsules which may contain “background” information or have not been successfully trained.
- DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism383
This work proposes DiffSinger, an acoustic model for SVS based on the diffusion probabilistic model, a parameterized Markov chain that iteratively converts the noise into mel-spectrogram conditioned on the music score that outperforms state-of-the-art SVS work.
- ProDiff264
ProDiff parameterizes the denoising model by directly predicting clean data to avoid distinct quality degradation in accelerating sampling, and enables a sampling speed of 24x faster than real-time on a single NVIDIA 2080Ti GPU, making diffusion models practically applicable to text-to-speech synthesis deployment for the first time.
- Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos251
A novel Cross-Modal Interaction Network (CMIN) is introduced to consider multiple crucial factors for this challenging task, including the syntactic structure of natural language queries; long-range semantic dependencies in video context and the sufficient cross-modal interaction.
- WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling240
WavTokenizer achieves state-of-the-art reconstruction quality with outstanding UTMOS scores and inherently contains richer semantic information by designing a broader VQ space, extended contextual windows, and improved attention networks, as well as introducing a powerful multi-scale discriminator and an inverse Fourier transform structure.
- Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers232
This paper introduces the use of object identifiers and object-centric representations to interact with scenes at the object level, and decomposes the input 3D scene into a set of object proposals, each assigned a unique identifier token, which enables efficient object referencing and grounding during user-assistant interactions.
- UniAudio: An Audio Foundation Model Toward Universal Audio Generation230
The UniAudio system is presented, which, unlike prior task-specific approaches, leverages LLM techniques to generate multiple types of audio with given input conditions, and has the potential to become a foundation model for universal audio generation.
- GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis225
GeneFace is proposed, a generalized and high-fidelity NeRF-based talking face generation method, which can generate natural results corresponding to various out-of-domain audio and introduces a domain adaptative post-net to calibrate the result.
- Multi-modal Attention Network Learning for Semantic Source Code Retrieval195
Comprehensive experiments and analysis on a large-scale real-world dataset show that the proposed MMAN model can accurately retrieve code snippets and outperforms the state-of-the-art methods.
- Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentences187
A novel Spatio-Temporal Graph Reasoning Network (STGRN) is proposed for this task, which builds a spatio-temporal region graph to capture the region relationships with temporal object dynamics, which involves the implicit and explicit spatial subgraphs in each frame and the temporal dynamic subgraph across frames.
- Weakly-Supervised Video Moment Retrieval via Semantic Completion Network178
This paper proposes a novel weakly-supervised moment retrieval framework requiring only coarse video-level annotations for training, and devise a proposal generation module that aggregates the context information to generate and score all candidate proposals in one single pass.
- 176
- Hierarchical Multi-label Text Classification172
This paper proposes a novel framework called Hierarchical Attention-based Recurrent Neural Network (HARNN) for classifying documents into the most relevant categories level by level via integrating texts and the hierarchical category structure, and designs a hybrid method capable of predicting the categories of each level while classifying all categories in the entire hierarchical structure precisely.
- Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes158
Chat-3D is presented, which combines the 3D visual perceptual ability of pre-trained 3D representations and the impressive reasoning and conversation capabilities of advanced LLMs to achieve the first universal dialogue systems for 3D scenes.
- Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding150
This paper introduces the diversity and discrimination simultaneously when generating proposals, and in doing so proposes Diversified and Discriminative Proposal Networks model (DDPN), a high performance baseline model for visual grounding and evaluates it on four benchmark datasets.
- PopMAG148
A novel MUlti-track MIDI representation (MuMIDI), which enables simultaneous multi-track generation in a single sequence and explicitly models the dependency of the notes from different tracks and is called PopMAG, which largely outperforms other state-of-the-art music accompaniment generation models and multi- track MIDI representations in terms of subjective and objective metrics.
- Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus140
Multi-Singer, a fast multi-singer vocoder with generative adversarial networks improves unseen singer singing voices modeling in both speed and quality over previous methods, and achieves strong robustness in the multi- singer singing voice synthesis pipeline.
- EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration137
This work introduces EAGER, a novel generative recommendation framework that seamlessly integrates both behavioral and semantic information, and proposes a two-stream generation architecture leveraging a shared encoder and two separate decoders to decode behavior tokens and semantic tokens.
- Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation135
This work proposes Make-an-Audio 2, a latent diffusion-based T2A method that outperforms baseline models in both objective and subjective metrics, and achieves significant gains in temporal information understanding, semantic consistency, and sound quality.
- 134
- 127
- Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis126
Real3D-Portrait is presented, a framework that improves the one-shot 3D reconstruction power with a large image-to-plane model that distills 3D prior knowledge from a 3D face generative model and generates more realistic talking portrait videos compared to previous methods.
- 126
- 124
- WavChat: A Survey of Spoken Dialogue Models123
This survey compiled existing spoken dialogue systems in the chronological order and categorized them into the cascaded and end-to-end paradigms, covering aspects such as speech representation, training paradigm, streaming, duplex, and interaction capabilities, and discusses the limitations of these technologies.
- 120
- 119
- Video Question Answering via Attribute-Augmented Attention Network Learning118
This paper proposes the attribute-augmented attention network learning framework that enables the joint frame-level attribute detection and unified video representation learning for video question answering and incorporates the multi-step reasoning process for the proposed attention network to further improve the performance.
- Video Question Answering via Hierarchical Spatio-Temporal Attention Networks117
The hierarchical spatio-temporal attention network is proposed for learning the joint representation of the dynamic video contents according to the given question and the multi-step reasoning process for open-ended video question answering is developed.
- SimulSpeech: End-to-End Simultaneous Speech to Text Translation115
Experiments on MuST-C English-Spanish and English-German spoken language translation datasets show that SimulSpeech achieves reasonable BLEU scores and lower delay compared to full-sentence end-to-end speech to text translation (without simultaneous translation), and better performance than the two-stage cascaded simultaneous translation model in terms of BLEu scores and translation delay.
- 113
- Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias103
Experimental results demonstrate that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS, speech editing, and cross-lingual TTS tasks, with superior naturalness, robustness, and speaker similarity due to the proper inductive bias of each module.
- 103
- Dialogue Act Recognition via CRF-Attentive Structured Network102
This paper tackles the problem of DAR from the viewpoint of extending richer Conditional Random Field (CRF) structured dependencies without abandoning end-to-end training and incorporates hierarchical semantic inference with memory mechanism on the utterance modeling at multiple levels.
- Multimodal Pretraining, Adaptation, and Generation for Recommendation: A Survey101
This survey seeks to provide a comprehensive exploration of the latest advancements and future trajectories in multimodal pretraining, adaptation, and generation techniques, as well as their applications in enhancing recommender systems.
- Counterfactual Contrastive Learning for Weakly-Supervised Vision-Language Grounding99
This paper designs three counterfactual transformation strategies from the feature-, interactionand relation-level, where the feature-level method damages the visual features of selected proposals, interactionlevel approach confuses the vision-language interaction and relation-level strategy destroys the context clues in proposal relationships.
- TextrolSpeech: A Text Style Control Speech Corpus with Codec Language Text-to-Speech Models93
This work proposes TextrolSpeech, which is the first large-scale speech emotion dataset annotated with rich text attributes and proposes an efficient architecture called Salle, which treats text controllable TTS as a language model task, utilizing audio codec codes as an intermediate representation to replace the conventional mel-spectrogram.
- 91
- Community-Based Question Answering via Heterogeneous Social Network Learning91
This paper proposes a novel framework which encodes not only the contents of question-answer(Q-A) but also the social interaction cues in the community to boost the cQA tasks and shows that leveraging the heterogeneous social information indeed achieves better performance than other state-of-the-art cZA methods.
- Gloss Attention for Gloss-free Sign Language Translation90
This work proposes gloss attention, which enables the model to keep its attention within video segments that have the same semantics locally, just as gloss helps existing models do, and transfers the knowledge of sentence-to-sentence similarity from the natural language model to the GASLT network to help it understand sign language videos at the sentence level.
- DeepSinger: Singing Voice Synthesis with Data Mined From the Web89
DeepSinger, a multi-lingual multi-singer singing voice synthesis (SVS) system built from scratch using singing training data mined from music websites, is developed, which is the first SVS system that directly mines training data fromMusic websites.
- 88
- Expert finding for community-based question answering via ranking metric network learning88
This paper proposes a novel ranking metric network learning framework for expert finding by exploiting both users' relative quality rank to given questions and their social relations and develops a random-walk based learning method with recurrent neural networks forranking metric network embedding.
- Personalized Hashtag Recommendation for Micro-videos87
A Graph Convolution Network based Personalized Hashtag Recommendation (GCN-PHR) model, which leverages recently advanced GCN techniques to model the complicate interactions among users, hashtags, and micro-videos and learn their representations.
- Regularized Two-Branch Proposal Networks for Weakly-Supervised Moment Retrieval in Videos86
This paper proposes a novel Regularized Two-Branch Proposal Network to simultaneously consider the inter-sample and intra-sample confrontments and applies the proposal regularization to stabilize the training process and improve model performance.
- 86
- 85
- Revisiting Over-Smoothness in Text to Speech81
This work conducts a comprehensive study on NAR-TTS models that use some advanced modeling methods and finds that Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity.
- Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models80
It is found that the reason that pre-trained DPMs fail to reconstruct an image from its latent variables is due to the information loss of forward process, which causes a gap between their predicted posterior mean and the true one.
- Long Short-Term Memory Network Design for Analog Computing80
It is shown that using current signals as internal transmission signals can largely reduce computation delay, compared to the digital implementation of long short-term memory network, which is compatible with digital CMOS technology.
- Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-Modal Structured Representations78
This paper presents an end-to-end framework Structure-CLIP, which integrates Scene Graph Knowledge (SGK) to enhance multi-modal structured representations and uses scene graphs to guide the construction of semantic negative examples, which results in an increased emphasis on learning structured representations.
- Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation78
The Re4 framework is proposed, which leverages the backward flow to reexamine each interest embedding and encapsulates three backward flows, and empirical studies validate that Re4 helps to learn distinct and effective multi-interest representations.
- Towards Fast and High-Quality Sign Language Production78
This paper proposes a novel Non-AuToregressive (NAT) model with a parallel decoding scheme, as well as an External Aligner for sequence alignment learning, which outperforms state-of-the-art autoregressive models in terms of speed and quality.
- 78
- SingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation75
SingGAN is the first work designed toward high-fidelity singing voice vocoding, and enables a sample speed of 50x faster than real-time on a single NVIDIA 2080Ti GPU.
- Semantic Proximity Search on Heterogeneous Graph by Proximity Embedding74
This work introduces a new concept of proximity embedding, which directly embeds the network structure between two possibly distant nodes and can easily estimate the proximity score between two nodes and enable search on the graph.
- RoboGround: Robotic Manipulation with Grounded Vision-Language Priors73
This work introduces ROBOGROUND, a grounding-aware robotic manipulation policy that leverages grounding masks as an intermediate representation to guide policy networks in object manipulation tasks, and proposes an automated pipeline for generating large-scale, simulated data with a diverse set of objects and instructions.
- Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching73
Frieren, a V2A model based on rectified flow matching that achieves state-of-the-art performance in both generation quality and temporal alignment on VGGSound, with alignment accuracy reaching 97.22%, and 6.2% improvement in inception score over the strong diffusion-based baseline.
- 72
- Personalized Response Generation via Domain adaptation72
A novel personalized response generation model via domain adaptation (PRG-DM) that learns the human responding style from large general data (without user-specific information) and fine tune the model on a small size of personalized data to generate personalized responses with a dual learning mechanism.
- A Study of Non-autoregressive Model for Sequence Generation71
An analysis model called CoMMA is proposed to characterize the difficulty of different NAR sequence generation tasks and has several interesting findings, including that among the NMT, ASR and TTS tasks,ASR has the most target-token dependency while TTS has the least.
- GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks70
GTSinger, a large global, multi-technique, free-to-use, high-quality singing corpus with realistic music scores, designed for all singing tasks, along with its benchmarks, is presented.
- 70
- Query-Biased Self-Attentive Network for Query-Focused Video Summarization69
This paper proposes a hierarchical self-attentive network to model the relative relationship at three levels, which are different frames from a segment, different segments of the same video, textual information of video description and its related visual contents, and the query-focused summary.
- Investigating the transferring capability of capsule networks for text classification68
The transferring capability of the proposed cross-domain capsule network (TL-Capsule) is demonstrated by applying it to two transfer learning applications: single-label to multi-label text classification and cross- domain sentiment classification.
- 67
- 66
- 66
- 66
- 64
- 64
- 62
- Abstractive Meeting Summarization via Hierarchical Adaptive Segmental Network Learning62
This paper proposes the hierarchical neural encoder based on adaptive recurrent networks to learn the semantic representation of meeting conversation with adaptive conversation segmentation and develops the reinforced decoder network to generate the high-quality summaries for abstractive meeting summarization.
- Connecting Multi-modal Contrastive Representations61
A novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR), which achieves state-of-the-art performance on audio- image retrieval, audio-visual source localization, and counterfactual audio-image recognition tasks.
- Multi-interaction Network with Object Relation for Video Question Answering61
A new attention mechanism called multi-interaction, which can capture both element-wise and segment-wise sequence interactions, simultaneously, simultaneously is proposed, which achieves the new state-of-the-art performance.
- Open-Ended Long-form Video Question Answering via Adaptive Hierarchical Reinforced Networks61
This paper proposes the adaptive hierarchical encoder network to learn the joint representation of the long-form video contents according to the question with adaptive video segmentation and develops the reinforced decodernetwork to generate the natural language answer for open-ended video question answering.
- Efficient algorithms for finding optimal meeting point on road networks61
In order to support spatial applications that involve large flow of queries and require fast response, an extremely efficient algorithm is proposed to find a high-quality near-optimal meeting point, which is orders of magnitude faster than the exact OMP algorithms.
- GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation60
GeneFace++ is proposed, which is the first NeRF-based method that achieves stable and real-time talking face generation with generalized audio-lip synchronization and outperforms state-of-the-art baselines in terms of subjective and objective evaluation.
- Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Tasks60
A novel Dual-Guided Spatial-Channel-Temporal (DG-SCT) attention mechanism that incorporates trainable cross-modal interaction layers into pre- trained audio-visual encoders, allowing adaptive extraction of crucial information from the current modality across spatial, channel, and temporal dimensions, while preserving the frozen parameters of large-scale pre-trained models.
- StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis59
StyleSinger is proposed, the first singing voice synthesis model for zero-shot style transfer of out-of-domain reference singing voice samples that outperforms baseline models in both audio quality and similarity to the reference singing voice samples.
- 59
- Community-Based Question Answering via Asymmetric Multi-Faceted Ranking Network Learning59
This paper proposes a novel asymmetric multi-faceted ranking network learning framework for community-based question answering by jointly exploiting the deep semantic relevance between question-answer pairs and the answerers' authority to the given question.
- Truth Discovery in Data Streams: A Single-Pass Probabilistic Approach59
This paper designs a streaming algorithm that infers the truth as well as source quality in real time and develops a one-pass algorithm, in which the inference of source quality is proved to be convergent and the accuracy is further improved.
- Hierarchical Cross-Modal Graph Consistency Learning for Video-Text Retrieval58
A Hierarchical Cross-Modal Graph Consistency Learning Network (HCGC) for video-text retrieval task, which considers multi-level graph consistency for video -text matching and designs three types of graph consistency: inter-graph parallel consistency, inter- graph cross consistency and intra-graph cross consistency.
- 58
- Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models57
Orient Anything is introduced, the first expert and foundational model designed to estimate object orientation in a single- and free-view image and achieves state-of-the-art orientation estimation accuracy in both rendered and real images and exhibits impressive zero-shot ability in various scenarios.
- TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation57
This work proposes TranSpeech, a speech-to-speech translation model with bilateral perturbation, which is the first to establish a non-autoregressive S2ST technique, which repeatedly masks and predicts unit choices and produces high-accuracy results in just a few cycles.
- Fine-Grained Predicates Learning for Scene Graph Generation57
This work proposes a method named Fine-Grained Predicates Learning (FGPL) which aims at differentiating among hard-to-distinguish predicates for Scene Graph Generation task, and introduces a Predicate Lattice that helps SGG models to figure out fine-grained predicate pairs.
- 55
- SimulSLT54
SimulSLT, the first end-to-end simultaneous sign language translation model, which can translate sign language videos into target text concurrently, is proposed, composed of a text decoder, a boundary predictor, and a masked encoder.
- Microblog Sentiment Classification via Recurrent Random Walk Network Learning54
This paper proposes a novel recurrent random walk network learning framework for the microblog sentiment classification problem by exploiting both users’ posted tweets and their social relations in microblogs, and introduces the deep recurrent neural networks with random-walk layer for heterogeneous MSC network embedding.
- Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment54
This paper considers learning a multi-modal embedding from the perspective of optimizing a pairwise ranking problem while enhancing both local alignment and global alignment and demonstrates the superiority of the proposed C2MLR due to its nature of multi- modal compositional embedding.
- Prosospeech: Enhancing Prosody with Quantized Vector Pre-Training in Text-To-Speech52
ProsoSpeech is proposed, which enhances the prosody using quantized latent vectors pre-trained on large-scale unpaired and low-quality text and speech data and can generate expressive speech conditioned on the predicted LPV.
- Lightweight dynamic conditional GAN with pyramid attention for text-to-image synthesis51
LD-CGAN is a compact and structured single-stream network that consists of one generator and two independent discriminators to regularize and generate 642 and 1282 images in one feed-forward process and significantly decreases the number of parameters and computation time.
- 50
- Multi-Turn Video Question Answering via Multi-Stream Hierarchical Attention Context Network50
This paper proposes the hierarchical attention context network for context-aware question understanding by modeling the hierarchically sequential conversation context structure and develops the multi-stream spatio-temporal attention network for learning the joint representation of the dynamic video contents and context- aware question embedding.
- A Multi-task Learning Approach for Image Captioning49
The experimental results demonstrate that the proposed Multi-task Learning Approach for Image Captioning achieves impressive results compared to other strong competitors.
- Dual Learning for Cross-domain Image Captioning49
A dual learning mechanism with a policy gradient method that generates highly rewarded captions is introduced that consistently outperforms previous methods for cross-domain image captioning.
- Personalized response generation by Dual-learning based domain adaptation48
Experimental results show that the proposed PRGDDA (Personalized Response Generation by Dual-learning based Domain Adaptation) model can generate better personalized responses for different users.
- OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces47
This work proposes remapping and binding the spaces of various pre-trained specialist models together, and presents OmniBind, large-scale multimodal joint representation models ranging in scale from 7 billion to 30 billion parameters, which support 3D, audio, image, and language inputs.
- Object-Aware Multi-Branch Relation Networks for Spatio-Temporal Video Grounding47
This work proposes a novel object-aware multi-branch relation network for object- Aware region modeling, where each branch focuses on a crucial object mentioned in the sentence to capture critical object relationships between the main branch and auxiliary branches.
- 46
- Video Question Answering via Hierarchical Dual-Level Attention Network Learning46
This paper develops the hierarchical duallevel attention networks to learn the question-aware video representations with word-level and question-level attention mechanisms, and devise thequestion-level fusion attention mechanism for the proposed networks to learning the questionaware joint video representation.
- 46
- Game cloud design with virtualized CPU/GPU servers and initial performance results46
This work designs a game cloud with a virtualized cluster of CPU/GPU servers at USC GamePipe Laboratory to enable interactive gaming by taking full advantage of the cloud and local resources for high quality of experience (QoE) gaming.
- Why Do We Click45
This work devise the local impression modeling module to simultaneously attend to decomposed details in the impression when understanding the semantic meaning of news title, which could explicitly get close to the process of users reading news.
- A Comparative Study of Team Formation in Social Networks45
This paper compares and contrast all the state-of-the-art team formation algorithms and proposes a benchmark that enables fair comparison amongst these algorithms, and presents the insights arising from the results of the comparison.
- 44
- 43
- Hierarchical human-like strategy for aspect-level sentiment classification with sentiment linguistic knowledge and reinforcement learning43
This work proposes a novel Hierarchical Human-like strategy for Aspect-level Sentiment classification (HHAS), which contains three major components, a sentiment-aware mutual attention module, an aspect-specific knowledge distillation module, and a reinforcement learning based re-reading module which are consistent with the stages of the human reading cognitive process.
- Crowd-Selection Query Processing in Crowdsourcing Databases: A Task-Driven Approach43
A bayesian model is developed that provides a principle and natural framework for capturing the latent skills of workers as well as the latent categories of crowdsourced tasks and a variational algorithm is devised that transforms the latent skill infer- ence with the proposed model into a standard optimization prob- lem, which can be solved efficiently.
- EAGER-LLM: Enhancing Large Language Models as Recommenders through Exogenous Behavior-Semantic Integration42
EAGER-LLM is proposed, a decoder-only llm-based generative recommendation framework that integrates endogenous and exogenous behavioral and semantic information in a non-intrusive manner and demonstrates EAGER-LLM's effectiveness through rigorous testing on three public benchmarks.
- Future-Aware Diverse Trends Framework for Recommendation42
This paper bridges the gap between the past preference and potential future preference by proposing the future-aware diverse trends (FAT) framework and demonstrates the proposed framework not only outperforms the state-of-the-art sequential recommendation methods across various metrics, but also makes complementary and fresh recommendations.
- 42
- Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference42
This paper proposes to transfer knowledge from some important discourse markers to augment the quality of the NLI model by using reinforcement learning to optimize a new objective function with a reward defined by the property of theNLI datasets to make full use of the labels information.
- Mining probabilistically frequent sequential patterns in uncertain databases42
Based on the prefix-projection strategy of the famous PrefixSpan algorithm, two new algorithms are developed, collectively called U-PrefixSpan, for p-FSP mining, which effectively avoids the problem of "possible world explosion", and when combined with the three pruning techniques and one validating technique, achieves good performance.
- TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control41
TCSinger is introduced, the first zero-shot SVS model for style transfer across cross-lingual speech and singing styles, along with multi-level style control, and outperforms all baseline models in synthesis quality, singer similarity, and style controllability across various tasks.
- WSRGlow: A Glow-Based Waveform Generative Model for Audio Super-Resolution41
WSRGlow, a Glow-based waveform generative model to perform audio super-resolution is proposed, which is easier to train and outperforms the previous works in terms of both objective and perceptual quality.
- Interactive Paths Embedding for Semantic Proximity Search on Heterogeneous Graphs41
This paper introduces a novel concept of interactive paths to model the inter-dependency among multiple paths between a query object and a target object, and proposes an Interactive Paths Embedding (IPE) model, which learns low-dimensional representations for the resulting interactive-paths structures for proximity estimation.
- ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editing40
ThinkSound is presented, a novel framework that leverages Chain-of-Thought (CoT) reasoning to enable stepwise, interactive audio generation and editing for videos and introduces AudioCoT, a comprehensive dataset with structured reasoning annotations that establishes connections between visual content, textual descriptions, and sound synthesis.
- 40
- Make-A-Voice: Unified Voice Synthesis With Discrete Representation40
Make-A-Voice is proposed, a unified framework for synthesizing and manipulating voice signals from discrete representations by re-synthesizing the discrete voice representations with prompt guidance and demonstrating superior audio quality and style similarity compared with competitive baseline models.
- 40
- Distance-Aware DAG Embedding for Proximity Search on Heterogeneous Graphs40
A novel Distance-aware DAG Embedding (D2AGE) model is developed, which is evaluated on three benchmark data sets with six semantic relations, and it is shown that D2AGE outperforms the state-of-the-art baselines.
- EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model39
This paper proposes a simple but efficient architecture for emotional speech synthesis called EMSpeech, which could predict emotion labels just from the input text and generate more expressive speech conditioned on the emotion embedding.
- Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval39
A deep progressive quantization (DPQ) model, as an alternative to PQ, for large scale image retrieval, that is trained once for different code lengths and therefore requires less computation time.
- Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis38
The Mega-TTS 2 model is introduced, a generic zero-shot multispeaker TTS model capable of synthesizing speech for unseen speakers with arbitrary-length prompts and a phoneme-level auto-regressive duration model is proposed to introduce in-context learning capabilities to duration modeling.
- Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation38
An error correction mechanism is introduced into NMT, which corrects the error information in the previous generated tokens to better predict the next token, and introduces two-stream self-attention from XLNet into NMT decoder.
- NGUARD38
The proposed game bot detection framework NGUARD has been implemented and deployed in multiple MMORPG productions in the NetEase Game portfolio, achieving remarkable performance improvement and acceleration compared to traditional methods.
- GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech36
GenerSpeech, a text-to-speech model towards high-fidelity zero-shot style transfer of OOD custom voice surpasses the state-of-the-art models in terms of audio quality and style similarity and the extension studies to adaptive style transfer further show that GenerSpeech performs robustly in the few-shot data setting.
- 36
- HERO: HiErarchical spatio-tempoRal reasOning with Contrastive Action Correspondence for End-to-End Video Object Grounding35
This paper introduces the weakly-supervised contrastive learning that classifies the video as action-consistent and action-independent frames relying on the video-caption action semantic correspondence and designs the hierarchical reasoning layers to decouple fully connected multi-head attention and remove the redundant interfering correlations.
- Interactive Dual Generative Adversarial Networks for Image Captioning35
This paper proposes an Interactive Dual Generative Adversarial Network (IDGAN) for image captioning, which mutually combines the retrieval-based and generation-based methods to learn a better image Captioning ensemble.
- A transfer learning based framework of crowd-selection on twitter34
This paper shows a framework that tackles the problem of crowdsourced task assignment on Twitter according to the social activities of its users, and transfers the knowledge from categorized Yahoo! Answers datasets for learning user expertise.
- Multichannel Attention Refinement for Video Question Answering33
Appearance, motion, and audio features are extracted from the video, and question-guided attentions are refined to generate the expressive clues that support the correct answer in VideoQA.
- 33
- Reinforced Visual Perception with Tools32
A novel RL algorithm based on GRPO is introduced, designed to train models to reason with a suite of four visual tools through reinforcement learning, and achieves state-of-the-art performance on several perception-heavy benchmarks, significantly outperforming the supervised and text-based RL finetuning baselines.
- 32
- MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition32
AVMuST-TED is presented, the first dataset for Audio-Visual Multilingual Speech Translation, derived from TED talks, and MixSpeech, a cross-modality self-learning framework that utilizes audio speech to regularize the training of visual speech tasks, is proposed.
- Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding32
A novel semantic matching model is designed that analyzes the semantic similarity between object proposals and sentences in a coarse-to-fine manner and distill the coarse- to-fine semantic matching knowledge into a typical two-stage 3D visual grounding model, which reduces inference costs and improves performance by taking full advantage of the well-studied structure of the existing architectures.
- 32
- Question retrieval for community-based question answering via heterogeneous social influential network32
A novel framework named HSIN is proposed which encodes not only the question contents but also the asker’s social interactions to enhance the question embedding performance and demonstrates that employing the heterogeneous social network information outperforms the other state-of-the-art solutions in this task.
- Efficient processing of optimal meeting point queries in Euclidean space and road networks32
In the setting of Euclidean space, a general framework for answering all OMP query variants is proposed and the best algorithms for particular types of OMP queries in the literature are identified.
- A model-based approach for RFID data stream cleansing32
A probabilistic model for object tracking in a mobile environment is proposed and a Bayesian inference based approach for cleansing RFID data using the model is developed, which is validated through extensive simulations and demonstrated by using two real RFID applications of human tracking and conveyor belt monitoring.
- Semantic Alignment for Multimodal Large Language Models31
Semantic Alignment for Multi-modal large language models (SAM) is introduced, which aims to enhance the preservation of linking information for coherent analysis and align the semantics of different images before feeding them into LLM.
- 30
- ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control29
ControlSpeech is presented, a text-to-speech (TTS) system capable of fully cloning the speaker's voice and enabling arbitrary control and adjustment of speaking style and a new style controllable dataset called VccmDataset is made available.
- Subgraph-augmented Path Embedding for Semantic User Search on Heterogeneous Social Network29
A new concept of subgraph-augmented path for semantic user search is introduced, which considers sampling a set of object paths from a query user to a target user, and replaces the linear object sequence between its every two neighboring users with their shared subgraph instances.
- AudioLCM: Text-to-Audio Generation with Latent Consistency Models28
This work introduces AudioLCM, a novel consistency-based model tailored for efficient and high-quality text-to-audio generation and proposes the Guided Latent Consistency Distillation with a multi-step Ordinary Differential Equation (ODE) solver to overcome the convergence issue inherent in LDMs with reduced sample iterations.
- Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation28
SarCode is proposed, which introduces an all-to-all interaction module to model fine-grained query-item interactions and significantly improves the accuracy of candidate item matching while retaining the same level of retrieval efficiency with two-tower models.
- Learning the Beauty in Songs: Neural Singing Voice Beautifier28
This work introduces Neural Singing Voice Beautifier (NSVB), the first generative model to solve the SVB task, which adopts a conditional variational autoencoder as the backbone and learns the latent representations of vocal tone.
- Prior Knowledge and Memory Enriched Transformer for Sign Language Translation28
This paper proposes a novel method called Prior knowledge and memory Enriched Transformer (PET) for SLT, which incorporates the auxiliary information into vanilla transformer, and designs a multi-stream memory structure to obtain higher-quality translations.
- Contrastive Disentangled Meta-Learning for Signer-Independent Sign Language Translation28
This paper proposes a novel framework called contrastive disentangled meta-learning (CDM), which develops several improvements in both deep architecture and training mode and mainly attacks the signer-independent setting and focuses on augmenting the generalization ability of translation model.
- Exploring Human-Like Reading Strategy for Abstractive Text Summarization28
This work proposes a novel Hybrid learning model for Abstractive Text Summarization (HATS), which consists of three major components, a knowledge-based attention network, a multitask encoder-decoder network, and a generative adversarial network, which are consistent with the different stages of the human-like reading strategy.
- A Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications27
This work categorizes recent studies on DPO based on key research questions to provide a thorough understanding of DPO’s current landscape, and proposes several future research directions to offer insights on model alignment for the research community.
- 27
- 27
- Contrastive Learning with Positive-Negative Frame Mask for Music Representation27
A novel contrastive learning objective to accommodate both self-augmented positives/negatives sampled from the same music and the experimental results of two music-related downstream tasks, music classification and cover song identification, demonstrate the generalization ability and transferability of music representation learned by PEMR.
- 27
- 26
- Answer Identification from Product Reviews for User Questions by Multi-Task Attentive Networks26
By analyzing the similarity and discrepancy between explicit answers and reviews that can be answers, a novel multi-task deep learning method with carefully designed attention mechanisms is developed and demonstrates its effectiveness and superiority over competitive baselines.
- FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation25
Experimental results on text-to-audio generation demonstrate that FlashAudio's one-step generation performance surpasses the diffusion-based models with hundreds of sampling steps on audio quality and enables a sampling speed of 400x faster than real-time on a single NVIDIA 4090Ti GPU.
- AudioLCM: Efficient and High-Quality Text-to-Audio Generation with Minimal Inference Steps25
To overcome the convergence issue inherent in LDMs with reduced sample iterations, the Guided Latent Consistency Distillation with a multi-step Ordinary Differential Equation (ODE) solver is proposed, thereby achieving fast convergence and high-quality generation.
- Text-Guided Image Inpainting25
A progressively coarse-to-fine cross-modal generative network is proposed and the text-image-text training schema is adopted to generate visually consistent and semantically coherent images.
- FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion24
This work proposes FreeBind, an idea that treats multimodal representation spaces as basic units, and freely augments pre-trained unified space by integrating knowledge from extra expert spaces via "space bonds", and introduces two kinds of basic space bonds.
- ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos24
ANetQA is presented, a large-scale benchmark that supports fine-grained compositional reasoning over the challenging untrimmed videos from ActivityNet and is an order of magnitude larger than AGQA with a similar number of videos.
- 24
- Seeking and Updating with Live Visual Knowledge23
This work introduces LiveVQA, the first-of-its-kind dataset featuring 107,143 samples and 12 categories data specifically designed to support research in both seeking and updating with live visual knowledge, and dives deeply to the critical balance between adapter capacity and model capability when updating MLLMs with new visual knowledge.
- Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment23
Melodist is developed, a two-stage text-to-song method that consists of singing voice synthesis (SVS) and vocal-to-accompaniment (V2A) synthesis that leverages tri-tower contrastive pretraining to learn more effective text representation for controllable V2A synthesis.
- MuVi: Video-to-Music Generation with Semantic Alignment and Rhythmic Synchronization23
This paper introduces a contrastive music-visual pre-training scheme to ensure synchronization, based on the periodicity nature of music phrases, and demonstrates that the flow-matching-based music generator has in-context learning ability, allowing it to control the style and genre of the generated music.
- 23
- 23
- NGUARD+23
A general game bot detection framework for massively multiplayer online role playing games termed NGUARD+ (denoting NetEase Games’ Guard), which captures user patterns in order to identify game bots from player behavior sequences and can reveal outstanding robustness for game bots in mutated patterns and even in completely unseen patterns.
- Cross-modal Image-Text Retrieval with Multitask Learning23
Two regularization terms (variance and consistency constraints) are introduced to the cross-modal embeddings such that the learned common information has large variance and is modality invariant and to enable large-scale cross- modal similarity search, a flexible binary transform network is designed to convert the text and imageembeddings into binary codes.
- SCSMiner: mining social coding sites for software developer recommendation with relevance propagation23
A generativ e probabilistic expert ranking model upon which a consistency among projects is incorporated as graph regularization to enhance the expert ranking and a perspective of relevance propagation illustration is introduced.
- Cross-domain Aspect/Sentiment-aware Abstractive Review Summarization23
The proposed model CASAS (neural attentive model for Cross-domain Aspect/Sentiment-aware Abstractive review Summarization) leverages domain classification task, working on datasets of both source and target domains, to recognize the domain information of texts and transfer knowledge from source domains to target domains.
- SocialTransfer: Transferring Social Knowledge for Cold-Start Cowdsourcing23
A SocialTransfer model for cold- start crowdsourcing is proposed, which not only infers the expertise of warm- start crowdworkers from their past activities, but also transfers the expertise knowledge to cold-start crowdworkers via social connections.
- DSI-Bench: A Benchmark for Dynamic Spatial Intelligence22
The DSI-Bench is proposed, a benchmark with nearly 1,000 dynamic videos and over 1,700 manually annotated questions covering nine decoupled motion patterns of observers and objects and provides valuable findings and insights about the future development of general and expertise models with dynamic spatial intelligence.
- Accompanied Singing Voice Synthesis with Fully Text-controlled Melody22
MelodyLM is the first TTSong model that generates high-quality song pieces with fully text-controlled melodies, achieving minimal user requirements and maximum control flexibility, and Experimental results indicate that MelodyLM achieves superior performance in terms of both objective and subjective metrics.
- Flow-Based Unconstrained Lip to Speech Generation22
A novel flow-based non-autoregressive lip-to-speech model (GlowLTS) to break autoregressive constraints and achieve faster inference and the superiority of the proposed method is demonstrated through objective and subjective evaluation.
- 22
- Identifying and Tracking Sentiments and Topics from Social Media Texts during Natural Disasters22
The experimental results demonstrate that LDST performs very well at discovering topics and sentiments from social media and tracking their shifts in different geographical regions during emergencies and disasters.
- Cold-Start Expert Finding in Community Question Answering via Graph Regularization22
This work finds that it is difficult to address the cold-start issue in finding the experts in Community-based Question Answering (CQA) systems such as Quora.
- TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis21
TCSinger 2, a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts, is introduced, a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts.
- 21
- 21
- Video Question Answering via Knowledge-based Progressive Spatial-Temporal Attention Network21
A knowledge-based progressive spatial-temporal attention network is proposed to tackle the problem of video question answering by taking the spatial and temporal dimension of video content into account and employing an external knowledge base to improve the answering ability of the network.
- 21
- VarietySound: Timbre-Controllable Video to Sound Generation Via Unsupervised Information Disentanglement20
This paper proposes the task of generating sound with a specific timbre given a silent video input and a reference audio sample, and uses three encoders to disentangle each target sound audio into temporal, acoustic, and background information respectively.
- TAVT: Towards Transferable Audio-Visual Text Generation20
A novel Transferable Audio- visual Text Generation framework, named TAVT, which consists of two key components: Audio-Visual Meta-Mapper (AVMM) and Dual Counterfactual Contrastive Learning (DCCL).
- Play and rewind: Context-aware video temporal action proposals20
This work proposes a context-aware temporal action proposal network which makes full use of the contextual information in two aspects and designs an action-attention based re-ranking network which considers both surrounding proposal and initial actionness scores to assign true action proposals with high confidence scores.
- Attentional Image Retweet Modeling via Multi-Faceted Ranking Network Learning20
A novel attentional multi-faceted ranking network learning framework with multi-modal neural networks is developed for the proposed heterogenous IRM network to learn the joint image tweet representations and user preference representations for prediction task.
- WavReward: Spoken Dialogue Models With Generalist Reward Evaluators19
WavReward is proposed, a reward feedback model based on audio language models that can evaluate both the IQ and EQ of spoken dialogue systems with speech input that outperforms previous state-of-the-art evaluation models across multiple spoken dialogue scenarios.
- Boosting Speech Recognition Robustness to Modality-Distortion with Contrast-Augmented Prompts19
PCD: cluster-Prompt with Contrastive Decomposition is proposed, a robust framework for modality-distortion speech recognition, specifically devised to transpose the pre-trained knowledge from high-resource domain to the targeted domain by leveraging contrast-augmented prompts.
- OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup19
Omni-modal Sound Separation (OmniSep), a novel framework capable of isolating clean soundtracks based on omni-modal queries, encompassing both single-modal and multi-modal composed queries, is introduced, which blends query features from different modalities during training.
- 19
- SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory19
A novel attention-guided adaptive memory is devised to organize semantic information of history segments and enhance the visual representations with acceptable computation-aware latency in the task of simultaneous lip reading.
- Temporality-enhanced knowledgememory network for factoid question answering19
This paper devise an architecture named the temporality-enhanced knowledge memory network (TE-KMN) and apply the model to a factoid question answering dataset from a trivia competition called quiz bowl and demonstrates that the method achieves better performance than several state-of-the-art methods.
- Monochromatic and bichromatic reverse nearest neighbor queries on land surfaces19
These proposed algorithms are able to localize query evaluation by accessing just a small fraction of the surface data near the query point, which helps avoid shortest path evaluation on a large surface.
- OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios18
This paper introduces ShareChatX, the first comprehensive, large-scale dataset for spoken dialogue that spans diverse scenarios, and introduces OmniChat, a multi-turn dialogue system with a heterogeneous feature fusion module, designed to optimize feature selection in different dialogue contexts.
- Low-rank Prompt Interaction for Continual Vision-Language Retrieval18
The Low-rank Prompt Interaction (LPI) is innovatively proposed, which considers both cross-modal and cross-task interactions, and introduces explicit task contrastive constraints in the prompt learning process based on task semantic distances.
- Artificial Intelligence in Gerontology: Data-Driven Health Management and Precision Medicine18
Key AI applications in gerontology are reviewed, focusing on its role in analyzing multimodal data such as electronic health records, genomic data, medical imaging, and wearable device metrics, to enhance geriatric care.
- Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems18
Through in-depth analysis, it is found that CausalD largely improves the performance of tail users, reduces the performance heterogeneity, and enhances the overall performance.
- FastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis18
This work proposes to converge advantages from GANs and diffusion models by incorporating both classes, introducing dual-empowered modeling perspectives and showing that both variants enjoy an efficient 4-step sampling process and demonstrate superior sample quality and diversity.
- User Personalized Satisfaction Prediction via Multiple Instance Deep Learning18
This paper designs an efficient framework exploiting multiple instance learning property with deep learning tactic to model the question-answer pairs relevance and rank the asker's satisfaction possibility and demonstrates the feasibility of the proposed framework in predicting asker personalized satisfaction.
- Social recommendation via multi-view user preference learning18
This paper introduces the social recommendation (SR) problem utilizing microblogging data and develops an efficient alternating direction method of multipliers (ADMM) scheme to solve the problem via multi-view user preference learning.
- SkinGEN: an Explainable Dermatology Diagnosis-to-Generation Framework with Interactive Vision-Language Models17
SkinGEN is proposed, a diagnosis-to-generation framework that leverages the stable diffusion (SD) model to generate reference demonstrations from diagnosis results provided by VLM, thereby enhancing the visual explainability for users.
- Action Imitation in Common Action Space for Customized Action Image Synthesis17
Extensive experiments demonstrate TwinAct’s superiority in generating accurate, context-independent customized actions while maintaining the identity consistency of different subjects, including animals, humans, and even customized actors.
- Multi-modal Action Chain Abductive Reasoning17
The proposed model significantly outperforms existing video-language models in terms of effectiveness on the newly created MAR dataset and design a novel symbolic module that can complete strict derivation of the progressive action chain layer by layer.
- 17
- 17
- Abstractive meeting summarization by hierarchical adaptive segmental network learning with multiple revising steps17
This paper proposes the hierarchical neural encoder based on adaptive recurrent networks to learn the semantic representation of meeting conversation with adaptive conversation segmentation with a multi-step revising mechanism to refine the learned semantic representation.
- Speech Watermarking with Discrete Intermediate Representations16
This paper proposes DiscreteWM, a novel speech watermarking framework that injects watermarks into the discrete intermediate representations of speech by mapping speech into discrete latent space with a vector-quantized autoencoder and changing the modular arithmetic relation of discrete IDs.
- Multi-trends Enhanced Dynamic Micro-video Recommendation16
The DMR framework is proposed to explicitly model dynamic multi-trends of users' current preference and make predictions based on both the history and future potential trends and is validated over multiple state-of-the-art micro-video recommenders on two publicly available real-world datasets.
- Semantic Codebook Learning for Dynamic Recommendation Models16
The semantic metacode and semantic codebook, which stores disentangled item representations, ensures robust and accurate parameter generation and demonstrates that SOLID consistently outperforms existing DSR, delivering more accurate, stable, and robust recommendations.
- Prosody-TTS: Improving Prosody with Masked Autoencoder and Conditional Diffusion Model For Expressive Text-to-Speech16
Prosody-TTS is proposed, a two-stage pipeline that enhances prosody modeling and sampling by introducing several components: a self-supervised masked autoencoder to model the prosodic representation without relying on text transcriptions or local prosody attributes, which ensures to cover diverse speaking voices with superior generalization.
- SLED: Structure Learning based Denoising for Recommendation16
It is identified that an interaction’s related structural patterns in its neighborhood graph are potentially correlated with some outcome of implicit feedback (i.e., users’ ratings after consuming items), analogous to findings in other domains such as social networks.
- 16
- VLAD-VSA: Cross-Domain Face Presentation Attack Detection with Vocabulary Separation and Adaptation16
The VLAD aggregation method is adopted to quantize local features with visual vocabulary locally partitioning the feature space, and hence preserve the local discriminability, and the proposed vocabulary separation and adaptation method imitates the maximization step of the k-means algorithm in the end-to-end training.
- 16
- Link Prediction via Ranking Metric Dual-Level Attention Network Learning16
This paper proposes a novel proximity ranking metric attention network learning framework by jointly exploiting both node-level and path-level attention proximity of the endpoints to their betweenness paths for learning the discriminative feature representation for link prediction.
- Towards Transformer-Based Aligned Generation with Self-Coherence Guidance15
Self-Coherence Guidance is introduced, a method that dynamically refines attention maps using masks derived from previous denoising steps, ensuring precise alignment without additional training in Transformer-based Text-Guided Diffusion Models.
- MC-SLT: Towards Low-Resource Signer-Adaptive Sign Language Translation15
This paper proposes MC-SLT, a novel meta-learning framework that could exploit additional new-signer data via a support set, and output a signer-adaptive model via a few-gradient-step update, and devise diversity-aware meta- Adaptive weights for the token-wise cross-entropy losses.
- Open-Ended Long-Form Video Question Answering via Hierarchical Convolutional Self-Attention Networks15
A hierarchical convolutional self-attention encoder to efficiently model long-form video contents, which builds the hierarchical structure for video sequences and captures question-aware long-range dependencies from video context and a multi-scale attentive decoder to incorporate multi-layer video representations for answer generation.
- 15
- 15
- Partial Multi-Modal Sparse Coding via Adaptive Similarity Structure Regularization15
This paper proposes a partial multi-modal sparse coding framework termed Adaptive Partial Multi-Modal Similarity Structure Regularization for Sparse Coding (AdaPM2SC), which preserves the similarity structure within the same modality and between different modalities.
- 15
- UniSinger: Unified End-to-End Singing Voice Synthesis With Cross-Modality Information Matching14
This work proposes UniSinger, a unified end-to-end singing voice synthesizer, which integrates three abilities related to singing voice generation: singing voice synthesis (SVS), singing voice conversion (SVC), and singing voice editing (SVE) into a single framework and proposes a cross-modality information matching method to close the distribution gap between multi-modal inputs.
- Detector Guidance for Multi-Object Text-to-Image Generation14
Detector Guidance is introduced, which integrates a latent object detection model to separate different objects during the generation process and provides an 8-22\% advantage in preventing the amalgamation of conflicting concepts and ensuring that each object possesses its unique region without any human involvement and additional iterations.
- FastLTS: Non-Autoregressive End-to-End Unconstrained Lip-to-Speech Synthesis14
FastLTS is proposed, a non-autoregressive end-to-end model which can directly synthesize high-quality speech audios from unconstrained talking videos with low latency, and has a relatively small model size.
- 14
- FastLR14
Three methods to reduce the gap between FastLR and AR model are introduced, which leverage integrate-and-fire (I&F) module to model the correspondence between source video frames and output text sequence and add an auxiliary autoregressive decoder to help the feature extraction of encoder.
- 14
- Localizing Unseen Activities in Video via Image Query14
This work considers a new task to localize unseen activities in videos via image queries, named Image-Based Activity Localization, and proposes a novel self-attention interaction localizer to retrieve unseen Activities in an end-to-end fashion.
- IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models13
The IRBridge framework is introduced, which enables the direct utilization of generative models within image restoration bridges, offering a more flexible and adaptable approach to image restoration.
- Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model13
PFlow-VC is introduced, a conditional flow matching voice conversion model that leverages fine-grained discrete pitch tokens and target speaker prompt information for expressive voice conversion (VC) and improves timbre similarity by combining global timbre embeddings with time-varying timbre tokens.
- Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners13
Make-A-Voice is introduced as a multi-modal voice LLM and Experimental results demonstrate that Make-A-Voice exhibits superior audio quality and style similarity compared with competitive baseline models in monolingual/cross-lingual voice generation.
- MUG: A General Meeting Understanding and Generation Benchmark13
A large-scale general Meeting Understanding and Generation Benchmark (MUG) is established to benchmark the performance of a wide range of SLP tasks, including topic segmentation, topic-level and session-level extractive summarization and topic title generation, keyphrase extraction, and action item detection.
- 13
- Video-Guided Curriculum Learning for Spoken Video Grounding13
It is proved that in the case of noisy sound, the proposed video-guided curriculum learning can facilitate the pre-training process to obtain a mutual audio encoder, significantly promoting the performance of spoken video grounding tasks.
- Six‐bit, reusable comparator stage‐based asynchronous binary‐search SAR ADC using smart switching network13
A reusable comparator stage-based asynchronous binary-search Successive Approximation Register (SAR) analogue-to-digital converter (ADC) with a smart reference range prediction network is presented in this brief.
- 13
- CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale12
CODESYNC is introduced, a data engine for identifying outdated code patterns and collecting real-time code knowledge updates from Python third-party libraries and CODESYNCBENCH, a comprehensive benchmark for assessing LLMs' ability to stay synchronized with code evolution, which covers real-world updates for 220 APIs from six Python libraries.
- Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning12
This work investigates weakly-supervised spoken video grounding and proposes Semantic Interaction Learning (SIL), a novel framework consisting of the acoustic-semantic pre-training (ASP) and acoustic-visual contrastive learning (AVCL), which directly strengthen the interaction between speech and video by maximizing their mutual information.
- Video-Audio Domain Generalization via Confounder Disentanglement12
DeVADG performs cross-modal disentanglement and obtains fine-grained confounders at both class-level and domain-level using half-sibling regression and unpaired domain transformation, which essentially identifies domain-variant factors and class-shared factors that cause spurious correlations between features and false labels.
- Multi-purpose Tactile Perception Based on Deep Learning in a New Tendon-driven Optical Tactile Sensor12
A perceiving out-of-view neural network (O2VNet) is created to separate the mixed information with unequal affection in a multi-point touch task and can not only achieve real-time temporal sequence prediction, but also attain the average classification accuracy of 99.06%.
- 12
- Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations11
A novel approach that leverages Unified Representations to map different paired modalities together is proposed, effectively adapting DG methods to MMDG by enabling synchronized multi-modal improvements within the unified space.
- MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis11
S-DiT is introduced, a TTS system featuring an innovative sparse alignment algorithm that guides the latent diffusion transformer (DiT) and achieves state-of-the-art zero-shot TTS speech quality and supports highly flexible con-trol over accent intensity.
- Uni-Dubbing: Zero-Shot Speech Synthesis from Visual Articulation11
This work employs a methodology that incorporates modality alignment during the pre-training phase on multimodal datasets, uniquely facilitating Zero-Shot generalization through the process of freezing the video modality feature extraction component and the encoder module within the pretrained weights, thereby enabling effective cross-modal and cross-lingual transfer.
- Rethinking the Multimodal Correlation of Multimodal Sequential Learning via Generalizable Attentional Results Alignment11
A new constrained scheme called Multimodal Contextual Contrast (MCC) is proposed, which could align the multiple attentional results from both local and global perspectives, making the information capture more efficient.
- 11
- 11
- 11
- 11
- CrowdSeed11
CrowdSeed is presented, a system that automatically integrates human input for processing queries imposed on microblogs, and the effectiveness and efficiency of the system using real world data, as well as presenting interesting results from a game called "Who is in the CrowdSeed?".
- AHa-Bench: Benchmarking Audio Hallucinations in Large Audio-Language Models10
Through the development of a comprehensive evaluation framework, AHa-Bench aims to enhance robustness of LALMs, fostering more reliable and nuanced audio understanding in LALMs.
- 10
- UnicEdit-10M: A Dataset and Benchmark Breaking the Scale-Quality Barrier via Unified Verification for Reasoning-Enriched Edits10
A lightweight data pipeline that replaces multi-toolchains with an end-to-end model and a unified post-verification stage is introduced and a general benchmark that extends beyond basic edits to explicitly assess spatial and knowledge-driven reasoning is proposed.
- 10
- Data-Efficiently Learn Large Language Model for Universal 3D Scene Perception10
Chat-3D combines the 3D visual perceptual ability of pre-trained 3D representations and the impressive reasoning and conversation capabilities of advanced LLMs to achieve the first universal dialogue systems for 3D scenes, thus enabling LLMs to perceive the 3D world.
- STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation10
STARS is presented, which is, to the knowledge, the first unified framework that simultaneously addresses singing transcription, alignment, and refined style annotation and pioneers new methodologies for controllable singing voice synthesis.
- 10
- 10
- 10
- Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation10
A novel generalizable framework named C ontrastive To ken-Wise M eta-l earning (CtoML), which strives to transfer recognition skills to unseen performers is developed, and two complementary global and local contrastive losses are developed to maintain inter-class relationships and promote domain-independence.
- 10
- Parallel and High-Fidelity Text-to-Lip Generation10
A parallel decoding model for fast and high-fidelity text-to-lip generation (ParaLip) is proposed that predicts the duration of the encoded linguistic features and model the target lip frames conditioned on the encode linguistic features with their duration in a non-autoregressive manner and incorporates the structural similarity index loss and adversarial learning.
- 10
- 10
- The forgettable-watcher model for video question answering10
A TGIF-QA dataset for video question answering with the help of automatic question generation and a forgettable-watcher model to better utilize the temporal structure of the videos and the phrasal structures of the answers is proposed.
- CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling9
This work proposes a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling, which assigns identifiers to each candidate and treats the generating identifier as the retrieval target, and explores an effective coarse-to-fine scheme.
- 9
- 9
- 9
- DisCover: Disentangled Music Representation Learning for Cover Song Identification9
The goal of disentangling version-specific and version-invariant factors could make it easier for the model to learn invariant music representations for unseen query songs, and the disentanglement framework (DisCover) is proposed for CSI.
- Unsupervised Domain Adaptation for Video Object Grounding with Cascaded Debiasing Learning9
This paper addresses the Unsupervised Domain Adaptation (UDA) for the dense frame prediction task - Video Object Grounding and measures Grounding Coordinate Offset based on the semantic distance of the model's prediction across domains, based on which it alleviate source-domain bias in the target model through adversarial learning.
- 9
- 9
- Uncovering Causal Effects of Online Short Videos on Consumer Behaviors9
Both causal effects (i.e., informative and persuasive effects) are significant and have a strong correlation with the short videos' subjectivity scores, and the signs of these correlations vary with the prices of the products.
- DualSign: Semi-Supervised Sign Language Production with Balanced Multi-Modal Multi-Task Dual Transformation9
This paper proposes DualSign, a semi-supervised two-staged SLP framework, which can effectively utilize partially gloss-annotated text-pose pairs and monolingual gloss data, and substantially improves previous state-of-the-art SLP methods.
- Hierarchical Temporal Fusion of Multi-grained Attention Features for Video Question Answering9
This work proposes the multi-granularity temporal attention network that enables to search for the specific frames in a video that are holistically and locally related to the answer.
- Improved Dynamic Memory Network for Dialogue Act Classification with Adversarial Training9
This paper first cast the problem into a question and answering problem and proposed an improved dynamic memory networks with hierarchical pyramidal utterance encoder, which is not only robust, but also achieves better performance when compared with some state-of-the-art baselines.
- Crowdsourced Query Processing on Microblogs9
This paper proposes a new problem of minimizing the cost of processing crowdsourced queries on microblogs, given a specified accuracy threshold of users’ votes, and develops a greedy algorithm with a quality guarantee.
- APO: Enhancing Reasoning Ability of MLLMs via Asymmetric Policy Optimization8
View-R1-3B significantly enhances reasoning capabilities, showing an average 7\% gain over the base model and outperforming larger MLLMs (7-11B) on various reasoning benchmarks, and maintaining consistent improvement, demonstrating superior generalization.
- PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards8
A scalable multi-subject data generation pipeline that leverages powerful single-subject generation models to construct diverse and high-quality multi-subject training data and design a set of Pairwise Subject-Consistency Rewards and general-purpose rewards, which are incorporated into a refined reinforcement learning stage.
- SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning8
This work proposes SyncTalklip, a novel dual-tower framework designed to overcome the challenges of synchronization while improving lip-reading performance, and designs AV-SyncNet, a pre-trained multi-task model aiming to achieve a dual-focus on synchronization and intelligibility.
- Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis8
Adaptive Text-to-Talking Avatar (Ada-TTA), which designs a generic zero-shot multi-speaker TTS model that well disentangles the text content, timbre, and prosody and embraces recent advances in neural rendering to achieve realistic audio-driven talking face video generation.
- 8
- 8
- 8
- Be Relevant, Non-Redundant, and Timely: Deep Reinforcement Learning for Real-Time Event Summarization8
A Deep Reinforcement learning framework for real-time Event Summarization (DRES), which shows promising performance for resolving all three challenges (i.e., relevance, non-redundancy, timeliness) in a unified framework.
- Weak Supervision Enhanced Generative Network for Question Generation8
The Weak Supervision Enhanced Generative Network (WeGen) is proposed which automatically discovers relevant features of the passage given the answer span in a weakly supervised manner to improve the quality of generated questions.
- Leveraging read rates of passive RFID tags for real-time indoor location tracking8
This paper proposes a new indoor location tracking system, named PassTrack, which relies on the read rates of passive RFID tags for location estimation, and outperforms the previous localization approaches in localization accuracy, tracking efficiency and space applicability.
- 7
- Calibrating Prompt from History for Continual Vision-Language Retrieval and Grounding7
This paper designs Historical Prompt Calibration that includes two objectives to calibrate prompts and identifies two critical limitations in existing methods: Unfamiliarity across tasks, which prevents task-specific prompts from achieving forward propagation andeterogeneity between modalities, which makes it difficult to guarantee a consistent optimization direction for prompts of different modalities.
- 7
- DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect7
DopperBAS distinctly improves the representative WarpNet and BinauralGrad backbones in the phase error metric and reaches a new state of the art (SOTA): 0.780 (versus the current SOTA 0.807).
- Unsupervised Domain Adaptation for Referring Semantic Segmentation7
A cycle-consistent vision-language matching network to narrow down the domain gap and ease adaptation difficulty is proposed and has significant practical applications since they are capable generalising to new data sources without requiring corresponding text annotations.
- Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech7
Dict-TTS is proposed, a semantic-aware generative text-to-speech model with an online website dictionary that outperforms several strong baseline models in terms of pronunciation accuracy and improves the prosody modeling of TTS systems.
- 7
- Video Dialog via Multi-Grained Convolutional Self-Attention Context Networks7
A novel approach for video dialog called multi-grained convolutional self-attention context network, which combines video information with dialog history and can achieve higher time efficiency and the extensive experiments also show the effectiveness of the method.
- Language-Codec: Bridging Discrete Codec Representations and Speech Language Models6
In the Language-Codec, a Masked Channel Residual Vector Quantization mechanism along with improved fourier transform structures and attention blocks, refined discriminator design to address the gaps between discrete codecs and downstream speech language models are introduced.
- A Multimodal Evaluation Framework for Spatial Audio Playback Systems: From Localization to Listener Preference6
This work proposes MESA, a multi-modal evaluation framework for spatial audio systems, and introduces PSA-MOS, a high-quality multi-scene spatial audio dataset and develops SAE-Encoder, a spatial audio encoder that captures both acoustic-spatial cues and fine-grained perceptual patterns.
- 6
- 6
- 6
- 6
- 6
- 6
- DATE: Domain Adaptive Product Seeker for E-Commerce6
A Domain Adaptive Product Seeker (DATE) framework, regarding PR and PG as Product Seeking problem at different levels, to assist the query date the product and design a semantics-aggregated feature extractor for each modality to obtain concentrated and comprehensive features for following efficient retrieval and fine-grained grounding tasks.
- 6
- 6
- Modeling High-order Interactions across Multi-interests for Micro-video Reommendation (Student Abstract)6
A Self-over-Co Attention module to enhance user’s interest representation and use co-attention to model correlation patterns across different levels and then use self attention to modelcorrelation patterns within a specific level.
- 6
- Bi-Decoder Augmented Network for Neural Machine Translation6
This paper proposes a novel Bi-Decoder Augmented Network (BiDAN) for the neural machine translation task, and adds an auxiliary decoder to generate back the source language sequence at the training time.
- 6
- Efficient location-based search of trajectories with location importance6
An effective method to infer the importance of trajectory points from the temporal information is described and efficient R-tree-based and grid-based algorithms to answer Important Connected Trajectories (k-ICT) queries are proposed.
- Multimodal Conditional Retrieval with High Controllability5
The proposed CCSR model is the first to apply contrastive learning to conditional retrieval and introduces a novel Mixture-of-Expert models (MOE) system to enable collaboration between multiple expert systems.
- 5
- 5
- Open-Set Cross Modal Generalization via Multimodal Unified Representation5
This paper extends Cross Modal Generalization to open-set environments by proposing the more challenging Open-set Cross Modal Generalization (OSCMG) task, and proposes MICU, comprising two key components: Fine-Coarse Masked multimodal InfoNCE and Cross modal Unified Jigsaw Puzzles.
- GTA: Towards Generative Text-To-Audio Retrieval via Multi-Scale Tokenizer5
This work proposes GTA to move towards a generative text-to-audio retrieval paradigm, utilizing a multi-scale audio tokenizer to embed audio semantics into identifiers, while incorporating a dual-alignment strategy to ensure consistency between audio and text semantics.
- 5
- 5
- MulliVC: Multi-lingual Voice Conversion With Cycle Consistency5
MulliVC is proposed, a novel voice conversion system that only converts timbre and keeps original content and source language prosody without multi-lingual paired data, demonstrating the system's efficacy and the viability of the three-step approach with cycle consistency.
- VoiceTuner: Self-Supervised Pre-training and Efficient Fine-tuning For Voice Generation5
This work proposes VoiceTuner, with a self-supervised pre-training and efficient fine-tuning approach for low-resource voice generation, and introduces a multiscale transformer adapter to effectively update only around 1% parameters as a plug-and-play module.
- 5
- Analyzing and Mitigating Inconsistency in Discrete Audio Tokens for Neural Codec Language Models5
This paper quantitatively analyzes the DRI phenomenon within popular audio tokenizers such as EnCodec and effectively mitigates the DRI phenomenon of the neural audio codec.
- MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer5
An adversarial training framework integrated with teacher-student learning is introduced to enhance training stability and reduce the complexity of high-dimensional optimization in the unsupervised text style transfer task.
- 5
- Weakly-supervised Disentanglement Network for Video Fingerspelling Detection5
This paper proposes the Weakly-supervised Disentanglement Network, namely WED, that requires no additional knowledge, and better exploits the video-sentence weak supervisions, contributing to the state-of-the-art performance for fingerspelling detection and recognition.
- 5
- Regional Relation Modeling for Visual Place Recognition5
A regional relation module is proposed which models the regional relationships and converts the convolutional feature maps to the relational feature maps and a cascaded pooling method is designed to get discriminative relation descriptors by preventing the influence of confusing relations and preserving as much useful information as possible.
- Video Dialog via Progressive Inference and Cross-Transformer5
This paper introduces a novel progressive inference mechanism for video dialog, which progressively updates query information based on dialog history and video content until the agent think the information is sufficient and unambiguous.
- 5
- 5
- A probabilistic convex hull query tool5
This work proposes and studies probabilistic convex hull queries based on the possible world semantics, which are able to retrieve the objects whose probability of being on the conveX hull is at least α.
- Figma2Code: Automating Multimodal Design to Code in the Wild4
Figma2Code, a new task that advances design-to-code into a multimodal setting and aims to automate design-to-code in the wild, is introduced and ten state-of-the-art open-source and proprietary MLLMs are benchmarked.
- EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Models4
This work designs a K-step noise approximation strategy to reduce cloud inference frequency by using noise gradients between steps and applying cloud inference periodically to adjust errors, and designs a two-stage greedy search algorithm to efficiently find the optimal parameters for noise approximation and edge model switching.
- ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignment4
ICG is a novel framework that integrates MLLM-based prompting with personalized preference alignment to generate high-quality, contextually relevant covers, and adopts a multi-reward learning strategy that combines public aesthetic and relevance rewards with a personalized preference model trained from user behavior.
- 4
- 4
- 4
- CCL4Rec: Contrast over Contrastive Learning for Micro-video Recommendation4
The Contrast over Contrastive Learning framework for training recommender models is proposed, named CCL4Rec, which models the nuances of different augmented views by further contrasting augmented positives/negatives with adaptive pulling/pushing strengths, i.e., the contrast over (vanilla) contrastive learning.
- AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism4
A novel architecture AntPivot is proposed to solve the problem of livestream highlight detection by converting the detection of highlight clips into the search for optimal decision sequences and use the fully integrated representations to predict the final results in a dynamic-programming mechanism.
- 4
- Exploiting cross-source knowledge for warming up community question answering services4
This paper proposes a topical expertise model from the perspective of multi-view learning that incorporates the consistency existing among multiple views into a unified probabilistic graphic model and demonstrates the performance on two real-world datasets.
- Learning Max-Margin GeoSocial Multimedia Network Representations for Point-of-Interest Suggestion4
This paper proposes a novel max-margin metric geosocial multimedia network representation learning framework by exploiting users' check-in behavior and their social relations and develops a random-walk based learning method with max- margin metric network embedding.
- 4
- Generative Reasoning Recommendation via LLMs3
This work explores how to build GRRMs by adapting pre-trained LLMs, which achieves a unified understanding-reasoning-prediction manner for recommendation tasks, and proposes GREAM, an end-to-end framework that integrates three components.
- MelodyEdit: Zero-shot Music Editing with Disentangled Inversion Control3
MelodyEdit is proposed, a novel zero-shot music editing system based on innovative Disentangled Inversion Control (DIC) technique, which comprises Harmonized Attention Control and Disentangled Inversion, which outperforms state-of-the-art inversion techniques in editing fidelity and content preservation.
- AntCritic: Argument Mining for Free-Form and Visually-Rich Financial Comments3
A novel dataset AntCritic is collected to serve as a helpful complement to argument mining and two simple but effective model architectures are designed and conducted to cope with the challenges brought by scenario expansion.
- 3
- 3
- 3
- 3
- 3
- 3
- GamePipe: A Virtualized Cloud Platform Design and Performance Evaluation3
Preliminary performance results on the game latency and frame rate are reported and it is anticipated game cloud to have a performance gain or QoS improvement of 14% ~ 38% over video gaming on a thin client device such as a smartphone or a tablet computer.
- 3
- AnomalyCoT: A Multi-Scenario Chain-of-Thought Dataset for Multimodal Large Language Models2
Anoma-lyCoT, a multimodal Chain-of-Thought (CoT) dataset for multi-scenario IAD tasks, which consists of 37,565 IAD samples with the CoT data and is defined by challenging composite IAD tasks, thereby improving visual comprehension of defects across different types.
- MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendation2
This work introduces MIRA, a pioneering framework for task instruction recommendation that enables intuitive one-touch AI tasking on smartphones, and demonstrates substantial improvements in the accuracy of instruction recommendation.
- Entropy-based Coarse and Compressed Semantic Speech Representation Learning2
Experiments on ASR, speech-to-text translation, and voice conversion tasks demonstrate that the compressed representations perform on par with or better than dense token sequences, demonstrating the effectiveness of the proposed approach.
- 2
- 2
- 2
- 2
- Language Model is a Branch Predictor for Simultaneous Machine Translation2
This work utilizes a language model as a branch predictor to predict potential branch directions, namely, future source words, and utilizes the predicted source words to decode the output in advance to reduce translation latency.
- 2
- Set-Based Face Recognition Beyond Disentanglement: Burstiness Suppression With Variance Vocabulary2
This work argues that the two crucial issues in SFR, the face quality and burstiness, are both identity-irrelevant and variance-relevant, and proposes to separate the identity features with the variance features in a light-weighted set-based disentanglement framework.
- 2
- MGD-GAN: Text-to-Pedestrian Generation Through Multi-grained Discrimination2
The Multi-Grained Discrimination enhanced Generative Adversarial Network (MGD-GAN) is proposed, that capitalizes a human-part-based Discriminator (HPD) and a self-cross-attended (SCA) global Discriminators in order to capture the coherence of the complex body structure.
- 2
- 2
- Multi-Speaker Video Dialog with Frame-Level Temporal Localization2
A novel task of Multi-Speaker Video Dialog with frame-level Temporal Localization (MSVD-TL) to make video dialog systems more applicable and focuses on the characteristic of the video dialog generation process and exploit the relation among the video fragment, the chat history, and the following response to refine their representations.
- Multi-Label Community-Based Question Classification via Personalized Sequence Memory Network Learning2
This work introduces the personalized sequence memory network that leverages not only the semantics of questions but also the personalized information of askers to provide the sequence tag learning function to capture the high-order tag dependency.
- Video question answering via multi-granularity temporal attention network learning2
This work proposes the multi-granularity temporal attention network (MGTA-Net) that enables to search for the specific frames in a video that are holistically and locally related to the answer.
- 2
- A Multiple Input Floating Gate Based Arithmetic Logic Unit with a Feedback Loop for Digital Calibration2
Compared to the same ALU built in static logic, the proposed one using multiple input floating gate logic has the advantages of energy saving and large tolerance in fan-out.
- 2
- 2
- 2
- 2
- Energy Consumption Prediction Based on Time-Series Models for CPU-Intensive Activities in the Cloud2
The results indicate that the performance of ES model is better than the ARMA model in predicting the energy consumption of known activities, and sliding windows segmentation model and bottom-up segmentation models can all have satisfactory performance but the former is slightly better thanThe later.
- 2
- 2
- 2
- 1
- 1
- 1
- ExpTalk: Diverse Emotional Expression via Adaptive Disentanglement and Refined Alignment for Speech-Driven 3D Facial Animation1
ExTalk is proposed, which first applies an Adaptive Disentanglement Variational Autoencoder to decouple facial expression aligned with spoken language and emotions of speech through contrastive learning, and a Refined Alignment Diffusion is employed to iteratively refine the decoupled facial expression priors through diffusion-based perturbations.
- 1
- SPMDM: Enhancing Masked Diffusion Models through Simplifying Sampling Path1
The Simple Path Masked Diffusion Model (SPMDM), which partitions sequences into fixed-length, non-overlapping subsequences and applies varying noise scales to learn token-level and cross-subsequence dependencies and captures structural rules effectively.
- 1
- Continual Cross-Modal Generalization1
This work proposes the Continual Mixture of Experts Adapter (CMoE-Adapter) to project diverse modalities into a unified space while preserving prior knowledge, and introduces a Pseudo-Modality Replay (PMR) mechanism with a dynamically expanding codebook, enabling the model to adaptively incorporate new modalities using learned ones as guidance.
- Unleashing the Power of Natural Audio Featuring Multiple Sound Sources1
ClearSep is proposed, an innovative framework that employs a data engine to decompose complex naturally mixed audio into multiple independent tracks, thereby allowing effective sound separation in real-world scenarios and proposes a series of training strategies tailored to these separated independent tracks to make the best use of them.
- 1
- 1
- 1
- MART: Learning Hierarchical Music Audio Representations with Part-Whole Transformer1
This work introduces MART, a hierarchical music representation learning approach that facilitates feature interactions among cropped music clips while considering their part-whole hierarchies, and proposes a hierarchical part-whole transformer to capture the structural relationships between music clips in a part-whole hierarchy.
- Quality control in a training course of off-pump coronary artery bypass grafting surgery1
It is feasible to perform quality control by funnel plots and CUSUM method in OPCAB surgery to ensure the safety of the training course.
- Denoising Multi-modal Sequential Recommenders with Contrastive Learning1
A weakly-supervised framework based on contrastive learning for denoising multi-modal recommenders (dubbed Demure) is proposed, which circumvents the requirement of users' explicit feedback and identifies the noises by analyzing the modalities of all interacted items from a given user.
- Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG)1
A large-scale meeting dataset is constructed and released, the AliMeeting4MUG Corpus, to facilitate MUG, and a review of the dataset, track settings and baselines, and major techniques used in the submissions are summarized.
- Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding1
Under the UDA setting, a novel Adversarial Multi-modal Domain Adaptation (AMDA) method is introduced to adaptively adjust the model's scene-related knowledge by incorporating insights from the target data and mitigate the semantic gap between different modalities by aligning video-query pairs with related semantics.
- Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering1
A weakly supervised question grounding (WSQG) setting is devised, where only QA annotations are used and the relevant temporal boundaries are generated according to the temporal attention scores, which helps to optimize the temporal Attention scores and hence improve the video-language understanding in VideoQA model.
- 1
- 1
- StackReader: An RNN-Free Reading Comprehension Model1
This paper proposes StackReader, an end-to-end neural network model, to solve machine comprehension of text problem, without recurrent neural network units and its variants.
- Calibration method to reduce the error in logarithmic conversion with its circuit implementation1
Here, based on Mitchell's logarithmic conversion, the authors propose a fast calibration method using a fixed binary code with case judgement, which suppresses the conversion error and developed a highly paralleled circuit serving the proposed calibration method.
- A novel switchable pin method for regulating power in chip-multiprocessor1
This work not only builds on power delivery network (PDN), but also designs switchable pin working for two cores at the layout level, and test results show that the proposed switchablePin can well compensate voltage loss in chip multiprocessor, and transfer time of two modes is very short.
- 1
- 1
- Research of the Digital Map Symbolization Based on Controlling Rules1
The application of this digital map symbolization technique on the map had a good effect, testifying the feasibility of this technique, and summary of mapping symbolization controlling rules were summarized and suggested through which it could be effectively accomplished.
- Controls of Cartography Expert's Knowledge in the Process of Map's Symbolization1
The dissertation illustrates how to control the visualization by cartography expert's knowledge formamized but expressed with uncertain human's language in order to get excellent map vision effect.
- An Open and Binding Way for Managing Vector Map1
The authors articulate the open way by which the management for vector map becomes generic and flexible.
- –
- Unleashing Chain-of-Thought Style Reasoning for Text-to-Speech Systems–
Experimental results show that Thought-TTS successfully leverages chain-of-thought reasoning to comprehend context cues and produce accurate style descriptions through progressive thinking, enabling novel applications in controllable speech generation.
- Hallucinations in LLM-Based Code Summarization: Unveiling, Detection, and Mitigation–
Hallu-Eval is constructed, a novel dataset for unveiling hallucination phenomena and rigorously evaluating the effectiveness of hallucination detection and mitigation in LLM-based code summarization, and Hallu-Shield is introduced, an inference-time mitigation approach that leverages an external value model to guide LLMs toward producing more faithful summaries without costly retraining of the LLM itself.
- –
- –
- Vinci: Deep Thinking in Text-to-Image Generation using Unified Model with Reinforcement Learning–
This work proposes Vinci, a novel framework designed to enable interleaved image generation and understanding through deep reasoning capabilities, and introduces a momentum-based reward function, which dynamically adjusts the reward distribution by considering historical improvements, ensuring the stability of the model across multiple generations.
- –
- –
- –
- Controllable Text-to-Speech Synthesis with Masked-Autoencoded Style-Rich Representation–
This work proposes a two-stage style-controllable TTS system with language models, utilizing a quantized masked-autoencoded style-rich representation as an intermediary, and explores fully controlling the speaker's timbre and other stylistic information, and adjusting attributes like emotion for a specified speaker.
- –
- –
- –
- Cross-modal Observation Hypothesis Inference–
A large-scale text-video dataset that contains 39,796 meticulously annotated hypothesis inference examples and auxiliary commonsense knowledge for key video characters is proposed, and a strong baseline, COINNet, is designed, which features two perspectives: aligning temporally displaced textual observations with target videos via transformer-based multi-task learning, and inferring the action flow with non-parametric graph-based inference grounded in graph theory.
- –
- –
- –
- –
- Weakly-Supervised Video Moment Retrieval via Regularized Two-Branch Proposal Networks with Erasing Mechanism–
A novel Regularized Two-Branch Proposal Network with Erasing Mechanism to consider the inter-sample and intra-sample confrontments simultaneously and introduces an attention-guided dynamic erasing mechanism in enhanced branch to discover the complementary video-sentence relation.
- –
- –
- –
- –
- –
- Cortical Surface Shape Analysis Based on Alexandrov Polyhedra–
It is shown that the proposed method to use the Alexandrov polyhedra as surface-based shape signatures for cortical morphometry analysis is effective in pinpointing regional cortical structural changes impacted by AD.
- –
- –
- –
- An entity construction method of urban roads considering geometric connectivity and name consistency–
- Location-Based End-to-End Speech Recognition with Multiple Language Models–
End-to-End deep learning approaches for Automatic Speech Recognition (ASR) and multiple language models (LMs) are applied into decoding, where one LM is used for selecting appropriate answers and others, considering both context and grammar, for further decision are used.
- –
- Techniques of Energy-Efficient VLSI Chip Design for High-Performance Computing–
It is suggested that within the context of a youth-services agency such as this, where children are involved in any kind of abuse, is a positive thing to do and should be a priority.
- Integrating Side Information for Boosting Machine Comprehension–
A novel attention-based architecture is presented which could feed the side information representations into word level embeddings to explore the comprehension performance and show consistent improvements of the model over various baselines.
- –
- Detecting Temporal Proposal for Action Localization with Tree-structured Search Policy–
A reinforcement learning based method, which trains a localizer that learns a search policy that finds an optimal search path to locate a salient proposal based on the currently observing video segment in a tree structure, therefore reduces the number of video segments fed into the proposal detector.
- An Effective Energy Testing Framework for Cloud Workflow Activities–
This paper proposes an effective energy testing framework that can help to accurately test and analyze the baseline energy of physical and virtual machines in the cloud environment, and then obtain the energy consumption data of cloud workflow activities.
- Temporal Verification for Scientific Cloud Workflows: State-of-the-Art and Research Challenges–
This paper systematically investigates the problem of workflow temporal verification as the major approach for workflow temporal QoS (Quality of Service) assurance and presents four key research issues based on the introduction of a generic temporal verification framework.
- –
- –
- –
- The Research Status and Existing Problems of Microscopic Visual Servoing Based on Image–
From three aspects, which are Jacobin matrix calculating, automatic focusing technology of microscopic lens, and microscopic image fusing technology, it proposes some key problems to be urgently solved about the microscopic visual servoing based on image.
- Design of micro-capacitance measurement system based on AD7746–
The micro-capacitance measuring system was designed from two aspects of hardware and software, the capacitance-to-digital converter AD7746, which used the inter-integrated circuit serial bus to communicate with microcontrollers, and Matlab in upper computer used the serial device object to communication with micro Controllers.
- –
- –
- POSITION DETERMINATION USING LANDMARK LOCALIZATION FOR A MOBILE ROBOT–
An artificial landmark-based, automatic navigation system developed for a small-scale field robot showed that LRF-based position estimation was much more accurate when compared to the dead- Reckoning and dead-reckoning with gyroscope correction approaches.
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.