Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Sharad MehrotraSuggested from co-authorship
- Thomas S. HuangSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works273 from public data
- Image Retrieval: Current Techniques, Promising Directions, and Open Issues2,642
The survey includes 100+ papers covering the research aspects of image feature representation and extraction, multidimensional indexing, and system design, three of the fundamental bases of content-based image retrieval.
- 2,618
- Relevance feedback: a power tool for interactive content-based image retrieval2,082
A relevance feedback based interactive retrieval approach that effectively takes into account the subjectivity of human perception of visual content and the gap between high-level concepts and low-level features in CBIR.
- 905
- 642
- Jointly Modeling Embedding and Translation to Bridge Video and Language597
A novel unified framework, named Long Short-Term Memory with visual-semantic Embedding (LSTM-E), which can simultaneously explore the learning of LSTM and visual- semantic embedding and outperforms several state-of-the-art techniques in predicting Subject-Verb-Object (SVO) triplets.
- 554
- Automatically extracting highlights for TV Baseball programs507
This paper explores how to provide for the ability to extract highlights automatically, so that viewing time can be reduced, and presents results comparing output of algorithms against human-selected highlights for a diverse collection of baseball games with very encouraging results.
- Correlative multi-label video annotation504
A third paradigm is proposed which simultaneously classifies concepts and models correlations between them in a single step by using a novel Correlative Multi-Label (CML) framework and is compared with the state-of-the-art approaches in the first and second paradigms on the widely used TRECVID data set.
- 499
- 448
- 409
- A Survey on Food Computing405
This is the first comprehensive survey that targets the study of computing technology for the food area and also offers a collection of research studies and technologies to benefit researchers and practitioners working in different food-related fields.
- 293
- 283
- 263
- Distributed meetings258
A system that provides speaker clustering, spatial indexing, and time compression to provide a rich experience for people who miss a meeting and want to watch it afterward and a user study evaluation of the system is described.
- 244
- Relevance feedback techniques in interactive content-based image retrieval244
A relevance feedback based interactive retrieval approach, which effectively takes into account the above two characteristics in CBIR and greatly reduces the user's effort of composing a query and captures the users' information need more precisely.
- Multimedia search reranking228
Categorize and evaluate algorithms for visual search reranking, which reorders visual documents based on multimodal cues to improve initial text-only searches, and discuss relevant issues such as data collection, evaluation metrics, and benchmarking.
- 220
- Supporting ranked Boolean similarity queries in MARS212
The ranking and retrieval algorithms developed in MARS based on the Boolean retrieval model are discussed and the results of the experiments that demonstrate the effectiveness of the developed model for image retrieval are described.
- Automatically Solving Number Word Problems by Semantic Parsing and Reasoning210
A new meaning representation language is designed to bridge natural language text and math expressions and a CFG parser is implemented based on 9,600 semi-automatically created grammar rules.
- Location Discriminative Vocabulary Coding for Mobile Landmark Search191
A Location Discriminative Vocabulary Coding (LDVC) scheme, which achieves extremely low bit rate query transmission, discriminative landmark description, as well as scalable descriptor delivery in a unified framework is presented.
- 189
- 185
- 180
- A novel relevance feedback technique in image retrieval175
This paper introduces the first technique that not only has a solid theoretical framework but also takes into account the multi-level image content model and formulates a vigorous optimization problem.
- 174
- 171
- 165
- Browsing digital video163
A software video browsing application that combines many such features to understand user behavior when such capabilities are available, and results show that the most frequently used features were time compression, pause removal, and navigation using shot boundaries.
- Regularity and Conformity161
This paper proposes a hybrid predictive model integrating both the regularity and conformity of human mobility as well as their mutual reinforcement, and evaluates the proposed model using several city-scale mobility datasets including location check-ins, GPS trajectories of taxis, and public transit data.
- 156
- 153
- 150
- 145
- Action Recognition by Learning Deep Multi-Granular Spatio-Temporal Video Representation142
This paper presents a novel framework to boost action recognition by learning a deep spatio-temporal video representation at hierarchical multi-granularity using 2D or 3D convolutional neural networks to learn both the spatial and temporal representations.
- 142
- ARTiFACIAL: Automated Reverse Turing test using FACIAL features142
This paper designs attacks using the best existing face detectors and demonstrated the challenge they presented to bots, and proposes a new HIP algorithm based on detecting human face and facial features.
- Pairwise constraints based multiview features fusion for scene classification129
A novel multiview dimensionality reduction method for scene classification that takes both intraclass and interclass geometries into consideration, and has the best performance in scene classification.
- 123
- 120
- Sparse transfer learning for interactive video search reranking120
This article proposes a novel dimension reduction tool, termed sparse transfer learning (STL), to effectively and efficiently encode user's labeling information and conducts extensive experiments on the TRECVID 2005, 2006 and 2007 benchmark datasets and compared STL with popular dimension reduction algorithms.
- 119
- Automatic Generation of Visual-Textual Presentation Layout114
This article presents a system to automatically generate visual-textual presentation layouts by investigating a set of aesthetic design principles, through which an average user can easily create visually appealing layouts, and demonstrates that the designs achieve the best reading experience compared with the reimplementation of parts of existing state-of-the-art designs.
- Trajectory based event tactics analysis in broadcast sports video114
An effective tactic representation called aggregate trajectory is proposed which is constructed based on multiple trajectories using a novel analysis of temporal-spatial interaction among the players and the ball.
- Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting111
This paper proposes a novel structure-guided diffusion model for image inpainting named StrDiffusion, to reformulate the conventional texture denoising process under the structure guidance to derive a simplified denoising objective for im-age inpainting.
- Clickage111
It is argued that the massive amount of click data from commercial search engines provides a data set that is unique in the bridging of the semantic and intent gap, and preliminary studies on the power of large-scale click data are presented.
- 105
- 104
- Automating lecture capture and broadcast: technology and videography103
The design of a complete system that captures and broadcasts lectures automatically and report on a user study and a detailed set of video-production rules obtained from professional videographers who critiqued the system, which has been deployed in this organization for 2 years.
- Saliency Detection on Light Field101
Experimental results show that this approach can achieve 0.6--6.7% relative improvements over state-of-the-art methods in terms of the F-measure and Precision metrics, which demonstrates the effectiveness of the proposed approach.
- Beyond the Words100
A Heterogeneous Information Ensemble framework, called HIE, is proposed to predict users' personality traits by integrating heterogeneous information including self-language usage, avatar, emoticon, and responsive patterns to improve the performance of personality prediction.
- Building Hierarchical Representations for Oracle Character and Sketch Recognition98
A novel hierarchical representation is proposed that combines a Gabor-related low-level representation and a sparse-encoder-related mid- level representation that has beaten humans at recognizing general sketches.
- Hierarchical User Intent Graph Network for Multimedia Recommendation97
A novel framework, Hierarchical User Intent Graph Network, is developed, which exhibits user intents in a hierarchical graph structure, from the fine-grained to coarse- grained intents, and achieves significant improvements over the state-of-the-art methods, including MMGCN and DisenGCN.
- Enhancing Person Re-identification in a Self-Trained Subspace92
A self-trained subspace learning paradigm for person re-ID is proposed that effectively utilizes both labeled and unlabeled data to learn a discriminative subspace where person images across disjoint camera views can be easily matched.
- 86
- Cross-media semantic representation via bi-directional learning to rank85
This paper proposes a general cross-media ranking algorithm to optimize the bi-directional listwise ranking loss with a latent space embedding, which it is called Bi- Directional Cross-Media Semantic Representation Model (Bi-CMSRM).
- 85
- Click-through-based cross-view learning for image search84
This paper proposes a novel cross-view learning method for image search, named Click-through-based Cross-view Learning (CCL), by jointly minimizing the distance between the mappings of query and image in the latent subspace and preserving the inherent structure in each original space.
- Monocular Video Foreground/Background Segmentation by Tracking Spatial-Color Gaussian Mixture Models81
- Building an intelligent camera management system80
Motivated by different roles in a professional video production team, a multi-cinematographer single-director camera management system performs lecturer tracking, audience tracking, and video editing all fully automatically, and offers quality close to that of human-operated systems.
- 78
- 77
- 76
- 75
- 70
- A Unified Framework for Video Summarization, Browsing, and Retrieval69
This chapter recapitulates the key components of video highlights extraction and video retrieval and proposes a unified framework for video summarization, browsing, and retrieval to enable a user to go back and forth between browsing and retrieval.
- Image search—from thousands to billions in 20 years68
The challenges and opportunities brought by the astonishing increase of dataset scales from thousands to billions in the same time period are emphasized, as well as future research directions in effective visual representation, image knowledge base construction, implicit user feedback and crowdsourcing, mobile image search, and creative multimedia interfaces.
- Relevance Feedback Techniques in Image Retrieval68
An image object model can be modeled as a function of the image data, features, and representations and it is shown that the information embedded in an image is far more complex than that in text.
- Web video topic discovery and tracking via bipartite graph reinforcement model66
A bipartite graph model is proposed that represents the correlation between web videos and their keywords, and automatic topic discovery is achieved through two steps - coarse topic filtering and fine topic re-ranking.
- Learning Word Representation Considering Proximity and Ambiguity64
Proximity-Ambiguity Sensitive (PAS) models are proposed to produce high quality distributed representations of words considering both word proximity and ambiguity, and the strength of pooling-structured neural networks in word representation learning is revealed.
- See Widely, Think Wisely: Toward Designing a Generative Multi-agent System to Burst Filter Bubbles63
This study developed a prototype featuring LLM-powered multi-agent characters that users could interact with while reading social media content and found that multi-agent dialogues with gamification incentives could motivate users to engage with opposing viewpoints.
- Graph Attention Transformer Network for Multi-label Image Classification63
A Graph Attention Transformer Network is proposed, a general framework for multi-label image classification by mining rich and effective label correlation by using the cosine similarity value of the pre-trained label word embedding as the initial correlation matrix.
- Sense beauty via face, dressing, and/or voice63
This work performs a comprehensive study of female attractiveness conveyed by single/multiple modalities of cues, i.e., face, dressing and/or voice, and aims to uncover how different modalities individually and collectively affect the human sense of beauty.
- 61
- Correlative multilabel video annotation with temporal kernels58
This article proposes another paradigm of the video annotation method that simultaneously annotates the concepts as well as model correlations between them in one step by the proposed Correlative Multilabel (CML) method, which benefits from the compensation of complementary information between different labels.
- Unified tag analysis with multi-edge graph56
A new concept of multi-edge graph is proposed, through which a unified solution is derived for the different tag analysis tasks, and a core vertex-vs-edge tag equation unique for multi- edge graph is developed to unify the image/vertex tag and region-pair/edge tag.
- Hierarchical semantic image matching using CNN feature pyramid55
This work proposes a hierarchal image matching method using the CNN feature pyramid, named as CNN Flow, which takes advantage of the complementarity of different layers using guidance from higher layer to lower layer, and demonstrates the superiority of CNN Flow in image dense matching under challenging variations.
- Towards low bit rate mobile visual search with multiple-channel coding55
Quantitative comparisons to the state-of-the-arts demonstrate the significant advantages in descriptor compactness and retrieval mAP in mobile landmark, product, and CD/book cover search.
- 55
- Constraint Based Region Matching for Image Retrieval53
This paper proposes a constraint-based region matching approach to image retrieval that includes image segmentation, local feature extraction, first- and second-order constraints, and probabilistic regionweight estimation.
- Hierarchical affective content analysis in arousal and valence dimensions52
Experimental results show that CRF-based hierarchical method outperforms the one-step method on emotion type detection and majority of the users are satisfied with the proposed emotion detection.
- 52
- 52
- A Survey on Video Moment Localization51
This survey paper aims to present a comprehensive review of existing video moment localization techniques, including supervised, weakly supervised, and unsupervised ones, and to discuss promising future directions for this field, in particular large-scale datasets and interpretable video moments localization models.
- Advances in deep learning approaches for image tagging51
The purpose of this paper is to categorize and evaluate different image tagging approaches based on deep learning techniques, including data collection, evaluation metrics, and existing commercial systems, and conclude the advantages of different image tags paradigms.
- 50
- 49
- AI-Oriented Large-Scale Video Management for Smart City: Technologies, Standards, and Beyond48
This paper envisions the future deep feature coding standard for the AI-oriented large-scale video management and discusses existing techniques, standards, and possible solutions for these open problems.
- Who Will Reply to/Retweet This Tweet?48
By incorporating the dynamics of friendship relations, this approach significantly outperforms state-of-the-art models in terms of multiple evaluation metrics, such as MAP, NDCG and Topmost Accuracy.
- 48
- 46
- 43
- 42
- 41
- Towards decrypting attractiveness via multi-modality cues40
A novel Latent Dual-supervised Feature-Attribute-Task (LDFAT) network is proposed, where latent attributes are combined with manual attributes to contribute to the final attractiveness estimation, and extensive experimental evaluations well demonstrate the effectiveness of the proposed DFAT and LDFAT networks.
- 40
- 40
- 39
- 38
- 38
- 37
- Tagging Personal Photos with Transfer Deep Learning36
An ontology consisting of a hierarchical vocabulary tailored for personal photos is designed, to solve the semantic distribution gap and discover the intermediate image representations and ontology priors by deep learning with bottom-up and top-down transfers across two domains.
- 35
- As-Rigid-As-Possible Stereo under Second Order Smoothness Priors34
A mixture model is presented to combine the benefits of these two kinds of priors, whose energy function consists of a Laplacian operator on the disparity map which imposes pixel-wise second-order smoothness and a segment-wise matching cost as a function of quadratic surface, which encourages “as-rigid-as-possible” smoothness.
- 32
- 32
- Delving Globally into Texture and Structure for Image Inpainting30
This model is orthogonal to the fashionable arts, such as Convolutional Neural Networks, Attention and Transformer model, from the perspective of texture and structure information for image inpainting.
- 29
- Bilateral Correspondence Model for Words-and-Pictures Association in Multimedia-Rich Microblogs29
This paper uses a data-driven approach to analyze and modeling the associations between multimodal contents in microblog streams and proposes a novel generative model called the Bilateral Correspondence Latent Dirichlet Allocation (BC-LDA) model, which can assign flexible associations between pictures and words and is able to allow picture-word co-occurrence with bilateral directions, but also single modal association.
- 29
- 28
- Self-Supervised Graph Neural Network for Multi-Source Domain Adaptation28
A novel Self-Supervised Graph Neural Network (SSG) is proposed, where a graph neural network is used as the bridge to enable more effective inter-task information exchange and knowledge sharing and more expressive representation is learned by adopting a mask token strategy to mask some domain information.
- HoloBoard: a Large-format Immersive Teaching Board based on pseudo HoloGraphics28
A controlled experimental study to compare a HoloBoard class with a normal class indicated that the lecture-based class utilizing HoloBoard lead to slightly better learning outcomes and a significantly higher level of student engagement.
- A Distributed Approach Toward Discriminative Distance Metric Learning28
This paper proposes a discriminative metric learning algorithm, develops a distributed scheme learning metrics on moderate-sized subsets of data, and aggregate the results into a global solution that scales well with the data size and can be controlled by the partition.
- Semantic Retrieval of Video28
Different research works in 3 types of video, i.e., video of meetings, movies and broadcast news, and sports video are reviewed, putting them into a general framework of video summarization, browsing, and retrieval.
- 27
- From Artificial Intelligence to Augmented Intelligence27
Yong Rui discusses a panel he attended, which featured various competitions between AI and humans, and considers AI's progress and remaining challenges and its potential to augment human intelligence.
- 26
- Exploration of Visual Data26
This chapter discusses low-level and intermediate-level visual descriptors, and the proposed modeling scheme, which aims to solve this problem by automating the very labor-intensive and therefore time-heavy and expensive process of manually cataloging and annotating images.
- 26
- LMAgent: A Large-Scale Multimodal Agents Society for Multi-User Simulation25
This paper presents LMAgent, a very large-scale and multimodal agents society based on multimodal LLMs, and introduces a self-consistency prompting mechanism to augment agents’ multimodal capabilities, resulting in significantly improved decision-making performance over the existing multi-agent system.
- Learning to personalize trending image search suggestion25
This paper proposes a learning-based framework including two novel components, i.e., trending-aware weight-regularized matrix factorization (TA-WRMF) and TASM, able to suggest personalized trending search queries by learning user preference from many users as well as auxiliary common searches.
- An Egocentric Action Anticipation Framework via Fusing Intuition and Analysis24
Qualitative and quantitative evaluation results validate the effectiveness of the IAI framework, and demonstrate the advantage of bridging visual gap by utilizing multi-modal information, including both visual features of observed segments and sequential instructions of actions.
- 24
- 23
- 23
- 22
- 22
- Knowledge Mining: A Cross-disciplinary Survey21
This work presents a cross-disciplinary review where traits from different fields were exposed to further stimulate research ideas and to try to build bridges among these fields.
- Learning hierarchical video representation for action recognition21
This work represents the hierarchical structure of video with multiple granularities including, from short to long, single frame, consecutive frames (motion), short clip, and the entire video and proposes a novel deep learning framework to model each granularity individually.
- User-Click-Data-Based Fine-Grained Image Recognition via Weakly Supervised Metric Learning20
This work presents a novel fine-grained image recognition framework using user click data, which can bridge the semantic gap in distinguishing categories that are similar in visual, and proposes a weakly-supervised metric and template leaning with smooth assumption and click prior method.
- 20
- Toward efficient indexing structure for scalable content-based music retrieval19
Effective music indexing framework (EMIF) is designed based on a “classification-and-indexing” principle and consists of two main functionality modules: music classification and indexing module—multiple local indexing structures, one for each semantic category to reduce query response time significantly.
- 19
- Towards multi-semantic image annotation with graph regularized exclusive group lasso19
This paper proposes a general framework for harmoniously integrating the above multiple semantics, and investigates the problem of learning to annotate images with training images labeled in two or more correlated semantic spaces, such as fascinating nighttime, or exciting cat.
- 19
- 19
- What If We Could Not See? Counterfactual Analysis for Egocentric Action Anticipation18
A counterfactual analysis framework for egocentric action anticipation (CA-EAA) is presented to enhance the capacity and reduce the side-effect caused by semantic labels via a comparison between factual andcounterfactual outcomes.
- Where2Stand18
This article investigates human-scenery positional relationships and construct a photographic assistance system to optimize the position of human subjects in a given background scene, thereby assisting the user in capturing high-quality souvenir photos.
- Share-and-Chat17
This paper presents a novel two-stage deep learning-based approach to automatic video commenting that outperforms several state-of-the-art methods and achieves human-level video commenting.
- 17
- 17
- 16
- Lenovo Schedules Laptop Manufacturing Using Deep Reinforcement Learning16
Lenovo Research teamed with members of the factory operations group at Lenovo’s largest laptop manufacturing facility, LCFC, to replace a manual production scheduling system with a decision-making platform built on a deep reinforcement learning architecture, enabling a 20% reduction in the backlog of production orders and a 23% improvement in the fulfillment rate.
- DNN Flow: DNN Feature Pyramid based Image Matching16
The coarse to fine matching based on DNN feature pyramid is formulated as a series of optimization problems considering the guidance from top level, and extensive experiments demonstrate the superiority of DNN Flow in image matching under challenge variations.
- Information Retrieval beyond the Text Document16
Some of the progress made over the years toward exploring information retrieval beyond the text domain are reported, including visual feature extraction, retrieval models, query reformulation techniques, efficient execution speed performance, and user interface considerations.
- Image2Text: A Multimodal Caption Generator15
This work presents the Image2Text system, which is a real-time captioning system that can generate human-level natural language description for any input image, and develops a user-friendly system to automatically generated human- level captions for users.
- Image2Text15
This work presents the Image2Text system, which is a real-time captioning system that can generate human-level natural language description for any input image, and develops a user-friendly system to automatically generated human- level captions for users.
- 15
- 15
- Adaptive tree similarity learning for image retrieval15
A new approach to interactive image retrieval based on an adaptive tree similarity model that does not require any prior knowledge of the data and supports incremental learning with a fast convergence rate and achieves better performance than most approaches.
- Towards Workplace Metaverse: A Human-Centered Approach for Designing and Evaluating XR Virtual Displays14
This article explored how XR glasses can be used for PC’s virtual extended displays, and conducted user interviews and usability tests to propose a systematic user experience design and evaluation framework for XR virtual displays, and identified clarity as the most significant factor that affects user satisfaction.
- 14
- A Region-Based Representation of Images in MARS14
Simulation results show that the proposed algorithms achieve coding performance which compares favorably, both perceptually and objectively, to that achieved using state-of-the-art image/video coding techniques while additionally providing region-based support.
- 13
- Excuse me, but are you human?13
This work designs attacks using the best existing face detectors and demonstrated the difficulty to bots, and proposes a new HIP algorithm based on detecting human face and facial features.
- Efficient indexing, browsing and retrieval of image/video content13
Novel approaches to feature analysis, content representation, indexing and retrieval are presented and the introduction of relevance feedback techniques to image retrieval greatly improves the retrieval performance and alleviates the user's query formulation burden.
- 12
- MSR-VTT: A Large Video Description Dataset for Bridging Video and Language [Supplementary Material]11
When organizing the Microsoft Research Video To Language challenge, some sentences annotated by AMT workers are identical in one video clip or very similar in one category so the performance cannot be well matched with what was reported in the CVPR paper.
- 11
- Learning Click-Based Deep Structure-Preserving Embeddings with Visual Attention10
A novel click-based Deep Structure-Preserving Embeddings with visual Attention (DSPEA) model is presented, which consists of two components: deep convolutional neural networks followed by image embedding layers for learning visual embedding, and a deep neural networks for generating query semantic embedding.
- Learning Cross Space Mapping via DNN Using Large Scale Click-Through Logs10
A unified DNN model for image-query similarity calculation by simultaneously modeling image and query in one network is proposed, named the cross space mapping (CSM) model, which contains two parts, a convolutional part and a query-embedding part.
- Sketch Recognition with Natural Correction and Editing10
Based on the proposed algorithms, a real-time sketch recognition system is built to recognize hand-drawn flowcharts and diagrams with flexible interactions.
- 9
- 9
- 9
- Multi-order visual phrase for scalable image search9
Multi-order visual phrase is proposed which contains two complementary clues: center visual word quantized from the local descriptor of each image keypoint and the visual and spatial clues of multiple nearby keypoints and is more robust to quantization error than existing visual phrases.
- Large-scale multilabel propagation based on efficient sparse graph construction8
A novel sparse graph based multilabel propagation (SGMP) scheme for super large scale datasets, which encodes the label information of an image as a unit label confidence vector and naturally imposes inter-label constraints and manipulates labels interactively.
- Video ChatBot7
A video chatbot that can generate human-level emotional comments referring to the videos shared by users and trigger a conversation with users is demonstrated, which has great potential to increase live social interactions.
- Resorting Relevance Evidences to Cumulative Citation Recommendation for Knowledge Base Acceleration7
Three types of relevance evidences including entities’ profiles, existing citations in KBs, and temporal signals are explored, to supplement the definitions of relevance levels and show that all these approaches outperform the corresponding baselines.
- 7
- 7
- Building Embodied EvoAgent: A Brain-inspired Paradigm for Bridging Multimodal Large Models and World Models6
A brain-inspired learning and evolution paradigm for embodied agents is proposed that significantly improves the performance of embodied agents in a series of tasks and enhances their generalization ability in zero-shot tasks through embodied exploration experience and online evolution.
- Semi-Supervised Unconstrained Head Pose Estimation in the Wild6
This work proposes the first semi-supervised unconstrained head pose estimation method SemiUHPE, which can leverage abundant easily available unlabeled head images and proposes dynamic entropy based filtering to adaptively remove unlabeled outliers as training progresses by updating the threshold in multiple stages.
- 6
- 6
- 6
- 6
- What Visual Attributes Characterize an Object Class?6
The approach ensures the learned visual attributes to be visually recognizable and representative, in contrast to manually constructed attributes that contain properties difficult to be visualized, e.g., “smelly,” “smart.”
- Annotating web images using NOVA6
A new sparsity-based approach NOVA (NOn-conVex group spArsity) is proposed, which is the first to introduce non-convex penalty for group selection in high-dimensional heterogeneous features space and achieves the consistency.
- 6
- 5
- Applications5
Video summarization, browsing, and retrieval in different applications is reviewed, finding the technology is successful in small or medium-scale applications such as highlight generation for a particular sport, but still faces great technical challenges for large-scale deployments such as airport security and general surveillance.
- Disentangled Partial Label Learning4
This paper makes the first attempt towards disentangled PLL and proposes a novel approach named TERIAL, which makes predictions according to derived disentangled representation of instances and label embeddings and theoretically proves that the neighborhood routing mechanism will converge to the point estimate that maximizes the marginal likelihood of observed PL training examples.
- 4
- Sequence-to-Sequence Learning via Shared Latent Representation4
This paper proposes a star-like framework for general and flexible sequence-to-sequence learning, where different types of media contents could be encoded to and decoded from a shared latent representation (SLR) (the central node).
- 4
- LEGO-MM: LEarning Structured Model by Probabilistic loGic Ontology Tree for MultiMedia4
A new framework is proposed, termed LEarning Structured Model by Probabilistic loGic Ontology Tree for MultiM edia (LEGO 1 -MM), which can seamlessly integrate both the new target training examples and the existing primitive concept models to infer the more complex concept models.
- 4
- MSR KMG at TREC 2014 KBA Track Vital Filtering Task4
The strategy for vital filtering for TREC 2014 KBA track Vital Filtering task is to first retrieve as many relevant documents as possible and then apply classification and ranking methods to differentiate vital documents from non-vital documents.
- 4
- Highlights Extraction from Unscripted Video4
This chapter has reported the results of sports highlights extraction based on audio classification and the correlation between the applause/cheering sound with exciting moments.
- 4
- SPEAKER TRACKING USING PARTICLE FILTER SENSOR FUSION4
A new fusion framework that combines both the bottom-up and top-down approaches to probabilistically fuse multiple sensing modalities and provides a natural scheme to evaluate the performance of the individual trackers and dynamically updates their object states in nonstationary environments and hence much more robust in longterm tracking.
- 4
- 4
- Media Content Management4
In one example embodiment, a system includes a first device configured to transmit a request for temporary user rights to media content and a server configured to issue the temporaryuser rights to the media content to a user account authenticated on the first device.
- 3
- Scalable Visual Instance Mining with Instance Graph3
This paper proposes a scalable mining method by leveraging the graph structure with images as vertices, capable of mining challenging small visual instances with diverse variations and outperformed the state of the arts on both datasets with higher recalls.
- Unsupervised Template Mining for Semantic Category Understanding3
An unsupervised approach to constructing templates from a large collection of semantic category names, and use the templates as the semantic representation of categories, achieves significantly better results than baseline methods.
- 3
- Learning concepts by modeling relationships3
This talk argues that within the annotation paradigm, the relationship-based annotation approach outperforms other existing annotation approaches, because individual concepts are considered jointly instead of independently.
- Conclusions3
A unique and unified framework is presented for video summarization, browsing, and retrieval to support going back and forth between the video ToC and the video index for scripted video, and between video highlights and videoIndex for unscripted video.
- 3
- A Framework for Garment Shopping over the Internet3
This chapter proposes a framework for integrated design, advertisement and retailing of garments over the Internet that would make use of the latest research in computer graphics, image processing, computer vision and artificial intelligence.
- Video key frame extraction by unsupervised clustering and feedback adjustment3
This paper introduces a new algorithm for key frame extraction based on unsupervised clustering and provides a feedback chain to adjust the granularity of the extraction result.
- Collective domain adversarial learning for unsupervised domain adaptation2
This work proposes a novel adversarial domain adaptation framework that leverages the collective assumption to model and exploit higher-order interactions among samples and achieves a more robust domain alignment, demonstrating enhanced resilience to noise and domain ambiguity.
- 2
- DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization2
This work constructs a novel dataset DomainVerse and proposes two methods called Domain CLIP and Domain++ CLIP for tuning-free adaptive domain generalization and proposes two methods called Domain CLIP and Domain++ CLIP for tuning-free adaptive domain generalization.
- Toward Egocentric Compositional Action Anticipation with Adaptive Semantic Debiasing2
This article devise a novel counterfactual analysis scheme to adaptively amplify or penalize the effect of semantic experience by considering the discrepancy both among categories and among examples, and demonstrates that the traditional counterfactual analysis scheme is a special case of the devised adaptive counterfactual analysis scheme.
- 2
- Predicting Smartphone Adoption in Social Networks2
A supervised model that takes all three factors for smartphone adoption and at the same time learns the relative contribution of each factor from the data is developed and experimental results demonstrate the effectiveness of the proposed model.
- CeleBrowser: An example of browsing big data on small device2
This demonstration shows how query logs collected for six months from two vertical search engines are leveraged for mining hot events and videos of celebrities for exploratory browsing of large number of Web videos on a device with small screen.
- Up-Fusion2
This article proposes a novel evolving fusion method based on the online portfolio selection theory that takes into account the correlation among different information sources and evolves the fusion model when new multimedia data is added.
- Multi-order visual phrase for scalable partial-duplicate visual search2
Multi-order visual phrase (MVP) is proposed which contains two complementary clues: center visual word quantized from the local descriptor of each image keypoint and the visual and spatial clues of multiple nearby keypoints and is more robust to quantization error than existing visual phrases.
- Big Data and Image Search2
Editor-in-Chief Yong Rui reflects on what has been achieved in the past 20 years of multimedia research and where to go next, specifically on the past and future of image search.
- Proceedings of the Second International Conference on Internet Multimedia Computing and Service2
The Fourth International Conference on Internet Multimedia Computing and Service (ICIMCS 2012) was held on the campus of Central China Normal University, Wuhan, Hubei, China during September 7-11, 2012 and received 60 regular papers and only 32 of them were accepted.
- 2
- 2
- A United Framework for Video Browsing and Retrieval2
This work focuses on video retrieval, which deals with how to use the representation of the video system to help viewers browse the video ontent, and the relationship between these four areas is illustrated in Figure 1.
- 1
- 1
- 1
- Up-fusion1
A new evolving fusion method is proposed based on the online portfolio selection theory that takes the correlation among different information sources into account, and evolves the fusion model when new multimedia data is added.
- Automated Lecture Services1
This chapter surveys existing approaches for providing automated lecture services and examines two major challenges in providing such services, namely, how to capture, analyze and render the lecture content automatically, and how to provide live/on-demand lecture viewing/browsing experience with an automated end-to-end system.
- 1
- 1
- Edge-Based Structural Features1
This chapter proposes edge-based structural features for image representation, develops an algorithm for effective extraction of such features, and explores the effectiveness of these features in aiding visual retrieval tasks.
- 1
- 1
- –
- Classifier-induced Reciprocal Points for Multi-label Open-set Recognition–
A novel framework CREM is proposed, i.e. Classifier-induced REciprocal point for Multi-label open-set recognition, which rethinks the above problem from the reciprocal point perspective, and reciprocal points are formulated by explicitly constraining the opposition feature space to a learnable bounded margin.
- Continual novel class discovery under domain shift with entropy-based selection and representation evolution–
This paper proposes a novel progressive method consisting of a discovery phase and an enhancement phase to improve model discriminative capability and demonstrates that it outperforms existing baseline methods under various conditions.
- Collaborative Dual Representations for Semi-Supervised Partial Label Learning–
This work proposes a novel approach named CODUAL, which jointly learns a pair of dual representations for each instance: the predictive class distribution and the low-dimensional embedding, which facilitates more effective use of unlabeled data for disambiguation.
- A Bootstrap Pipeline for Chat-Based Image Retrieval with Effective Question Generation–
Under a fair comparison protocol, the proposed method can not only address the critical data gap but also achieve state-of-the-art results, which substantially surpass GPT-4o and GPT-4-Turbo through the fine-tuning of an 8B model.
- –
- –
- –
- –
- –
- Epistemic Graph: A Plug-And-Play Module For Hybrid Representation Learning–
A novel Epistemic Graph Layer (EGLayer) is developed to enable hybrid learning, such that the information can be exchanged more effectively between the deep features and a structured knowledge graph.
- MMPT'21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding–
This workshop aims to gather peer researchers on related topics for more insightful discussion and to attract more researchers to explore and investigate more opportunities of designing and using innovative pre-training models for multimedia tasks.
- –
- Changes on the Horizon for the Multimedia Community–
In his final EIC Message, Yong Rui looks at how the field of multimedia has changed over the years and introduces the new Editor in Chief, whose term will start in 2018.
- Understanding Multimedia–
EIC Yong Rui reflects on how much AI and multimedia technologies have advanced in the past 60 years, discussing progress made in visual media understanding and the challenges ahead for the video2text problem.
- Recognizing Exceptional Contributions–
EIC Yong Rui announces the winners of IEEE MultiMedia's Best Article and Best Department awards for 2016, both of which appeared in the July-Sept.
- Working with the Domain Experts–
A classification and recognition system for all entities in the world is difficult, but if the authors start with some important vertical domains, they might be able to achieve good results with help from domain experts.
- Establishing Best Papers for IEEE MultiMedia–
The first winners of these two new awards starting this year--best article and best department of IEEE MultiMedia are announced.
- Multimedia Goes Beyond Content–
This 3C model of multimedia--including content, context, and connectivity--lets us process and understand multimedia in a new way.
- –
- Deep Neural Networks: Another Tool for Multimedia Computing–
Editor-in-Chief Yong Rui reflects on deep neural networks, which have quickly gained momentum in various pattern recognition and multimedia applications.
- –
- –
- –
- 14 Automated Lecture Services–
This chapter surveys existing approaches for providing automated lecture services and examines two major challenges in providing such services, namely, how to capture, analyze and render the lecture content automatically, and how to provide live/on-demand lecture viewing/browsing experience with an automated end-to-end system.
- A Unifying Multi-Label Temporal Kernel Machine with Its Application to Video Annotation–
This paper proposes a new paradigm of the video annotation method that simultaneously annotates the concepts as well as model correlations between them in one step by the proposed Correlative Multi-Label (CML) method and extends the proposed method to exploit the rich temporal information in the videos.
- –
- Content‐Based Multimedia Retrieval–
Both classic CBMR and the current mainstream are introduced, where QBE is the most typical scenario for multimedia retrieval before the year of 2000, whereas query by text and by combination of text and examples have become two new mainstream scenarios thereafter.
- Guest Editors' Introduction: Advances in Multimedia Computing–
The guest editors discuss their selection of the best papers from the ACM Multimedia 2007 Conference.
- Video Indexing–
This chapter concludes that audio features are a rich source of content semantics, which enables much easier access to content semantics than visual analysis.
- Video Table-of-Content Generation–
This chapter presents an effective scene-level ToC construction technique based on intelligent, unsupervised clustering based on Gaussian normalization that provides an open framework for structure analysis of video features.
- –
- –
- –
- Improving retrieval performance by region constraints and relevance feedback–
In this paper, region features and relevance feedback are used to improve the performance of CBIR and simultaneously models both region properties and their spatial relationships in a probabilistic framework.
- Relevance Feedback for Visual Data Retrieval–
The on-line learning problem for content-based multimedia information retrieval is focused on, i.e., the on-linespecialization of the effectiveness of the representation and the selection of a similarity metric during the retrieval process.
- Future Research Directions–
To extend and continue the research carried out in this book, the emphasis may be put on the following challenging topics.
- –
- Toward Unification of Keywords and Low-Level Contents–
A WARF (word association via relevance feedback) formula is proposed as a pseudoclassification algorithm for the learning of the term similarity matrix during user interaction that can be applied for keyword semantic grouping, thesaurus construction, and soft query expansion during intelligent image retrieval with user-in-the-loop.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.