Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks28 from public data
- A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook142
This article introduces seminal foundation models proposed or adaptable for reasoning, highlighting the latest advancements in various reasoning tasks, methods, and benchmarks, and delves into the potential future directions behind the emergence of reasoning abilities within foundation models.
- PIXART-$$\Sigma $$: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation112
- 88
- 79
- A Survey of Reasoning with Foundation Models73
This paper introduces seminal foundation models proposed or adaptable for reasoning, highlighting the latest advancements in various reasoning tasks, methods, and benchmarks and discusses the relevance of multimodal learning, autonomous agents, and super alignment in the context of reasoning.
- ScheMoE: An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling61
This work proposes an extensible and efficient MoE training system, ScheMoE, which is equipped with several features and outperforms existing state-of-the-art MoE systems, Tutel and Faster-MoE, by 9%-30%.
- SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator55
This work has identified a key pattern: certain seemingly meaningless separator tokens contribute disproportionately to attention scores compared to semantically meaningful tokens, which suggests that information of the segments between these separator tokens can be effectively condensed into the separator tokens themselves without significant information loss.
- EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge50
This paper proposes EdgeFM, a novel edge-cloud cooperative system with open-set recognition capability that selectively uploads unlabeled data to query the FM on the cloud and customizes the specific knowledge and architectures for edge models.
- DAPE: Data-Adaptive Positional Encoding for Length Extrapolation42
A Data-Adaptive Positional Encoding (DAPE) method, which dynamically and semantically adjusts based on input context and learned fixed priors is proposed, which enhances model performances in terms of trained length and length generalization.
- AutoBERT-Zero: Evolving BERT Backbone from Scratch42
This work makes the first attempt to automatically discover novel pre-trained language model (PLM) backbone on a flexible search space containing the most fundamental operations from scratch and proposes an Operation-Priority Neural Architecture Search (OP-NAS) algorithm, which optimizes both the search algorithm and evaluation of candidate models.
- EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation28
This paper has a critical insight that improving the feed-forward network (FFN) in BERT has a higher gain than improving the multi-head attention (MHA) since the computational cost of FFN is 2$\sim$3 times larger than MHA.
- 26
- TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs22
This paper introduces TaskSense, a novel system that coordinates multiple sensor systems in response to users' complex queries using a sensor language that automatically translates the capabilities and data dependencies of sensor systems into vocabularies and grammar rules that can be understood by LLMs.
- Scaling Law for Language Models Training Considering Batch Size20
This paper empirically investigates how a critical hyper-parameter, i.e., the global batch size, influences the LLM training prdocess, and establishes a basic scaling law on model size and training data amount.
- 13
- 10
- 10
- 8
- 8
- 8
- 5
- A Study on Transformer Configuration and Training Objective5
Bamboo, an idea of using deeper and narrower transformer configurations, for masked autoencoder training, is proposed, effective in alleviating the over-smoothing issue in deep transformer training.
- An Efficient Edge-Cloud Collaboration System With Foundational Models for Open-Set IoT Applications4
DeepEdgeFM is introduced, an edge-cloud collaborative system with FMs that enables open-set learning, simultaneously achieving generalizability and efficiency for IoT applications and outperforms state-of-the-art baselines.
- 4
- 2
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.