🔥 Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion

💡 The paper introduces Orthrus, a dual architecture framework that combines the strengths of autoregressive large language models and diffusion models to achieve fast parallel token generation while maintaining exact inference fidelity. The problem with standard autoregressive decoding is that it is sequential, which represents a fundamental bottleneck for high throughput inference. Diffusion language models try to address this issue with parallel generation, but they suffer from performance degradation, high training costs, and lack of convergence guarantees.

The Orthrus framework resolves this issue by augmenting a frozen large language model with a lightweight trainable module to create a parallel diffusion view alongside the standard autoregressive view. Both views attend to the same high fidelity key value cache, where the autoregressive head executes context pre filling to construct accurate key value representations, and the diffusion head executes parallel generation. The framework employs an exact consensus mechanism between the two views to guarantee lossless inference.

The results show that Orthrus delivers a speedup of up to 7.8 times with only a constant memory cache overhead and minimal parameter additions. This is achieved by sharing key value caches and using a consensus mechanism, which allows the framework to maintain exact inference fidelity while generating tokens in parallel. Overall, the Orthrus framework provides a simple and efficient solution to the problem of slow sequential decoding in autoregressive large language models, and it has the potential to be seamlessly integrated into existing transformer architectures.

📅 Published on May 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.12825
• PDF: https://arxiv.org/pdf/2605.12825

🤖 Models citing this paper:
• https://huggingface.co/chiennv/Orthrus-Qwen3-8B
• https://huggingface.co/chiennv/Orthrus-Qwen3-4B
• https://huggingface.co/chiennv/Orthrus-Qwen3-1.7B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#DiffusionLanguageModels #ParallelTokenGeneration #AutoregressiveDecoding #DualViewDiffusion #LargeLanguageModels

🔥 DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI

💡 The paper introduces DataFlow, a framework for unified data preparation and workflow automation in the context of large language models. The problem addressed is the current lack of scalable and reliable data preparation pipelines, which are often dominated by ad-hoc scripts and loosely specified workflows, hindering reproducibility and model performance.

To address this challenge, the authors propose DataFlow, a framework that provides system-level abstractions for modular, reusable, and composable data transformations. It includes a PyTorch-style pipeline construction API and nearly 200 reusable operators, as well as six domain-general pipelines for various tasks such as text, mathematical reasoning, and code.

The framework also includes DataFlow-Agent, which can automatically translate natural-language specifications into executable pipelines. This is achieved through operator synthesis, pipeline planning, and iterative verification.

The results show that DataFlow consistently improves downstream large language model performance across six representative use cases. The framework outperforms curated human datasets and specialized synthetic baselines, achieving significant gains in execution accuracy and average improvements on code benchmarks.

For example, the math, code, and text pipelines achieve up to 3 percent execution accuracy in Text-to-SQL, 7 percent average improvements on code benchmarks, and 1-3 point gains on math benchmarks. Additionally, a unified dataset produced by DataFlow enables base models to surpass counterparts trained on larger datasets.

Overall, the paper demonstrates that DataFlow provides a practical and high-performance substrate for reliable, reproducible, and scalable large language model data preparation, and establishes a system-level foundation for future data-centric AI development.

📅 Published on Dec 18, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2512.16676
• PDF: https://arxiv.org/pdf/2512.16676
• Project Page: https://github.com/OpenDCAI/DataFlow

📊 Datasets citing this paper:
• https://huggingface.co/datasets/OpenDCAI/dataflow-demo-Text2SQL
• https://huggingface.co/datasets/OpenDCAI/dataflow-mm-context_vqa
• https://huggingface.co/datasets/OpenDCAI/dataflow-instruct-10k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#DataCentricAI #LLMDrivenFrameworks #UnifiedDataPreparation #WorkflowAutomation #LargeLanguageModels

🔥 SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution

💡 The paper introduces SkillsVote, a governance framework for managing reusable skills in long-horizon large language model agents. The problem addressed is that raw trajectories of agent experiences are noisy and hard to govern, making it difficult to reuse and improve agent skills. To solve this, the authors propose treating agent skills as an experience schema that combines executable scripts with non-executable guidance on procedures.

The SkillsVote framework consists of three main processes: collection, recommendation, and evolution of agent skills. It starts by profiling a large open-source corpus of skills to identify environment requirements, quality, and verifiability. Then, it synthesizes tasks for verifiable skills and performs a search over a structured skill library to provide instructional context before execution. After execution, it decomposes trajectories into skill-linked subtasks, attributes outcomes to skill use, and admits only successful reusable discoveries to updates.

The evaluation of SkillsVote shows promising results, with offline evolution improving performance on Terminal-Bench 2.0 by up to 7.9 percentage points and online evolution improving performance on SWE-Bench Pro by up to 2.6 percentage points. The key contribution of the paper is that governed external skill libraries can improve frozen agents without requiring model updates, as long as systems control exposure, credit, and preservation of skills. Overall, the SkillsVote framework provides a structured approach to managing and improving agent skills, enabling more efficient and effective reuse of experience and knowledge.

📅 Published on May 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.18401
• PDF: https://arxiv.org/pdf/2605.18401
• Project Page: https://skills.vote

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#AgentGovernance #LargeLanguageModels #SkillEvolution #ReusableSkills #LifecycleManagement

🔥 Code as Agent Harness

💡 The paper discusses the concept of code as agent harness, where large language models are used as operational substrates for agent reasoning and execution in agentic systems. The authors argue that code is no longer just a target output, but serves as a unified infrastructure layer across multiple domains and applications. They introduce a unified view that centers code as the basis for agent infrastructure, and organize their survey around three connected layers: the harness interface, harness mechanisms, and scaling the harness.

The harness interface layer explores how code connects agents to reasoning, action, and environment modeling. The harness mechanisms layer examines planning, memory, and tool use for long-horizon execution, as well as feedback-driven control and optimization. The scaling layer discusses how to extend the harness from single-agent systems to multi-agent settings, where shared code artifacts support multi-agent coordination, review, and verification.

The authors summarize representative methods and practical applications of code as agent harness, including coding assistants, GUI/OS automation, embodied agents, scientific discovery, personalization and recommendation, DevOps, and enterprise workflows. They also outline open challenges for harness engineering, such as evaluation beyond final task success, verification under incomplete feedback, regression-free harness improvement, consistent shared state across multiple agents, human oversight for safety-critical actions, and extensions to multimodal environments.

The paper provides a unified roadmap toward executable, verifiable, and stateful AI agent systems by centering code as the harness of agentic AI. The authors demonstrate the potential of code as agent harness to enable more efficient, adaptable, and reliable agent systems, and highlight the need for further research in harness engineering to address the open challenges and limitations of this approach. Overall, the paper contributes to the development of agentic systems by providing a new perspective on the role of code in agent infrastructure and highlighting the potential benefits and challenges of this approach.

📅 Published on May 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.18747
• PDF: https://arxiv.org/pdf/2605.18747

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#AgenticSystems #LargeLanguageModels #AgentReasoning #CodeAsInfrastructure #ArtificialIntelligence

🔥 MemOS: A Memory OS for AI System

💡 The paper introduces MemOS, a memory operating system designed for Large Language Models to address the challenges of memory management. Current models lack a well-defined memory management system, relying on static parameters and short-lived contextual states, which limits their ability to track user preferences or update knowledge over time. The proposed MemOS system unifies plaintext, activation-based, and parameter-level memories, enabling efficient storage, retrieval, and continual learning.

The key contribution of MemOS is the introduction of a basic unit called a MemCube, which encapsulates both memory content and metadata such as provenance and versioning. MemCubes can be composed, migrated, and fused over time, allowing for flexible transitions between memory types and bridging retrieval with parameter-based learning.

By treating memory as a manageable system resource, MemOS establishes a memory-centric system framework that brings controllability, plasticity, and evolvability to Large Language Models. This framework enables cost-efficient storage and retrieval, laying the foundation for continual learning and personalized modeling. The proposed system has the potential to address the broader challenges of managing heterogeneous knowledge spanning different temporal scales and sources, and can substantially reduce the training and inference costs of Large Language Models.

Overall, the paper proposes a novel approach to memory management for Large Language Models, which can improve their ability to learn and adapt over time, and can pave the way for the development of more advanced Artificial General Intelligence systems. The results of the paper demonstrate the effectiveness of the proposed MemOS system in addressing the challenges of memory management in Large Language Models, and highlight its potential to enable more efficient and effective learning and adaptation in these models.

📅 Published on Jul 4, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2507.03724
• PDF: https://arxiv.org/pdf/2507.03724
• Project Page: https://memos.openmem.net/

🤖 Models citing this paper:
• https://huggingface.co/kagvi13/HMP

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MemTensor/MemOS_eval_result

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#MemoryOperatingSystem #LargeLanguageModels #MemoryManagementSystems #ContinualLearningAlgorithms #ArtificialIntelligenceArchitecture

🔥 OpenGuardrails: An Open-Source Context-Aware AI Guardrails Platform

💡 OpenGuardrails is an open source project that provides a unified model for detecting content safety and model manipulation risks in large language models. The project aims to address the critical issue of safeguarding large language models against unsafe, malicious, or privacy violating content. The OpenGuardrails platform offers a comprehensive solution that includes a context aware safety and manipulation detection model, as well as a separate named entity recognition pipeline for identifying and redacting sensitive data.

The platform protects against various types of risks, including content safety risks, model manipulation attacks such as prompt injection and jailbreaking, and data leakage. The content safety and model manipulation detection are implemented using a unified large model, while data leakage identification and redaction are performed using a separate lightweight named entity recognition pipeline.

The OpenGuardrails system can be deployed in various ways, including as a security gateway or an API based service, with enterprise grade deployment options that ensure fully private deployment. The project achieves state of the art performance on safety benchmarks, excelling in both prompt and response classification across multiple languages, including English, Chinese, and multilingual tasks.

The key contributions of the OpenGuardrails project include providing a unified model for content safety and model manipulation detection, offering a separate named entity recognition pipeline for data leakage identification and redaction, and achieving state of the art performance on safety benchmarks. The project also makes all models available under the Apache 2.0 license for public use, allowing for widespread adoption and further development of the technology. Overall, OpenGuardrails provides a comprehensive and effective solution for safeguarding large language models against various types of risks, and its open source nature makes it a valuable resource for the data science community.

📅 Published on Oct 22, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2510.19169
• PDF: https://arxiv.org/pdf/2510.19169
• Project Page: https://openguardrails.com

🤖 Models citing this paper:
• https://huggingface.co/openguardrails/OpenGuardrails-Text-2510
• https://huggingface.co/openguardrails/OpenGuardrails-Text-4B-0124

📊 Datasets citing this paper:
• https://huggingface.co/datasets/openguardrails/OpenGuardrailsMixZh_97k
• https://huggingface.co/datasets/qtqtqtqt/OpenGuardrailsMixZh_97k
• https://huggingface.co/datasets/ruishen123/OpenGuardrailsMixZh_97k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#ContextAwareAI #LargeLanguageModels #ContentSafety #ModelManipulation #NamedEntityRecognition

🔥 MiniCPM4: Ultra-Efficient LLMs on End Devices

💡 The paper introduces MiniCPM4, a highly efficient large language model designed for end-side devices. The goal is to achieve superior performance while being efficient, which is a challenge for large language models due to their computational requirements. To address this, the authors propose innovations in four key areas: model architecture, training data, training algorithms, and inference systems.

In terms of model architecture, the authors propose InfLLM v2, a trainable sparse attention mechanism that accelerates both prefilling and decoding phases for long-context processing. For training data, they propose UltraClean, an efficient and accurate pre-training data filtering and generation strategy, and UltraChat v2, a comprehensive supervised fine-tuning dataset. These datasets enable satisfactory model performance to be achieved using just 8 trillion training tokens.

The authors also propose ModelTunnel v2 for efficient pre-training strategy search and improve existing post-training methods by introducing chunk-wise rollout for load-balanced reinforcement learning and data-efficient ternary LLM, BitCPM. For inference systems, they propose CPM.cu, which integrates sparse attention, model quantization, and speculative sampling to achieve efficient prefilling and decoding.

The MiniCPM4 model is available in two versions, with 0.5B and 8B parameters, respectively. The evaluation results show that MiniCPM4 outperforms open-source models of similar size across multiple benchmarks, highlighting both its efficiency and effectiveness. Notably, MiniCPM4-8B demonstrates significant speed improvements over Qwen3-8B when processing long sequences.

The results also show that MiniCPM4 can be adapted to power diverse applications, including trustworthy survey generation and tool use with model context protocol, clearly showcasing its broad usability. Overall, the paper presents a highly efficient large language model that achieves superior performance on end-side devices, making it a significant contribution to the field of natural language processing.

📅 Published on Jun 9, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2506.07900
• PDF: https://arxiv.org/pdf/2506.07900
• Project Page: https://huggingface.co/collections/openbmb/minicpm4-6841ab29d180257e940baa9b

🤖 Models citing this paper:
• https://huggingface.co/openbmb/MiniCPM4.1-8B
• https://huggingface.co/openbmb/MiniCPM5-1B
• https://huggingface.co/openbmb/MiniCPM4-8B

📊 Datasets citing this paper:
• https://huggingface.co/datasets/openbmb/Ultra-FineWeb

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/openbmb/MiniCPM5-1B-Demo
• https://huggingface.co/spaces/openbmb/Ultra-FineWeb-L2-Selector
• https://huggingface.co/spaces/openbmb/MiniCPM4.1-8B-Demo

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#EfficientLLMs #LargeLanguageModels #SparseAttentionMechanisms #EndDeviceComputing #LowResourceNLP

🔥 ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents

💡 The paper presents ProRL Agent, a scalable infrastructure for reinforcement learning training of multi-turn large language model agents. The problem addressed is the difficulty in generating and managing large numbers of sandboxed rollout trajectories required for reinforcement learning, which is a key component for improving the long-horizon behavior of these agents. Existing infrastructures often combine rollout orchestration with the training loop, making systems hard to migrate and maintain.

To solve this problem, the authors propose a rollout-as-a-service approach, where ProRL Agent serves the full agentic rollout lifecycle through an API service. This allows for decoupling rollout orchestration from the training loop, making the system more flexible and easier to maintain. Additionally, ProRL Agent provides standardized and extensible sandbox environments that support diverse agentic tasks in high-performance computing settings.

The authors validate ProRL Agent by applying it to reinforcement learning training on various tasks, including software engineering, math, STEM, and coding. The results demonstrate the effectiveness of ProRL Agent in supporting scalable and efficient reinforcement learning training. Furthermore, ProRL Agent is open-sourced and integrated as part of NVIDIA NeMo Gym, making it accessible to the research community. Overall, the paper contributes a scalable and flexible infrastructure for reinforcement learning training of multi-turn large language model agents, which can facilitate advancements in complex, interactive tasks.

📅 Published on Mar 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2603.18815
• PDF: https://arxiv.org/pdf/2603.18815

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#ReinforcementLearning #LargeLanguageModels #MultiTurnDialogue #RolloutOptimization #RLTrainingInfrastructure

🔥 LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards

💡 The paper LongTraceRL addresses the challenge of long-context reasoning in large language models. Long-context reasoning is a central challenge for these models as they often fail to locate and integrate key information in extensive distracting content. Existing methods using reinforcement learning with verifiable rewards have shown promise but are limited by low-confusability distractors and sparse reward signals that cannot supervise intermediate reasoning steps.

To address these issues, the authors introduce LongTraceRL, a method that uses tiered distractor construction and rubric reward design to improve reasoning quality. For data construction, the authors generate multi-hop questions via knowledge graph random walks and leverage search agent trajectories to build tiered distractors. These distractors include documents the agent read but did not cite, which are high in confusability, and documents that appeared in search results but were never opened, which are low in confusability. This approach produces training contexts that are far more challenging than those built by random sampling or one-shot search.

The authors also propose a rubric reward that uses gold entities along each reasoning chain as fine-grained, entity-level process supervision. This reward is applied only to responses with correct final answers, which distinguishes the reasoning quality among correct responses and prevents reward hacking.

The experiments on three reasoning large language models across five long-context benchmarks demonstrate that LongTraceRL consistently outperforms strong baselines and encourages comprehensive, evidence-grounded reasoning. The results show that LongTraceRL is effective in improving the long-context reasoning capabilities of large language models. The codes, datasets, and models are available for further research and development. Overall, LongTraceRL provides a new approach to addressing the challenge of long-context reasoning in large language models and has the potential to improve the performance of these models in a variety of applications.

📅 Published on May 29

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.31584
• PDF: https://arxiv.org/pdf/2605.31584

🤖 Models citing this paper:
• https://huggingface.co/THU-KEG/LongTraceRL-4B
• https://huggingface.co/THU-KEG/LongTraceRL-8B
• https://huggingface.co/THU-KEG/LongTraceRL-30B

📊 Datasets citing this paper:
• https://huggingface.co/datasets/THU-KEG/LongTraceRL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#LongContextReasoning #ReinforcementLearning #LargeLanguageModels #RubricRewards #SearchAgentTrajectories

🔥 KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks

💡 The paper introduces KVarN, a new method for quantizing KV-cache in large language models to reduce error accumulation during autoregressive decoding. The problem addressed is that test-time scaling, which improves reasoning in large language models, becomes memory-bottlenecked during long-horizon decoding due to the growing KV-cache. Existing KV-cache quantization methods are not effective in this setting because they are evaluated under prefill-like settings, where errors behave differently than in autoregressive decoding. In autoregressive decoding, quantization errors accumulate across timesteps, primarily due to incorrect token scales.

The KVarN method addresses this issue by applying a Hadamard rotation followed by a dual-scaling variance normalization across both axes of the K and V matrices. This combination fixes outlying token-scale errors and substantially reduces error accumulation. The method is calibration-free, meaning it does not require any additional calibration steps.

The results show that KVarN establishes a new state-of-the-art for KV-cache quantization on generative benchmarks, including MATH500, AIME24, and HumanEval, at 2-bit precision. This means that KVarN is able to achieve better performance than existing methods while using less memory. The KVarN method is also available for implementation in large language models, providing a practical solution to the problem of error accumulation in autoregressive decoding. Overall, the paper contributes a new and effective method for quantizing KV-cache in large language models, which can improve the performance and efficiency of these models in reasoning tasks.

📅 Published on Jun 2

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.03458
• PDF: https://arxiv.org/pdf/2606.03458
• Project Page: https://github.com/huawei-csl/KVarN

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#KVCacheQuantization #AutoregressiveDecoding #LargeLanguageModels #ErrorAccumulationMitigation #QuantizationMethodsForReasoningTasks

🔥 Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolution

💡 The paper introduces the Role-Agent framework, which aims to improve the performance of Large Language Model agents by addressing the limitations of inefficient interaction feedback and static training environments. The problem with current Large Language Model agents is that their learning is hindered by the lack of effective feedback and the inability to adapt to changing environments, resulting in limited generalization.

To address this issue, the Role-Agent framework enables a single Large Language Model to function as both the agent and the environment, allowing for a bootstrapped co-evolution process. This framework consists of two components: World-In-Agent and Agent-In-World. The World-In-Agent component uses the Large Language Model as the agent to predict future states after each action, and the alignment between predicted and actual states is used as a reward to encourage environment-aware reasoning.

The Agent-In-World component analyzes failure modes from failed trajectories and retrieves tasks with similar failure patterns, thereby reshaping the training data distribution for targeted practice. This allows the Large Language Model to focus on improving its performance in areas where it is struggling.

The results of the experiments show that the Role-Agent framework consistently improves performance, with an average gain of over 4 percent over strong baselines. This demonstrates the effectiveness of the Role-Agent framework in improving the performance of Large Language Model agents by enabling them to adapt to changing environments and focus on targeted practice. Overall, the Role-Agent framework provides a novel approach to improving the performance of Large Language Model agents, and its results have significant implications for the development of more effective and adaptive language models.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.10917
• PDF: https://arxiv.org/pdf/2606.10917

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#LargeLanguageModels #AgentEnvironmentInteraction #DualRoleEvolution #BootstrappedLearning #CoEvolutionaryAlgorithms

🔥 LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference

💡 The paper presents LMCache, an efficient key-value cache layer for large language model inference at the enterprise scale. The problem addressed is the traditional storage of key-value caches in GPU memory, which limits cache reuse across different queries and inference engines. As the total key-value cache stored by users grows rapidly, exceeding the capacity of GPU memory, there is a need to move caches outside GPU devices.

The authors propose LMCache as a solution, which extracts and stores key-value caches generated by modern large language model engines out of the GPU memory and shares them across engines and queries. LMCache supports cache offloading and prefill-decode disaggregation, allowing for cross-engine and GPU cache transfer. The key contributions of LMCache include highly optimized key-value cache data movement, a modular cache connector component that decouples LMCache from the evolution of inference engines, and a control API for flexible cache orchestration across different layers.

The evaluation of LMCache shows significant improvements in throughput, with up to 15 times improvement when combined with a large language model engine. The adoption of LMCache in enterprise settings provides valuable insights, such as the benefits of fetching key-value caches from remote storage and the impact of context truncation on prefix cache hit ratio. Overall, LMCache is presented as an efficient and open-source key-value caching solution that addresses the need for efficient cache management in large language model inference.

📅 Published on Oct 8, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2510.09665
• PDF: https://arxiv.org/pdf/2510.09665
• Project Page: https://huggingface.co/collections/dvps/dvps-scientific-watch

🤖 Models citing this paper:
• https://huggingface.co/enfinity7B/apac

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#LargeLanguageModels #LLMInference #KVCacheOptimization #EnterpriseScaleAI #GPUAcceleratedInference

🔥 Foundations of Large Language Models

💡 The book Foundations of Large Language Models provides a comprehensive overview of the fundamental concepts underlying large language models. The book is structured into four main chapters, each focusing on a key area: pre-training, generative models, prompting techniques, and alignment methods. The authors aim to provide a foundational understanding of large language models, rather than a comprehensive coverage of all cutting-edge technologies. The book is intended for college students, professionals, and practitioners in natural language processing and related fields, serving as a reference for anyone interested in large language models.

The problem addressed by the book is the need for a clear understanding of the foundational concepts of large language models, which are becoming increasingly important in natural language processing. The method used to address this problem is a structured approach, dividing the topic into four key areas and exploring each in depth. The results of this approach are a book that provides a solid foundation for understanding large language models, which can be used as a reference by students, professionals, and practitioners in the field.

Overall, the book provides a foundational understanding of large language models, covering key areas such as pre-training, generative models, prompting techniques, and alignment methods, and is intended to serve as a reference for those interested in this topic. The book does not aim to cover all cutting-edge technologies, but rather provides a solid foundation for understanding the underlying concepts of large language models.

📅 Published on Jan 16, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2501.09223
• PDF: https://arxiv.org/pdf/2501.09223

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#LargeLanguageModels #NaturalLanguageProcessing #PreTrainingMethods #GenerativeModels #LanguageModelAlignment

🔥 FastContext: Training Efficient Repository Explorer for Coding Agents

💡 The paper introduces FastContext, a dedicated exploration subagent designed to improve the efficiency of repository exploration in large language model coding agents. The problem addressed is that repository exploration is a major bottleneck in coding agents, consuming a substantial token budget and polluting the agent's context with irrelevant code snippets.

The method involves separating repository exploration from code solving using specialized exploration models. FastContext is invoked on demand and issues parallel tool calls to return concise file paths and line ranges as focused context. The exploration models used in FastContext are powered by 4B-30B parameters and are bootstrapped from strong reference-model trajectories. They are then refined with task-grounded rewards for broad first-turn search, multi-turn evidence gathering, and precise citation generation.

The results show that integrating FastContext into a coding agent improves end-to-end resolution rates by up to 5.5 percent while reducing coding-agent token consumption by up to 60 percent, with minimal overhead. The paper demonstrates that repository exploration can be effectively handled by specialized models, separate from the code solving process. The code and data for FastContext are made available, allowing for further research and development in this area. Overall, the paper presents a significant contribution to the field of coding agents and software engineering, providing a more efficient and effective approach to repository exploration.

📅 Published on Jun 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.14066
• PDF: https://arxiv.org/pdf/2606.14066
• Project Page: https://huggingface.co/microsoft/FastContext-1.0-4B-SFT

🤖 Models citing this paper:
• https://huggingface.co/microsoft/FastContext-1.0-4B-SFT
• https://huggingface.co/microsoft/FastContext-1.0-4B-RL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#EfficientRepositoryExploration #CodingAgents #LargeLanguageModels #RepositoryExplorationSubagents #SpecializedExplorationModels

🔥 From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

💡 The paper discusses the challenges of continual learning in large language models and how current methods such as retrieval-augmented generation have limitations in mimicking human long-term memory. The authors propose a new framework called HippoRAG 2 which builds upon previous work and enhances it with deeper passage integration and more effective online use of a large language model. This approach improves performance across factual, sense-making, and associative memory tasks, addressing the deterioration in performance seen in previous methods that tried to augment vector embeddings with structures like knowledge graphs. The results show that HippoRAG 2 outperforms standard retrieval-augmented generation comprehensively, achieving a 7 percent improvement in associative memory tasks over the state-of-the-art embedding model, while also exhibiting superior factual knowledge and sense-making memory capabilities. The work contributes to non-parametric continual learning for large language models, paving the way for more effective and human-like memory capabilities in artificial intelligence systems.

📅 Published on Feb 20, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2502.14802
• PDF: https://arxiv.org/pdf/2502.14802

🤖 Models citing this paper:
• https://huggingface.co/muthuk1/graphrag-inference-hackathon

📊 Datasets citing this paper:
• https://huggingface.co/datasets/osunlp/HippoRAG_2
• https://huggingface.co/datasets/g7haha/HippoRAG_2

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#ContinualLearning #LargeLanguageModels #NonParametricLearning #RetrievalAugmentedGeneration #LongTermMemory

🔥 Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models

💡 The paper addresses the challenge of financial sentiment analysis, which is crucial for investment decision-making. Traditional natural language processing models are limited by their size and training data, resulting in poor generalization and effectiveness. Large Language Models, despite their superior performance in various NLP tasks, also face challenges in financial sentiment analysis due to the discrepancy between their pre-training objective and the task of predicting sentiment labels. Additionally, the concise nature of financial news often lacks sufficient context, which can compromise the reliability of Large Language Models' sentiment analysis.

To overcome these challenges, the authors propose a retrieval-augmented Large Language Model framework. This framework consists of two modules: an instruction-tuned Large Language Model module that ensures the model behaves as a predictor of sentiment labels, and a retrieval-augmentation module that retrieves additional context from reliable external sources. This approach enables the model to leverage external context to improve its sentiment analysis capabilities.

The authors evaluate their framework against traditional models and other Large Language Models, such as ChatGPT and LLaMA. The results show that their approach achieves a significant performance gain, with improvements in accuracy and F1 score ranging from 15% to 48%. This demonstrates the effectiveness of the proposed retrieval-augmented Large Language Model framework in enhancing financial sentiment analysis. Overall, the paper contributes to the development of more accurate and reliable financial sentiment analysis models, which can inform better investment decisions.

📅 Published on Oct 6, 2023

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2310.04027
• PDF: https://arxiv.org/pdf/2310.04027

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#FinancialSentimentAnalysis #RetrievalAugmentedModels #LargeLanguageModels #NaturalLanguageProcessing #FinancialTextAnalysis

🔥 Efficient Guided Generation for Large Language Models

💡 The paper presents an efficient method for guiding large language model text generation using regular expressions and context-free grammars. The problem addressed is that guided generation can be impractical due to significant overhead. The authors propose an approach that adds minimal overhead to the token sequence generation process. This method makes guided generation feasible in practice. The approach is implemented in the open source Python library Outlines, providing a practical solution for efficient guided generation. The results indicate that the method is effective, allowing for guided generation with little to no overhead, which is a significant contribution to the field of natural language processing.

📅 Published on Jul 19, 2023

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2307.09702
• PDF: https://arxiv.org/pdf/2307.09702

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#LargeLanguageModels #GuidedTextGeneration #RegularExpressions #ContextFreeGrammars #EfficientGenerationMethods

🔥 JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

💡 The paper introduces JetSpec, a speculative decoding framework designed to improve the inference speed and acceptance rates of large language models. The problem addressed is the scaling limitation of speculative decoding, which accelerates autoregressive large language models by drafting multiple tokens and verifying them in parallel. However, increasing the draft budget only improves speed when acceptance remains high and drafting overhead stays low, creating a scaling ceiling.

The proposed JetSpec framework combines efficient forward drafting with causal conditioning to break this ceiling. It trains a causal parallel draft head over fused hidden states from the frozen target model, producing candidate trees whose scores align with the target model's autoregressive factorization. This approach enables JetSpec to convert larger draft budgets into longer accepted prefixes and higher end-to-end speedup.

The method is compared to bidirectional-head and tree-based speculative decoding baselines across various benchmarks, including math, coding, and chat tasks on dense and MoE models. The results show that JetSpec consistently outperforms these baselines, achieving significant speedup on different workloads. Specifically, JetSpec achieves up to 9.64x speedup on math tasks and 4.58x on open-ended conversational workloads, with further latency gains demonstrated through integration with virtual large language models under realistic serving loads.

Overall, the paper contributes a novel speculative decoding framework that breaks the scaling ceiling of prior methods, enabling faster and more efficient large language model inference. The code and models are made available for further research and development.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.18394
• PDF: https://arxiv.org/pdf/2606.18394
• Project Page: https://jetspec-project.github.io/jetspec-web/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#SpeculativeDecoding #LargeLanguageModels #AutoregressiveModeling #ParallelTreeDrafting #CausalConditioning

🔥 Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

💡 The paper introduces Agents-A1, a 35 billion parameter Mixture-of-Experts Agentic Model that achieves performance comparable to trillion-parameter models by scaling the agent horizon instead of the parameters. The problem addressed is how to improve the performance of large language models on long-horizon tasks without increasing the number of parameters. The method used is a three-stage training approach, which includes supervised fine-tuning, domain-level teacher models, and multi-teacher distillation. The model is trained on a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories with an average length of 45,000 tokens. The results show that Agents-A1 achieves strong and broad performance on long-horizon agent benchmarks, outperforming or matching the results of 1 trillion parameter models on several tasks, including SEAL-0, IFBench, HiPhO, FrontierScience-Olympiad, and MolBench-Bind. The paper provides a practical path for scaling the horizon using a smaller model that can reach or match the performance of larger models on long-horizon tasks.

📅 Published on Jun 29

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.30616
• PDF: https://arxiv.org/pdf/2606.30616
• Project Page: https://internscience.github.io/Agents-A1/

🤖 Models citing this paper:
• https://huggingface.co/InternScience/Agents-A1
• https://huggingface.co/InternScience/Agents-A1-FP8-dynamic
• https://huggingface.co/Abiray/Agents-A1-Q4_K_M-GGUF

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#MixtureOfExperts #AgenticModels #LongHorizonTasks #LargeLanguageModels #ParameterEfficientTraining

🔥 VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

💡 The paper introduces VideoChat3, a fully open, efficient, and generalist video-centric multimodal large language model for video understanding. Current open-source models are limited in several ways, struggling to generalize across diverse video types and being computationally demanding, which restricts their efficiency and scalability. Most models are also only partially open, with key components such as training code, strategy, or datasets unavailable, hindering reproducibility and slowing community-driven development.

To address these issues, VideoChat3 advances video understanding through two complementary designs. For efficiency, it introduces the Inflated 3D Vision Transformer and Adaptive Frame Resolution for Streaming Video Perception, enabling efficient spatiotemporal representation and reducing the cost of processing video inputs during training and inference. For effectiveness, it develops a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets, covering general, long-form, and streaming video scenarios, which improves the model's generalization across domains.

By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts, using only 4B parameters and achieving higher efficiency. The paper's contributions include a fully open and efficient video understanding model, a scalable video data synthesis pipeline, and state-of-the-art results on various video understanding benchmarks.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.14935
• PDF: https://arxiv.org/pdf/2607.14935
• Project Page: https://mcg-nju.github.io/VideoChat3

🤖 Models citing this paper:
• https://huggingface.co/MCG-NJU/VideoChat3-4B
• https://huggingface.co/MCG-NJU/I3D-ViT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-LV116k
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-Academic2M
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-OL617k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#VideoUnderstanding #MultimodalLearning #LargeLanguageModels #VideoCentricAI #EfficientComputerVision