🔥 ShutterMuse: Capture-Time Photography Guidance with MLLMs

💡 The paper introduces a new benchmark and dataset for photography assistance, focusing on capture-time guidance for both camera framing and subject pose. Existing models primarily evaluate post-hoc crop prediction and overlook subject-side recommendations, leaving a gap in the capabilities of multimodal large language models. To address this, the researchers developed CaptureGuide-Bench, a benchmark with two tasks: photographer-side composition decision and refinement, and subject-side scene-conditioned pose recommendation. They also constructed CaptureGuide-Dataset, comprising 130K samples with textual rationales and visual annotations.

The researchers then developed ShutterMuse, a unified multimodal large language model trained with supervised and reinforcement fine-tuning. ShutterMuse provides both composition guidance and pose recommendations during image capture. The evaluation reveals that general-purpose models can make composition decisions but lack precise refinement localization, while specialized aesthetic cropping models localize crops effectively but are limited to refinement and do not provide pose guidance.

The experiments on CaptureGuide-Bench show that ShutterMuse achieves the best overall photographer-side performance among evaluated baselines and competitive subject-side pose recommendation with lower inference cost. This demonstrates the potential of multimodal large language models as interactive assistants for photography during image capture, addressing the need for capture-time guidance in real-world photography. The paper contributes to the development of models that can provide effective guidance for both camera framing and subject pose, making it a significant step forward in the field of photography assistance.

📅 Published on Jun 24

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.25763
• PDF: https://arxiv.org/pdf/2606.25763
• Project Page: https://lijayutnt.github.io/ShutterMuse/

🤖 Models citing this paper:
• https://huggingface.co/ShutterMuse/ShutterMuse

📊 Datasets citing this paper:
• https://huggingface.co/datasets/ShutterMuse/CaptureGuide-Bench

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/ShutterMuse/ShutterMuse-Video-Demo

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#MultimodalLargeLanguageModels #CaptureTimeGuidance #PhotographyAssistance #CameraFramingTechniques #MultimodalLearningModels

🔥 MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revision

💡 The paper presents MemSlides, a hierarchical memory framework for personalized presentation agents that enables stable personalization and reliable local edits across multi-turn revisions. The problem addressed is that current presentation generation methods are unable to preserve stable user preferences across tasks and retain newly introduced preferences and constraints during multi-turn revision. To solve this, MemSlides separates long-term user profiles, working memory for session constraints, and tool memory for reusable execution experiences.

The method involves a hierarchical memory design that stores intent-conditioned profiles for round-0 personalization in user profile memory, carries active preferences and session constraints across revision rounds in working memory, and stores reusable execution experience for reliable localized editing in tool memory. This memory design is paired with scoped slide-local revision, which allows targeted updates to act on the smallest affected region instead of regenerating the full deck.

The results show that user profile memory improves persona-alignment judgments, tool-memory injection improves closed-loop modify behavior, and working memory is able to carry over preferences. The experiments were conducted in controlled settings, including a multi-persona, multi-intent profile bank and diagnostic matched-pair settings. The results suggest that effective personalization in presentation authoring depends on separating persistent user profiles, session-level working memory, and reusable execution experience across generation and localized revision. Overall, MemSlides provides a framework for personalized presentation agents that can generate and revise presentations in a way that is tailored to individual users and their preferences.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.17162
• PDF: https://arxiv.org/pdf/2606.17162
• Project Page: https://memslides.github.io/

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/huohua325/MemSlides

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#PersonalizedPresentationAgents #HierarchicalMemoryFramework #MultiTurnRevision #LocalEditPreservation #IntelligentSlideGeneration

🔥 UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating

💡 The paper presents UnityShots, a memory-driven audio-video generation system that can generate coherent multi-shot videos. The problem addressed is that existing approaches to generating multi-shot videos either cannot scale or do not maintain consistent subject appearance and audio across video cuts. To solve this, UnityShots uses a combination of fixed-size long-term and short-term memory slots, boundary-conditioned gates, and discrete cut-type priors to maintain consistency across shots. The system consists of two streams, video and audio, where the video stream uses two fixed-size slots to store information about the opening shot and the immediately preceding shot, and the audio stream uses a reference speaker token to preserve vocal timbre. The boundary-conditioned gate updates the memory slots at every cut, and the discrete cut-type prior allows for control over transition strength between shots. The system was trained on annotated cinematic and music-video shots and evaluated on a benchmark of 200 multi-cultural multi-shot sequences. The results show that UnityShots outperforms open-source baselines on every cross-shot coherence metric and matches the performance of the strongest closed-source system. The paper also releases a benchmark of multi-shot sequences with per-shot reference identities, reference audio, and per-boundary transition labels, which can be used for future research. Overall, UnityShots provides a new approach to generating coherent multi-shot videos that can maintain consistent subject appearance and audio across video cuts.

📅 Published on Jun 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.21661
• PDF: https://arxiv.org/pdf/2606.21661
• Project Page: https://jackailab.github.io/Projects/UnityShots/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/KlingTeam/UnityShotsBench

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#AudioVideoGeneration #MultiShotVideo #BoundaryAwareGating #MemoryDrivenGeneration #CoherentVideoGeneration

There are hundreds of AI channels on YouTube. Here's why we made another one.

Most AI content does one of two things: it stays so surface-level it teaches you nothing, or it goes so deep you need a PhD to follow along.

We built Guidely for everyone in between.

→ We start with absolute beginners in mind
→ Then take you deeper, until the details actually click
→ Every guide is reviewed by experienced AI engineers
→ We don't make more content. We make better content.
Whether you build, design, or market products, our goal is simple: leave you thinking "I've never seen it broken down this well."

Two good places to start 👇

→ AI vs ML vs Deep Learning vs GenAI ... But Done Right!
The terms everyone uses. The distinctions are almost never explained clearly. We fix that: youtu.be/72yyLA2wRWc

→ How to Break into AI Engineering in 2026
A senior applied scientist shares what actually matters:  youtu.be/42vE7Ij4kdU

If AI has ever felt overwhelming or noisy, this channel is for you. If the content resonates with you, please don’t forget to like and subscribe.

🔥 ViQ: Text-Aligned Visual Quantized Representations at Any Resolution

💡 The paper introduces ViQ, a visual quantization framework that aims to balance semantic richness and detail preservation in discrete representations of images. The goal is to create a unified representation for text and vision that enables simpler multimodal modeling and more efficient training. Existing methods struggle to balance low-level details and high-level semantics in discrete representations, often resulting in severe information loss.

The ViQ framework addresses this issue by structuring quantization learning into two stages: text-aligned pre-training and feature discretization. In the first stage, the visual encoder is pre-trained with semantic-rich supervision from a pre-trained language model, allowing it to process native-resolution visual inputs. In the second stage, a proximal representation learning strategy is used to progressively compact the feature space, along with a position-aware head-wise quantization mechanism that enables flexible processing of arbitrary resolutions.

The results show that ViQ achieves competitive performance compared to state-of-the-art multimodal vision encoders with continuous and high-dimensional visual features, while maintaining high precision in low-level reconstruction. Additionally, multimodal training with visual quantized representations using ViQ leads to significant efficiency improvements, with up to 20-70 percent acceleration in training time compared to different base language models and training recipes. Overall, the paper presents a novel approach to visual quantization that balances semantics and details in discrete representations, enabling more efficient and effective multimodal modeling.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.27313
• PDF: https://arxiv.org/pdf/2606.27313

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#VisualQuantization #MultimodalModeling #TextVisionAlignment #DiscreteRepresentations #ImageQuantization

🔥 JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

💡 The paper introduces JetSpec, a speculative decoding framework designed to improve the inference speed and acceptance rates of large language models. The problem addressed is the scaling limitation of speculative decoding, which accelerates autoregressive large language models by drafting multiple tokens and verifying them in parallel. However, increasing the draft budget only improves speed when acceptance remains high and drafting overhead stays low, creating a scaling ceiling.

The proposed JetSpec framework combines efficient forward drafting with causal conditioning to break this ceiling. It trains a causal parallel draft head over fused hidden states from the frozen target model, producing candidate trees whose scores align with the target model's autoregressive factorization. This approach enables JetSpec to convert larger draft budgets into longer accepted prefixes and higher end-to-end speedup.

The method is compared to bidirectional-head and tree-based speculative decoding baselines across various benchmarks, including math, coding, and chat tasks on dense and MoE models. The results show that JetSpec consistently outperforms these baselines, achieving significant speedup on different workloads. Specifically, JetSpec achieves up to 9.64x speedup on math tasks and 4.58x on open-ended conversational workloads, with further latency gains demonstrated through integration with virtual large language models under realistic serving loads.

Overall, the paper contributes a novel speculative decoding framework that breaks the scaling ceiling of prior methods, enabling faster and more efficient large language model inference. The code and models are made available for further research and development.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.18394
• PDF: https://arxiv.org/pdf/2606.18394
• Project Page: https://jetspec-project.github.io/jetspec-web/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#SpeculativeDecoding #LargeLanguageModels #AutoregressiveModeling #ParallelTreeDrafting #CausalConditioning

🔥 OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning

💡 The paper proposes a framework called On-Policy Skill Distillation, or OPID, which aims to improve the efficiency and performance of language agent training in reinforcement learning. The problem addressed is that outcome-based reinforcement learning provides sparse rewards that do not offer sufficient guidance on intermediate decisions, while existing self-distillation methods often rely on external skill memories that can be costly to maintain and may not match the current policy.

OPID addresses this issue by extracting skill supervision directly from completed on-policy trajectories, representing trajectory hindsight as hierarchical skills that capture both global and local decision knowledge. The framework uses a critical-first routing mechanism to select the most relevant skill and inject it into the interaction history, allowing the old policy to re-score responses under both original and skill-augmented contexts. This yields a token-level self-distillation advantage that is combined with the outcome advantage for policy optimization.

The results of the experiments demonstrate that OPID generally improves agent performance, sample efficiency, and robustness over outcome-only reinforcement learning and existing skill-distillation baselines. The framework preserves reinforcement learning as the primary training objective while introducing dense, distribution-matched hindsight supervision. The experiments were conducted on several datasets, including ALFWorld, WebShop, and Search-based QA, and the code is available for further research. Overall, OPID offers a novel approach to skill distillation that can enhance the training of language agents in reinforcement learning.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.26790
• PDF: https://arxiv.org/pdf/2606.26790

🤖 Models citing this paper:
• https://huggingface.co/Jinyang23/OPID-ALFWorld-1.7B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#AgenticReinforcementLearning #OnPolicyLearning #SkillDistillation #ReinforcementLearningFrameworks #HierarchicalReinforcementLearning

🔥 Terrain Diffusion: A Diffusion-Based Successor to Perlin Noise in Infinite, Real-Time Terrain Generation

💡 The paper introduces Terrain Diffusion, a new method for generating realistic and infinite procedural worlds in real-time. The current method, Perlin noise, is fast and infinite but lacks realism and large-scale coherence. Terrain Diffusion uses diffusion models and a novel algorithm called InfiniteDiffusion to address these limitations. The InfiniteDiffusion algorithm enables seamless and real-time synthesis of boundless landscapes by coupling planetary context with local detail through a hierarchical stack of diffusion models. The method also uses a compact Laplacian encoding to stabilize outputs across large dynamic ranges and an open-source infinite-tensor framework to support constant-memory manipulation of unbounded tensors. Additionally, few-step consistency distillation enables efficient generation. The results show that Terrain Diffusion can synthesize entire planets coherently, controllably, and without limits, making it a practical foundation for procedural world generation. The method provides constant-time random access, seamless infinite extent, and seed-consistency, making it a suitable successor to Perlin noise. Overall, the paper presents a significant contribution to the field of procedural world generation, enabling the creation of realistic and infinite worlds in real-time.

📅 Published on Dec 9, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2512.08309
• PDF: https://arxiv.org/pdf/2512.08309
• Project Page: https://xandergos.github.io/terrain-diffusion/

🤖 Models citing this paper:
• https://huggingface.co/xandergos/terrain-diffusion-30m
• https://huggingface.co/xandergos/terrain-diffusion-90m
• https://huggingface.co/xandergos/TerrainDiffusion-Consistency-Base-192x3

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#ProceduralTerrainGeneration #InfiniteWorlds #DiffusionModels #RealTimeTerrainSynthesis #PerlinNoiseAlternatives

🔥 Hallucination in World Models is Predictable and Preventable

💡 The paper addresses the issue of hallucination in world models, which occurs when the model generates unrealistic futures despite appearing visually fluent. The authors hypothesize that hallucination happens in low-data regions of the state-action space and can be detected and mitigated using data-centric signals and coverage-aware sampling techniques.

To test this hypothesis, the authors created a large dataset called MMBench2, consisting of 427 hours of data and 210 tasks for visual world modeling, with ground-truth actions and rewards. They trained a 350M-parameter world model on this dataset and identified three distinct modes of hallucination: perceptual, action-marginalized, and scene-diverging.

The authors developed three signals that can accurately predict where the model will fail and used these signals to develop a coverage-aware sampling technique to close coverage gaps during training. They also used the hallucination predictors as curiosity rewards for targeted data collection to adapt the pretrained world model to new environments with as few as 50 real environment trajectories.

The results show that hallucination in world models is indeed a data coverage issue and that the same signals used to detect it can also be used for mitigation. The authors provide a data-efficient finetuning recipe that can adapt the pretrained world model to entirely unseen environments, demonstrating the effectiveness of their approach. Overall, the paper contributes to a better understanding of hallucination in world models and provides a practical solution to prevent and mitigate it.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.27326
• PDF: https://arxiv.org/pdf/2606.27326
• Project Page: https://www.nicklashansen.com/mmbench2

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#HallucinationInAI #WorldModelingTechniques #PredictiveModelingForRobotics #DataCentricSignalProcessing #VisualWorldModeling

🔥 Geometric Context Transformer for Streaming 3D Reconstruction

💡 The paper presents a new approach to streaming 3D reconstruction, which involves recovering 3D information such as camera poses and point clouds from a video stream. This task requires geometric accuracy, temporal consistency, and computational efficiency. To address this problem, the authors introduce LingBot-Map, a feed-forward 3D foundation model that uses a geometric context transformer architecture. The key component of this architecture is a specialized attention mechanism that integrates three main elements: an anchor context, a pose-reference window, and a trajectory memory. These elements work together to address coordinate grounding, dense geometric cues, and long-range drift correction, allowing the model to maintain a compact streaming state while retaining rich geometric context. The result is a model that can perform stable and efficient inference at around 20 frames per second on high-resolution inputs, even over long sequences exceeding 10,000 frames. The authors evaluate their approach on various benchmarks and demonstrate that it outperforms both existing streaming and iterative optimization-based methods, achieving superior performance in terms of geometric accuracy and temporal consistency. Overall, the paper contributes a novel and effective approach to streaming 3D reconstruction, with potential applications in areas such as robotics, computer vision, and virtual reality.

📅 Published on Apr 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2604.14141
• PDF: https://arxiv.org/pdf/2604.14141
• Project Page: https://technology.robbyant.com/lingbot-map

🤖 Models citing this paper:
• https://huggingface.co/robbyant/lingbot-map
• https://huggingface.co/agramoi/lingbot-map
• https://huggingface.co/maujim/lingbot-map-long-only

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/limonsyrah/lingbot-3d
• https://huggingface.co/spaces/mohan007/lingbot-3d
• https://huggingface.co/spaces/Fifthoply/lingbot-3d-ZERO

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://t.iss.one/PaperNexus

#GeometricDeepLearning #3DReconstructionAlgorithms #StreamingComputerVision #TransformerArchitectures #PointCloudProcessing