TAP-Vid: A Benchmark for Tracking Any Point in a Video

📝Generic motion understanding from video involves not only tracking objects, but also perceiving how their surfaces deform and move.
https://github.com/deepmind/tapnet

OneFlow: Redesign the Distributed Deep Learning Framework from Scratch

📝Aiming at a simple, neat redesign of distributed deep learning frameworks for various parallelism paradigms, we present OneFlow, a novel distributed training framework based on an SBP (split, broadcast and partial-value) abstraction and the actor model.
https://github.com/Oneflow-Inc/oneflow

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

📝The tremendous success of CLIP (Radford et al., 2021) has promoted the research and application of contrastive learning for vision-language pretraining.
https://github.com/ofa-sys/chinese-clip

PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping

📝Previous generative adversarial network (GAN)-based neural vocoders are trained to reconstruct the exact ground truth waveform from the paired mel-spectrogram and do not consider the one-to-many relationship of speech synthesis.
https://github.com/mindslab-ai/phaseaug

Example-Based Named Entity Recognition

📝We present a novel approach to named entity recognition (NER) in the presence of scarce data that we call example-based NER.
https://github.com/sayef/fsner

Fine-Tuning Language Models from Human Preferences

📝Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks.
https://github.com/lvwerra/trl

DI-engine

📝OpenDILab Decision AI Engine
https://github.com/opendilab/DI-engine

DiffusionInst: Diffusion Model for Instance Segmentation

📝This paper proposes DiffusionInst, a novel framework that represents instances as instance-aware filters and formulates instance segmentation as a noise-to-filter denoising process.
https://github.com/chenhaoxing/DiffusionInst

DAMO-YOLO : A Report on Real-Time Object Detection Design

📝In this report, we present a fast and accurate object detection method dubbed DAMO-YOLO, which achieves higher performance than the state-of-the-art YOLO series.
https://github.com/tinyvision/damo-yolo

Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation

📝The introductory programming sequence has been the focus of much research in computing education.
https://github.com/deepmind/code_contests

Images Speak in Images: A Generalist Painter for In-Context Visual Learning

📝In this work, we present Painter, a generalist model which addresses these obstacles with an "image"-centric solution, that is, to redefine the output of core vision tasks as images, and specify task prompts as also images.
https://github.com/baaivision/painter

ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency

📝In the learning phase, each agent minimizes the TD error that is dependent on how the subsequent agents have reacted to their chosen action.
https://github.com/opendilab/ace

Learning Video Representations from Large Language Models

📝We introduce LaViLa, a new approach to learning video-language representations by leveraging Large Language Models (LLMs).
https://github.com/facebookresearch/lavila

EVA: Exploring the Limits of Masked Visual Representation Learning at Scale

📝We launch EVA, a vision-centric foundation model to explore the limits of visual representation at scale using only publicly accessible data.
https://github.com/baaivision/eva

ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders

📝This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition benchmarks, including ImageNet classification, COCO detection, and ADE20K segmentation.

https://github.com/facebookresearch/convnext-v2

Cramming: Training a Language Model on a Single GPU in One Day

📝Recent trends in language modeling have focused on increasing performance through scaling, and have resulted in an environment where training language models is out of reach for most researchers and practitioners.

https://github.com/jonasgeiping/cramming

Muse: Text-To-Image Generation via Masked Generative Transformers

📝Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.

https://github.com/lucidrains/muse-pytorch

A Survey for In-context Learning

📝With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples.

https://github.com/dqxiu/icl_paperlist

Reasoning over Different Types of Knowledge Graphs: Static, Temporal and Multi-Modal

📝The early works in this domain mainly focus on static KGR and tend to directly apply general knowledge graph embedding models to the reasoning task.

https://github.com/liangke23/awesome-knowledge-graph-reasoning

BanditPAM: Almost Linear Time k-Medoids Clustering via Multi-Armed Bandits

📝In these experiments, we observe that BanditPAM returns the same results as state-of-the-art PAM-like algorithms up to 4x faster while performing up to 200x fewer distance computations.https://github.com/ThrunGroup/BanditPAM