Learning Video Representations from Large Language Models

📝We introduce LaViLa, a new approach to learning video-language representations by leveraging Large Language Models (LLMs).
https://github.com/facebookresearch/lavila

EVA: Exploring the Limits of Masked Visual Representation Learning at Scale

📝We launch EVA, a vision-centric foundation model to explore the limits of visual representation at scale using only publicly accessible data.
https://github.com/baaivision/eva

ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders

📝This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition benchmarks, including ImageNet classification, COCO detection, and ADE20K segmentation.

https://github.com/facebookresearch/convnext-v2

Cramming: Training a Language Model on a Single GPU in One Day

📝Recent trends in language modeling have focused on increasing performance through scaling, and have resulted in an environment where training language models is out of reach for most researchers and practitioners.

https://github.com/jonasgeiping/cramming

Muse: Text-To-Image Generation via Masked Generative Transformers

📝Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.

https://github.com/lucidrains/muse-pytorch

A Survey for In-context Learning

📝With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples.

https://github.com/dqxiu/icl_paperlist

Reasoning over Different Types of Knowledge Graphs: Static, Temporal and Multi-Modal

📝The early works in this domain mainly focus on static KGR and tend to directly apply general knowledge graph embedding models to the reasoning task.

https://github.com/liangke23/awesome-knowledge-graph-reasoning

BanditPAM: Almost Linear Time k-Medoids Clustering via Multi-Armed Bandits

📝In these experiments, we observe that BanditPAM returns the same results as state-of-the-art PAM-like algorithms up to 4x faster while performing up to 200x fewer distance computations.https://github.com/ThrunGroup/BanditPAM

SegGPT: Segmenting Everything In Context

📝We unify various segmentation tasks into a generalist in-context learning framework that accommodates different kinds of segmentation data by transforming them into the same format of images.https://github.com/baaivision/painter

Instruction Tuning with GPT-4

📝Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed.https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace

📝Solving complicated AI tasks with different domains and modalities is a key step toward advanced artificial intelligence.https://github.com/microsoft/JARVIS

Segment Anything

📝We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation.https://github.com/facebookresearch/segment-anything

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

📝In this paper, we address this challenge, and propose GPTQ, a new one-shot weight quantization method based on approximate second-order information, that is both highly-accurate and highly-efficient.https://github.com/thudm/chatglm-6b

SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation📝We present SadTalker, which generates 3D motion coefficients (head pose, expression) of the 3DMM from audio and implicitly modulates a novel 3D-aware face render for talking head generation.https://github.com/winfredy/sadtalker

Segment Everything Everywhere All at Once

📝https://github.com/ux-decoder/segment-everything-everywhere-all-at-once

ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

📝https://github.com/thudm/imagereward

Self-Instruct: Aligning Language Model with Self Generated Instructions

📝https://github.com/databrickslabs/dolly

CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society

📝https://github.com/lightaime/camel

AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

📝https://github.com/microsoft/agieval

A Method for Animating Children's Drawings of the Human Figure

📝https://github.com/facebookresearch/AnimatedDrawings

OpenAssistant Conversations -- Democratizing Large Language Model Alignment

📝https://github.com/laion-ai/open-assistant

HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge

📝https://github.com/scir-hi/huatuo-llama-med-chinese