Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation (CVPR 2023)

Novel Diffusion Audio-Gesture Transformer is devised to better attend to the information from multiple modalities and model the long-term temporal dependency.

🖥 Github: https://github.com/advocate99/diffgesture

⏩ Paper: https://arxiv.org/abs/2303.09119v1

💨 Dataset: https://paperswithcode.com/dataset/beat

https://t.iss.one/DataScienceT

Deep Metric Learning for Unsupervised CD

🖥 Github: https://github.com/wgcban/metric-cd

⏩ Paper: https://arxiv.org/abs/2303.09536v1

https://t.iss.one/DataScienceT

⚜️ ViperGPT: Visual Inference via Python Execution for Reasoning

ViperGPT, a framework that leverages code-generation models to compose vision-and-language models into subroutines to produce a result for any query.

🖥 Github: https://github.com/cvlab-columbia/viper

⏩ Paper: https://arxiv.org/pdf/2303.08128.pdf

💨 Project: https://paperswithcode.com/dataset/beat

https://t.iss.one/DataScienceT

MIT Introduction to Deep Learning - 2023 Starting soon! MIT Intro to DL is one of the most concise AI courses on the web that cover basic deep learning techniques, architectures, and applications.

2023 lectures are starting in just one day, Jan 9th!

Link to register:
https://introtodeeplearning.com

MIT Introduction to Deep Learning The 2022 lectures can be found here:

https://m.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI

https://t.iss.one/DataScienceT

Train your ControlNet with diffusers 🧨

ControlNet is a neural network structure that allows fine-grained control of diffusion models by adding extra conditions.

🤗 Hugging face: https://huggingface.co/blog/train-your-controlnet#

🖥 Github: https://github.com/huggingface/blog/blob/main/train-your-controlnet.md

⏩ ControlNet training example: https://github.com/huggingface/diffusers/tree/main/examples/controlnet

https://t.iss.one/DataScienceT

🔥 Fix the Noise: Disentangling Source Feature for Controllable Domain Translation

A new approach for high-quality domain translation with better controllability.

🖥 Github: https://github.com/LeeDongYeun/FixNoise

⏩ Paper: https://arxiv.org/abs/2303.11545v1

💨 Dataset: https://paperswithcode.com/dataset/metfaces

https://t.iss.one/DataScienceT

"A panda is playing guitar on times square"

Text2Video-Zero

Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators

Paper: https://arxiv.org/abs/2303.13439
Video Result: video result link
Source code: https://github.com/picsart-ai-research/text2video-zero

https://t.iss.one/DataScienceT

Conditional Image-to-Video Generation with Latent Flow Diffusion Models

New approach for cI2V using novel latent flow diffusion models (LFDM) that synthesize an optical flow sequence in the latent space based on the given condition to warp the given image.

🖥 Github: https://github.com/nihaomiao/cvpr23_lfdm

⏩ Paper: https://arxiv.org/abs/2303.13744v1

💨 Dataset: https://drive.google.com/file/d/1dRn1wl5TUaZJiiDpIQADt1JJ0_q36MVG/view?usp=share_link

https://t.iss.one/DataScienceT

What's your gender?

Test of Time: Instilling Video-Language Models with a Sense of Time

GPT-5 will likely have video abilities, but will it have a sense of time? Here is answer to this question in #CVPR2023 paper by student of University of Amsterdam to learn how to instil time into video-language foundation models.

Paper:
https://arxiv.org/abs/2301.02074

Code:
https://github.com/bpiyush/TestOfTime

Project Page:
https://bpiyush.github.io/testoftime-website/

https://t.iss.one/DataScienceT

One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer

🖥 Github: https://github.com/IDEA-Research/OSX

⏩ Paper: https://arxiv.org/abs/2303.16160

⭐️ Project: https://osx-ubody.github.io

💨 Dataset: https://paperswithcode.com/dataset/expose

https://t.iss.one/DataScienceT

ViperGPT: Visual Inference via Python Execution for Reasoning

ViperGPT, a framework that leverages code-generation models to compose vision-and-language models into subroutines to produce a result for any query.


Github:
https://github.com/cvlab-columbia/viper

Paper:
https://arxiv.org/pdf/2303.08128.pdf

Project:
https://paperswithcode.com/dataset/beat

https://t.iss.one/DataScienceT

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research

Propose a three-stage processing pipeline for filtering noisy data and generating high-quality captions, where ChatGPT.

🖥 Github: https://github.com/xinhaomei/wavcaps

⏩ Paper: https://arxiv.org/abs/2303.17395v1

💨 Dataset: https://paperswithcode.com/dataset/sounddescs

https://t.iss.one/DataScienceT

DPF: Learning Dense Prediction Fields with Weak Supervision

🖥 Github: https://github.com/cxx226/dpf

⏩ Paper: https://arxiv.org/abs/2303.16890v1

💨 Dataset: https://paperswithcode.com/dataset/pascal-context

https://t.iss.one/DataScienceT

Human Guided Ground-truth Generation for Realistic Image Super-resolution

🖥 Github: https://github.com/chrisdud0257/hggt

⏩ Paper: https://arxiv.org/abs/2303.13069

💨 Dataset: https://paperswithcode.com/dataset/div2k

https://t.iss.one/DataScienceT

ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing

🖥 Github: https://github.com/alibaba/easyrobust/tree/main/benchmarks/imagenet-e

⏩ Paper: https://arxiv.org/abs/2303.17096v1

💨 Dataset: https://paperswithcode.com/dataset/objectnet

https://t.iss.one/DataScienceT

⚡️Token Merging for Stable Diffusion

Token Merging (ToMe) speeds up transformers by merging redundant tokens, which means the transformer has to do less work.

pip install tomesd

🖥 Github: https://github.com/dbolya/tomesd

⏩ Paper: https://arxiv.org/abs/2303.17604v1

💨 Blog: https://research.facebook.com/blog/2023/2/token-merging-your-vit-but-faster/

https://t.iss.one/DataScienceT

⭐️ HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace

Language serves as an interface for LLMs to connect numerous AI models for solving complicated AI tasks!

🖥 Github: https://github.com/microsoft/JARVIS

⏩ Paper: https://arxiv.org/abs/2303.17604v1

https://t.iss.one/DataScienceT