ViperGPT: Visual Inference via Python Execution for Reasoning

ViperGPT, a framework that leverages code-generation models to compose vision-and-language models into subroutines to produce a result for any query.


Github:
https://github.com/cvlab-columbia/viper

Paper:
https://arxiv.org/pdf/2303.08128.pdf

Project:
https://paperswithcode.com/dataset/beat

https://t.iss.one/DataScienceT

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research

Propose a three-stage processing pipeline for filtering noisy data and generating high-quality captions, where ChatGPT.

🖥 Github: https://github.com/xinhaomei/wavcaps

⏩ Paper: https://arxiv.org/abs/2303.17395v1

💨 Dataset: https://paperswithcode.com/dataset/sounddescs

https://t.iss.one/DataScienceT

DPF: Learning Dense Prediction Fields with Weak Supervision

🖥 Github: https://github.com/cxx226/dpf

⏩ Paper: https://arxiv.org/abs/2303.16890v1

💨 Dataset: https://paperswithcode.com/dataset/pascal-context

https://t.iss.one/DataScienceT

Human Guided Ground-truth Generation for Realistic Image Super-resolution

🖥 Github: https://github.com/chrisdud0257/hggt

⏩ Paper: https://arxiv.org/abs/2303.13069

💨 Dataset: https://paperswithcode.com/dataset/div2k

https://t.iss.one/DataScienceT

ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing

🖥 Github: https://github.com/alibaba/easyrobust/tree/main/benchmarks/imagenet-e

⏩ Paper: https://arxiv.org/abs/2303.17096v1

💨 Dataset: https://paperswithcode.com/dataset/objectnet

https://t.iss.one/DataScienceT

⚡️Token Merging for Stable Diffusion

Token Merging (ToMe) speeds up transformers by merging redundant tokens, which means the transformer has to do less work.

pip install tomesd

🖥 Github: https://github.com/dbolya/tomesd

⏩ Paper: https://arxiv.org/abs/2303.17604v1

💨 Blog: https://research.facebook.com/blog/2023/2/token-merging-your-vit-but-faster/

https://t.iss.one/DataScienceT

⭐️ HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace

Language serves as an interface for LLMs to connect numerous AI models for solving complicated AI tasks!

🖥 Github: https://github.com/microsoft/JARVIS

⏩ Paper: https://arxiv.org/abs/2303.17604v1

https://t.iss.one/DataScienceT

Test of Time: Instilling Video-Language Models with a Sense of Time

GPT-5 will likely have video abilities, but will it have a sense of time? Here is answer to this question in #CVPR2023 paper by student of University of Amsterdam to learn how to instil time into video-language foundation models.

Paper:
https://arxiv.org/abs/2301.02074

Code:
https://github.com/bpiyush/TestOfTime

Project Page:
https://bpiyush.github.io/testoftime-website/

https://t.iss.one/DataScienceT

Segment Anything

The Segment Anything Model (SAM) produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image.

🖥 Github: https://github.com/facebookresearch/segment-anything

⭐️ Project: https://segment-anything.com/

⏩ Paper: https://arxiv.org/abs/2304.02643v1

💨 Dataset: https://segment-anything.com/dataset/index.html

https://t.iss.one/DataScienceT

Painter → SegGPT: Vision Foundation Models from BAAI

SegGPT, a generalist model for segmenting everything in context.

🖥 Github: https://github.com/baaivision/painter

⏩ Paper: https://arxiv.org/abs/2304.03284v1

⏩ Demo: https://huggingface.co/spaces/BAAI/SegGPT

💨 Dataset: https://paperswithcode.com/dataset/youtube-vos

https://t.iss.one/DataScienceT

To watch paid channel content
All you have to do is subscribe to the paid channel. The paid channel includes multiple and huge programming courses, in addition to very useful books that are not available for free except in the paid channel.

To request a subscription: talk to @Hussein_Sheikho

Channel link: https://t.iss.one/+LnCmAFJO3tNmYjUy

⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️

We launched a special bot some time ago to download all scientific, software and mathematics books The bot contains more than thirty million books, and new books are downloaded first, In addition to the possibility of downloading all articles and scientific papers for free

To request a subscription: talk to @Hussein_Sheikho

Instruction Tuning with GPT-4

First attempt to use GPT-4 to generate instruction-following data for LLM finetuning.

🖥 Github: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM

⏩ Paper: https://arxiv.org/abs/2304.03277v1

⏩ Project: https://instruction-tuning-with-gpt-4.github.io/

https://t.iss.one/DataScienceT

⚜️ OpenAGI: When LLM Meets Domain Experts

Reinforcement Learning from Task Feedback (RLTF) mechanism, which uses the task-solving result as feedback to improve the LLM's task-solving ability

git clone https://github.com/agiresearch/OpenAGI.git

🖥 Github: https://github.com/agiresearch/openagi

⏩ Paper: https://arxiv.org/pdf/2304.04370.pdf

⭐️ Dataset: https://drive.google.com/drive/folders/1AjT6y7qLIMxcmHhUBG5IE1_5SnCPR57e?usp=share_link

https://t.iss.one/DataScienceT

⭐️ Hard Patches Mining for Masked Image Modeling

We observe that the reconstruction loss can naturally be the metric of the difficulty of the pre-training task.

🖥 Github: https://github.com/haochen-wang409/hpm

⏩ Paper: https://arxiv.org/abs/2304.05919v1

⭐️ Dataset: https://paperswithcode.com/dataset/ade20k

https://t.iss.one/DataScienceT

👀SEEM: Segment Everything Everywhere All at Once

Universal, interactive multi-modal interface for any types of segmentation with ONE SINGLE MODE.

🖥 Github: https://github.com/ux-decoder/segment-everything-everywhere-all-at-once

⏩ Paper: https://arxiv.org/abs/2304.06718v1

⭐️ Dataset: https://paperswithcode.com/dataset/refcoco

https://t.iss.one/DataScienceT

🎨 Animated Drawings

A Method for Automatically Animating Children's Drawings of the Human Figure

🖥 Github: https://github.com/facebookresearch/AnimatedDrawings

⭐️Project: https://fairanimateddrawings.com/site/home

⏩ Paper: arxiv.org/pdf/2303.12741.pdf

https://t.iss.one/DataScienceT

​​InceptionNeXt: When Inception Meets ConvNeXt

Large-kernel convolutions, such as those employed in ConvNeXt, can improve model performance but often come at the cost of efficiency due to high memory access costs. Although reducing kernel size may increase speed, it often leads to significant performance degradation.

To address this issue, the authors propose InceptionNeXt, which decomposes large-kernel depthwise convolution into four parallel branches along the channel dimension. This new Inception depthwise convolution results in networks with high throughputs and competitive performance. For example, InceptionNeXt-T achieves 1.6x higher training throughputs than ConvNeX-T and a 0.2% top-1 accuracy improvement on ImageNet-1K. InceptionNeXt has the potential to serve as an economical baseline for future architecture design, helping to reduce carbon footprint.

A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-inceptionnext

Paper link:https://arxiv.org/abs/2303.16900

Code link: https://github.com/sail-sg/inceptionnext

https://t.iss.one/DataScienceT

💻 Graph classification with Transformers

This notebook shows how to fine-tune the Graphormer model for Graph Classification on a dataset available on the hub.

🤗Hugging face blog: https://huggingface.co/blog/graphml-classification

⏩ Intro to Graphs: https://t.iss.one/ai_machinelearning_big_data/3214

🖥 Github: https://github.com/huggingface/blog/blob/main/notebooks/graphml-classification.ipynb

⏩ Paper: https://arxiv.org/abs/2106.05234

⭐️Dataset: https://ogb.stanford.edu/

https://t.iss.one/DataScienceT