Test of Time: Instilling Video-Language Models with a Sense of Time

GPT-5 will likely have video abilities, but will it have a sense of time? Here is answer to this question in #CVPR2023 paper by student of University of Amsterdam to learn how to instil time into video-language foundation models.

Paper:
https://arxiv.org/abs/2301.02074

Code:
https://github.com/bpiyush/TestOfTime

Project Page:
https://bpiyush.github.io/testoftime-website/

https://t.iss.one/DataScienceT

Segment Anything

The Segment Anything Model (SAM) produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image.

🖥 Github: https://github.com/facebookresearch/segment-anything

⭐️ Project: https://segment-anything.com/

⏩ Paper: https://arxiv.org/abs/2304.02643v1

💨 Dataset: https://segment-anything.com/dataset/index.html

https://t.iss.one/DataScienceT

Painter → SegGPT: Vision Foundation Models from BAAI

SegGPT, a generalist model for segmenting everything in context.

🖥 Github: https://github.com/baaivision/painter

⏩ Paper: https://arxiv.org/abs/2304.03284v1

⏩ Demo: https://huggingface.co/spaces/BAAI/SegGPT

💨 Dataset: https://paperswithcode.com/dataset/youtube-vos

https://t.iss.one/DataScienceT

To watch paid channel content
All you have to do is subscribe to the paid channel. The paid channel includes multiple and huge programming courses, in addition to very useful books that are not available for free except in the paid channel.

To request a subscription: talk to @Hussein_Sheikho

Channel link: https://t.iss.one/+LnCmAFJO3tNmYjUy

⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️

We launched a special bot some time ago to download all scientific, software and mathematics books The bot contains more than thirty million books, and new books are downloaded first, In addition to the possibility of downloading all articles and scientific papers for free

To request a subscription: talk to @Hussein_Sheikho

Instruction Tuning with GPT-4

First attempt to use GPT-4 to generate instruction-following data for LLM finetuning.

🖥 Github: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM

⏩ Paper: https://arxiv.org/abs/2304.03277v1

⏩ Project: https://instruction-tuning-with-gpt-4.github.io/

https://t.iss.one/DataScienceT

⚜️ OpenAGI: When LLM Meets Domain Experts

Reinforcement Learning from Task Feedback (RLTF) mechanism, which uses the task-solving result as feedback to improve the LLM's task-solving ability

git clone https://github.com/agiresearch/OpenAGI.git

🖥 Github: https://github.com/agiresearch/openagi

⏩ Paper: https://arxiv.org/pdf/2304.04370.pdf

⭐️ Dataset: https://drive.google.com/drive/folders/1AjT6y7qLIMxcmHhUBG5IE1_5SnCPR57e?usp=share_link

https://t.iss.one/DataScienceT

⭐️ Hard Patches Mining for Masked Image Modeling

We observe that the reconstruction loss can naturally be the metric of the difficulty of the pre-training task.

🖥 Github: https://github.com/haochen-wang409/hpm

⏩ Paper: https://arxiv.org/abs/2304.05919v1

⭐️ Dataset: https://paperswithcode.com/dataset/ade20k

https://t.iss.one/DataScienceT

👀SEEM: Segment Everything Everywhere All at Once

Universal, interactive multi-modal interface for any types of segmentation with ONE SINGLE MODE.

🖥 Github: https://github.com/ux-decoder/segment-everything-everywhere-all-at-once

⏩ Paper: https://arxiv.org/abs/2304.06718v1

⭐️ Dataset: https://paperswithcode.com/dataset/refcoco

https://t.iss.one/DataScienceT

🎨 Animated Drawings

A Method for Automatically Animating Children's Drawings of the Human Figure

🖥 Github: https://github.com/facebookresearch/AnimatedDrawings

⭐️Project: https://fairanimateddrawings.com/site/home

⏩ Paper: arxiv.org/pdf/2303.12741.pdf

https://t.iss.one/DataScienceT

​​InceptionNeXt: When Inception Meets ConvNeXt

Large-kernel convolutions, such as those employed in ConvNeXt, can improve model performance but often come at the cost of efficiency due to high memory access costs. Although reducing kernel size may increase speed, it often leads to significant performance degradation.

To address this issue, the authors propose InceptionNeXt, which decomposes large-kernel depthwise convolution into four parallel branches along the channel dimension. This new Inception depthwise convolution results in networks with high throughputs and competitive performance. For example, InceptionNeXt-T achieves 1.6x higher training throughputs than ConvNeX-T and a 0.2% top-1 accuracy improvement on ImageNet-1K. InceptionNeXt has the potential to serve as an economical baseline for future architecture design, helping to reduce carbon footprint.

A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-inceptionnext

Paper link:https://arxiv.org/abs/2303.16900

Code link: https://github.com/sail-sg/inceptionnext

https://t.iss.one/DataScienceT

💻 Graph classification with Transformers

This notebook shows how to fine-tune the Graphormer model for Graph Classification on a dataset available on the hub.

🤗Hugging face blog: https://huggingface.co/blog/graphml-classification

⏩ Intro to Graphs: https://t.iss.one/ai_machinelearning_big_data/3214

🖥 Github: https://github.com/huggingface/blog/blob/main/notebooks/graphml-classification.ipynb

⏩ Paper: https://arxiv.org/abs/2106.05234

⭐️Dataset: https://ogb.stanford.edu/

https://t.iss.one/DataScienceT

📝 An open, billion-scale corpus of images interleaved with text.

MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

🖥 Github: https://github.com/allenai/mmc4

⏩ Paper: https://arxiv.org/abs/2304.06939v1

⭐️ Dataset: https://paperswithcode.com/dataset/c4

https://t.iss.one/DataScienceT

STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training

🖥 Github: https://github.com/ziyan-huang/stu-net

⏩ Paper: https://arxiv.org/pdf/2304.06716v1.pdf

💨 Dataset: https://paperswithcode.com/dataset/abdomenct-1k

https://t.iss.one/DataScienceT

Count anything

An empirical study on few-shot counting using segment anything

🖥 Github: https://github.com/vision-intelligence-and-robots-group/count-anything

⏩ Paper: https://arxiv.org/abs/2304.10817v1

🤗 Hugging face: https://huggingface.co/spaces/nebula/counting-anything

📌 Dataset: https://drive.google.com/file/d/1ymDYrGs9DSRicfZbSCDiOu0ikGDh5k6S/view?usp=sharing

https://t.iss.one/DataScienceT

Best Data Science Channels and groups on Telegram:

https://t.iss.one/addlist/8_rRW2scgfRhOTc0

Only click on OK and Will automatically add you to all channels

Update telegram version

🔍 Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System

Self-Controlled Memory (SCM) system to unleash infinite-length input capacity for large-scale language models.

🖥 Github: https://github.com/toufunao/SCM4LLMs

⏩ Paper: https://arxiv.org/abs/2304.13343v1

📌 Tasks: https://paperswithcode.com/task/language-modelling

https://t.iss.one/DataScienceT

🖌 Edit Everything: A Text-Guided Generative System for Images Editing

A text-guided generative system without any finetuning (zero-shot).

🖥 Github: https://github.com/defengxie/edit_everything

⏩ Paper: https://arxiv.org/abs/2304.14006v1

🚀 Dataset: https://paperswithcode.com/dataset/wukong

https://t.iss.one/DataScienceT