🍋GATO: agent for multiple tasks🍋

👉The same network with the same weights can play Atari, caption pics, chat, and more🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅General-purpose agent, multiple tasks
✅Multi-modal-task, multi-embodiment
✅Inspired by large-scale language model

More: https://bit.ly/3LbBOWb

🪐NeRF powered by keypoints🪐

👉ETHZ + META unveil how to encode relative spatial #3D info via sparse 3D keypoints

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Sparse 3D keypoints for SOTA avatars
✅Unseen subjects from 2/3 views
✅Never-before-seen iPhone captures

More: https://bit.ly/39NQqhe

🐌Self-Supervised human co-evolution🐌

👉Self-supervised 3D by co-evolution of pose estimator, imitator, and hallucinator

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel self-supervised 3D pose
✅Co-evo of pose, imitator, hallucinator
✅Realist 3D pose and 2D-3D supervision
✅Source code / model under MIT license

More: https://bit.ly/37J5ImL

🐲 Diff-SDF #3D Rendering 🐲

👉Reconstruction with no complex reg. or priors, using only a per-pixel RGB loss

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Diff-render to optimize geometry/albedo
✅No ad-hoc object mask or supervision
✅Extended sphere tracing algorithm

More: https://bit.ly/3yKWPnI

👄LVD: new SOTA for #3D human👄

👉Corona et al. unveils a novel 3D human model fitting

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Solution via neural field
✅Not sensitive to initialization
✅SOTA in shape from single pic
✅SOTA in fitting 3D scans

More: https://bit.ly/3Ng4lLr

🏳️‍🌈Deep Clustering on ImageNet & Co.🏳️‍🌈

👉World's first deep nonparametric clustering on large dataset such as ImageNet

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Deep clustering that infers nr. of clusters
✅Loss: amortized inference in mixt-models
✅Deep nonparametric clustering on ImageNet
✅Code and model available under MIT license

More: https://bit.ly/38p62rn

💥HQ-E²FGVI just released💥💥

👉Flow-Guided Video Inpainting through three trainable modules

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Flow, pixel-prop, content hallucination
✅Three stage-modules, jointly optimized
✅The new SOTA, promising efficiency
✅Code and Models under MIT license

More: https://bit.ly/3Ln0ICj

🪔 AvatarCLIP: Text-Driven Avatar 🪔

👉Zero-shot text-driven for #3D avatar in #metaverse

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅First text-driven synthesis
✅Shape, texture, and motion
✅Animation-ready, HQ texture/geometry
✅Zero-shot text-guided ref-based motion
✅Code and model under MIT license

More: https://bit.ly/3LjTWgB

🔥#AIwithPapers: we are 2,500!🔥

💙💛Only 2 Billion papers remaining on arXiv. The more we are, the faster we read💙💛

😈 Invite your friends -> https://t.iss.one/AI_DeepLearning

💥Podcasting AI & CV💥

👉🏼For people fluent in Italian: 1 hour podcast in which I talk about AI, CV, Startup and more (included this wonderful project).

More: https://bit.ly/38DtBwB

🔥Inpainting: new SOTA! INSANE🔥

👉Novel two-stream approach: inpainting at the next level!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅High-freq locally, low-freq globally
✅Local to global -> error correction
✅44% / 26% improvements FID/scores
✅Source code, more clips available

More: https://bit.ly/3ltIX9R

🔥Super-Human Crossword Solver🔥

👉Solving crosswords outperforming best humans

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Crossword solving based on NNs
✅Q&A, structured decoding, local search
✅Wide domains with perfect accuracy
✅Large question-answer dataset

More: https://bit.ly/3a3zzqQ

🥸Imagen: far beyond DALL·E 2🥸

👉#Google: unprecedented photorealism and deep level of language understanding

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Dynamic thresh diffusion sampling
✅Efficient U-Net, efficient++ variant
✅DrawBench, new text-to-image
✅The new SOTA, COCO FID of 7.27

More: https://bit.ly/3lVtkbz

🪤Tracking over SOTA detectors🪤

👉Lightweight Python lib for real-time 2D object tracking 💥

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Layer of tracking over SOTA detectors
✅Suitable for complex video processing
✅Source code under BSD 3-Clause
✅Maintained by Tryolabs team

More: https://bit.ly/3wKtGqg

🥷🏿 FCA: #3D Neural Camouflage 🥷🏿

👉#3D full-camouflage adversarial patch to fool neural detectors

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Attack by diff-neural render
✅E2E physical adversarial attack
✅Envs, vehicles & detectors
✅Source code available!

More: https://bit.ly/38kKyfa

🍋 One-Shot Object Pose 🍋

👉A novel one-shot object pose estimator

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Visual localization pipeline for object pose
✅Handling novel objects without CAD model
✅Novel graph attention for 2D-3D matching
✅Large dataset for one-shot object pose

More: https://bit.ly/3MTogjJ

☄️STEVE: Slot-TransformEr for VidEos☄️

👉STEVE: unsupervised model for object-centric learning in videos

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Adoption of a slot decoder (SLATE)
✅SLATE with slot-level recurrence model
✅Complex and naturalistic videos
✅Significantly outperforms previous SOTA

More: https://bit.ly/3PNxxM3

🦔 CogVideo: insane text-to-clip 🦔

👉CogVideo: 9B-parameters world's first large scale open-source text-to-video 😵

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Largest open-source T2C transformer
✅Finetuning of text-to-image model
✅Multi-frame-rate hierarchical training
✅From pretrained model CogView2

More: https://bit.ly/3Gzfl4n

🦄Time-Aware Neural Voxels🦄

👉TiNeuVox: "NeRF" with time-aware voxel features 😵

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Dynamic scene w/ optimizable structure
✅Temporal information in radiance net
✅Small/large motion w/ single-res of feats
✅192× faster than previous Hyper-NeRF

More: https://bit.ly/3wR4O08

🫐Neural Anomaly Detection by AWS🫐

👉Ultra-competitive inference and SOTA for both detection and localization

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Locally aggregated, mid-level feats patch
✅Maximizing nominal information at test time
✅Reducing biases towards ImageNet classes
✅Image-level anomaly AUROC of up to 99.6%

More: https://bit.ly/3t7Ndjg