🦀 Event-aided Direct Sparse Odometry 🦀

👉EDS: direct monocular visual odometry using events/frames

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Mono 6-DOF visual odometry + events
✅Direct photometric bundle adjustment
✅Camera motion tracking by sparse pixels
✅A new dataset with HQ events and frame

More: https://bit.ly/3s9FiBN

🫀BlobGAN: Blob-Disentangled Scene🫀

👉Unsupervised, mid-level (blobs) generation of scenes

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Spatial, depth-ordered Gaussian blobs
✅Reaching for supervised level, and more
✅Source under BSD-2 "Simplified" License

More: https://bit.ly/3kRyGnj

🦕E2EVE editor via pre-trained artist🦕

👉E2EVE generates a new version of the source image that resembles the "driver" one

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Blending regions by driver image
✅E2E cond-probability of the edits
✅S.S. augmenting in target domain
✅Implemented as SOTA transformer
✅Code/models available (soon)

More: https://bit.ly/3P9TDYW

🐶 Bringing pets in #metaverse 🐶

👉ARTEMIS: pipeline for generating articulated neural pets for virtual worlds

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅ARTiculated, appEarance, Mo-synthesIS
✅Motion control, animation & rendering
✅Neural-generated (NGI) animal engine
✅SOTA animal mocap + neural control

More: https://bit.ly/3LZSLDU

😍Animated hand in 1972, damn romantic😍

👉Q: is #VR the technology that developed least in the last 30 years? 🤔

More: https://bit.ly/3snxNaq

⏏️Ensembling models for GAN training⏏️

👉Pretrained vision models to improve the GAN training. FID by 1.5 to 2×!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅CV models as ensemble of discriminators
✅Improving GAN in limited / large-scale set
✅10k samples matches StyleGAN2 w/ 1.6M
✅Source code / models under MIT license

More: https://bit.ly/3wgUVsr

🤯Cooperative Driving + AUTOCASTSIM🤯

👉COOPERNAUT: cross-vehicle perception for vision-based cooperative driving

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅UTexas + #Stanford + #Sony #AI
✅LiDAR into compact point-based
✅Network-augmented simulator
✅Source code and models available

More: https://bit.ly/3sr5HLk

💄NeuralHDHair: 3D Neural Hair💄

👉NeuralHDHair: fully automatic system for modeling HD hair from a single image

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅IRHairNet for hair geometric features
✅GrowingNet: 3D hair strands in parallel
✅VIFu: novel voxel-aligned implicit function
✅SOTA in 3D hair modeling from single pic

More: https://bit.ly/38iR0mQ

🐡DyNeRF: Neural 3D Video Synthesis🐡

👉#Meta unveils DyNeRF, novel rendering HQ 3D video

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel NeRF-based on temp-latent codes
✅Novel training based on hierarchical step
✅Datasets of time-synch/calibrated clips
✅Attribution-NonCommercial 4.0 Int.

More: https://bit.ly/3MlBRA9

🍋GATO: agent for multiple tasks🍋

👉The same network with the same weights can play Atari, caption pics, chat, and more🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅General-purpose agent, multiple tasks
✅Multi-modal-task, multi-embodiment
✅Inspired by large-scale language model

More: https://bit.ly/3LbBOWb

🪐NeRF powered by keypoints🪐

👉ETHZ + META unveil how to encode relative spatial #3D info via sparse 3D keypoints

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Sparse 3D keypoints for SOTA avatars
✅Unseen subjects from 2/3 views
✅Never-before-seen iPhone captures

More: https://bit.ly/39NQqhe

🐌Self-Supervised human co-evolution🐌

👉Self-supervised 3D by co-evolution of pose estimator, imitator, and hallucinator

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel self-supervised 3D pose
✅Co-evo of pose, imitator, hallucinator
✅Realist 3D pose and 2D-3D supervision
✅Source code / model under MIT license

More: https://bit.ly/37J5ImL

🐲 Diff-SDF #3D Rendering 🐲

👉Reconstruction with no complex reg. or priors, using only a per-pixel RGB loss

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Diff-render to optimize geometry/albedo
✅No ad-hoc object mask or supervision
✅Extended sphere tracing algorithm

More: https://bit.ly/3yKWPnI

👄LVD: new SOTA for #3D human👄

👉Corona et al. unveils a novel 3D human model fitting

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Solution via neural field
✅Not sensitive to initialization
✅SOTA in shape from single pic
✅SOTA in fitting 3D scans

More: https://bit.ly/3Ng4lLr

🏳️‍🌈Deep Clustering on ImageNet & Co.🏳️‍🌈

👉World's first deep nonparametric clustering on large dataset such as ImageNet

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Deep clustering that infers nr. of clusters
✅Loss: amortized inference in mixt-models
✅Deep nonparametric clustering on ImageNet
✅Code and model available under MIT license

More: https://bit.ly/38p62rn

💥HQ-E²FGVI just released💥💥

👉Flow-Guided Video Inpainting through three trainable modules

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Flow, pixel-prop, content hallucination
✅Three stage-modules, jointly optimized
✅The new SOTA, promising efficiency
✅Code and Models under MIT license

More: https://bit.ly/3Ln0ICj

🪔 AvatarCLIP: Text-Driven Avatar 🪔

👉Zero-shot text-driven for #3D avatar in #metaverse

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅First text-driven synthesis
✅Shape, texture, and motion
✅Animation-ready, HQ texture/geometry
✅Zero-shot text-guided ref-based motion
✅Code and model under MIT license

More: https://bit.ly/3LjTWgB

🔥#AIwithPapers: we are 2,500!🔥

💙💛Only 2 Billion papers remaining on arXiv. The more we are, the faster we read💙💛

😈 Invite your friends -> https://t.iss.one/AI_DeepLearning

💥Podcasting AI & CV💥

👉🏼For people fluent in Italian: 1 hour podcast in which I talk about AI, CV, Startup and more (included this wonderful project).

More: https://bit.ly/38DtBwB

🔥Inpainting: new SOTA! INSANE🔥

👉Novel two-stream approach: inpainting at the next level!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅High-freq locally, low-freq globally
✅Local to global -> error correction
✅44% / 26% improvements FID/scores
✅Source code, more clips available

More: https://bit.ly/3ltIX9R