🎭Novel pre-training strategy for #AI🎭

👉EPFL unveils the Multi-modal Multi-task Masked Autoencoders (MultiMAE)

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Multimodal: additional modal. over RGB
✅Multi-task: multiple outputs over RGB
✅General: MultiMAE by pseudo-labeling
✅Classification, segmentation, depth
✅Code under NonCommercial 4.0 Int.

More: https://bit.ly/3jRhNsN

🧪 A new SOTA in Dataset Distillation 🧪

👉A new approach by Matching Training Trajectories is out!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Distilling data "to match" bigger one
✅Distilled data to guide a network
✅Trajectories of experts from real data
✅SOTA + distilling higher-res visual data

More: https://bit.ly/3JwYOxW

🧤 Two-Hand tracking via GCN 🧤

👉The first-ever GCN for two interacting hands in single RGB image

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Reconstruction by GCN mesh regression
✅PIFA: pyramid attention for local occlusion
✅CHA: cross hand attention for interaction
✅SOTA + generalization in-the-wild scenario
✅Source code available under GNU 🤯

More: https://bit.ly/3KH5FWO

🕹️Video K-Net, SOTA in Segmentation🕹️

👉Simple, strong, and unified framework for fully end-to-end video panoptic segmentation

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Learnable kernels from K-Net
✅K-Net learns to segment & track
✅Appearance / cross-T kernel interaction
✅New SOTA without bells and whistles 🤷‍♂️

More: https://bit.ly/3uEEZQR

🐭DeepLabCut: tracking animals in the wild🐭

👉A toolbox for markerless pose estimation of animals performing various tasks

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Multi-animal pose estimation
✅Datasets for multi-animal pose
✅Key-points, limbs, animal identity
✅Optimal key-points without input

More: https://bit.ly/37L1mLE

🍡Neural Articulated Human Body🍡

👉Novel neural implicit representation for articulated body

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅COmpositional Articulated People
✅Large variety of shapes & poses
✅Novel encoder-decoder architecture

More: https://bit.ly/3xvn7dl

🦚 2K Resolution Generative #AI 🦚

👉Novel continuous-scale training with variable output resolutions

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Mixed-resolution data
✅Arbitrary scales during training
✅Generations beyond 1024×1024
✅Variant of FID metric for scales
✅Source code under MIT license

More: https://bit.ly/3uNfVY6

🐍DS Unsupervised Video Decomposition🐍

👉Novel method to extract persistent elements of a scene

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Scene element as Deformable Sprite (DS)
✅Deformable Sprites by video auto-encoder
✅Canonical texture image for appearance
✅Non-rigid geom. transformation

More: https://bit.ly/37WV9w1

🥓 L-SVPE for Deep Deblurring 🥓

👉L-SVPE to deblur scenes while recovering high-freq details

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Learned Spatially Varying Pixel Exposures
✅Next-gen focal-plane sensor + DL
✅Deep conv decoder for motion deblurring
✅Superior results over non-optimized exp.

More: https://bit.ly/3uRYQMT

🧧Hyper-Fast Instance Segmentation🧧

👉Novel Temporally Efficient Vision Transformer (TeViT) for VIS

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Video instance segmentation transformer
✅Contextual-info at frame/instance level
✅Nearly convolution-free framework 🤷‍♂️
✅The new SOTA for VIS, ~70 FPS!
✅Code & models under MIT license

More: https://bit.ly/3rCMXIn

📗Unified Scene Text/Layout Detection📗

👉World's first hierarchical scene text dataset + novel detection method

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Unified detection & geometric layout
✅Hierarchical annotations in natural scenes
✅Word, line, & paragraph level annotations
✅Source under CC Attribution Share Alike 4.0

More: https://bit.ly/3jRpezV

🙌 #Oculus' new Hand Tracking 🙌

👉Hands are able to move as naturally and intuitively in the #metaverse as do in real life

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Hands2.0 powered by CV & ML
✅Tracking hand-over-hand interactions
✅Crossing hands, clapping, high-fives
✅Accurate thumbs-up gesture

More: https://bit.ly/3JXPvY2

🎗️New SOTA in #3D human avatar🎗️

👉PHORHUM: photorealistic 3D human from mono-RGB

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Pixel-aligned method for 3D geometry
✅Unshaded surface color + illumination
✅Patch-based rendering losses for visible
✅Plausible color estimation for non-visible

More: https://bit.ly/3MkvBrA

📟 What's in your hands (#3D) ? 📟

👉Reconstructing hand-held objects (from single RGB) without knowing their 3D templates🤷‍♂️

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Hand is highly predictive of object shape
✅Conditional-based on the articulation
✅Visual feats. / articulation-aware coords.
✅Code and models available!

More: https://bit.ly/3vuYn2a

🔋YODO: You Only Demonstrate Once🔋

👉A novel category-level manipulation learned in sim from single demonstration video🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅One-shot IL, model-free 6D pose tracking
✅Demonstration BY single 3rd-person-view
✅manipulation including hi-precision tasks
✅Category-level Behavior Cloning
✅Attention for dynamic coords selection
✅Generalizability to novel unseen obj/env

More: https://bit.ly/3v0V4R4

👗 Dress Code for Virtual Try-On 👗

👉UniMORE (+ YOOX) unveils a novel dataset/approach for virtual try-on.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Hi-Res paired front-view / full-body
✅Pixel-level Semantic-Aware Discriminator
✅9 SOTA VTON approaches / 3 baselines
✅New SOTA considering res. & garments

More: https://bit.ly/3xKXSUw

🍃Deep Equilibrium for Optical Flow🍃

👉DEQ: converge faster, less memory, often more accurate

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel formulation of optical flow method
✅Compatible with prior modeling/data-related
✅Sparse fixed-point correction for stability
✅Code/models under GNU Affero GPL v3.0

More: https://bit.ly/3v4fZmi

🌳Ultra High-Resolution Neural Saliency🌳

👉A novel ultra high-resolution saliency detector with dataset!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Ultra Hi-Res Saliency Detection
✅5,920 pics at 4K-8K resolution
✅Pyramid Grafting Network
✅Cross-Model Grafting Module
✅AGL: Attention Guided Loss
✅Code/models under MIT

More: https://bit.ly/3MnU1Rf

🪆StyleGAN-Human for fashion 🪆

👉A novel unconditional human generation based on StyleGAN is out!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅200,000+ labeled sample (pose/texture)
✅1024x512 StyleGAN-Human StyleGAN3
✅512x256 StyleGAN-Human StyleGAN1
✅Face model for downstream: InsetGAN
✅Source code and model available!

More: https://bit.ly/3xMg5B2