🎷🎷OMNI3D: #3D Objects in the Wild🎷🎷

👉#3D detection: 234k images, 3M+ instances & 97 categories

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅OMNI3D from publicly released dataset
✅234k pics, 3M+ annotation with 3D box
✅97 categories such as sofa, table, cars
✅Fast (450x) and exact algorithm for IoU
✅Cube R-CNN: novel 3D object detector

More: https://bit.ly/3cznjzG

👹Multiface Neural Rendering 👹

👉A new multi-view, Hi-Res data collected at #META Reality Labs for neural face

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Mugsy, large scale multi-cam apparatus
✅High-Res sync facial performance
✅Closing the gap in accessing HQ data
✅Suitable for #VR & #mixedreality

More: https://bit.ly/3b6XfeL

💄DEVIANT: SOTA in mono-3D detection💄

👉A novel Depth EquiVarIAnt NeTwork for 3D monocular detection in the wild

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Michigan + #Meta + Ford 🤯
✅Depth-equi. + scale equiv. steerable
✅New SOTA on KITTI & Waymo
✅Ok cross-dataset -> generalization

More: https://bit.ly/3OEFtgK

🧱 Assembling #LEGO with #AI 🧱

👉Step-by-step assembly manual created by human into machine-interpretable instructions

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Stanford + MIT + #Google 🤯
✅MEPNet: Manual-to-Executable-Plan Net
✅Manual to machine-executable plan
✅2D manual - 3D geometric shape
✅Reasoning on 3D alignments of legos

More: https://bit.ly/3PCwn5C

🔥🔥 UPDATE 🔥🔥

Code Released: https://github.com/apple/ml-mobileone

🎃New SOTA in UDA Semantic Seg.🎃

👉HRDA: multi-res Unsupervised Domain Adaptive Semantic Seg. -> SOTA

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅ETH + MPG + KU Leuven 🤯
✅HRDA: multi-res approach for UDA
✅Manageable GPU memory footprint
✅Small objects & fine segmentation detail
✅New SOTA on GTA and Synthia dataset

More: https://bit.ly/3cKtDEp

⚗️ SemAbs: 3D Scene Understanding ⚗️

👉Framework that equips 2D Vision-Language Models (VLMs) with new 3D spatial capabilities

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅2D VLMs with 3D reasoning skills
✅ViTs Efficient MS Relevancy Extraction
✅Novel Open-World understanding tasks
✅Completing partially observed objects
✅Finding hidden objects from language

More: https://bit.ly/3PYYk7d

🦚 TinyCD: Neural Change Detection 🦚

👉TinyCD: new SOTA in change detection with up to 150x fewer parameters.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅SOTA with up to 150X fewer params
✅Mixing blocks for s.t. cross-correlation
✅PW-MLP for pixel wise classification
✅MAMB: novel block for skip connection

More: https://bit.ly/3zFEngk

🦊 3D-Aware "StyleGANv2" version 🦊

👉Upgrading StyleGANv2 into a novel 3D-aware GAN with just a minimal set of changes🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅MPI-like 3D-aware GAN w/ single-view
✅GMPI: generative multiplane image
✅2D GAN 3D-aware with a minimal changes
✅Encoding 3D-aware inductive biases

More: https://bit.ly/3OJ5gnS

📺 NeRF-ing "The Big Bang Theory" 📺

👉Berkeley unveils an approach for accurate estimation of actor’s 3D pose & location

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Input: images across the whole season
✅3D context (i.e. cams, structure, body)
✅Integrating context in 3D estimation
✅Re-ID, gaze, cinematography, pic editing
✅Knock, Knock, Penny!

More: https://bit.ly/3OLuaUb

🎩ShAPO: SOTA in object understanding🎩

👉Joint multi-object detection, #3D texture, 6D object pose & size estimation.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Disentangled shape & appearance
✅Efficient octree-based differentiable
✅Object-centric understanding pipeline
✅Detection, reconstruction , 6D & size
✅SOTA in reconstruction & pose est.

More: https://bit.ly/3oHN5EQ

🏙️ CityNeRF: Neural Rendering of City Scenes 🏙️

👉Progressive NeRF model and training set on city-scenes

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅BungeeNeRF: novel progressive NeRF
✅Details on drastically varied scales
✅Growing with residual block structure
✅Inclusive multi-level data supervision

More: https://bit.ly/3cS9vk7

🍦🍦 Rewriting Geometry of GAN 🍦🍦

👉Drive GAN synthesizing many unseen objects with the desired shape

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅User-friendly "warping" with geometry
✅Low-rank update to layer for editing
✅Latent augmentation based on style-mix
✅Endless objects with defined changes
✅Latent space interpolation, image editing

More: https://bit.ly/3zIfOj8

🍏🍏 GAUDI: the Neural Architect 🍏🍏

👉Novel generative model for immersive 3D scenes from a moving camera

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Hundreds of thousands pics/scenes
✅Novel denoising optimization objective
✅New SOTA across multiple datasets
✅Un/conditional on images/text

More: https://bit.ly/3Bt65ye

🚜NeDDF: the NeRF evolution!🚜

👉Novel 3D representation that reciprocally constrains distance & density fields

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅NeRF provides no distance
✅Extending for arbitrary density
✅Density via dist-field & gradient
✅Alleviating the instability

More: https://bit.ly/3Bte8LC

🔥AND/OR: Composable Diffusion Models🔥

👉Novel neural compositional generation via Composable Diffusion Models

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅DM as energy-based models
✅Connecting diffusion models
✅Conjunction & negation, on top of DM
✅Zero-shot combinatorial generalization

More: https://bit.ly/3PYv1Cs

🔥 MobileNeRF is out -> Pure Fire! 🔥

👉MobileNeRF is out: the mobile evolution of NeRF via textured polygons.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Same quality, 10x faster than SNeRG
✅Memory-- by storing surface textures
✅Integrated GPUs: less memory/power
✅Suitable for browser & viewer is HTML

More: https://bit.ly/3PUKPWy

🔥🔥UPDATE🔥🔥

Code Released: https://github.com/andreas128/RePaint

🧣NeRF for Outdoor Scene Relighting🧣

👉NeRF-OSR: the first neural radiance fields approach for outdoor scene relighting

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅NeRF-method for outdoor relighting
✅Simultaneous illumination/viewpoint
✅Control over shading, shadow, albedo
✅Self-Supervised training from outdoor
✅Dataset: 3240 viewpoints, 110+ times

More: https://bit.ly/3vBiH2G

👩‍🦰 Real-Time Neural Hair 👩‍🦰

👉Accurate hair geometry & appearance from multi-pics

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Bonn, CMU and Reality Labs
✅Photorealistic Real-Time render
✅HQ strand geometry/appearance
✅Novel scalp texture description
✅Intuitive manipulation of 3D hair

More: https://bit.ly/3vBiH2G