Castle in the Sky
Dynamic Sky Replacement and Harmonization in Videos
Fascinating and ready to be applied for work. (With colab notebook)
The authors proposed a method to replace the sky in the video that works well in high resolution. The results are very impressive. The method runs in real-time and produces video almost without glitches and artifacts. Also, can generate for example lightning and glow on target video.
The pipeline is quite complicated and contains several tasks:
– A sky matting network to segmentation sky on video frames
– A motion estimator for sky objects
– A skybox for blending where sky and other environments on video are relighting and recoloring.
Authors say their work, in a nutshell, proposes a new framework for sky augmentation in outdoor videos. The solution is purely vision-based and it can be applied to both online and offline scenarios.
But let's take a closer look.
A sky matting module is a ResNet-like encoder and several layers upsampling decoder to solve sky pixel-wise segmentation tasks followed by a refinement stage with guided image filtering.
A motion estimator directly estimates the motion of the objects in the sky. The motion patterns are modeled by an affine matrix and optical flow.
The sky image blending module is a decoder that models a linear combination of target sky matte and aligned sky template.
Overall, the network architecture is ResNet-50 as encoder and decoder with coordConv upsampling layers with skip connections and implemented in Pytorch,
The result is presented in a very cool video https://youtu.be/zal9Ues0aOQ
site: https://jiupinjia.github.io/skyar/
paper: https://arxiv.org/abs/2010.11800
github: https://github.com/jiupinjia/SkyAR
#sky #CV #video #cool #resnet
Dynamic Sky Replacement and Harmonization in Videos
Fascinating and ready to be applied for work. (With colab notebook)
The authors proposed a method to replace the sky in the video that works well in high resolution. The results are very impressive. The method runs in real-time and produces video almost without glitches and artifacts. Also, can generate for example lightning and glow on target video.
The pipeline is quite complicated and contains several tasks:
– A sky matting network to segmentation sky on video frames
– A motion estimator for sky objects
– A skybox for blending where sky and other environments on video are relighting and recoloring.
Authors say their work, in a nutshell, proposes a new framework for sky augmentation in outdoor videos. The solution is purely vision-based and it can be applied to both online and offline scenarios.
But let's take a closer look.
A sky matting module is a ResNet-like encoder and several layers upsampling decoder to solve sky pixel-wise segmentation tasks followed by a refinement stage with guided image filtering.
A motion estimator directly estimates the motion of the objects in the sky. The motion patterns are modeled by an affine matrix and optical flow.
The sky image blending module is a decoder that models a linear combination of target sky matte and aligned sky template.
Overall, the network architecture is ResNet-50 as encoder and decoder with coordConv upsampling layers with skip connections and implemented in Pytorch,
The result is presented in a very cool video https://youtu.be/zal9Ues0aOQ
site: https://jiupinjia.github.io/skyar/
paper: https://arxiv.org/abs/2010.11800
github: https://github.com/jiupinjia/SkyAR
#sky #CV #video #cool #resnet
YouTube
Dynamic Sky Replacement and Harmonization in Videos
Preprint: Castle in the Sky: Zhengxia Zou, Dynamic Sky Replacement and Harmonization in Videos, 2020.
Project page: https://jiupinjia.github.io/skyar/
Project page: https://jiupinjia.github.io/skyar/
👍1
Forwarded from Находки в опенсорсе
Higher Kinded Types in #python!
I am happy to announce that the first version of Higher Kinded Types
emulation is released with full mypy support as a part of
Source code and docs: https://github.com/dry-python/returns
Try it: https://gist.github.com/sobolevn/7f8ffd885aec70e55dd47928a1fb3e61
In the nearest future, we will share some practical libraries using HKTs in Python, including type-safe business-validation, smart and simple framework-agnostic REST framework, and so on! This is a big day for Python and its ecosystem.
In this article I am going to show how HKT works and why it is useful.
I am happy to announce that the first version of Higher Kinded Types
emulation is released with full mypy support as a part of
dry-python/[email protected].Source code and docs: https://github.com/dry-python/returns
Try it: https://gist.github.com/sobolevn/7f8ffd885aec70e55dd47928a1fb3e61
In the nearest future, we will share some practical libraries using HKTs in Python, including type-safe business-validation, smart and simple framework-agnostic REST framework, and so on! This is a big day for Python and its ecosystem.
In this article I am going to show how HKT works and why it is useful.
Forwarded from Professor M
“Simplicity is the ultimate sophistication”
In early math classes, we learn that division by zero is a banned operation. If you could violate the rule and stealthily divide by zero in your derivations, you could, for example, prove absurdities such as 2=1. Later on, calculus introduces limits. Division by a number indistinguishable from zero (in the limit) is no longer banned; it produces infinity (in the limit).
When I code in Python, however, I divide by zero all the time. Why do I do this?
Occasionally, you want the script to stop execution when certain conditions occur. Perhaps, you’re still developing the program and, before moving forward, want to report the values of some variables when reaching a specific juncture in the code and then halt. How would you terminate the Python script?
The textbook way of doing so is with the command sys.exit(). Plus, you have to import the sys library with import sys. Twenty characters in total; pretty simple. But what if you could achieve that same objective of halting the script with three characters. How?
Divide by zero. Type 1/0 and the script would stop at this line because of the division-by-zero error. If you can achieve an objective with three characters, why would you use twenty?
When I use STATA for statistical analysis, I also sometimes want the script to terminate at specific points. In STATA, however, I don’t even know the textbook way of doing so. So how do I do it? I use a one-word command: stop. It turns out there is no pre-programmed command stop in STATA. When the software reaches this line, it doesn’t know what to do. It reports an error and then—stops.
I had frowned upon such seemingly non-elegant solutions before acquiring a taste for them. Wearing a tuxedo for a black-tie event is elegant; wearing one for breakfast reveals confusing priorities.
In early math classes, we learn that division by zero is a banned operation. If you could violate the rule and stealthily divide by zero in your derivations, you could, for example, prove absurdities such as 2=1. Later on, calculus introduces limits. Division by a number indistinguishable from zero (in the limit) is no longer banned; it produces infinity (in the limit).
When I code in Python, however, I divide by zero all the time. Why do I do this?
Occasionally, you want the script to stop execution when certain conditions occur. Perhaps, you’re still developing the program and, before moving forward, want to report the values of some variables when reaching a specific juncture in the code and then halt. How would you terminate the Python script?
The textbook way of doing so is with the command sys.exit(). Plus, you have to import the sys library with import sys. Twenty characters in total; pretty simple. But what if you could achieve that same objective of halting the script with three characters. How?
Divide by zero. Type 1/0 and the script would stop at this line because of the division-by-zero error. If you can achieve an objective with three characters, why would you use twenty?
When I use STATA for statistical analysis, I also sometimes want the script to terminate at specific points. In STATA, however, I don’t even know the textbook way of doing so. So how do I do it? I use a one-word command: stop. It turns out there is no pre-programmed command stop in STATA. When the software reaches this line, it doesn’t know what to do. It reports an error and then—stops.
I had frowned upon such seemingly non-elegant solutions before acquiring a taste for them. Wearing a tuxedo for a black-tie event is elegant; wearing one for breakfast reveals confusing priorities.
👍1
Forwarded from DL in NLP (nlpcontroller_bot)
The story behind Paranoid Transformer 🔥🔥
GPT generator + BERT filter + hand-writing generation RNN
https://medium.com/altsoph/paranoid-transformer-80a960ddc90a
GPT generator + BERT filter + hand-writing generation RNN
https://medium.com/altsoph/paranoid-transformer-80a960ddc90a
Medium
PARANOID TRANSFORMER
The pre-order of my book, Paranoid Transformer, generated by a bunch of neural networks, is now open. Here is the story…
News on new Macbook Pro 13:
* M1 Apple chip with built in stuff for ML — but anyway you won't build models on the laptop
* Max 16 Gb RAM — so you won't be able to open more tabs in Chrome / Safari
* 100% recycled alluminium — good for nature
* Improved microphones and camera — collegues will see better picture of you and listen to your cats meowing clearer
And still no reasons to update if you are doing any DS.
#Apple
* M1 Apple chip with built in stuff for ML — but anyway you won't build models on the laptop
* Max 16 Gb RAM — so you won't be able to open more tabs in Chrome / Safari
* 100% recycled alluminium — good for nature
* Improved microphones and camera — collegues will see better picture of you and listen to your cats meowing clearer
And still no reasons to update if you are doing any DS.
#Apple
Benford’s Law, DS and the 2020 Election
This law can be used for the very basic check on wether the data was artificially generated or not. It assumes that lower digits have higher probability of occuring.
And there can be nothing better for #reproducibleresearch concept promotion, than #openresearch on poll data, because it shows that those can and should be transparent and open.
With the help of the repo below anyone can check compliance of poll data results with the #BenfordsLaw on unofficial (or official if you are able to get that data).
KDnuggets tutorial: https://www.kdnuggets.com/2020/09/diy-election-fraud-analysis-benfords-law.html
Github repo with examples on unofficial US election data: https://github.com/cjph8914/2020_benfords
#statistics
This law can be used for the very basic check on wether the data was artificially generated or not. It assumes that lower digits have higher probability of occuring.
And there can be nothing better for #reproducibleresearch concept promotion, than #openresearch on poll data, because it shows that those can and should be transparent and open.
With the help of the repo below anyone can check compliance of poll data results with the #BenfordsLaw on unofficial (or official if you are able to get that data).
KDnuggets tutorial: https://www.kdnuggets.com/2020/09/diy-election-fraud-analysis-benfords-law.html
Github repo with examples on unofficial US election data: https://github.com/cjph8914/2020_benfords
#statistics
KDnuggets
DIY Election Fraud Analysis Using Benford's Law - KDnuggets
In this article, we will talk about a Do-It-Yourself approach towards election analysis and coming to a conclusion whether the elections were conducted fairly or not.
Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes
#3DRCAN for denoising, super resolution and expansion microscopy.
GitHub: https://github.com/AiviaCommunity/3D-RCAN
ArXiV: https://www.biorxiv.org/content/10.1101/2020.08.27.270439v1
#biolearning #cv #dl
#3DRCAN for denoising, super resolution and expansion microscopy.
GitHub: https://github.com/AiviaCommunity/3D-RCAN
ArXiV: https://www.biorxiv.org/content/10.1101/2020.08.27.270439v1
#biolearning #cv #dl
Tutorial on Generative Adversarial Networks (GANs) with Keras and TensorFlow
Nice tutorial with enough theory to understand what you are doing and code to get it done.
Link: https://www.pyimagesearch.com/2020/11/16/gans-with-keras-and-tensorflow/
#Keras #TensorFlow #tutorial #wheretostart #GAN
Nice tutorial with enough theory to understand what you are doing and code to get it done.
Link: https://www.pyimagesearch.com/2020/11/16/gans-with-keras-and-tensorflow/
#Keras #TensorFlow #tutorial #wheretostart #GAN
🔥Breaking: Discord plugin for JetBrains IDEs integration
This plugin allows you to show what are you working on.
Compatible with PyChar, DataGrip and other JetBrains IDEs
Link: https://plugins.jetbrains.com/plugin/10233-discord-integration
#JetBrains #Discord #productivity #effectiveness #success
This plugin allows you to show what are you working on.
Compatible with PyChar, DataGrip and other JetBrains IDEs
Link: https://plugins.jetbrains.com/plugin/10233-discord-integration
#JetBrains #Discord #productivity #effectiveness #success
JetBrains Marketplace
Discord Integration - IntelliJ IDEs Plugin | Marketplace
Show everyone on Discord what awesome projects you're working on! Features:Publish information about your current project to DiscordHighly customizable in using the...
👍2
DeepMind significally (+100%) improved protein folding modelling
Why is this important: protein folding = protein structure = protein function = how protein works in the living speciment and what it does.
What this means: better vaccines, better meds, more curable diseases and more calamities easen by the medications or better understanding.
Dataset: ~170000 available protein structures from PDB
Hardware: 128 TPUv3 cores (roughly equivalent to ~100-200 GPUs)
Link: https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology
#DL #NLU #proteinmodelling #bio #biolearning #insilico #deepmind #AlphaFold
Why is this important: protein folding = protein structure = protein function = how protein works in the living speciment and what it does.
What this means: better vaccines, better meds, more curable diseases and more calamities easen by the medications or better understanding.
Dataset: ~170000 available protein structures from PDB
Hardware: 128 TPUv3 cores (roughly equivalent to ~100-200 GPUs)
Link: https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology
#DL #NLU #proteinmodelling #bio #biolearning #insilico #deepmind #AlphaFold
Protein Folding explained
#deepmind didn’t stop at delivering the improved technology for folding #proteinfolding . They also provided really cool video explaining why is that so cool.
YouTube: https://www.youtube.com/watch?v=KpedmJdrTpY&feature=emb_title
#explained #bio #proteinmodelling
#deepmind didn’t stop at delivering the improved technology for folding #proteinfolding . They also provided really cool video explaining why is that so cool.
YouTube: https://www.youtube.com/watch?v=KpedmJdrTpY&feature=emb_title
#explained #bio #proteinmodelling
YouTube
Protein folding explained
Join DeepMind Science Engineer Kathryn Tunyasuvunakool to explore the hidden world of proteins.
These tiny molecular machines underpin every biological process in every living thing and each one has a unique 3D shape that determines how it works and what…
These tiny molecular machines underpin every biological process in every living thing and each one has a unique 3D shape that determines how it works and what…
Animating Pictures with Eulerian Motion Fields
New method for single image animation. Authors promised to release code soon!
Website: https://eulerian.cs.washington.edu
Paper: https://eulerian.cs.washington.edu/animating_pictures_2020.pdf
ArXiV: https://arxiv.org/abs/2011.15128
YouTube: https://www.youtube.com/watch?v=4zKliOMilGY
#DL #animation #WashingtonUni
New method for single image animation. Authors promised to release code soon!
Website: https://eulerian.cs.washington.edu
Paper: https://eulerian.cs.washington.edu/animating_pictures_2020.pdf
ArXiV: https://arxiv.org/abs/2011.15128
YouTube: https://www.youtube.com/watch?v=4zKliOMilGY
#DL #animation #WashingtonUni
YouTube
Animating Pictures with Eulerian Motion Fields
Aleksander Holynski, Brian Curless, Steven M. Seitz, Richard Szeliski
Project Website: https://eulerian.cs.washington.edu
Project Website: https://eulerian.cs.washington.edu
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
This technology allows to move camera a bit on any video, slow down time or do both. Great application for video producers and motion designers.
Website: https://www.cs.cornell.edu/~zl548/NSFF/
ArXiV: https://arxiv.org/abs/2011.13084
YouTube: https://youtu.be/qsMIH7gYRCc
#Nerf #videointerpolation #DL
This technology allows to move camera a bit on any video, slow down time or do both. Great application for video producers and motion designers.
Website: https://www.cs.cornell.edu/~zl548/NSFF/
ArXiV: https://arxiv.org/abs/2011.13084
YouTube: https://youtu.be/qsMIH7gYRCc
#Nerf #videointerpolation #DL
Forwarded from Spark in me (Alexander)
Trying Out New Ampere GPUs and MIG
Our hands on experience with new Ampere GPUs - 3090 and A100 (with multi instance GPU)
https://habr.com/ru/post/531436/
Please like / share / repost!
#hardware
#deep_learning
Our hands on experience with new Ampere GPUs - 3090 and A100 (with multi instance GPU)
https://habr.com/ru/post/531436/
Please like / share / repost!
#hardware
#deep_learning
Habr
Playing with Nvidia's New Ampere GPUs and Trying MIG
Every time when the essential question arises, whether to upgrade the cards in the server room or not, I look through similar articles and watch such videos.
👩🎓Online lectures on Special Topics in AI: Deep Learning
Fresh free and open playlist on special topics in #DL from University of Wisconsin-Madison. Topics covering reliable deep learning, generalization, learning with less supervision, lifelong learning, deep generative models and more.
Overview Lecture: https://www.youtube.com/watch?v=6LSErxKe634&list=PLKvO2FVLnI9SYLe1umkXsOfIWmEez04Ii
YouTube Playlist: https://www.youtube.com/playlist?list=PLKvO2FVLnI9SYLe1umkXsOfIWmEez04Ii
Syllabus: https://pages.cs.wisc.edu/~sharonli/courses/cs839_fall2020/schedule.html
#wheretostart #lectures #YouTube
Fresh free and open playlist on special topics in #DL from University of Wisconsin-Madison. Topics covering reliable deep learning, generalization, learning with less supervision, lifelong learning, deep generative models and more.
Overview Lecture: https://www.youtube.com/watch?v=6LSErxKe634&list=PLKvO2FVLnI9SYLe1umkXsOfIWmEez04Ii
YouTube Playlist: https://www.youtube.com/playlist?list=PLKvO2FVLnI9SYLe1umkXsOfIWmEez04Ii
Syllabus: https://pages.cs.wisc.edu/~sharonli/courses/cs839_fall2020/schedule.html
#wheretostart #lectures #YouTube
YouTube
CS839 Special Topics in Deep Learning: Course Overview (Lecture 1)
👍1
Opinion: Remote jobs are to stay even after pandemic
As Packy McCormick (author of Notboring blog) writes in his recent post: we are never going back. Pandemic has catalized the global switch to remote jobs and acceptance of it (including consideration of stock options for remote workers).
Given that we as @opendatascience community are able to admit that those are only opinions and the future is more complex and unpredictable, we are posting a list of remote job aggregators so every reader can explore those opportunities if needed.
Blog entry: https://notboring.substack.com/p/were-never-going-back
Audio version: https://open.spotify.com/show/6k1YLBvORRMyosKy3x1xIl?si=_Z7mdecqTSSYrwhHexwGEA
Telegram bots:
@sixnomads_bot — a bot that connects you with relevant remote and full-time jobs that fit your tech stack, desired time zone and salary
@remotejobss — a channel that posts new remote opportunities daily
@datasciencejobseeker — a chat for jobs in data science
@remotejobpositions — a channel with interesting remote jobs for developers.
Here are some websites that might be a good alternative for you:
remoteok.io — a colorful job board with remote jobs in tech companies (from the creators of Nomadlist)
weworkremotely.com — another job board with new remote opportunities updated every day.
remote.co — a hub to learn new tips on working remotely and find your new remote job.
#hr #career #job #remote
As Packy McCormick (author of Notboring blog) writes in his recent post: we are never going back. Pandemic has catalized the global switch to remote jobs and acceptance of it (including consideration of stock options for remote workers).
Given that we as @opendatascience community are able to admit that those are only opinions and the future is more complex and unpredictable, we are posting a list of remote job aggregators so every reader can explore those opportunities if needed.
Blog entry: https://notboring.substack.com/p/were-never-going-back
Audio version: https://open.spotify.com/show/6k1YLBvORRMyosKy3x1xIl?si=_Z7mdecqTSSYrwhHexwGEA
Telegram bots:
@sixnomads_bot — a bot that connects you with relevant remote and full-time jobs that fit your tech stack, desired time zone and salary
@remotejobss — a channel that posts new remote opportunities daily
@datasciencejobseeker — a chat for jobs in data science
@remotejobpositions — a channel with interesting remote jobs for developers.
Here are some websites that might be a good alternative for you:
remoteok.io — a colorful job board with remote jobs in tech companies (from the creators of Nomadlist)
weworkremotely.com — another job board with new remote opportunities updated every day.
remote.co — a hub to learn new tips on working remotely and find your new remote job.
#hr #career #job #remote
Not Boring by Packy McCormick
We're Never Going Back
The future of work is remote, and companies don't really have a choice.
Tool for restoration of pixelated images
Tool uses De Bruijn sequence to restore the original information
Github: https://github.com/beurtschipper/Depix
#pixelization #github
Tool uses De Bruijn sequence to restore the original information
Github: https://github.com/beurtschipper/Depix
#pixelization #github
MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
Work on conditional image generation
ArXiV: https://arxiv.org/abs/2012.02821
#GAN #DL #food2vec
Work on conditional image generation
ArXiV: https://arxiv.org/abs/2012.02821
#GAN #DL #food2vec
Data Science by ODS.ai 🦜
Tool for restoration of pixelated images Tool uses De Bruijn sequence to restore the original information Github: https://github.com/beurtschipper/Depix #pixelization #github
Bayesian Image Reconstruction using Deep Generative Models
Really impressive results reported, but there is no code yet.
ArXiV: https://arxiv.org/abs/2012.04567
#DL #BayesianLearning #Reconstruction
Really impressive results reported, but there is no code yet.
ArXiV: https://arxiv.org/abs/2012.04567
#DL #BayesianLearning #Reconstruction