Forwarded from opendatasciencebot
Microsoft's FLAML - Fast and Lightweight AutoML
Github: https://github.com/microsoft/FLAML
Code: https://github.com/microsoft/FLAML/tree/main/notebook/
Paper: https://arxiv.org/abs/2106.04815v1
@a
Github: https://github.com/microsoft/FLAML
Code: https://github.com/microsoft/FLAML/tree/main/notebook/
Paper: https://arxiv.org/abs/2106.04815v1
@a
👍1
Forwarded from Gradient Dude
Chinese researchers are very fond of doing extensive surveys of a particular sub-field of machine learning, listing the main works and the major breakthrough ideas. There are so many articles published every day, and it is impossible to read everything. Therefore, such reviews are valuable (if they are well written, of course, which is quite rare).
Recently there was a very good paper reviewing various variants of Transformers with a focus on language modeling (NLP). This is a must-read for anyone getting into the world of NLP and interested in Transformers. The paper discusses the basic principles of self-attention and such details of modern variants of Transformers as architecture modifications, pre-training, and various applications.
📝Paper: A Survey of Transformers.
Recently there was a very good paper reviewing various variants of Transformers with a focus on language modeling (NLP). This is a must-read for anyone getting into the world of NLP and interested in Transformers. The paper discusses the basic principles of self-attention and such details of modern variants of Transformers as architecture modifications, pre-training, and various applications.
📝Paper: A Survey of Transformers.
Color2Style: Real-Time Exemplar-Based Image Colorization with Self-Reference Learning and Deep Feature Modulation
ArXiV: https://arxiv.org/pdf/2106.08017.pdf
#colorization #dl
ArXiV: https://arxiv.org/pdf/2106.08017.pdf
#colorization #dl
Semi-Autoregressive Transformer for Image Captioning
Current state-of-the-art image captioning models use autoregressive decoders - they generate one word after another, which leads to heavy latency during inference. Non-autoregressive models predict all the words in parallel; however, they suffer from quality degradation as they remove word dependence excessively.
The authors suggest a semi-autoregressive approach to image captioning to improve a trade-off between speed and quality: the model keeps the autoregressive property in global but generates words parallelly in local. Experiments on MSCOCO show that SATIC can achieve a better trade-off without bells and whistles.
Paper: https://arxiv.org/abs/2106.09436
A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-satic
#imagecaptioning #deeplearning #transformer
Current state-of-the-art image captioning models use autoregressive decoders - they generate one word after another, which leads to heavy latency during inference. Non-autoregressive models predict all the words in parallel; however, they suffer from quality degradation as they remove word dependence excessively.
The authors suggest a semi-autoregressive approach to image captioning to improve a trade-off between speed and quality: the model keeps the autoregressive property in global but generates words parallelly in local. Experiments on MSCOCO show that SATIC can achieve a better trade-off without bells and whistles.
Paper: https://arxiv.org/abs/2106.09436
A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-satic
#imagecaptioning #deeplearning #transformer
Forwarded from Spark in me (Alexander)
Transformer Module Optimization
Article on how to apply different methods to make your transformer network up to 10x smaller and faster:
- Plain model optimization and PyTorch tricks;
- How and why to use FFT instead of self-attention;
- Model Factorization and quantization;
https://habr.com/ru/post/563778/
#deep_learning
Article on how to apply different methods to make your transformer network up to 10x smaller and faster:
- Plain model optimization and PyTorch tricks;
- How and why to use FFT instead of self-attention;
- Model Factorization and quantization;
https://habr.com/ru/post/563778/
#deep_learning
Хабр
Сжимаем трансформеры: простые, универсальные и прикладные способы cделать их компактными и быстрыми
Сейчас в сфере ML постоянно слышно про невероятные "успехи" трансформеров в разных областях. Но появляется все больше статей о том, что многие из этих успехов м...
Forwarded from Towards NLP🇺🇦
DocNLI
Natural Language Inference (NLI) is the task of determining whether a “hypothesis” is true (entailment), false (contradiction), or undetermined (neutral) given a “premise”.
Previously, this task was solved for sentence-level texts. A new work "DOCNLI: A Large-scale Dataset for Document-level Natural Language Inference" to be appeared in ACL 2021 presenting the study for document/paragraph level NLI:
https://arxiv.org/abs/2106.09449v1
In Github repo you can find data and pretrained weights of RoBERTa:
https://github.com/salesforce/DocNLI
For release in HuggingFace we, probably, should wait...
P.S. I am already waiting to test this setup for fake news detection🙃
Natural Language Inference (NLI) is the task of determining whether a “hypothesis” is true (entailment), false (contradiction), or undetermined (neutral) given a “premise”.
Previously, this task was solved for sentence-level texts. A new work "DOCNLI: A Large-scale Dataset for Document-level Natural Language Inference" to be appeared in ACL 2021 presenting the study for document/paragraph level NLI:
https://arxiv.org/abs/2106.09449v1
In Github repo you can find data and pretrained weights of RoBERTa:
https://github.com/salesforce/DocNLI
For release in HuggingFace we, probably, should wait...
P.S. I am already waiting to test this setup for fake news detection🙃
Article on how to use #XGBoost for #timeseries forcasting
Link: https://machinelearningmastery.com/xgboost-for-time-series-forecasting/
Link: https://machinelearningmastery.com/xgboost-for-time-series-forecasting/
👍2
Forwarded from Denis Sexy IT 🇬🇧
Recently I have found an Instagram of artist from Tomsk, Evgeny Schwenk – he redraws characters from Soviet cartoons as if they were real people. I have applied neural.love neural network which made his drawings even more realistic. Just a bit of Photoshop (mainly for hats) and here we go.
I guess Karlsson-on-the-Roof is my best result.
I guess Karlsson-on-the-Roof is my best result.
👍2
RL + NLP + Minecraft = Awesomeness
The video from Data Fest Online 2021 about IGLU Competition which was accepted at competition track of NeurIPS 2021
Link: https://youtu.be/mbDY8uxk9bs
The video from Data Fest Online 2021 about IGLU Competition which was accepted at competition track of NeurIPS 2021
Link: https://youtu.be/mbDY8uxk9bs
YouTube
Data Fest Online 2021 | IGLU Competition @ NeurIPS 2021
Data Fest Online 2021 https://fest.ai/2021/
RL + Catalyst track https://ods.ai/tracks/catalyst-and-rl-df2021
RL + Catalyst track https://ods.ai/tracks/catalyst-and-rl-df2021
New Coding Assistant Tool From OpenAI and Microsoft
Github announced new tool for improving coding experience: Github's copilot, developed with Microsoft and OpenAI's help. This looks really promosing, at least from the announce perspective: imaging just typing convert_datetime_to_date and getting function for that. Looking forward to the actual demo.
Project: https://copilot.github.com
Blog entry: https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/
CNBC news post: https://www.cnbc.com/2021/06/29/microsoft-github-copilot-ai-offers-coding-suggestions.html
#OpenAI #microsoft #coding #CS #computerlanguageunderstanding #CLU #Github
Github announced new tool for improving coding experience: Github's copilot, developed with Microsoft and OpenAI's help. This looks really promosing, at least from the announce perspective: imaging just typing convert_datetime_to_date and getting function for that. Looking forward to the actual demo.
Project: https://copilot.github.com
Blog entry: https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/
CNBC news post: https://www.cnbc.com/2021/06/29/microsoft-github-copilot-ai-offers-coding-suggestions.html
#OpenAI #microsoft #coding #CS #computerlanguageunderstanding #CLU #Github