Судя по https://algorithmsbook.com/#outline — очень крутая книга.
#machinelearning #reinforcementlearning #multiagent #uncertainty #optimization #julia

SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allocation with local explanations using the classic Shapley values from game theory and their related extensions:
https://github.com/slundberg/shap#citations

#machinelearning #explainableai

https://econml.azurewebsites.net/spec/spec.html
The EconML Python SDK, developed by the ALICE team at MSR New England, incorporates individual machine learning steps into interpretable causal models.

Pretty cool. Docs feature introduction into the topic and the methods.

#datascience #causalinference #machinelearning

#machinelearning for 4-graders (~10 years old)
https://orangedatamining.com/blog/2022/2022-06-01-blog-minions-new/

Most important points IMO:
- Single simple task: classification with decision trees
- Guide pupils to invent the method themselves on simplified visual and familiar synthetic data
- Show automation on data pupils collected themselves, manually retrace the generated tree
- Discuss issues with data and problems they generate down the line

https://mlss.cc/

#mlss #machinelearning summer schools — all over the world. Some lectures #video available.

https://arxiv.org/abs/2210.16294
"Learning Modular Simulations for Homogeneous Systems" by Jayesh K. Gupta, Sai Vemprala, Ashish Kapoor

A pretty cool paper from Microsoft Research merging Neural ODEs with Graph Neural Networks ideas. It presents data-driven continuous-time dynamical system identification problem as learning a system of NeuralODE nodes connected with links forming a graph, and using (synthetic) message-passing along the graph edges. Thus the MP-NODE name of the technique.

OK, what does all of that mean?
Most of physical (continuous-time) systems are modelled as differential equations, and we all know that knowing either initial conditions or boundary conditions we can (numerically) solve the equation obtaining a function that tells us the state of the system at any point in time and space. But to do so we need to know precisely the function governing system's evolution. For most real-world systems we can't know it, and instead we have a series of measurements of the system's state at different times. So we have a data-driven system identification problem.

NeuralODEs present one approach to this problem, approximating unknown evolution function with a Neural Network so we can train the network in a way that ODE solution becomes as close to the data at hand as possible. And we need to go deeper! 😊

Usually real-world systems get composed from smaller systems, that can be described with differential equations, connected together. Thus it would be pretty natural (and hopefully computationally more efficient) to model them the same way: as a number of (smaller) equations somehow connected and synchronized.

Enter MP-NODE: the authors propose augmenting NeuralODE's state vectors with a "message" part, account for connections between different nodes, and learn the whole thing at once from the data.

The caveat is all the nodes in the system are governed by exactly the same NeuralODE, thus the "Homogeneous" part of the title. But such (pretty) homogeneous systems are all around us: for instance water supply system consists of almost identical pumping stations connected into a single network.

The results are pretty impressive and promising! The MP-NODE models learned from smaller systems (in terms of the number of nodes in a graph) can be easily and efficiently fine-tuned to larger systems. And if we keep the size of the graph but change the topology (connectivity) the model performs surprisingly accurate even without any additional training at all!

Also the "ODEs connected with links in a graph" idea reminds me about https://github.com/AlgebraicJulia/AlgebraicDynamics.jl from the Topos Institute. Though their approach has nothing to do with Machine Learning. But for some reason they chose Julia anyway too! 😃

#julia #machinelearning #ode

https://statmodeling.stat.columbia.edu/2023/01/03/explanation-and-reproducibility-in-data-driven-science-new-course/

WOW, a great reading list on #statistics and #machinelearning ! And an important topic for a course. Especially targeting CS students.

ДМК Пресс выпускает перевод книги "Algorithms for Decision Making".
Оригинал доступен для скачивания бесплатно по второй ссылке. Там же можно глянуть оглавление — оно внушает уважение. 😊

#free #book #algorithms #machinelearning #reinforcementlearning

https://fleuret.org/francois/lbdl.html
The Little Book of Deep Learning

A nice concise #free #book covering the fundamentals of #machinelearning and deep architectures, including skip connections, dropout, batch normalisation, attention, autoencoders, transformers and computer vision applications (and some other things).

Free PDFs come in two forms: mobile phone format, which is a bit annoying on a PC, and ready-to-print booklet format, which is also annoying to read on a PC. I guess they nudge you to buy a physical book, which is fair enough.

There's also a lecture course which discusses mostly the same topics, but to a greater depth, I guess. Video recordings, slides, handouts, even a virtual machine with everything installed — all available for free. François Fleuret did a great job here, for real.

https://www.stochasticlifestyle.com/chatgpt-performs-better-on-julia-than-python-and-r-for-large-language-model-llm-code-generation-why/

Evidently ChatGPT 3.5 "understands" #Julia significantly better than other languages including Python and JavaScript, not even mentioning Go, C and C++: https://arxiv.org/abs/2308.04477

Chris Rackauckas gives some points as to why it's not that surprising from the perspective of teaching novices. And also speculates that large volume of so-so tutorials and examples for very popular languages might hurt LLMs' learning.

He also advertises his diffeqpy library connecting optimized solvers implemented in Julia (including code generation for GPUs) to #Python #machinelearning libraries, and points to some curious papers with impressive benchmarks. 😊

Again on my favorite topic of correlation, causation and control systems.

"When causation does not imply correlation" presents pretty technical analysis (with a couple of theorems) of conditions when a control system "breaks the circuit" and decorrelates variables:
https://arxiv.org/abs/1505.03118

I wrestled with this issue when I did a control system parameters' identification from sensor data using machine learning. Judging from data some actions had no effect because they were kicking in precisely in order to counteract another force and keep the readings the same.

Then the "Slime Mold" guys rediscovered this effect, and they provide nice, approachable illustrations:
https://slimemoldtimemold.com/2022/03/15/control-and-correlation/

More comments from Gelman's blog including long historic roots of this observation:
https://statmodeling.stat.columbia.edu/2024/01/15/a-feedback-loop-can-destroy-correlation-this-idea-comes-up-in-many-places/

#statistics #machinelearning

https://causalml-book.org/

#free #book on #causalinference and CausalML #machinelearning with a lot of Google Colab Notebooks in Python and R using EconML and DoubleML packages.