Meet the new ALVI Interface: a breakthrough in intuitive prosthetic control.

This technology offers individuals with hand differences a new movement experience:
✨ Wrist rotation.
🖐 Finger movement.
🕹 Interaction with objects in VR.

Discover how we're turning futuristic dreams into today's reality. Be among the first to step into this new era of possibilities.

Our demo:
https://youtu.be/Dx_6Id2clZ0?si=jF9pX3u7tSiKobM5
Twitter/X

P. S. We are going to build a hand prosthesis powered by ALVI Interface. We need your support to do it.

How to adapt MAE for multimodal data generation?

Background.
We know that MAE is very good approach to learn good features. It can be used for several modalities. For example, it outperforms MAE with trained on pairs text and image via transferable features: M3AE

Also there is MaskGit model which uses MAE for generation tasks. Shortly speaking it unmasks image step by step (aka diffusion but without diffusion).
And deep mind published paper: Muse(3 Jan 2023) in which they adapt mask git for text to image.

Very interesting idea 💡

Few words about my research)
I'm developing a prosthesis control system at ALVI Labs. My goal is to decode hand motions from muscle activity(EMG).

So what if these approaches merge?
In my opinion using multimodal MAE for EMG and Motions concurrent prediction might be ... well promising.

How?
1) Just use M3AE pipeline and actually get better MAE features for Muscle.
2) Add MaskGit generation pipeline.

Expected results
Universal model which can unmask/generate 2 modalities.
- EMG -> Motion
- Motion -> EMG
- Better EMG features.

What do you think?☺️

P.S. In theory, it might be used for any data: ECoG, Spikes, EEG, fMRI and so on.

Recent advances in neural population decoding

In this recent paper (NeurIPS) the researchers adapted transformers for flexible decoding of large-scale spiking populations. The model POYO-1 was trained across sessions and participants (non-human primates), thus being effectively a good pre-trained model for whichever decoding paradigm you might want to use. Transfer learning across sessions and participants is enabled in a very "DL engineer" manner: let's just create an embedding of the session and process it along-side with spike embeddings - it seems to work well.

Unfortunately they did not release the code yet, but in the meantime you can check the website of a project.

One of the potential limitations is data was collected from M1, PMd and S1, which was shown to contain the task-relevant information, usually in the population vector coding format. Does the pre-trained transformer generalize to, for example, auditory cortex to decode imagined speech, remains unclear.

For me, seems like a hype paper, cannot wait until the code/weights are released to try it out. Let us know in the comments, which pitfalls you notice - it always feels good to debunk crowd decision making results 🙃

***

If you're rather interested in applying a model to any kind of data to infer low-dimensional trajectories representing the behavior check out CEBRA (paper). They allow you to use any kind of time series data (calcium imaging, spikes, voltages) along side with behavior. They also enable transfer learning and generalization across participants by explicitly encoding trial_ID, session_ID, subject_ID for the model.

👉 Here is the website: https://cebra.ai/

A collegue from Max Planck Institute in Tübingen talks about his foundational model of a cortical neuron, able to capture rich dynamics of real neurons.

https://twitter.com/OpenNeuroMorph/status/1762523010895081807?s=19

https://arxiv.org/abs/2306.16922

Figure x OpenAI 2024

This status update just blows my mind. Really amazing progress. Especially the level of hand dexterity.

Figure combines a vision language model from openAI with a control system.

They did it in 13 days after signing a partnership🔥

For more details: X post

Grok 1 by X. 314B MoE.

Model Details
- Base model trained on a large amount of text data, not fine-tuned for any particular task.
- 314B parameter Mixture-of-Experts model with 25% of the weights active on a given token.
- Trained from scratch by xAI using a custom training stack on top of JAX and Rust in October 2023.

Link: https://x.ai/blog/grok-os

Brain-To-Text Competition 2024

This is the most fascinating BCI competition yet, organized by Stanford. Everyone has one month to develop the world's best brain-to-speech decoder!

Task: Predict attempted speech from brain activity.

Deadline: June 2, 2024

Dataset: They've recorded 12,100 sentences from a patient who can no longer speak intelligibly due to amyotrophic lateral sclerosis (ALS).

For each sentence, we provide the transcript of what the participant was attempting to say, along with the corresponding time series of neural spiking activity recorded from 256 microelectrodes in speech-related areas of cortex.

Just letting you know we're jumping into this challenge!
Together with @Altime, @kovalev_alvi and the team of ALVI Labs, we're going to create something interesting.

Like this post if you want to follow our updates❤️