Training Language Models via Neural Cellular Automata: marimo demo

Interactive marimo notebook unpacking how NCA pre-pre-training improves language-model reasoning and transfer.

marimo research llms github

What it is

This is an interactive marimo notebook based on the paper Training Language Models via Neural Cellular Automata. It turns the paper into a hands-on walkthrough with visual explanations, sliders, and experiments.

The notebook focuses on how NCA-generated synthetic sequences can improve downstream language-model transfer and reasoning efficiency before standard language pre-training.

Where to access it

Interactive notebook: rkaushik29.github.io/alphaxiv-marimo-notebook-comp/
GitHub repo: github.com/rkaushik29/alphaxiv-marimo-notebook-comp
Paper: arxiv.org/abs/2603.10055

How to run locally

git clone https://github.com/rkaushik29/alphaxiv-marimo-notebook-comp.git
cd alphaxiv-marimo-notebook-comp
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install marimo numpy
marimo run notebook.py

For notebook editing mode:

marimo edit notebook.py

Inside the demo

Side-by-side comparison of Game of Life and Neural Cellular Automata dynamics
Interactive complexity explorer using gzip-based trajectory scoring
Transfer-learning intuition for why NCA pre-pre-training can help language models