An earlier version of this page described this research under the name Hyperion. The lineage, per the repo's own kept-visible history: the system layer was originally called RAIN-Net, and the compositional kernel came from Hyperion, a sibling research project; the fused work is what's now public as RAIN. This page has been revised to match exactly what the public repo measures and claims today, because corrections you can audit beat clean stories you have to trust.
What it is
RAIN is a small, fast, transparent system that learns the rules of a language or puzzle from training examples, then applies those rules to brand-new cases it has never seen before.
Think of it like watching someone solve three crossword puzzles, then being able to solve every crossword of that type forever after, without practicing thousands more, without a giant brain, and without forgetting.
Researchers measure this ability, called compositional generalization, with standardized public benchmarks: does a system apply learned rules to new combinations, or does it just memorize patterns? Those benchmarks are where large language models have historically struggled, and they're where RAIN is measured.
The results
These numbers come from running the code in the open-source repo against the official public datasets. The baselines in the right column aren't quoted from papers, they were measured on the same test splits, by the same harness, and the raw result files ship in the repo.
| Benchmark | What it measures | RAIN | Measured on the same test |
|---|---|---|---|
| SCAN (addprim_jump, 7,706 held-out) | Novel combinations of simple commands | 100% | GPT-5.5: 86% · Claude Sonnet 5: 72% · from-scratch transformer: 0% |
| COGS (generalization set, 21,000 cases) | English-like sentences never seen before | 99.75% | GPT-5.5: 46% · Claude Sonnet 5: 36% · Llama 3.2 3B (a small local model): 0% |
| PCFG SET (paired, 1,000 cases) | Nested string-edit operations | 100% | Transformer with 15.7 million parameters: 0.5% |
The part I'd underline: the frontier-model numbers aren't strawmen. GPT-5.5 and Claude Sonnet 5 were measured on the same held-out splits through the same harness. They're good, 72 to 86% on SCAN, and they still aren't exact. RAIN's fit runs in seconds on a CPU and its answers are deterministic: same input, same output, every time, with the rule it applied visible. Repo's here: github.com/NORTHTEKDevs/rain.
Why this might matter
I hold two framings on this at once.
The conservative read: RAIN is a strong, fully reproducible baseline on well-known compositional-generalization benchmarks, with same-split measured comparisons instead of quoted ones. It runs in seconds, and the code is auditable. On its own, that's a research-grade result.
The more ambitious read: it's evidence for a hypothesis I've been chasing, that this kind of rule-based reasoning might not need giant neural networks at all. It might just need the problem represented the right structured way, plus systematic search over candidate rules. If that holds on harder problems (still open), it points toward a different kind of AI for rule-bound work: small, transparent, energy-efficient, and exact on the cases it handles.
And the honest middle, which the repo states up front: published prompting techniques can get frontier models to roughly 99% on some of these splits (cited in the repo, not reproduced there). So the differentiator isn't "AI can't do this." It's that RAIN does it exactly, deterministically, auditably, and for pennies of compute, where a large model does it approximately, probabilistically, and opaquely. Whether that difference matters depends on your use case. For anything where "roughly right" isn't good enough, it matters a lot.
What it isn't
I'd rather you trust this than be impressed by it, so here's what RAIN cannot do:
- It's not a ChatGPT replacement. You can't have a conversation with it, and it can't summarize a PDF or write you a poem.
- It only works in domains with a learnable rule structure. Formal languages, structured puzzles, simplified grammars. Not real internet text, not arbitrary natural language.
- The COGS learner knows the task's shape. It encodes the benchmark's role ontology and construction signatures, a task-specific inductive bias the repo discloses everywhere the number appears. What it learns from data is the lexicon and the parameter values. This isn't "discover the language from scratch."
- It won't scale to general natural language. The mechanism depends on surface regularity in the inputs, and real internet text isn't regular like that.
How it works, in plain English
The one-sentence version the repo uses: learned selection plus exact symbolic composition. A tiny learned component (in one case, 65 parameters, small enough to print on a page) decides which rule applies where. The actual reasoning, combining those rules into an answer, is done by exact algebra, not a neural network. That's why the answers are exact and repeatable: the part that composes the answer literally cannot be approximately right.
For each benchmark, RAIN pulls the underlying rule out of the training examples, then applies it to test inputs it has never seen. On SCAN, that's algebra over high-dimensional vectors that store and retrieve role-filler pairs exactly. On COGS, a template learner reads off the sentence's structure and fills in learned mappings. Same pattern underneath everywhere: a structured hypothesis space, systematic search, and parameters induced from training data only.
Checked against cheating
Every headline number ships with a negative control: shuffled-target and label-flip checks that break the training signal on purpose and require the evaluation to notice. The repo's own stated rule is that an evaluation that cannot fail is treated as broken. And every claim traces back to a raw result file in the repo, produced by a checked-in script, so you can follow any number on this page down to the run that generated it.
How to verify the numbers yourself
Don't take any AI claim on faith, mine included. The repo's README walks through it, and it comes down to:
git clone https://github.com/NORTHTEKDevs/rain.git && cd rain
python -m venv .venv && source .venv/bin/activate
pip install -e ".[raincg,dev]"
python scripts/download_data.py
python -m pytest raincg/tests/ -q
python -m raincg.bench.results_table --out raincg/RESULTS.mdThe datasets download from their official public sources, the tests assert the accuracy thresholds, and the results table regenerates from the raw result files. If it passes on your machine, the numbers are real.
What this is part of
RAIN is the research arm of Northtek, my broader work on alternative AI architectures. The production stack that pays the bills uses standard tools where standard tools are the right answer. The research is where I get to ask: is there a smaller, more transparent way to do what large language models do, in the cases where their downsides actually matter?
Right now, for these benchmarks, the answer is yes, with the caveats stated plainly above. The code is Apache-2.0 licensed. Fork it, use it, build on it, cite it. Issues and pull requests welcome.
See the code
The full repo is on GitHub. Clone it, run the tests, audit the work.
github.com/NORTHTEKDevs/rain →Built by Kristian Baer / Northtek. Research code, Apache-2.0 license.