Learning State-Tracking from Code Using Linear RNNs

We convert state-tracking benchmarks into code via REPL traces and study which sequence models can learn long-horizon state tracking.