Simon Ramstedt

AI Research & Development. PhD dropout at Mila and McGill, Master's at University of Montreal and undergrad at TU Darmstadt. Spent summers at Microsoft Research and ElementAI.

Reinforcement Learning with Random Delays Simon Ramstedt, Yann Bouteiller, G. Beltrame, Christopher Pal & Jonathan Binas [ICLR 2021] [arXiv:2010.02966] A provably lower-bias, lower-variance value estimator for environments with random action and observation delays. The estimator is used and evaluated with our new Delay-Correcting Actor-Critic algorithm.
Real-Time Reinforcement Learning Simon Ramstedt & Christopher Pal [NeurIPS 2019] [arXiv:1911.04448] A new framework for Reinforcement Learning in which states and actions evolve simultaneously. It explicitly takes into account that action selection takes time. We introduce the Real-Time Actor-Critic algorithm.

Qn 2025-now [github.com/rmst/qn] A lightweight, Node.js-compatible JavaScript runtime built on Fabrice Bellard's QuickJS and libuv.
Dependency Explorer 2026-now [simonramstedt.com/tools/dependency-explorer] An interactive dependency graph explorer for npm, pypi and nix packages. Makes it easy to inspect transitive dependencies when deciding what to rely on.
Jix 2025-now [github.com/rmst/jix] A tool enabling functional, declarative system and development configurations to be written in JavaScript. Internally, it constructs a Merkle-DAG of configuration units called "effects" (a generalization of Nix derivations). It's more powerful than YAML-based configuration tools (e.g. Ansible, docker-compose) and more ergonomic and flexible than Nix.

Robin VLM 2023 [github.com/cerc-aai/robin] Data, training code and weights for an open VLM based on Mistral-7B and SigLIP, implemented using PyTorch and DeepSpeed and trained on eight A100 GPUs on the HessianAI computing cluster.
Uniton 2021 [github.com/rmst/uniton][Demo Video] A Python package to control the Unity game engine. It automatically translates function calls from Python to Unity/C# and dispatches them asynchronously. It also coordinates the Python and Unity/C# garbage collectors.
RLRD 2020 [github.com/rmst/rlrd] Code accompanying our Reinforcement Learning with Random Delays paper with an implementation of our Delay-Correcting Actor-Critic algorithm in Pytorch.
RTRL 2019 [github.com/rmst/rtrl] Code accompanying our Real-Time Reinforcement Learning paper with implementations of Real-Time Actor-Critic and Soft Actor-Critic in Python and Pytorch.
Yoke 2018-2021 [github.com/rmst/yoke] Android app turning the phone into a gamepad for Windows/Mac/Linux. Low latency communication over Wifi via UDP.
Avenue 2017-2019 [github.com/elementai/avenue] A fast, easy-to-use, high-fidelity car simulator based on the Unity game engine.
DDPG 2016 [github.com/rmst/ddpg] The first open reproduction of the Deep Deterministic Policy Gradient algorithm by Lillicrap, et al., 2015, in both MATLAB with manually-coded gradient computation and also the (at the time) brand new Tensorflow automatic differentiation framework.

Versatile Npm-Free Web Stack 2026-04-24 · SWE Preact and Hono provide what React and Express used to offer but without dependencies. We install a web stack straight from source without npm packages.
Npm Slop & Wonky Software Supply Chains 2026-04-09 · SWE Packages on npm and pip are developer-uploaded bundles, not source. Attestation narrows the trust gap but doesn't help with building from source.
2021
Reinforcement Learning with Random Delays(ICLR 2021) 2021-05-03 · AI
Driving the Unity Game Engine from Python 2021-03-30 · SWE AI
Does OpenAI Lose Money on GPT-3 Inference? 2021-03-28 · AI
2020
Motion, Depth and Object Perception 2020-12-22 · IDEAS AI
2019
Real-Time Reinforcement Learning(NeurIPS 2019) 2019-12-08 · AI
2018
Deep Learning with Integer Activations 2018-02-10 · PAPER REVIEW AI
2016
Deep Reinforcement Learning for Continuous Control 2016-04-30 · BACHELOR THESIS AI