Reinforcement Learning
Train a reinforcement learning agent in your browser and watch it learn: Q-learning on FrozenLake, policy gradients on CartPole.
About this page
Reinforcement learning trains an agent by trial and error: it acts in an environment, collects a reward, and changes what it does next. On FrozenLake, a grid of ice and holes, tabular Q-learning fills in a table of action values, drawn as arrows that turn towards the goal as it learns. You choose the 4×4 or 8×8 map, slippery or not, and set the learning rate α, the discount γ, the exploration rate ε and its decay, the number of episodes and the seed. On CartPole, REINFORCE and Actor-Critic learn to balance a pole on a moving cart, and Run both trains the two policy gradient methods one after the other on the same seed.
Both environments are transcribed from Gymnasium, and the results are checked against references rather than against a rising curve: the Q-table is compared with value iteration, which solves the grid exactly, and the CartPole physics matches a trajectory generated by Gymnasium itself. Compare runs across seeds rather than trusting one. The 8×8 grid needs about 20 000 episodes, and roughly one seed in three never finds its goal.
Nothing is downloaded: there is no pretrained policy, and the agent starts from zero. Training runs on the CPU in a Web Worker, on purpose. A Q-learning step is an array lookup, about 15 million a second, and at batch size one a round trip to the GPU costs more than the arithmetic it would do. Nothing leaves the tab. Decision Transformer and reinforcement learning for language models are explained on the page, but neither can run here.