fromziro/MrPong

Mr. Pong

In the realm of Ping Pong, Mr. Pong is no ordinary paddle. Feared across the table as the Blue Beast, he commands every rally with ruthless precision, striking despair into the hearts of all who dare face him.

In other words, he is a reinforcement learning (RL) agent trained to compete at a grandmaster level in Ping Pong.

Model Architecture

• Base Architecture: MrPongMLPForRL
• Hidden Size: 160
• Observation Size: 16
• Number Of Layers: 3
• Action Space: 3 (stay, up, down)
• Activation: tanh

Mr. Pong uses a shared Actor-Critic MLP designed for 2D table tennis control. The 16-dimensional observation vector is passed through two hidden layers of 192 units each, with Tanh activations between them. The resulting 192-dimensional representation is shared by both the actor and critic heads. The actor outputs categorical logits across the 3 movement actions (stay, up, down), while the critic uses the same representation to estimate the scalar state value V(s). This shared setup keeps the network small and fast on CPU while letting the model combine ball coordinates, velocities, paddle momentum, raycasted intercept points, and opponent court openings into a single control decision.

Training Configuration

• Number of Environments: 12
• Rollout Steps: `128
• PPO Epochs: 4
• Minibatch Size: 64
• Gamma: 0.99
• GAE Lambda: 0.95
• Clip Coefficient: 0.20
• Value Coefficient: 0.50
• Entropy Coefficient: 0.02
• Maximum Gradient Norm: 0.75
• Learning Rate: 3.5e-4
• Adam Epsilon: 1e-5
• Anneal Learning Rate: true
• Total Timesteps: 10,000,000
• Maximum Rally Steps: 1500

Training Opponents

• Realistic Hard Logic (human-like perception horizon and bounce raycasting)
• Medium Logic (linear trajectory extrapolation)
• Minimax Lookahead (depth 1 and depth 2)
• Current Self-Play (mirror matches against the active policy)
• Historical Self-Play (past checkpoints sampled between 5 and 75 saves ago)
• Easy Logic (reaction delay with targeting noise)
• Impossible Hard Logic (0ms reaction time)
• Random Agent

Training Results

Opponent Name	Win %	Draw %	Loss %	Record (W / D / L)	Avg Rally
Easy Logic	99.7%	0.0%	0.3%	997W / 0D / 3L	1.7 hits
Medium Logic	99.0%	0.2%	0.8%	990W / 2D / 8L	17.3 hits
Realistic Hard	86.6%	8.1%	5.3%	866W / 81D / 53L	33.9 hits
Impossible Hard	0.0%	98.4%	1.6%	0W / 984D / 16L	64.8 hits
Minimax Depth 1	43.0%	55.8%	1.2%	430W / 558D / 12L	32.2 hits
Minimax Depth 2	78.4%	20.3%	1.3%	784W / 203D / 13L	23.7 hits
Random Agent	100.0%	0.0%	0.0%	1000W / 0D / 0L	0.9 hits
Self-Play Mirror	1.6%	96.1%	2.3%	16W / 961D / 23L	50.5 hits

Mr. Pong has demonstrated the he has mastered the game Ping Pong; losing only <=5% of games.

Inference

First install the required dependencies:

pip install torch transformers

Next, download inference.py and run:

python inference.py

• Use --mode play to play against the agent live in the terminal.
• Use --mode simulate to run AI match simulations (default).
• Add --opponent to choose a specific opponent (realistic_hard, medium, minimax_d1, minimax_d2, impossible_hard, easy, random).
• Add --video to export an MP4 recording of the match.

License

Apache 2.0.

Citation

@misc{mrbalance,
  title        = {Mr. Pong: Teaching RL agents to Play Ping Pong},
  organization = {FromZero},
  authors      = {Paul Courneya},
  year         = {2026},
  url          = {https://huggingface.co/fromziro/MrBalance]
}