What this project is
A Double DQN implementation and evaluation pipeline for analysing value overestimation and training stability.
Why it matters
Many value-based methods can look stable while quietly accumulating biased estimates. Double DQN offers a direct way to reduce this bias.
Architecture overview
- Replay buffer and target network pipeline.
- Double DQN target calculation.
- Training/evaluation split with fixed seeds.
- Monitoring of value estimates and policy performance.
Current status
- Baseline training loop and replay handling implemented.
- Initial comparisons against vanilla DQN complete.
- Experiment tracking templates established.
Selected results
- Reduced overestimation relative to vanilla baseline in early experiments.
Roadmap
- Expand ablations over replay and target-update frequency.
- Improve run-to-run variance analysis.