Model-free in situ training of diffractive optical processors via proximal policy optimization

We introduce a model-free in situ training framework for diffractive optical processors based on Proximal Policy Optimization (PPO), a reinforcement learning algorithm well-suited for optimization under noisy and uncertain conditions. PPO enables more stable and efficient updates by reusing experimental data and enforcing conservative policy changes during each iteration. We experimentally validated this approach across a range of optical tasks, including energy focusing through unknown scattering media, holographic image reconstruction, aberration correction, and optical image classification. In all cases, PPO consistently achieved faster convergence and superior performance compared to conventional approaches. Our strategy operates directly on the physical system, inherently accounting for unknown distortions, misalignments and variability without requiring prior knowledge of these factors or calibration steps. This makes it especially well-suited for complex optical setups with feedback-driven dynamics. As such, our framework offers a scalable and general solution for training a broad class of analog optical/physical computing systems.