Hyperparameter Landscape Smoothing Algorithms For Stable Training Of Gans

Abstract

It has been demonstrated that Generative Adversarial Networks (GANs) have made tremendous progress in generating synthetic images, but still, the adversarial optimization comes with many problems such as unstable training, mode collapse, and oscillatory convergence. To achieve stable training, the researchers in this article have proposed the Hyperparameter Landscape Smoothing GAN (HLS-GAN) framework that adaptively smooths the optimization trajectory based on the hyperparameter landscapes generated by the parameters. The proposed model is able to dynamically control the variation of gradients and stabilize the generator-discriminator interaction during training. The experimental analysis was carried out with the MNIST dataset of 70,000 handwritten digit images. A set of metrics to evaluate the framework, such as Fréchet Inception Distance (FID), Inception Score (IS), convergence variance, and training stability, were used. The experimental results showed that the proposed HLS-GAN achieved the FID score of 18.7 and the Inception Score of 8.1, which is better than conventional GAN, DCGAN, and WGAN. Moreover, the proposed smoothing mechanism decreased loss variance from 0.82 to 0.23, so that it had better convergence consistency without gradient oscillation. Mode collapse events were minimized, and structure was clearer in the produced handwritten digit samples. In conclusion, the proposed Hyperparameter Landscape Smoothing framework offers a computationally stable and efficient deep learning optimization method to better control the convergence reliability and synthetic image generation quality of GANs.