Tesla p100 graphics processing unit

The proposed network was trained in an end-to-end fashion to optimize the loss function; the convolution layers’ weights were initialized with a zero-mean Gaussian distribution, with a standard deviation of 2/m, where $m=f^2\times n_f$ , f is a filter size, and $n_f$ is the number of filters; this initialization can relieve diminishing gradients and improve the convergence of deep network architectures²⁷ .
The discriminators’ learning rate ${\gamma }_D$ was set to ${10}^{-5}$ equally for $D_{\text{HR}}$ and $D_{\text{LR}}$ , while the learning rate for the generators G and F was set to ${\gamma }_G={\gamma }_D/2$ , following the Two Times Update Rule (TTUR)²⁸ , to improve GAN convergence under mild assumptions. Dropout regularization layers, applied in the generators, were initialized with the rate $p_{\text{Dropout}}=0.8$ . Leaky ReLU layers were initialized with the negative slope coefficient $\alpha =0.1$ . The loss weights ${\lambda }_1$ , ${\lambda }_2$ , and ${\lambda }_3$ were set to 1, 0.5 and 0.00001, respectively.
The training used the Adam optimizer with exponential decay rates of ${\beta }_1=0.5$ and ${\beta }_2=0.9$ during 100 epochs with batches of 16 images. On average, the training took 9-11 hours per iteration, using TensorFlow (version 2.3.0) on a shared HPC workspace with an Nvidia Tesla P100 Graphics Processing Unit (GPU). The implemented code is available under the GNU license on https://github.com/erickcfarias/SR-CIRCLE-GAN.