chore: update docs with recent trained weights (#12)

* chore: update docs with recent trained weights

* adjust table

* test align center

* center & adjust

* remove unnecessary toc items

* only allow gauss filter if transmission map is used

* still default to false

* try fix for `gauss_blur_model` model

* revert and fix gauss

* also add legacy weights

* center last table

* adjust epochs

* adjustments

* revert unnecessary changes

* clear all outputs

* fill comparison table and adjust links

* flipped images

README.md

@@ -1,3 +1,6 @@
+> [!NOTE]  
+> November 2024: New Pre-trained Models are available, check the [Pre-trained Models](#pre-trained-models) section.
+
 # Foggy-CycleGAN
 
 <p align="center">
@@ -9,37 +12,71 @@ This project is the implementation for my Computer Science MSc thesis in the Uni
 Dissertation: 
 <a href="./dissertation/Simulating%20Weather%20Conditions%20on%20Digital%20Images%20-%20Final.pdf" target="_blank">[PDF] Simulating Weather Conditions on Digital Images</a> (Debrecen, 2020).
 
+# Table of Content
+- [Description](#description)
+- [Code](#code)
+- [Notebook](#notebook)
+- [Results](#results)
+- [Pre-trained Models](#pre-trained-models)
+
 ## Description
 **Foggy-CycleGAN** is a
 <a href="https://junyanz.github.io/CycleGAN/" target="_blank">CycleGAN</a> model trained to synthesize fog on clear images. More details in the dissertation above.
 
 ## Code
 The full source code is available under GPL-3.0 License in my Github repository <a href="https://github.com/ghaiszaher/Foggy-CycleGAN" target="_blank">ghaiszaher/Foggy-CycleGAN</a>
 
-## Pre-trained Models
-A version of pre-trained models used in the thesis can be found [here](https://drive.google.com/drive/folders/1QKsiaGkMFvtGcp072IG57MfY1o_D-L3k?usp=sharing).
-
 ## Notebook <a href="https://colab.research.google.com/github/ghaiszaher/Foggy-CycleGAN/blob/master/Foggy_CycleGAN.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
 A Jupyter Notebook file <a href="https://github.com/ghaiszaher/Foggy-CycleGAN/blob/master/Foggy_CycleGAN.ipynb" target="_blank">Foggy_CycleGAN.ipynb</a> is available in the repository.
-
 
 ## Results
+(as of June 2020)
 <p align="center">
- <img src="images/result-animated-01.gif">
+ <img src="images/results/2020-06/result-animated-01.gif">
 </p>
 
 <p align="center">
- <img src="images/result-sample-0.2.jpg">
+ <img src="images/results/2020-06/result-sample-0.2.jpg">
 </p>
 
 <p align="center">
- <img src="images/result-sample-0.3.jpg">
+ <img src="images/results/2020-06/result-sample-0.3.jpg">
 </p>
 
 <p align="center">
- <img src="images/result-sample-0.25.jpg">
+ <img src="images/results/2020-06/result-sample-0.25.jpg">
 </p>
 
-<div align="right">
+<div align="center">
 &copy; Ghais Zaher 2020
 </div>
+
+## Pre-trained Models
+As previous pre-trained models are no longer compatible with newer Keras/Tensorflow versions, I have retrained the model and made the new weights available to download.
+
+Each of the following models was trained in Google Colab using the same dataset, the parameters for building the models and number of trained epochs are a bit different:
+<div align="center">
+
+| Model                                                                                                       | Trained Epochs | Config                                                                              |
+|-------------------------------------------------------------------------------------------------------------|----------------|-------------------------------------------------------------------------------------|
+| [2020-06 (legacy)](https://drive.google.com/drive/folders/1QKsiaGkMFvtGcp072IG57MfY1o_D-L3k?usp=sharing)    | 145            | `use_transmission_map=False`<br>`use_gauss_filter=False`<br>`use_resize_conv=False` |
+| [2024-11-17-rev1-000](https://drive.google.com/drive/folders/1--W53NNrVxS5pvrf8jDKCRmg4h4vD5lx?usp=sharing) | 522            | `use_transmission_map=False`<br>`use_gauss_filter=False`<br>`use_resize_conv=False` |
+| [2024-11-17-rev2-110](https://drive.google.com/drive/folders/1rQ7jmsv63uv6v45IVZmZ8w9CVktqJAfn?usp=sharing) | 100            | `use_transmission_map=True`<br>`use_gauss_filter=True`<br>`use_resize_conv=False`   |
+| [2024-11-17-rev3-111](https://drive.google.com/drive/folders/1-0-z7KTMXTrwwUdeJtkUOBCWkwD6behO?usp=sharing) | 103            | `use_transmission_map=True`<br>`use_gauss_filter=True`<br>`use_resize_conv=True`    |
+| [2024-11-17-rev4-001](https://drive.google.com/drive/folders/1hDxJtU0agbnPO2XrrPo26RQJKOePa6WX?usp=sharing) | 39             | `use_transmission_map=False`<br>`use_gauss_filter=False`<br>`use_resize_conv=True`  |
+
+</div>
+
+### Results
+The results of the new models are similar to the previous ones, here are some samples:
+<div align="center">
+
+| Clear                                                   | 2024-11-17-rev1-000                                        | 2024-11-17-rev2-110                                        | 2024-11-17-rev3-111                                        | 2024-11-17-rev4-001                                        |
+|---------------------------------------------------------|------------------------------------------------------------|------------------------------------------------------------|------------------------------------------------------------|------------------------------------------------------------|
+| <img src="images/results/2024-11-17/clear/sample1.jpg"> | <img src="images/results/2024-11-17/rev1-000/sample1.gif"> | <img src="images/results/2024-11-17/rev2-110/sample1.gif"> | <img src="images/results/2024-11-17/rev3-111/sample1.gif"> | <img src="images/results/2024-11-17/rev4-001/sample1.gif"> |
+| <img src="images/results/2024-11-17/clear/sample2.jpg"> | <img src="images/results/2024-11-17/rev1-000/sample2.gif"> | <img src="images/results/2024-11-17/rev2-110/sample2.gif"> | <img src="images/results/2024-11-17/rev3-111/sample2.gif"> | <img src="images/results/2024-11-17/rev4-001/sample2.gif"> |
+| <img src="images/results/2024-11-17/clear/sample3.jpg"> | <img src="images/results/2024-11-17/rev1-000/sample3.gif"> | <img src="images/results/2024-11-17/rev2-110/sample3.gif"> | <img src="images/results/2024-11-17/rev3-111/sample3.gif"> | <img src="images/results/2024-11-17/rev4-001/sample3.gif"> |
+| <img src="images/results/2024-11-17/clear/sample4.jpg"> | <img src="images/results/2024-11-17/rev1-000/sample4.gif"> | <img src="images/results/2024-11-17/rev2-110/sample4.gif"> | <img src="images/results/2024-11-17/rev3-111/sample4.gif"> | <img src="images/results/2024-11-17/rev4-001/sample4.gif"> |
+| <img src="images/results/2024-11-17/clear/sample5.jpg"> | <img src="images/results/2024-11-17/rev1-000/sample5.gif"> | <img src="images/results/2024-11-17/rev2-110/sample5.gif"> | <img src="images/results/2024-11-17/rev3-111/sample5.gif"> | <img src="images/results/2024-11-17/rev4-001/sample5.gif"> |
+
+</div>

lib/gauss.py

@@ -1,12 +1,11 @@
 def gauss_blur_model(input_shape, kernel_size=19, sigma=5, **kwargs):
     import tensorflow as tf
     import numpy as np
+
     def matlab_style_gauss2D(shape=(3, 3), sigma=0.5):
         """
         2D gaussian mask - should give the same result as MATLAB's
         fspecial('gaussian',[shape],[sigma])
-        #https://stackoverflow.com/questions/55643675/how-do-i-implement-gaussian-blurring-layer-in-keras
-        #https://stackoverflow.com/questions/17190649/how-to-obtain-a-gaussian-filter-in-python/17201686#17201686
         """
         m, n = [(ss - 1.) / 2. for ss in shape]
         y, x = np.ogrid[-m:m + 1, -n:n + 1]
@@ -18,10 +17,8 @@ def matlab_style_gauss2D(shape=(3, 3), sigma=0.5):
         return h
 
     class SymmetricPadding2D(tf.keras.layers.Layer):
-        # Source: https://stackoverflow.com/a/55210905/11394663
-        def __init__(self, output_dim, padding=(1, 1),
+        def __init__(self, padding=(1, 1),
                      data_format="channels_last", **kwargs):
-            self.output_dim = output_dim
             self.data_format = data_format
             self.padding = padding
             super(SymmetricPadding2D, self).__init__(**kwargs)
@@ -30,35 +27,49 @@ def build(self, input_shape):
             super(SymmetricPadding2D, self).build(input_shape)
 
         def call(self, inputs, **kwargs):
-            if self.data_format is "channels_last":
-                # (batch, depth, rows, cols, channels)
-                pad = [[0, 0]] + [[i, i] for i in self.padding] + [[0, 0]]
-            # elif self.data_format is "channels_first":
+            if self.data_format == "channels_last":
+                # (batch, rows, cols, channels)
+                pad = [[0, 0]] + [[p, p] for p in self.padding] + [[0, 0]]
             else:
-                # (batch, channels, depth, rows, cols)
-                pad = [[0, 0], [0, 0]] + [[i, i] for i in self.padding]
+                # (batch, channels, rows, cols)
+                pad = [[0, 0], [0, 0]] + [[p, p] for p in self.padding]
             paddings = tf.constant(pad)
-            out = tf.pad(inputs, paddings, "REFLECT")
-            return out
+            return tf.pad(inputs, paddings, "REFLECT")
 
         def compute_output_shape(self, input_shape):
-            return input_shape[0], self.output_dim
+            if self.data_format == "channels_last":
+                return (input_shape[0],
+                        input_shape[1] + 2 * self.padding[0],
+                        input_shape[2] + 2 * self.padding[1],
+                        input_shape[3])
+            else:
+                return (input_shape[0],
+                        input_shape[1],
+                        input_shape[2] + 2 * self.padding[0],
+                        input_shape[3] + 2 * self.padding[1])
 
     if kernel_size % 2 == 0:
         raise Exception("kernel size should be an odd number")
+
     gauss_inputs = tf.keras.layers.Input(shape=input_shape)
 
     kernel_weights = matlab_style_gauss2D(shape=(kernel_size, kernel_size), sigma=sigma)
     in_channels = input_shape[-1]
     kernel_weights = np.expand_dims(kernel_weights, axis=-1)
     kernel_weights = np.repeat(kernel_weights, in_channels, axis=-1)  # apply the same filter on all the input channels
     kernel_weights = np.expand_dims(kernel_weights, axis=-1)  # for shape compatibility reasons
+
     gauss_layer = tf.keras.layers.DepthwiseConv2D(kernel_size, use_bias=False, padding='valid')
     p = (kernel_size - 1) // 2
-    # noinspection PyCallingNonCallable
-    x = SymmetricPadding2D(0, padding=[p, p])(gauss_inputs)
+
+    # Apply symmetric padding
+    x = SymmetricPadding2D(padding=[p, p])(gauss_inputs)
+
+    # Ensure the input to DepthwiseConv2D has the correct shape
     x = gauss_layer(x)
-    ########################
+
+    # Set the weights for the gaussian filter
     gauss_layer.set_weights([kernel_weights])
     gauss_layer.trainable = False
+
     return tf.keras.Model(inputs=gauss_inputs, outputs=x, **kwargs)