Antes de empezar, da click en el boton “Open in Colab” de arriba. Esto abrirá este notebook de python directamente en Colab. Luego, para poder editar y ejecutar el código, deberas copiar el notebook en Google drive utilizando el boton:

De esta manera, este notebook quedará almacenado en Google drive, en la carpeta Colab Notebooks

Introducción

HDSP Laboratory: Deep Learning for inverse problem

This laboratory is based on the paper Deep Learning Techniques for Inverse Problems in Imaging.

1. Before starting, please read the document

Inverse problem:

It is the task of reconstructing an unknown signal $x$ (image, voice, multidimensional data), from projected observation $y$. This observation can be modeled with a forward model as \begin{equation} y = A(x) + \epsilon
\end{equation} where $A(\cdot)$ is usually non-invertible and ill-posed. Therefore, a unique solution that fits the observations is difficult or impossible without some prior knowledge about the data.

Exercise 1:

Example Inverse problems (Super-resolution):

The super-resolution problem is an inverse problem where the measurements $y$ is a downsampling version of an image $x$. This problem can be expressed as a linear system where \begin{equation} y = Ax
\end{equation}

A visual example of this problem is illustrated as follows.

Figure 1: (left) Low-resolution images $y$ (right) High resolution images $x$

Assuming a resolution factor of $4$, i.e., $4\times 4$ pixel in the original images are the average of a pixel in the measurements, read an image a made the decimation matrix $A$ to obtain $y$, using $y=Ax$

# Read a images (preference a gray-sacle)
from google.colab import files  
files.upload()

import numpy as np
import cv2 
import matplotlib.pyplot as plt

img = cv2.imread('name_image.png') # we can visualize the image
plt.imshow(img)

#--- we need to vectorize the image x

#Lines for vectorize the image  
x = 

# --- We need to build the matrix A  (recomentation: We can use np.kron)

# Lines for bould the linear matrix 
 A =

# ----- sensing model 

y = np.multiply(A,x)

#-------- reshape y as a matrix

# Lines for reshape y 
Y_matrix = 

plt.imshow(Y_matrix)

Inverse problem examples

With the growth of available data, deep learning has helped improve the performance of these problems. An example of these is the following.

classification of Inverse Problem

The majority of the deep inverse problem can be generalices in the following table

Forward Model Fully Known During Training and Testing

when the degradation model is known, for example $A$, in the super-resoution model, an initial estimate can be calculated to return to the image dimensions

$$\tilde{x}=A^Ty$$

and this can be used as input of a model

Dowload the MNIST DATASET

import tensorflow as tf

mnist = tf.keras.datasets.mnist
(x_train, x_labels), (x_test, y_labels) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 0s 0us/step

Build a decimation matrix with a supe-resolution factor of 2, for the MNIST dataset, i.e., the low-resolution image result of $14\times 14$

do the following operations:

1. reshape de data
2. y_train = A*x_train
3. x_estimate = A*y_train
4. reshape de x_estimate as a image

y_train = 
x_estimate = 

1. check that spatial dimensions of $x_estimate$ are $28x28$
2. Build a convolutional model, where the input and the output received an image of $28\times 28$

model = 

Now your model must be training using this estimate as input data and the original image as output data.

model.compile(optimizer=optimizad, loss='mean_squared_error',metrics=['mse','mae'])
      
history = model.fit(x_estimate, x_train, epochs=100, batch_size=FLAGS.batch_size)

Evaluate your model with x_test data, (Notice that you need to do the same process, i.e, obtain the initialization)

y_test = 
x_estimate_test = 

x_reconstructed = model.predict(x_estimate_test)

# calculate error metrics
MSE = 
MAE = 

well done, you just created your first model to solve an inverse problem

Now you can improve your model using a network with skip connections

model = 

Questions

1. What is a inverse problem?
2. What kind of reverse problem was analyzed in the laboratory?
3. Why the need to use Deep learning in this problem?
4. How can you improve the performance of your model?