Welcome back. With some of the more recent image restoration approaches. We are interested in obtaining an estimate of the posterior, not just its maximum value, as with map estimation. Coupled with a hierarchical paradigm, prior models for unknown signals are defined, that is for the noise, the original image. But also for the points function of the blurring system for the blind restoration case. And also models are defined for the parameters which define such priors. Then the posterior for all unknown parameters is four. Since its estimation is not directly feasible an approximation of it is sought after. With such an approach, the original image and poise function of the blur are estimated along with the unknown parameters, such as the noise variants, automatically based on the available data. Since, it becomes a rather complicated estimation problem. We do not show any of the algorithmic details. Instead, we show some experimental results, which represent one of the state of the art results. So, let us now proceed with the material for this segment. I would like to make a few comments based on what we have seen. In the previous few slides, we saw that the result of the maximum like Wood and MAP estimation, resulted in some nice, closed form solutions. This clearly may not be the case, and simply depends on the noise model in the image prior we use. So, depending on these choices we may have to address some challenging optimization problems which are outside the scope of what we are doing here. The second comment is that in all these derivations, we assume that the image and noise covariances. In the general case or the specific parameters alpha and beta, where assumed to be known. So, the, our techniques to try to estimate such parameters offline but in general. We need to address the estimation of this parameters, as well. Also, we have been dealing with the non-blind restoration problem, according to which the blurring function H. The impulsive part of the system, the matrix HR known ahead of time. But clearly, very often this may not be the case and therefore, one needs to address the blind restoration problem as well. This is a topic that has seen quite some activity, and there are techniques that. First, estimate the blur and then perform the restoration problem, so it's a separate and sequential estimation process. However, the last two points can also be addressed through the hierarchical Bayesian paradigm. I would like just to give you a flavor of this paradigm, just mention it, describe it at a very high level simply because we end up with challenging optimization. Approaches in many cases, and this is something, again, outside the scope of this course. However having this basic information, I believe that you'll be able, for those of you interested, to look into the literature and, and dive a bit deeper in mm, looking at some of these very interesting and exciting results. What we discuss here is an extension of the material we already covered. So we're interested in forming this joint distribution shown here. And the distribution of f the unknown original image we tried to estimate h, the unknown impulse response of the degradation system, so we are addressing blind restoration here. This set omega which incorporates all the unknown parameters, model parameters we introduced. So omega refers to as the hyper parameters. And y is the observed noise in blurred image. Now the hyperparamaters are treated themselves as random variables, therefore we bring in information about them through the probability of omega. This is a hyperprior. So according to the product rule of probability we have a disjoint equals the likelihood, function times the prior knowledge we bring to the problem, and the prior consists of a prior on the original image f, a prior on the unknown. Impulse response of the degradation system h and the prior, on the hyper parameters. So, if we look at a, for example, at quad hyperprior we saw that for the likelihood there was the parameter beta as, as an unknown that a minus 1 is the noise variance actually. That image prior involved the parameter alpha and let assume that we have a prior on h. Lets move this prior, for example, that involves another parameter gamma. Then the set of hyper parameter omega is simply these three parameters, alpha, beta, gamma that also I need to estimate. So, we try to draw inference again based on the posterior. The probability of the unknown parameters, again, the original image, the impulse degredation system and the hyper parameters given the observed image. According to base rule, is equal to this fraction. And depending on the approach we follow, we can draw inference again from this posterior function. So, we have introduced two additional parameters, h and omega but other than that the framework is again an extension of what we already covered. It's called the hierarchical paradigm, Bayesian paradigm, because in the first stage we model the noise, the original image and the, the gradation system, and in the second stage we model the. Hyper parameters through these hyper priors. The different ways for draw inference from the posterior, we showed in the previous slide. One of which is again maximum likelihood. We try to maximize the likelihood function here, with respect to the three unknowns. It's the original image. The impulse is possible the system and the set of the hyper parameters. Clearly now, there are three unknowns involved and their optimization problem is more challenging. One general approach is to alternate optimization, so you fix two parameters, you fix each of the mega for example, optimized with respect to f. The result of the optimization is fixed on f then let's say, h is also fixed and to iterate on omega and keep iterating this way. Similarly, a map estimate can be obtained by maximizing the posterior. It's shown here again, this is the numerator of the posterior, the denominators independent of the optimization. This is now, even more challenging than maximum likelihood. Again, three unknowns alternating optimization is again a general approach that one can follow. Both these approaches provide so called point estimate. Just one answer for the unknown parameter. One might be interested in obtaining an estimate of the posterior itself. In that case, one can follow a variation or approximation approach to the posterior. So as the name implies. A function that is better behaving is used to approximate the posterior and then this other function is optimized and provides the desired form of the, the posterior. This is advantageous because now one can obtain additional information about the estimation process and possibly the different points to represent the restored image as well as the unknown impulses points. Alternatively one could sample the posterior through some modicarlo approaches, and that way an estimate of the posterior can also be obtained. In all of these cases, however, the resulting optimization is outside the scope of this course. But again, for those of you interested, I believe we have some basic understanding here, of this material. And you should be able to quote the literature and look deeper into any of these approaches. As mentioned multiple times so far the amount and quality of the prior information about the image, the blur, and the noise is of the utmost importance in establishing the quality of the restored image. When it comes to Bayesian methods, this prior knowledge is incorporated into the algorithm through the use of prior models. We discussed such prior models for the noise and the image in previous slides. Another image prior model is a total variation model shown here. It has become very popular recently in the restoration literature because of it's edge preserving property, by not over penalizing this continuity in the image while imposing smoothness. Z of alpha here is the so-called partition function. Delta of i, h is the horizontal gradient of the i'th pixel, while delta i, v is the vertical gradient at the i'th pixel. This actually represents a quadratic approximation to the TV prior. And alpha is the hyper parameter that needs to be estimated. When it comes to modelling the blur, one of the models that has been used is also a SAR of the blur. As we discussed, such a model imposes smoothness on h in this particular case. It's a reasonable model for a good number of the blurs that we encounter in practice, like the ones I showed at the beginning of the restoration segment of the course. So, here the hyper parameter is denoted by gamma and it needs to be modelled and estimated through this [UNKNOWN] framework. The literature is rich with prior models for both the images and the blur. It maybe goes without saying that the more complicated mathematically speaking the model, the, hopefully better the result, but also harder the optimization problem, one has to solve. In the following, I will show some experimental results, when these two priors are used. The tv for the image and the SAR for the blur. I will not show any of the mathematical details. This variation or approximation. Approach that they mentioned the previous slide is utilized. But again the main point is just to give you a flavor of what is achievable. I will show here in a synthetic experiment results blind and non blind restoration results utilizing the variation of Bayesian algorithm that I just mentioned a few slides earlier. The results are fully automated in the sense that all the parameters that are needed. Such as the noise variants and the image and blur model parameters, are estimated automatically from the data. So, we show here the degraded. The blur has Gaussian shape and variance nine, and the blurred signal to noise ratio is 40 dB. This is the result of blind restoration, when SAR models are used, both for the image and the blur. When now, a TV model is used for the image, while the same SAR for the blur, this is result we obtain. And if we perform a different implementation of the TV image model, we obtain the image shown here. By comparing the blind restorations here. Most people, one would agree that the TV provides better restorations. The TV image prior. And this is also reflected by the improvement in signal-to-noise ratio number. Now, if the blur is known, and this is a synthetic experiment so we know exactly the blur so it's a non-blind restoration. Then these are the corresponding results we obtain within this three combination of models I showed here. Let me switch back and forth a few times to see. What is the additional improvement one obtains by knowing exactly the blur? So, a similar comment can be made here that the TV image prior provides. Better results than the SAR image prior visually and also by comparing the ISNR numbers. Actually you see here the piecewise smooth result when you look at the TV results that is the, is what the TV model really promotes. We show here in red, the original point script function of the [UNKNOWN] system. I draw over it, so that it's visible and then below, you see three different estimates of, of the blur when the, the blind desolation algorithm is run and the difference SAR in parameters that. One has to choose in determining the hyper priors, the priors to impose on the hyper parameters that we need to estimate. So, by and large, all these three estimates shown here, do a reasonable job in approximating the real or original estimated function. We finally show a state of the odd blind restoration result utilizing the variational beta framework I just mentioned a few slides earlier. A sparse image prior is used, a so called super Gaussian prior. Since we are modeling the edges of the im.age which is sparse. We will be actually talking about sparsity during the last week of the course. So here is the acquired image and some close ups. This is an image I also showed at the beginning of the lecture on restoration. It's actually the building outside my office at North Western University. So, while the picture was taken, while the aperture was open, the camera moved or shook in an unspecified way. Applying this fully automated algorithm that is all the required parameters, were estimated from the data. Here is the restoration we obtained. So quite a lot of information is revealed. Looking also at the close up windows down here. We can see the people the people walking there and a lot, a lot of detail. The estimated blur is also shown here. So, the shaking of the camera did not follow a linear trajectory, as was the case with the horizontal motion blur we studied earlier.
