# 1: The Terminology

## The filter - its properties, characteristics and functionality

First off, what is a filter? A filter is any kind of processing that has one characteristic: it takes a signal as the input and produces a relevant output signal after processing it in some way. A filter can always be mathematically described - think of it as a function, for instance:**Fig. 1.1: The least elaborate form of a simple filter that only takes one argument (an input signal or sample) and produces an output signal or sample**

**Fig. 1.2: a 1D filter**

**Fig. 1.3: a 2D filter**

## The spatial, frequency and time domains

The above terms actually define two different "spaces". The catch here lies that while the time domain only exists for 1D signals (such as an audio signal), the spatial domain describes a 2D signal (an image). Both of these domains can be converted into an alternate domain - the frequency domain. The process used to do the conversion is called the Fourier Transform and will not be discussed in this article because it has very little use in image filtering. That is, while most audio filters (high pass, low pass, band pass and others) function in the frequency domain, image information is encoded mostly in the spatial domain, requiring no conversion at all. The moral here - if you look at an image, think of the spatial domain; in the case of an audio signal think of the time domain and when looking at a frequency spectrum (as in WinAMP), think of the frequency domain. For now, let's forget all about everything but the spatial domain.## The kernel and the PSF

Again, these two terms mean the exact same thing with the difference that a kernel generally denotes the use of a 1D filter while the PSF, which stands for point spread function, denotes the use of a 2D filter. There is yet another name that is used: the impulse response, which is arguably the most relevant for both cases. Looking back at figures 1.2 and 1.3, one can see that the explanations below them are flawed. The figures do not show what a filter looks like, but rather the impulse responses of the respective 1D and 2D filters. We will be using all of the terms interchangeably for all filter types.## Convolution as a mathematical operation

Applying a simple filter to an input signal requires some math to be done. The method used for kernel-type filters is called convolution. It is a ternary mathematical operation (taking two arguments and producing one output value) just as addition or multiplication. Compared to the two, however, it is more like multiplication due to one very important characteristic in signal processing: linearity. Linearity There are two types of systems that surround us: a system can either be linear or non-linear. Most real life systems (such as temperature changes) can be characterized as non-linear. This means that there is a lot of uncertainty involved. Mathematics introduces some very well known non-linear operations and functions: integration, division, multiplication, taking the logarithm, etc. This means that it is somewhat challenging to know the source data by analyzing the output data. Fortunately mathematics also introduces inverse functions and operations to most of the commonly used ones: division for multiplication, logarithm for exponentiation, etc. Other types of systems are linear - systems that we can and most of the time do know a great deal about. It isn't uncommon that a linear system is perfectly defined - that is, we know everything about it. Two linear mathematical operations are addition and subtraction. Convolution belongs to the non-linear class - that means that once a signal is convolved, it isn't at all that trivial to reproduce the original signal. Although the inverse of convolution, deconvolution, exists, it is not discussed in this article. In mathematics, convolution is generally denoted with the asterisk (*). This notation is what we will be using so do not confuse it with multiplication! Again, mathematically speaking, here's how the whole thing works:**Eq. 1.1: 1D convolution (the black dot denotes multiplication). y is the output signal, x is the input signal and h is the filter kernel**

**Eq. 1.2: 2D convolution**

**Fig. 1.4: a convolution example of a 2D signal with a 2D filter with c=-1/8. The asterisk (*), as mentioned above, denotes convolution.**

**Fig. 1.5: a sample and impulse response combination that produces a negative output value if c < 0**

# 2. Filtering an input image

Even though the only limitation for filter dimensions is the fact that it has to have an odd number of samples, problems arise when we use a filter kernel with d > 1 with more than one non-zero value. That is, the only filter that will not introduce artefacts in the output signal is the impulse filter, which has the following kernel:**Fig. 2.1: a simple two-dimensional impulse where n?N. If n=1 the impulse response becomes a unity impulse, e.g. it will not change the input signal**

# 3. Designing a filter

This may seem surprising, but high-profile filter design comprises 40% logic and predicate analysis and 60% experimentation. We will now look at different filters commonly used and the changes they introduce if we vary some parameters. To demonstrate a possible output of the following filters we will be applying them to this source image:**Img 3.1: the original reference image used hence forth**

Since the sample program accompanying this article only handles greyscale images (that are converted into 24 bit DIB's upon export), the reference images produced in Paint Shop Pro 4.12 have first been converted to greyscale and then filtered.

Even though the images produced by Paint Shop Pro are interchangeably similar to the ones produced by our software, notice the discrepancies at the edges of the images we've presented - this "artefact" has been discussed above. Just know that it is not a bug!

## Edge detection

The single most important feature one could extract from an image is the edge data. The edge information can be parallelized to the frequency spectrum of an audio signal - one can use it to do a lot of very neat tricks with the image. For instance, if the edges are fuzzy and hard to make out, a modified version of edge detection - edge enhancement - can be used to enforce the contours. Edge detection is the most important thing in image recognition - for instance, if we have a reference image and some target image that we want to locate (or verify if it exists) in the reference image, we first find the edges in both images and do a check using the edge data instead of some weird interpolation that uses pixel information. Cell shading can be accomplished through edge detection - a neat effect to be used in games. Once again, the kernel of a traditional edge detection filter looks like this (this has been covered above):**Fig. 3.1: the traditional filter kernel of an edge detection filter where c = -1/8**

## Edge enhancement

If we're only stuck with a blurry image, we can use a parameterized edge detection filter to make the edges stick out more. To do this, we modify the constant c in the edge detection filter with the value c = -k/8. Here are the results:**Img 3.3: the edge enhancement filter does nothing more than makes the edges in the image more prevalent. We used k = 3 in this case**

## Sharpen and unsharpen (blur)

Sharpening is pretty much the same as edge enhancement - it accentuates the edges, unsharpening, or blurring, being its immediate inverse. We won't be comparing the output images for these filters as we're not using the same constants as Paint Shop Pro (our filter sharpens more at a time). Here are the filter kernels used:**Fig 3.1: the sharpening filter kernel. We're using c=-1 in the sample software**

**Fig 3.2: the unsharpening filter kernel**

## Emboss and engrave

Embossing is probably one of the 'coolest' effects that can be applied to an image. Its close relative, bumpmapping, is one of those few texture effects that you can often toggle on and off in a game. Bumpmapping relies on the presence of a "bump image", or the bump map which is essentially nothing more than a simple embossed image that can be created using your favourite image editing program. By combining this bump image with the texture map you can create the final bumpmapped image. Using what we've learnt so far, it isn't difficult to construct a simple embossing filter that does the whole business of creating the bump image for us without the need for fancy software. Here is a set of the filter kernels used for embossing an image:**Fig 3.3: a 3x3 emboss filter kernel**

**Fig 3.4: a 5x5 emboss filter kernel**

**Fig 3.5: a 7x7 emboss filter kernel**

**Img 3.4: using different filter kernel sizes creates different depths in the embossed image. Can you hazard a guess which kernel size Paint Shop Pro might be using? (images counter-clockwise from top right: 5x5 filter kernel, 3x3 filter, 7x7 filter and PSP)**