Yes this can certainly be done.  The maths arent that hard.  

To filter an image, you go through every pixel (or in your case, only the pixels you wish to filter), and assign that pixel a new value based on the values of itself, and the surrounding pixels.  This is called convolution.

Think of a grayscale image to start with until you are comfortable.

So for example, a blur filter would assign a pixel a new value based on an average of the surrounding pixels' values.  Easy enough.

But, *how* is it done?  Well thats where the FILTER KERNEL comes in.  A filter kernel, in the most basic case is a 3x3 matrix of values.  This matrix represents the amount of influence each surrounding pixel will have on the output.  

An example will help here. A blur kernel is:

1,1,1
1,1,1  <-- number in the middle is 'current pixel'
1,1,1  

As you can see, this blur filter takes the current pixel, multiplies it by 1, takes all the currounding pixels, multiplies each of them by one.  Then you add them all together, and divide by the TOTAL SUM of the kernel weights, in this case 9.  

You then assign this resultant value to the current pixel, and you have averaged it with its neighbors.  If you perform this function on all the pixels in the image then you get a blurred image.  There are filter kernels available for all kinds of interesting effects, just use google.

So the algorithm is this:

For each pixel in the image:
- Multiply center kernel value by current pixel(x,y)
- Multiply top left kernel value by pixel (x-1,y-1)
- Multiply top center kernel value by pixel (x,y-1)
- Multiply top right kernel value by pixel (x+1,y-1)
- Do this for all the surrounding pixels
- Add these up and divide by the kernel's sum
- Assign this value to the current location (x,y) in
an output image.

Again, there is *so* much information available on this.  Google is your friend, use it!  

Hope that helps