CS 180 Fall 2024

Overview

I explored the usage of image pyramid for image alignement in this project to create RGB values from seperated channel values. I first use skio to pre-process the given image, which contains 3 channel values. I split the imput image into 3 equal heights and then crop them by 8% from each side of the boundary. With the 3 channel pixels, I then min-max normalized each of them to account for cases were exposure on the three channels are different. Then I started trying to align the pixels directly using for loop across some (dx, dy) (in my case I use -16 to 16). Do note that I used the Normalized Cross-Correlation (NCC) value to evaluate on whether the offset is good. In contrast to Euclidean Distance, I found that NCC performs better when exposure difference between the channels are strong. This worked for small jpeg images. However, for the bigger tif inputs, I soon realize that I have to use image pyramid. To solve the issue, I recursivly called the align function. The function will return starting from image that is scaled down to no more than 256 pixels (base case). Note that this will be our biggest movement and as the recursion returns to previous layers, the alignment movement will be more and more fine grined (detailed).
I run into some issues in parameter tuning. All the files but emir.jpg resulted rather neat. I took a closer look at emir and reazlied that the green_blue alignment is perfect yet the red_blue alignment is off by a huge scale (See end of document). When I try to fix emir, other images started to act weird which made me realize emir may have its own unique issue. Upon inspecting the input data, I believe that the error may appear from the extreme difference in exposure among the 3 channels. However, all other images displayed fine so I believe emir is its own edge case and thus the algorithm does work in general for large tif images.

Official Results

red offset: [3, 12] green offset: [2, 5]

red offset: [-4, 58] green offset: [4, 52]

red offset: [-493, 147] green offset: [24, 49]

red offset: [13, 124] green offset: [16, 59]

red offset: [23, 89] green offset: [17, 41]

red offset: [11, 111] green offset: [9, 49]

red offset: [13, 179] green offset: [10, 82]

red offset: [2, 3] green offset: [2, -3]

red offset: [36, 108] green offset: [26, 51]

red offset: [-27, 140] green offset: [-11, 33]

red offset: [37, 176] green offset: [29, 78]

red offset: [11, 112] green offset: [14, 53]

red offset: [3, 6] green offset: [2, 3]

red offset: [32, 87] green offset: [5, 42]

Personal Results

red offset: [1, -4] green offset: [1, -3]

red offset: [1, 7] green offset: [2, 2]

red offset: [-3, 9] green offset: [-1, 5]

red offset: [-8, 75] green offset: [-2, -2]