CS 180 Fall 2024

Project 4 by Lawrence Shieh

Part A

Part 1. Shooting Pictures

Using my phone camera, I have taken some pictures on campus and at my appartnment to be used in this project.

Image 1 for Part 1.1
Image To Rectify 1
Image 2 for Part 1.1
Image To Rectify 2
Image 3 for Part 1.1
Blend Pair 1-1
Image 3 for Part 1.1
Blend Pair 1-2
Image 2 for Part 1.1
Blend Pair 2-1
Image 2 for Part 1.1
Blend Pair 2-2
Image 2 for Part 1.1
Blend Pair 2-1
Image 2 for Part 1.1
Blend Pair 2-2

Part 2. Recover Homographies"

The goal here is to find the homography matrix H. Since there are 8 unkown entries in the matrix, we need at least 4 corresponding pairs of points to find the matrix. Note that the scale factor "i" here can be set to 1 so in reality there is only 8 unkonwns shown in the follwing matrix. Also note that we are using OSL if we provide more than 4 points.

Image 1 for Part 1.1
p' = Hp from lecture slide
Image 1 for Part 1.1
OSL from lecture slide
Image 1 for Part 1.1
Warped Image 1-1 To 1-2

Part 3. Rectify Images

1. Take picture of image that contains a rectangle somewhere.
2. Define the coordinates on the 4 corner of the rectangle.
3. Self define a set of corresponding coordinates to those 4 corners.
4. Use part 2 to find the homography matrix H.
5. Apply the homography matrix to the original image we took.
For image 1 I target the green looking rectangle above the light switch. For image 2 I target the poster itself.

Image 1 for Part 1.1
Image 1 Original
Image 1 for Part 1.1
Image 1 Rectified
Image 1 for Part 1.1
Image 2 Original
Image 1 for Part 1.1
Image 2 Rectified

Part 4. Belnd Images

1. Define corresponding points to two images.
2. Warp one of the images onto the other like part 3.
3. Using the given offsets (if not already shown in the output) to align image.
4. Blend them together using concepts from project 2.

Image 1 for Part 1.1
Original 1-1
Image 1 for Part 1.1
Original 1-2
Image 1 for Part 1.1
Final
Image 1 for Part 1.1
Original 2-1
Image 1 for Part 1.1
Original 2-2
Image 1 for Part 1.1
Final
Image 1 for Part 1.1
Original 3-1
Image 1 for Part 1.1
Original 3-2
Image 1 for Part 1.1
Final

Part B

I went on a trip to Lake Tahoe in Califonia and found a pretty picture spot. Although the images all look similar, they are all unique if you look closely.

Step 1: Apply Harris Corners

I applied the harris corners algorithm to extract a bunch of harris corner points.

Image 1 for Part 1.1
Harris 1-1
Image 1 for Part 1.1
Harris 1-2
Image 1 for Part 1.1
Harris 2-1
Image 1 for Part 1.1
Harris 2-2
Image 1 for Part 1.1
Harris 3-1
Image 1 for Part 1.1
Harris 3-1

Step 2: Apply Adaptive Non-Maximal Suppression

I applied the Adaptive Non-Maximal Suppression algorithm to reduce the amount of Harris points.

Image 1 for Part 1.1
anms 1-1
Image 1 for Part 1.1
anms 1-2
Image 1 for Part 1.1
anms 2-1
Image 1 for Part 1.1
anms 2-2
Image 1 for Part 1.1
anms 3-1
Image 1 for Part 1.1
anms 3-1

Step 3: Apply Feature Descriptor Extraction & Feature Matching

For each harris points, create a 40 by 40 grid around.

Image 1 for Part 1.1
boxes 1-1
Image 1 for Part 1.1
boxes 1-2
Image 1 for Part 1.1
boxes 2-1
Image 1 for Part 1.1
boxes 2-2
Image 1 for Part 1.1
boxes 3-1
Image 1 for Part 1.1
boxes 3-1

Sub-sample each 40 by 40 box down to 8 by 8. Normalize the descriptor. Each of these image here contains many small 8 by 8 normalized image. Note that some of these values are "negative" so it may not make much visual sense.

Image 1 for Part 1.1
features 1-1
Image 1 for Part 1.1
features 1-2
Image 1 for Part 1.1
features 2-1
Image 1 for Part 1.1
features 2-2
Image 1 for Part 1.1
features 3-1
Image 1 for Part 1.1
features 3-1

Implement Feature Matching using Lowe thresholding. If you look closely the corresponding images have corresponding numbers to each of the matching points.

Image 1 for Part 1.1
numbered 1-1
Image 1 for Part 1.1
numbered 1-2
Image 1 for Part 1.1
numbered 2-1
Image 1 for Part 1.1
numbered 2-2
Image 1 for Part 1.1
numbered 3-1
Image 1 for Part 1.1
numbered 3-1

Step 4: Apply RANSAC

I applied the ransac algorithm to find the best homography function between two sets of corresponding points. I blended the image using the homography and the results are as shown.
1. Select four feature pairs (at random)
2. Compute homography H (exact)
3. Compute inliers where dist(pi’, H pi ) < ε
4. Keep largest set of inliers
5. Re-compute least-squares H estimate on all of the inliers

Image 1 for Part 1.1
Original 1-1
Image 1 for Part 1.1
Original 1-2
Image 1 for Part 1.1
Blended 1
Image 1 for Part 1.1
Original 2-1
Image 1 for Part 1.1
Original 2-2
Image 1 for Part 1.1
Blended 2
Image 1 for Part 1.1
Original 3-1
Image 1 for Part 1.1
Original 3-2
Image 1 for Part 1.1
Blended 3

Short Reflection

The coolest thing I have learned is how powerful harris corners can get. In project 3 we have to manually pick all the corresponding points and that was really annoying. What we have implmented here is essentailly a pipeline that requires minimal human interference to stich two images together into a mosaic.