Assignment 3 ΔΙΔΑΣΚΩΝ: ΑΛΕΞΑΝΔΡΟΣ ΜΑΚΡΗΣ

Advances in Digital Imaging and Computer Vision 16/12/2018

ΠΑΠΑΔΑΚΗ ΜΑΡΙΑ Α.Μ. ΜΗ92 Σ΢ΑΡΟΤΧΙΔΟΤ ΧΑΡΙΚΛΕΙΑ Α.Μ. ΜΗ93

ΣΕΧΝΟΛΟΓΙΚΟ ΕΚΠΑΙΔΕΤΣΙΚΟ ΙΔΡΤΜΑ ΚΡΗΣΗ΢

ΠΡΟΗΓΜΕΝΑ ΢Τ΢ΣΗΜΑΣΑ ΠΑΡΑΓΩΓΗ΢ ΑΤΣΟΜΑΣΙ΢ΜΟΤ ΚΑΙ ΡΟΜΠΟΣΙΚΗ΢

PAGE 1

Assignment 3

Blob-Hough

Exercise 1 – Blob Detection (30%) 01 Blob Detection a) Develop a simple 'blob' detector. The detector takes as arguments an image and the blob radius and return the detected blob positions. The detector should use Hough – Circle the LoG filter so the possible detections are the local minima and maxima of the 02 Detection filter response. b) Add an extra parameter to the detector to select either: i) dark blobs, ii) light

blobs, or iii) both. c) Extend the detector so instead of a single radius it accepts a range (min, max). Test the detector firstly in the images 'black_dots.jpg', 'white_dots.jpg' and then also in 'coins.tiff' and 'circles_%%.jpg' . Present the results by showing the detected blobs in the initial images. Exercise 2 – Hough – Circle Detection (70%) a) Develop a system that detects circular objects of known radius r in an image. For their detection you should implement the Hough transform. Visualize the detected objects on the initial image. Test your system using the images 'coins.tiff' and 'circles_%%.jpg' .

b) Extend your system to detect circular objects within a given radius interval rmin-rmax.

c) Use the gradient orientation to render your system more robust to noise.

d) Evaluate the final detector on the image 'circles_05.jpg'. To that end evaluate the precision and recall metrics for various detection thresholds and plot the precision-recall curve. A detection should be considered as correct if the ratio of overlap between the detection bounding box and the groundtruth bounding box divided by the area of the detection bounding box is above 0.8. The groundtruth bounding boxes are provided ('circles_05_gt.mat').

PAGE 2

a) Develop a simple 'blob' detector. The detector takes as arguments an image and the blob radius and return the detected blob positions. The detector should use the LoG filter so the possible detections are the local minima and maxima of the filter response.

Before we apply the ‘blob’ detector first in 'black_dots.jpg' image with radius equals 50, we need to create the Blob detector function. (We named it blob_det(I,radius)). Inside the function we first convert our image (I) to greyscale and unit8. Then we create the LoG filter and filter the image. We defined that σ is equal with the cycle’s radius divided by the square root of 2. As well as the filter size to be 10 to 10. Then we show the image with the threshold and we find the centroid of the same image. Next we draw the red cycles around the centers and finally we show the result. Below is shown the matlab code for the blod detector function and the results we got for the image ‘black_dots.jpg'. (Image Frame 3.01) MATLAB CODE:

function [A]=blob_det(I,radius)

I1 = rgb2gray(I); %in case our image rgb converting it to grayscale I1=im2uint8(I1); %converting our image in unit8 s=radius/sqrt(2); %defining our s filter_size=[10, 10]; I2=double(I1); figure(1); imshow(I);

LoG=s^2*fspecial('log',filter_size,s); %creating LoG filter I3 = imfilter(I2,LoG); %filtering the image thr=I3>0.22; %threashold figure(3) imshow(I3);

figure(4); imshow(thr); Centers=regionprops(thr,'centroid'); centroids=cat(1,Centers.Centroid); imshow(thr); hold on

A=[centroids(:,1),centroids(:,2)]; % hold on, figure(3) plot(centroids(:,1),centroids(:,2),'g*') ; R(1:size(centroids,1),1) = radius; K = int16([A R]); red = uint8([255 0 0]); J = vision.ShapeInserter('Shape', 'Circles','BorderColor','Custom','CustomBorderColor', red); Y = step(J, I, K); imshow(Y); end

PAGE 3

Our Image Drawing the cicles

Finding the centers

Image Frame 3.01 Now we will apply the function for the image ‘white_dots.jpg’ image with radius equals 25 and the results that we got are shown in the Image Frames 3.02.

Image Cycles Drawn

Image Frames 3.02 PAGE 4

Next, we apply the function for the image ‘coins.tiff’ and take the results for the Image Frame 3.03.

Image Cycles Drawn

Image Frames 3.03

Next, we apply the function for the image ‘circles_01.jpg’ with radius equals 45, the filter size equals with 2*radius, threshold around 30 and take the results for the Image Frame 3.04.

Image Draw Cycle

Image Frames 3.04

Last for a, we apply the function for the image ‘circles_02.jpg’ with radius equals 15, the filter size equals with 2*radius, threshold around 30 and take the results for the Image Frame 3.05.

PAGE 5

Image Draw Cycle

Image Frames 3.05

b) Add an extra parameter to the detector to select either: i) dark blobs, ii) light blobs, or iii) both. Here we will add an extra parameter to our script we will choose whether we want dark circles to be assumed, let us assume option = 1, dark-circles option = 2 or all circles option 3. The code will change to the following and we will get the results below. We will just add a third variable ‘option’ to our function and simply use the if..else statement to our code. MATLAB CODE:

function [A]=blob_det1(I,radius,option)

… if option==1 thr=I3>25; if option==2 thr=I3<-25; else thr=(I3<-0.22 & I3>0.22); end end …

PAGE 6

c) Extend the detector so instead of a single radius it accepts a range (min, max)

MATLAB CODE I=imread(black_dots.jpg); I2=im2uint8(I); Is=size(I2); if (length(Is)==3) I2=rgb2gray(I2); end r_min=25; r_max=30; I_double=double(I2); figure(1); imshow(I); count=1; for i=r_min:r_max r=i; s=r/sqrt(2); filter_size=2*r; Fsx=s^2*fspecial('log',filter_size,s); th= imfilter(I_double,Fsx); I_fil_thr=th<-20; E(:,:,count)=I_fil_thr; count=count+1; end S = regionprops(E, 'centroid'); centroids=cat(1,S.Centroid); figure(3) imshow(I); hold on C=[centroids(:,1),centroids(:,2)] plot(centroids(:,1),centroids(:,2),'g*') ; R(1:size(centroids,1),1) = r; PTS = int16([C R]); red = uint8([255 0 0]); J = vision.ShapeInserter('Shape', 'Circles','BorderColor','Custom','CustomBorderColor', red); RGB = repmat(I,[1,1,3]); Y = step(J, I, PTS); figure(4) imshow(Y);

PAGE 7

Image Frames 3.06

a) Develop a system that detects circular objects of known radius r in an image. For their detection you should implement the Hough transform. Visualize the detected objects on the initial image. Test your system using the images 'coins.tiff' and 'circles_%%.jpg'.

Here we'll describe how to detect circles (which are quite important in computer vision application) using a technique Hough transform. So, the flow of events is something like this:

I. Load an image 'circles_01.jpg'. II. Detect edges and generate a binary image. III. For every 'edge' pixel, generate a circle in the space. IV. For every point on the circle in the space, cast 'votes' in the accumulator cells. V. The cells with greater number of votes are the centers. (Step 4 and 5 are created by using the AddCircle that has already being given to us. “Lucky us!” VI. We used a sobel operator to get the images. And finally, for every white pixel in the image, it creates a circle in the space. VII. The horizontal axis is the 'a' axis, the vertical axis is the 'b' axis. The brighter a spot, more the number of votes case at the point. And more votes imply a greater probability of a point being a center. Note: When the radius is unknown, the simplest solution is to just guess. Assume R = 1, and then run the same algorithm. Then assume R = 2, and run it again. Assume R = 3.... and so on.

PAGE 8

MATLAB CODE:

I=imread('circles_01.jpg'); radius=50; s=size(I); if (length(s)==3) I=rgb2gray(I); End

I2=double(I); a=min(min(I2)); b=max(max(I2));

In=(I2 - a) * 255 / (b - a); Gaus_filer=fspecial('gaussian',[3,3],0.5); Sobel_filter= fspecial('sobel');

Ix=imfilter(Gaus_filer,Sobel_filter); Iy=imfilter(Gaus_filer,Sobel_filter') ; I5=imfilter(In,Ix); I6=imfilter(In,Iy); I7=sqrt(I5.^2+I6.^2); I7=I7>235; figure(1) imshow(I7) hough=I7.*0; [K2,K1]=find(I7); x=length(K1); for i=1:x hough=AddCircle(hough,radius,K1(i),K2(i),0.011); %votes end figure(2) imshow(hough) hough=hough>0.50; %threshold S = regionprops(hough, 'centroid'); centroids=cat(1,S.Centroid); figure(3) imshow(I); hold on

C=[centroids(:,1),centroids(:,2)]; plot(centroids(:,1),centroids(:,2),'g*') ; R(1:size(centroids,1),1) = radius; PTS = int16([C R]); red = uint8([255 0 0]);

J = vision.ShapeInserter('Shape', 'Circles','BorderColor',Custom', …'CustomBorderColor', red); RGB = repmat(I,[1,1,3]); Y = step(J, RGB, PTS); figure(4) imshow(Y);

PAGE 9

Image: 'circles_01.jpg' Radius: 50 Threshold: 0.50

Image Frames 3.07

PAGE 10

Image: 'circles_02.jpg' Radius: 18 Threshold: 0.35 (We note that this time it detected another cycle not the one we were supposed to find.)

Image Frames 3.08

PAGE 11

Image: 'circles_03.jpg' Radius: 75 Threshold: 0.70

Image Frames 3.09

PAGE 12

Image: 'circles_04.jpg' Radius: 30 Threshold: 0.30

Image Frames 3.10

PAGE 13

Image: 'circles_05.jpg' Radius: 18 Threshold: 0.30

Image Frames 3.11

PAGE 14

b) Extend your system to detect circular objects within a given radius interval rmin-rmax.

The next step we want to use upper and lower limit for the radius that was given. We will only make a small change to our code adding the rmin for lower limit and rmax for the upper limit of the radius that was given. Our code can be view below as well as the results. There is not anything different from the above code. Only the loop to find a better result using the upper and lower limit. MATLAB CODE:

I=imread('coins.tiff'); I=im2uint8(I); s=size(I);

if (length(s)==3) I=rgb2gray(I); End

r_min=25; r_max=30; count=1;

for r=r_min:r_max I_d=double(I); a=min(min(I_d)); b=max(max(I_d)); In=(I_d - a) * 255 / (b - a);

Gaus_filer=fspecial('gaussian',[3,3],0.5); SObel_filter= fspecial('sobel'); Ix=imfilter(Gaus_filer,SObel_filter); Iy=imfilter(Gaus_filer,SObel_filter') ; I5=imfilter(In,Ix); I6=imfilter(In,Iy); I7=sqrt(I5.^2+I6.^2); I7=I7>235;

figure(1), imshow(I7); hough=I7.*0; [K2,K1]=find(I7); x=length(K1); for i=1:x hough=AddCircle(hough,r,K1(i),K2(i),0.011); end figure(2),imshow(hough);

hough=hough>0.9; E(:,:,count)=hough; count=count+1; end S = regionprops(E, 'centroid'); centroids=cat(1,S.Centroid); figure(3),imshow(I);

hold on C=[centroids(:,1),centroids(:,2)]; plot(centroids(:,1),centroids(:,2),'g*') ; R(1:size(centroids,1),1) = r; PTS = int16([C R]); red = uint8([255 0 0]); J = vision.ShapeInserter('Shape', 'Circles','BorderColor','Custom','CustomBorderColor', red); RGB = repmat(I,[1,1,3]); Y = step(J, RGB, PTS); figure(4),imshow(Y);

PAGE 15

Image Frames 3.12

Its already noticeable that the centers, as well as the drawn cycle around the coins are more accurate. Only by adding a minim and maximum to the radius it has improved our results. (Yay!)

PAGE 16

c) Use the gradient orientation to render your system more robust to noise.

This example demonstrates how to reduce noise associated with computing image gradients. Image gradients are used to highlight interesting features in images and are used in many feature detection algorithms like edge/corner detection. Reducing noise in gradient computations is crucial to detecting accurate features. We have made the changes that were need in our code (we added it below) as well as the results we got after running it.

MATLAB CODE: I=imread('coins.tiff') radius=25; s=size(I);

if (length(s)==3) I=rgb2gray(I); End

I2=double(I); a=min(min(I2)); b=max(max(I2));

In=(I2 - a) * 255 / (b - a); Gaus_filter=fspecial('gaussian',[3,3],0.5); Sobel_filter= fspecial('sobel');

Ix=imfilter(Gaus_filter,Sobel_filter); Iy=imfilter(Gaus_filter,Sobel_filter') ; I5=imfilter(In,Ix); I6=imfilter(In,Iy); I7=sqrt(I5.^2+I6.^2); th=atan(I6./I5);

I7=I7>240; figure(1) imshow(I7)

dx=floor(radius.*cos(th)); dy=floor(radius.*sin(th)); hough=I7.*0; [K2,K1]=find(I7); x=length(K1); for i=1:x q1=K2(i)+dx(K2(i),K1(i)); q2=K1(i)+dy(K2(i),K1(i)); q3=K2(i)-dx(K2(i),K1(i)); q4=K1(i)-dy(K2(i),K1(i)); if (((q10)) && (q20) && (q4>0) && (q4

hf= fspecial('average'); hough=imfilter(hough,hf); figure(2) imshow(hough);

PAGE 17

hough=hough>0.20;

S = regionprops(hough, 'centroid'); centroids=cat(1,S.Centroid); figure(3) imshow(I);

hold on C=[centroids(:,1),centroids(:,2)]; plot(centroids(:,1),centroids(:,2),'g*') ;

R(1:size(centroids,1),1) = radius;

PTS = int16([C R]); red = uint8([255 0 0]);

J = vision.ShapeInserter('Shape','Circles','BorderColor','Custom', 'CustomBorderColor', red);

RGB = repmat(I,[1,1,3]);

Y = step(J, RGB, PTS); figure(4) imshow(Y);

PAGE 18

IMAGE: ‘COINS.TIFF’ RADIUS: 25

Image Frames 3.12

PAGE 19

IMAGE: ‘CIRCLES_05.JPG'RADIUS: 18

Image Frames 3.13

PAGE 20

Image: 'circles_04.jpg' Radius: 30

Image Frames 3.14

PAGE 21

Image: 'circles_03.jpg' Radius: 75

Image Frames 3.15

PAGE 22

EVALUATING OBJECT DETECTORS In object detection, evaluation is non-trivial, because there are two distinct tasks to measure:

 Determining whether an object exists in the image (classification)  Determining the location of the object (localization, a regression task).

Furthermore, in a typical data set there will be many and their distribution is non-uniform (for example there might be many more oranges than cycles). So a simple accuracy-based metric will introduce biases. It is also important to assess the risk of misclassifications. Thus, there is the need to associate a “confidence score” or model score with each bounding box detected and to assess the model at various level of confidence.

MATLAB CODE: I=imread('circles_05.jpg') load('circles_05_gt.mat');

I=imread('circles_05.jpg'); radius=18; s=size(I); if (length(s)==3) I=rgb2gray(I); end result = BB(:,1,:,:); result1 = BB(:,2,:,:); groundTruthBoxes = result boundingBoxes = result1;

figure hold on for i=1:34 rectangle('Position',groundTruthBoxes(i,:),'EdgeColor','r'); end for i=1:34 rectangle('Position',boundingBoxes(i,:),'EdgeColor','b'); end

[precision,recall] = bboxPrecisionRecall(boundingBoxes,groundTruthBoxes)

PAGE 23

Image Frames 3.16

REFERENCE: http://vision.stanford.edu/teaching/cs231a_autumn1112/lecture/lecture4_edges_lines_cs231a_marked.pdf https://www.mathworks.com/help/images/ref/imfindcircles.html http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.6622&rep=rep1&type=pdf https://www.mathworks.com/matlabcentral/answers/384288-foreground-detection-and-blob-detection

PAGE 24