Detect, Compute, and Match Descriptors
This program demonstrates how to detect, compute, and match descriptors using various algorithms: ORB, BRISK, and AKAZE.
Sources:
load a pair of grayscale images
img1 = cv.imread(fullfile(mexopencv.root(),'test','basketball1.png'), 'Grayscale',true); img2 = cv.imread(fullfile(mexopencv.root(),'test','basketball2.png'), 'Grayscale',true);
descriptors we are going to detect and compute
features = {'AKAZEUpright', 'AKAZE', 'KAZE', 'ORB', 'BRISK'};
if true
features = [features, 'SIFT', 'SURF']; % xfeatures2d from opencv_contrib
endalgorithms used to match descriptors
matchers = strcat('BruteForce', {'', '-L1', '-Hamming', '-Hamming(2)'});
descriptors loop
D = nan(numel(features), numel(matchers)); for i=1:numel(features) % create Feature2D object switch features{i} case 'AKAZEUpright' b = cv.AKAZE('DescriptorType','KAZEUpright'); case 'AKAZE' b = cv.AKAZE(); case 'KAZE' b = cv.KAZE(); case 'ORB' b = cv.ORB(); case 'BRISK' b = cv.BRISK(); case 'SIFT' b = cv.SIFT(); case 'SURF' b = cv.SURF(); end % detect and compute descriptors in both images [kpts1, desc1] = b.detectAndCompute(img1); [kpts2, desc2] = b.detectAndCompute(img2); % matchers loop for j=1:numel(matchers) % create DescriptorMatcher object m = cv.DescriptorMatcher(matchers{j}); % warn about invalid combinations of descriptors and matchers valid = true; mHam = ~isempty(strfind(matchers{j}, 'Hamming')); bHam = ~isempty(strfind(b.defaultNorm(), 'Hamming')); if mHam && (~bHam || strcmp(b.descriptorType, 'single')) fprintf(2, 'Hamming distance only for binary descriptors\n'); if mexopencv.isOctave() %HACK: avoid unhandled C++ expections, they can crash Octave! valid = false; end end if ~mHam && bHam fprintf(2, 'L1 or L2 distance not for binary descriptors\n'); end fprintf('%s + %s:\n', features{i}, matchers{j}); fprintf(' %d keypoints in img1\n', numel(kpts1)); fprintf(' %d keypoints in img2\n', numel(kpts2)); try % match descriptors assert(valid); matches = m.match(desc1, desc2); fprintf(' %d matches\n', numel(matches)); % save matches to file fname = sprintf('%s_%s.yml', features{i}, matchers{j}); fname = fullfile(tempdir(), fname); cv.FileStorage(fname, struct('Matches',matches)); % Keep only best 30 matches to have a nice drawing (sorted by dist) [~,ord] = sort([matches.distance]); ord(31:end) = []; matches = matches(ord); % pretty print matches if ~mexopencv.isOctave() t = struct2table(matches); t.imgIdx = []; display(t) else %HACK: no tables in Octave t = cat(1, matches.distance); display(t) end % draw matches and show result out = cv.drawMatches(img1, kpts1, img2, kpts2, matches); figure, imshow(out) title(sprintf('%s + %s', features{i}, matchers{j})) % compute total distances between keypoint matches pts1 = cat(1, kpts1([matches.queryIdx]+1).pt); pts2 = cat(1, kpts2([matches.trainIdx]+1).pt); D(i,j) = sum(sqrt(sum((pts1 - pts2).^2, 2))); fprintf(' %f total error\n', D(i,j)); catch ME fprintf(' matching failed\n'); %warning(getReport(ME)) end fprintf('%s\n', repmat('-',1,80)); end end
AKAZEUpright + BruteForce:
509 keypoints in img1
489 keypoints in img2
509 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ _________
225 212 0.0061414
425 400 0.0065284
228 216 0.0078475
432 407 0.0078775
285 270 0.0089423
259 244 0.0089759
413 388 0.0093736
1 0 0.009867
286 272 0.0099593
105 91 0.010512
115 102 0.010668
486 464 0.010733
287 273 0.011009
233 219 0.011101
3 1 0.011432
108 94 0.012
291 277 0.012291
288 274 0.01242
246 231 0.012453
6 4 0.013142
276 262 0.015391
114 100 0.015577
112 97 0.016037
273 261 0.016045
121 108 0.016096
4 2 0.016369
294 279 0.016634
292 276 0.017646
111 98 0.017932
110 96 0.018505
76.347928 total error
--------------------------------------------------------------------------------
AKAZEUpright + BruteForce-L1:
509 keypoints in img1
489 keypoints in img2
509 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
225 212 0.035796
425 400 0.039139
432 407 0.045195
228 216 0.047979
286 272 0.04935
259 244 0.052619
285 270 0.052759
413 388 0.055712
1 0 0.056515
486 464 0.060291
105 91 0.060574
115 102 0.061157
3 1 0.06201
288 274 0.065543
233 219 0.065866
291 277 0.065992
287 273 0.067931
108 94 0.07174
246 231 0.07656
6 4 0.078552
276 262 0.079296
112 97 0.091108
273 261 0.091113
114 100 0.091542
294 279 0.092278
398 376 0.092863
4 2 0.092881
121 108 0.093999
292 276 0.094359
111 98 0.097353
75.509662 total error
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
AKAZEUpright + BruteForce-Hamming:
509 keypoints in img1
489 keypoints in img2
matching failed
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
AKAZEUpright + BruteForce-Hamming(2):
509 keypoints in img1
489 keypoints in img2
matching failed
--------------------------------------------------------------------------------
L1 or L2 distance not for binary descriptors
AKAZE + BruteForce:
589 keypoints in img1
569 keypoints in img2
589 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
444 421 0
461 438 2
432 411 5.6569
310 296 8.2462
318 302 8.3666
39 25 8.4853
439 416 12.042
42 31 14.595
275 262 16.031
316 300 17.205
312 298 17.889
225 215 23.431
265 254 23.685
135 123 29.086
469 445 32.016
125 113 32.062
127 115 37.736
44 34 42.544
542 517 43.543
309 294 48.052
167 155 62.137
255 243 65.315
315 301 68.066
122 108 72.166
249 237 74.135
448 424 75.472
153 145 93.145
535 507 94.366
547 524 96
537 510 99.479
77.536100 total error
--------------------------------------------------------------------------------
L1 or L2 distance not for binary descriptors
AKAZE + BruteForce-L1:
589 keypoints in img1
569 keypoints in img2
589 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
444 421 0
461 438 2
432 411 8
310 296 10
39 25 12
318 302 12
275 262 17
439 416 21
312 298 24
316 300 28
42 31 31
469 445 33
125 113 34
225 215 43
265 254 45
309 294 51
127 115 68
135 123 80
44 34 86
542 517 100
448 424 104
315 301 105
122 108 112
249 237 128
118 105 146
167 155 151
547 524 160
535 507 169
131 119 173
295 106 180
80.196240 total error
--------------------------------------------------------------------------------
AKAZE + BruteForce-Hamming:
589 keypoints in img1
569 keypoints in img2
589 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
444 421 0
461 438 1
125 113 2
275 262 2
310 296 2
312 298 2
432 411 2
469 445 2
39 25 3
118 105 3
305 291 3
547 524 3
115 102 4
137 127 4
309 294 4
318 302 4
325 310 4
439 416 4
448 424 4
127 115 5
131 119 5
316 300 5
21 15 6
42 31 6
122 108 6
225 215 6
265 254 6
295 106 6
311 297 6
11 5 7
103.266482 total error
--------------------------------------------------------------------------------
AKAZE + BruteForce-Hamming(2):
589 keypoints in img1
569 keypoints in img2
589 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
444 421 0
461 438 1
125 113 2
275 262 2
310 296 2
312 298 2
432 411 2
469 445 2
39 25 3
118 105 3
305 291 3
309 294 3
547 524 3
115 102 4
137 127 4
318 302 4
325 310 4
439 416 4
448 424 4
127 115 5
131 119 5
316 300 5
21 15 6
42 31 6
122 108 6
225 215 6
265 254 6
295 106 6
311 297 6
11 5 7
103.266482 total error
--------------------------------------------------------------------------------
KAZE + BruteForce:
962 keypoints in img1
909 keypoints in img2
962 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ _________
405 388 0.0085198
646 621 0.0088553
457 432 0.0093376
18 16 0.0096003
208 193 0.010034
449 425 0.010102
187 176 0.010281
511 480 0.01029
2 1 0.010337
216 202 0.010546
131 123 0.011074
818 764 0.011187
149 142 0.011195
0 0 0.011843
389 370 0.012087
191 179 0.012396
207 191 0.013254
196 183 0.013399
144 138 0.014298
14 12 0.014497
634 610 0.014566
454 429 0.016669
204 188 0.01698
96 84 0.020786
220 206 0.021669
145 139 0.021678
921 863 0.021951
532 502 0.023943
19 17 0.024116
447 423 0.024717
50.046663 total error
--------------------------------------------------------------------------------
KAZE + BruteForce-L1:
962 keypoints in img1
909 keypoints in img2
962 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
449 425 0.046328
457 432 0.050101
405 388 0.052212
646 621 0.053055
2 1 0.055326
18 16 0.056384
0 0 0.057026
818 764 0.057264
208 193 0.057709
187 176 0.057777
511 480 0.058332
216 202 0.058883
149 142 0.061423
131 123 0.068747
389 370 0.071617
191 179 0.075301
634 610 0.080895
196 183 0.082588
207 191 0.084627
204 188 0.086547
14 12 0.090572
144 138 0.093892
96 84 0.099536
454 429 0.10454
220 206 0.11729
145 139 0.12303
921 863 0.12421
536 505 0.13232
532 502 0.13498
19 17 0.13511
50.707891 total error
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
KAZE + BruteForce-Hamming:
962 keypoints in img1
909 keypoints in img2
matching failed
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
KAZE + BruteForce-Hamming(2):
962 keypoints in img1
909 keypoints in img2
matching failed
--------------------------------------------------------------------------------
L1 or L2 distance not for binary descriptors
ORB + BruteForce:
500 keypoints in img1
500 keypoints in img2
500 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
2 98 11.314
192 173 34.176
80 61 46.54
77 7 51.769
81 90 57.991
247 221 65.276
174 122 66.091
137 139 72.076
149 151 73.607
206 237 73.79
319 316 73.844
13 75 80.144
199 225 84.06
392 392 95.682
252 236 97.432
368 341 97.499
60 8 101.18
249 278 105.33
345 369 107.87
166 146 109.8
124 120 112.03
357 363 118.08
351 346 120.96
216 356 125.39
46 97 129.22
115 144 130.32
123 114 132.79
71 40 133.2
163 147 134.42
193 155 134.6
78.299105 total error
--------------------------------------------------------------------------------
L1 or L2 distance not for binary descriptors
ORB + BruteForce-L1:
500 keypoints in img1
500 keypoints in img2
500 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
2 98 16
192 173 52
247 221 83
174 122 84
80 61 88
137 139 111
77 7 120
149 151 122
206 237 123
81 90 145
319 316 147
13 75 153
46 97 157
115 144 173
71 40 191
123 114 194
223 244 198
345 369 203
175 166 210
99 104 213
163 147 216
60 8 218
199 225 218
392 392 219
193 155 220
351 346 220
211 249 223
35 68 232
272 272 236
124 120 237
70.072139 total error
--------------------------------------------------------------------------------
ORB + BruteForce-Hamming:
500 keypoints in img1
500 keypoints in img2
500 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
2 98 2
174 122 3
192 173 4
247 221 5
46 97 6
99 104 6
223 244 6
149 151 7
206 237 7
71 40 8
136 138 8
137 139 8
1 3 9
13 75 9
35 68 9
80 61 9
115 144 9
345 369 9
351 346 9
358 355 9
394 419 9
22 92 10
77 7 10
123 114 10
147 129 10
163 147 10
211 249 10
272 272 10
339 371 10
374 340 10
48.577527 total error
--------------------------------------------------------------------------------
ORB + BruteForce-Hamming(2):
500 keypoints in img1
500 keypoints in img2
500 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
2 98 2
174 122 3
192 173 4
247 221 5
46 97 6
99 104 6
223 244 6
149 151 7
206 237 7
71 40 8
115 144 8
136 138 8
137 139 8
1 3 9
13 75 9
22 92 9
35 68 9
80 61 9
123 114 9
345 369 9
351 346 9
358 355 9
394 419 9
77 7 10
147 129 10
163 147 10
211 249 10
272 272 10
295 276 10
339 371 10
50.305524 total error
--------------------------------------------------------------------------------
L1 or L2 distance not for binary descriptors
BRISK + BruteForce:
767 keypoints in img1
735 keypoints in img2
767 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
433 418 30.594
523 505 32
401 383 33.956
664 637 64.031
224 217 75.266
185 178 75.286
413 395 78.39
648 621 80.623
357 331 84.581
0 0 90.708
673 555 92.027
549 637 92.26
14 14 113.82
153 144 119.92
655 628 120.09
374 348 128.09
277 271 133.6
41 38 135.1
272 404 136.87
361 336 139.23
646 619 143.11
204 196 146.64
423 410 146.87
270 266 148.03
574 555 149.63
402 384 153.21
638 614 158.52
576 559 159.86
418 265 160.02
172 163 160.55
60.398485 total error
--------------------------------------------------------------------------------
L1 or L2 distance not for binary descriptors
BRISK + BruteForce-L1:
767 keypoints in img1
735 keypoints in img2
767 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
523 505 32
401 383 49
664 637 66
433 418 68
413 395 129
374 348 136
224 217 137
0 0 138
185 178 140
673 555 151
549 637 152
648 621 162
646 619 192
277 271 204
357 331 214
204 196 224
423 410 234
361 336 253
272 404 255
270 266 266
153 144 292
206 197 313
172 163 316
14 14 318
402 384 325
638 614 325
3 5 327
574 555 331
643 618 337
576 559 338
49.992817 total error
--------------------------------------------------------------------------------
BRISK + BruteForce-Hamming:
767 keypoints in img1
735 keypoints in img2
767 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
523 505 1
646 619 2
664 637 2
204 196 3
374 348 4
401 383 4
413 395 4
549 637 4
0 0 5
423 410 5
96 87 6
673 555 6
167 157 7
224 217 7
433 418 8
648 621 8
172 163 9
185 178 10
315 295 10
724 696 10
411 393 11
3 5 12
270 266 12
277 271 12
361 336 12
402 384 12
420 406 12
89 82 13
197 189 13
731 704 13
56.118373 total error
--------------------------------------------------------------------------------
BRISK + BruteForce-Hamming(2):
767 keypoints in img1
735 keypoints in img2
767 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
523 505 1
646 619 2
664 637 2
204 196 3
374 348 4
401 383 4
413 395 4
549 637 4
0 0 5
423 410 5
96 87 6
167 157 6
673 555 6
224 217 7
433 418 8
648 621 8
172 163 9
185 178 10
315 295 10
411 393 10
724 696 10
402 384 11
3 5 12
270 266 12
277 271 12
361 336 12
420 406 12
89 82 13
197 189 13
377 521 13
56.575280 total error
--------------------------------------------------------------------------------
SIFT + BruteForce:
538 keypoints in img1
554 keypoints in img2
538 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
151 158 7.6158
200 206 9.434
169 183 9.8995
133 138 10.05
122 126 11.18
145 152 12.124
135 140 12.806
180 192 13.304
128 131 13.342
154 165 14.107
189 198 14.248
153 162 17.117
138 143 17.378
137 142 17.578
14 13 17.972
76 77 19.209
181 193 20.224
85 91 21.237
20 20 21.307
129 133 21.794
110 117 21.932
107 114 22.226
440 451 22.383
162 172 22.583
164 175 23.937
163 174 24.678
81 88 24.839
171 184 25
38 37 27.129
288 308 27.532
22.399791 total error
--------------------------------------------------------------------------------
SIFT + BruteForce-L1:
538 keypoints in img1
554 keypoints in img2
538 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
151 158 38
200 206 51
133 138 59
145 152 59
135 140 64
169 183 64
122 126 67
180 192 73
154 165 75
128 131 78
137 142 89
138 143 92
189 198 93
76 77 95
153 162 95
129 133 105
14 13 109
81 88 117
85 91 117
181 193 125
162 172 130
164 175 141
440 451 145
182 194 147
184 196 147
20 20 150
82 89 153
171 184 153
288 308 154
163 174 155
21.775604 total error
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
SIFT + BruteForce-Hamming:
538 keypoints in img1
554 keypoints in img2
matching failed
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
SIFT + BruteForce-Hamming(2):
538 keypoints in img1
554 keypoints in img2
matching failed
--------------------------------------------------------------------------------
SURF + BruteForce:
833 keypoints in img1
819 keypoints in img2
833 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ _________
21 22 0.0044549
25 26 0.0075029
741 752 0.0083626
332 324 0.0084082
371 351 0.0086561
265 248 0.0094784
129 123 0.0098364
119 112 0.010511
20 17 0.01079
104 100 0.011485
434 393 0.011719
87 93 0.011821
224 211 0.012183
72 65 0.013086
814 740 0.01342
171 165 0.013746
508 487 0.01501
380 356 0.01558
731 712 0.015842
69 68 0.015907
500 446 0.016303
502 473 0.016388
753 729 0.016895
61 67 0.017205
559 531 0.01759
663 633 0.017897
793 769 0.017943
560 475 0.018021
509 483 0.018098
403 377 0.018622
39.197591 total error
--------------------------------------------------------------------------------
SURF + BruteForce-L1:
833 keypoints in img1
819 keypoints in img2
833 matches
t =
30×3 table
queryIdx trainIdx distance
________ ________ ________
21 22 0.017737
25 26 0.031595
332 324 0.036884
265 248 0.039954
371 351 0.042173
129 123 0.043942
741 752 0.045758
104 100 0.049184
119 112 0.04946
434 393 0.0545
72 65 0.054785
87 93 0.054964
224 211 0.056004
20 17 0.05609
380 356 0.06616
171 165 0.066818
69 68 0.067284
814 740 0.067873
508 487 0.069274
500 446 0.071465
731 712 0.075041
61 67 0.079448
481 424 0.079876
502 473 0.081175
793 769 0.081626
247 234 0.084103
403 377 0.084347
559 531 0.084814
666 639 0.086527
535 508 0.087287
30.480832 total error
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
SURF + BruteForce-Hamming:
833 keypoints in img1
819 keypoints in img2
matching failed
--------------------------------------------------------------------------------
Hamming distance only for binary descriptors
SURF + BruteForce-Hamming(2):
833 keypoints in img1
819 keypoints in img2
matching failed
--------------------------------------------------------------------------------
Cumulative distances between matched keypoint for different matchers and feature detectors
if ~mexopencv.isOctave() t = array2table(D, 'RowNames',features, ... 'VariableNames',regexprep(matchers,'\W','')); display(t) else %HACK: no tables in Octave display(D) end
t =
7×4 table
BruteForce BruteForceL1 BruteForceHamming BruteForceHamming2
__________ ____________ _________________ __________________
AKAZEUpright 76.348 75.51 NaN NaN
AKAZE 77.536 80.196 103.27 103.27
KAZE 50.047 50.708 NaN NaN
ORB 78.299 70.072 48.578 50.306
BRISK 60.398 49.993 56.118 56.575
SIFT 22.4 21.776 NaN NaN
SURF 39.198 30.481 NaN NaN