-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.html
673 lines (582 loc) · 101 KB
/
README.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
<!DOCTYPE html><html><head>
<title>README</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="file:///c:\Users\hank\.vscode\extensions\shd101wyy.markdown-preview-enhanced-0.5.16\node_modules\@shd101wyy\mume\dependencies\katex\katex.min.css">
<style>
/**
* prism.js Github theme based on GitHub's theme.
* @author Sam Clarke
*/
code[class*="language-"],
pre[class*="language-"] {
color: #333;
background: none;
font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace;
text-align: left;
white-space: pre;
word-spacing: normal;
word-break: normal;
word-wrap: normal;
line-height: 1.4;
-moz-tab-size: 8;
-o-tab-size: 8;
tab-size: 8;
-webkit-hyphens: none;
-moz-hyphens: none;
-ms-hyphens: none;
hyphens: none;
}
/* Code blocks */
pre[class*="language-"] {
padding: .8em;
overflow: auto;
/* border: 1px solid #ddd; */
border-radius: 3px;
/* background: #fff; */
background: #f5f5f5;
}
/* Inline code */
:not(pre) > code[class*="language-"] {
padding: .1em;
border-radius: .3em;
white-space: normal;
background: #f5f5f5;
}
.token.comment,
.token.blockquote {
color: #969896;
}
.token.cdata {
color: #183691;
}
.token.doctype,
.token.punctuation,
.token.variable,
.token.macro.property {
color: #333;
}
.token.operator,
.token.important,
.token.keyword,
.token.rule,
.token.builtin {
color: #a71d5d;
}
.token.string,
.token.url,
.token.regex,
.token.attr-value {
color: #183691;
}
.token.property,
.token.number,
.token.boolean,
.token.entity,
.token.atrule,
.token.constant,
.token.symbol,
.token.command,
.token.code {
color: #0086b3;
}
.token.tag,
.token.selector,
.token.prolog {
color: #63a35c;
}
.token.function,
.token.namespace,
.token.pseudo-element,
.token.class,
.token.class-name,
.token.pseudo-class,
.token.id,
.token.url-reference .token.variable,
.token.attr-name {
color: #795da3;
}
.token.entity {
cursor: help;
}
.token.title,
.token.title .token.punctuation {
font-weight: bold;
color: #1d3e81;
}
.token.list {
color: #ed6a43;
}
.token.inserted {
background-color: #eaffea;
color: #55a532;
}
.token.deleted {
background-color: #ffecec;
color: #bd2c00;
}
.token.bold {
font-weight: bold;
}
.token.italic {
font-style: italic;
}
/* JSON */
.language-json .token.property {
color: #183691;
}
.language-markup .token.tag .token.punctuation {
color: #333;
}
/* CSS */
code.language-css,
.language-css .token.function {
color: #0086b3;
}
/* YAML */
.language-yaml .token.atrule {
color: #63a35c;
}
code.language-yaml {
color: #183691;
}
/* Ruby */
.language-ruby .token.function {
color: #333;
}
/* Markdown */
.language-markdown .token.url {
color: #795da3;
}
/* Makefile */
.language-makefile .token.symbol {
color: #795da3;
}
.language-makefile .token.variable {
color: #183691;
}
.language-makefile .token.builtin {
color: #0086b3;
}
/* Bash */
.language-bash .token.keyword {
color: #0086b3;
}
/* highlight */
pre[data-line] {
position: relative;
padding: 1em 0 1em 3em;
}
pre[data-line] .line-highlight-wrapper {
position: absolute;
top: 0;
left: 0;
background-color: transparent;
display: block;
width: 100%;
}
pre[data-line] .line-highlight {
position: absolute;
left: 0;
right: 0;
padding: inherit 0;
margin-top: 1em;
background: hsla(24, 20%, 50%,.08);
background: linear-gradient(to right, hsla(24, 20%, 50%,.1) 70%, hsla(24, 20%, 50%,0));
pointer-events: none;
line-height: inherit;
white-space: pre;
}
pre[data-line] .line-highlight:before,
pre[data-line] .line-highlight[data-end]:after {
content: attr(data-start);
position: absolute;
top: .4em;
left: .6em;
min-width: 1em;
padding: 0 .5em;
background-color: hsla(24, 20%, 50%,.4);
color: hsl(24, 20%, 95%);
font: bold 65%/1.5 sans-serif;
text-align: center;
vertical-align: .3em;
border-radius: 999px;
text-shadow: none;
box-shadow: 0 1px white;
}
pre[data-line] .line-highlight[data-end]:after {
content: attr(data-end);
top: auto;
bottom: .4em;
}html body{font-family:"Helvetica Neue",Helvetica,"Segoe UI",Arial,freesans,sans-serif;font-size:16px;line-height:1.6;color:#333;background-color:#fff;overflow:initial;box-sizing:border-box;word-wrap:break-word}html body>:first-child{margin-top:0}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{line-height:1.2;margin-top:1em;margin-bottom:16px;color:#000}html body h1{font-size:2.25em;font-weight:300;padding-bottom:.3em}html body h2{font-size:1.75em;font-weight:400;padding-bottom:.3em}html body h3{font-size:1.5em;font-weight:500}html body h4{font-size:1.25em;font-weight:600}html body h5{font-size:1.1em;font-weight:600}html body h6{font-size:1em;font-weight:600}html body h1,html body h2,html body h3,html body h4,html body h5{font-weight:600}html body h5{font-size:1em}html body h6{color:#5c5c5c}html body strong{color:#000}html body del{color:#5c5c5c}html body a:not([href]){color:inherit;text-decoration:none}html body a{color:#08c;text-decoration:none}html body a:hover{color:#00a3f5;text-decoration:none}html body img{max-width:100%}html body>p{margin-top:0;margin-bottom:16px;word-wrap:break-word}html body>ul,html body>ol{margin-bottom:16px}html body ul,html body ol{padding-left:2em}html body ul.no-list,html body ol.no-list{padding:0;list-style-type:none}html body ul ul,html body ul ol,html body ol ol,html body ol ul{margin-top:0;margin-bottom:0}html body li{margin-bottom:0}html body li.task-list-item{list-style:none}html body li>p{margin-top:0;margin-bottom:0}html body .task-list-item-checkbox{margin:0 .2em .25em -1.8em;vertical-align:middle}html body .task-list-item-checkbox:hover{cursor:pointer}html body blockquote{margin:16px 0;font-size:inherit;padding:0 15px;color:#5c5c5c;background-color:#f0f0f0;border-left:4px solid #d6d6d6}html body blockquote>:first-child{margin-top:0}html body blockquote>:last-child{margin-bottom:0}html body hr{height:4px;margin:32px 0;background-color:#d6d6d6;border:0 none}html body table{margin:10px 0 15px 0;border-collapse:collapse;border-spacing:0;display:block;width:100%;overflow:auto;word-break:normal;word-break:keep-all}html body table th{font-weight:bold;color:#000}html body table td,html body table th{border:1px solid #d6d6d6;padding:6px 13px}html body dl{padding:0}html body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:bold}html body dl dd{padding:0 16px;margin-bottom:16px}html body code{font-family:Menlo,Monaco,Consolas,'Courier New',monospace;font-size:.85em !important;color:#000;background-color:#f0f0f0;border-radius:3px;padding:.2em 0}html body code::before,html body code::after{letter-spacing:-0.2em;content:"\00a0"}html body pre>code{padding:0;margin:0;font-size:.85em !important;word-break:normal;white-space:pre;background:transparent;border:0}html body .highlight{margin-bottom:16px}html body .highlight pre,html body pre{padding:1em;overflow:auto;font-size:.85em !important;line-height:1.45;border:#d6d6d6;border-radius:3px}html body .highlight pre{margin-bottom:0;word-break:normal}html body pre code,html body pre tt{display:inline;max-width:initial;padding:0;margin:0;overflow:initial;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}html body pre code:before,html body pre tt:before,html body pre code:after,html body pre tt:after{content:normal}html body p,html body blockquote,html body ul,html body ol,html body dl,html body pre{margin-top:0;margin-bottom:16px}html body kbd{color:#000;border:1px solid #d6d6d6;border-bottom:2px solid #c7c7c7;padding:2px 4px;background-color:#f0f0f0;border-radius:3px}@media print{html body{background-color:#fff}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{color:#000;page-break-after:avoid}html body blockquote{color:#5c5c5c}html body pre{page-break-inside:avoid}html body table{display:table}html body img{display:block;max-width:100%;max-height:100%}html body pre,html body code{word-wrap:break-word;white-space:pre}}.markdown-preview{width:100%;height:100%;box-sizing:border-box}.markdown-preview .pagebreak,.markdown-preview .newpage{page-break-before:always}.markdown-preview pre.line-numbers{position:relative;padding-left:3.8em;counter-reset:linenumber}.markdown-preview pre.line-numbers>code{position:relative}.markdown-preview pre.line-numbers .line-numbers-rows{position:absolute;pointer-events:none;top:1em;font-size:100%;left:0;width:3em;letter-spacing:-1px;border-right:1px solid #999;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.markdown-preview pre.line-numbers .line-numbers-rows>span{pointer-events:none;display:block;counter-increment:linenumber}.markdown-preview pre.line-numbers .line-numbers-rows>span:before{content:counter(linenumber);color:#999;display:block;padding-right:.8em;text-align:right}.markdown-preview .mathjax-exps .MathJax_Display{text-align:center !important}.markdown-preview:not([for="preview"]) .code-chunk .btn-group{display:none}.markdown-preview:not([for="preview"]) .code-chunk .status{display:none}.markdown-preview:not([for="preview"]) .code-chunk .output-div{margin-bottom:16px}.scrollbar-style::-webkit-scrollbar{width:8px}.scrollbar-style::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}.scrollbar-style::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,0.66);border:4px solid rgba(150,150,150,0.66);background-clip:content-box}html body[for="html-export"]:not([data-presentation-mode]){position:relative;width:100%;height:100%;top:0;left:0;margin:0;padding:0;overflow:auto}html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{position:relative;top:0}@media screen and (min-width:914px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{padding:2em calc(50% - 457px + 2em)}}@media screen and (max-width:914px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{font-size:14px !important;padding:1em}}@media print{html body[for="html-export"]:not([data-presentation-mode]) #sidebar-toc-btn{display:none}}html body[for="html-export"]:not([data-presentation-mode]) #sidebar-toc-btn{position:fixed;bottom:8px;left:8px;font-size:28px;cursor:pointer;color:inherit;z-index:99;width:32px;text-align:center;opacity:.4}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] #sidebar-toc-btn{opacity:1}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc{position:fixed;top:0;left:0;width:300px;height:100%;padding:32px 0 48px 0;font-size:14px;box-shadow:0 0 4px rgba(150,150,150,0.33);box-sizing:border-box;overflow:auto;background-color:inherit}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar{width:8px}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,0.66);border:4px solid rgba(150,150,150,0.66);background-clip:content-box}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc a{text-decoration:none}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc ul{padding:0 1.6em;margin-top:.8em}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc li{margin-bottom:.8em}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc ul{list-style-type:none}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{left:300px;width:calc(100% - 300px);padding:2em calc(50% - 457px - 150px);margin:0;box-sizing:border-box}@media screen and (max-width:1274px){html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{width:100%}}html body[for="html-export"]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .markdown-preview{left:50%;transform:translateX(-50%)}html body[for="html-export"]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .md-sidebar-toc{display:none}
/* Please visit the URL below for more information: */
/* https://shd101wyy.github.io/markdown-preview-enhanced/#/customize-css */
</style>
</head>
<body for="html-export">
<div class="mume markdown-preview ">
<h1 class="mume-header" id="realtime-fpga-video-filter">Realtime-FPGA-Video-Filter</h1>
<p>A realtime video filter on FPGA Nexys4.</p>
<ul>
<li><a href="#realtime-fpga-video-filter">Realtime-FPGA-Video-Filter</a>
<ul>
<li><a href="#%E5%AE%9E%E9%AA%8C%E7%AE%80%E4%BB%8B">实验简介</a></li>
<li><a href="#%E5%AE%9E%E9%AA%8C%E7%8E%AF%E5%A2%83">实验环境</a></li>
<li><a href="#%E5%B7%A5%E7%A8%8B%E7%BB%93%E6%9E%84">工程结构</a></li>
<li><a href="#%E7%AE%97%E6%B3%95%E5%8E%9F%E7%90%86%E5%8F%8A%E5%AE%9E%E7%8E%B0">算法原理及实现</a>
<ul>
<li><a href="#%E6%91%84%E5%83%8F%E6%9C%BA%E9%85%8D%E7%BD%AE">摄像机配置</a>
<ul>
<li><a href="#omnivision-%E4%B8%B2%E8%A1%8C%E6%91%84%E5%83%8F%E6%9C%BA%E6%8E%A7%E5%88%B6%E6%80%BB%E7%BA%BF">OmniVision 串行摄像机控制总线</a></li>
<li><a href="#ov7670-%E6%91%84%E5%83%8F%E6%9C%BA">OV7670 摄像机</a>
<ul>
<li><a href="#%E5%AF%84%E5%AD%98%E5%99%A8%E9%85%8D%E7%BD%AE">寄存器配置</a></li>
</ul>
</li>
<li><a href="#%E8%A7%86%E9%A2%91%E5%B8%A7%E9%87%87%E9%9B%86">视频帧采集</a></li>
</ul>
</li>
<li><a href="#%E5%9D%87%E5%80%BC%E6%BB%A4%E6%B3%A2-%E4%B8%8E-%E4%B8%AD%E5%80%BC%E6%BB%A4%E6%B3%A2">均值滤波 与 中值滤波</a>
<ul>
<li><a href="#%E5%9D%87%E5%80%BC%E6%BB%A4%E6%B3%A2">均值滤波</a></li>
<li><a href="#%E4%B8%AD%E5%80%BC%E6%BB%A4%E6%B3%A2">中值滤波</a></li>
</ul>
</li>
<li><a href="#%E7%81%B0%E5%BA%A6%E4%B8%8E%E5%8F%8D%E7%81%B0%E5%BA%A6">灰度与反灰度</a></li>
<li><a href="#sobel%E7%AE%97%E5%AD%90">Sobel算子</a>
<ul>
<li><a href="#%E5%8D%B7%E7%A7%AF%E6%93%8D%E4%BD%9C%E7%9A%84%E4%BB%A3%E7%A0%81%E5%AE%9E%E7%8E%B0">卷积操作的代码实现</a></li>
</ul>
</li>
<li><a href="#%E6%BB%91%E5%8A%A8%E7%AA%97%E5%8F%A3">滑动窗口</a></li>
<li><a href="#vga%E6%98%BE%E7%A4%BA">VGA显示</a></li>
</ul>
</li>
<li><a href="#%E6%95%88%E6%9E%9C%E6%BC%94%E7%A4%BA">效果演示</a>
<ul>
<li><a href="#%E5%8E%9F%E5%9B%BE%E5%83%8F">原图像</a></li>
<li><a href="#%E5%90%84%E5%B1%82%E6%95%88%E6%9E%9C">各层效果</a>
<ul>
<li><a href="#%E5%9D%87%E5%80%BC%E6%BB%A4%E6%B3%A2-%E5%92%8C-%E4%B8%AD%E5%80%BC%E6%BB%A4%E6%B3%A2">均值滤波 和 中值滤波</a></li>
<li><a href="#%E7%81%B0%E5%BA%A6%E5%8C%96-%E5%92%8C-%E5%8F%8D%E7%81%B0%E5%BA%A6%E5%8C%96">灰度化 和 反灰度化</a></li>
<li><a href="#%E8%BE%B9%E7%BC%98%E6%A3%80%E6%B5%8B%E6%8F%90%E5%8F%96">边缘检测提取</a></li>
</ul>
</li>
<li><a href="#%E4%B8%8D%E5%90%8C%E9%98%88%E5%80%BC%E7%9A%84%E7%BB%93%E6%9E%9C">不同阈值的结果</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h2 class="mume-header" id="%E5%AE%9E%E9%AA%8C%E7%AE%80%E4%BB%8B">实验简介</h2>
<p>本实验作为 <code>嵌入式系统设计方法</code> 课程的大作业, 利用了 FPGA 的高并行性质来进行视频流处理, 利用了<code>均值滤波</code>, <code>中值滤波</code>, <code>灰度化</code>(及<code>反灰度化</code>)等技术来处理图像, 并最终产生<code>图像边沿检测</code>结果.</p>
<h2 class="mume-header" id="%E5%AE%9E%E9%AA%8C%E7%8E%AF%E5%A2%83">实验环境</h2>
<ul>
<li>EDA 工具: Vivado 2020.2</li>
<li>实验平台: Windows 10</li>
<li>实验设备: Nexys 4 DDR xc7a100tcsg324-1 及 OV7670 摄像头</li>
</ul>
<h2 class="mume-header" id="%E5%B7%A5%E7%A8%8B%E7%BB%93%E6%9E%84">工程结构</h2>
<p>我们使用了像素流水线的方式组织图像的处理。OV 7670 capture module每次给出一个像素的颜色数据，通过各级流水线的图像处理算法可以对这个像素数据进行一定的处理，并将这个像素数据送入SRAM储存，最终通过VGA显示模块显示在显示器上。通过各级流水线前的MUX，可以很方便地选择数据通过的模块。</p>
<p>通过像素流水线可以最大程度的复用各种模块（例如Sobel模块需要灰度模块的预处理才能使用），这样有利于节约FPGA资源；并且通过像素流水线可以很方便地关闭和使用某个图像处理模块，从而便于对比每个图像处理算法的效果。</p>
<p><img src="./image/diagram.jpg" alt="diagram"></p>
<h2 class="mume-header" id="%E7%AE%97%E6%B3%95%E5%8E%9F%E7%90%86%E5%8F%8A%E5%AE%9E%E7%8E%B0">算法原理及实现</h2>
<h3 class="mume-header" id="%E6%91%84%E5%83%8F%E6%9C%BA%E9%85%8D%E7%BD%AE">摄像机配置</h3>
<p>实验中我们采用 OV7670 摄像头, 它的工作电压低，提供单片 VGA 摄像头和影像处理器的所有功能. 通过 SCCB 总线控制, 可以输出整帧, 子采<br>
样, 取窗口等方式的各种分辨率 8 位影响数据. 此外该产品 VGA 图像最高达到 30 帧/秒, 高于人眼可分辨频率. 另外, 这一款 OV7670 也能够同时兼容 I2C 协议来进行控制.<br>
<img src="image/ov7670.jpg" alt="Picture of Real Product of OV7670"></p>
<h4 class="mume-header" id="omnivision-%E4%B8%B2%E8%A1%8C%E6%91%84%E5%83%8F%E6%9C%BA%E6%8E%A7%E5%88%B6%E6%80%BB%E7%BA%BF">OmniVision 串行摄像机控制总线</h4>
<p>OmniVision 串行摄像机控制总线 (即 OmniVision Serial Camera Control Bus, 简称SCCB) 是一种专用于控制 OmniVision 摄像机芯片的协议, 它最主要的接口为 <code>SIO_C</code> 与 <code>SIO_D</code>.</p>
<ul>
<li>SCCB_E: 片选信号</li>
<li>SIO_C: 用以进行芯片的控制</li>
<li>SIO_D: 用以传输芯片数据</li>
<li>PWDN: OV 芯片的电源开关</li>
</ul>
<p>该协议主要依赖于一系列状态转换来完成数据读写.</p>
<ul>
<li>Phase 1 -- ID 地址: 设置设备 ID 地址, 其时序图如下:<br>
<img src="image/sccb_sequence_chart.png" alt="SCCB Sequence Chart"></li>
<li>Phase 2 -- 子地址 / 数据读取: 这个 phase 内可以读取数据, 或者选择子地址以期后续写入.</li>
<li>Phase 3 -- 写数据: 即写入数据, 最常用于初始化时配置寄存器.</li>
</ul>
<p>为了完成这一系列操作, 我们代码中的状态机状态主要有这样几个:</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token keyword">localparam</span> FSM_IDLE <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_START_SIGNAL <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_LOAD_BYTE <span class="token operator">=</span> <span class="token number">2</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_TX_BYTE_1 <span class="token operator">=</span> <span class="token number">3</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_TX_BYTE_2 <span class="token operator">=</span> <span class="token number">4</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_TX_BYTE_3 <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_TX_BYTE_4 <span class="token operator">=</span> <span class="token number">6</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_END_SIGNAL_1 <span class="token operator">=</span> <span class="token number">7</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_END_SIGNAL_2 <span class="token operator">=</span> <span class="token number">8</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_END_SIGNAL_3 <span class="token operator">=</span> <span class="token number">9</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_END_SIGNAL_4 <span class="token operator">=</span> <span class="token number">10</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_DONE <span class="token operator">=</span> <span class="token number">11</span><span class="token punctuation">;</span>
<span class="token keyword">localparam</span> FSM_TIMER <span class="token operator">=</span> <span class="token number">12</span><span class="token punctuation">;</span>
</pre><h4 class="mume-header" id="ov7670-%E6%91%84%E5%83%8F%E6%9C%BA">OV7670 摄像机</h4>
<p>本款摄像机的结构图如下所示:<br>
<img src="image/ov7670_circuit.png" alt="OV7670 Circuit and Interfaces"><br>
除了电源接口 <code>3.3V</code> 和 <code>GND</code> 以外, 我们需要控制的接口主要有:</p>
<ul>
<li>SCL 和 SDA: 分别对应于上述的 <code>SIO_C</code>, <code>SIO_D</code>. 此二者只需要按照上述 SCCB 协议的过程进行控制, 一般用于配置芯片寄存器.</li>
<li>VS(output): 帧同步信号, 用以读取图片的时候进行同步</li>
<li>HS(output): 行同步信号, 用以读取图片的时候进行同步</li>
<li>PLK(output): 像素时钟</li>
<li>XLK(input): 系统时钟</li>
<li>D7~D0: 分别是 SCCB 控制协议中的数据位, 共计 1 个字节</li>
<li>RET: 复位</li>
<li>PWDN: Power Down, OV 芯片的电源开关</li>
</ul>
<h5 class="mume-header" id="%E5%AF%84%E5%AD%98%E5%99%A8%E9%85%8D%E7%BD%AE">寄存器配置</h5>
<p>为了适配我们的实验, 配置 OV7670 的寄存器有一些需要特别注意的地方. 此外的其余配置大多只需要使用常用配置即可.</p>
<ul>
<li>MVFP(地址为 1E): 需要配置为水平镜像使能, 以期拍摄书籍等能够显示出正向的图片.</li>
<li>RGB444(地址为 8C): 虽然该 OV7670 能够提供 RGB565 的更为精确的图像色彩信息, 但由于我们的 Nexys4 DDR 开发板上的 VGA 接口仅能够支持 12 bit 的 RGB444 色彩输出, 因此我们用不上 RGB565, 所以需要使能 OV7670 的 RGB444 采集模式.</li>
</ul>
<p>要做到配置的更改, 我们需要修改寄存器配置 ROM 模块(<code>ov7670_config_rom_rgb444.v</code> 中的 <code>ov7670_config_rom_rgb444</code> 模块), 例如下面这句:</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token number">73</span><span class="token punctuation">:</span> dout <span class="token operator"><=</span> <span class="token number">16'h8c_02</span><span class="token punctuation">;</span> <span class="token comment">// RGB444 enable xRGB444</span>
</pre><h4 class="mume-header" id="%E8%A7%86%E9%A2%91%E5%B8%A7%E9%87%87%E9%9B%86">视频帧采集</h4>
<p>配置好了寄存器以后, 我们需要读取视频帧. OV7670 的视频帧读取非常简单, 每个 PCLK 时钟周期内, 我们都能够先后从 D7~D0 线上读取到像素的 16 个色彩比特. 然后每当读完一组, 我们只要根据手册给出的映射关系(如下图)读取数据就能获取该像素对应的色彩.<br>
<img src="image/ov7670_rgb444_sequence_chart.png" alt="OV7670 RGB444 Sequence Chart"><br>
整个视频帧采集的代码中, 最为核心的就是下面这一段. 每当读完整个像素的色彩信息, 就让地址 <code>address</code> 自增. 如果在读的只是前一半 (由 <code>pixel_half</code> 指示), 则只是单纯地将 <code>p_data</code> 写入到 <code>pixel_data[7:0]</code>.</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog">pixel_half <span class="token operator"><=</span> <span class="token operator">~</span> pixel_half<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>pixel_half<span class="token punctuation">)</span> pixel_data<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator"><=</span> p_data<span class="token punctuation">;</span>
<span class="token keyword">else</span> <span class="token keyword">begin</span>
pixel_data<span class="token punctuation">[</span><span class="token number">15</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator"><=</span> p_data<span class="token punctuation">;</span>
address <span class="token operator"><=</span> address <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token keyword">end</span>
</pre><h3 class="mume-header" id="%E5%9D%87%E5%80%BC%E6%BB%A4%E6%B3%A2-%E4%B8%8E-%E4%B8%AD%E5%80%BC%E6%BB%A4%E6%B3%A2">均值滤波 与 中值滤波</h3>
<p>二者滤波的主要原理都是以将要计算的像素点为中心, 选取一个卷积核(如 3x3 的卷积核), 并依据卷积核内的其他数据计算出该像素的最终数据.</p>
<p><img src="image/mean_and_mid_filter_diagram.jpeg" alt></p>
<p>所不同的是, 均值滤波中, 我们可以针对滤波目的来设定卷积核内各个像素点的权重, 而中值滤波则是简单地选取中位数.</p>
<h4 class="mume-header" id="%E5%9D%87%E5%80%BC%E6%BB%A4%E6%B3%A2">均值滤波</h4>
<p>在本实验中, 我们选取的均值滤波权重如下:</p>
<table style="vertical-align:middle;">
<tbody><tr><td>1</td><td>2</td><td>1</td></tr>
<tr><td>2</td><td>4</td><td>2</td></tr>
<tr><td>1</td><td>2</td><td>1</td></tr>
</tbody></table>
<p>之所以选择这样一组数据, 是因为这样能够做到每个数据都能用左移或右移来得到, 而不必做复杂的乘法, 并且最后除以总权重的时候, 也是除以 16, 这只需要右移 4 位, 或者直接取高位就能得到, 这是非常 verilog 友好的写法.</p>
<p>其中最核心的是下面这个代码. 这里 <code>matrix_??</code> 是读入的卷积核数据. 至于如何一次性读入处理这 9 个数据, 可参见后文的介绍.</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog">stage1_data_r <span class="token operator"><=</span>
<span class="token punctuation">(</span>matrix_00<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_01<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_02<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token punctuation">)</span>
<span class="token operator">+</span> <span class="token punctuation">(</span>matrix_10<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_11<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">2</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_12<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span>
<span class="token operator">+</span> <span class="token punctuation">(</span>matrix_20<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_21<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_22<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span>
stage1_data_g <span class="token operator"><=</span>
<span class="token punctuation">(</span>matrix_00<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_01<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_02<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token punctuation">)</span>
<span class="token operator">+</span> <span class="token punctuation">(</span>matrix_10<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_11<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">2</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_12<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span>
<span class="token operator">+</span> <span class="token punctuation">(</span>matrix_20<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_21<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_22<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span>
stage1_data_b <span class="token operator"><=</span>
<span class="token punctuation">(</span>matrix_00<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_01<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_02<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token punctuation">)</span>
<span class="token operator">+</span> <span class="token punctuation">(</span>matrix_10<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_11<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">2</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_12<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span>
<span class="token operator">+</span> <span class="token punctuation">(</span>matrix_20<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_21<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>matrix_22<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span>
stage1_addr <span class="token operator"><=</span> addr_pixel<span class="token punctuation">;</span>
</pre><h4 class="mume-header" id="%E4%B8%AD%E5%80%BC%E6%BB%A4%E6%B3%A2">中值滤波</h4>
<p>中值滤波也是类似的, 但为了在尽量少的时钟周期内获取中位数, 我们需要一个更好的算法. 这里我们参考了论文 <a href="http://www.c-s-a.org.cn/csa/article/pdf/20150542">改进中值滤波方法的图像预处理技术</a> 提出的方法来求取.</p>
<p>为了方便, 我们利用了 verilog 提供的 <code>task</code> 的形式来完成, 这里实现了 <code>get_max_mid_min</code>, <code>get_min</code>, <code>get_max</code>, <code>get_mid</code> 四个主要的 <code>task</code>, 这很大程度上增强了代码的可读性与易维护性.</p>
<p>在第一个时钟周期, 我们计算每行的 最小值, 最大值 和 中位数:</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token function">get_max_mid_min</span><span class="token punctuation">(</span>matrix_00<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> matrix_01<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> matrix_02<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> stage1_data_r_maxh0<span class="token punctuation">,</span> stage1_data_r_midh0<span class="token punctuation">,</span> stage1_data_r_minh0<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token function">get_max_mid_min</span><span class="token punctuation">(</span>matrix_10<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> matrix_11<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> matrix_12<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> stage1_data_r_maxh1<span class="token punctuation">,</span> stage1_data_r_midh1<span class="token punctuation">,</span> stage1_data_r_minh1<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token function">get_max_mid_min</span><span class="token punctuation">(</span>matrix_20<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> matrix_21<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> matrix_22<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span><span class="token punctuation">,</span> stage1_data_r_maxh2<span class="token punctuation">,</span> stage1_data_r_midh2<span class="token punctuation">,</span> stage1_data_r_minh2<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>
stage1_addr <span class="token operator"><=</span> addr_pixel<span class="token punctuation">;</span>
</pre><p>第二个时钟周期中, 我们计算上述最大值中的最小值, 最小值中的最大值, 中位数中的中位数:</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token comment">// min of the maxes in each line</span>
<span class="token function">get_min</span><span class="token punctuation">(</span>stage1_data_r_maxh0<span class="token punctuation">,</span> stage1_data_r_maxh1<span class="token punctuation">,</span> stage1_data_r_maxh2<span class="token punctuation">,</span> stage2_r_minmax<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>
<span class="token comment">// mid of the mids in each lin</span>
<span class="token function">get_min</span><span class="token punctuation">(</span>stage1_data_r_midh0<span class="token punctuation">,</span> stage1_data_r_midh1<span class="token punctuation">,</span> stage1_data_r_midh2<span class="token punctuation">,</span> stage2_r_midmid<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>
<span class="token comment">// max of the mins in each line</span>
<span class="token function">get_max</span><span class="token punctuation">(</span>stage1_data_r_minh0<span class="token punctuation">,</span> stage1_data_r_minh1<span class="token punctuation">,</span> stage1_data_r_minh2<span class="token punctuation">,</span> stage2_r_maxmin<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>
</pre><p>第三个时钟周期中, 我们计算第二步得到的值的中位数, 即为整个卷积核的中位数:</p>
<pre data-role="codeBlock" data-info class="language-"><code>// mid of minmax, midmid, maxmin
get_mid(stage2_r_minmax, stage2_r_midmid, stage2_r_maxmin, stage3_data_r);
get_mid(stage2_g_minmax, stage2_g_midmid, stage2_g_maxmin, stage3_data_g);
get_mid(stage2_b_minmax, stage2_b_midmid, stage2_b_maxmin, stage3_data_b);
stage3_addr <= stage2_addr;
</code></pre><h3 class="mume-header" id="%E7%81%B0%E5%BA%A6%E4%B8%8E%E5%8F%8D%E7%81%B0%E5%BA%A6">灰度与反灰度</h3>
<p>相比于滤波, 灰度与反灰度的处理则相对简单.</p>
<p>著名心理学公式给出, 对人眼而言最为舒适的灰度图满足</p>
<pre data-role="codeBlock" data-info="python" class="language-python">Gray <span class="token operator">=</span> Red <span class="token operator">*</span> <span class="token number">0.299</span> <span class="token operator">+</span> G <span class="token operator">*</span> <span class="token number">0.587</span> <span class="token operator">+</span> B <span class="token operator">*</span> <span class="token number">0.114</span>
</pre><p>理论上, 我们针对每个像素, 按照这个公式进行处理, 就能得到最优的灰度图. 但是对于 FPGA 来说, 过多地使用浮点数乘除法显然会加大它的负担. 鉴于我们的硬件资源限制, 我们原本能够提供的色彩失真也比较大(只有 RGB444 可供使用), 因此我们不必如此精确. 经过各种选择的调试, 最终我们采用的灰度计算公式如下:</p>
<pre data-role="codeBlock" data-info="python" class="language-python">Gray <span class="token operator">=</span> <span class="token punctuation">(</span>Red <span class="token operator">>></span> <span class="token number">1</span> <span class="token operator">+</span> Green <span class="token operator">*</span> <span class="token number">5</span> <span class="token operator">+</span> Blue<span class="token punctuation">)</span> <span class="token operator">>></span> <span class="token number">3</span>
</pre><p>在这个公式下, 我们既可以完成一个比较好的灰度滤镜, 也可以尽可能小地占用硬件资源.</p>
<p>核心代码如下:</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token keyword">assign</span> tmp_gray <span class="token operator">=</span> data<span class="token punctuation">[</span><span class="token number">11</span><span class="token punctuation">:</span><span class="token number">8</span><span class="token punctuation">]</span> <span class="token operator">*</span> <span class="token number">2</span> <span class="token operator">+</span> data<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span> <span class="token operator">*</span> <span class="token number">5</span> <span class="token operator">+</span> data<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
<span class="token keyword">assign</span> gray <span class="token operator">=</span> <span class="token number">4'hf</span> <span class="token operator">-</span> tmp_gray<span class="token punctuation">[</span><span class="token number">6</span><span class="token punctuation">:</span><span class="token number">3</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
<span class="token keyword">assign</span> gray_pixel <span class="token operator">=</span> <span class="token operator">{</span>gray<span class="token punctuation">,</span> gray<span class="token punctuation">,</span> gray<span class="token operator">}</span><span class="token punctuation">;</span>
<span class="token keyword">assign</span> gray_pixel_dark <span class="token operator">=</span> <span class="token operator">{</span>tmp_gray<span class="token punctuation">[</span><span class="token number">6</span><span class="token punctuation">:</span><span class="token number">3</span><span class="token punctuation">]</span><span class="token punctuation">,</span> tmp_gray<span class="token punctuation">[</span><span class="token number">6</span><span class="token punctuation">:</span><span class="token number">3</span><span class="token punctuation">]</span><span class="token punctuation">,</span> tmp_gray<span class="token punctuation">[</span><span class="token number">6</span><span class="token punctuation">:</span><span class="token number">3</span><span class="token punctuation">]</span><span class="token operator">}</span><span class="token punctuation">;</span>
</pre><p>由于逻辑比较简单, 就采用组合电路来完成.</p>
<h3 class="mume-header" id="sobel%E7%AE%97%E5%AD%90">Sobel算子</h3>
<p>Sobel算子通常用于图像的边缘提取。在技术上，它是一离散性差分算子，用来运算图像亮度函数的梯度之近似值。在概念上，Sobel算子就是一个小且是整数的滤波器对整张影像在水平及垂直方向上做卷积，因此它所需的运算资源相对较少，另一方面，对于影像中的频率变化较高的地方，它所得的梯度之近似值也比较粗糙。</p>
<p>由于使用的是FPGA，所以在资源利用率方面要求比较高，所以最终选择了简单且资源占用较少的Sobel算子作为边缘提取的核心。</p>
<p>该算子包含两组3x3的矩阵，分别为横向及纵向。</p>
<img src="image/sobel.png" alt="img" style="zoom:50%;">
<p>使用一个<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>的滑动窗口(关于滑动窗口的实现可参考<a href="#%E6%BB%91%E5%8A%A8%E7%AA%97%E5%8F%A3">滑动窗口</a>)，从原始图像中取出<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>的图像矩阵<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>A</mi></mrow><annotation encoding="application/x-tex">A</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal">A</span></span></span></span>并分别与<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>G</mi><mi>x</mi></msub><mo separator="true">,</mo><msub><mi>G</mi><mi>y</mi></msub></mrow><annotation encoding="application/x-tex">G_x,G_y</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.969438em;vertical-align:-0.286108em;"></span><span class="mord"><span class="mord mathnormal">G</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">x</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathnormal">G</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.15139200000000003em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.286108em;"><span></span></span></span></span></span></span></span></span></span>做卷积操作。</p>
<p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>A</mi><mi>x</mi></msub><mo>=</mo><msub><mi>G</mi><mi>x</mi></msub><mo>∗</mo><mi>A</mi><mspace linebreak="newline"></mspace><msub><mi>A</mi><mi>y</mi></msub><mo>=</mo><msub><mi>G</mi><mi>y</mi></msub><mo>∗</mo><mi>A</mi></mrow><annotation encoding="application/x-tex">A_x = G_x*A \\ A_y=G_y*A</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.83333em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">A</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">x</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.83333em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">G</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">x</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal">A</span></span><span class="mspace newline"></span><span class="base"><span class="strut" style="height:0.969438em;vertical-align:-0.286108em;"></span><span class="mord"><span class="mord mathnormal">A</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.15139200000000003em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.286108em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.969438em;vertical-align:-0.286108em;"></span><span class="mord"><span class="mord mathnormal">G</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.15139200000000003em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.286108em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal">A</span></span></span></span></span></p>
<blockquote>
<p>卷积操作的实现可以参考如下动图</p>
<p><img src="image/convolution.gif" alt="图像卷积运算示例"></p>
</blockquote>
<p>图像的每一个像素的横向及纵向梯度近似值可用以下的公式结合，来计算梯度的大小。</p>
<p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>G</mi><mo>=</mo><msqrt><mrow><msup><msub><mi>A</mi><mi>x</mi></msub><mn>2</mn></msup><mo>+</mo><msup><msub><mi>A</mi><mi>y</mi></msub><mn>2</mn></msup></mrow></msqrt></mrow><annotation encoding="application/x-tex">G=\sqrt{{A_x}^2+{A_y}^2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal">G</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1.84em;vertical-align:-0.5055099999999999em;"></span><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.3344900000000002em;"><span class="svg-align" style="top:-3.8em;"><span class="pstrut" style="height:3.8em;"></span><span class="mord" style="padding-left:1em;"><span class="mord"><span class="mord"><span class="mord"><span class="mord mathnormal">A</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">x</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.887338em;"><span style="top:-3.1362300000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord mathnormal">A</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.15139200000000003em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.286108em;"><span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.887338em;"><span style="top:-3.1362300000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span><span style="top:-3.2944899999999997em;"><span class="pstrut" style="height:3.8em;"></span><span class="hide-tail" style="min-width:1.02em;height:1.8800000000000001em;"><svg width="400em" height="1.8800000000000001em" viewBox="0 0 400000 1944" preserveAspectRatio="xMinYMin slice"><path d="M983 90
l0 -0
c4,-6.7,10,-10,18,-10 H400000v40
H1013.1s-83.4,268,-264.1,840c-180.7,572,-277,876.3,-289,913c-4.7,4.7,-12.7,7,-24,7
s-12,0,-12,0c-1.3,-3.3,-3.7,-11.7,-7,-25c-35.3,-125.3,-106.7,-373.3,-214,-744
c-10,12,-21,25,-33,39s-32,39,-32,39c-6,-5.3,-15,-14,-27,-26s25,-30,25,-30
c26.7,-32.7,52,-63,76,-91s52,-60,52,-60s208,722,208,722
c56,-175.3,126.3,-397.3,211,-666c84.7,-268.7,153.8,-488.2,207.5,-658.5
c53.7,-170.3,84.5,-266.8,92.5,-289.5z
M1001 80h400000v40h-400000z"/></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.5055099999999999em;"><span></span></span></span></span></span></span></span></span></span></p>
<p>由于FPGA的硬件资源有限，所以通常不使用开平方操作，而是使用以下公式近似</p>
<p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>G</mi><mo>=</mo><mi mathvariant="normal">∣</mi><msub><mi>A</mi><mi>x</mi></msub><mi mathvariant="normal">∣</mi><mo>+</mo><mi mathvariant="normal">∣</mi><msub><mi>A</mi><mi>y</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">G=|A_x| + |A_y|</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal">G</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">A</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">x</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.036108em;vertical-align:-0.286108em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">A</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.15139200000000003em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.286108em;"><span></span></span></span></span></span></span><span class="mord">∣</span></span></span></span></span></p>
<p>最后使用一个阈值来判断该像素位置的值是否为边缘，如果大于该阈值则该像素显示为黑色，否则显示为白色。</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog">data_out <span class="token operator"><=</span> <span class="token punctuation">(</span>final_data <span class="token operator">></span> threshold<span class="token punctuation">[</span><span class="token number">7</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">?</span> <span class="token number">12'h000</span> <span class="token punctuation">:</span> <span class="token number">12'hfff</span><span class="token punctuation">;</span>
</pre><h4 class="mume-header" id="%E5%8D%B7%E7%A7%AF%E6%93%8D%E4%BD%9C%E7%9A%84%E4%BB%A3%E7%A0%81%E5%AE%9E%E7%8E%B0">卷积操作的代码实现</h4>
<p>由于卷积操作涉及多个数字的相加和相乘，所以为了方式时序问题，我们采用流水线的方式实现卷积操作，第一个时钟周期计算每一行卷积的结果，第二个时钟周期计算每一个卷积核的卷积结果，第三个周期计算两个卷积核的卷积结果的绝对值之和。</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token important">always@</span><span class="token punctuation">(</span><span class="token keyword">posedge</span> CLK_100MHZ<span class="token punctuation">)</span>
<span class="token keyword">begin</span>
stage1_data_line1_x <span class="token operator"><=</span> martix_02<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">-</span> martix_00<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
stage1_data_line2_x <span class="token operator"><=</span> <span class="token punctuation">(</span>martix_12<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">-</span> martix_10<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">;</span>
stage1_data_line3_x <span class="token operator"><=</span> martix_22<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">-</span> martix_20<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
stage1_data_line1_y <span class="token operator"><=</span> martix_00<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">-</span> martix_20<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
stage1_data_line2_y <span class="token operator"><=</span> <span class="token punctuation">(</span>martix_01<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">-</span> martix_02<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator"><<</span> <span class="token number">1</span><span class="token punctuation">;</span>
stage1_data_line3_y <span class="token operator"><=</span> martix_02<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">-</span> martix_22<span class="token punctuation">[</span><span class="token number">3</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
stage2_data_x <span class="token operator"><=</span> stage1_data_line1_x <span class="token operator">+</span> stage1_data_line2_x <span class="token operator">+</span> stage1_data_line3_x<span class="token punctuation">;</span>
stage2_data_y <span class="token operator"><=</span> stage1_data_line1_y <span class="token operator">+</span> stage1_data_line2_y <span class="token operator">+</span> stage1_data_line3_y<span class="token punctuation">;</span>
<span class="token keyword">end</span>
<span class="token keyword">assign</span> final_data <span class="token operator">=</span> <span class="token punctuation">(</span>stage2_data_x <span class="token operator">></span> <span class="token number">0</span> <span class="token operator">?</span> stage2_data_x <span class="token punctuation">:</span> <span class="token operator">-</span>stage2_data_x<span class="token punctuation">)</span> <span class="token operator">+</span>
<span class="token punctuation">(</span>stage2_data_y <span class="token operator">></span> <span class="token number">0</span> <span class="token operator">?</span> stage2_data_y <span class="token punctuation">:</span> <span class="token operator">-</span>stage2_data_y<span class="token punctuation">)</span><span class="token punctuation">;</span>
</pre><h3 class="mume-header" id="%E6%BB%91%E5%8A%A8%E7%AA%97%E5%8F%A3">滑动窗口</h3>
<p>图片的像素数据被存入一个FIFO，我们只要从FIFO中特定的位置取出像素数据，就可以得到图片中任意位置的像素。</p>
<p>由于Sobel、均值滤波和中值滤波需要使用<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>的图像矩阵，所以滑动窗口模块用于每次从图像中取出纵向3个像素的值，通过3个周期的时间，就可以取出一个完整的<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>矩阵，如下图所示（下图中它一次性取出连续的3个数据，所以不需要3个周期就可以取出3*3的图像矩阵）。</p>
<p><img src="image/photo.png" alt="img"></p>
<p>使用Vivado提供的shift ram可以简单的实现滑动窗口。shift ram类似于一个限定了大小的FIFO，通过将3个大小为640的shift ram首尾相接，就可以同时缓存3行的视频像素数据。图解如下：</p>
<p><img src="image/shiftram.jpg" alt></p>
<p>通过每次读取pixel data out[2:0]中的数据，就可以获得一列3个像素的颜色数据，再将其存入一个<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>移位寄存器，就可以得到一个<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>矩阵。</p>
<p>实现<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>3</mn><mo>∗</mo><mn>3</mn></mrow><annotation encoding="application/x-tex">3*3</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">3</span></span></span></span>矩阵的代码如下</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token important">always@</span><span class="token punctuation">(</span><span class="token keyword">posedge</span> valid_pause<span class="token punctuation">)</span>
<span class="token keyword">begin</span>
martix_02 <span class="token operator"><=</span> line1<span class="token punctuation">;</span>
martix_01 <span class="token operator"><=</span> martix_02<span class="token punctuation">;</span>
martix_00 <span class="token operator"><=</span> martix_01<span class="token punctuation">;</span>
martix_12 <span class="token operator"><=</span> line2<span class="token punctuation">;</span>
martix_11 <span class="token operator"><=</span> martix_12<span class="token punctuation">;</span>
martix_10 <span class="token operator"><=</span> martix_11<span class="token punctuation">;</span>
martix_22 <span class="token operator"><=</span> line3<span class="token punctuation">;</span>
martix_21 <span class="token operator"><=</span> martix_22<span class="token punctuation">;</span>
martix_20 <span class="token operator"><=</span> martix_21<span class="token punctuation">;</span>
<span class="token keyword">end</span>
<span class="token comment">// 实现移位寄存器，缓存3行数据，每周期输出一列3个像素数据line1,line2,line3</span>
<span class="token function">line_cache</span><span class="token punctuation">(</span>
<span class="token punctuation">.</span><span class="token function">CLK_100MHZ</span><span class="token punctuation">(</span>CLK_100MHZ<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">RST</span><span class="token punctuation">(</span>RST<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">valid</span><span class="token punctuation">(</span>valid_pause<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">data</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">address</span><span class="token punctuation">(</span>addr<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">line1</span><span class="token punctuation">(</span>line1<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">line2</span><span class="token punctuation">(</span>line2<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">line3</span><span class="token punctuation">(</span>line3<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">addr_out</span><span class="token punctuation">(</span>addr_pixel<span class="token punctuation">)</span>
<span class="token punctuation">)</span><span class="token punctuation">;</span>
</pre><h3 class="mume-header" id="vga%E6%98%BE%E7%A4%BA">VGA显示</h3>
<p>我们使用的是<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>640</mn><mo>∗</mo><mn>480</mn></mrow><annotation encoding="application/x-tex">640*480</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">6</span><span class="mord">4</span><span class="mord">0</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">4</span><span class="mord">8</span><span class="mord">0</span></span></span></span> 60fps的VGA显示，该显示格式的VGA时序如下。</p>
<table>
<thead>
<tr>
<th style="text-align:left">Format</th>
<th style="text-align:left">Pixel Clock (MHz)</th>
<th style="text-align:left">Active Video</th>
<th style="text-align:left">Front Porch</th>
<th style="text-align:left">Sync Pulse</th>
<th style="text-align:left">Back Porch</th>
<th style="text-align:left">Active Video</th>
<th style="text-align:left">Front Porch</th>
<th>Sync Pulse</th>
<th>Back Porch</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:left">640x480, 60Hz</td>
<td style="text-align:left">25.175</td>
<td style="text-align:left">640</td>
<td style="text-align:left">16</td>
<td style="text-align:left">96</td>
<td style="text-align:left">48</td>
<td style="text-align:left">480</td>
<td style="text-align:left">11</td>
<td>2</td>
<td>31</td>
</tr>
</tbody>
</table>
<p>该模块从一个<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>640</mn><mo>∗</mo><mn>480</mn></mrow><annotation encoding="application/x-tex">640*480</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">6</span><span class="mord">4</span><span class="mord">0</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">4</span><span class="mord">8</span><span class="mord">0</span></span></span></span>的SRAM中读取像素的颜色信息，并将结果显示到外接的VGA显示器上。</p>
<p>通过VGA模块内部计算得到的当前正在扫描的像素坐标，从SRAM中取出对应像素位置的颜色信息。</p>
<p>模块的使用如下：</p>
<pre data-role="codeBlock" data-info="verilog" class="language-verilog"><span class="token comment">// VGA control</span>
<span class="token keyword">wire</span> active<span class="token punctuation">;</span>
<span class="token keyword">wire</span> <span class="token punctuation">[</span><span class="token number">9</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> x<span class="token punctuation">;</span>
<span class="token keyword">wire</span> <span class="token punctuation">[</span><span class="token number">8</span><span class="token punctuation">:</span><span class="token number">0</span><span class="token punctuation">]</span> y<span class="token punctuation">;</span>
vga640x480 display <span class="token punctuation">(</span>
<span class="token punctuation">.</span><span class="token function">clk</span><span class="token punctuation">(</span>CLK_100MHZ<span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token comment">// 100Mhz board clock</span>
<span class="token punctuation">.</span><span class="token function">clk_vga</span><span class="token punctuation">(</span>CLK_25MHZ<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">//25Mhz clock for Pixel Clock</span>
<span class="token punctuation">.</span><span class="token function">rst</span><span class="token punctuation">(</span>RST<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">.</span><span class="token function">hs</span><span class="token punctuation">(</span>VGA_HS<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// H Sync for vga port</span>
<span class="token punctuation">.</span><span class="token function">vs</span><span class="token punctuation">(</span>VGA_VS<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// V Sync for vga port</span>
<span class="token punctuation">.</span><span class="token function">x</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// current display x coordinate on vga monitor</span>
<span class="token punctuation">.</span><span class="token function">y</span><span class="token punctuation">(</span>y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// current display y coordinate on vga monitor</span>
<span class="token punctuation">.</span><span class="token function">active</span><span class="token punctuation">(</span>active<span class="token punctuation">)</span> <span class="token comment">// high when not in blanking interval</span>
<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token important">always @</span><span class="token punctuation">(</span><span class="token keyword">posedge</span> CLK_100MHZ<span class="token punctuation">)</span>
<span class="token keyword">begin</span>
r_addr <span class="token operator"><=</span> y <span class="token operator">*</span> <span class="token number">640</span> <span class="token operator">+</span> x<span class="token punctuation">;</span> <span class="token comment">// calculate sram address for pixel data</span>
<span class="token keyword">end</span>
<span class="token keyword">assign</span> <span class="token operator">{</span>VGA_R<span class="token punctuation">,</span> VGA_G<span class="token punctuation">,</span> VGA_B<span class="token operator">}</span> <span class="token operator">=</span> active <span class="token operator">?</span> sram_out <span class="token punctuation">:</span> <span class="token number">12'h000</span><span class="token punctuation">;</span> <span class="token comment">// display color</span>
</pre><h2 class="mume-header" id="%E6%95%88%E6%9E%9C%E6%BC%94%E7%A4%BA">效果演示</h2>
<p>我们以拍摄<code>农夫山泉瓶装矿泉水</code>的结果作为演示.</p>
<h3 class="mume-header" id="%E5%8E%9F%E5%9B%BE%E5%83%8F">原图像</h3>
<table>
<tbody><tr>
<td><p><img src="image/origin.jpg">原图</p></td>
</tr>
</tbody></table>
<h3 class="mume-header" id="%E5%90%84%E5%B1%82%E6%95%88%E6%9E%9C">各层效果</h3>
<h4 class="mume-header" id="%E5%9D%87%E5%80%BC%E6%BB%A4%E6%B3%A2-%E5%92%8C-%E4%B8%AD%E5%80%BC%E6%BB%A4%E6%B3%A2">均值滤波 和 中值滤波</h4>
<table>
<tbody><tr>
<td><p><img src="image/mean.jpg">均值滤波</p></td>
<td><p><img src="image/mid.jpg">中值滤波</p></td>
</tr>
</tbody></table>
<h4 class="mume-header" id="%E7%81%B0%E5%BA%A6%E5%8C%96-%E5%92%8C-%E5%8F%8D%E7%81%B0%E5%BA%A6%E5%8C%96">灰度化 和 反灰度化</h4>
<table>
<tbody><tr>
<td><p><img src="image/antigray.jpg">反灰度化</p></td>
<td><p><img src="image/gray.jpg">灰度化</p></td>
</tr>
</tbody></table>
<h4 class="mume-header" id="%E8%BE%B9%E7%BC%98%E6%A3%80%E6%B5%8B%E6%8F%90%E5%8F%96">边缘检测提取</h4>
<table>
<tbody><tr>
<td><p><img src="image/edge.jpg">边缘检测提取(threshold=128)</p></td>
</tr>
</tbody></table>
<h3 class="mume-header" id="%E4%B8%8D%E5%90%8C%E9%98%88%E5%80%BC%E7%9A%84%E7%BB%93%E6%9E%9C">不同阈值的结果</h3>
<p>另外, 作为对比, 在阈值分别为 128, 32, 8, 2 的情况下, 有下列结果:</p>
<table>
<tbody><tr>
<td><p><img src="image/edge.jpg">threshold=128</p></td>
<td><p><img src="image/edge32.jpg">threshold=32</p></td>
</tr>
<tr>
<td><p><img src="image/edge8.jpg">threshold=8</p></td>
<td><p><img src="image/edge2.jpg">threshold=2</p></td>
</tr>
</tbody></table>
</div>
</body></html>