Altera Fpga Based Picture In Picture Application Information Technology Essay
Nowadays, many types of electronic devices are being used as you can see people around you may have a mobile at least and some of them may carry more than one of these gadgets; mobile phone, laptop, MP3 player, PDA, etc. Therefore, we might have to accept the truth that these equipments importantly influence on our day-to-day activities. Visual applications of the electronic tool are highly significant as well as the others because we can say that almost houses will have at least one of visual electronic device such as TV (Television Set), PC (Personal Computer), etc. The household appliance will have a monitor/screen and it can be watched very often every day thus we can attach some equipment to them and make them more useful than they have been using. The additional tools need to provide more functionality to the existing one.
Picture-in-picture is another interesting technology to merge into this application because we can display another source of image or video on the same display unit. The technology is basically used in TV to display another channel(s) in inset screen(s) at the same time as another full TV screen displaying.
Figure 1 – Picture-in-picture in TV sets.
To understand the design process, we need to do the experiment over and over until we obtained the most effective result before implementation of the real production. Thus this requirement of fast and flexible solution is vital, if we have to make the final product every time we have tested. It would not be appropriate in terms of cost and time wasting; therefore, field-programmable gate array (FPGA) can be another solution for the designing. We can design, debug, verify and simulate the prototype with many changes or algorithms using complex operations again and again within short period of time which can save the development time.
Figure 2 – Altera FPGA DE2-70 Development Board Layout and Peripherals
Altera FPGA DE2-70 Development Board is an option for this project design because the device has many peripherals which can provide almost adequate resources of project requirement.
Cyclone II EP2C70 FPGA
68,416 LEs
250 M4K RAM blocks
150 embedded multipliers
4 PLLs
622 user I/O pins
SDRAM
Two 32-Mbyte Single Data Rate Synchronous Dynamic RAM memory chips
Organized as 4M x 16 bits x 4 banks
Accessible as memory for the Nios II processor and by the DE2-70 Control Panel
VGA OUTPUT
Uses the ADV7123 240-MHz triple 10-bit high-speed video DAC
With 15-pin high-density D-sub connector
Supports up to 1600 x 1200 at 100-Hz refresh rate
NTSC/PAL/ SECAM TV DECODER
Uses two ADV7180 Multi-format SDTV Video Decoders
Supports worldwide NTSC/PAL/SECAM color demodulation
One 10-bit ADC, 4X over-sampling for CVBS
Supports Composite Video (CVBS) RCA jack input
Supports digital output formats : 8-bit ITU-R BT.656 YCrCb 4:2:2 output + HS, VS, and FIELD
The design would be based on digital signal processing (DSP) which also be involved with arithmetic calculation, memory management, analog-to-digital converter and digital-to-analog converter.
One of Hardware Description Language (HDL) will be used to control FPGA and Verilog is one of the most commonly used in the hierarchical design, verification and implementation of digital logic chips at the register transfer level ( RTL ). Verilog HDL has some similarity of C programming language which can be an advantage for designers who has experienced from the programming language. Verilog HDL allows designers to implement switches, RTL, gates and custom behavioral code within the same design.
CHAPTER II
REVIEW OF THE LITERATURE
2.1 COMPOSITE VIDEO TO DIGITAL VIDEO CONVERTER
In composite video system, the luminance and chrominance signal can be multiplexed within the same channel. The signal can be received by both of monochrome and colour receivers. The National Television Systems Committee (NTSC) systems take the advantage of the fact that the sensitivity of human eyes is the most at the green light, less at the red one and least at the blue one by using lower bandwidth for the chrominance (colour) signals compare to the luminance (brightness) signal. This solution can reduce the bandwidth of the transmission without losing the colour signal quality.
In general, there are four common steps to convert analog signals to digital ones. The system consists of filter, sampler, quantiser and encoder. The video signals are Y, Cr and Cb which can be discovered from these steps.
Figure 3 Conversion of component analog TV signals to digital TV signals
The definition of NTSC colour space is known as YIQ which consist of these components: luminance (Y), hue (I) and saturation (Q). YIQ can be converted to RGB components or YUV components by the following:
γ = 0.299R + 0.587G + 0.114B
І = 0.596R – 0.274G – 0.322B
= -(sin 33°)U + (cos 33°)V
Q = 0.211R – 0.523G + 0.311B
= (cos 33°)U + (sin 33°)V
From the above formulas, it has clearly seen that γ is being used for the intensity of the luminance in the colour space thus we can use the signal for grayscale image conversion.
There is another colour space called YCbCr which has been the scaling and offsetting of YUV colour space. The colour space is related to 8-bit RGB components as described below:
Y = 0.257R + 0.504G + 0.098B + 16
Cr = 0.439R – 0.368G – 0.071B + 128
Cb = -0.148R – 0.291G + 0.439B + 128
YCbCr could be more precise and it can be fitted in unsigned integer 8-bit data but Y is allowed to have the range of value from 16 to 235 and both of Cb and Cr can have value between 16 and 240. The sampling rate of the luminance is 13.5 MHz and the chrominance is 6.75 MHz. NTSC (30 frames/second) will have 720 active pels per line and 480 active lines per one frame.
CHROMA SUBSAMPLING
Due to the sensitivity of human to the luminance or the brightness, we can save the bandwidth of signals which are being transmitted by compressing the chrominance or colour signals. The sampling rate of 13.5MHz is being used for Y signal thus YCbCr can be designated as 4:n:n. The bandwidth of Cb and Cr should be less than 13.5 MHz or equal.
THE 4:2:2 FORMAT
This technique can reduce the bandwidth of the chrominance signals by half which means if a luminance sampling frequency is 13.5 MHz therefore chrominance ones would be 6.75 MHz. The method perform by sampling Cb and Cr one-half of the number of luminance frequency. The chrominance resolution will be vertically sampled as figure below.
Figure 4 : The YCbCr 4:2:2 Format
THE 4:4:4 FORMAT
This method will not attempt to reduce the usage of bandwidth and the additional bandwidth could not be clearly seen the difference however it is being widely used in colour space conversion. For instance, when we want to convert video format from one colour space to another, the signals need to be resampled in 4:4:4 format first. The luminance and the chrominance signals will be transmitted at the same frequency. “The 4:4:4 format is useful mainly in computer graphics applications, where full chrominance resolution is required not only in the vertical but also in the horizontal direction.”(Haskell, Puri, and Netravali, 2000, p. 94)
Figure 5 : The YCbCr 4:2:2 Format
INTERLACED AND NON-INTERLACED VIDEO
Interlaced video is represented by using 2 distinct images per frame on the same screen. One image exist on odd-numbered raster lines and another image is on even-numbered raster lines however both will be combined to exhibit a same image at the same frame.
Figure 6 : Interlaced Video Scanning
Non-interlaced video is a type of raster scanning which sequentially scans raster lines from top to bottom within each frame. The whole screen has only one particular image per frame. Therefore, all the vertically adjacent scan lines are also temporally adjacent as well as motion images. The video system is also known as ‘Progressive video’.
Figure 7 : Interlaced and Non-Interlaced Video Scanning
2.2 VIDEO GRAPHICS ARRAY (VGA)
VGA CONNECTOR
VGA connector is 3-row of 5 pins DE-15 connector which is also known as D-SUB 15 RGB connector. The connector is widely used in many computer monitor, video graphic cards and high definition television sets. Although the connector consists of 15 pins but there are only 5 important pins used to transmit video signals to the display devices.
Figure 8 : DE-15 or D-SUB 15 Connector
VGA VIDEO SIGNAL
There are 5 active analog signals will be transmitted through pin 1, 2, 3, 13 and 14 of DE-15 connector. The first two signals are horizontal and vertical synchronization signals which are compatible with TTL logic levels. The other three signals are R (red), G (green) and B (blue) and they have 0.7 to 1.0 Volt peak-to-peak level. Each signal will represent primary base colour; however, the difference and combination of the analog signals level will control the generation of video colour on VGA output devices.
Figure 9 : RGB Signals and Colours
VIDEO DISPLAY, REFRESH AND TIMING
VGA computer monitor is a progressive scan system therefore an image on the screen will be generated from a sequence of horizontal lines. For example, the screen resolution 640 by 480 image elements ( pixels ). The screen should be able to redraw the whole screen 60 times per second for an image and to reduce flicker. The period is well-known as “Refresh Rate”.
The flicker can be noticed by human eyes when the refresh rate is less than 30 – 60 Hz. PC monitors sometimes use 70 Hz of refresh rate rather than 60 Hz to reduce flicker from fluorescent light source interference. Every scanned pixel will be generated by the value of RGB signal. For 640×480 resolution and 60 Hz (16.67 ms) refresh rate, one pixel needs approximately 40 ns (25 MHz) thus 640 pixels will need ~25.60 µs the refresh rate will need to be increased for higher resolution by higher clock rate.
Figure 10 : VGA Signals and Timing
Table 1 : VGA Signals and Timing
A frame of image on monitor will be started from top left of the screen and one pixel at a particular time will be plotted on the screen from left to right once the last pixel of the row has been created. The row address will be increased and the column address will also be reset to the fist left column again. The row address has been controlled by the horizontal sync signal (HSYNC). The process continues until the row address come to the last row of the screen. At this time an entire frame of image has been successfully generated and both of column and row address will be reset to the top left position or pixel [0] [0] again by consulting the vertical sync signal (VSYNC).
Figure 11 : VGA HSYNC and VSYNC Signals
Mode
Resolution (HxV)
Refresh Rate
Pixel clock(MHz)
VGA
640×480
(60Hz)
25 (640/c)
VGA
640×480
(85Hz)
36 (640/c)
SVGA
800×600
(60Hz)
40 (800/c)
SVGA
800×600
(75Hz)
49 (800/c)
SVGA
800×600
(85Hz)
56 (800/c)
XGA
1024×768
(60Hz)
65 (1024/c)
XGA
1024×768
(70Hz)
75 (1024/c)
XGA
1024×768
(85Hz)
95 (1024/c)
1280×1024
1280×1024
(60Hz)
108 (1280/c)
Table 2 : VGA Mode, Resolution, Refresh Rate and Pixel Clock Rate
Computer monitor will detect the synchronous signals automatically and will not active if sync signals are not correct. Active mode of the screen can be indicated by observing the colour of an LED of the monitor. The LED colour will turn into green if the sync signals are correct but it will be orange when the sync signals are out of range.
2.3 DIGITAL IMAGE PROCESSING
A digital image can be represented as a two-dimensional function or M x N mathematic array. The M is the number of row and N is the number of column of an image frame. The function can also be described as f(x, y) when x is the column number start from top-left and y is the row number start from top of an image.
The coordination between x and y can be called pixel as well as the value at a particular pixel is well-known as intensity in monochrome image however the value will be formed by a combination of primary colour : red, green and blue in colour image.
Figure 12 : Digital Image Coordination as Array
Mode of colour can be categorized by the depth of pixel value. The following are some examples.
1-bit monochrome image : the value of each pixel will be either ‘0’ or ‘1’ only thus the image will have black and white colour. It is also known as bi-level, two-level, binary or black-and-white image.
Figure 13 : Binary Image
8-bit grayscale image : the value of each pixel can be any value from the range of 0 – 255. The weakest intensity will be considered as 0 or black and the strongest one will have a value of 255 or white.
Figure 14 : Grayscale Image
24-bit RGB image : each pixel will have values of three 8-bit RGB colour The combination will provide 24 bits of data per pixel which means each pixel will have an unsigned integer value 0 – 255.
32-bit RGB image : the image has 24-bit RGB value per pixel including an additional 8-bit for transparency. The image can overlay on another picture on the screen.
Figure 14 : 24-bit RGB Image
2.3.1 RGB TO GRAYSCALE CONVERSION
The intensity of each grayscale image’s pixel can be converted from RGB value by using the equation below:
Grayscale = 0.299R + 0.587G + 0.114B
2.3.2 GRAYSCALE TO BINARY CONVERSION
Binary image would be using in applications of image segmentation but we need to consider the intensity of the image which pixel should be judged as black or white pixel. To overcome the problem, thresholding will be consulted. There are two ways to determine threshold: global thresholding and local thresholding.
Figure 15 : Thresholding of Image Histogram
GLOBAL THRESHOLDING
The intensity histogram of an image can be used to separate objects from background. Pixels of object and background will have two different groups of intensity levels. To make a decision, a threshold (T) of an image needs to be selected and used to compare every pixel of the image.
AUTOMATIC GLOBAL THRESHOLDING
A threshold value (T) between the minimum and maximum intensity levels will be initially chosen and it will be able to segment the image into two groups
G1 : The intensity value is more than or equal T
G2 : The intensity value is less than T
Average intensity values of pixels in ranges G1 and G2 will be calculated and known as µ1 and µ2. The average intensity values will be used in the new threshold calculation and continues the above procedure until predefined threshold value is met.
One histogram-based method called “Otsu’s Algorithm” is being widely used to choose an optimum threshold value automatically. The algorithm believes that there are two different region values within an image (e.g. background and foreground image) and can be recognized by firstly starting looking for a discrete probability intensity level.
= Total number of pixels in the image.
= Number of pixels that have intensity level
L = Total number of possible intensity levels in the image
A threshold k will be selected for a set of intensities [0, 1,…, k-1] or C0 and C1 will represent a set of pixel intensity [k, k+1,… , L-1] . The algorithm will choose the maximum threshold value k from the between-class variance Æ 2B.
MANUAL GLOBAL THRESHOLDING
Threshold values can be chosen by trial and error until an appropriate value is observed. This process is normally done by observers and the method is good for interactive operation. However, indicators will need to be provided for the operators.
LOCAL THRESHOLDING
We can have an incorrect threshold value from the global thresholding method when the background illumination is not even. We need to solve the illumination problem by compensation or preprocessing the image before using the global thresholding method to calculate the threshold value of the picture.
when
Figure 16 : Grayscale and Binary Image
2.3.3 EDGE DETECTION
Th