Graphics

Pixels

At first glance, the word 'Pixels' here seems to have quite well-formed letters with smooth outlines. When we look more closely, however, we can see the individual ‘pixels’, or picture elements.

Similarly, the image of a camel looks quite smooth when it's small, but is actually very blocky from up close.

2-D graphics resolution

		No. pixels horizontally	No. pixels vertically	No. bits per pixel	Total no. Mbits
Telephone1		96	65	1
Palm/pocket2		160	160	8	0.2
		320	240	16	1.2
		640	240	16	2.4
X-ray CT		512	512	8	2
SVGA3		800	600	8	4
Laser printer4		3300	2550	1	8
Workstation		1600	1200	24	46
Digital camera 5 Mpixels		2560	1920	24	118
X-ray film		4000	4000	12	192
35-mm film		4000	3000	30	360
Digital camera 11 Mpixels		4064	2704	36	396
Typography5		28000	21000	1	588
70-mm film		6000	4600	30	830

¹ Nokia 7110 is 96×65, Ericsson R320s is 101×65, both include Web browsers.

² PalmOS-based PDA's have 160×160 screens. The Nokia 9000i telephone-cum-PDA has a 640×200 screen in addition to a small telephone-type screen. WindowsCE-based handheld computers typically have 640×240 screens.

³ CGA = 640×200×1, EGA = 640×350×4, VGA = 640×480×4, XGA = 1024×768.

⁴ At 300 dpi. Many printers are now 600 dpi or higher.

⁵ Approx. 11"×8.25" at 2540 dpi, black and white.

Prices for computer terminals, from 1985.

Display technology

Cathode-ray tube

• Storage vector
• Refreshed raster

LCD

Light is generated by a fluorescent backlight then filtered by the liquid-crystal molecules.

Plasma

Light is generated by phosphors.

Comparison of plasma with LCD:

• bigger
• better blacks
• better contrast in the dark
• better colour
• a bit faster
• heavy, fragile
• take more power (?)
• some burn-in

Some of these statements may be debatable, especially if you sell one type or the other.

E-paper

One technique is E Ink: microscopic charged black & white particles.

• no refresh rate problem
• viewed by reflected light
• low power
• thin, light, even flexible

Generation of raster-scan images

An image is created on a cathode-ray tube (CRT) by firing electrons at a chemical (the phosphor) on a glass screen. The electron beam is swept back and forth, and top to bottom, in a zigzag pattern.

The phosphor exhibits persistence, so when the screen is refreshed the pixels appear to be illuminated continuously.

Refresh rate

How often you need to refresh, to avoid flicker, depends on

• the persistence of the source, and
• the fusion characteristics of the eye.

Incandescent lights used to flicker with 25 Hz AC power, but not with modern 50 or 60 Hz.

These days it's recommended that computer monitors should be refreshed at 70 Hz or more.
With a million pixels on the screen, that requires very fast electronics!

Generation of straight lines

To generate a straight line from (x1,y1) to (x2,y2)

Generation of horizontal line from (0,0) to (20,0)

Generation of diagonal line from (0,0) to (10,10)

Generation of oblique line from (0,0) to (20,10)

Generation of oblique line from (0,0) to (5,10)

Generation of oblique line from (0,0) to (3,10)

The algorithm doesn't work in this simple form because of the fixed step size.

The algorithm is also computationally expensive because it involves floating-point multiplications and divisions.

This algorithm uses only integer arithmetic, and no multiplications or divisions.
Generating a straight line from (2,2) to (17,8)
and so on ...

Of course, it takes a lot more than one small straight line ...

Graphics primitives

• straight lines
• curved lines (e.g., arcs, splines)
• polygons (e.g., triangles)
• shading
• textures
• glyphs (characters, symbols)

And then there are 3-D primitives ...

Graphics processing units

GPU's are becoming complex and programmable.
(Haines, E (2006): An introductory tour of interactive rendering. IEEE Computer Graphics and Applications 26(1): 76- 87

Realistic 3-D rendering: occlusion, shading, textures, perspective, etc.

Being used in for non-graphics applications.

Colour combination

Subtractive

Used in colour printing: cyan, magenta & yellow primaries.

Additive

Used in light-emitting displays: red, green & blue primaries.

Colour display

In digital displays, typically 8 bits for each of red, green and blue.

Amount of video memory (MBytes) required for different resolutions:

Resolution		8 bits/pixel (256)	16 bits/pixel (65,536)	24 bits/pixel (‘true colour’)	32 bits/pixel
VGA1	640 × 480	0.31 MB	0.61 MB	0.92 MB	1.22
SVGA2	800 × 600	0.48 MB	0.96 MB	1.44 MB	1.92
XGA2	1024 × 768	0.79 MB	1.57 MB	2.36 MB	3.14
SXGA	1280 × 1024	1.31 MB	2.62 MB	3.93 MB	5.24
UXGA	1600 × 1200	1.92 MB	3.84 MB	5.76 MB	7.68
WUXGA	1920 × 1200	2.30 MB	4.61 MB	6.91 MB	9.22
NTSC TV	≤720 × 480 (240 lines interlaced)
HDTV	1280 × 720
	1920 × 1080

¹The original VGA had only 4 bits at 640×480.

²The original SVGA and XGA had only 8 bits.

In true-colour mode, 24 bits (3 bytes) are stored for every pixel. This gives millions of colours (2²⁴ = 16,777,216).

Memory can be saved by storing only 15 bits (2 bytes with 1 bit unused) for every pixel. This gives thousands of colours (2¹⁵ = 32,768).

Colour palettes

One can save a lot of memory by storing only 8 bits per pixel ...

... using a lookup table, or palette, to convert from 8 bits to 24 bits for each pixel.

The 8-bit value for a particular pixel is used as an address in the palette, and the 24 bits at that location define the red, green and blue values for that pixel.

With 8 bits, or 256 colours:

• 3/3/2 - 3 bits of red, 3 of green, 2 of blue (because darker)
• 5/6/5 - 5 shades of red, 6 of green, 5 of blue (6x7x6=252) plus some grays
• gray scale
• 'false colour'
• flesh tones
• shades of H&E, ...
• ...

Colours should be reserved for GUI, or left free for other programmes. Standard palette in Netscape uses only 216 colours.

Microsoft Internet Explorer has a different 216-colour 'standard' palette, just to be different.

To represent a colour which is not in the current palette, one can either choose the nearest colour, or use dithering.

This image actually uses only 41 colours, which are dithered to simulate 256 colours.

Various methods are available for dithering. The three images below were created by converting a 24-bit image to 8 bits with no dithering (i.e., using nearest colour), with ordered dithering, and with error-diffusion dithering, respectively.

The two images below show the different patterns produced by ordered dithering and error-diffusion dithering.

Colour gamut

Colour rendition and perception are not really as simple as just combining three primaries.

The eye's sensitivity to colour involves overlapping frequency curves.

The range of perceptible colours is represented in the CIE chromaticity diagram, using a special set of standardized coördinate axes.

(Cf. Eugene Vishnevsky's Introduction to Color)

Using any particular set of primary colours, the gamut of colours which can be rendered is delimited by a triangle.

Some colour printers use more than 3 colours of ink, e.g., CcMmY.

Many colour printers also use a separate ink cartridge for black.

Colour perception also involves a lot of neural processing.

Something white will be perceived as white even under quite different lighting.
Cf. white balance in digital cameras.

Limits to colour invariance.

Hey, Fred - turn on the blue light, the man wants to see a blue suit.

© 1995, E. H. Adelson, MIT, reproduced with permission

Another illusion: The squares marked A and B are the same shade of grey.
Discussion.

Encoding of text and graphics

Character codes

• ASCII
• ISO-Latin1, etc. (example)
• Chinese, Japanese, Korean, etc. (Big-5, JIS, etc.)
• Unicode: a 16-bit encoding for the scripts of the world's principal languages and for many historic and archaic scripts
  (a subset of the 32-bit ISO/IEC 10646-1:1993 standard)

Graphics formats

• Rows & columns of pixels
• Information about image, e.g.,
  • image size
  • no. bits/pixel
  • type of colour palette, gamma correction, etc.
  • type of compression
  • timings for multiple frames
  • resolution (dots/inch), mainly for printing
  • names of artist, software, etc.
  • date, comments, copyright, etc.

TIFF format

The specification (121 pp.) is available from Adobe.

DICOM format

‘The goals of DICOM are to achieve compatibility and to improve workflow efficiency between imaging systems and other information systems in healthcare environments worldwide.’ (ref.)

The specification is available from NEMA:

• Part 1: Introduction and Overview (21 pp.)
• Part 2: Conformance (285 pp.)
• Part 3: Information Object Definitions (894 pp.)
• Part 4: Service Class Specifications (352 pp.)
• Part 5: Data Structure and Semantics (106 pp.)
• Part 6: Data Dictionary (95 pp.)
• Part 7: Message Exchange (118 pp.)
• Part 8: Network Communication Support for Message Exchange (56 pp.)
• Part 10: Media Storage and File Format for Media Interchange (33 pp.)
• Part 11: Media Storage Application Profiles (75 pp.)
• Part 12: Media Formats and Physical Media for Media Interchange (54 pp.)
• Part 14: Grayscale Standard Display Function (55 pp.)
• Part 15: Security and System Management Profiles (75 pp.)
• Part 16: Content Mapping Resource (646 pp.)
• Part 17: Explanatory Information (201 pp.)
• Part 18: Web Access to DICOM Persistent Objects (WADO) (22 pp.)

Common Web formats

Web browsers typically handle GIF, JPEG and PNG files themselves, but usually have to call a helper application to handle TIFF or other files.

Compression

Lossless compression

Used for images (including faxes), text, executable files, word-processing files, and digital data in general. E.g.:

• run-length encoding (RLE), including PackBits (Mac), Huffman, etc.
• Lempel-Ziv & Welch (LZW), as in GIF files (algorithm is patented; patent expired 2003 Jun in the U.S.A. but is still in force elsewhere)
• ‘deflate’ compression, as in PNG files (patent-free)

Lossy compression

Used for images, video, audio, and analogue data in general. E.g.:

• JPEG (discrete cosine transforms; since 2002 a company has been trying to enforce a 1987 patent that they claim covers the JPEG compression algorithm)
• JPEG 2000 (wavelets)
• fractal
• AZTEC (for ECG's)

Some compression schemes (e.g., RLE) are better for images with large areas of uniform colour, while others (e.g., JPEG) are better for complex images with a lot of shading.

This sequence of images shows the effects of increasing the JPEG compression parameter from 0 (lossless) to 100:

000 (11 kB)	010	020	030
040	050 (2 kB)	060	070
080	090	100 (1 kB)

One can also apply smoothing to the image. These two images have smoothing parameters of 0 and 100, respectively (with the compression parameter set to zero). Smoothing reduced the file size by about 16%.

Graphics software

Paint software

• GIMP
• Corel PHOTO-PAINT
• PaintShop Pro
• Adobe Photoshop
• etc.

Draw software

• Inkscape
• OpenOffice.org Draw
• CorelDRAW
• Adobe Illustrator
• etc.

Paint vs. draw

Paint	Draw
Manipulate pixels directly	Manipulate geometric primitives, e.g., lines, circles, rectangles, splines, text
Edit by erasing and redoing	Edit by adjusting primitives
Save as GIF, PNG, JPEG, TIFF, etc.	Save in proprietary format†; export to GIF, PNG, JPEG, TIFF, etc.
Good for editing scanned images, and for drawings requiring control over individual pixels	Good for drawings which are likely to be modified or resized
May also use proprietary format for retaining limited information about objects	May permit import of images, for tracing and limited manipulation

† Inkscape's file format is SVG, a W3C standard based on XML. It can be viewed directly in some Web browsers.

Also data-plotting software like gnuplot

and 3-D graphics software like Blender

Video

• QuickTime (Apple)
• AVI (Microsoft)
• RealVideo (RealNetworks)
• MPEG (ISO standards)
  • MPEG-1 is lower quality, lower bandwidth
  • MPEG-2 is used in HDTV, will give compression of about 50:1
  • MPEG-4 (1999) combines 'real' & computer-generated content; handles anything from very-low bit rates to very high quality; allows interaction (see IEEE Spectrum 36(2): 26-33, 1999)
• etc.

Medical imaging

Principle of tomography

Multiple views

Reconstruction

Levels of detail

Clinical CT
middle-ear ossicles are barely visible

Reconstruction from microCT
glorious detail

Histology
far more detail, but also artefacts

Image segmentation

Image analysis

http://www.cse.lehigh.edu/prr/hip2005/

Pattern recognition is easy for humans, hard for computers.

http://curious.astro.cornell.edu/history.php

Maybe too easy for humans.