Extended ASCII
|
|
The term extended ASCII (or high ASCII) describes eight-bit or larger character encodings that include the standard seven-bit ASCII characters as well as others. The use of the term has sometimes been criticized, because it can be mistakenly interpreted that the ASCII standard has been updated to include more than 127 characters, which is untrue.
| Contents |
Motives for extending
Because the number of written symbols used in common natural languages far exceeds the limited range of the ASCII code, many extensions to it have been used to facilitate handling of those languages. Markets for computers and communication equipment outside English-speaking countries were historically open long before standards bodies had time to deliberate upon the best way to accommodate them, so there are many incompatible proprietary extensions to ASCII.
Since ASCII is a seven-bit code and most computers manipulate data in eight-bit bytes, many extensions use the additional 128 codes available by using all eight bits of each byte. This helps include many languages otherwise not easily representable in ASCII, but still not enough to cover all languages of countries in which computers are sold, so even these eight-bit extensions had to have local variants.
Proprietary extensions
Various proprietary extensions appeared on non-EBCDIC mainframe and mini-computers, especially in universities. Commodore microcomputers added many graphic symbols to their non-standard ASCII (PETSCII, based on the original ASCII standard of 1963). IBM introduced eight-bit extended ASCII codes on the original IBM PC and later produced variations for different languages and cultures. IBM called such character sets code pages and assigned numbers to both those they themselves invented as well as many invented and used by other manufacturers. Accordingly, character sets are very often indicated by their IBM code page number. In ASCII-compatibile code pages, the lower 128 characters maintained their standard US-ASCII values, and different pages (or sets of characters) could be made available in the upper 128 characters. DOS computers built for the North American market, for example, used code page 437, which included accented characters needed for French, German, and a few other European languages, as well as some graphical line-drawing characters. The larger character set made it possible to create documents in a combination of languages such as English and French, but not, for example, in English and Greek (which required code page 737).
A set with less characters but more letter and diacritic combinations was used by the Digital VT-220 terminal based on draft versions of a ISO standard that was being developed.
ISO 8859 and proprietary adaptions
Eventually, ISO released this standard as ISO 8859 describing its own set of eight-bit ASCII extensions. The most popular was ISO 8859-1, also called ISO Latin1, which contained characters sufficient for the most common Western European languages. Variations were standardized for other languages as well: ISO 8859-2 for Eastern European languages and ISO 8859-5 for Cyrillic languages, for example. One notable way in which ISO character sets differ from code pages is that the character positions 128 to 159, corresponding to ASCII control characters with the high-order bit set, are specifically unused and undefined in the ISO standards, though they had often been used for printable characters in proprietary code pages, a breaking of ISO standards that was almost universal. Microsoft later created code page 1252, a compatible superset of ISO 8859-1 with extra characters in the ISO unused range. Code page 1252 is the standard character encoding of western European language versions of Microsoft Windows, including English versions. ISO 8859-1 is the common character encoding used by the X Window System, and most Internet standards. The Apple Macintosh, under Mac OS X, currently uses Unicode as its default encoding. Under Mac OS, it used MacRoman.
Input methods
One problem with eight-bit codes is that computer keyboards were originally designed for seven-bit ASCII, and users became accustomed to them. Different manufacturers have solved this problem in different ways, most by using additional shift-type keys labelled "Alt" or "Meta", and sometimes by interpreting multi-keystroke sequences. Also, MS-DOS allowed the user to enter any character by typing its three-digit code point while holding down the Alt key. While it did allow users to take advantage of the full MS-DOS code page 437 character set, it was difficult to remember and caused problems when users switched to other character sets (including Microsoft's switch to code page 1252 beginning with Windows 3.0). To ease the transition, two slightly different numeric entry methods are available in Microsoft Windows: typing a three-digit number with the Alt key down enters a character from code page 437; typing a four-digit number (beginning with 0) will enter the character from code page 1252. For example, code page 437 used code point 151 for the lowercase u with grave accent (ù). Typing Alt+151 on a Windows machine will actually produce the character at code point 249 from code page 1252, which is where code page 1252 has ù. If you want the character that is in position 151 of code page 1252 (which is the em dash, —), you must type Alt+0151. But various international keyboard drivers are available through the Control Panel in which the right Alt-key can be used to select alternate characters without this cumbersome typing of numbers. The user can easily switch between multiple keyboards. The free utility AllChars (http://allchars.zwolnet.com) is also available.
Character set confusion
Because these ASCII extensions have so many variants, it is necessary to identify which set is being used for a particular text for it to be interpreted correctly. However, because the most-used characters (those in ASCII, the seven-bit code points) are common to all sets--even most proprietary ones like the Macintosh--failure to correctly identify a character set often suffers no adverse consequences if the user is typing in English. Further, because many Internet standards use ISO 8859-1, and because Microsoft Windows (using the code page 1252 superset of ISO 8859-1) is the dominant operating system for personal computers today, unannounced use of ISO 8859-1 is quite commonplace, and should generally be assumed without evidence to the contrary.
In many protocols, most importantly e-mail and HTTP the character encoding of content has to be tagged with IANA-assigned character set identifiers.
Unicode
A proposal called Unicode was made in 1991 to address many of these problems, and is now widely accepted. Unicode reserves 1,114,112 (= 220 + 216) code points, and currently assigns characters to more than 96,000 of those code points. The first 256 codes precisely match those of ISO 8859-1. The majority of the 96,000 code points are, at this time, used for Chinese and Korean characters.
External links
- Character Sets and Code Pages at the Push of a Button (http://www.i18nguy.com/unicode/codepages.html)
- AllChars Utility for Windows (http://allchars.zwolnet.com)
- Apple's page about internationalization support for Mac OS X (http://developer.apple.com/intl/)
- Unicode (http://www.unicode.org/)sv:Utökad ASCII