Misplaced Pages

This is an old revision of this page, as edited by Carey Evans (talk | contribs) at 03:02, 12 February 2002 (start coverting to tables). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Many characters not in the repertoire of standard ASCII will be useful--even necessary--for Wiki pages, especially the international pages. This page contains my recommendations for which characters are safe to use and how to use them. There are three ways to enter a non-ASCII character into a Wiki page:

• Enter the character directly from a foreign keyboard, or by cut and paste from a "character map" type application, or by some special means provided by the operating system or text editing application. The web server should then be configured to announce which 8-bit character set is being used.
• Use an HTML named character entity reference like à. This is the most reliable method, and is unambiguous even when the server does not announce the use of any special character set, and even when the character does not display properly on some browsers.
• Use an HTML numeric character entity reference like ¡. This is not recommended, because many browsers incorrectly interpret these as references to the native character set. It is, however, the only way to enter Unicode values for which there is no named entity, such as the Turkish letters. Note that because the code points 128 to 159 are unused in both ISO-8859-1 and Unicode, character references in that range such as ƒ are illegal and ambiguous, though they are commonly used by many web sites.

Generally speaking, Western European languages such as Spanish, French, and German pose few problems. For specific details about other langauges, see: Turkish (more will be added to this list as contributors in other languages appear).

ISO-8859-1 Characters

The following extended ASCII characters are safe for use in all Wiki pages. The table below shows the character itself, lists the code for each character in hexadecimal and decimal, shows the HTML entity name, and gives the common name of the character.

These characters are a subset of the most common extended ASCII character set in use on the Internet, ISO 8859-1. Misplaced Pages pages are identified by the server as containing ISO-8859-1 text. The characters above are a subset selected to improve compatibility with other machines.

For example, the Apple Macintosh is in common use on the Internet, is not limited to any specific language, and its native character set (which is not ISO-8859-1) contains many of the common international characters. Many Macintosh browsers will correctly translate ISO text into the native character set, as long as the characters used are available. So the table above is the subset of ISO-8859-1 characters that are also available on the native Macintosh character set. Microsoft Windows standard code page 1252 set is a superset of ISO-8859-1, so these characters will be readable as is on Windows machines. The most common Latin character sets other than ISO-8859-1 are MS-DOS (pre-Windows) code page 437, Macintosh Roman, and other ISO sets such as ISO-8859-2. The number of pre-Windows MS-DOS machines with web browsers is small and they are often dedicated-purpose machines that wouldn't be using Misplaced Pages anyway, so it is reasonably safe to sacrifice compatibility with them for the sake of needed foreign characters. Other ISO sets are generally intended to be read by other browsers using those same sets in the same country, and so those pages should use a language-specific set.

These characters can be entered either as HTML named character entity references such as à, directly from foreign keyboards, or with whatever facilities are available to the Wiki author for entering these characters. For example, Wiki authors using Windows machines can enter these by holding down the Alt key while typing the 4-digit decimal code of the character on the numeric pad of the keyboard. It is important that all 4 digits (including the leading 0) be typed; typing a 3-digit code will enter characters from the obsolete code page 437. Wiki authors using Macintosh machines should take care to either use special facilities to enter these in ISO-8859-1 format rather than with the native character set, or else use HTML named character entity references. Note that some Windows users may have trouble with versions of Microsoft Internet Explorer that use "Alt-Left-Arrow" and "Alt-Right-Arrow" for page movement. These will interfere with entering codes that contain the digits 4 and 6. Use HTML named character entity references in this case.

The characters from the table above can be used directly as 8-bit characters in all Wiki pages, and are sufficient for all pages primarily in English, Spanish, French, German, and languages that require no more special characters than those (such as Catalan). Despite their general safety, at this time, it is not possible to use these characters in Wiki page titles in the English Misplaced Pages, though some of the International Wikipedias are configured to allow them.

Unsafe characters

Note especially what is missing here from the full ISO-8859-1 set: The broken bar (0166=¦), soft hyphen (0173=­), superscript digits (0178=², 0179=³), vulgar fractions (0188=¼, 0189=½, 0190=¾), Old English eth and thorn (0208=Ð, 0240=ð, 0222=Þ, 0254=þ), and multiply sign (0215=×). These should be considered unsafe (and adequate substitutes are available for most of them).

Special care should be taken with characters that do exist in the native character set of popular machines but not in the above set. These are not safe, even though they may display correctly to you when you use them. Characters from Windows code page 1252 not in ISO-8859-1 include the euro sign (€), dagger and double dagger (†, ‡), bullet (•), trade mark sign (™), typeset-style punctuation (see below), per mille sign (‰), some Eastern European caron-accented letters, and the oe ligatures. Characters from the Macintosh Roman set not in ISO-8859-1 include dagger and double dagger, bullet, trade mark sign, a few math symbols such as infinity (∞) and not equal (≠), a few commonly-used Greek letters such as pi (π), ligatures like oe and fl, typeset-style punctuation, per mille sign, and lone accents such as the breve, onogek, and caron.

HTML 4.0 defines named character entities for some Latin characters not in ISO-8859-1 that are used by popular languages, such as OE ligature (Œ, œ), uppercase Y with diaeresis (Ÿ), and some Eastern European accented characters like š. These are also unsafe, though if they entered as HTML named character entity references, they may display on some machines.

In short, don't assume that it is safe to use a special character just because it looks correct on your machine. Use the ones from the table above, and read and understand how to use others shown below.

Possibly usable non-ISO characters

Some characters not listed as safe above may still be usable when entered as named HTML character entity references, because web browsers will recognize them and render them correctly, perhaps by switching to alternate fonts as needed. All of these should be considered less safe to use than those above, but only in the sense that they may not display properly, though in the form of HTML character entity references they are unambiguous, and preserve data integrity.

For many of these, adequate substitutes and workarounds are available, and should be used when the value of making the text available to users of older computers and software exceeds the value of good presentation to those with newer software (in the judgment of the author or editor).

Typeset-style Punctuation

Absent from the ISO-8859-1 character set, but commonly used and present in both Macintosh Roman and Windows code page 1252 character sets, are proper English quotation marks and dashes. These can be entered as character entity references, and should appear correctly on most machines running recent software. Even on ISO-based machines such as Unix/X, browsers should be able to interpret these references and make appropriate substitutes using plain ASCII straight quotes and hyphens (Mozilla does this correctly, for example). These references were not present in older versions of HTML, so may not be recognized by older software. Since using these characters maintains data integrity even on those machines that may not display them correctly, it should be considered safe to use these unless proper display on old software is critical. German "low-9" quotation marks are a similar case, but are less commonly translated by browsing software, and so are not quite as safe. The table below shows these characters next to a capital letter "O" for better visibility:

Many web sites targeted for a Windows-using audience use code page 1252 references for these characters: for example, using — for the em dash. This is not a recommended practice. To ensure future data integrity and maximum compatibility, recode these as named references such as —.

Greek letters and math symbols

Web standards for writing about mathematics are very recent (in fact MathML 2.0 was just released in February of 2001), so many browsers made before these standards were in place try to compensate by at least allowing characters commonly used in mathematics, including most of the Greek alphabet. These are necessarily entered as character entity references. Browsers often render these by switching to a "Symbol" font or something similar.

Upper- and lowercase Greek letters simply use their full names for character entities. These should, of course, only be used for occasional Greek letters in primarily-Latin text. Actual Greek-language text should be written using a Greek character set to avoid bloated files and slow response. Here are a few samples:

α  α      Γ  Γ
β  β       Λ  Λ
γ  γ      Σ  Σ
π  π         Π  Π
σ  σ      Ω  Ω
ς  ς ("final" sigma, lowercase only)

Other common math symbols:

≠  ≠         ′  ′
≤  ≤         ″  ″
≥  ≥         ∂  ∂
≡  ≡      ∫  ∫
≈  ≈      ∑  ∑
∞  ∞      ∏  ∏
√  √

Many of the symbols in the Windows "Symbol" font commonly used for rendering mathematics (such as the expandable bracket parts) are not present on most other machines, and not even present in Unicode 3.1 or as HTML named entities (though they are planned for Unicode 3.2). These are used by products such as TtH to reder equations. You should be aware that if you use these symbols, you are restricting your audience to Windows users (whether or not that's acceptable is a judgment you will have to make as an author).

Other common symbols

Some characters such as the bullet, euro currency sign, and trade mark sign are special cases. They are likely to be understood and rendered in some way by many browsers. Because they are important for international trade, many computers specifically add them to fonts at some non-standard location and render them when requested, or else render them in special ways that don't require them to be present in a font. See below for how your browser renders these:

•  •    bullet
€  €    euro currency sign
™  ™   trade mark sign

Other somewhat less commonly used symbols include these:

†  †  dagger             ♠  ♠  black spade suit
‡  ‡  double dagger      ♣  ♣   black club suit
◊  ◊     lozenge            ♥  ♥  black heart suit
‰  ‰  per mille sign     ♦  ♦   black diamond suit
←  ←    leftward arrow     ‹  ‹  single left-pointing angle quote
↑  ↑    upward arrow       ›  ›  single right-pointing angle quote
→  →    rightward arrow
↓  ↓    downward arrow

These should be considered unsafe to use except perhaps on pages intended for a specific audience likely to have very up-to-date software on popular machines.

Unicode

The Unicode character encoding UCS-4 is the official character encoding of HTML 4.0. Many browsers, though, are only capable of displaying a small subset of the full UCS-4 repertoire. For example, the codes Й ק م display on your browser as Й, ק, and م, which ideally look like the Cyrillic letter "Short I", the Hebrew letter "Qof", and the Arabic letter "Meem", respectively. It is unlikely that your computer has all of those fonts and will display them all correctly, though it may display a subset of them. Because they are encoded according to the standard, though, they will display correctly on any system that is compliant and does have the characters available. Numeric character entity references are the only way to enter these characters into a Wiki page at present. Note that encoding them using decimal rather than hexadecimal (e.g. Й instead of Й) will increase the number of browsers on which they will work.

see also Unicode and HTML for character entities tables.

/Windows

Talk