Reading and Writing Text Files in Different Encodings in Objective-C

Drawing QR Code ISO/IEC18004 in Objective-C Reading and Writing Text Files in Different Encodings

Text-editing applications such as TextEdit commonly include an option in their Open and Save dialog boxes for specifying which encoding you want them to use when reading and writing a plaintext file. Similarly, Standard Additions read and write commands give you some control over how you read and write plain-text files in AppleScript by way of their optional as parameters. In both commands, the as parameter takes a class name constant that indicates how the character data is (or should be) encoded in the text file. You can use three class names for specifying how AppleScript reads and writes text files: string, class utf8 , and Unicode text.

To read and write text files in MacRoman (or whatever your primary encoding is), use string for the as parameter. For example: read the_file as string And for example: write the_text to the_file as string If you don t specify an as parameter for the read command, it will use string as the default. The write command s default behavior is a bit trickier to describe, however. In Panther and earlier, the write command s default as parameter was also string; however, this behavior has significantly changed in Tiger. I ll discuss the details and implications of the new behavior later in the Changes to the write Command in Tiger section.

To read and write Unicode text files, you need to use either class utf8 or Unicode text. Which one you should use depends on how the Unicode character data has been (or will be) encoded in the text file. The two main Unicode encoding schemes in common use today are UTF-8 and UTF-16. (In fact, there s actually two kinds of UTF-16 encodings: UTF-16LE and UTF-16BE; I ll discuss the difference in this section.) The UTF-8 encoding scheme is a cleverly designed encoding that uses a variable number of bytes to represent each character. The first 128 characters in the Unicode character set this includes commonly used characters such as A Z, a z, 0 9, and basic punctuation are represented by a single byte each, and all other characters are represented using unique combinations of two to six bytes apiece. The first 128 characters in the Unicode character set are also identical to the 128 characters of the 7-bit ASCII character set, an even older system that has been influential over the decades. Many other character sets, including MacRoman, are also supersets of ASCII. This overlap gives UTF-8 a degree of backward compatibility, since tools that understand ASCII (much of Mac OS X s Unix-based command line, for example) can still make some sense of UTF-8 encoded files. It also means that UTF-8 files containing mostly English text are more compact than UTF-16 files, which can be handy if you need to keep file size to a minimum. These advantages make UTF-8 a popular modern encoding for plain-text files. To read and write UTF-8 encoded text files, use this: read the_file as class utf8 and this: write the_text to the_file as class utf8

If you accidentally read a UTF-8 encoded text file as if it was MacRoman, the result will be readable where characters 0 127 are used since both UTF-8 and MacRoman represent those characters in the same way. Other characters will appear as gibberish, however. For example, a UTF-8 encoded text file containing the phrase 37 5 C would show up as 37 5 C if interpreted as MacRoman instead of UTF-8. Each of the number and letter characters is represented by the same single byte in both the MacRoman and UTF-8 encoding systems, so would appear correct either way. However, the decimal point and degree sign are both represented in UTF-8 using two bytes apiece, so they show up as nonsense characters when interpreted as MacRoman.

The UTF-16 encoding scheme is a bit more straightforward than UTF-8, although it lacks the backward compatibility and compactness aspects when dealing with English text. In UTF-16, the majority of characters are encoded as two bytes of data each. (Some characters are actually represented using four bytes, but you don t need to worry about this.) To read and write UTF-16 encoded text files, use this: read the_file as Unicode text and this: write the_text to the_file as Unicode text