Next
Previous
Contents
You can now already use any Unicode characters in file names. No kernel
or file utilities need modifications. This is because file names in the
kernel can be anything not containing a null byte, and '/' is used to
delimit subdirectories. When encoded using UTF-8, non-ASCII characters
will never be encoded using null bytes or slashes. All that happens is
that file and directory names occupy more bytes than they contain characters.
For example, a filename consisting of five greek characters will appear
to the kernel as a 10-byte filename. The kernel does not know (and does
not need to know) that these bytes are displayed as greek.
This is the general theory, as long as your files stay inside Linux. On
filesystems which are used from other operating systems, you have mount
options to control conversion of filenames to/from UTF-8:
The other filesystems (nfs, smbfs, ncpfs, hpfs, etc.) don't convert
filenames; therefore they support Unicode file names in UTF-8 encoding only
if the other operating system supports them.
Recall that to enable a mount option for all future remounts, you add it to
the fourth column of the corresponding /etc/fstab line.
glibc-2.2 supports multibyte locales, in particular UTF-8 locales. But
glibc-2.1.x and earlier C libraries do not support it. Therefore you need
to upgrade to glibc-2.2. Upgrading from glibc-2.1.x is riskless, because
glibc-2.2 is binary compatible with glibc-2.1.x (at least on i386 platforms,
and except for IPv6). Nevertheless, I recommend to have a bootable rescue
disk handy in case something goes wrong.
Prepare the kernel sources. You must have them unpacked and configured.
/usr/src/linux/include/linux/autoconf.h must exist. Building the kernel
is not needed.
Retrieve the glibc sources
ftp://ftp.gnu.org/pub/gnu/glibc/,
su to root, then unpack, build and install it:
# unset LD_PRELOAD
# unset LD_LIBRARY_PATH
# tar xvfz glibc-2.2.tar.gz
# tar xvfz glibc-linuxthreads-2.2.tar.gz -C glibc-2.2
# mkdir glibc-2.2-build
# cd glibc-2.2-build
# ../glibc-2.2/configure --prefix=/usr --with-headers=/usr/src/linux/include --enable-add-ons
# make
# make check
# make info
# LC_ALL=C make install
# make localedata/install-locales
Upgrading from glibc versions earlier than 2.1.x cannot be done this way;
consider first installing a Linux distribution based on glibc-2.1.x, and
then upgrading to glibc-2.2 as described above.
Note that if -- for any reason -- you want to rebuild GCC after having
installed glibc-2.2, you need to first apply this patch
gcc-glibc-2.2-compat.diff
to the GCC sources.
You will need a program to convert your locally (probably ISO-8859-1) encoded
texts to UTF-8. (The alternative would be to keep using texts in different
encodings on the same machine; this is not fun in the long run.)
One such program is `iconv', which comes with glibc-2.2. Simply use
$ iconv --from-code=ISO-8859-1 --to-code=UTF-8 < old_file > new_file
Here are two handy shell scripts, called "i2u"
i2u.sh
(for ISO to UTF conversion) and "u2i"
u2i.sh
(for UTF to ISO conversion).
Adapt according to your current 8-bit character set.
If you don't have glibc-2.2 and iconv installed, you can use GNU recode 3.6
instead.
"i2u"
i2u_recode.sh is
"recode ISO-8859-1..UTF-8", and
"u2i"
u2i_recode.sh is
"recode UTF-8..ISO-8859-1".
ftp://ftp.gnu.org/pub/gnu/recode/recode-3.6.tar.gz Or you can also use CLISP instead. Here are
"i2u"
i2u.lisp and
"u2i"
u2i.lisp
written in Lisp. Note: You need a CLISP version from July 1999 or newer.
ftp://clisp.cons.org/pub/lisp/clisp/source/clispsrc.tar.gz.
Other data conversion programs, less powerful than GNU recode, are
`trans'
ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/charsets/trans113.tar.gz,
`tcs' from the Plan9 operating system
ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/charsets/tcs.tar.gz,
and
`utrans'/`uhtrans'/`hutrans'
ftp://ftp.cdrom.com/pub/FreeBSD/distfiles/i18ntools-1.0.tar.gz
by G. Adam Stanislav
<adam@whizkidtech.net>.
For the repeated conversion of files to UTF-8 from different character sets,
a semi-automatic tool can be used:
to-utf8
presents the non-ASCII parts of a file to the user, lets him decide about the
file's original character set, and then converts the file to UTF-8.
You may have the following environment variables set, containing locale
names:
- LANGUAGE
override for LC_MESSAGES, used by GNU gettext only
- LC_ALL
override for all other LC_* variables
- LC_CTYPE, LC_MESSAGES, LC_COLLATE, LC_NUMERIC, LC_MONETARY, LC_TIME
individual variables for:
character types and encoding,
natural language messages,
sorting rules,
number formatting,
money amount formatting,
date and time display
- LANG
default value for all LC_* variables
(See `man 7 locale ' for a detailed description.)
Each of the LC_* and LANG variables can contain a locale name of the
following form:
language[_territory[.codeset]][@modifier]
where language is an
ISO 639
language code (lower case), territory is an
ISO 3166
country code (upper case), codeset denotes a character set, and
modifier stands for other particular attributes (for example indicating
a particular language dialect, or a nonstandard orthography).
LANGUAGE can contain several locale names, separated by colons.
In order to tell your system and all applications that you are using UTF-8,
you need to add a codeset suffix of UTF-8 to your locale names. For example,
if you were using
LC_CTYPE=de_DE
you would change this to
LC_CTYPE=de_DE.UTF-8
You do not need to change your LANGUAGE environment variable.
GNU gettext in glibc-2.2 has the ability to convert translations to the right
encoding.
You create using localedef the support files for each UTF-8 locale
you intend to use, for example:
$ localedef -v -c -i de_DE -f UTF-8 de_DE.UTF-8
You typically don't need to create locales named "de" or "fr" without
country suffix, because these locales are normally only used by the
LANGUAGE variable and not by the LC_* variables, and LANGUAGE is only
used as an override for LC_MESSAGES.
Next
Previous
Contents
|