The NLS SETUP Application and TRABASE.SAS: Easy ways to customise character conversions using the SAS System Manfred Kiefer, SAS Institute European Headquarters Abstract All software used in a multi-dimensional environment must account for the differences in character sets and encoding schemes. It must also accommodate differences in conventional usage between languages, such as the differing usage of upper and lower cases. The SAS System provides several features to ensure that applications can be written to use local conventions and provide national language support (NLS). Areas of concern include the following: • moving data and applications between hosts • management of text-strings • displaying and printing national characters other than the standard upper and lower case A-Z, as they are encoded by various ASCII and EBCDIC formats. Historically, the SAS System has provided internal translation tables, called TRANTAB entries, that convert one character encoding standard to another. Currently shipping as a sample application with the Orlando Release, NLSsetup fully automates the creation of TRANTABs, key maps, and device maps, and provides an easy point-and-click interface for users to transparently specify language features. The Problem for SAS System Users As shown in Table 1, each host or platform on which the SAS System runs uses different standards for encoding characters. As a result, you must convert or map characters when you move data across platforms. Table 1: Operating Systems (Hosts) Grouped by Character-Encoding Standard EBCDIC hosts: • CMS • MVS • VSE ASCII-ISO hosts (those that use character set(s) that are defined by the ISO 8859 standard): • AIX-RS/6000 • Convex • DG/UX • HP-UX • Intel ABI • MIPS ABI • OpenVMS-VAX • OpenVMS-AXP • Digital UNIX • Solaris 2 • SunOS 4.1 • ULTRIX ASCII-ANSI hosts,. which is the MS-Windows ANSI character set. (This is essentially ISO 8859, but it is called ASCII-ANSI because it was originally based on an ANSI draft standard) • Windows 3.1 • Windows 32s • Windows NT • Windows 95 ASCII-MAC hosts (those that use character set(s) that are defined by the vendor-specific Apple Macintosh character set): • Macintosh System 7.5 for Motorola 68020-, 68030-, and 68040-based systems • PowerPC-based Macintosh systems ASCII-OEM host (vendor-specific IBM PC-ASCII character set): • OS/2 When you transfer data with the "standard" A-Z characters, character conversion from one encoding standard to another is not a problem. You simply rely on the default conversion mechanisms. However, when you have data with national characters such as the æ, ø, and å in Danish; the ä, ö, ß and ö in German; and the accented characters, such as á, é, ú, and ñ in Spanish, different conversion mechanisms are involved, and unique character encoding standards are used on each platform. The NLSsetup application enables you to adapt conversion (translation) tables for each language. Although this is typically a system administrator's task, you should understand the process so that you can add your own customised tables or modify existing ones. The problem for a SAS system administrator is to enable users to transfer data and applications from one host to another, or transparently access data on one host from another host without concern about character conversion from one coded character set to another. The SAS System provides a number of ways of transporting data and applications across hosts. However, the processes and trantabs that are involved differ, depending on the mechanisms you use. The REMOTE engine feature of SAS/CONNECT and SAS/SHARE software uses host-to-host trantabs. PROCs UPLOAD, DOWNLOAD, CPORT, and CIMPORT use transport-format trantabs. Each of these mechanisms is explained in the following sections. Host-to-Host Trantabs: Transporting Data via the REMOTE Engine The REMOTE engine is a feature of SAS/CONNECT and SAS/SHARE software that allows you to access remote data. When you move data across platforms, the REMOTE engine translates character sets directly from the source platform's encoding standard to the target platform's encoding standard, as shown in the following diagram. source target platform <------> translation <--------> platform (host-to-host trantabs) For example, if you are using the REMOTE engine to access data on an MVS host, which uses EBCDIC encoding, from a PC client, which uses IBM PC-ASCII encoding, characters are translated directly from EBCDIC to IBM PC-ASCII and vice-versa. Table 2 shows the trantabs that the SAS System provides for direct host-to-host character-set translation. Table 2: SAS Host-to-Host Trantabs Trantab Name Entry and Function Specific Hosts ------------------------------------------------------------------------------------------- On EBCDIC hosts (IBM mainframes) _0000030 (0) import from ASCII-ISO to EBCDIC connecting MVS, CMS, or VSE to (1) export from EBCDIC to ASCII-ISO OpenVMS or UNIX systems _0000060 (0) import from ASCII-ANSI to EBCDIC connecting MVS, CMS, or VSE to (1) export from EBCDIC to ASCII-ANSI Windows _00000A0 (0) import from ASCII-OEM to EBCDIC connecting MVS, CMS, or VSE to (1) export from EBCDIC to ASCII-OEM OS/2 _0000120 (0) import from ASCII-MAC to EBCDIC connecting MVS, CMS, or VSE to (1) export from EBCDIC to ASCII-MAC MAC ------------------------------------------------------------------------------------------- On Windows hosts - Windows in ANSI mode: _0000050 (0) import from ASCII-ISO to ASCII-ANSI connecting Windows to (1) export from ASCII-ANSI to ASCII-ISO OpenVMS and UNIX systems _0000060 (0) import from EBCDIC to ASCII-ANSI connecting Windows to MVS (1) export from ASCII-ANSI to EBCDIC CMS, or VSE _00000C0 (0) import from ASCII-OEM to ASCII-ANSI connecting Windows to OS/2 (1) export from ASCII-ANSI to ASCII-OEM or to Windows in ASCII-OEM mode _0000140 (0) import from ASCII-MAC to ASCII-ANSI connecting Windows to MAC (1) export from ASCII-ANSI to ASCII-MAC - Windows in OEM mode: _0000090 (0) import from ASCII-ISO to ASCII-OEM connecting Windows to (1) export from ASCII-OEM to ASCII-ISO OpenVMS or UNIX systems _00000A0 (0) import from EBCDIC to ASCII-OEM connecting Windows to (1) export from ASCII-OEM to EBCDIC MVS, CMS, or VSE _00000C0 (0) import from ASCII-ANSI to ASCII-OEM connecting Windows to (1) export from ASCII-OEM to ASCII-ANSI Windows in ASCII-ANSI mode _0000180 (0) import from ASCII-MAC to ASCII-OEM connecting Windows to MAC (1) export from ASCII-OEM to ASCII-MAC -------------------------------------------------------------------------------------------- On OS/2 _0000090 (0) import from ASCII-ISO to ASCII-OEM connecting OS/2 to OpenVMS (1) export from ASCII-OEM to ASCII-ISO or UNIX systems _00000A0 (0) import from EBCDIC to ASCII-OEM connecting OS/2 to MVS, (1) export from ASCII-OEM to EBCDIC CMS, or VSE _00000C0 (0) import from ASCII-ANSI to ASCII-OEM connecting OS/2 to Windows (1) export from ASCII-OEM to ASCII-ANSI _0000180 (0) import from ASCII-MAC to ASCII-OEM connecting OS/2 to MAC (1) export from ASCII-OEM to ASCII-MAC -------------------------------------------------------------------------------------------- On MAC _0000110 (0) import from ASCII-ISO to ASCII-MAC connecting MAC to OpenVMS (1) export from ASCII-MAC to ASCII-ISO or UNIX systems _0000120 (0) import from EBCDIC to ASCII-MAC connecting MAC to MVS, (1) export from ASCII-MAC to EBCDIC CMS, or VSE _0000140 (0) import from ASCII-ANSI to ASCII-MAC connecting MAC to Windows (1) export from ASCII-MAC to ASCII-ANSI _0000180 (0) import from ASCII-OEM to ASCII-MAC connecting MAC to OS/2 (1) export from ASCII-MAC to ASCII-OEM -------------------------------------------------------------------------------------------- On OpenVMS and UNIX hosts _0000030 (0) import from EBCDIC to ASCII-ISO connecting OpenVMS or UNIX to (1) export from ASCII-ISO to EBCDIC MVS, CMS, or VSE _0000050 (0) import from ASCII-ANSI to ASCII-ISO connecting OpenVMS or UNIX (1) export from ASCII-ISO to ASCII-ANSI to Windows _0000090 (0) import from ASCII-OEM to ASCII-ISO connecting OpenVMS or UNIX (1) export from ASCII-ISO to ASCII-OEM to OS/2 _0000110 (0) import from ASCII-MAC to ASCII-ISO connecting OpenVMS or UNIX (1) export from ASCII-ISO to ASCII-MAC to MAC -------------------------------------------------------------------------------------------- The same trantabs are used for all connectivity mechanisms, including APPC, TCP/IP, NETBIOS, and DECnet. As you can see from Tables 1 and 2, the conversion subsystem distinguishes the following character architectures: • EBCDIC • ASCII-ISO (ISO 8859) • ASCII-OEM (which, for our purposes, includes only the IBM PC-ASCII standard) • Microsoft's ASCII-ANSI • Apple's ASCII-MAC. Each host-to-host trantab actually consists of two halves, or "entries": • ordered entry 0 (for importing) • entry 1 (for exporting). For example, on UNIX hosts, the _0000030 (EBCDIC to ASCII-ISO) trantab is shown in Table 3 as it appears when both halves are listed by the TRANTAB procedure. Table 3: The _0000030 Trantab Table name is _0000030. 0 1 2 3 4 5 6 7 8 9 A B C D E F 00 '000102039C09867F978D8E0B0C0D0E0F'x 10 '101112139D8508871819928F1C1D1E1F'x 20 '80818283840A171B88898A8B8C050607'x 30 '909116939495960498999A9B14159E1A'x 40 '20A0A1A2A3A4A5A6A7A8D52E3C282B7C'x -> 50 '26A9AAABACADAEAFB0B121242A293B5E'x 60 '2D2FB2B3B4B5B6B7B8B9E52C255F3E3F'x 70 'BABBBCBDBEBFC0C1C2603A2340273D22'x 80 'C3616263646566676869C4C5C6C7C8C9'x 90 'CA6A6B6C6D6E6F707172CBCCCDCECFD0'x A0 'D17E737475767778797AD2D3D45BD6D7'x B0 'D8D9DADBDCDDDEDFE0E1E2E3E45DE6E7'x C0 '7B414243444546474849E8E9EAEBECED'x