Developing a Pan-CJK IVD Collection Dr Ken Lunde | Sr Computer Scientist 2 | CJKV Type Development | Adobe Systems Incorporated

@ken_lunde | @CJKType | @IVD_Registrar http://adobe.ly/2ffq6yR

• A database of registered Ideographic Variation Sequences (IVSes)

• A database of registered Ideographic Variation Sequences (IVSes) • Each registered IVS is associated with a registered IVD collection • Shared IVSes are associated with two or more IVD collections

• Five versions

• Five versions • Four registered collections

• 2007-12-14 • Adobe-Japan1 collection

• 2007-12-14 • Adobe-Japan1 collection • 2010-11-14 • Hanyo-Denshi collection

• 2007-12-14 • Adobe-Japan1 collection • 2010-11-14 • Hanyo-Denshi collection • 2012-03-02 • Additional registered IVSes for the Adobe-Japan1 and Hanyo-Denshi collections

• 2007-12-14 • Adobe-Japan1 collection • 2010-11-14 • Hanyo-Denshi collection • 2012-03-02 • Additional registered IVSes for the Adobe-Japan1 and Hanyo-Denshi collections • 2014-05-16 • Moji_Joho collection

• Adobe-Japan1 • 14,679 sequences in two registrations

• Adobe-Japan1 • 14,679 sequences in two registrations • Hanyo-Denshi • 13,045 sequences in two registrations

• Adobe-Japan1 • 14,679 sequences in two registrations • Hanyo-Denshi • 13,045 sequences in two registrations • Moji_Joho • 10,710 sequences • 9,685 of its sequences are shared with the Hanyo-Denshi collection

• An IVS is a sequence of a base character followed by a variation selector • Any CJK Unified Ideograph—80,388 in Unicode Version 9.0—can serve as a valid base character • There are 240 variation selectors in Plane 14

• Almost all CJK fonts are single-region • One glyph per code point, except for region-specific variants • Region-specific variants can be handled via a registered IVD collection, such as Adobe-Japan1 • Developing Pan-CJK fonts is a lot of work • Multiple glyphs for tens of thousands of code points

• All currently-registered IVD collections are region-specific • Three for Japan • One for Macao SAR

• All currently-registered IVD collections are region-specific • Three for Japan • One for Macao SAR • A multiple-region (aka Pan-CJK) IVD collection makes a lot of sense • Regional variation is a fundamental characteristic of CJK Unified Ideographs • Regional variation using a single font resource requires language-tagging and the ‘locl’ GSUB feature • Very few environments support this functionality—Adobe InDesign and modern browsers • Regional variation is not preserved in “plain text” • Regional variation can be preserved in “plain text” environments via IVSes

♧

♧ 㶶̠㶵

♧ 㶶̠㶵 낧̠남̠낦

♧ 㶶̠㶵 낧̠남̠낦 划̠刓̠刑̠

♧ 㶶̠㶵 낧̠남̠낦 划̠刓̠刑̠ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ⌍ ⌌ ծկ̠

• The extent to which glyphs can be shared across regions can vary… • …by typeface style • …by typeface design

• The extent to which glyphs can be shared across regions can vary… • …by typeface style • …by typeface design • This would lead to multiple and conflicting Pan-CJK IVD collections… • …if actual variation were used as the basis • One Solution: Register a uniform and future-proof Pan-CJK IVD collection

• 10 regions plus one pseudo-region • CN, SG, TW, HK, MO, MY, JP, KR, KP & VN • XK as a pseudo-region for Kāngxī Dictionary (䐁擳㶶Ⱙ) forms

• 10 regions plus one pseudo-region • CN, SG, TW, HK, MO, MY, JP, KR, KP & VN • XK as a pseudo-region for Kāngxī Dictionary (䐁擳㶶Ⱙ) forms • The last 11 variation selectors are reserved for this purpose • U+E01E5 (VS246) through U+E01EF (VS256) • 80,388 CJK Unified Ideographs—Unicode Version 9.0 • 11 regions × 80,388 CJK Unified Ideographs = 884,268 IVSes • Two sample implementations based on Source Han Sans / Noto Sans CJK • Four supported regions × 29,777 supported CJK Unified Ideographs = 119,108 IVSes • 11 regions × 29,777 supported CJK Unified Ideographs = 327,547 IVSes • The seven unsupported regions are appropriately mapped/aliased to the four supported ones

• Defines the requirements and process for registering IVD collections

• Defines the requirements and process for registering IVD collections • Three basic requirements: 1) A base character—the IVD Registrar assigns the variation selector 2) A unique (within the collection) sequence identifier 3) A representative glyph • All submissions are subject to a public review period of at least 90 days • The IVD Registrar, not the UTC, issues the PRI (Public Review Issue)

• The UTC did not have any strong objection about reserving VS246 through VS256

• The UTC did not have any strong objection about reserving VS246 through VS256 • The UTC primarily objected to… • …the lack of representative glyphs for all regions

• The UTC did not have any strong objection about reserving VS246 through VS256 • The UTC primarily objected to… • …the lack of representative glyphs for all regions • Other feedback • The private use “XK” ISO 3166-1 alpha-2 country code is being used by CLDR for Kosovo • This feedback reinforced my belief that… • …horizontal extensions for existing characters are important • …source references and representative glyphs are useful beyond the code charts

• Prepare and submit a revised “PanCJKV” IVD collection acceptance proposal to the UTC

• Prepare and submit a revised “PanCJKV” IVD collection acceptance proposal to the UTC • Submit the IVD collection and its first sequences for registration • The scope will be limited to the base characters supported by Source Han Sans / Noto Sans CJK • The scope will be further limited to the existing source references of those base characters

• Stress the importance of and promote the submission of horizontal extensions • Horizontal extension = source reference + representative glyph • Some regions seriously lack source references

• Stress the importance of and promote the submission of horizontal extensions • Horizontal extension = source reference + representative glyph • Some regions seriously lack source references • Continue to register new IVSes… • …as more base characters (CJK Unified Ideographs) are supported in the model implementation • …as more horizontal extensions are submitted and accepted

• PanCJKV IVD Collection (UNREGISTERED)—GitHub • https://github.com/adobe-type-tools/pancjkv-ivd-collection/ • CJK Type Blog articles • https://blogs.adobe.com/CCJKType/2016/02/experimental-pancjkv-ivd-collection.html • https://blogs.adobe.com/CCJKType/2016/02/experimental-pancjkv-ivd-collection-redux.html • https://blogs.adobe.com/CCJKType/2016/02/pancjkv-ivd-collection-unregistered.html • L2/16-063: Proposal to accept the submission to register the “PanCJKV” IVD collection • http://www.unicode.org/L2/L2016/16063-pancjkv-ivd-collection.pdf • Ideographic Variation Database—IVD • http://www.unicode.org/ivd/ • UTS #37: Unicode Ideographic Variation Database • http://www.unicode.org/reports/tr37/