JDK:CharacterData和CharacterDataLatin1

CharacterData類是一個抽象類,這個抽象類中定義了許多判斷字符屬性的抽象方法,這些方法的具體實現都在Character0X類中。其實Character類中有許多對應的方法,CharacterData子類實現抽象類的方法來實現字符屬性的判斷。我們並不關心這個字符由哪個具體類中的方法來判斷,如果以後還增加了一些增補字符,那麼只需要實現抽象類並且稍加修改of()方法即可。這就是使用策略模式的好處。

package java.lang;

abstract class CharacterData {
abstract int getProperties(int ch);
abstract int getType(int ch);
abstract boolean isWhitespace(int ch);
abstract boolean isMirrored(int ch);
abstract boolean isJavaIdentifierStart(int ch);
abstract boolean isJavaIdentifierPart(int ch);
abstract boolean isUnicodeIdentifierStart(int ch);
abstract boolean isUnicodeIdentifierPart(int ch);
abstract boolean isIdentifierIgnorable(int ch);
abstract int toLowerCase(int ch);
abstract int toUpperCase(int ch);
abstract int toTitleCase(int ch);
abstract int digit(int ch, int radix);
abstract int getNumericValue(int ch);
abstract byte getDirectionality(int ch);

//need to implement for JSR204
int toUpperCaseEx(int ch) {
    return toUpperCase(ch);
}
char[] toUpperCaseCharArray(int ch) {
    return null;
}
boolean isOtherLowercase(int ch) {
    return false;
}
boolean isOtherUppercase(int ch) {
    return false;
}
boolean isOtherAlphabetic(int ch) {
    return false;
}
boolean isIdeographic(int ch) {
    return false;
}

// Character <= 0xff (basic latin) is handled by internal fast-path
// to avoid initializing large tables.
// Note: performance of this "fast-path" code may be sub-optimal
// in negative cases for some accessors due to complicated ranges.
// Should revisit after optimization of table initialization.

static final CharacterData of(int ch) {
    if (ch >>> 8 == 0) {     // fast-path
        return CharacterDataLatin1.instance;
    } else {
        switch(ch >>> 16) {  //plane 00-16
        case(0):
            return CharacterData00.instance;
        case(1):
            return CharacterData01.instance;
        case(2):
            return CharacterData02.instance;
        case(14):
            return CharacterData0E.instance;
        case(15):   // Private Use
        case(16):   // Private Use
            return CharacterDataPrivateUse.instance;
        default:
            return CharacterDataUndefined.instance;
        }
    }
}

}

那麼Java是怎麼判斷這些字符的屬性的呢?其實每一個Java字符都用一個32位,也就是4個字節來表示這個屬性。

舉例說明:
當我們傳入一個'0'字符時,實際上通過static final CharacterData of(int ch),'0'對應ASCII碼爲48,方法判斷後,最終會調用CharacterDataLatin1類中對應的方法去處理。

CharacterDataLatin1源碼:

package java.lang;

/* The CharacterData class encapsulates the large tables found in
Java.lang.Character.
/

class CharacterDataLatin1 extends CharacterData {

/* The character properties are currently encoded into 32 bits in the following manner:
    1 bit   mirrored property
    4 bits  directionality property
    9 bits  signed offset used for converting case
    1 bit   if 1, adding the signed offset converts the character to lowercase
    1 bit   if 1, subtracting the signed offset converts the character to uppercase
    1 bit   if 1, this character has a titlecase equivalent (possibly itself)
    3 bits  0  may not be part of an identifier
            1  ignorable control; may continue a Unicode identifier or Java identifier
            2  may continue a Java identifier but not a Unicode identifier (unused)
            3  may continue a Unicode identifier or Java identifier
            4  is a Java whitespace character
            5  may start or continue a Java identifier;
               may continue but not start a Unicode identifier (underscores)
            6  may start or continue a Java identifier but not a Unicode identifier ($)
            7  may start or continue a Unicode identifier or Java identifier
            Thus:
               5, 6, 7 may start a Java identifier
               1, 2, 3, 5, 6, 7 may continue a Java identifier
               7 may start a Unicode identifier
               1, 3, 5, 7 may continue a Unicode identifier
               1 is ignorable within an identifier
               4 is Java whitespace
    2 bits  0  this character has no numeric property
            1  adding the digit offset to the character code and then
               masking with 0x1F will produce the desired numeric value
            2  this character has a "strange" numeric value
            3  a Java supradecimal digit: adding the digit offset to the
               character code, then masking with 0x1F, then adding 10
               will produce the desired numeric value
    5 bits  digit offset
    5 bits  character type

    The encoding of character properties is subject to change at any time.
 */

int getProperties(int ch) {
    char offset = (char)ch;
    int props = A[offset];
    return props;
}

int getPropertiesEx(int ch) {
    char offset = (char)ch;
    int props = B[offset];
    return props;
}

boolean isOtherLowercase(int ch) {
    int props = getPropertiesEx(ch);
    return (props & 0x0001) != 0;
}

boolean isOtherUppercase(int ch) {
    int props = getPropertiesEx(ch);
    return (props & 0x0002) != 0;
}

boolean isOtherAlphabetic(int ch) {
    int props = getPropertiesEx(ch);
    return (props & 0x0004) != 0;
}

boolean isIdeographic(int ch) {
    int props = getPropertiesEx(ch);
    return (props & 0x0010) != 0;
}

int getType(int ch) {
    int props = getProperties(ch);
    return (props & 0x1F);
}

boolean isJavaIdentifierStart(int ch) {
    int props = getProperties(ch);
    return ((props & 0x00007000) >= 0x00005000);
}

boolean isJavaIdentifierPart(int ch) {
    int props = getProperties(ch);
    return ((props & 0x00003000) != 0);
}

boolean isUnicodeIdentifierStart(int ch) {
    int props = getProperties(ch);
    return ((props & 0x00007000) == 0x00007000);
}

boolean isUnicodeIdentifierPart(int ch) {
    int props = getProperties(ch);
    return ((props & 0x00001000) != 0);
}

boolean isIdentifierIgnorable(int ch) {
    int props = getProperties(ch);
    return ((props & 0x00007000) == 0x00001000);
}

int toLowerCase(int ch) {
    int mapChar = ch;
    int val = getProperties(ch);

    if (((val & 0x00020000) != 0) && 
            ((val & 0x07FC0000) != 0x07FC0000)) { 
        int offset = val << 5 >> (5+18);
        mapChar = ch + offset;
    }
    return mapChar;
}

int toUpperCase(int ch) {
    int mapChar = ch;
    int val = getProperties(ch);

    if ((val & 0x00010000) != 0) {
        if ((val & 0x07FC0000) != 0x07FC0000) {
            int offset = val  << 5 >> (5+18);
            mapChar =  ch - offset;
        } else if (ch == 0x00B5) {
            mapChar = 0x039C;
        }
    }
    return mapChar;
}

int toTitleCase(int ch) {
    return toUpperCase(ch);
}

int digit(int ch, int radix) {
    int value = -1;
    if (radix >= Character.MIN_RADIX && radix <= Character.MAX_RADIX) {
        int val = getProperties(ch);
        int kind = val & 0x1F;
        if (kind == Character.DECIMAL_DIGIT_NUMBER) {
            value = ch + ((val & 0x3E0) >> 5) & 0x1F;
        }
        else if ((val & 0xC00) == 0x00000C00) {
            // Java supradecimal digit
            value = (ch + ((val & 0x3E0) >> 5) & 0x1F) + 10;
        }
    }
    return (value < radix) ? value : -1;
}

int getNumericValue(int ch) {
    int val = getProperties(ch);
    int retval = -1;

    switch (val & 0xC00) {
        default: // cannot occur
        case (0x00000000):         // not numeric
            retval = -1;
            break;
        case (0x00000400):              // simple numeric
            retval = ch + ((val & 0x3E0) >> 5) & 0x1F;
            break;
        case (0x00000800)      :       // "strange" numeric
             retval = -2; 
             break;
        case (0x00000C00):           // Java supradecimal
            retval = (ch + ((val & 0x3E0) >> 5) & 0x1F) + 10;
            break;
    }
    return retval;
}

boolean isWhitespace(int ch) {
    int props = getProperties(ch);
    return ((props & 0x00007000) == 0x00004000);
}

byte getDirectionality(int ch) {
    int val = getProperties(ch);
    byte directionality = (byte)((val & 0x78000000) >> 27);

    if (directionality == 0xF ) {
        directionality = -1;
    }
    return directionality;
}

boolean isMirrored(int ch) {
    int props = getProperties(ch);
    return ((props & 0x80000000) != 0);
}

int toUpperCaseEx(int ch) {
    int mapChar = ch;
    int val = getProperties(ch);

    if ((val & 0x00010000) != 0) {
        if ((val & 0x07FC0000) != 0x07FC0000) {
            int offset = val  << 5 >> (5+18);
            mapChar =  ch - offset;
        }
        else {
            switch(ch) {
                // map overflow characters
                case 0x00B5 : mapChar = 0x039C; break;
                default       : mapChar = Character.ERROR; break;
            }
        }
    }
    return mapChar;
}

static char[] sharpsMap = new char[] {'S', 'S'};

char[] toUpperCaseCharArray(int ch) {
    char[] upperMap = {(char)ch};
    if (ch == 0x00DF) {
        upperMap = sharpsMap;
    }
    return upperMap;
}

static final CharacterDataLatin1 instance = new CharacterDataLatin1();
private CharacterDataLatin1() {};

// The following tables and code generated using:

// java GenerateCharacter -template ../../tools/GenerateCharacter/CharacterDataLatin1.java.template -spec ../../tools/UnicodeData/UnicodeData.txt -specialcasing ../../tools/UnicodeData/SpecialCasing.txt -proplist ../../tools/UnicodeData/PropList.txt -o C:/re/jdk7u80/2329/build/windows-amd64/gensrc/java/lang/CharacterDataLatin1.java -string -usecharforbyte -latin1 8
// The A table has 256 entries for a total of 1024 bytes.

static final int A[] = new int[256];
static final String A_DATA =
"\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800"+
"\u100F\u4800\u100F\u4800\u100F\u5800\u400F\u5000\u400F\u5800\u400F\u6000\u400F"+
"\u5000\u400F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800"+
"\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F"+
"\u4800\u100F\u4800\u100F\u5000\u400F\u5000\u400F\u5000\u400F\u5800\u400F\u6000"+
"\u400C\u6800\030\u6800\030\u2800\030\u2800\u601A\u2800\030\u6800\030\u6800"+
"\030\uE800\025\uE800\026\u6800\030\u2000\031\u3800\030\u2000\024\u3800\030"+
"\u3800\030\u1800\u3609\u1800\u3609\u1800\u3609\u1800\u3609\u1800\u3609\u1800"+
"\u3609\u1800\u3609\u1800\u3609\u1800\u3609\u1800\u3609\u3800\030\u6800\030"+
"\uE800\031\u6800\031\uE800\031\u6800\030\u6800\030\202\u7FE1\202\u7FE1\202"+
"\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1"+
"\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202"+
"\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1\202\u7FE1"+
"\202\u7FE1\uE800\025\u6800\030\uE800\026\u6800\033\u6800\u5017\u6800\033\201"+
"\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2"+
"\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201"+
"\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2\201\u7FE2"+
"\201\u7FE2\201\u7FE2\201\u7FE2\uE800\025\u6800\031\uE800\026\u6800\031\u4800"+
"\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u5000\u100F"+
"\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800"+
"\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F"+
"\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800"+
"\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F\u4800\u100F"+
"\u3800\014\u6800\030\u2800\u601A\u2800\u601A\u2800\u601A\u2800\u601A\u6800"+
"\034\u6800\034\u6800\033\u6800\034\000\u7002\uE800\035\u6800\031\u4800\u1010"+
"\u6800\034\u6800\033\u2800\034\u2800\031\u1800\u060B\u1800\u060B\u6800\033"+
"\u07FD\u7002\u6800\034\u6800\030\u6800\033\u1800\u050B\000\u7002\uE800\036"+
"\u6800\u080B\u6800\u080B\u6800\u080B\u6800\030\202\u7001\202\u7001\202\u7001"+
"\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202"+
"\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001"+
"\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\u6800\031\202\u7001\202"+
"\u7001\202\u7001\202\u7001\202\u7001\202\u7001\202\u7001\u07FD\u7002\201\u7002"+
"\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201"+
"\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002"+
"\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\u6800"+
"\031\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002\201\u7002"+
"\u061D\u7002";

// The B table has 256 entries for a total of 512 bytes.

static final char B[] = (
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"+
"\000\000\000\000\000\000\000\000\000").toCharArray();

// In all, the character property tables require 1024 bytes.

static {
            { // THIS CODE WAS AUTOMATICALLY CREATED BY GenerateCharacter:
        char[] data = A_DATA.toCharArray();
        assert (data.length == (256 * 2));
        int i = 0, j = 0;
        while (i < (256 * 2)) {
            int entry = data[i++] << 16;
            A[j++] = entry | data[i++];
        }
    }

}        

}

說一下Latin1編碼,Latin1是ISO-8859-1的別名,有些環境下寫作Latin-1。
 ISO-8859-1編碼是單字節編碼,向下兼容ASCII,其編碼範圍是0x00-0xFF,0x00-0x7F之間完全和ASCII一致,0x80-0x9F之間是控制字符,0xA0-0xFF之間是文字符號。
ISO-8859-1收錄的字符除ASCII收錄的字符外,還包括西歐語言、希臘語、泰語、阿拉伯語、希伯來語對應的文字符號。歐元符號出現的比較晚,沒有被收錄在ISO-8859-1當中。
因爲ISO-8859-1編碼範圍使用了單字節內的所有空間,在支持ISO-8859-1的系統中傳輸和存儲其他任何編碼的字節流都不會被拋棄。換言之,把其他任何編碼的字節流當作ISO-8859-1編碼看待都沒有問題。這是個很重要的特性,MySQL數據庫默認編碼是Latin1就是利用了這個特性。ASCII編碼是一個7位的容器,ISO-8859-1編碼是一個8位的容器。
回到我們的源代碼中,可以看到最終A[]中存儲了256個整數,就是使用有4個字節,32bits來存儲的數,但是不能將這256個數當作一個整數來看待,沒有任何的意義,需要讀取32個比特位中特定的位的值,因爲他代表着字符的屬性。舉個例子:ASCII表中的49代表'0'字符,獲取這個字符對應的屬性值爲A[49],轉換後的二進制值如下:
  0- 0011-000   000000-0-0   0-011-01-10   000-01001

1位:0表示沒有mirrored property,如果是'(','[',這些字符,這個位置的值爲1
4位:3
9位:無偏移
1位:無小寫
1位:無大寫
1位:無首字母大寫屬性
3位:3 表示是一個合法的Unicode標識符或Java標識符
2位:1 有數字的屬性
5位:數字移位爲0
5位:字符類型代表的值爲9
既然能夠得到每個字符的代表屬性的整數,接下來當然就是編寫方法取出特定二進制位上的值了。如要查看一個字符的類型,而這個類型由二進制位的最後5位表示,取出後5位的方法如下:

int getPropertiesEx(int ch) {
    char offset = (char)ch;
    int props = B[offset];
    return props;
}

int getType(int ch) {
        int props = getProperties(ch);
        return (props & 0x1F);
}

附件:ASCII表

JDK:CharacterData和CharacterDataLatin1

JDK:CharacterData和CharacterDataLatin1

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章