avro learning

sample1:

schema:

{
    "namespace":"com.test.avro",
    "type":"record",
    "name":"User",
    "fields":[
        {"name":"name", "type":"string"},
        {"name":"age", "type":"int"}
    ]
}

轉成java對象文件，官網上又maven插件（http://avro.apache.org/docs/current/gettingstartedjava.html），不過我在eclipse上使用有問題，將就着手動轉轉

java -jar avro-tools-1.7.5.jar compile schema D:\...\user.avsc D:\...\src\main\java

serializing:

File file = new File("users.avro");
DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
dataFileWriter.create(user.getSchema(), file);
dataFileWriter.append(user);
dataFileWriter.close();

deserializing:

File file = new File("users.avro");

DatumReader<User> userDatumReader = new SpecificDatumReader<User>(User.class);
DataFileReader<User> dataFileReader = new DataFileReader<User>(file, userDatumReader);
while (dataFileReader.hasNext()) { 
User users = dataFileReader.next();
}
dataFileReader.close();

sample2:

schema

{
    "namespace": "com.test.avro",
    "type": "record",
    "name":"Fin",
    "fields": [
        {"name": "id", "type": "long"},
        {"name": "items", 
         "type": {
                    "type": "array", 
                    "items": {
                        "namespace": "com.test.test.avro",
                        "type": "record",
                        "name":"Item",
                        "fields": [
                            {"name": "cd", "type": "string"},
                            {"name": "value", "type": "string"}
                        ]
                    }
                  }
        }
    ]
}

schema2會生成2個java文件，Fin&Item，也可以寫在2個schema文件裏，然後在一條命令裏編譯，如：

{
    "namespace": "com.test.test.avro",
    "type": "record",
    "name":"Fin",
    "fields": [
        {"name": "id", "type": "long"},
        {"name": "items", 
         "type": {"type": "array", "items": "com.test.test.avro.Item"}
        }
    ]
}

{
    "namespace": "com.test.test.avro",
    "type": "record",
    "name":"Item",
    "aliases": ["finItem"], 
    "fields": [
        {"name": "cd", "type": "string"},
        {"name": "value", "type": "string"}
    ]
}

java -jar avro-tools-1.7.5.jar compile schema D:\...\item.avsc D:\...\fin.avsc D:\...\src\main\java
PS：item一定要在fin前面，不然報錯找不到item

create without code generation

Schema finSchema= new Parser().parse(new File("fin.avsc"));
Schema itemSchema= new Parser().parse(new File("item.avsc"));

GenericRecord fin = new GenericData.Record(finSchema);

f1.put("id", 3L);

GenericRecord item = new GenericData.Record(itemSchema);
item.put("cd", "1");
item.put("value", "1000");

List<GenericRecord> items = new ArrayList<GenericRecord>();
items.add(item);
fin.put("items", items);