I have a file in Azure Storage which is zipped and then encoded by Avro as Blob. I read it and decode it as you see in the following code:
public static int decodeAvroFile(String avroFile) throws Exception {
GenericDatumReader<Object> reader=new GenericDatumReader<Object>();
org.apache.avro.file.FileReader<Object> fileReader= DataFileReader.openReader(new File(avroFile),reader);
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
Schema schema=fileReader.getSchema();
DatumWriter<Object> writer=new GenericDatumWriter<Object>(schema);
JsonEncoder encoder = EncoderFactory.get().jsonEncoder(schema, os);
for (Object datum : fileReader) writer.write(datum,encoder);
encoder.flush();
SaveJasonInDB saveJson = new SaveJasonInDB();
saveJson.zipJsonParser(os, avroFile + "-decodedFile");
}
finally {
fileReader.close();
}
return 0;
}
After that, I have to pars it to get the content in zip file via JSON:
public void zipJsonParser(ByteArrayOutputStream os,String jsonFile) throws ParseException, IOException {
JSONParser parser = new JSONParser();
String st = os.toString();
JSONObject json = (JSONObject) parser.parse(st);
System.out.println(json.get("EnqueuedTimeUtc"));
JSONObject body = (JSONObject) json.get("Body");
createZipFile((String) body.get("bytes"), jsonFile);
}
Finaly I want to unzip the zip file:
public void unzip(String zipFilePath, String destDirectory) throws IOException {
File destDir = new File(destDirectory);
if (!destDir.exists()) {
destDir.mkdir();
}
ZipInputStream zipIn = new ZipInputStream(new FileInputStream(zipFilePath));
ZipEntry entry = zipIn.getNextEntry();
// iterates over entries in the zip file
while (entry != null) {
String filePath = destDirectory + File.separator + entry.getName();
if (!entry.isDirectory()) {
// if the entry is a file, extracts it
extractFile(zipIn, filePath);
} else {
// if the entry is a directory, make the directory
File dir = new File(filePath);
dir.mkdir();
}
zipIn.closeEntry();
entry = zipIn.getNextEntry();
}
zipIn.close();
}
private void extractFile(ZipInputStream zipIn, String filePath) throws IOException {
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(filePath));
byte[] bytesIn = new byte[BUFFER_SIZE];
int read = 0;
while ((read = zipIn.read(bytesIn)) != -1) {
bos.write(bytesIn, 0, read);
}
bos.close();
}
Everything is ok when I try the last part of code to unzip a normal zip file. But in this example, we have to decode Avro file and extract the zip file from a JSON element, it doesn't work. You see a part of JSON here:
{"EnqueuedTimeUtc":"2017-11-28T09:45:55.4560000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"firstSAM","contentEncoding":"","connectionDeviceGenerationId":"636451410108166633","contentType":"application/json","connectionAuthMethod":"{\"scope\":\"hub\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}","enqueuedTime":"2017-11-28T09:45:55.4560000Z"},"Body":{"bytes":"PK\u0003\u0004\n\u0000\u0000\u0000\u0000\u0000SK|K\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000
Could you please help me??