Below, you will find a listing on how to copy the content of an Oracle table into an avro file. The trick is quite straight forward. A table is read via a cursor. Each record is then appended to an avro file. The scheme is:
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "number", "type": ["int", "null"]},
{"name": "local", "type": ["string", "null"]}
]
}
So:
import avro.schema
from avro.datafile import DataFileWriter
from avro.io import DatumWriter
import cx_Oracle
teller =0
schema = avro.schema.Parse(open("user.avsc").read())
writer = DataFileWriter(open("users.avro", "wb"), DatumWriter(), schema)
con = cx_Oracle.connect('C##HR/xxx12345@van-maanen.com/orcl')
cur = con.cursor()
cur.execute('select name, local from countries')
for result in cur:
teller = teller + 1
writer.append({"name": result[0], "number": teller, "local": result[1]})
cur.close()
con.close()
writer.close()