Manipulating Avro
tom
Avro files are binary files that contain data and the description of the files. Thereby it is a very interesting file format. One may send this file to any application…
Parquet format
As we know, we may store table definitions in the metastore. These table definitions then refer to a location where the data are stored. The format of the data might…
Avro format
In Hive, we see a situation where a table definition is stored in a metastore. This table definition is linked to a directory where the data are stored. It is…
Create a Hive table – 3 ways
In this little note, I want to show three different ways to create a table on Hive. The first one starts with a file on HDFS that is available and…
With Python in Hive
In this small note, it is described how an HDFS file can be stored in a Hive context. In it stored in a Hive context, it can be accessed from…
Oracle ODI
The successor to OWB is the Oracle Data Integrator. This tool has more functionalities than OWB. Next to that, it has an interface that more or less steers the user…
Dataflow in Oracle Warehouse Builder
I know that Oracle Warehouse Builder (OWB) is at end of life. On the other hand, I encounter OWB quite often and it is interesting to see how it works.…
Docker container
Only this weekend I downloaded a Docker package from https://docs.docker.com/docker-for-windows. This package allows you to run very small light weight containers on your server than act as components to perform…
reading an HDFS file in Python
In this note, I show you how to get data from an HDFS platform into a Python programme. The idea is that we have data on HDFS and we would…
Putting a file on HDFS
Putting a file on HDFS is relatively easy. There are a few steps to take. Let us assume the file is on a linux system. The first step is to…