Flume: sending data via stream
tom
It is possible to capture streaming data in HDFS files. A tool to do this is Flume. The idea is that we have 3 elements: sources that provide a stream,…
Partitioned Table in Hive
It is possible to partition the tables in Hive. Remember the data are stored in files. So we expect the files to be partitioned. This is accomplished by a split…
Manipulating Avro
Avro files are binary files that contain data and the description of the files. Thereby it is a very interesting file format. One may send this file to any application…
Parquet format
As we know, we may store table definitions in the metastore. These table definitions then refer to a location where the data are stored. The format of the data might…
Avro format
In Hive, we see a situation where a table definition is stored in a metastore. This table definition is linked to a directory where the data are stored. It is…
Create a Hive table – 3 ways
In this little note, I want to show three different ways to create a table on Hive. The first one starts with a file on HDFS that is available and…