Data, data en nog eens data

Author: tom

  • Hadoop: my first java programme

    —

    by

    tom
    in

    Today, I created a Java programme to get myself acquainted with the usage of Hadoop. I took an existing java programme to start with. This existing programme can be found at ” https://github.com/tomwhite/hadoop-book/blob/master/ch02/src/main/java/OldMaxTemperature.java “. I tweaked this programme to adjust it to my existing situation.

  • AWK to investigate files on Unix

    —

    by

    tom
    in

    Today, I worked with the Unix’ awk utility. This is an extremely potent utility to investigate text files on a Unix platform. It can be invoked from the terminal command line. The command must start with awk. The keyword awk is followed by a script that is positioned between quotes. After the quotes, the textfile…

  • Hadoop

    —

    by

    tom
    in

    Everyone talks about big data and Hadoop. Someone even compared it to teenage sex: everone talks about it, everyone knows someone who does it but no-one yet does it. I just tried hadoop to see what it is all about. I made two attempts to install hadoop. One attempt was about installing Hadoop 1.0.3. I…

  • Slowly Changing Dimensions Type 2

    —

    by

    tom
    in

    Just to get myself acquainted with the new Informatica version, I created a mapping in which SCD 2 was inplemented. The mapping is shown here. In the first step, the input data are read.Let us assume that these records are read. The records contain a number and a name: number Name 1 Tom 2 ine…

  • Use Case, Business Events and Time Events

    —

    by

    tom
    in

    In a previous post, I showed the context diagramme. I then continued by saying that each of the arrows that flow to and fro the bubble in the middle can be translated as use cases. But one may take a slightly different view: each of these arrows are either business events or time events. A business…

  • Access odbc

    —

    by

    tom
    in

    Just a little note. I discovered that one may access the 32 bit ODBC drivers on a 64 bit platform via: C:\Windows\SysWOW64\odbcad32.exe and the 64 bit ODBC drivers via: C:\Windows\System32\odbcad32.exe This may be handy if one must access a 32 bit application (like Access 2007) via ODBC in a 64 bit environment. The standard ODBC…

  • Use Case

    —

    by

    tom
    in

    A Use Case is something that can be written after a context diagramme is written. A context diagramme provides an overview of the whole system with its interfaces to the outside world. The use cases are written on base of each of the individual interactions between the system an one of its external systems. As…

  • Requirements analysis

    —

    by

    tom
    in

    Recently I read a very interesting book on requirements analysis “Mastering the Requirements Process: Getting Requirements Right” by Suzanne Robertson and James Robertson. They wrote a practical book that lead to a template for such a document They advocate to spend enough time on requirements analysis, which can be described shortly as “getting to know…

  • Context diagrammes – understanding a scope

    —

    by

    tom
    in

    I came accross a very nice technique that can be used to show the scope of a project. It is called the Context Diagramme. The idea is that the system is shown as one item. The internal structure within the system is not displayed. Instead the diagramme shows the interaction from the system with external…

  • Yet another checklist

    —

    by

    tom
    in

    I often come across checklists. Such checklists are used to assess the quality of, say, a database. See here for an example of such a checklist. In this case, we have a checklist that is created to assess the quality of a Teradata database. Such checklist are very interesting. They are generally based on a…