Category: Allgemein

This short article describes how to add two files with data to a SAS dataset. Assume both files use the same format for the data. The first approach uses a more or less standard technique wereby a global variable (fileref) is created that refers to two files. filename gegs (‘C:\Users\tomva\SynologyDrive\SAS\gegs.txt’ ‘C:\Users\tomva\SynologyDrive\SAS\gegs2.txt’); data ff; infile gegs;…

Recently, I came across the so-called fast export facility in Teradata. This facility allows you to export data from Teradata in a relative fast way. Let us look at the script: .LOGTABLE tom.invoer_log; .logon 192.168.178.13/tom,******; database tom; .BEGIN EXPORT SESSIONS 2; .EXPORT OUTFILE C:\Users\tomva\Documents\invoer.txt MODE RECORD FORMAT TEXT; SELECT CAST(leeftijd AS CHAR(10)), CAST(nummer AS CHAR(10))…

Teradata is a DBMS that is more or less created to handle massive amounts of data. I now work with it and this DBMS is able to handle 500 million rows in a split second. Brilliant. Teradata has a tool BTEQ that can be used to run scripts. Let us show a very simple script:…

Recently, I came accross the SVN version control mechanism. The idea is that we have a so-called working directory that acts as a source for a final version of some documents. Once we have a final version, the final version can be committed to another platform. This other platform can be accessed by others. If…

Currently, I work with SAS and teradata. I notice, we have two possibilities to connect SAS to teradata. One possibility is to create a remote library. The definition is: libname Td_direc teradata tdpid =”127.0.0.1″ user=”tduser” password=”tduser” database=”financial”; Another possibility is to create an explicit connection: proc sql;connect to teradata as td(tdpid=”127.0.0.1″user=”tduser”password=”tduser”database=”financial”);create table work.dataX asselect *from…

Once an API is running, we may start a client. This client fires a call. This call is fired with “requests.get”. Between the brackets, we see a call that could be run from any webbrowser. The reponse can then be received with response.json() The json is very similar to a dataframe. The dataframe can then…

I found a brilliant script to handle API requests. I noticed three elements here: it contains a server process, it is able to receive a call and it is able to return results. The script: import flask from flask import request, jsonify import pyodbc import pandas as pd app = flask.Flask(name) app.config[“DEBUG”] = True def…

A small example on using an API is provided below. I have attempted to make the programme as small as possible. It shows how the response is derived. This response is then translated into a dataframe. this dataframe is then split into keys and values. import requests import pandas as pd response = requests.get(“http://api.open-notify.org/astros.json”) if…

This script combines many Python features, I used in the last few months. The idea is a file is read that contains parameters that are used is a API call. The result is then analysed and split in different data element. Finally, some data are stored in a dataframe that is subsequently read in a…

Whenever we have a data in a dataframe, we might have the urge to change the values. Below, three situations will be investigated. The first situation is the situation, whereby one field will be taken into account. The first approach is a one line creation of such transformation. df[“Birthdate”].astype(str).apply(lambda x : datetime.strptime(x, ‘%Y-%m-%d’) if x…