A few months back, I was taking part to the Qlik Master API Summit where I made my first experience with Qlik Core, a Dockerized version of Qlik’s associative data engine. During the experience I had the honor the meet a young (that is not valid anymore for me :-)) Qlik enthusiast like me, Alexander Torngren, and together we started our journey building up a fully based Qlik Core Web-app covering the refugees and migration topic.

First of all – What is Qlik Core?

Qlik Core is an analytics development platform built around our powerful Associative engine and Qlik-authored library. Build, extend and deploy fast, interactive data-driven applications delivered at massive scale within any cloud environment: Ingest and combine disparate data sources, no matter how large”

Basically it is a software development product, which delivers the Qlik Associative engine where all the data can be brought together with an associative experience without any limit in data exploration (slice and down, drill down, drill up).   All the data resides in memory, processed and indexed by the Engine and consumed by third party application via a web-socket connection. Basically you get delivered the ❤ of Qlik. Sounds not bad. Right?

Are you ready to read about my personal Qlik Core Journey ?



All has started during the Qlik Core Hackathon challenge at the Master Summit in Sweden. The challenge was to use Qlik Core and the Engine API to build up nice visualizations integrated into a Web App covering the actual Refugees and Migration topic. The great guys of the Qlik Master Summit API Team handed out some good material to start with:

  • Refugees Data in form of .csv files
  • Qlik Sense .qvf Refugees sample App

Together with buddy Alex we started with some ideas and brainstorming sessions with a rough planning of activities, which become then a routine for the next 1-2 months after the Summit with regular weekly Skype meetings besides our daily work, till we reached our final and desired status of our Qlik Core Refugees Web App.

During our meetings following questions and points were tackled:

  • What questions can be answered with the delivered data and app?
  • Do we have the possibility to enhance and integrate some additional external data sources (e.g. external refugees news feed provider)?
  • Which kind of UI and Design we are going to use for our web-app?
  • Which kind of visualization libraries can be used to visualize our data?
  • Which JavaScript web front-end framework to choose?
  • How to setup the container using docker?

Plenty a lot of questions. Right? Which will find their detailed answer in my coming soon video series related to DevOps and Qlik. So stay tuned!
But in beforehand already some high level answers to the question above.

Which Front-end JS framework did we choose?

We decided to move out of our comfort zone and we decided us for React and Redux. For both the first experience, as for me a good chance to dig in in this new JS development methodology, which at the end was resulting very much straight forward. As preparation I took a really good Udemy course covering the React and Redux topics by Maximillian Schwarzmüller 😍 which helped me to deep dive in the subject and which made me productive already after 1 week of exercises and prototypes. Highly recommended to everyone who is interested in learning React!

What about UI and UX?

As for the design we took inspiration from already some pretty looking existing web projects covering this topic, completely re-worked them out and enhanced it with additional UI elements, using the React component approach. For some our UI components, we used the Material UI React component framework as base library, which already provides some nice widget, e.g. the select components, but for the base structure we just used HTML5 and CSS3.

With which visualization library to go?

The choice with which visualization library to go, was a crucial part when it comes to visualize our refugees data. In the past I had some experiences with D3 and Picasso.js, but together with Alex we took the decision to use the EChart visualization library. It is a completely open source library committed by the great Chinese (Asiatic) community. Very easy to use, fast learning curve and with great features and capabilities. The kernel of ECharts is a suite of declarative visual design language that customizes built-in chart types based on HTML5 Canvas. I was really impressed about the amount of available visualizations and the great look and feel. Especially the map charts are amazing! If you have some spare time and you have some dash-boarding projects I highly recommended to look into it.

And what about the Qlik Integration ?

Yes I almost forgot about this part…no joking :-). As I mentioned early on in this post we used the containerized version of the Qlik Engine by using Docker for Windows. To be able to communicate with the container and the docker engine we have created a docker-compose file (configuration file) with some settings in it, which basically includes the Qlik Engine service which we would like to run in the container itself with some settings like showing below. In one of my next video you will have a step by step introduction of how you can setup Qlik Core with Docker container with all the settings explained :-).

version: "2"

services:
  engine:
    image: qlikcore/engine:12.424.0
    restart: always
    command: -S AcceptEULA=yes -S DocumentDirectory=/data -S EnableFilePolling=1
    ports:
      - "19076:9076"
      - "19090:9090"
    volumes: 
      - ./data:/data
      - ./log:/home/nobody/Qlik/Sense/Log

For the communication with the Qlik Engine we used the Enigma.js library and the Engine API . In our Web-app we build a kind of Qlik Communication Layer which takes care about the communication between the components and the Qlik Engine. All the logic and API calls is encapsulated in this service class which in fact acts as a bridge.

export default class QlikService {

    static async clearAll(app, softLock=true){
        return new Promise((resolve) => {
           const r = app.clearAll(softLock);
           resolve(r);
        });   
    }

    static async clearField (app, field) {
        return new Promise((resolve) => {
            app.getField(field).then(f => {
                f.clear().then(r => {
                    resolve(r);
                });
            });
        });
    }

    static async createSessionObject (app, def){

        const model = await app.createSessionObject(def);
        const layout = await model.getLayout();

        // eslint-disable-next-line object-shorthand
        return {model: model, layout: layout }

Finally an external JS definition file was used with all the definitions of the Qlik JS object, which are sent to the Qlik Engine via Enigma, with the purpose to get back the data to be rendered on the UI.

export const defBarChart = {
  qInfo: {
    qType: "BarChart"
  },
  qHyperCubeDef: {
    qDimensions: [
      {
        qDef: {
          qFieldDefs: ["[Year]"]
        },
        qNullSuppression: true
      }
    ],
    qMeasures: [
      {
        qSortBy: {
          qSortByState: 0,
          qSortByFrequency: 0,
          qSortByNumeric: 0,
          qSortByAscii: 0,
          qSortByLoadOrder: 0,
          qSortByExpression: -1,
          qExpression: {
            qv: ""
          }
        },
        qDef: {
          qDef: `Sum({<Year={">=$(=Max(Year)-6)<=$(=Max(Year))"}>}[PersonCount])`,
          qLabel: "Person"
        }
      }
    ],
    qEffectiveInterColumnSortOrder: [1, 0],
    qInitialDataFetch: [{ qTop: 0, qLeft: 0, qWidth: 2, qHeight: 5000 }]
  }
};

The final result of our work is shown here

 

What are my personal take-aways of my Qlik Core journey?

It was big fun to play with Qlik Core. Definitely! And I am really happy about the result, which we could deliver in our spare time. Honestly I do not know if I will ever going to use Qlik Core in my working environment, as I still do not see the big ROI of developing with Qlik Core compared to the development of a standard mashup solution with the Qlik Analytic Platform (QAP) and it’s build-In capabilities. It is just Qlik Sense Enterprise with a lot of missing features, e.g. no built-in visualizations, no repository, no orchestrations etc.. BUT with the big advantage that it is a fully containerized product with the possibility of deploying highly scalable and cloud ready solution. It is more a software development component rather than a complete BI dash-boarding product. Overall I had a nice journey which made me learn a lot of new things and not only related to the Qlik topics. I had the possibility to deep dive in new front-end development methodologies by using the React JS framework, I learned new concept around Docker and Container and can now use them in my daily job. Sounds great, isn’t it? ❤

Project Links:

The Links below are including the deployment instructions and all the resources used => HAVE FUN!