BroadOakData.UK

Tag: flink

  • Using Apache Flink to process data from Kafka and MySQL database

    —

    by

    broadoakdata
    in

    I need to use Apache Flink to process data which stored in Kafka and MySql. In my previous article I shared my notes on how to use a free MySQL server (db4free.net) instance for development work. Apache Flink is good processing engine and has nice features for manipulating data using Batch and/or Streaming processing. I…

  • Using Apache Flink to process apache web log files

    —

    by

    broadoakdata
    in

    In my previous article I shared my approach for processing apache web server log files using PySpark. Here I will try to accomplish the same task using Apache Flink. I am using Apache Flink python (PyFlink) package and Flink SQL. Flink SQL is an ANSI standard compliant SQL engine that can process data both using…

  • Streaming data processing using Apache Kafka and Flink

    —

    by

    broadoakdata
    in

    We will use Apache Kafka and Apache Flink to process data from Companies House Stream API . First, we will setup Apache Kafka and Flink in Google Colab platform. You can download the script to install both software from here – https://gitlab.com/akkasali910/companieshouse-data . Or you can use the following bash codes: After running the above…

  • Quick look at Apache Flink

    —

    by

    broadoakdata
    in

    What is Apache Flink? According to Apache Flink’s website: Stateful Computations over Data Streams Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speed and at any scale. https://flink.apache.org We will…