Python Big Data Engineer ETL, Airflow, CircleCI & Cloud //CLOSED

Keywords : Spark, azure data factory, airflow,ETL, Python, Dutch native speaker mandatory

Context:

We wish to build up customer knowledge at our client in order to further improve our service. Both outbound in intelligent interactions with our customers, as well as business in a more targeted use of investment resources to us better match supply to demand

 

Description

You will join an agile team with the mentality of a startup. In addition to the technical development of pipelines, you can also understand how to get value from this data. You can first gain exploration insights and build production-worthy models. Open constructive content and technical adjustment is in your nature. We therefore expect more than just technical implementation. You have experience with technical integrations and you can think / build with this (API, event based, pipeline scheduling and transaction handling…). You think along with you functionally. You are available at least 4/5 to strengthen the team on-site in Mechelen.

Profile: (red= mandatory)

  • Minimum 5 years of professional experience in IT programming
  • Dutch native with fluent English.
  • Experience in Design, build and deploy a set of batch data applications in spark / azure batch
  • Knowledge of data science and machine learning algorithms to ensure that your data deliveries are ideal for building models.
  • Experience in workflow mgmt and job scheduling of data loads (airflow, azure data factory)
  • Experience in real time data streaming and processing is a plus (azure functions / event hubs / streaming analytics)
  • Experience in cloud Azure is a plus. General cloud experience is a must
  • Knowledge of ETL for large data sets on distributed systems
  • Advanced knowledge in programming languages ​​(python, sql, hcl, …) and in exposing data (API, sql server, event streams, ..)
  • Experience with data exploration: Jupyter notebooks, dashboarding … and getting value from data.
  • We expect that you can identify business opportunities or service improvements through your own analyzes and insights.
  • You have enough courage to bring forward new assumptions. You have demonstrable experience in business rules and to be able to weigh complexities
  • Experience with ETLs with aggregations, deduplication and linking to identities
  • Experience in build automation and devops CI / CD tooling: git, Terraform / ARM templates, Circle CI
  • Experience with web environments that use big data intel / recommendations. Javascript, Java, Node Js, API with data exposition

 

For more information: Contact us now with your CV