Setting up Apache Airflow on macOS using Rancher Desktop involves several steps. In this guide, we'll walk you through installing Rancher Desktop, deploying a Kubernetes cluster, and deploying Airflow using…
This article is about writing end-to-end test for a data pipeline. It will cover Airflow, as one of the most popular data pipeline scheduler now days and one of the…
The best way to familiarize yourself with the Hadoop ecosystem or to do proof of concept: is to play with it in a sandbox. Cloudera provides 2 Quick Start options:…
Consider the following situation: A bundle of .avro files is stored on HDFS. They need to be converted to Impala tables. Schemas are not provided with files, at least not…
Kate Belova