DataOps: Code of Conduct
Kate BelovaI'm often asked what DataOps is and how to be effective in it. DataOps is a relatively new term that refers to the practice of applying DevOps principles to data…
HowTo: Apache Airflow with Docker on Linux
Here's a guide to setting up Apache Airflow with Docker on a Linux machine, with shared DAGs and plugins folder, extra plugins, specific Python packages on Airflow workers, and a…
HowTo: install Apache Airflow with Rancher Desktop (MacOS)
Setting up Apache Airflow on macOS using Rancher Desktop involves several steps. In this guide, we'll walk you through installing Rancher Desktop, deploying a Kubernetes cluster, and deploying Airflow using…
HowTo: Run Cloudera Quickstart in Docker
The best way to familiarize yourself with the Hadoop ecosystem or to do proof of concept: is to play with it in a sandbox. Cloudera provides 2 Quick Start options:…
How to create avro based table in Impala
Consider the following situation: A bundle of .avro files is stored on HDFS. They need to be converted to Impala tables. Schemas are not provided with files, at least not…