Shopping for a Data Platform
With more and more people relying on online grocery shopping, supermarkets process enormous amounts of data to ensure timely, correct delivery. After all: no data, no delivery. Unfortunately, this retailer’s self-build data ingestion solution was wonky: the supermarket lost important insights because of data arriving later than expected. And if that isn’t frustrating enough, try spending days figuring out what’s wrong and hours debugging.
Fullstaq’s engineers reviewed the set-up and decided with the in-house data scientist that they wouldn’t stop there. To future-proof the supermarket’s e-commerce data solution, we designed and developed a new data platform on Google Cloud with serverless data warehouse BigQuery. In addition, we incorporated CI/CD (using GitHub Actions), version control, monitoring, re-usable components, and building blocks for data ingestion.
Lifting the Shop’s Data Processing Capabilities
The new platform has many advantages; take data collection, for example. Before, the supermarket could only batch process data. Now, they do that smoothly via Apache Airflow, and they can also stream process data through Pub/Sub and streaming components running on Kubernetes. The supermarket can analyze its data in real-time and act on it: prompt e-mails for abandoned shopping carts, swift recommendations, you name it.
We also designed and developed a standardized way to bring scheduled queries and analytical views to the cloud, which makes for some happy data scientists! And some jealous colleagues – quite a few company divisions are eager to adopt the new set-up, and Fullstaq is keen to help.
An Air-Gapped Kubernetes Implementation
Implementing Kubernetes in an air-gapped environment.
Building Kubernetes as a Service for de Volksbank
Building a production-grade Kubernetes cluster in only one month.