#024: 8 mini-projects to help you practice

A common challenge for data engineers is finding ways to practice.

The options are seemingly endless, which isn’t always a good thing.

So you aimlessly search online or become overwhelmed and stop trying.

To help give a little guidance, every Friday for the last 2 months I’ve posted a “mini-project” idea on Twitter & YouTube Community.

Each post shares a few steps and a general goal.

Some may take you 20 minutes, others an entire weekend.

And they are purposely a bit vague to encourage self-learning.

But today I’ve consolidated all 8 mini-projects and listed them below so you can see them in one place.

My hope is that perhaps even one of these will help nudge you in a new direction.

Enjoy!

Project #1

Goal: See how startup scripts work on VMs

1. Write bash script to create a directory
2. Create virtual machine on a cloud platform
3. Set startup script -> your bash script
4. Start the VM
5. Open a terminal & confirm directory exists

Project #2

Goal: Understand virtual environments & aliases

1. Install python
2. Create a virtual environment
3. Install any package via pip
4. Deactivate & reactivate, notice the difference
5. Add alias to bash/zsh profile
6. Reactivate w/ alias

Project #3

Goal: See a GitHub workflow in action

1. Signup for GitHub
2. Create a new repository
3. Confirm/Install git locally
4. Create a random SQL file
5. Add remote refs to GH
6. Add, commit, push file to GH rep
7. Create new branch, repeat & merge

Project #4

Goal: Better understand unstructured data types

1. List your 5 favorite songs
2. Under each, list artist, album & year
3. Under each artist, list hometown city
4. Convert to XML
5. Convert to JSON
6. Convert to YAML

Project #5

Goal: Become familiar w/ no-code automation tools

1. Create a Google Sheet
2. Sign up for Zapier (free plan)
3. Connect w/ any other tool
4. Trigger event by adding a new row
5. Experiment w/ other tiggers

Project #6

Goal: Learn different components of SQL

1. Write a SQL query about a fun topic
2. Convert some logic into a function
3. Call function in query
4. Turn query into Stored Procedure
5. Run Stored Procedure on a schedule

Project #7

Goal: Learn how websites use databases

1. Create a MySQL DB
2. Create a local webpage w/ PHP
3. Connect to DB w/ PHP
4. Run SQL to print a record on page

(No graphics - just plain text w/ a value from the DB.)

Project #8

Goal: Learn how to remotely interact with (and automate) cloud storage

1. Sign up for AWS
2. Create a S3 Bucket
3. Create a local CSV file
4. Connect to AWS & load the file w/ Python

As you continue your journey in this wild world of data engineering, don’t forget to practice.

It will help you stay engaged, keep your skills sharp and be a reminder of why you were interested in data in the first place.