Image source1
I’m fortunate that where I live I can walk to the kitchen, turn on the faucet, and fill my cup with clean drinking water. I don’t have to think about the complex process that produced the water or how the water was delivered to my kitchen. Other people were responsible for finding suitable water sources and blending them, monitoring the purification process, testing the water to ensure safety, and maintaining the infrastructure and pipes that delivered the water to my kitchen.
I feel better knowing that experts handle each step, that I’m not responsible for finding clean drinking water. By being sheltered from the burden of trying to turn something abundant into something useful, I can avoid the sense of overwhelm described in the 1834 poem “The Rime of the Ancient Mariner” by Samuel Taylor Cooleridge:
Water, water, every where,
Nor any drop to drink
Nearly 200 years after these words were written, this same sentiment applies to working with big data:
Data, data, every where,
Not a bit to use
Data engineers take on job of turning seas of data into something useful. They make sure data is cleaned, processed, and delivered to the researchers, product designers, analysts, and data scientists who need it.
So the next time you view a dashboard, a recommendation to buy something, or your medical history on a tiny screen, pause and reflect on the process needed to get that data to you.