Hr Library
Trending

11 Questions About Data Engineers: What’s the profession about, and where’s it heading?

hope my answers will be useful to novice data engineers and anyone interested in data engineering

By | | www.kdnuggets.com

I switched to data engineering from development in 2017. Previously I had worked for ten years in desktop development, backend (mainly Java), and a little bit of frontend. Despite my strong IT experience, it wasn’t easy at first to figure out what data engineers do, how they differ from database administrators, how they’re connected to data analytics, and what they have to do with Big Data. 

It was the magic of the phrase “Big Data” that determined what I’m doing now (this link provides a well-established definition of Big Data with the triad “Volume, Variety, Velocity” + an informative video from Amazon AWS). For me, the Big Data field looked like a challenge that I had to accept. 

I became more interested in distributed systems, scalable technologies, and clouds, and went to conferences where relevant Big Data products like Hadoop, Kafka, Spark, etc. were analyzed. Colleagues who worked with Big Data appeared around me, and I flooded them with questions. Sometimes the answers I received weren’t entirely clear, which further fueled my curiosity.

Click here to read the full article

Source
www.kdnuggets.com
Show More

Related Articles

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Back to top button