Ashraf Miah
Oct 29, 2022

--

If your Data Science tasks look like Data Engineering ones that makes sense. If on the otherhand you need understand the data by trying to do a plot, the better pattern is taking a sample that allows you to explore it effectively.

Since no one uses Spark where Pandas works, the use cases don’t quite line up for a direct comparison.

Edit: typo

--

--

Ashraf Miah

CTO, Data Scientist & Chartered Engineer (MEng CEng EUR ING MRAeS) with over 20 years experience in the Aerospace, Rail & Energy Industry.