Which one to choose, SQL or Pandas for basic Data manipulation ?

Have you often wondered, as why we use a certain tool for manipulation ? sometimes we would be asked to do in sql, sometimes in python or any other etc.,

Well they have their purpose and reasons, and one such as why or when we use pandas and sql is what we are going to learn in this post.

So coming back to our question, What do you prefer for basic data manipulation, SQL or pandas ?

It depends on the situation,

(For basic manipulations)

If you have data manipulation to be done from different sources, then go for pandas.

Or, if the data is from a database, and you would want to perform manipulations, then define your question, for what are you performing these manipulations for ?

Do you want it to directly impact the database and retrieve/extract later based on use case ?

Or, do you want to use these for further analysis ?

So if it’s the first case, then go for SQL, else go for pandas.

The reason for the above mentioned answer is, though SQL is a really powerful tool, it fails at many places, it can’t load data from multiple sources, and it doesn’t have many functionalities as that of pandas. But at the same time, if the data is quite huge, transformation in pandas does take a lot of time, as it extracts the data first, manipulates, and then sends it back again. Whereas SQL does this really quick in a giffy as it directly impacts the database.

So define your question first, what kind of data you have, varied…big or small ?. Next, what do I want achieve by performing those manipulations. These two questions would lead you to your answer.