Sumit Mittal

A lot of candidates asked me, sir how can we learn Python

I did some research,

Here are 5 Excellent Free Resources to learn Python

1. Python with Shradha Khapra
https://lnkd.in/g_WWQSk7

2. Python with Nana Janashia
https://lnkd.in/gCFiKCZQ

3. Python by Code with Mosh
https://lnkd.in/guUVdJKp

4. Python by Telusko
https://lnkd.in/gHWyQfQX

5. Python by freeCodeCamp
https://lnkd.in/g-zx7hDy

Huge shout out to the above creators for this amazing content!

If you know more good resources, please add in comments!

Follow Sumit Mittal for more such insights!

#python #bigdata #dataengineering #dataanalytics #dataanalysis

Lets say you have 5 TB of data stored in your Amazon S3 bucket.

Lets say this data has 500 million records, 100 columns.

Suppose there are 100 cities and you want to get the data for a particular city, and you want to retrieve only 10 columns.

~ considering each city has equal amount of records,
we want to get 1% of data in terms of number of rows
and 10% in terms of columns

thats roughly 0.1% of the actual data which might be 5 GB roughly.

Now lets the pricing if you are using serverless technology like AWS Athena

- the worst case you end up having the data in a csv format (row based) with no compression. you end up scanning the entire 5 TB data and you pay $25 for this query. (The charges are $5 for each TB of data scanned)

Now lets try to improve it..

- use a columnar file format like parquet with snappy compression which takes lesser space so your 5 TB data might roughly become 2 TB (actually it will be even lesser)

- partition this based on city so that we have 1 folder for each city.

This way you have 2 TB data sitting across 100 folders, but you have to scan just one folder which is 20 GB,

Not just this you need 10 columns out of 100 so roughly you scan 10% of 20 GB (as we are using columnar file format)

This comes out to be 2 GB only.

so how much do we pay?
just $.01 which is 2500 times lesser than what you paid earlier.

This is how you save cost.

what we did?

- using columnar file formats for column pruning
- using partitioning for row pruning
- using efficient compression techniques

Knowing the internals play a very vital role in saving the cost.

I hope you enjoyed this post and the scenario shared.

PS~ I offer a Ultimate big data program and talk a lot of these things. New batch starting tomorrow. DM to know more!

Complete RoadMap to become a Data Analyst (Free Resources)

1. SQL - One should have a good grip over SQL.

https://lnkd.in/gdnhRk8b

2. Excel - In Excel you should go upto advanced level

https://lnkd.in/gR6MUag9

3. Python & R - One programming Language Preferably Python is quite important for Data analyst

Python - https://lnkd.in/gHWyQfQX

R Programming- https://lnkd.in/gEhr6u-2

4. Tableau & Power BI For visualization

One of these visualization tools is good enough

Tableau - https://lnkd.in/gUaZWbyk

Power BI - https://lnkd.in/gAEP3umj

Note - The role of a Data Analyst is to analyse the Data, and above tools & technologies are really important to make an entry in this field.

Conceptually one should know Statistics & Probability to some extent which you can learn from - https://lnkd.in/g_d4Zy8k

Follow this complete path & definitely this should help you gain the right confidence!

#dataanalytics #dataanalyst #sql #database #python #tableau #powerbi #excel #statistics #visualization #data

Here Comes the Gold mine for Data Engineers

Python Complete Playlist (4 videos already released)
lecture 1 - https://lnkd.in/gh_gVsTm
lecture 2 - https://lnkd.in/gCxba2zB
lecture 3 - https://lnkd.in/g6DVh4va
lecture 4 - https://lnkd.in/gs9TFEWK

The 5th lecture is coming on Tuesday (26th march)

Free SQL Complete Course (one shot video 9 hours)
https://lnkd.in/gXTvqk3i

Data Engineering Mock Interviews (Data Engineers Club)
Interview 1 - https://lnkd.in/gjascwBm
Interview 2 - https://lnkd.in/g7AB8TNt
Interview 3 - https://lnkd.in/gzZBRcKu
Interview 4 - https://lnkd.in/gx9kg95h
Interview 5 - https://lnkd.in/gQ9kVUrf

Everyday we are releasing one mock interview at 7 pm on my youtube channel.

Also more videos will be coming in the python playlist.

Do subscribe to my channel so that you do not miss out on anything.

Link to the channel - https://lnkd.in/gun_wZeK

Enjoy the series!

#bigdata #dataengineering #interview #apachespark #sql #python

How a SQL query gets executed internally - Lets see step by step!

We all know SQL, but most of us do not understand the internals of it.

Let me take an example to explain this better.

Select p.plan_name, count(plan_id) as total_count
From plans p
Join subscriptions s on s.plan_id=p.plan_id
Where p.plan_name !=’premium’
Group by p.plan_name
Having total_count > 100
Order by p.plan_name
Limit 10;

Step 01: Get the table data required to run the sql query
Operations: FROM, JOIN (From plans p, Join subscriptions s)

Step 02: Filter the data rows
Operations: WHERE (where p.plan_name=’premium’)

Step 03: Group the data
Operations: GROUP (group by p.plan_name)

Step 04: Filter the grouped data
Operations: HAVING (having total_count > 100)

Step 05: Select the data columns
Operations: SELECT (select p.plan_name, count(p.plan_id)

Step 06: Order the data
Operations: ORDER BY (order by p.plan_name)

Step 07: Limit the data rows
Operations: LIMIT (limit 100)

Knowing the Internals really help.

PS~ I just launched a 8 weeks SQL champion program . DM to know more!

#database #sql

Amazing SQL notes by one of the learners @Atul Singh who is following my SQL series on YouTube.

if you have not yet started with it.. then here is the complete plan.

SQL Basics (14 videos)

📌 SQL Fundamentals, CRUD Operations & Setting Environment - 
https://lnkd.in/ekBxGU2c

📌 Primary Key vs Unique Key, Auto Increment Values -
https://lnkd.in/eXSugBVX

📌 DDL vs DML, Truncate vs Delete -
https://lnkd.in/eCEj6NHc

📌 Foreign Key Constraint -
https://lnkd.in/ebfYyM2b

📌 Distinct, Order By, Limit, Like Keyword -
https://lnkd.in/ec-McKnC

📌 Order of execution in SQL -
https://lnkd.in/eShPzDCJ

📌 Aggregate Functions in SQL -
https://lnkd.in/e2HQQZj3

📌 Datatypes in SQL -
https://lnkd.in/eJ7prXMR

📌 Logical Operators in SQL -
https://lnkd.in/eubjUHeD

📌 Joins in SQL - 
https://lnkd.in/e63jvjec

📌 Difference between where and having in mysql -
https://lnkd.in/eTwb9pcJ

📌 Over Clause & Partition By Clause -
https://lnkd.in/ewspqCVS

📌 Row Number Function in MySQL -
https://lnkd.in/eK9-Ef4P

📌 Rank & Dense Rank -
https://lnkd.in/en83Pr5V

SQL Advanced (2 videos)

📌 CTE in SQL -
https://lnkd.in/e-cKsd89

📌 SQL internals -
https://lnkd.in/erwxZY8J

SQL Leetcode (8 problems)

📌 LeetCode 175 - combine 2 tables
https://lnkd.in/eMmX8DQa

📌 Leetcode 176 - Second highest salary (3 approaches)
https://lnkd.in/eaDRxzSd

📌 Leetcode 178,180,181 - Rank Scores, Consecutive Numbers, Employees Earning More Than Their Managers
https://lnkd.in/efibXrXG

📌 Leetcode 182,183 - Duplicate Emails, Customers Who Never Order
https://lnkd.in/e9ZSr9s2

📌 LeetCode 184 - Department Highest Salary
https://lnkd.in/evpeZrJh

I will soon be adding more videos under the Advanced and LeetCode Playlists.

Link to subscribe to my youtube channel -
https://lnkd.in/geJt-sMS

what topics in SQL you want me to cover?

#bigdata #sql #dataengineering #sumitteaches #dataanalytics

Recently, our startup ecosystem nearly fired 9000 people and to put this in practical nature, we have 9000 more unemployed folks in the job market.

People talk about loyalty of employees towards their company all the time, but where's the loyalty from company's side ?

The CEO's, CFO's and other executives fire people to cut down expenses, then why it is wrong for employees to switch for a better offer and a better job ?

Do give this a thought !

#startup #job #salary #layoffs

I have seen a lot of candidates who keep postponing the interviews.

They keep saying I am still preparing, or I am not prepared.

They feel they need more time, and want perfection.

Remember one thing, you will never feel that you are 100% prepared.

My suggestion, decide on a date, and after this date you start giving interviews no matter what.

Failing interviews is far better than not giving interviews.

why? because Failed interviews at least teach you something that is going to help you further.

So start giving interviews, and it's okay even if you get rejected in 20 interviews.

There will soon be a time when you will crack most of them.

#dataengineering #bigdata #interviews

The Absolute Big Data Fundamentals - Chapter 101

Amazing handwritten notes by one of my student preeti singh

If you are someone who is starting your journey in Big Data then this should definitely help you out.

happy learning!

#SumitTeaches

#bigdata #dataengineering #hadoop

Similar is with tech interviews. You should always be in touch with your concepts and always be ready for interviews.

If you are ever ready, then there's hardly any company that you'll be afraid of. Always prepare as if you have to interview tomorrow.

#interview