PySpark interview roadmap

All 50 problems in a suggested order, from fundamentals to advanced patterns; each step goes from easy to hard.

1. Filtering & Selection

select, filter, sort and de-duplicate rows

• Filter High Earners · Easy · Filtering & Selection
• Add a Bonus Column · Easy · Filtering & Selection
• Distinct User Actions · Easy · Filtering & Selection
• Combine Batches with Schema Drift · Medium · Filtering & Selection

2. Conditional Logic

when/otherwise and derived columns

• Categorize Orders · Easy · Conditional Logic

3. Null Handling

coalesce, fillna and three-valued logic

• Replace Missing Values · Easy · Null Handling
• Data Quality: Null Count per Column · Medium · Null Handling

4. Aggregations

groupBy, agg, percentiles and conditional metrics

• Average Salary by Department · Easy · Aggregations
• Departments with Large Teams · Easy · Aggregations
• Conditional Aggregation · Medium · Aggregations
• Entry and Exit Pages (MAX_BY / MIN_BY) · Medium · Aggregations
• New Users per Day · Medium · Aggregations
• Median and 90th Percentile · Medium · Aggregations
• Conversion Funnel · Hard · Aggregations

5. Strings

clean and transform text columns

• Parse Application Log Lines · Medium · Strings

6. Dates

parse, shift and compare dates

• Month Start, Month End and Due Dates · Medium · Dates

7. Joins

inner, left, anti and semi joins across DataFrames

• Customers Without Orders · Medium · Joins
• Employees and Their Departments · Medium · Joins
• Year-over-Year Growth · Medium · Joins
• Employees Earning More than Their Manager · Medium · Joins
• Joining on Nullable Keys · Medium · Joins
• Price Valid at Order Time (Range Join) · Hard · Joins
• Reconcile Two Sources (Full Outer Join) · Hard · Joins
• Fill Missing Dates (Calendar Spine) · Hard · Joins
• Customers Who Bought Every Product · Hard · Joins
• Monthly Retention by Cohort · Hard · Joins

8. Arrays

split, explode and work with array columns

• Word Count · Medium · Arrays
• Item Positions in a Basket (POSEXPLODE) · Medium · Arrays
• Keep Users with No Interests (EXPLODE_OUTER) · Medium · Arrays
• Sorted Distinct Purchase List · Medium · Arrays

9. Window Functions

rank, lag, lead and running totals over partitions

• Latest Order per Customer · Medium · Window Functions
• Running Total by Region · Medium · Window Functions
• Top Two Salary Levels per Department · Medium · Window Functions
• Month-over-Month Change · Medium · Window Functions
• 7-Day Moving Average · Medium · Window Functions
• Share of Category Total · Medium · Window Functions
• Customer Spend Quartiles (NTILE) · Medium · Window Functions
• Percent Rank and Cumulative Distribution · Medium · Window Functions
• Three-Day Login Streak · Hard · Window Functions
• Trailing 3 Calendar Days (Range Frame) · Hard · Window Functions
• Forward-Fill Missing Readings · Hard · Window Functions
• Sessionize a Clickstream · Hard · Window Functions
• Consecutive Status Runs (Gaps and Islands) · Hard · Window Functions
• Build an SCD Type 2 History · Hard · Window Functions
• Upsert: Merge New Records into a Table · Hard · Window Functions

10. Pivot, Unpivot & Rollup

pivot, unpivot, rollup, cube and grouping sets

• Pivot Quarterly Revenue · Medium · Pivot, Unpivot & Rollup
• Unpivot Wide Columns to Rows · Medium · Pivot, Unpivot & Rollup
• Subtotals with ROLLUP · Hard · Pivot, Unpivot & Rollup
• All Combinations with CUBE · Hard · Pivot, Unpivot & Rollup
• Pivot with Several Aggregations · Hard · Pivot, Unpivot & Rollup