PySpark interview roadmap
All 69 problems in a suggested order, from fundamentals to advanced patterns; each step goes from easy to hard.
1. Filtering & Selection
select, filter, sort and de-duplicate rows
- Filter High Earners · Easy · Filtering & Selection
- Add a Bonus Column · Easy · Filtering & Selection
- Distinct User Actions · Easy · Filtering & Selection
- Build a Price List · Easy · Filtering & Selection
- Paid Orders in March · Easy · Filtering & Selection
- Sort Delivery Options · Easy · Filtering & Selection
- Combine Batches with Schema Drift · Medium · Filtering & Selection
2. Conditional Logic
when/otherwise and derived columns
- Categorize Orders · Easy · Conditional Logic
3. Null Handling
coalesce, fillna and three-valued logic
- Replace Missing Values · Easy · Null Handling
- Data Quality: Null Count per Column · Medium · Null Handling
4. Aggregations
groupBy, agg, percentiles and conditional metrics
- Average Salary by Department · Easy · Aggregations
- Departments with Large Teams · Easy · Aggregations
- Find Partitions That Need Compaction · Easy · Aggregations
- Find Repeat Customers · Easy · Aggregations
- Conditional Aggregation · Medium · Aggregations
- Entry and Exit Pages (MAX_BY / MIN_BY) · Medium · Aggregations
- New Users per Day · Medium · Aggregations
- Median and 90th Percentile · Medium · Aggregations
- Iceberg Hidden Partition Layout · Medium · Aggregations
- Conversion Funnel · Hard · Aggregations
- Net Changes from a Change Feed · Hard · Aggregations
5. Strings
clean and transform text columns
- Parse Application Log Lines · Medium · Strings
- Parse UTM Parameters · Medium · Strings
6. Dates
parse, shift and compare dates
- Label Late Shipments · Easy · Dates
- Month Start, Month End and Due Dates · Medium · Dates
7. Joins
inner, left, anti and semi joins across DataFrames
- Order Lines with Product Details · Easy · Joins
- Customers Without Orders · Medium · Joins
- Employees and Their Departments · Medium · Joins
- Year-over-Year Growth · Medium · Joins
- Employees Earning More than Their Manager · Medium · Joins
- Joining on Nullable Keys · Medium · Joins
- Rebuild a Table from Its Transaction Log · Medium · Joins
- Files VACUUM Can Delete · Medium · Joins
- How Much Does Each Query Read? · Medium · Joins
- Pick the Join Strategy · Medium · Joins
- Price Valid at Order Time (Range Join) · Hard · Joins
- Reconcile Two Sources (Full Outer Join) · Hard · Joins
- Fill Missing Dates (Calendar Spine) · Hard · Joins
- Customers Who Bought Every Product · Hard · Joins
- Monthly Retention by Cohort · Hard · Joins
- Find Hot Keys and Plan Salting · Hard · Joins
- What Changed Between Two Versions · Hard · Joins
8. Arrays
split, explode and work with array columns
- Word Count · Medium · Arrays
- Item Positions in a Basket (POSEXPLODE) · Medium · Arrays
- Keep Users with No Interests (EXPLODE_OUTER) · Medium · Arrays
- Sorted Distinct Purchase List · Medium · Arrays
- Learning Path per Student · Medium · Arrays
9. Window Functions
rank, lag, lead and running totals over partitions
- Latest Order per Customer · Medium · Window Functions
- Running Total by Region · Medium · Window Functions
- Top Two Salary Levels per Department · Medium · Window Functions
- Month-over-Month Change · Medium · Window Functions
- 7-Day Moving Average · Medium · Window Functions
- Share of Category Total · Medium · Window Functions
- Customer Spend Quartiles (NTILE) · Medium · Window Functions
- Percent Rank and Cumulative Distribution · Medium · Window Functions
- Three-Day Login Streak · Hard · Window Functions
- Trailing 3 Calendar Days (Range Frame) · Hard · Window Functions
- Forward-Fill Missing Readings · Hard · Window Functions
- Sessionize a Clickstream · Hard · Window Functions
- Consecutive Status Runs (Gaps and Islands) · Hard · Window Functions
- Build an SCD Type 2 History · Hard · Window Functions
- Upsert: Merge New Records into a Table · Hard · Window Functions
- Apply a CDC Batch (MERGE Semantics) · Hard · Window Functions
10. Pivot, Unpivot & Rollup
pivot, unpivot, rollup, cube and grouping sets
- Pivot Quarterly Revenue · Medium · Pivot, Unpivot & Rollup
- Unpivot Wide Columns to Rows · Medium · Pivot, Unpivot & Rollup
- Survey Scores by Question · Medium · Pivot, Unpivot & Rollup
- Subtotals with ROLLUP · Hard · Pivot, Unpivot & Rollup
- All Combinations with CUBE · Hard · Pivot, Unpivot & Rollup
- Pivot with Several Aggregations · Hard · Pivot, Unpivot & Rollup