PySpark Filtering & Selection

31 interview problems on select, filter, sort and de-duplicate rows, from easy to hard.

• Filter High Earners · Easy · Filtering & Selection
• Add a Bonus Column · Easy · Filtering & Selection
• Distinct User Actions · Easy · Filtering & Selection
• Categorize Orders · Easy · Conditional Logic
• Departments with Large Teams · Easy · Aggregations
• Customers Without Orders · Medium · Joins
• Employees and Their Departments · Medium · Joins
• Latest Order per Customer · Medium · Window Functions
• Top Two Salary Levels per Department · Medium · Window Functions
• Word Count · Medium · Arrays
• Unpivot Wide Columns to Rows · Medium · Pivot, Unpivot & Rollup
• Year-over-Year Growth · Medium · Joins
• Employees Earning More than Their Manager · Medium · Joins
• Joining on Nullable Keys · Medium · Joins
• Combine Batches with Schema Drift · Medium · Filtering & Selection
• Data Quality: Null Count per Column · Medium · Null Handling
• Item Positions in a Basket (POSEXPLODE) · Medium · Arrays
• Keep Users with No Interests (EXPLODE_OUTER) · Medium · Arrays
• Month Start, Month End and Due Dates · Medium · Dates
• Parse Application Log Lines · Medium · Strings
• Three-Day Login Streak · Hard · Window Functions
• Subtotals with ROLLUP · Hard · Pivot, Unpivot & Rollup
• Trailing 3 Calendar Days (Range Frame) · Hard · Window Functions
• Sessionize a Clickstream · Hard · Window Functions
• Build an SCD Type 2 History · Hard · Window Functions
• Upsert: Merge New Records into a Table · Hard · Window Functions
• Price Valid at Order Time (Range Join) · Hard · Joins
• Reconcile Two Sources (Full Outer Join) · Hard · Joins
• Fill Missing Dates (Calendar Spine) · Hard · Joins
• Customers Who Bought Every Product · Hard · Joins
• Monthly Retention by Cohort · Hard · Joins

Browse

Topics: Window Functions · Joins · Aggregations · Pivot, Unpivot & Rollup · Arrays · Null Handling · Conditional Logic · Dates · Filtering & Selection · Strings

Difficulty: Easy · Medium · Hard · PySpark interview roadmap · All problems