Apache Spark Optimization Techniques
Before discuss various optimization techniques have a quick review how does spark run
How does spark run:
User submits an application using spark-submit in cluster mode (there are local and client modes too, but considering production situation).
The spark-submit utility will then communicate with Resource Manager to start the…