Using Spark and Python in same IDE

Question

I am using Spyder(Anaconda) on Mac for Python development. I also have installed PySpark on my machine, which i use from the terminal. Is it possible to use both of them in Spyder, or somehow manage to import the spark context into my python 2.7?

Community · Accepted Answer · 2017-05-23 10:28:11Z

0

yes it is possible just install

pip install findspark

then run

findspark.init()

https://stackoverflow.com/a/34763240

then try to import pyspark if it works then good or else add pyspark to pythonpath & try again

# Add the PySpark classes to the Python path:
export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH

edited May 23, 2017 at 10:28

CommunityBot

11 silver badge

answered May 6, 2016 at 6:03

shivsn

7,9141 gold badge28 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using Spark and Python in same IDE

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related