Working with Python + Spark code
A Text Editor
The Python universe is simple enough that any good text editor will do just fine.
PyCharm
If you really like IDEs, PyCharm is good.
For the Spark dependencies in PyCharm, download or find the Spark distribution, the same as you need to run Spark. In your PyCharm project: File → Settings → Project → Project Structure → Add Content Root. In the Spark distribution, add python/lib/pyspark.zip
.
On the lab Macs, you can get PyCharm by clicking the “SFU” in the upper-right of the screen, “Software Updates” and install PyCharm. In a few minutes, it will be installed in the Applications folder.
Updated Thu Aug. 22 2024, 11:06 by ggbaker.