sedona.spark package

Subpackages

Submodules

sedona.spark.SedonaContext module

class sedona.spark.SedonaContext.SedonaContext[source]

Bases: object

Entry point for using Sedona from PySpark.

SedonaContext.builder returns a SparkSession builder configured with Sedona’s Kryo serializer, and SedonaContext.create registers Sedona’s spatial types, functions and optimizations on a SparkSession.

Examples

>>> from sedona.spark import SedonaContext
>>> config = SedonaContext.builder().getOrCreate()  
>>> sedona = SedonaContext.create(config)  

__init__() → None: Method generated by attrs for class SedonaContext.

classmethod builder() → <pyspark.sql.session.SparkSession.Builder object at 0x7f92c67e5c90>[source]: This method adds the basic Sedona configuration to the SparkSession builder. Usually the user does not need to call this method directly, as it is configured when a cluster is created. This method is needed when the user wants to manually configure Sedona :return: SparkSession.builder

classmethod create(spark: SparkSession) → SparkSession[source]

This is the core of whole package, It uses py4j to run wrapper which takes existing SparkSession and register the core logics of Apache Sedona, for this SparkSession.

Parameters:: spark – pyspark.sql.SparkSession, spark session instance
Returns:: SedonaContext which is an instance of SparkSession

Module contents

sedona.spark.to_sedonadb(self, connection=None)[source]: Converts a SedonaSpark DataFrame to a SedonaDB DataFrame. :param connection: Optional SedonaDB connection object. If None, a new connection will be created. :return: SedonaDB DataFrame