sedona.spark.geoarrow package

Submodules

sedona.spark.geoarrow.geoarrow module

sedona.spark.geoarrow.geoarrow.create_spatial_dataframe(spark: SparkSession, gdf: gpd.GeoDataFrame) → DataFrame[source]

Create a Spark DataFrame from a GeoPandas GeoDataFrame.

Older runtimes use Sedona’s Arrow transfer path. Spark 4.2+ receives native geometry values through createDataFrame and its configured transport. Geometry columns retain their geometry schema, and other columns use the Spark types corresponding to their Arrow types.

Parameters:
  • spark – The Spark session.

  • gdf – The GeoPandas GeoDataFrame to convert. It must have at least one geometry column.

Returns:

A Spark DataFrame with the columns of gdf.

Raises:

ValueError – If gdf has no geometry column.

sedona.spark.geoarrow.geoarrow.crs_to_json(crs)[source]
sedona.spark.geoarrow.geoarrow.dataframe_to_arrow(df, crs=None)[source]

Collect a DataFrame as a PyArrow Table

In the output Table, geometry will be encoded as a GeoArrow extension type. The resulting output is compatible with lonboard.viz(), geopandas.GeoDataFrame.from_arrow(), or any library compatible with GeoArrow extension types.

Parameters:
  • df – A Spark DataFrame

  • crs – A CRS-like object (e.g., pyproj.CRS or string interpretable by pyproj.CRS). If provided, this will override any CRS present in the output geometries. If omitted, the CRS will be inferred from the values present in the output if exactly one CRS is present in the output.

Returns:

A pyarrow.Table.

sedona.spark.geoarrow.geoarrow.dataframe_to_arrow_raw(df)[source]

Backport of toArrow() (available in Spark 4.0)

sedona.spark.geoarrow.geoarrow.infer_schema(gdf: gpd.GeoDataFrame) → StructType[source]
sedona.spark.geoarrow.geoarrow.try_register_extension_types()[source]

Try to register extension types using geoarrow-types

Do this defensively, because it can fail if the extension type was registered in some other way (notably: old versions of geoarrow-pyarrow, which is a dependency of Kepler).

sedona.spark.geoarrow.geoarrow.unique_srid_from_ewkb(obj)[source]
sedona.spark.geoarrow.geoarrow.wrap_geoarrow_extension(col, spec, crs)[source]
sedona.spark.geoarrow.geoarrow.wrap_geoarrow_field(field, col, crs)[source]

Module contents