sedona.spark.geoarrow package
Submodules
sedona.spark.geoarrow.geoarrow module
- sedona.spark.geoarrow.geoarrow.create_spatial_dataframe(spark: SparkSession, gdf: gpd.GeoDataFrame) DataFrame[source]
Create a Spark DataFrame from a GeoPandas GeoDataFrame.
Older runtimes use Sedona’s Arrow transfer path. Spark 4.2+ receives native geometry values through
createDataFrameand its configured transport. Geometry columns retain their geometry schema, and other columns use the Spark types corresponding to their Arrow types.- Parameters:
spark – The Spark session.
gdf – The GeoPandas GeoDataFrame to convert. It must have at least one geometry column.
- Returns:
A Spark DataFrame with the columns of
gdf.- Raises:
ValueError – If
gdfhas no geometry column.
- sedona.spark.geoarrow.geoarrow.dataframe_to_arrow(df, crs=None)[source]
Collect a DataFrame as a PyArrow Table
In the output Table, geometry will be encoded as a GeoArrow extension type. The resulting output is compatible with lonboard.viz(), geopandas.GeoDataFrame.from_arrow(), or any library compatible with GeoArrow extension types.
- Parameters:
df – A Spark DataFrame
crs – A CRS-like object (e.g., pyproj.CRS or string interpretable by pyproj.CRS). If provided, this will override any CRS present in the output geometries. If omitted, the CRS will be inferred from the values present in the output if exactly one CRS is present in the output.
- Returns:
A
pyarrow.Table.
- sedona.spark.geoarrow.geoarrow.dataframe_to_arrow_raw(df)[source]
Backport of toArrow() (available in Spark 4.0)
- sedona.spark.geoarrow.geoarrow.infer_schema(gdf: gpd.GeoDataFrame) StructType[source]