Skip to content

Release notes

Note

Support of Spark 2.X and Scala 2.11 was removed in Sedona 1.3.0+ although some parts of the source code might still be compatible. Sedona 1.3.0+ releases binary for both Scala 2.12 and 2.13.

Note

Sedona before 1.6.0 only works with Shapely 1.x. If you want to work with Shapely 2.x, please use Sedona no earlier than 1.6.0.

If you use Sedona < 1.6.0, please use GeoPandas <= 0.11.1 since GeoPandas > 0.11.1 will automatically install Shapely 2.0. If you use Shapely, please use <= 1.8.5.

Sedona 1.5.1

Sedona 1.5.1 is compiled against Spark 3.3 / Spark 3.4 / Spark 3.5, Flink 1.12, Snowflake 7+, Java 8.

Highlights

  • Sedona Snowflake Add support for Snowflake
  • Sedona Spark Support Spark 3.5
  • Sedona Spark Support Snowflake 7+
  • Sedona Spark Added 20+ raster functions (or variants)
  • Sedona Spark/Flink/Snowflake Added 7 vector functions (or variants)
  • Sedona Spark GeoParquet reader and writer supports projjson in metadata
  • Sedona Spark GeoParquet reader and writer conform to GeoParquet spec 1.0.0 instead of 1.0.0-beta1
  • Sedona Spark Added a legacyMode in GeoParquet reader for 1.5.1+ users to read Parquet files written by Sedona 1.3.1 and earlier
  • Sedona Spark Fixed a bug in GeoParquet writer so 1.3.1 and earlier users can read Parquet files written by 1.5.1+

Behavior change

  • All raster functions that take a geometry will implicitly transform the CRS of the geometry if needed.
  • The default CRS for these functions is 4326 for raster and geometry involved in raster functions, if not specified.
  • KeplerGL and DeckGL become optional dependencies for Sedona Spark Python.

New Contributors

Bug

  • [SEDONA-414] - ST_MakeLine in sedona-spark does not work with array inputs
  • [SEDONA-417] - Fix SedonaUtils.display_image
  • [SEDONA-419] - SedonaKepler and SedonaPyDeck should not be in `sedona.spark`
  • [SEDONA-420] - Make SedonaKepler and SedonaPydeck optional dependencies
  • [SEDONA-424] - Specify jt-jiffle as a provided dependency
  • [SEDONA-426] - Change cloning of rasters to be able to include metadata.
  • [SEDONA-440] - GeoParquet reader should support filter pushdown on nested fields
  • [SEDONA-443] - Upload-artifact leads to 503 error
  • [SEDONA-453] - Performance degrade when indexing points using Quadtree
  • [SEDONA-456] - SedonaKepler cannot work with geopandas >= 0.13.0 correctly

New Feature

Improvement

  • [SEDONA-339] - Skip irrelevant GitHub actions
  • [SEDONA-416] - importing SedonaContext, kepler.gl is not found.
  • [SEDONA-429] - geoparquet reader/writer should print "1.0.0" in its version
  • [SEDONA-434] - Improve reliability by resolve the nondeterministic of the order of the Map
  • [SEDONA-436] - Fix RS_SetValues bug
  • [SEDONA-437] - Add implicit CRS transformation
  • [SEDONA-446] - Add floating point datatype support in RS_AsBase64
  • [SEDONA-448] - RS_SetBandNoDataValue should have `replace` option
  • [SEDONA-454] - Change the default value of sedona.global.indextype from quadtree to rtree
  • [SEDONA-457] - Don't write GeometryUDT into org.apache.spark.sql.parquet.row.metadata when writing GeoParquet files
  • [SEDONA-464] - ST_Valid should have integer flags
  • [SEDONA-466] - RS_AsRaster does not use the weight and height of the raster in its parameters.

Test

  • [SEDONA-410] - pre-commit: check that scripts with shebangs are executable
  • [SEDONA-412] - pre-commit: add hook `end-of-file-fixer`
  • [SEDONA-423] - pre-commit: apply hook `end-of-file-fixer` to more files
  • [SEDONA-442] - pre-commit: add hook markdown-lint
  • [SEDONA-444] - pre-commit: add hook to trim trailing whitespace
  • [SEDONA-445] - pre-commit: apply hook end-of-file-fixer to more files
  • [SEDONA-447] - pre-commit: apply end-of-file-fixer to more files
  • [SEDONA-463] - Add a Makefile for convenience

    • Task

    Sedona 1.5.0

    Sedona 1.5.0 is compiled against Spark 3.3 / Spark 3.4 / Flink 1.12, Java 8.

    Highlights

    API breaking changes:

    • The following functions in Sedona requires the input data must be in longitude/latitude order otherwise they might throw errors. You can use FlipCoordinates to swap X and Y.
      • ST_Transform
      • ST_DistanceSphere
      • ST_DistanceSpheroid
      • ST_GeoHash
      • All ST_H3 functions
      • All ST_S2 functions
      • All RS constructors
      • All RS predicates
      • Spark RDD: CRStransform
    • Rename RS_Count to RS_CountValue
    • Drop RS_HTML
    • Unshaded Sedona Spark code are all merged to a single jar sedona-spark

    New features

    • Add 18 more ST functions for vector data processing in Sedona Spark and Sedona Flink
    • Add 36 more RS functions in Sedona Spark to support comprehensive raster data ETL and analytics
      • You can now directly join vector and raster datasets together
      • Flexible map algebra equations: SELECT RS_MapAlgebra(rast, 'D', 'out = (rast[3] - rast[0]) / (rast[3] + rast[0]);') as ndvi FROM raster_table
    • Add native support of Uber H3 functions in Sedona Spark and Sedona Flink.
    • Add SedonaKepler and SedonaPyDeck for interactive map visualization on Sedona Spark.

    Bug

    • [SEDONA-318] - SerDe for RasterUDT performs poorly
    • [SEDONA-319] - RS_AddBandFromArray does not always produce serializable rasters
    • [SEDONA-322] - The "Scala and Java build" CI job occasionally fail
    • [SEDONA-325] - RS_FromGeoTiff is leaking file descriptors
    • [SEDONA-329] - Remove geometry_col parameter from SedonaKepler APIs
    • [SEDONA-330] - Fix bugs in SedonaPyDeck
    • [SEDONA-332] - RS_Value and RS_Values don't need to fetch all the pixel data
    • [SEDONA-337] - Failure falling back to pure python implementation when geomserde_speedup is unavailable
    • [SEDONA-338] - Refactor Raster construction in sedona to use AffineTransform instead of envelope
    • [SEDONA-358] - Refactor Functions to remove geotools dependency for most vector functions
    • [SEDONA-362] - RS_BandAsArray truncates the decimal part of float/double pixel values.
    • [SEDONA-373] - Move RasterPredicates to correct raster package to prevent redundant imports
    • [SEDONA-394] - fix RS_Band data type bug
    • [SEDONA-401] - Handle null values in RS_AsMatrix
    • [SEDONA-402] - Floor grid coordinates received from geotools
    • [SEDONA-403] - Add Null tolerance to RS_AddBandFromArray
    • [SEDONA-405] - Sedona driver Out of Memory on 1.4.1

    New Feature

    Improvement

    • [SEDONA-39] - Fix the Lon/lat order issue in Sedona
    • [SEDONA-114] - Add ST_MakeLine to Apache Sedona
    • [SEDONA-142] - Add ST_Collect to Flink Catalog
    • [SEDONA-311] - Refactor InferredExpression to handle functions with arbitrary arity
    • [SEDONA-313] - Refactor ST_Affine to support signature like PostGIS
    • [SEDONA-324] - R – Fix failing tests
    • [SEDONA-326] - Improve raster band algebra functions for easier preprocessing of raster data
    • [SEDONA-327] - Refactor InferredExpression to handle GridCoverage2D
    • [SEDONA-333] - Support EWKT parser in ST_GeomFromWKT
    • [SEDONA-347] - Centralize usages of transform()
    • [SEDONA-350] - Refactor RS_AddBandFromArray to allow adding a custom noDataValue
    • [SEDONA-352] - Refactor MakeEmptyRaster to allow setting custom datatype for the raster
    • [SEDONA-360] - Handle nodata values of raster bands in a more concise way
    • [SEDONA-365] - Refactor RS_Count to RS_CountValue
    • [SEDONA-374] - RS predicates should support (geom, rast) and (rast, rast) as arguments, and use the convex hull of rasters for spatial relationship testing
    • [SEDONA-385] - Set the Maven Central to be the first repository to check
    • [SEDONA-386] - Speed up GridCoverage2D serialization
    • [SEDONA-392] - Add five more pre-commit hooks
    • [SEDONA-399] - Support Uber H3 cells
    • [SEDONA-400] - pre-commit add hook to ensure that links to vcs websites are permalinks
    • [SEDONA-408] - Set a reasonable default size for RasterUDT

    Task

    • [SEDONA-316] - Refactor Sedona Jupyter notebook examples with unified SedonaContext entrypoint
    • [SEDONA-317] - Change map visualization in Jupyter notebooks with KeplerGL
    • [SEDONA-341] - Move RS_Envelope to GeometryFunctions
    • [SEDONA-356] - Change CRS transformation from lat/lon to lon/lat order
    • [SEDONA-370] - Completely drop the old GeoTiff reader and writer
    • [SEDONA-377] - Change sphere/spheroid functions to work with coordinates in lon/lat order
    • [SEDONA-380] - Merge all Sedona Spark module to a single module
    • [SEDONA-381] - Merge python-adapter to sql module
    • [SEDONA-382] - Merge SQL and Core module to a single Spark module
    • [SEDONA-384] - Merge viz module to the spark module
    • [SEDONA-397] - Move Map Algebra functions

    Sedona 1.4.1

    Sedona 1.4.1 is compiled against Spark 3.3 / Spark 3.4 / Flink 1.12, Java 8.

    Highlights

    • Sedona Spark More raster functions and bridge RasterUDT and Map Algebra operators. See Raster based operators and Raster to Map Algebra operators.
    • Sedona Spark & Flink Added geodesic / geography functions:
      • ST_DistanceSphere
      • ST_DistanceSpheroid
      • ST_AreaSpheroid
      • ST_LengthSpheroid
    • Sedona Spark & Flink Introduced SedonaContext to unify Sedona entry points.
    • Sedona Spark Support Spark 3.4.
    • Sedona Spark Added a number of new ST functions.
    • Zeppelin Zeppelin helium plugin supports plotting geometries like linestring, polygon.

    API change

    • Sedona Spark & Flink Introduced a new entry point called SedonaContext to unify all Sedona entry points in different compute engines and deprecate old Sedona register entry points. Users no longer have to register Sedona kryo serializer and import many tedious Python classes.
      • Sedona Spark:
        • Scala: 
          import org.apache.sedona.spark.SedonaContext
val sedona = SedonaContext.create(SedonaContext.builder().master("local[*]").getOrCreate())
sedona.sql("SELECT ST_GeomFromWKT(XXX) FROM")
        • Python: 
          from sedona.spark import *

config = SedonaContext.builder().\
   config('spark.jars.packages',
       'org.apache.sedona:sedona-spark-shaded-3.0_2.12:1.4.1,'
       'org.datasyslab:geotools-wrapper:1.4.0-28.2'). \
   getOrCreate()
sedona = SedonaContext.create(config)
sedona.sql("SELECT ST_GeomFromWKT(XXX) FROM")
      • Sedona Flink: 
        import org.apache.sedona.flink.SedonaContext
StreamTableEnvironment sedona = SedonaContext.create(env, tableEnv);
sedona.sqlQuery("SELECT ST_GeomFromWKT(XXX) FROM")

    Bug

    • [SEDONA-266] - RS_Values throws UnsupportedOperationException for shuffled point arrays
    • [SEDONA-267] - Cannot pip install apache-sedona 1.4.0 from source distribution
    • [SEDONA-273] - Set a upper bound for Shapely, Pandas and GeoPandas
    • [SEDONA-277] - Sedona spark artifacts for scala 2.13 do not have proper POMs
    • [SEDONA-283] - Artifacts were deployed twice when running mvn clean deploy
    • [SEDONA-284] - Property values in dependency deduced POMs for shaded modules were not substituted

    New Feature

    • [SEDONA-196] - Add ST_Force3D to Sedona
    • [SEDONA-239] - Implement ST_NumPoints
    • [SEDONA-264] - zeppelin helium plugin supports plotting geometry like linestring, polygon
    • [SEDONA-280] - Add ST_GeometricMedian
    • [SEDONA-281] - Support geodesic / geography functions
    • [SEDONA-286] - Support optimized distance join on ST_DistanceSpheroid and ST_DistanceSphere
    • [SEDONA-287] - Use SedonaContext to unify Sedona entry points
    • [SEDONA-292] - Bridge Sedona Raster and Map Algebra operators
    • [SEDONA-297] - Implement ST_NRings
    • [SEDONA-302] - Implement ST_Translate

    Improvement

    • [SEDONA-167] - Add __pycache__ to Python .gitignore
    • [SEDONA-265] - Migrate all ST functions to Sedona Inferred Expressions
    • [SEDONA-269] - Add data source for writing binary files
    • [SEDONA-270] - Remove redundant serialization for rasters
    • [SEDONA-271] - Add raster function RS_SRID
    • [SEDONA-274] - Move all ST function logics to Sedona common
    • [SEDONA-275] - Add raster function RS_SetSRID
    • [SEDONA-276] - Add support for Spark 3.4
    • [SEDONA-279] - Sedona-Flink should not depend on Sedona-Spark modules
    • [SEDONA-282] - R – Add raster write function
    • [SEDONA-290] - RDD Spatial Joins should follow the iterator model

    Sedona 1.4.0

    Sedona 1.4.0 is compiled against, Spark 3.3 / Flink 1.12, Java 8.

    Highlights

    • Sedona Spark & Flink Serialize and deserialize geometries 3 - 7X faster
    • Sedona Spark & Flink Google S2 based spatial join for fast approximate point-in-polygon join. See Join query in Spark and Join query in Flink
    • Sedona Spark Pushdown spatial predicate on GeoParquet to reduce memory consumption by 10X: see explanation
    • Sedona Spark Automatically use broadcast index spatial join for small datasets
    • Sedona Spark New RasterUDT added to Sedona GeoTiff reader.
    • Sedona Spark A number of bug fixes and improvement to the Sedona R module.

    API change

    • Sedona Spark & Flink Packaging strategy changed. See Maven Coordinate. Please change your Sedona dependencies if needed. We recommend sedona-spark-shaded-3.0_2.12-1.4.0 and sedona-flink-shaded_2.12-1.4.0
    • Sedona Spark & Flink GeoTools-wrapper version upgraded. Please use geotools-wrapper-1.4.0-28.2.

    Behavior change

    • Sedona Flink Sedona Flink no longer outputs any LinearRing type geometry. All LinearRing are changed to LineString.
    • Sedona Spark Join optimization strategy changed. Sedona no longer optimizes spatial join when use a spatial predicate together with a equijoin predicate. By default, it prefers equijoin whenever possible. SedonaConf adds a config option called sedona.join.optimizationmode, it can be configured as one of the following values:
      • all: optimize all joins having spatial predicate in join conditions. This was the behavior of Apache Sedona prior to 1.4.0.
      • none: disable spatial join optimization.
      • nonequi: only enable spatial join optimization on non-equi joins. This is the default mode.

    When sedona.join.optimizationmode is configured as nonequi, it won't optimize join queries such as SELECT * FROM A, B WHERE A.x = B.x AND ST_Contains(A.geom, B.geom), since it is an equi-join with equi-condition A.x = B.x. Sedona will optimize for SELECT * FROM A, B WHERE ST_Contains(A.geom, B.geom)

    Bug

    • [SEDONA-218] - Flaky test caused by improper handling of null struct values in Adapter.toDf
    • [SEDONA-221] - Outer join throws NPE for null geometries
    • [SEDONA-222] - GeoParquet reader does not work in non-local mode
    • [SEDONA-224] - java.lang.NoSuchMethodError when loading GeoParquet files using Spark 3.0.x ~ 3.2.x
    • [SEDONA-225] - Cannot count dataframes loaded from GeoParquet files
    • [SEDONA-227] - Python SerDe Performance Degradation
    • [SEDONA-230] - rdd.saveAsGeoJSON should generate feature properties with field names
    • [SEDONA-233] - Incorrect results for several joins in a single stage
    • [SEDONA-236] - Flakey python tests in tests.serialization.test_[de]serializers
    • [SEDONA-242] - Update jars dependencies in Sedona R to Sedona 1.4.0 version
    • [SEDONA-250] - R Deprecate use of Spark 2.4
    • [SEDONA-252] - Fix disabled RS_Base64 test
    • [SEDONA-255] - R – Translation issue for ST_Point and ST_PolygonFromEnvelope
    • [SEDONA-258] - Cannot directly assign raw spatial RDD to CircleRDD using Python binding
    • [SEDONA-259] - Adapter.toSpatialRdd in Python binding does not have valid implementation for specifying custom field names for user data
    • [SEDONA-261] - Cannot run distance join using broadcast index join when the distance expression references to attributes from the right-side relation

    New Feature

    • [SEDONA-156] - predicate pushdown support for GeoParquet
    • [SEDONA-215] - Add ST_ConcaveHull
    • [SEDONA-216] - Upgrade jts version to 1.19.0
    • [SEDONA-235] - Create ST_S2CellIds in Sedona
    • [SEDONA-246] - R GeoTiff read/write
    • [SEDONA-254] - R – Add raster type
    • [SEDONA-262] - Don't optimize equi-join by default, add an option to configure when to optimize spatial joins

    Improvement

    • [SEDONA-205] - Use BinaryType in GeometryUDT in Sedona Spark
    • [SEDONA-207] - Faster serialization/deserialization of geometry objects
    • [SEDONA-212] - Move shading to separate maven modules
    • [SEDONA-217] - Automatically broadcast small datasets
    • [SEDONA-220] - Upgrade Ubuntu build image from 18.04 to 20.04
    • [SEDONA-226] - Support reading and writing GeoParquet file metadata
    • [SEDONA-228] - Standardize logging dependencies
    • [SEDONA-231] - Redundant Serde Removal
    • [SEDONA-234] - ST_Point inconsistencies
    • [SEDONA-243] - Improve Sedona R file readers: GeoParquet and Shapefile
    • [SEDONA-244] - Align R read/write functions with the Sparklyr framework
    • [SEDONA-249] - Add jvm flags for running tests on Java 17
    • [SEDONA-251] - Add raster type to Sedona
    • [SEDONA-253] - Upgrade geotools to version 28.2
    • [SEDONA-260] - More intuitive configuration of partition and index-build side of spatial joins in Sedona SQL

    Sedona 1.3.1

    This version is a minor release on Sedoma 1.3.0 line. It fixes a few critical bugs in 1.3.0. We suggest all 1.3.0 users to migrate to this version.

    Bug fixes

    • SEDONA-204 - Init value in X/Y/Z max should be -Double.MAX
    • SEDONA-206 - Performance regression of ST_Transform in 1.3.0-incubating
    • SEDONA-210 - 1.3.0-incubating doesn't work with Scala 2.12 sbt projects
    • SEDONA-211 - Enforce release managers to use JDK 8
    • SEDONA-201 - Implement ST_MLineFromText and ST_MPolyFromText methods

    New Feature

    Improvement

    • SEDONA-194 - Merge org.datasyslab.sernetcdf into Sedona
    • SEDONA-208 - Use Spark RuntimeConfig in SedonaConf

    Sedona 1.3.0

    This version is a major release on Sedona 1.3.0 line and consists of 50 PRs. It includes many new functions, optimization and bug fixes.

    Highlights

    • Sedona on Spark in this release is compiled against Spark 3.3.
    • Sedona on Flink in this release is compiled against Flink 1.14.
    • Scala 2.11 support is removed.
    • Spark 2.X support is removed.
    • Python 3.10 support is added.
    • Aggregators in Flink are added
    • Correctness fixes for corner cases in range join and distance join.
    • Native GeoParquet read and write (../../tutorial/sql/#load-geoparquet).
      • df = spark.read.format("geoparquet").option("fieldGeometry", "myGeometryColumn").load("PATH/TO/MYFILE.parquet")
      • df.write.format("geoparquet").save("PATH/TO/MYFILE.parquet")
    • DataFrame style API (../../tutorial/sql/#dataframe-style-api)
      • df.select(ST_Point(min_value, max_value).as("point"))
    • Allow WKT format CRS in ST_Transform
      • ST_Transform(geom, "srcWktString", "tgtWktString")
    GEOGCS["WGS 84",
  DATUM["WGS_1984",
  SPHEROID["WGS 84",6378137,298.257223563,
  AUTHORITY["EPSG","7030"]],
  AUTHORITY["EPSG","6326"]],
  PRIMEM["Greenwich",0,
  AUTHORITY["EPSG","8901"]],
  UNIT["degree",0.0174532925199433,
  AUTHORITY["EPSG","9122"]],
  AUTHORITY["EPSG","4326"]]

    Bug fixes

    • SEDONA-119 - ST_Touches join query returns true for polygons whose interiors intersect
    • SEDONA-136 - Enable testAsEWKT for Flink
    • SEDONA-137 - Fix ST_Buffer for Flink to work
    • SEDONA-138 - Fix ST_GeoHash for Flink to work
    • SEDONA-153 - Python Serialization Fails with Nulls
    • SEDONA-158 - Fix wrong description about ST_GeometryN in the API docs
    • SEDONA-169 - Fix ST_RemovePoint in accordance with the API document
    • SEDONA-178 - Correctness issue in distance join queries
    • SEDONA-182 - ST_AsText should not return SRID
    • SEDONA-186 - collecting result rows of a spatial join query with SELECT * fails with serde error
    • SEDONA-188 - Python warns about missing jars even when some are found
    • SEDONA-193 - ST_AsBinary produces EWKB by mistake

    New Features

    Improvement

    • SEDONA-121 - Add equivalent constructors left over from Spark to Flink
    • SEDONA-132 - Create common module for SQL functions
    • SEDONA-133 - Allow user-defined schemas in Adapter.toDf()
    • SEDONA-139 - Fix wrong argument order in Flink unit tests
    • SEDONA-140 - Update Sedona Dependencies in R Package
    • SEDONA-143 - Add missing unit tests for the Flink predicates
    • SEDONA-144 - Add ST_AsGeoJSON to the Flink API
    • SEDONA-145 - Fix ST_AsEWKT to reserve the Z coordinate
    • SEDONA-146 - Add missing output functions to the Flink API
    • SEDONA-147 - Add SRID functions to the Flink API
    • SEDONA-148 - Add boolean functions to the Flink API
    • SEDONA-149 - Add Python 3.10 support
    • SEDONA-151 - Add ST aggregators to Sedona Flink
    • SEDONA-152 - Add reader/writer functions for GML and KML
    • SEDONA-154 - Add measurement functions to the Flink API
    • SEDONA-157 - Add coordinate accessors to the Flink API
    • SEDONA-159 - Add Nth accessor functions to the Flink API
    • SEDONA-160 - Fix geoparquetIOTests.scala to cleanup after test
    • SEDONA-161 - Add ST_Boundary to the Flink API
    • SEDONA-162 - Add ST_Envelope to the Flink API
    • SEDONA-163 - Better handle of unsupported types in shapefile reader
    • SEDONA-164 - Add geometry count functions to the Flink API
    • SEDONA-165 - Upgrade Apache Rat to 0.14
    • SEDONA-170 - Add ST_AddPoint and ST_RemovePoint to the Flink API
    • SEDONA-172 - Add ST_LineFromMultiPoint to Apache Sedona
    • SEDONA-176 - Make ST_Contains conform with OGC standard, and add ST_Covers and ST_CoveredBy functions.
    • SEDONA-177 - Support spatial predicates other than INTERSECTS and COVERS/COVERED_BY in RangeQuery.SpatialRangeQuery and JoinQuery.SpatialJoinQuery
    • SEDONA-181 - Build fails with java.lang.IllegalAccessError: class org.apache.spark.storage.StorageUtils$
    • SEDONA-189 - Prepare geometries in broadcast join
    • SEDONA-192 - Null handling in predicates
    • SEDONA-195 - Add wkt validation and an optional srid to ST_GeomFromWKT/ST_GeomFromText

    Task

    • SEDONA-150 - Drop Spark 2.4 and Scala 2.11 support

    Sedona 1.2.1

    This version is a maintenance release on Sedona 1.2.0 line. It includes bug fixes.

    Sedona on Spark is now compiled against Spark 3.3, instead of Spark 3.2.

    SQL (for Spark)

    Bug fixes:

    • SEDONA-104: Bug in reading band values of GeoTiff images
    • SEDONA-118: Fix the wrong result in ST_Within
    • SEDONA-123: Fix the check for invalid lat/lon in ST_GeoHash

    Improvement:

    New features:

    New features:

    Sedona 1.2.0

    This version is a major release on Sedona 1.2.0 line. It includes bug fixes and new features: Sedona with Apache Flink.

    RDD

    Bug fix:

    • SEDONA-18: Fix an error reading Shapefile
    • SEDONA-73: Exclude scala-library from scala-collection-compat

    Improvement:

    • SEDONA-77: Refactor Format readers and spatial partitioning functions to be standalone libraries. So they can be used by Flink and others.

    SQL

    New features:

    • SEDONA-4: Handle nulls in SQL functions
    • SEDONA-65: Create ST_Difference function
    • SEDONA-68 Add St_Collect function.
    • SEDONA-82: Create ST_SymmDifference function
    • SEDONA-75: Add support for "3D" geometries: Preserve Z coordinates on geometries when serializing, ST_AsText, ST_Z, ST_3DDistance
    • SEDONA-86: Support empty geometries in ST_AsBinary and ST_AsEWKB
    • SEDONA-90: Add ST_Union
    • SEDONA-100: Add st_multi function

    Bug fix:

    • SEDONA-89: GeometryUDT equals should test equivalence of the other object

    Major update:

    • SEDONA-80: Geospatial stream processing support in Flink Table API
    • SEDONA-85: ST_Geohash function in Flink
    • SEDONA-87: Support Flink Table and DataStream conversion
    • SEDONA-93: Add ST_GeomFromGeoJSON

    Sedona 1.1.1

    This version is a maintenance release on Sedona 1.1.X line. It includes bug fixes and a few new functions.

    Global

    New feature:

    • SEDONA-73: Scala source code supports Scala 2.13

    SQL

    Bug fix:

    New features:

    • SEDONA-43: Add ST_GeoHash and ST_GeomFromGeoHash
    • SEDONA-45: Add ST_MakePolygon
    • SEDONA-71: Add ST_AsBinary, ST_AsEWKB, ST_SRID, ST_SetSRID

    Sedona 1.1.0

    This version is a major release on Sedona 1.1.0 line. It includes bug fixes and new features: R language API, Raster data and Map algebra support

    Global

    Dependency upgrade:

    • SEDONA-30: Use Geotools-wrapper 1.1.0-24.1 to include geotools GeoTiff libraries.

    Improvement on join queries in core and SQL:

    • SEDONA-63: Skip empty partitions in NestedLoopJudgement
    • SEDONA-64: Broadcast dedupParams to improve performance

    Behavior change:

    • SEDONA-62: Ignore HDF test in order to avoid NASA copyright issue

    Core

    Bug fix:

    • SEDONA-41: Fix rangeFilter bug when the leftCoveredByRight para is false
    • SEDONA-53: Fix SpatialKnnQuery NullPointerException

    SQL

    Major update:

    • SEDONA-30: Add GeoTiff raster I/O and Map Algebra function

    New function:

    • SEDONA-27: Add ST_Subdivide and ST_SubdivideExplode functions

    Bug fix:

    • SEDONA-56: Fix broadcast join with Adapter Query Engine enabled
    • SEDONA-22, SEDONA-60: Fix join queries in SparkSQL when one side has no rows or only one row

    Viz

    N/A

    Python

    Improvement:

    • SEDONA-59: Make pyspark dependency of Sedona Python optional

    Bug fix:

    • SEDONA-50: Remove problematic logging conf that leads to errors on Databricks
    • Fix the issue: Spark dependency in setup.py was configured to be < v3.1.0 by mistake.

    R

    Major update:

    Sedona 1.0.1

    This version is a maintenance release on Sedona 1.0.0 line. It includes bug fixes, some new features, one API change

    Known issue

    In Sedona v1.0.1 and earlier versions, the Spark dependency in setup.py was configured to be < v3.1.0 by mistake. When you install Sedona Python (apache-sedona v1.0.1) from PyPI, pip might uninstall PySpark 3.1.1 and install PySpark 3.0.2 on your machine.

    Three ways to fix this:

    1. After install apache-sedona v1.0.1, uninstall PySpark 3.0.2 and reinstall PySpark 3.1.1

    2. Ask pip not to install Sedona dependencies: pip install --no-deps apache-sedona

    3. Install Sedona from the latest setup.py (on GitHub) manually.

    Global

    Dependency upgrade:

    • SEDONA-16: Use a GeoTools Maven Central wrapper to fix failed Jupyter notebook examples
    • SEDONA-29: upgrade to Spark 3.1.1
    • SEDONA-33: jts2geojson version from 0.14.3 to 0.16.1

    Core

    Bug fix:

    • SEDONA-35: Address user-data mutability issue with Adapter.toDF()

    SQL

    Bug fix:

    • SEDONA-14: Saving dataframe to CSV or Parquet fails due to unknown type
    • SEDONA-15: Add ST_MinimumBoundingRadius and ST_MinimumBoundingCircle functions
    • SEDONA-19: Global indexing does not work with SQL joins
    • SEDONA-20: Case object GeometryUDT and GeometryUDT instance not equal in Spark 3.0.2

    New function:

    • SEDONA-21: allows Sedona to be used in pure SQL environment
    • SEDONA-24: Add ST_LineSubString and ST_LineInterpolatePoint
    • SEDONA-26: Add broadcast join support

    Viz

    Improvement:

    API change:

    • SEDONA-29: Upgrade to Spark 3.1.1 and fix ST_Pixelize

    Python

    Bug fix:

    • SEDONA-19: Global indexing does not work with SQL joins

    Sedona 1.0.0

    This version is the first Sedona release since it joins the Apache Incubator. It includes new functions, bug fixes, and API changes.

    Global

    Key dependency upgrade:

    • SEDONA-1: upgrade to JTS 1.18
    • upgrade to GeoTools 24.0
    • upgrade to jts2geojson 0.14.3

    Key dependency packaging strategy change:

    • JTS, GeoTools, jts2geojson are no longer packaged in Sedona jars. End users need to add them manually. See here.

    Key compilation target change:

    • SEDONA-3: Paths and class names have been changed to Apache Sedona
    • SEDONA-7: build the source code for Spark 2.4, 3.0, Scala 2.11, 2.12, Python 3.7, 3.8, 3.9. See here.

    Sedona-core

    Bug fix:

    • PR 443: read multiple Shape Files by multiPartitions
    • PR 451 (API change): modify CRSTransform to ignore datum shift

    New function:

    • SEDONA-8: spatialRDD.flipCoordinates()

    API / behavior change:

    • PR 488: JoinQuery.SpatialJoinQuery/DistanceJoinQuery now returns <Geometry, List> instead of <Geometry, HashSet> because we can no longer use HashSet in Sedona for duplicates removal. All original duplicates in both input RDDs will be preserved in the output.

    Sedona-sql

    Bug fix:

    • SEDONA-8 (API change): ST_Transform slow due to lock contention.
    • PR 427: ST_Point and ST_PolygonFromEnvelope now allows Double type

    New function:

    • PR 499: ST_Azimuth, ST_X, ST_Y, ST_StartPoint, ST_Boundary, ST_EndPoint, ST_ExteriorRing, ST_GeometryN, ST_InteriorRingN, ST_Dump, ST_DumpPoints, ST_IsClosed, ST_NumInteriorRings, ST_AddPoint, ST_RemovePoint, ST_IsRing
    • PR 459: ST_LineMerge
    • PR 460: ST_NumGeometries
    • PR 469: ST_AsGeoJSON
    • SEDONA-8: ST_FlipCoordinates

    Behavior change:

    • PR 480: Aggregate Functions rewrite for new Aggregator API. The functions can be used as typed functions in code and enable compilation-time type check.

    API change:

    • SEDONA-11: Adapter.toDf() will directly generate a geometry type column. ST_GeomFromWKT is no longer needed.

    Sedona-viz

    API change: Drop the function which can generate SVG vector images because the required library has an incompatible license and the SVG image is not good at plotting big data

    Sedona Python

    API/Behavior change:

    • Python-to-Sedona adapter is moved to a separate module. To use Sedona Python, see here

    New function:

    • PR 448: Add support for partition number in spatialPartitioning function spatial_rdd.spatialPartitioning(grid_type, NUM_PARTITION)