org.apache.sedona.core.spatialPartitioning

Class BroadcastedSpatialPartitioner

java.lang.Object

org.apache.spark.Partitioner

org.apache.sedona.core.spatialPartitioning.SpatialPartitioner

org.apache.sedona.core.spatialPartitioning.BroadcastedSpatialPartitioner

All Implemented Interfaces:

Serializable

public class BroadcastedSpatialPartitioner
extends SpatialPartitioner

The SpatialPartitioner may contain a large number of grids, which may make the serialized tasks to be larger than 1MB and trigger a warning: "WARN DAGScheduler: Broadcasting large task binary with size XXXX KB". This class is a wrapper around a SpatialPartitioner that is broadcasted to reduce the size of serialized tasks.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.sedona.core.spatialPartitioning.SpatialPartitioner

grids, gridType

Constructor Summary

Constructors

Constructor and Description
BroadcastedSpatialPartitioner(org.apache.spark.broadcast.Broadcast<SpatialPartitioner> partitioner)

Method Summary

Modifier and Type	Method and Description
DedupParams	getDedupParams()
List<org.locationtech.jts.geom.Envelope>	getGrids()
int	numPartitions()
<T extends org.locationtech.jts.geom.Geometry> Iterator<scala.Tuple2<Integer,T>>	placeObject(T spatialObject) Given a geometry, returns a list of partitions it overlaps.

Methods inherited from class org.apache.sedona.core.spatialPartitioning.SpatialPartitioner

getGridType, getPartition

Methods inherited from class org.apache.spark.Partitioner

defaultPartitioner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BroadcastedSpatialPartitioner

public BroadcastedSpatialPartitioner(org.apache.spark.broadcast.Broadcast<SpatialPartitioner> partitioner)

Method Detail

placeObject

public <T extends org.locationtech.jts.geom.Geometry> Iterator<scala.Tuple2<Integer,T>> placeObject(T spatialObject)
                                                                                             throws Exception

Description copied from class: SpatialPartitioner

Given a geometry, returns a list of partitions it overlaps.

For points, returns exactly one partition as long as grid type is non-overlapping. For other geometry types or for overlapping grid types, may return multiple partitions.

Specified by:

placeObject in class SpatialPartitioner

Throws:

Exception

getDedupParams

@Nullable
public DedupParams getDedupParams()

Specified by:

getDedupParams in class SpatialPartitioner

getGrids

public List<org.locationtech.jts.geom.Envelope> getGrids()

Overrides:

getGrids in class SpatialPartitioner

numPartitions

public int numPartitions()

Specified by:

numPartitions in class org.apache.spark.Partitioner