Class BroadcastedSpatialPartitioner
- java.lang.Object
-
- org.apache.spark.Partitioner
-
- org.apache.sedona.core.spatialPartitioning.SpatialPartitioner
-
- org.apache.sedona.core.spatialPartitioning.BroadcastedSpatialPartitioner
-
- All Implemented Interfaces:
Serializable
,scala.Serializable
public class BroadcastedSpatialPartitioner extends SpatialPartitioner
The SpatialPartitioner may contain a large number of grids, which may make the serialized tasks to be larger than 1MB and trigger a warning: "WARN DAGScheduler: Broadcasting large task binary with size XXXX KB". This class is a wrapper around a SpatialPartitioner that is broadcasted to reduce the size of serialized tasks.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.sedona.core.spatialPartitioning.SpatialPartitioner
grids, gridType
-
-
Constructor Summary
Constructors Constructor Description BroadcastedSpatialPartitioner(org.apache.spark.broadcast.Broadcast<SpatialPartitioner> partitioner)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description DedupParams
getDedupParams()
List<org.locationtech.jts.geom.Envelope>
getGrids()
int
numPartitions()
<T extends org.locationtech.jts.geom.Geometry>
Iterator<scala.Tuple2<Integer,T>>placeObject(T spatialObject)
Given a geometry, returns a list of partitions it overlaps.-
Methods inherited from class org.apache.sedona.core.spatialPartitioning.SpatialPartitioner
getGridType, getPartition
-
-
-
-
Constructor Detail
-
BroadcastedSpatialPartitioner
public BroadcastedSpatialPartitioner(org.apache.spark.broadcast.Broadcast<SpatialPartitioner> partitioner)
-
-
Method Detail
-
placeObject
public <T extends org.locationtech.jts.geom.Geometry> Iterator<scala.Tuple2<Integer,T>> placeObject(T spatialObject) throws Exception
Description copied from class:SpatialPartitioner
Given a geometry, returns a list of partitions it overlaps.For points, returns exactly one partition as long as grid type is non-overlapping. For other geometry types or for overlapping grid types, may return multiple partitions.
- Specified by:
placeObject
in classSpatialPartitioner
- Throws:
Exception
-
getDedupParams
@Nullable public DedupParams getDedupParams()
- Specified by:
getDedupParams
in classSpatialPartitioner
-
getGrids
public List<org.locationtech.jts.geom.Envelope> getGrids()
- Overrides:
getGrids
in classSpatialPartitioner
-
numPartitions
public int numPartitions()
- Specified by:
numPartitions
in classorg.apache.spark.Partitioner
-
-