Class BroadcastedSpatialPartitioner

  • All Implemented Interfaces:
    Serializable, scala.Serializable

    public class BroadcastedSpatialPartitioner
    extends SpatialPartitioner
    The SpatialPartitioner may contain a large number of grids, which may make the serialized tasks to be larger than 1MB and trigger a warning: "WARN DAGScheduler: Broadcasting large task binary with size XXXX KB". This class is a wrapper around a SpatialPartitioner that is broadcasted to reduce the size of serialized tasks.
    See Also:
    Serialized Form
    • Constructor Detail

      • BroadcastedSpatialPartitioner

        public BroadcastedSpatialPartitioner​(org.apache.spark.broadcast.Broadcast<SpatialPartitioner> partitioner)
    • Method Detail

      • placeObject

        public <T extends org.locationtech.jts.geom.Geometry> Iterator<scala.Tuple2<Integer,​T>> placeObject​(T spatialObject)
                                                                                                           throws Exception
        Description copied from class: SpatialPartitioner
        Given a geometry, returns a list of partitions it overlaps.

        For points, returns exactly one partition as long as grid type is non-overlapping. For other geometry types or for overlapping grid types, may return multiple partitions.

        Specified by:
        placeObject in class SpatialPartitioner
        Throws:
        Exception
      • numPartitions

        public int numPartitions()
        Specified by:
        numPartitions in class org.apache.spark.Partitioner