org.apache.sedona.core.joinJudgement

Class DynamicIndexLookupJudgement<T extends org.locationtech.jts.geom.Geometry,U extends org.locationtech.jts.geom.Geometry>

java.lang.Object

org.apache.sedona.core.joinJudgement.DynamicIndexLookupJudgement<T,U>

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.FlatMapFunction2<Iterator<U>,Iterator<T>,org.apache.commons.lang3.tuple.Pair<U,T>>

public class DynamicIndexLookupJudgement<T extends org.locationtech.jts.geom.Geometry,U extends org.locationtech.jts.geom.Geometry>
extends Object
implements org.apache.spark.api.java.function.FlatMapFunction2<Iterator<U>,Iterator<T>,org.apache.commons.lang3.tuple.Pair<U,T>>, Serializable

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.spark.util.LongAccumulator	buildCount
protected org.apache.spark.util.LongAccumulator	candidateCount
protected org.apache.spark.util.LongAccumulator	resultCount
protected org.apache.spark.util.LongAccumulator	streamCount

Constructor Summary

Constructors

Constructor and Description
DynamicIndexLookupJudgement(SpatialPredicate spatialPredicate, IndexType indexType, JoinBuildSide joinBuildSide, org.apache.spark.util.LongAccumulator buildCount, org.apache.spark.util.LongAccumulator streamCount, org.apache.spark.util.LongAccumulator resultCount, org.apache.spark.util.LongAccumulator candidateCount)

Method Summary

Modifier and Type	Method and Description
Iterator<org.apache.commons.lang3.tuple.Pair<U,T>>	call(Iterator<U> leftShapes, Iterator<T> rightShapes)
protected boolean	hasNextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes) Iterator model for the nest loop join.
protected boolean	hasNextBase(org.locationtech.jts.index.SpatialIndex spatialIndex, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes, boolean buildLeft) Iterator model for the index-based join.
protected void	initPartition() Looks up the extent of the current partition.
protected void	log(String message, Object... params)
protected org.apache.commons.lang3.tuple.Pair<U,T>	nextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes) Iterator model for the nest loop join.
protected org.apache.commons.lang3.tuple.Pair<U,T>	nextBase(org.locationtech.jts.index.SpatialIndex spatialIndex, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes, boolean buildLeft) Iterator model for the index-based join.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

buildCount

protected final org.apache.spark.util.LongAccumulator buildCount

streamCount

protected final org.apache.spark.util.LongAccumulator streamCount

resultCount

protected final org.apache.spark.util.LongAccumulator resultCount

candidateCount

protected final org.apache.spark.util.LongAccumulator candidateCount

Constructor Detail

DynamicIndexLookupJudgement

public DynamicIndexLookupJudgement(SpatialPredicate spatialPredicate,
                                   IndexType indexType,
                                   JoinBuildSide joinBuildSide,
                                   org.apache.spark.util.LongAccumulator buildCount,
                                   org.apache.spark.util.LongAccumulator streamCount,
                                   org.apache.spark.util.LongAccumulator resultCount,
                                   org.apache.spark.util.LongAccumulator candidateCount)

See Also:

JudgementBase

Method Detail

call

public Iterator<org.apache.commons.lang3.tuple.Pair<U,T>> call(Iterator<U> leftShapes,
                                                               Iterator<T> rightShapes)
                                                        throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.FlatMapFunction2<Iterator<U extends org.locationtech.jts.geom.Geometry>,Iterator<T extends org.locationtech.jts.geom.Geometry>,org.apache.commons.lang3.tuple.Pair<U extends org.locationtech.jts.geom.Geometry,T extends org.locationtech.jts.geom.Geometry>>

Throws:

Exception

initPartition

protected void initPartition()

Looks up the extent of the current partition. If found, `match` method will activate the logic to avoid emitting duplicate join results from multiple partitions.

Must be called before processing a partition. Must be called from the same instance that will be used to process the partition.

hasNextBase

protected boolean hasNextBase(org.locationtech.jts.index.SpatialIndex spatialIndex,
                              Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes,
                              boolean buildLeft)

Iterator model for the index-based join. It checks if there is a next match and populate it to the result.

Parameters:

spatialIndex -

streamShapes -

buildLeft -

Returns:

hasNextBase

protected boolean hasNextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes,
                              Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes)

Iterator model for the nest loop join. It checks if there is a next match and populate it to the result.

Parameters:

buildShapes -

streamShapes -

Returns:

nextBase

protected org.apache.commons.lang3.tuple.Pair<U,T> nextBase(org.locationtech.jts.index.SpatialIndex spatialIndex,
                                                            Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes,
                                                            boolean buildLeft)

Iterator model for the index-based join. It returns 1 pair in the current batch. Each batch contains a list of pairs of geometries that satisfy the join condition. The current batch is the result of the current stream shape against all the build shapes.

Parameters:

spatialIndex -

streamShapes -

buildLeft -

Returns:

nextBase

protected org.apache.commons.lang3.tuple.Pair<U,T> nextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes,
                                                            Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes)

Iterator model for the nest loop join. It returns 1 pair in the current batch. Each batch contains a list of pairs of geometries that satisfy the join condition. The current batch is the result of the current stream shape against all the build shapes.

Parameters:

buildShapes -

streamShapes -

Returns:

log

protected void log(String message,
                   Object... params)