org.apache.sedona.core.joinJudgement

Class KnnJoinIndexJudgement<T extends org.locationtech.jts.geom.Geometry,U extends org.locationtech.jts.geom.Geometry>

java.lang.Object

org.apache.sedona.core.joinJudgement.KnnJoinIndexJudgement<T,U>

Type Parameters:

T - extends Geometry - the type of geometries in the left set

U - extends Geometry - the type of geometries in the right set

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.FlatMapFunction2<Iterator<T>,Iterator<org.locationtech.jts.index.SpatialIndex>,org.apache.commons.lang3.tuple.Pair<U,T>>

public class KnnJoinIndexJudgement<T extends org.locationtech.jts.geom.Geometry,U extends org.locationtech.jts.geom.Geometry>
extends Object
implements org.apache.spark.api.java.function.FlatMapFunction2<Iterator<T>,Iterator<org.locationtech.jts.index.SpatialIndex>,org.apache.commons.lang3.tuple.Pair<U,T>>, Serializable

This class is responsible for performing a K-nearest neighbors (KNN) join operation using a spatial index. It extends the JudgementBase class and implements the FlatMapFunction2 interface.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.spark.util.LongAccumulator	buildCount
protected org.apache.spark.util.LongAccumulator	candidateCount
protected org.apache.spark.util.LongAccumulator	resultCount
protected org.apache.spark.util.LongAccumulator	streamCount

Constructor Summary

Constructors

Constructor and Description

KnnJoinIndexJudgement(int k, DistanceMetric distanceMetric, boolean includeTies, org.apache.spark.broadcast.Broadcast<STRtree> broadcastedTreeIndex, org.apache.spark.util.LongAccumulator buildCount, org.apache.spark.util.LongAccumulator streamCount, org.apache.spark.util.LongAccumulator resultCount, org.apache.spark.util.LongAccumulator candidateCount) Constructor for the KnnJoinIndexJudgement class.

Method Summary

Modifier and Type	Method and Description
Iterator<org.apache.commons.lang3.tuple.Pair<U,T>>	call(Iterator<T> streamShapes, Iterator<org.locationtech.jts.index.SpatialIndex> treeIndexes) This method performs the KNN join operation.
protected boolean	hasNextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes) Iterator model for the nest loop join.
protected boolean	hasNextBase(org.locationtech.jts.index.SpatialIndex spatialIndex, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes, boolean buildLeft) Iterator model for the index-based join.
protected void	initPartition() Looks up the extent of the current partition.
protected void	log(String message, Object... params)
protected org.apache.commons.lang3.tuple.Pair<U,T>	nextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes) Iterator model for the nest loop join.
protected org.apache.commons.lang3.tuple.Pair<U,T>	nextBase(org.locationtech.jts.index.SpatialIndex spatialIndex, Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes, boolean buildLeft) Iterator model for the index-based join.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

buildCount

protected final org.apache.spark.util.LongAccumulator buildCount

streamCount

protected final org.apache.spark.util.LongAccumulator streamCount

resultCount

protected final org.apache.spark.util.LongAccumulator resultCount

candidateCount

protected final org.apache.spark.util.LongAccumulator candidateCount

Constructor Detail

KnnJoinIndexJudgement

public KnnJoinIndexJudgement(int k,
                             DistanceMetric distanceMetric,
                             boolean includeTies,
                             org.apache.spark.broadcast.Broadcast<STRtree> broadcastedTreeIndex,
                             org.apache.spark.util.LongAccumulator buildCount,
                             org.apache.spark.util.LongAccumulator streamCount,
                             org.apache.spark.util.LongAccumulator resultCount,
                             org.apache.spark.util.LongAccumulator candidateCount)

Constructor for the KnnJoinIndexJudgement class.

Parameters:

k - the number of nearest neighbors to find

distanceMetric - the distance metric to use

buildCount - accumulator for the number of geometries processed from the build side

streamCount - accumulator for the number of geometries processed from the stream side

resultCount - accumulator for the number of join results

candidateCount - accumulator for the number of candidate matches

broadcastedTreeIndex - the broadcasted spatial index

Method Detail

call

public Iterator<org.apache.commons.lang3.tuple.Pair<U,T>> call(Iterator<T> streamShapes,
                                                               Iterator<org.locationtech.jts.index.SpatialIndex> treeIndexes)
                                                        throws Exception

This method performs the KNN join operation. It iterates over the geometries in the stream side and uses the spatial index to find the k nearest neighbors for each geometry. The method returns an iterator over the join results.

Specified by:

call in interface org.apache.spark.api.java.function.FlatMapFunction2<Iterator<T extends org.locationtech.jts.geom.Geometry>,Iterator<org.locationtech.jts.index.SpatialIndex>,org.apache.commons.lang3.tuple.Pair<U extends org.locationtech.jts.geom.Geometry,T extends org.locationtech.jts.geom.Geometry>>

Parameters:

streamShapes - iterator over the geometries in the stream side

treeIndexes - iterator over the spatial indexes

Returns:

an iterator over the join results

Throws:

Exception - if the spatial index is not of type STRtree

initPartition

protected void initPartition()

Looks up the extent of the current partition. If found, `match` method will activate the logic to avoid emitting duplicate join results from multiple partitions.

Must be called before processing a partition. Must be called from the same instance that will be used to process the partition.

hasNextBase

protected boolean hasNextBase(org.locationtech.jts.index.SpatialIndex spatialIndex,
                              Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes,
                              boolean buildLeft)

Iterator model for the index-based join. It checks if there is a next match and populate it to the result.

Parameters:

spatialIndex -

streamShapes -

buildLeft -

Returns:

hasNextBase

protected boolean hasNextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes,
                              Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes)

Iterator model for the nest loop join. It checks if there is a next match and populate it to the result.

Parameters:

buildShapes -

streamShapes -

Returns:

nextBase

protected org.apache.commons.lang3.tuple.Pair<U,T> nextBase(org.locationtech.jts.index.SpatialIndex spatialIndex,
                                                            Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes,
                                                            boolean buildLeft)

Iterator model for the index-based join. It returns 1 pair in the current batch. Each batch contains a list of pairs of geometries that satisfy the join condition. The current batch is the result of the current stream shape against all the build shapes.

Parameters:

spatialIndex -

streamShapes -

buildLeft -

Returns:

nextBase

protected org.apache.commons.lang3.tuple.Pair<U,T> nextBase(List<? extends org.locationtech.jts.geom.Geometry> buildShapes,
                                                            Iterator<? extends org.locationtech.jts.geom.Geometry> streamShapes)

Iterator model for the nest loop join. It returns 1 pair in the current batch. Each batch contains a list of pairs of geometries that satisfy the join condition. The current batch is the result of the current stream shape against all the build shapes.

Parameters:

buildShapes -

streamShapes -

Returns:

log

protected void log(String message,
                   Object... params)