org.apache.sedona.core.spatialRDD

Class SpatialRDD<T extends org.locationtech.jts.geom.Geometry>

java.lang.Object

org.apache.sedona.core.spatialRDD.SpatialRDD<T>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

CircleRDD, LineStringRDD, PointRDD, PolygonRDD, RectangleRDD

public class SpatialRDD<T extends org.locationtech.jts.geom.Geometry>
extends Object
implements Serializable

The Class SpatialRDD.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
long	approximateTotalCount The total number of records.
org.locationtech.jts.geom.Envelope	boundaryEnvelope The boundary envelope.
protected boolean	CRStransformation The CR stransformation.
List<String>	fieldNames
org.apache.spark.api.java.JavaRDD<org.locationtech.jts.index.SpatialIndex>	indexedRawRDD The indexed raw RDD.
org.apache.spark.api.java.JavaRDD<org.locationtech.jts.index.SpatialIndex>	indexedRDD The indexed RDD.
org.apache.spark.api.java.JavaRDD<T>	rawSpatialRDD The raw spatial RDD.
org.apache.spark.sql.types.StructType	schema
protected String	sourceEpsgCode The source epsg code.
org.apache.spark.api.java.JavaRDD<T>	spatialPartitionedRDD The spatial partitioned RDD.
protected String	targetEpgsgCode The target epgsg code.

Constructor Summary

Constructors

Constructor and Description
SpatialRDD()

Method Summary

Modifier and Type	Method and Description
boolean	analyze() Analyze.
boolean	analyze(org.locationtech.jts.geom.Envelope datasetBoundary, Integer approximateTotalCount)
boolean	analyze(org.apache.spark.storage.StorageLevel newLevel) Analyze.
org.locationtech.jts.geom.Envelope	boundary() Deprecated. Call analyze() instead
void	buildIndex(IndexType indexType, boolean buildIndexOnSpatialPartitionedRDD) Builds the index.
void	calc_partitioner(GridType gridType, int numPartitions) Spatial partitioning.
STRtree	coalesceAndBuildRawIndex(IndexType indexType) Builds the index on coalesced raw spatial RDD.
long	countWithoutDuplicates() Count without duplicates.
long	countWithoutDuplicatesSPRDD() Count without duplicates SPRDD.
boolean	CRSTransform(String sourceEpsgCRSCode, String targetEpsgCRSCode) CRS transform.
boolean	CRSTransform(String sourceEpsgCRSCode, String targetEpsgCRSCode, boolean lenient) CRS transform.
void	flipCoordinates()
boolean	getCRStransformation() Gets the CR stransformation.
SpatialPartitioner	getPartitioner()
org.apache.spark.api.java.JavaRDD<T>	getRawSpatialRDD() Gets the raw spatial RDD.
int	getSampleNumber()
String	getSourceEpsgCode() Gets the source epsg code.
String	getTargetEpgsgCode() Gets the target epgsg code.
RectangleRDD	MinimumBoundingRectangle() Deprecated.
void	saveAsGeoJSON(String outputLocation) Save as geo JSON.
void	saveAsWKB(String outputLocation) Save as WKB.
void	saveAsWKT(String outputLocation) Save as WKT
void	setNeighborSampleNumber(int neighborSampleNumber) Sets the neighbor sample number.
void	setRawSpatialRDD(org.apache.spark.api.java.JavaRDD<T> rawSpatialRDD) Sets the raw spatial RDD.
void	setSampleNumber(int sampleNumber) Sets the sample number.
boolean	spatialPartitioning(GridType gridType)
void	spatialPartitioning(GridType gridType, int numPartitions)
boolean	spatialPartitioning(List<org.locationtech.jts.geom.Envelope> otherGrids) Deprecated. Use spatialPartitioning(SpatialPartitioner partitioner)
void	spatialPartitioning(SpatialPartitioner partitioner)
boolean	spatialPartitioning(StandardQuadTree partitionTree) Deprecated. Use spatialPartitioning(SpatialPartitioner partitioner)
boolean	spatialPartitioningWithoutDuplicates(GridType gridType)
void	spatialPartitioningWithoutDuplicates(GridType gridType, int numPartitions) Calculate non-duplicate inducing partitioning
boolean	spatialPartitioningWithoutDuplicates(List<org.locationtech.jts.geom.Envelope> otherGrids) Calculate non-duplicate inducing partitioning based on a list of existing envelopes
void	spatialPartitioningWithoutDuplicates(SpatialPartitioner partitioner) Calculate non-duplicate inducing partitioning from an existing SpatialPartitioner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

approximateTotalCount

public long approximateTotalCount

The total number of records.

boundaryEnvelope

public org.locationtech.jts.geom.Envelope boundaryEnvelope

The boundary envelope.

spatialPartitionedRDD

public org.apache.spark.api.java.JavaRDD<T extends org.locationtech.jts.geom.Geometry> spatialPartitionedRDD

The spatial partitioned RDD.

indexedRDD

public org.apache.spark.api.java.JavaRDD<org.locationtech.jts.index.SpatialIndex> indexedRDD

The indexed RDD.

indexedRawRDD

public org.apache.spark.api.java.JavaRDD<org.locationtech.jts.index.SpatialIndex> indexedRawRDD

The indexed raw RDD.

rawSpatialRDD

public org.apache.spark.api.java.JavaRDD<T extends org.locationtech.jts.geom.Geometry> rawSpatialRDD

The raw spatial RDD.

fieldNames

public List<String> fieldNames

schema

public org.apache.spark.sql.types.StructType schema

CRStransformation

protected boolean CRStransformation

The CR stransformation.

sourceEpsgCode

protected String sourceEpsgCode

The source epsg code.

targetEpgsgCode

protected String targetEpgsgCode

The target epgsg code.

Constructor Detail

SpatialRDD

public SpatialRDD()

Method Detail

getSampleNumber

public int getSampleNumber()

setSampleNumber

public void setSampleNumber(int sampleNumber)

Sets the sample number.

Parameters:

sampleNumber - the new sample number

setNeighborSampleNumber

public void setNeighborSampleNumber(int neighborSampleNumber)

Sets the neighbor sample number.

Parameters:

neighborSampleNumber - the new neighbor sample number

CRSTransform

public boolean CRSTransform(String sourceEpsgCRSCode,
                            String targetEpsgCRSCode,
                            boolean lenient)

CRS transform.

Parameters:

sourceEpsgCRSCode - the source epsg CRS code

targetEpsgCRSCode - the target epsg CRS code

lenient - consider the difference of the geodetic datum between the two coordinate systems, if true, never throw an exception "Bursa-Wolf Parameters Required", but not recommended for careful analysis work

Returns:

true, if successful

CRSTransform

public boolean CRSTransform(String sourceEpsgCRSCode,
                            String targetEpsgCRSCode)

CRS transform.

Parameters:

sourceEpsgCRSCode - the source epsg CRS code

targetEpsgCRSCode - the target epsg CRS code

Returns:

true, if successful

spatialPartitioning

public boolean spatialPartitioning(GridType gridType)
                            throws Exception

Throws:

Exception

spatialPartitioningWithoutDuplicates

public boolean spatialPartitioningWithoutDuplicates(GridType gridType)
                                             throws Exception

Throws:

Exception

spatialPartitioningWithoutDuplicates

public void spatialPartitioningWithoutDuplicates(GridType gridType,
                                                 int numPartitions)
                                          throws Exception

Calculate non-duplicate inducing partitioning

Note that non-duplicating partitioners are intended for use by distributed partitioned writers and not able to be used for spatial joins.

Parameters:

gridType - The target GridType

numPartitions - The target number of partitions

Throws:

Exception

spatialPartitioningWithoutDuplicates

public void spatialPartitioningWithoutDuplicates(SpatialPartitioner partitioner)

Calculate non-duplicate inducing partitioning from an existing SpatialPartitioner

Note that non-duplicating partitioners are intended for use by distributed partitioned writers and not able to be used for spatial joins.

Parameters:

partitioner - An existing partitioner obtained from the partitioning of another SpatialRDD.

Throws:

Exception

spatialPartitioningWithoutDuplicates

public boolean spatialPartitioningWithoutDuplicates(List<org.locationtech.jts.geom.Envelope> otherGrids)
                                             throws Exception

Calculate non-duplicate inducing partitioning based on a list of existing envelopes

This is shorthand for spatialPartitioningWithoutDuplicates(new IndexedGridPartitioner()). Using spatialPartitioningWithoutDuplicates(gridType, numPartitions) is typically more appropriate because it is able to adapt to the content of the partition and is able to produce more consistently balanced partitions.

Note that non-duplicating partitioners are intended for use by distributed partitioned writers and not able to be used for spatial joins.

Parameters:

otherGrids - A list of existing envelopes

Returns:

true on success

Throws:

Exception

calc_partitioner

public void calc_partitioner(GridType gridType,
                             int numPartitions)
                      throws Exception

Spatial partitioning.

Parameters:

gridType - the grid type

numPartitions - the target number of partitions

Throws:

Exception - the exception

spatialPartitioning

public void spatialPartitioning(GridType gridType,
                                int numPartitions)
                         throws Exception

Throws:

Exception

getPartitioner

public SpatialPartitioner getPartitioner()

spatialPartitioning

public void spatialPartitioning(SpatialPartitioner partitioner)

spatialPartitioning

public boolean spatialPartitioning(List<org.locationtech.jts.geom.Envelope> otherGrids)
                            throws Exception

Deprecated. Use spatialPartitioning(SpatialPartitioner partitioner)

Throws:

Exception

spatialPartitioning

public boolean spatialPartitioning(StandardQuadTree partitionTree)
                            throws Exception

Deprecated. Use spatialPartitioning(SpatialPartitioner partitioner)

Throws:

Exception

countWithoutDuplicates

public long countWithoutDuplicates()

Count without duplicates.

Returns:

the long

countWithoutDuplicatesSPRDD

public long countWithoutDuplicatesSPRDD()

Count without duplicates SPRDD.

Returns:

the long

buildIndex

public void buildIndex(IndexType indexType,
                       boolean buildIndexOnSpatialPartitionedRDD)
                throws Exception

Builds the index.

Parameters:

indexType - the index type

buildIndexOnSpatialPartitionedRDD - the build index on spatial partitioned RDD

Throws:

Exception - the exception

coalesceAndBuildRawIndex

public STRtree coalesceAndBuildRawIndex(IndexType indexType)

Builds the index on coalesced raw spatial RDD.

Parameters:

indexType - the index type

Throws:

Exception - the exception

boundary

public org.locationtech.jts.geom.Envelope boundary()

Deprecated. Call analyze() instead

Boundary.

Returns:

the envelope

getRawSpatialRDD

public org.apache.spark.api.java.JavaRDD<T> getRawSpatialRDD()

Gets the raw spatial RDD.

Returns:

the raw spatial RDD

setRawSpatialRDD

public void setRawSpatialRDD(org.apache.spark.api.java.JavaRDD<T> rawSpatialRDD)

Sets the raw spatial RDD.

Parameters:

rawSpatialRDD - the new raw spatial RDD

analyze

public boolean analyze(org.apache.spark.storage.StorageLevel newLevel)

Analyze.

Parameters:

newLevel - the new level

Returns:

true, if successful

analyze

public boolean analyze()

Analyze.

Returns:

true, if successful

analyze

public boolean analyze(org.locationtech.jts.geom.Envelope datasetBoundary,
                       Integer approximateTotalCount)

saveAsWKB

public void saveAsWKB(String outputLocation)

Save as WKB.

Parameters:

outputLocation - the output location

saveAsWKT

public void saveAsWKT(String outputLocation)

Save as WKT

saveAsGeoJSON

public void saveAsGeoJSON(String outputLocation)

Save as geo JSON.

Parameters:

outputLocation - the output location

MinimumBoundingRectangle

@Deprecated
public RectangleRDD MinimumBoundingRectangle()

Deprecated.

Minimum bounding rectangle.

Returns:

the rectangle RDD

getCRStransformation

public boolean getCRStransformation()

Gets the CR stransformation.

Returns:

the CR stransformation

getSourceEpsgCode

public String getSourceEpsgCode()

Gets the source epsg code.

Returns:

the source epsg code

getTargetEpgsgCode

public String getTargetEpgsgCode()

Gets the target epgsg code.

Returns:

the target epgsg code

flipCoordinates

public void flipCoordinates()