DataFrame/SQL

Quick start

The detailed explanation is here: Visualize Spatial DataFrame/RDD.

1. Add GeoSpark-core, GeoSparkSQL, GeoSparkViz into your project POM.xml or build.sbt
2. Declare your Spark Session

   sparkSession = SparkSession.builder().
   config("spark.serializer",classOf[KryoSerializer].getName).
   config("spark.kryo.registrator", classOf[GeoSparkVizKryoRegistrator].getName).
   master("local[*]").appName("myGeoSparkVizDemo").getOrCreate()
   

3. Add the following lines after your SparkSession declaration:

   GeoSparkSQLRegistrator.registerAll(sparkSession)
   GeoSparkVizRegistrator.registerAll(sparkSession)
   

Regular functions

ST_Pixelize

Introduction: Return a pixel for a given resolution

Format: ST_Pixelize (A:geometry, ResolutionX:int, ResolutionY:int, Boundary:geometry)

Since: v1.2.0

Spark SQL example:

SELECT ST_Pixelize(shape, 256, 256, (ST_Envelope_Aggr(shape) FROM pointtable))
FROM polygondf

ST_TileName

Introduction: Return the map tile name for a given zoom level. Please refer to OpenStreetMap ZoomLevel and OpenStreetMap tile name.

Note

Tile name is formatted as a "Z-X-Y" string. Z is zoom level. X is tile coordinate on X axis. Y is tile coordinate on Y axis.

Format: ST_TileName (A:pixel, ZoomLevel:int)

Since: v1.2.0

Spark SQL example:

SELECT ST_TileName(pixels.px, 3)
FROM pixels

ST_Colorize

Introduction: Given the weight of a pixel, return the corresponding color. The weight can be the spatial aggregation of spatial objects or spatial observations such as temperature and humidity.

Note

The color is encoded to an Integer type value in DataFrame. When you print it, it will show some nonsense values. You can just treat them as colors in GeoSparkViz.

Format: ST_Colorize (weight:Double, maxWeight:Double, mandatory color: string (Optional))

Since: v1.2.0

Produce various colors - heat map

This function will normalize the weight according to the max weight among all pixels. Different pixel obtains different color.

Spark SQL example:

SELECT pixels.px, ST_Colorize(pixels.weight, 999) AS color
FROM pixels

Produce uniform colors - scatter plot

If a mandatory color name is put as the third input argument, this function will directly ouput this color, without considering the weights. In this case, every pixel will possess the same color.

Spark SQL example:

SELECT pixels.px, ST_Colorize(pixels.weight, 999, 'red') AS color
FROM pixels

Here are some example color names can be entered:

"firebrick"
"#aa38e0"
"0x40A8CC"
"rgba(112,36,228,0.9)"

Please refer to AWT Colors for a list of pre-defined colors.

ST_EncodeImage

Introduction: Return the base64 string representation of a Java PNG BufferedImage. This is specific for the server-client environment. For example, transfer the base64 string from GeoSparkViz to Apache Zeppelin.

Format: ST_EncodeImage (A:image)

Since: v1.2.0

Spark SQL example:

SELECT ST_EncodeImage(images.img)
FROM images

Aggregate functions

ST_Render

Introduction: Given a group of pixels and their colors, return a single Java PNG BufferedImage

Format: ST_Render (A:pixel, B:color)

Since: v1.2.0

Spark SQL example:

SELECT tilename, ST_Render(pixels.px, pixels.color) AS tileimg
FROM pixels
GROUP BY tilename

---

Last update: September 15, 2020 23:40:05