2014-01-10

Create Raster from Serialized Cell Values

(FME 2014 Beta build 14227)

I have a lot of DEM datasets, each of them contains grid cell values (altitudes) in a standard mesh, i.e. rectangular area.
Mesh size, number of columns and number of rows per a mesh are fixed as followings.
- mesh width = 45 seconds
- mesh height = 30 seconds
- number of rows = 150
- number of columns = 225
Therefore the number of cells is 33750 (= 150 x 225) per a mesh and cell size is 0.2x0.2 seconds.
Cell values will be stored in a list attribute named "_alti{}" - an array containing sequenced altitude values from south-west cell (0, 0) to north-east cell (149, 224). And also each mesh feature has the coordinate of its south-west corner as attributes named "_xmin" and "_ymin".

Consider creating a raster geometry for each mesh in such condition.
My strategy is:
1) Initially create a raster whose origin is (0, 0) and cell size is (1 x 1).
2) Scale the raster by appropriate ratio.
3) Move the raster so that the origin is located at the correct coordinate.

At first, extract cell values with a ListExploder, then create cell center points in 3D with a VertexCreator (2D/3DPointReplacer/Adder have been integrated into it). Since the cell size of initial raster is (1 x 1), the expressions for calculating (x, y) of each point are simple.
And then, create a raster with a NumericRasterizer. The parameter settings are also simple.
After creating raster, recover the original attributes using a FeatureMerger.

In the next Bookmark, scale the raster by appropriate ratio using a Scaler and move it to correct location using an Offsetter. Done!

Of course the raster can be created by the final cell size and location from the beginning. But in my trial, the way above was a little more efficient in running speed, even though having additional transformers for scaling and offsetting.
Above all, I think it's nice that calculation and parameter settings are very simple.

P.S. I feel the "VertexCreator and NumericRasterizer" method is not so efficient, but it seems to be the only way to create a raster geometry based on serialized grid cell values using existing transformers only. I looked for a way to create a raster more efficiently by Python scripting, but couldn't find functions to do that in FME Objects Python API unfortunately.
Note (2014-01-14): Inserting a PointCloudCombiner transformer between the VertexCreator and the NumericRasterizer could be effective to improve efficiency as mentioned below.

=====
2014-01-11: There are examples in which "Scaler and Offsetter" method can be used effectively.
> Community: Shrink / fit an object to another
> Community: Scale by a point
> Advanced Geometric Operation: Create Normal Lines of a Line Segment
=====
2014-01-14: Considerations on improving efficiency
1) Create 3D Points using Python script
As mentioned above FME Objects Python API doesn't provide functions to create a raster. But 3D points can be created easily with Python, and also  it could be more efficient. For example, the ListExploder and the VertexCreator can be replaced with a PythonCaller performing this script.
-----
# Example 1: Create 3D Points
import fmeobjects
class GridCellPointCreator(object):
def __init__(self):
pass

def input(self, feature):
id = feature.getAttribute('_feature_id')
for i, a in enumerate(feature.getAttribute('_alti{}')):
x, y, z = i % 225 + 0.5, i / 225 + 0.5, float(a)
point = fmeobjects.FMEFeature()
point.setGeometry(fmeobjects.FMEPoint(x, y, z))
point.setAttribute('_feature_id', id)
self.pyoutput(point)

def close(self):
pass
-----
Depending on the condition, creating a 3D multi point geometry could be more efficient.
-----
# Example 2: Create 3D Multi Point
# Unnecessary attributes should be removed before rasterizing.
import fmeobjects
def createGridCellPoints(feature):
points = fmeobjects.FMEMultiPoint()
for i, a in enumerate(feature.getAttribute('_alti{}')):
x, y, z = i % 225 + 0.5, i / 225 + 0.5, float(a)
points.appendPart(fmeobjects.FMEPoint(x, y, z))
feature.setGeometry(points)
-----
2) Transform Vector Geometry to a Point Cloud before rasterizing
The NumericRasterizer can also create a raster based on a point cloud. In my trial, transforming 3D points (or 3D multi point) to a point cloud using a PointCloudCombiner transformer before creating a raster was indeed more efficient. Furthermore, since the PointCloudCombiner can transform any geometry other than points to a point cloud, creating a 3D line geometry with Python could be also an option.
Dale and Chris suggested to me the capability of the point cloud technology. Thanks!
=====
2014-01-15
-----
# Example 3: Create 3D Line
import fmeobjects
def create3DLine(feature):
coords = []
for i, a in enumerate(feature.getAttribute('_alti{}')):
coords.append((i % 225 + 0.5, i / 225 + 0.5, float(a)))
feature.setGeometry(fmeobjects.FMELine(coords))
-----