Why not publish Parallel Processing parameters for the Custom Transformer?

Maybe someone is wondering why I didn't publish Parallel Processing parameters in the RandomPointGenerator implementation (see the previous post).

If you publish the "Parallel Process By" parameter (Right-click > Create User Parameter), you can make the transformer as a block-based one easily. i.e. the parameter works like "Group By".

Originally I had published the parameter while developing the transformer, but I noticed that this warning message appears on the Log window after the translation.
The message always appears if the workspace has the following conditions.
-----
Warning: not all FMESessions that were created were destroyed before shutdown. This may cause instability
-----
Conditions:
1) The custom transformer contains a StatisticsCalculator.
2) The Main workspace containing the custom transformer also contains a StatisticsCalculator.
3) Specify attribute(s) to the "Parallel Process By" parameter of the custom transformer.
4) Select "No Parallelism" (default) for the "Parallel Processing Level" parameter of the custom transformer.
-----
FME 2014 SP3 build 14391

I know that the warning may not cause any problem if the workspace is running with FME Desktop. But I'm afraid that it could cause a serious problem if the workspace is running with FME Server. And also I cannot confirm whether there are any other conditions which cause the same situation.

Therefore, I will not publish Parallel Processing parameters for a custom transformer unless the issue is resolved.

P.S. The same warning has occurred when using the AttributePivoter. These two transformers - StatisticsCalculator and AttributePivoter are implemented with Python. I guess there is an issue in the architecture of Python based transformers.

RandomPointGenerator

I created a custom transformer named RandomPointGenerator, which creates random points within entire input area features.
Find the transformer in Transformer Gallery (FME Store / Pragmatica).
FME 2014 SP3 build 14391

The implementation has been inspired by this thread in FME Community Answers.
Stratified random point in FME

Here, I illustrate outline of the procedure.

1. Input areas example.
Assume that these areas belong to a same group; these could be parts of an aggregated feature; there may be overlapped areas.

2. Resolve the overlapped area; calculate area for every area.

Deaggregator
AreaOnAreaOverlayer
AreaCalculator







3. Create bounding boxes for each individual area.

BoundingBoxReplacer









4. Dissolve the boxes, and create bounding boxes again; calculate area for every box.

Dissolver
BoundingBoxReplacer
AreaCalculator







5. Compute preferable number of points for each box  (based on the area calculation results and the required number of points), and then randomly create points within the boxes, so that the density will be approximately identical.

StatisticsCalculator
BoundsExtractor
AttributeCreator
ExpressionEvaluator
VertexCreator
Cloner
Offsetter



6. Filter the points by the original areas.

SpatialFilter









7. Randomly take points of specified number.

ExpressionEvaluator
Sorter
Sampler

Concatenate Attribute Values based on Attribute List Parameter

A custom transformer can receive space delimited attribute names through "Attribute List (space delimited)" type user parameter.

Typically, a parameter of this type would be used to link to "Group By" of transformer(s) in the custom transformer. In such a case, it's not necessary to retrieve values of the attributes.
But there could also be some cases where those values are necessary. e.g. use concatenated attribute values as Counter Name of a Counter.

Value of the parameter is a string. If specified attributes are "attr1", "attr2", and "attr3", for example, the value of parameter will be "attr1 attr2 attr3" (space delimited attribute names). And if an attribute name contains white spaces, it will be quoted by double quotations.

I think Tcl scripting is an easy way to concatenate specified attribute values.
First, use a ParameterFetcher to store the parameter value (i.e. specified attribute names) as an attribute. e.g. named "_attr_names".
And then, use a TclCaller with this script to create a concatenated string.
If no attributes were specified, this procedure returns an empty string.
-----
proc concatAttributeValues {} {
    set values {}
    foreach attr [FME_GetAttribute "_attr_names"] {
        lappend values [FME_GetAttribute $attr]
    }
    return [join $values {_}]
}
-----

It worked fine even if specified attribute names contain white spaces.

FME 2014 SP3 build 14391

Computational Error in DMSCalculator

The DMSCalculator is useful to convert decimal degree value to degrees, minutes and seconds. But there may be cases where the slight computational error causes a trouble.

=====
2014-10-03: I had discussed about the computational error in the DMSCaluculator (Version 0) with Safe's developer, then the new DMSCaluclator (Version 1) has become available in FME 2015 Beta build 15187. It's better on handling the computational error than the previous version, you will not need to consider the following workaround, in FME 2015.
=====
Although computational error cannot be avoided, there are ways to reduce it in some cases.
In the DMS calculation, I prefer to define math expressions myself, rather than using the DMSCalculator, so that the computational error can be reduced.
For example:

1) ExpressionEvaluator

2) AttributeCreator
=====
2014-09-24: Sorry, I didn't take care of negative source values - i.e. south/west hemisphere latitude/longitude. I'm living in north/east hemisphere of the earth.
Try these expressions for calculating minutes and seconds, if the source may be negative.
When the source is negative, a minus sign will be added to degrees value only.
-----
_minutes: @int(@abs(@Value(_seconds)))%3600/60
_seconds: @fmod(@abs(@Value(_seconds)),60)
=====

3) Result

(FME 2014 SP3 build 14391)

Transform 3D Triangular Polygons into TIN Surface

Assume that the source polygons are clean. i.e.
- There are no gaps and no overlaps between adjoining triangles.
- Common vertex of adjoining triangles has same elevation (z-coordinate).

There is a solution in this article.
How to clip TIN Surfaces

Summary:
1) FaceReplacer replaces the triangular polygons with faces.
2) Aggregator creates a multi-surface from the faces.
3) Triangulator transforms the multi-surface into a TIN surface.
If orientation of triangles could be Right Hand Rule, their orientation should be changed to Left Hand Rule before the FaceReplacer.

The purpose of the article is not to create new TIN surface, is to clip a given TIN surface. So, the solution is appropriate one. But it's not so efficient for newly creating a TIN surface based on millions triangular polygons.
If you allow that the original triangle shapes will not be preserved exactly as parts of the resulting TIN, this may be a much more efficient approach.

1) Chopper decomposes the triangles into vertex points.
2) CoordinateExtractor extracts x, y of each point.
3) AttributeRounder rounds x, y to avoid mismatching caused by slight computational error.
4) StringConcatenator concatenates x, y to create a string value.
5) DuplicateRemover removes duplicate points using the coordinate string as key.
6) TINGenerator creates a TIN surface based on the points.
In general, the Matcher can be used to remove duplicate points. But it's too inefficient (consumes long time and huge memory space) if there are millions points.

There could be more efficient solutions. If you know, please inform to me!

FME 2014 SP3 build 14391