2014-09-21

Why not publish Parallel Processing parameters for the Custom Transformer?

Maybe someone is wondering why I didn't publish Parallel Processing parameters in the RandomPointGenerator implementation (see the previous post).













If you publish the "Parallel Process By" parameter (Right-click > Create User Parameter), you can make the transformer as a block-based one easily. i.e. the parameter works like "Group By".

Originally I had published the parameter while developing the transformer, but I noticed that this warning message appears on the Log window after the translation.
The message always appears if the workspace has the following conditions.
-----
Warning: not all FMESessions that were created were destroyed before shutdown. This may cause instability
-----
Conditions:
1) The custom transformer contains a StatisticsCalculator.
2) The Main workspace containing the custom transformer also contains a StatisticsCalculator.
3) Specify attribute(s) to the "Parallel Process By" parameter of the custom transformer.
4) Select "No Parallelism" (default) for the "Parallel Processing Level" parameter of the custom transformer.
-----
FME 2014 SP3 build 14391

I know that the warning may not cause any problem if the workspace is running with FME Desktop. But I'm afraid that it could cause a serious problem if the workspace is running with FME Server. And also I cannot confirm whether there are any other conditions which cause the same situation.

Therefore, I will not publish Parallel Processing parameters for a custom transformer unless the issue is resolved.

P.S. The same warning has occurred when using the AttributePivoter. These two transformers - StatisticsCalculator and AttributePivoter are implemented with Python. I guess there is an issue in the architecture of Python based transformers.

No comments:

Post a Comment