2013-12-14

Convert Simple List to Complex List

(FME 2013 SP4 Build 13547)

List attributes often appear in FME workspace, and also take important role to achieve the project purpose in many cases.

"simple list" (or just called "list") and "complex list" are types of list attribute.
I don't know whether those type names are official terminologies, but "complex list" is used in this documentation. David pointed it out before, thanks.
> FMEpedia: List Attributes

Simple List Example:
_list{0}, _list{1}, _list{2}, ...

Complex List Example:
_list{0}.foo, _list{1}.foo, _list{2}.foo, ...
_list{0}.bar, _list{1}.bar, _list{2}.bar, ...

Note: There is one more list type - "nested list", but I don't touch it in this article.
=====
2013-12-18: "complex list" is also called "structured list" in other documentation.
"The function also accepts a "structured list" specification, such as "attrInfo{}.name", ..." 
-- description about FMEFeature.getAttribute function, FME Objects Python API, [FME_HOME]/fmeobjects/python/apidoc/index.html (FME 2014 Beta build 14218)
=====

Well, there are two feature types named "foo" and "bar". Both of them have a simple list attribute named "_list{}", and also have a merging key attribute named "_key".
Assume that both "foo" and "bar" don't have list attributes other than "_list{}".

Consider merging those features.  Merged features should have a complex list attribute which retains every element of the original list attributes from "foo" and "bar" features.
i.e.
_list{i} of "foo" should be converted to _list{i}.foo
_list{i} of "bar" should be converted to _list{i}.bar

I found the BulkAttributeRenamer can be used to do that (FME 2013 Build 13547).
This workflow does it. The functionality is similar to "zip" function of Python.















The "zip trick" with the BulkAttributeRenamer and the FeatureMerger flashed on this thread.
Although the SchemaMapper would be suitable in this case, I think there should be some cases that "zip trick" can be effective.

Moreover, I expect that the ListRenamer transformer will be upgraded to have new options for conversion between simple list and complex list.

=====
2013-12-17: A complex list (e.g. _list{}.foo) can be converted to a simple list (e.g. _list{}) using a BulkAttributeRenamer with this setting as well, for what it's worth.
-----
Rename: All Attributes
Action: Regular Expression Replace
Text to Find: }.foo$
String: }
-----
=====
2014-01-27: The BulkAttributeRenamer of FME 2014 (build 14234) also works for converting type of list with the way mentioned above. However, I noticed that the list name shown on the Canvas will not change, even though internal list name has been changed.
As a meantime workaround, the AttributeExposer can be used to expose the correct list name after renaming. But the original simple list name ("_list{}") cannot be hidden (:-(









=====
2014-01-28: I noticed the PythonCaller can be used to expose and hide list names. Just specify "Attributes to Expose" and "Lists to Hide" parameters. The script doesn't need to do any thing.
-----
def processFeature(feature):
    # do nothing
    pass
-----
Although it's a weird usage, can be effective.
Of course using the AttributeExposer or the PythonCaller is a temporary workaround. I expect the BulkAttributeRenamer will be fixed in the near future.

No comments:

Post a Comment