FME 2014 starts supporting null as attribute values.
In my understanding, null is a special value, should be distinguished from both an empty string and a missing (not existing) attribute.
We can set null to an attribute using the AttributeCreator.
And the Tester can be used to test whether an attribute value is null. This test condition passes the input feature if "attrNull" holds null.
The Tester provides "Attribute Is Null", "Attribute Is Empty String" and "Attribute Is Missing" operators, and those can strictly determine whether the Left Value is null, empty string or missing.
Well, "attrEmpty" holds an empty string now. If I set the test condition like this, the feature goes to which port, Passed or Failed?
I thought the feature would go to the Failed port, but the result was not so. The feature goes to the Passed port. I got a little confused. How should I think of this result?
Null is a special value, but it could be treated as an empty string in some cases?
Have to continue to explore about the null.
=====
2013-12-20: I noticed that the feature goes to the Passed port even if "attrEmpty" is missing. null seems to be also treated as missing attribute in some cases. hmm...
=====
2013-12-21: Tai clarified the design concept on comparing null, empty string and missing attribute. See his comment.
The mist in my brain has cleared up. Thanks, Tai.
Thank you for taking the time to post this article! We hope the response below will be of some assistance.
ReplyDeleteThe Tester supports two test clause modes: String, and Numeric. Automatic mode invokes Numeric mode where possible, and then falls back on String mode. Numeric mode, similarly, falls back on String mode when a numeric comparison is not possible. So now, imagine that we have an integer attribute with value 0, call it i, and a string attribute with value "0", call it s. If we tried i = s under Test Clauses, what would the Tester give us? No matter which mode we chose, Tester would consider i = s to be true, and the feature comes out of the PASSED port.
Some users would expect that i = s should fail the test condition because i and s have different value types. However at this time Tester does not have a mode to differentiate value types. Supporting value types as part of a new mode is no simple task as user expectations vary on whether a single float should match a double float, on whether an integer should match a short, on whether a string with encoding X should match a string with encoding Y, and so on. Further, if a user typed in a numeric literal 0 in a test clause, it is not clear what value type to assign to that literal.
Now let us consider null, missing, and empty string attribute values, in the context of the modes that Tester currently supports: String and Numeric. What are their string representations? When they are forced to be represented by strings, the only reasonable representation for these three types of nothing is the empty string. Next, what are their numeric representations? None of them have numeric representations, so a numeric comparison is not possible. Thus, the Tester falls back on a String comparison, and as we've just seen, these three types of nothing have the same string representation.
Therefore, the Tester behavior posted in this blog article is as designed. Thank you for your time and interest in exploring the wonderful world of null, empty, and missing values! We trust you will find that FME 2014 will give you more tools than ever before to handle the various types of nothing in your data flows!
Tai Meng, Safe Developer
Thanks Tai for the clarification. We are considering doing a whole webinar on "Null" (there are apparently more then 15 kinds of null even) sometime in the new year. A niche topic, but one rich in terms of the intricacies and potential references to Seinfeld and Shakespeare.
ReplyDeleteThank you for the clarification, Tai. I've understood the design concept on testing null value.
ReplyDeleteI think many questions about null would be posted in the Community. I believe the Webinar will be very helpful for all users.
I'll continue to explore the null world!