2013-08-24

An Example of Copying Files

For example, there are these files in my computer disk system:
C:\tmp\a\001.txt
C:\tmp\a\002.txt
C:\tmp\b\001.txt
C:\tmp\b\002.txt

and I want to copy them to another directory with renaming like this:
C:\tmp\folder\a_001.txt
C:\tmp\folder\a_002.txt
C:\tmp\folder\b_001.txt
C:\tmp\folder\b_002.txt

Directory and File Pathnames Reader and File Copy Writer are very useful to do such a operation.

Directory and File Pathnames Reader retrieves properties of every file in the specified directories, creates non-spatial features having attributes named "path_****". Each attribute stores a file property - file full path, directory path, file name, extension etc.
Source Directory and File Pathnames Directory: C:\tmp\a,C:\tmp\b

FilenamePartExtractor extracts parts of the file path.
Source File Name: path_windows
Full Filename Attribute: _filename
Directory Name Attribute: _dirname

AttributeCreator renames the attribute storing source file path, and creates the destination filename as a new attribute. "filecopy_****" is a format attribute of the File Copy Writer.
Attribute Name <- Value
filecopy_source_dataset <- path_windows
filecopy_dest_filename <- @Value(_dirname)_@Value(_filename)

File Copy Writer copies "filecopy_source_dataset" to "filecopy_dest_filename" in the specified directory.
Destination File Copy Directory: C:\tmp\folder

This is likely to be applicable in various scenes.

2013-08-23

Can ArcGIS Data Interoperability run existing FME workspaces?

I finally found a clear answer to this question.

> FME & Esri ArcGIS Data Interoperability FAQ
Can ArcGIS Data Interoperability run my existing FME workspaces?
No. ArcGIS Data Interoperability cannot run existing FME workspaces. If an FME user has workspaces, they will have to recreate the functionality of their existing workspaces within ArcGIS. This may or may not be successful, as the ArcGIS Data Interoperability extension doesn’t contain full FME functionality (see next question).
Note: It is possible to copy and paste transformers and data models between workspaces made with FME into custom tools made with Workbench inside of ArcGIS.

So, when I need to transplant an existing workspace to ArcGIS Data Interoperability, I have to re-create that in the ArcGIS Interoperability environment or copy & paste all the items of the workspace. However it may not be successfull because there are formats or transformers unsupported by ArcGIS Interoperability.

Associated links:
http://www.safe.com/solutions/for-applications/esri-arcgis/formats/
http://www.safe.com/solutions/for-applications/esri-arcgis/transformers/
http://www.safe.com/solutions/for-applications/esri-arcgis/data-interop-comparison/

2013-07-21

Multiple Feature Attribute Support of the AttributeCreator in FME 2013 SP2

The Multiple Feature Attribute Support functionality has been added to the AttributeCreator in FME 2013 SP2, it realizes accessing freely to attributes of backward / forward features in a sequence of features. Using this functionality, for example, the Fibonacci sequence can be created easily by a Counter and an AttributeCreator with the settings like:

Counter
Count Output Attribute: _fibonacci
Count Start: 0

AttributeCreator
<Mutliple Feature Attribute Support>
Number of Prior Features: 2
<Attribute To Set>
Attribute Name: _fibonacci
Value (Conditional):
  Test Condition: 1 < @Value(feature[0]._fibonacci)
  Output Value: @Evaluate(@Value(feature[-2]._fibonacci)+@Value(feature[-1]._fibonacci))

After this, using an AttributeAccumulator and a ListConcatenator, got a correct Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ...

The point is that we can access any attributes of prior / subsequent  features with the syntax of feature[n].<attribute name>n is an integer value which indicates the relative position of a feature based on the position of the current feature in the sequence. Negative n means a prior feature, positive n means a subsequent feature, and naturally 0 means the current feature. The range of available n value will be determined according to "Number of Prior Features" and "Number of Subsequent Features" parameters.

Does the Fibonacci sequence seem useless?
OK, see:
Great! Multiple Feature Attribute Support in AttributeCreator FME 2013 SP2

2013-08-26
In FME 2013 SP3, we don't need to use the syntax of feature[0].<attribute name> for the current feature attribute. Just <attribute name> indicates the current feature attribute in condition expressions. For example:
   Test Condition: 1 < @Value(_fibonacci)

Next: Use Case of Multiple Feature Attribute Support

2013-07-13

BulkAttributeRenamer + AttributeAccumulator creates single record from multiple records which are storing attributes of a feature dispersedly

Sometimes, we have a table that stores attributes of a feature in multiple rows dispersedly like this:
id, attrName, attrValue
001, aaa, 1
001, bbb, 2
002, aaa, 3
002, bbb, 4

How to transform this table to the following structure?
id, aaa, bbb
001, 1, 2
002, 3, 4

One possible way is to accumulate attributes grouped by id, and then to restructure the table using a Python script:
-----
import fmeobjects
def restructureTable(feature):
    names = feature.getAttribute('_list{}.attrName')
    values = feature.getAttribute('_list{}.attrValue')
    for name, value in zip(names, values):
        feature.setAttribute(name, value)
-----

It works well but is a little inefficient, because it needs a list attribute which will not be used afterward. But I didn't know better ways.
Now,  I've learned a solution using the BulkAttributeRenamer with the following parameter settings:
Selected Attributes: attrValue
Action: Regular Expression Replace
Text to Find: .*
String: attrName

After renaming, I can create a table having the preferable schema by the AttributeAccumulator without creating list attributes.

Community > RecordBuilder / CSV import
Community > Attribute renaming using attribute values

2013-07-12

How to specify multiple xfMaps to XML reader

To specify multiple xfMaps to the XML reader (FME 2013), the paths should be delimited with semicolons like:
"C:\tmp\xfmaps\sample01_2.xml ; C:\tmp\xfmaps\sample01_1.xml"
Nobody noticed this until now?
Community > How to specify multiple xfMap files to an XML reader

=====
2014-09-29: As I mentioned in the post linked above, this issue has been resolved in FME 2014. In FME 2014, you don't need delimiters when specifying multiple xfMap files to an XML reader.
=====