2013-11-03

Advanced Geometric Operation: Create Normal Lines of a Line Segment

Create normal (perpendicular) lines against a line segment. Make the start node of the resultant lines be equal to the start node of the input line segment.
This image illustrates the expected result.











Workflow Example (FME 2013 SP4 Build 13547)
One possible approach is: offset the line segment so that its start node is located at (0, 0); rotate the line about (0, 0) by 90 (Left) / -90 (Right) degree; scale the lines into specified lengths; offset the scaled lines so that those start nodes are equal to the start node of the input line segment.
If the Scaler origin could be set to any location, the Offsetters would not be necessary. But options of the current Scaler origin are "0, 0" and "Center Point" only, so the workflow became cluttered a little.
A custom transformer which performs scaling based on specified origin coordinate can be created, and it might be useful to not only this workflow but also many scenarios.













Addition 2013-11-04: There is another approach using characteristics of unit vector and normal vector. This workflow looks more elegant and also efficient to me.











Application Examples:
These images were created with the workflow and several existing transformers.






The PythoCaller with this script can do the same operation as the workflow.
-----
# Example for PythonCaller (1): Create Normal Lines of a Line Segment
# FME 2013 SP4 Build 13547
# Assume input feature is a line segment; omit error handling.
import fmeobjects, math

class NormalLineCreator(object):
    def __init__(self):
        pass
        
    def input(self, feature):
        # Check geometry type and number of coordinates.
        if feature.getGeometryType() != fmeobjects.FME_GEOM_LINE \
           or feature.numCoords() != 2:            
            return # *** error ***
            
        # Extract coordinate of start node, offset feature,
        # and get length of the line segment.
        coord = feature.getCoordinate(0)
        feature.offset(-coord[0], -coord[1])
        lenS = feature.getGeometry().getLength(False)

        # Lengths of resultant normal lines.
        lenL = float(feature.getAttribute('_length_left'))
        lenR = float(feature.getAttribute('_length_right'))
            
        # Create resultant normal lines and output them.
        data = [('left', lenL / lenS, 90.0), ('right', lenR / lenS, -90.0)]
        for direction, scale, angle in data:
            line = feature.clone()
            line.rotate2D((0.0, 0.0), angle)
            line.scale(scale, scale)
            line.offset(coord[0], coord[1])
            line.setAttribute('_normal_direction', direction)
            self.pyoutput(line)
            
    def close(self):
        pass
-----
# 2013-11-04
# Example for PythonCaller (2): Create Normal Lines of a Line Segment
# FME 2013 SP4 Build 13547
# Assume input feature is a line segment; omit error handling.
import fmeobjects, math

class NormalLineCreator(object):
    def __init__(self):
        pass
     
    def input(self, feature):
        # Check geometry type and number of coordinates.
        if feature.getGeometryType() != fmeobjects.FME_GEOM_LINE \
           or feature.numCoords() != 2:          
            return # *** error ***
     
        # Calculate unit vector of the line segment.
        seg = feature.removeGeometry()
        start, end = seg.getPointAt(0), seg.getPointAt(1)
        (x0, y0, z0), (x1, y1, z1) = start.getXYZ(), end.getXYZ()
        lenS = seg.getLength(False)
        ux, uy = (x1 - x0) / lenS, (y1 - y0) / lenS
     
        # Transform the feature to the start node point.
        feature.setGeometry(start)
     
        # Lengths of resultant normal lines.
        lenL = float(feature.getAttribute('_length_left'))
        lenR = float(feature.getAttribute('_length_right'))
     
        # Create resultant normal lines and output them.
        data = [('left', lenL, -uy, ux), ('right', lenR, uy, -ux)]
        for direction, len, nx, ny in data:
            line = feature.clone()
            line.addCoordinate(x0 + nx * len, y0 + ny * len, z0)
            line.setAttribute('_normal_direction', direction)
            self.pyoutput(line)
         
    def close(self):
        pass
-----

2013-11-01

FME 2013 SP4 has been upgraded

Current newest Build Number is 13547, it's now downloadable.
The bug on max / min function (FME Math Functions) has been fixed.
Community > ExpressionEvaluator Answer Changed

2013-10-31

Getting Started with Tcl in FME: List Attribute Manipulation in TclCaller

Previous: Startup / Shutdown Script

A list attribute contains multiple values under the same name, each element of a list is distinguished from others by its index. The index is usually 0-based sequential number quoted by braces (curly brackets); it's a part of the attribute name indicating an element.
Names of individual list elements are typically represented like these:
_list{0}, _list{1}, _list{2} ...
_list{0}.sub, _list{1}.sub, _list{2}.sub ...

Every element can be treated as well as a non-list attribute using its name (including index) in the Tcl script. And also, since the name contains an index, repetitive processing commands (for, foreach etc.) would be effective in many cases.
Here I'll give some Tcl procedure examples for the TclCaller.

Convert Multiple Attributes to a List Attribute
This procedure collects 3 non-list attribute values and create a list attribute named _code which contains original 3 attribute values as its elements. This functionality is similar to the ListPopulator transformer.
-----
proc multiAttributesToList {} {
  set i 0
  foreach name {"CODE_A" "CODE_B" "CODE_C"} {
    FME_SetAttribute "_code{$i}" [FME_GetAttribute $name]
    incr i
  }
}
-----

Concatenate List Elements
This procedure concatenates every element in a list attribute named _list, and returns the result as a comma separated string. Similar to the ListConcatenator transformer.
-----
proc concatList {} {
  set elements {}
  for {set i 0} {[FME_AttributeExists "_list{$i}"]} {incr i} {
    lappend elements [FME_GetAttribute "_list{$i}"]
  }
  return [join $elements {,}]
}
-----

Calculate Basic Statistics of List Elements
This procedure calculates sum, minimum, maximum and average of elements of a list attribute named _src, sets the results as new non-list attributes, and returns number of the elements. Assume all elements of the list are numeric representation.
-----
proc calcBasicStatistics {} {
  for {set count 0} {[FME_AttributeExists "_src{$count}"]} {incr count} {
    set val [FME_GetAttribute "_src{$count}"]
    if {$count == 0} {
      set sum [set min [set max $val]]
    } else {
      set sum [expr $sum + $val]
      if {$val < $min} {set min $val}
      if {$max < $val} {set max $val}
    }
  }
  if {0 < $count} {
    FME_SetAttribute "_sum" $sum
    FME_SetAttribute "_min" $min
    FME_SetAttribute "_max" $max
    FME_SetAttribute "_ave" [expr double($sum) / $count]
  }
  return $count
}
-----

"Getting Started with Tcl in FME" series ends here. My Tcl skill made progress a little???

"Getting Started with Tcl in FME" Titles
TclCaller
AttributeCreator
Other Transformers
Scripted Parameter
Startup / Shutdown Script
List Attribute Manipulation in TclCaller *this post

2013-10-30

Advanced Geometric Operation: Divide a Line at Equal Interval

There are many transformers for geometric operations in FME Workbench; one of them or combination of some transformers can do most of all geometric operations we need usually. But sometimes, I encounter a case which needs to create a custom transformer or Python scripting.

I think that dividing a line at equal interval is one of such cases, and also that is relatively frequent required operation.
The Snipper transformer can be used to divide a line. However, it divides a line into only two parts (snipped and remnant) per one processing. If I set the length of the snipped part as the specified interval, the dividing operation has to be performed repeatedly while the length of the remnant part is greater than specified interval.

So, I think a custom transformer having a loop is an effective solution to do that.
This screenshot is a simplified example without any validation against input features.
Note: Actually the LengthCalculator and the Tester may not be essential. But when lacking them, the Workbench will display warning messages.
=====
2013-11-25
It is a kind of "reinventing the wheel". FME Store provides the IterativeSnipper_2013 custom transformer. Look at this as an exercise for the loop functionality of a custom transformer.
=====










Similar operation can be also defined as Python script for the PythonCaller.
-----
# Example: Divide Line at Equal Interval
# 2013-11-27 Modified
import fmeobjects

class LineDivider(object):
    def __init__(self):
        self.geomCloner = fmeobjects.FMEFeature()
   
    def input(self, feature):
        try:
            interval = float(feature.getAttribute('_interval'))
            if interval < 1.0e-6:
                raise fmeobjects.FMEException()
           
            dim = int(feature.getAttribute('_length_dimension'))
            measure3D = (True if dim == 3 else False)
       
            remnant = feature.getGeometry()
            if isinstance(remnant, fmeobjects.FMESimpleArea):
                remnant = remnant.getBoundaryAsCurve()
       
            curveList = []
            while interval < remnant.getLength(measure3D):
                self.geomCloner.setGeometry(remnant)
                curve = self.geomCloner.getGeometry()
                curve.snip(fmeobjects.SNIP_DISTANCE, measure3D, 0, interval)
                curveList.append(curve)
                remnant.snip(fmeobjects.SNIP_DISTANCE, measure3D, interval, -1)
            if 0.0 < remnant.getLength(measure3D):
                curveList.append(remnant)
       
            for i, geom in enumerate(curveList):
                feat = feature.cloneAttributes()
                feat.setGeometry(geom)
                feat.setAttribute('_index', i)
                feat.setAttribute('_result', 'success')
                feat.setCoordSys(feature.getCoordSys())
                self.pyoutput(feat)
        except:
            feature.setAttribute('_result', 'fail')
            self.pyoutput(feature)
       
    def close(self):
        pass
-----

Next: Effective Cloner

2013-10-27

Specify Log File Name

By default, the log file name will be same as the workspace name, except its extension. If we need to change the log file name, can specify a preferable name to a Workspace Parameter named "Log File". For example:
$(FME_MF_DIR)MyLogFile.log

FME_MF_DIR is a macro (system parameter) name, indicates the current directory path. "./MyLogFile.log" can be also specified here.

But the log file name specified with such a way is fixed, so the file will be overwritten when the workspace is executed multiple times repeatedly. When we need to preserve every log file with unique name for each run, how can we do that?

1. Rename in the Shutdown Script
One possible approach is to rename the log file name in the Shutdown Script.
Note:This approach cannot be applied in a certain situation as mentioned later.
The log file has been closed before shutdown process; we can access the saved log file path via a global variable named "FME_LogFileName" in the script.
This Shutdown Python Script appends a timestamp (and sequential number when the timestamp also conflicts) to the log file name.
-----
import os, time, re
src = FME_LogFileName
if src and os.path.exists(src):
    stamp = time.strftime('%Y%m%d_%H%M%S')
    name = '%s_%s' % (re.sub('^(.+)\\.log$', '\\1', src), stamp)
    base = name
    count = 1
    while os.path.exists('%s.log' % name):
        name = '%s_%d' % (base, count)
        count += 1
    os.rename(src, '%s.log' % name)
-----

2. Define a Scripted Parameter
If the translation process is executed always one by one, the renaming approach works well. However, if two or more processes for the same workspace will be executed simultaneously - e.g. when the WorkspaceRunner specified "No" to "Wait for Job Complete" runs the workspace several times, we cannot get the expected result. Because the original log file name conflicts between the processes.
In such a situation, we should create a unique log file name as a scripted parameter, and link the Workspace Parameter "Log File" to it. The script can be like this, for example.
-----
import os, time, re
src = FME_MacroValues['FME_MF_DIR'] + FME_MacroValues['FME_MF_NAME']
stamp = time.strftime('%Y%m%d_%H%M%S')
name = '%s_%s' % (re.sub('^(.+)\\.fmw$', '\\1', src), stamp)
base = name
count = 1
while os.path.exists('%s.log' % name):
    name = '%s_%d' % (base, count)
    count += 1
return '%s.log' % name
-----

Appendix: Tcl Script Examples
The following Tcl scripts work as well as the Python examples above.

Shutdown Tcl Script: Rename the Log File Name
------
set src $FME_LogFileName
if {[string length $src] && [file exists $src]} {
  set stamp [clock format [clock seconds] -format {%Y%m%d_%H%M%S}]
  set name [format "%s_%s" [regsub {^(.+)\.log$} $src {\1}] $stamp]
  set base $name
  for {set count 1} {[file exists [format "%s.log" $name]]} {incr count} {
    set name [format "%s_%d" $base $count]
  }
  file rename $src [format "%s.log" $name]
}
------

Scripted (Tcl) Parameter: Create a Unique Log File Name
------
set src "$FME_MacroValues(FME_MF_DIR)$FME_MacroValues(FME_MF_NAME)"
set stamp [clock format [clock seconds] -format {%Y%m%d_%H%M%S}]
set name [format "%s_%s" [regsub {^(.+)\.fmw$} $src {\1}] $stamp]
set base $name
for {set count 1} {[file exists [format "%s.log" $name]]} {incr count} {
  set name [format "%s_%d" $base $count]
}
return [format "%s.log" $name]
------

See another example here:
> Community: FME -> log file