TIP DataFlowDiaPlugin is not installed on Foswiki.org.

Data Flow Diagram Plugin

Plugin containing macros for generating data flow diagrams

Purpose

The purpose of this plugin is to provide a means, when writing system/software design documentation, for understanding the interactions between processes without requiring a priori knowledge.

The plugin works by using macros to define individual components (entities) that make up the system, and how those components themselves interact with nearest-neighbors. The job of determining who those neighbors actually are is left up to the internals of the plug-in using said definitions.

Definitions

Data Type
A named collection of information. From a software design perspective, this might be a class or struct.
Group
An aggregation of multiple Data Types. From a software design perspective, this might be a library.
Process
Anything that produces or consumes instances of Data Types. From a software design perspective, this might be an application.
Translation
An indication that a Process ingests one Data Type and turns it into another Data Type before providing the new data to other Processes.
Locale
A grouping of processes. This may be a physical location, a host, a component within an embedded system; anything where processes might be grouped together. A process may have multiple locales or none at all. Locale definitions allow for associating a readable locale name with a locale ID used in macros.
Transport
A named mechanism for transferring instances of Data Types between Processes.
deprecated
An attribute for design elements indicating that the element has been removed from the implementation.
Orphan
Any design element whose documentation is incomplete or non-existent.

The following diagram indicates the relationships between the entities used by this plugin:

DataFlowDiaPlugin objects UML diagram

Plugin Preferences

ALERT! Note: This plugin does not read settings from this topic. Copy the settings below to your SitePreferences topic and remove the # if changes are required. These settings can be placed in SitePreferences, WebPreferences, User topics, or any individual topic. Refer to preference settings for more information.

ALERT! Note: This plugin utilizes DirectedGraphPlugin, therefore its preferences will affect the appearance of graph output for this plugin.

   * #Set DATAFLOWDIAPLUGIN_DEBUG = 0
   * #Set DATAFLOWDIAPLUGIN_DOTTAGOPTS = inline="svg" map="1" vectorformats="dot"
   * #Set DATAFLOWDIAPLUGIN_GRAPHDEFAULTS = rankdir="LR",labelloc="t"
   * #Set DATAFLOWDIAPLUGIN_EDGEDEFAULTS = fontsize=8
   * #Set DATAFLOWDIAPLUGIN_NODEDEFAULTS = style=filled,fontsize=9,fillcolor=white
   * #Set DATAFLOWDIAPLUGIN_PROCDEFAULTS = shape="ellipse"
   * #Set DATAFLOWDIAPLUGIN_DEPPROCDEFAULTS = shape="ellipse",fillcolor=red
   * #Set DATAFLOWDIAPLUGIN_DATADEFAULTS = shape="note"
   * #Set DATAFLOWDIAPLUGIN_DEPDATADEFAULTS = shape="note",fillcolor=red
   * #Set DATAFLOWDIAPLUGIN_DEPMARKUP = del

Description of the settings

Changes to the settings marked ALERT! will cause changes to the file hash. This will result in the attached graphics being regenerated on the next view.

All settings are prefixed with DATAFLOWDIAPLUGIN_ as shown above The prefix is ommited from this table to improve layout.

Setting Default Description
DEBUG 0 Produce debugging output in Foswiki debug.log
DOTTAGOPTS inline="svg" map="1" vectorformats="dot" ALERT! Parameters specified in the <dot></dot> tag (refer to DirectedGraphPlugin for more information).
GRAPHDEFAULTS rankdir="LR",labelloc="t" ALERT! Graphviz graph default settings
EDGEDEFAULTS fontsize=8 ALERT! Graphviz edge default settings
NODEDEFAULTS style=filled,fontsize=9,fillcolor=white ALERT! Graphviz node default settings
PROCDEFAULTS shape="ellipse" ALERT! Graphviz node settings for non-deprecated processes
DEPPROCDEFAULTS shape="ellipse",fillcolor=red ALERT! Graphviz node settings for deprecated processes
DATADEFAULTS shape="note" ALERT! Graphviz node settings for non-deprecated data types
DEPDATADEFAULTS shape="note",fillcolor=red ALERT! Graphviz node settings for deprecated data types
DEPMARKUP del HTML tag used for deprecated entities when rendered as links

Macros

The data flow diagram generation macros defined by this plugin build a database of cross reference information. This is done using the id parameter of the macro. The id parameter is used for linking between diagrams.

DFDPROC{...} -- renders a data flow diagram for processes

  • Parameters:
    Parameter Description Default
    id Process ID none, required parameter
    name Process name value of id
    url Specify a URL to be used when clicking on this particular process node in data flow diagrams Wiki page where this data flow diagram is found
    deprecated Indicates whether this process is still in use and how and when it will be rendered 0
    inxport Data transport to use for inputs unless otherwise specified in the inputs list Transport defined for the data type
    outxport Data transport to use for outputs unless otherwise specified in the outputs list Transport defined for the data type
    inoutxport Data transport to use for in/outs unless otherwise specified in the inouts list Transport defined for the data type
    inputs List of data type names taken as inputs to proc none
    outputs List of data type names produced as outputs to proc none
    inouts List of data type names produced as outputs to proc, where those same produced messages are also taken as inputs (e.g. for state recovery) none
    locales List of locales where this process is used none
    translation List of data translations performed by this process (see below) none

The values of inputs, outputs and inouts should be a comma-delimited list of data type identifiers. If linking from the data type nodes in the diagram to a wiki or web page is desired, the values should be used in an id parameter of an DFDDATA macro on the wiki. If no link is available for a given data type ID, the ID text as specified in the list is used and no linking is generated.

An example:
%DFDPROC{id="cc" inputs="code#file object"}%

The values of locales should be a comma-delimited list of locale identifiers.

The values of translation should be a comma-delimited list of from/to pairs of data type identifiers. The pair is separated by a > as in the following example:
%DFDDATA{id="ex1"}%
%DFDDATA{id="ex2"}%
%DFDPROC{id="proc" inputs="ex1" outputs="ex2" translation="ex1>ex2"}%

Diagram edges representing this transport will be labeled using name, which if not specified is the value of id.

If deprecated is non-zero, graphs will render the process according to DATAFLOWDIAPLUGIN_DEPPROCDEFAULTS. Tables will render the process thus.

DFDDATA{...} -- renders a data flow diagram for data types

  • Parameters:
    Parameter Description Default
    id Data type ID none, required parameter
    name Data type name value of id
    url Specify a URL to be used when clicking on this particular data type node in data flow diagrams Wiki page where this data flow diagram is found
    deprecated Indicates whether this data type is still in use and how and when it will be rendered 0
    group A group name used when annotating data type nodes in data flow diagrams, usually the name of the software library containing the definition/implementation of the data type none
    xport Name of transport used for this data type. If linking from the transport to a wiki or web page is desired, the value should be one that is in an id parameter of an DFDTRANSPORT macro on the wiki. This transport will be used as the label for edges to and from this data type. none

Data nodes in diagrams will be labeled with two or three lines of text, the top being the group and the bottom being the name, or id if name is not specified. When rendering process-centric diagrams using %DFDPROC, a 3rd line may be present with a data sub-id or sub-type, however graphs generated using this macro will not be labeled this way. The generated nodes will link to the value in url if specified, or the wiki topic where the DFDDATA macro is present.

If deprecated is non-zero, graphs will render the data type according to DATAFLOWDIAPLUGIN_DEPDATADEFAULTS.
  • Parameters:
    Parameter Description Default
    id ID of the transport type none, required parameter
    name Name of the transport type value of id
    url Specify a URL to be used when clicking on this particular transport type edge in data flow diagrams Wiki page where this macro is found
    deprecated Indicates whether this transport is still in use and how and when it will be rendered 0

This macro does not actually render anything on the wiki page where it is used. Instead it only provides an anchor to use when linking data flow diagrams to transport mechanisms. The link used in those instances is the value in url (which is the location of the DFDTRANSPORT macro if not specified).

Diagram edges representing this transport will be labeled using name, which if not specified is the value of id.

DFDLOCALE{...} -- renders a formatted list of data type translations by process

  • Parameters:
    Parameter Description Default
    id Process ID none, required parameter
    name A text description of this locale value of id
    url Specify a URL to be used when clicking on this particular locale in data flow diagrams Wiki page where this data flow diagram is found
    deprecated Indicates whether this locale is still in use and how and when it will be rendered 0
    connect Connections via transports to other locales none

Locales are rendered in diagrams as Graphviz cluster subgraphs. The subgraph will be labeled using name. The connect parameter is used to determine whether non-multicast data can flow between two locales.

The format of the connect parameter is a comma-separated list of pairs of locale|transport. As an example:

%DFDLOCALE{id="here" name="My Place" connect="there|ICMP"}%
%DFDLOCALE{id="there" name="Your Place"}%

In the above example, data utilizing the ICMP transport can flow from here to there, but not from there to here.

Note: There are no defaults for connections even for the self-locale. This means that in the above example, data utilizing the ICMP transport cannot be sent from here to here successfully. To remedy that, define your locale in the following fashion:

%DFDLOCALE{id="elsewhere" name="Another Place" connect="elsewhere|ICMP, there|ICMP, here|ICMP"}%

The above example allows the elsewhere locale to send data via ICMP to itself, here and there.

DFDGROUP{...} -- defines a group for aggregating entities

  • Parameters:
    Parameter Description Default
    id ID of the group none, required parameter
    name Name of the group value of id
    url Specify a URL to be used when clicking on this particular group type edge in data flow diagrams Wiki page where this macro is found
    deprecated Indicates whether this group is still in use and how and when it will be rendered 0

This macro does not actually render anything on the wiki page where it is used. Instead it only provides an anchor to use when linking data flow diagrams to group mechanisms. The link used in those instances is the value in url (which is the location of the DFDGROUP macro if not specified).

The primary use of GROUPs is in the organizion of Data Types. Generally speaking, the creationg of groups is unnecessary. Groups are not rendered at all, but are intended to be used in DFDSEARCH and DFDCONNECT queries, by ID.

DFDSEARCH{...} -- Perform a low-level XPath query on the DataFlowDiaPlugin data

  • Parameters:
    Parameter Description Default
    query The XPath query to perform on the data store (see below). none, required parameter
    aggregate If 0, apply "format" to each matching XML node. See below. 1
    atomempty If 1, insert the empty value for individual (atomic) XPath subexpressions in format that have no results. 0
    separator Specifies text to insert between each record of formatted output. $n() (newline)
    newline Text to insert in the output, if aggregate is 1, between aggregated matches. <br/>
    format Specifies the output format using an XPath expression. none
    header Specifies text to insert at the top of formatted output. none
    footer Specifies text to insert at the bottom of formatted output. none
    empty Text to insert into formatted output if an XPath expression matches nothing.  
    zeroresults Text to display if there are no matches to the query expression. No results.$n
    label Indicate how wiki links are labeled, with the Entity id, spec, aspec (abbreviated spec) or topic name

The query option is where half the utility of this macro resides. Use this option to specify an XPath query that will match XML nodes in the data store that are of interest. Example:
/datacrossref/data[@deprecated='1' and @web='Extensions']

See DataFlowDiaPluginFormatting for details on formatted text output.

Some noteworthy comments:
  • All entity nodes have an @id, @name, @web, @topic and @defined attribute.
  • Entities other than group may have group child nodes.
  • As a work-around for the fact that XPath 1.0 does not support querying node names, all nodes for searching have a @nodename attribute.
    • Aside: The nodename() function is supported by XPath 2.0 but libxml2 does not and apparently will not support XPath 2.0. DataFlowDiaPlugin is developed using a Perl wrapper for libxml2.
  • Cross references defined in DFD macros are typically accessible to DFDSEARCH using the singular (non-plural) form of the name, e.g. inputs becomes multiple XML nodes called input.

DFDCONNECT{...} -- renders an IPC diagram or list

The DFDCONNECT macro renders either a graph of the interconnections between entities (data type or process), or a formatted text (or HTML or wiki mark-up) representation of that graph.

  • Parameters:
    Parameter Description Default
    id Entity ID none, required parameter
    type Graph type, see below none, required parameter
    dir See below 3 (both)
    level The depth of the diagram, i.e. how many links away from id will be included. 0
    datanodes If non-zero, render data types as nodes in graphs 0 (do not render data types as nodes)
    printself If 0, the proc indicated by id will not be listed in the output. Valid with format only. Given proc is listed in output.
    nolocales For graphs, processes are clustered by locale. If this parameter is non-zero, the nodes are not clustered 0
    aggregate If 0, apply "format" to each matching XML node. See below. 1
    atomempty If 1, insert the empty value for individual (atomic) XPath subexpressions in format that have no results. 0
    hidedeprecated If 1, deprecated entities will not be rendered 0
    separator Specifies text to insert between each record of formatted output. $n() (newline)
    newline Text to insert in the output, if aggregate is 1, between aggregated matches. <br/>
    empty Text to insert into formatted output if an XPath expression matches nothing.  
    zeroresults Text to display if there are no matches to the query expression. No results.$n
    label Indicate how wiki links are labeled, with the Entity id, spec, aspec (abbreviated spec) or topic name
    format Specifies the output format using an XPath expression. none
    header Specifies text to insert at the top of formatted output. none
    footer Specifies text to insert at the bottom of formatted output. none
    locales Only render entities within in the given list of comma-separated locale IDs all locales are included
    exclocales Do not render entities withing the given list of comma-separated locale IDs no locales are excluded

Output is generated using data from DFDDATA macros and DFDPROC macros.

Graph Types
proc
Render a connection diagram between Process and Data Type entities, using a Process entity as a starting point. May have multiple starting points, depending on Locale associations and macro parameters.
data
Render a connection diagram between Process and Data Type entities, using a Data Type entity as a starting point. Aside from the obvious, such a diagram may also have multiple "starting points" due to the fact that multiple Processes may utilize the requested Data Type. The only Data Type entities appearing in the graph will be those matching the specified ID.
group
Render a connection diagram between Process and Data Type entities, where only Data Types that are members of the specified group are rendered, along with the Process entities that utilize those Data Types.
translation
Similar to a "data" graph type; additionally takes into account the translation between Data Types as specified in Process definitions (via DFDPROC macros).

See DataFlowDiaPluginFormatting for details on formatted text output.

Examples

Examples can be found in DataFlowDiaPluginExamples.

Extensive sanity checks can be found in PluginTestDataFlowDiaPlugin.

Plugin Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Installation

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

Dependencies

Package Description
DirectedGraphPlugin Allows embedding dynamic diagrams in wiki documents
XML::LibXML Perl module for reading and saving XML files

Known Bugs / Issues

  • Macro parameters are not sanitized.

TODO

Plugin Info

Change History:  
06 Oct 2015 2.0.1
  1. Remove entities from store when removed from topic
  2. Don't store inherited definitions in XML
  3. Fix various transport issues
28 Sep 2015 2.0.0 Rewrite to include web discrimination and XPath queries
03 Mar 2015 1.1.0
  1. Add customization preferences
  2. Add data type translation tracking
  3. Change from registered macros to commonTagsHandler to allow nested macros
12 Feb 2015 1.0.1 Initial release

PackageForm edit

Author Foswiki:Main.JohnKnutson
Version v2.0.1
Release 2.0.1
Description
Copyright © 2015, Applied Research Laboratories, the University of Texas at Austin
License GPLv3
Home http://foswiki.org/Extensions/DataFlowDiaPlugin
Support http://foswiki.org/Support/DataFlowDiaPlugin
Repository https://github.com/foswiki/DataFlowDiaPlugin
ExtensionClassification Interface and Visualisation
ExtensionType PluginPackage
Compatibility
IncompatibleWith
ImageUrl
DemoUrl http://
SupportUrl DataFlowDiaPlugin
I Attachment Action Size Date Who Comment
COPYING.txttxt COPYING.txt manage 34 K 21 Oct 2016 - 02:31 JohnKnutson GNU General Public License v3
DataFlowDiaPlugin.md5md5 DataFlowDiaPlugin.md5 manage 174 bytes 21 Oct 2016 - 02:33 JohnKnutson  
DataFlowDiaPlugin.sha1sha1 DataFlowDiaPlugin.sha1 manage 198 bytes 21 Oct 2016 - 02:33 JohnKnutson  
DataFlowDiaPlugin.tgztgz DataFlowDiaPlugin.tgz manage 557 K 21 Oct 2016 - 02:33 JohnKnutson  
DataFlowDiaPlugin.zipzip DataFlowDiaPlugin.zip manage 620 K 21 Oct 2016 - 02:32 JohnKnutson  
DataFlowDiaPlugin_installerEXT DataFlowDiaPlugin_installer manage 14 K 21 Oct 2016 - 02:33 JohnKnutson  
datatype-search.xsdxsd datatype-search.xsd manage 4 K 21 Oct 2016 - 02:31 JohnKnutson XML Schema for DFDDATA definition searches
group-search.xsdxsd group-search.xsd manage 2 K 21 Oct 2016 - 02:31 JohnKnutson XML Schema for DFDDATA definition searches
locale-search.xsdxsd locale-search.xsd manage 4 K 21 Oct 2016 - 02:31 JohnKnutson XML Schema for DFDDATA definition searches
objects.pngpng objects.png manage 14 K 21 Oct 2016 - 02:32 JohnKnutson Entity Relation Object Diagram
process-search.xsdxsd process-search.xsd manage 4 K 21 Oct 2016 - 02:32 JohnKnutson XML Schema for DFDDATA definition searches
transport-search.xsdxsd transport-search.xsd manage 3 K 21 Oct 2016 - 02:32 JohnKnutson XML Schema for DFDDATA definition searches
Topic revision: r2 - 27 Mar 2017, JohnKnutson
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy