Data Flow Diagram Plugin

Plugin containing macros for generating data flow diagrams

Purpose

The purpose of this plugin is to provide a means, when writing system/software design documentation, for understanding the interactions between processes without requiring a priori knowledge.

The plugin works by using macros to define individual components (entities) that make up the system, and how those components themselves interact with nearest-neighbors. The job of determining who those neighbors actually are is left up to the internals of the plug-in using said definitions.

Definitions

Data Type

A named collection of information. From a software design perspective, this might be a class or struct.

Group

An aggregation of multiple Data Types. From a software design perspective, this might be a library.

Process

Anything that produces or consumes instances of Data Types. From a software design perspective, this might be an application.

Translation

An indication that a Process ingests one Data Type and turns it into another Data Type before providing the new data to other Processes.

Locale

A grouping of processes. This may be a physical location, a host, a component within an embedded system; anything where processes might be grouped together. A process may have multiple locales or none at all. Locale definitions allow for associating a readable locale name with a locale ID used in macros.

Transport

A named mechanism for transferring instances of Data Types between Processes.

deprecated

An attribute for design elements indicating that the element has been removed from the implementation.

Orphan

Any design element whose documentation is incomplete or non-existent.

The following diagram indicates the relationships between the entities used by this plugin:

Plugin Preferences

Note: This plugin does not read settings from this topic. Copy the settings below to your SitePreferences topic and remove the # if changes are required. These settings can be placed in SitePreferences, WebPreferences, User topics, or any individual topic. Refer to preference settings for more information.

Note: This plugin utilizes DirectedGraphPlugin, therefore its preferences will affect the appearance of graph output for this plugin.

   * #Set DATAFLOWDIAPLUGIN_DEBUG = 0
   * #Set DATAFLOWDIAPLUGIN_DOTTAGOPTS = inline="svg" map="1" vectorformats="dot"
   * #Set DATAFLOWDIAPLUGIN_GRAPHDEFAULTS = rankdir="LR",labelloc="t"
   * #Set DATAFLOWDIAPLUGIN_EDGEDEFAULTS = fontsize=8
   * #Set DATAFLOWDIAPLUGIN_NODEDEFAULTS = style=filled,fontsize=9,fillcolor=white
   * #Set DATAFLOWDIAPLUGIN_PROCDEFAULTS = shape="ellipse"
   * #Set DATAFLOWDIAPLUGIN_DEPPROCDEFAULTS = shape="ellipse",fillcolor=red
   * #Set DATAFLOWDIAPLUGIN_DATADEFAULTS = shape="note"
   * #Set DATAFLOWDIAPLUGIN_DEPDATADEFAULTS = shape="note",fillcolor=red
   * #Set DATAFLOWDIAPLUGIN_DEPMARKUP = del

Description of the settings

Changes to the settings marked will cause changes to the file hash. This will result in the attached graphics being regenerated on the next view.

All settings are prefixed with DATAFLOWDIAPLUGIN_ as shown above The prefix is ommited from this table to improve layout.

Setting	Default	Description
DEBUG	0	Produce debugging output in Foswiki debug.log
DOTTAGOPTS	inline="svg" map="1" vectorformats="dot"	Parameters specified in the <dot></dot> tag (refer to DirectedGraphPlugin for more information).
GRAPHDEFAULTS	rankdir="LR",labelloc="t"	Graphviz graph default settings
EDGEDEFAULTS	fontsize=8	Graphviz edge default settings
NODEDEFAULTS	style=filled,fontsize=9,fillcolor=white	Graphviz node default settings
PROCDEFAULTS	shape="ellipse"	Graphviz node settings for non-deprecated processes
DEPPROCDEFAULTS	shape="ellipse",fillcolor=red	Graphviz node settings for deprecated processes
DATADEFAULTS	shape="note"	Graphviz node settings for non-deprecated data types
DEPDATADEFAULTS	shape="note",fillcolor=red	Graphviz node settings for deprecated data types
DEPMARKUP	del	HTML tag used for deprecated entities when rendered as links

Macros

The data flow diagram generation macros defined by this plugin build a database of cross reference information. This is done using the id parameter of the macro. The id parameter is used for linking between diagrams.

DFDPROC{...} -- renders a data flow diagram for processes

• Parameters:

  Parameter	Description	Default
  id	Process ID	none, required parameter
  name	Process name	value of id
  url	Specify a URL to be used when clicking on this particular process node in data flow diagrams	Wiki page where this data flow diagram is found
  deprecated	Indicates whether this process is still in use and how and when it will be rendered	0
  inxport	Data transport to use for inputs unless otherwise specified in the inputs list	Transport defined for the data type
  outxport	Data transport to use for outputs unless otherwise specified in the outputs list	Transport defined for the data type
  inoutxport	Data transport to use for in/outs unless otherwise specified in the inouts list	Transport defined for the data type
  inputs	List of data type names taken as inputs to proc	none
  outputs	List of data type names produced as outputs to proc	none
  inouts	List of data type names produced as outputs to proc, where those same produced messages are also taken as inputs (e.g. for state recovery)	none
  locales	List of locales where this process is used	none
  translation	List of data translations performed by this process (see below)	none

The values of inputs, outputs and inouts should be a comma-delimited list of data type identifiers. If linking from the data type nodes in the diagram to a wiki or web page is desired, the values should be used in an id parameter of an DFDDATA macro on the wiki. If no link is available for a given data type ID, the ID text as specified in the list is used and no linking is generated.

An example:

%DFDPROC{id="cc" inputs="code#file object"}%

The values of locales should be a comma-delimited list of locale identifiers.

The values of translation should be a comma-delimited list of from/to pairs of data type identifiers. The pair is separated by a > as in the following example:

%DFDDATA{id="ex1"}%
%DFDDATA{id="ex2"}%
%DFDPROC{id="proc" inputs="ex1" outputs="ex2" translation="ex1>ex2"}%

Diagram edges representing this transport will be labeled using name, which if not specified is the value of id.

If deprecated is non-zero, graphs will render the process according to DATAFLOWDIAPLUGIN_DEPPROCDEFAULTS. Tables will render the process thus.

DFDDATA{...} -- renders a data flow diagram for data types

• Parameters:

  Parameter	Description	Default
  id	Data type ID	none, required parameter
  name	Data type name	value of id
  url	Specify a URL to be used when clicking on this particular data type node in data flow diagrams	Wiki page where this data flow diagram is found
  deprecated	Indicates whether this data type is still in use and how and when it will be rendered	0
  group	A group name used when annotating data type nodes in data flow diagrams, usually the name of the software library containing the definition/implementation of the data type	none
  xport	Name of transport used for this data type. If linking from the transport to a wiki or web page is desired, the value should be one that is in an id parameter of an DFDTRANSPORT macro on the wiki. This transport will be used as the label for edges to and from this data type.	none

Data nodes in diagrams will be labeled with two or three lines of text, the top being the group and the bottom being the name, or id if name is not specified. When rendering process-centric diagrams using %DFDPROC, a 3rd line may be present with a data sub-id or sub-type, however graphs generated using this macro will not be labeled this way. The generated nodes will link to the value in url if specified, or the wiki topic where the DFDDATA macro is present.

If deprecated is non-zero, graphs will render the data type according to DATAFLOWDIAPLUGIN_DEPDATADEFAULTS.

DFDTRANSPORT{...} -- specifies a target location for data flow diagram transport links

• Parameters:

  Parameter	Description	Default
  id	ID of the transport type	none, required parameter
  name	Name of the transport type	value of id
  url	Specify a URL to be used when clicking on this particular transport type edge in data flow diagrams	Wiki page where this macro is found
  deprecated	Indicates whether this transport is still in use and how and when it will be rendered	0

This macro does not actually render anything on the wiki page where it is used. Instead it only provides an anchor to use when linking data flow diagrams to transport mechanisms. The link used in those instances is the value in url (which is the location of the DFDTRANSPORT macro if not specified).

Diagram edges representing this transport will be labeled using name, which if not specified is the value of id.

DFDLOCALE{...} -- renders a formatted list of data type translations by process

• Parameters:

  Parameter	Description	Default
  id	Process ID	none, required parameter
  name	A text description of this locale	value of id
  url	Specify a URL to be used when clicking on this particular locale in data flow diagrams	Wiki page where this data flow diagram is found
  deprecated	Indicates whether this locale is still in use and how and when it will be rendered	0
  connect	Connections via transports to other locales	none

Locales are rendered in diagrams as Graphviz cluster subgraphs. The subgraph will be labeled using name. The connect parameter is used to determine whether non-multicast data can flow between two locales.

The format of the connect parameter is a comma-separated list of pairs of locale|transport. As an example:

%DFDLOCALE{id="here" name="My Place" connect="there|ICMP"}%
%DFDLOCALE{id="there" name="Your Place"}%

In the above example, data utilizing the ICMP transport can flow from here to there, but not from there to here.

Note: There are no defaults for connections even for the self-locale. This means that in the above example, data utilizing the ICMP transport cannot be sent from here to here successfully. To remedy that, define your locale in the following fashion:

%DFDLOCALE{id="elsewhere" name="Another Place" connect="elsewhere|ICMP, there|ICMP, here|ICMP"}%

The above example allows the elsewhere locale to send data via ICMP to itself, here and there.

DFDGROUP{...} -- defines a group for aggregating entities

• Parameters:

  Parameter	Description	Default
  id	ID of the group	none, required parameter
  name	Name of the group	value of id
  url	Specify a URL to be used when clicking on this particular group type edge in data flow diagrams	Wiki page where this macro is found
  deprecated	Indicates whether this group is still in use and how and when it will be rendered	0

This macro does not actually render anything on the wiki page where it is used. Instead it only provides an anchor to use when linking data flow diagrams to group mechanisms. The link used in those instances is the value in url (which is the location of the DFDGROUP macro if not specified).

The primary use of GROUPs is in the organizion of Data Types. Generally speaking, the creationg of groups is unnecessary. Groups are not rendered at all, but are intended to be used in DFDSEARCH and DFDCONNECT queries, by ID.

DFDSEARCH{...} -- Perform a low-level XPath query on the DataFlowDiaPlugin data

• Parameters:

  Parameter	Description	Default
  query	The XPath query to perform on the data store (see below).	none, required parameter
  aggregate	If 0, apply "format" to each matching XML node. See below.	1
  atomempty	If 1, insert the empty value for individual (atomic) XPath subexpressions in format that have no results.	0
  separator	Specifies text to insert between each record of formatted output.	$n() (newline)
  newline	Text to insert in the output, if aggregate is 1, between aggregated matches.	<br/>
  format	Specifies the output format using an XPath expression.	none
  header	Specifies text to insert at the top of formatted output.	none
  footer	Specifies text to insert at the bottom of formatted output.	none
  empty	Text to insert into formatted output if an XPath expression matches nothing.
  zeroresults	Text to display if there are no matches to the query expression.	No results.$n
  label	Indicate how wiki links are labeled, with the Entity id, spec, aspec (abbreviated spec) or topic	name

The query option is where half the utility of this macro resides. Use this option to specify an XPath query that will match XML nodes in the data store that are of interest. Example:

/datacrossref/data[@deprecated='1' and @web='Extensions']

See DataFlowDiaPluginFormatting for details on formatted text output.

Some noteworthy comments:

• All entity nodes have an @id, @name, @web, @topic and @defined attribute.
• Entities other than group may have group child nodes.
• As a work-around for the fact that XPath 1.0 does not support querying node names, all nodes for searching have a @nodename attribute.
  • Aside: The nodename() function is supported by XPath 2.0 but libxml2 does not and apparently will not support XPath 2.0. DataFlowDiaPlugin is developed using a Perl wrapper for libxml2.
• Cross references defined in DFD macros are typically accessible to DFDSEARCH using the singular (non-plural) form of the name, e.g. inputs becomes multiple XML nodes called input.

DFDCONNECT{...} -- renders an IPC diagram or list

The DFDCONNECT macro renders either a graph of the interconnections between entities (data type or process), or a formatted text (or HTML or wiki mark-up) representation of that graph.

• Parameters:

  Parameter	Description	Default
  id	Entity ID	none, required parameter
  type	Graph type, see below	none, required parameter
  dir	See below	3 (both)
  level	The depth of the diagram, i.e. how many links away from id will be included.	0
  datanodes	If non-zero, render data types as nodes in graphs	0 (do not render data types as nodes)
  printself	If 0, the proc indicated by id will not be listed in the output. Valid with format only.	Given proc is listed in output.
  nolocales	For graphs, processes are clustered by locale. If this parameter is non-zero, the nodes are not clustered	0
  aggregate	If 0, apply "format" to each matching XML node. See below.	1
  atomempty	If 1, insert the empty value for individual (atomic) XPath subexpressions in format that have no results.	0
  hidedeprecated	If 1, deprecated entities will not be rendered	0
  separator	Specifies text to insert between each record of formatted output.	$n() (newline)
  newline	Text to insert in the output, if aggregate is 1, between aggregated matches.	<br/>
  empty	Text to insert into formatted output if an XPath expression matches nothing.
  zeroresults	Text to display if there are no matches to the query expression.	No results.$n
  label	Indicate how wiki links are labeled, with the Entity id, spec, aspec (abbreviated spec) or topic	name
  format	Specifies the output format using an XPath expression.	none
  header	Specifies text to insert at the top of formatted output.	none
  footer	Specifies text to insert at the bottom of formatted output.	none
  locales	Only render entities within in the given list of comma-separated locale IDs	all locales are included
  exclocales	Do not render entities withing the given list of comma-separated locale IDs	no locales are excluded

Output is generated using data from DFDDATA macros and DFDPROC macros.

Graph Types

proc

Render a connection diagram between Process and Data Type entities, using a Process entity as a starting point. May have multiple starting points, depending on Locale associations and macro parameters.

data

Render a connection diagram between Process and Data Type entities, using a Data Type entity as a starting point. Aside from the obvious, such a diagram may also have multiple "starting points" due to the fact that multiple Processes may utilize the requested Data Type. The only Data Type entities appearing in the graph will be those matching the specified ID.

group

Render a connection diagram between Process and Data Type entities, where only Data Types that are members of the specified group are rendered, along with the Process entities that utilize those Data Types.

translation

Similar to a "data" graph type; additionally takes into account the translation between Data Types as specified in Process definitions (via DFDPROC macros).

See DataFlowDiaPluginFormatting for details on formatted text output.

Examples

Examples can be found in DataFlowDiaPluginExamples.

Extensive sanity checks can be found in PluginTestDataFlowDiaPlugin.

Plugin Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Installation

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

Dependencies

Package	Description
DirectedGraphPlugin	Allows embedding dynamic diagrams in wiki documents
XML::LibXML	Perl module for reading and saving XML files

Known Bugs / Issues

• Macro parameters are not sanitized.

TODO

Plugin Info

Change History:
06 Oct 2015	2.0.1 Remove entities from store when removed from topic Don't store inherited definitions in XML Fix various transport issues
28 Sep 2015	2.0.0 Rewrite to include web discrimination and XPath queries
03 Mar 2015	1.1.0 Add customization preferences Add data type translation tracking Change from registered macros to commonTagsHandler to allow nested macros
12 Feb 2015	1.0.1 Initial release