Dataflow programming with dflibg™ is very different from traditional, imperative-style javascript programming.

dflibg-based javascript computations are completely data (and event) driven. A computation occurs when its data is 'ready' – i.e., available. Components of a dflibg-based application are nodes where computation occurs, datastreams between these nodes, and modules which organize related nodes.

In traditional, imperative javascript programming, computation occurs as directed by a sequence of steps – one operation following another, ... – not by when data is available. Frequently, in order to deal with 'when data is available', imperative javascript becomes 'call-back hell'.

As dflibg-based application programming is very different from traditional javascript for dHTML, so, too, must be the set of developer tools. Where traditional browser developer tools focus on instruction step sequences, functions, stack frames, etc., dataflow developer tools must focus on how data moves between the components. dflibg dfirefly has been created to meet that need.

dflibg™ dfirefly dataflow developer tools are intended to be used in concert with other developer tools – e.g., use Google Chrome™ developer tools, Mozilla Firebug™, etc., for inspecting resources and debugging the individual node's javascript implementation functions; use dfirefly for inspecting nodes, datastreams and events.

dflibg™ dfirefly includes a number of tools to assist in the development of dflibg-based applications, the illumination of dataflow design inconsistencies, and the debugging of the compiled dataflow application.

examine	is the dataflow equivalent of inspect {source}; instead of scripts/functions, it inspects the components of dataflow: the properties of nodes (including their implementation functions), and the dfe's and modules into which they are organized
linkage	or dcomexamines the datastream connections between the document and the dataflow modules – the document sources of input and the attachments through which data is written to the DOM.
watch	monitors dataflow in the executing machine; where an imperative tool 'watches' a variable for changes in value, dfirefly watches datastreams (node outputs).
debug	is our old friend, the breakpoint tool; however, instead of breakpointing instructions, dataflow events are breakpoints – when a node is fired because data became ready.

Each tool has its own welcome page, included in this set. Clicking the home icon in the menu bar will bring you back to these welcome pages. Browse with the nav bar, below, or select information pages, above.

Additional dflibg™ dfirefly tools dig deeper into the dataflow aspects of dflibg-based applications, the datastreams between components.

xcom	examines dataflow communications - not between nodes in the same module, but the datastream connections between modules, threads, frames, windows, extensions, and servers - i.e., promises, proxies, servers and component dataflow, oh my!
network	identifies/monitors asynchronous communications through proxy nodes, nodes which send and receive from external servers in response to their input dataflow, and through surrogate nodes, nodes which create and manage objects (e.g., windows) in response to input dataflow
console	allows one to 1) retrieve and follow the console log and the exceptions log issued by dflibg library, and 2) inject data into and/or configure dataflow components – in the 'inspected' window/page.
lint	dflibg.lint dynamically identifies inconsistencies and ambiguities – orphans, black-holes, cycles, missing datastreams, etc. – as well as other dataflow design analyses – multipath analsys, etc.
map/graph	while dfe and modules form the static design of a dataflow application, the dataflow design – the dataflow graph of the paths data follows from its injection into the dataflow engine until it is exported or written to a document – is analyzed via map/graph

The next page describes some the panels; following that are the individual dfTool introductions.

A number of the tools use a page layout similar to the one you see now –

• a dataflow component tree (or graph) outline as a sidebar panel,
• a detail panel (the one with the text you are reading now)
• a menu bar (shown above) with the tool name (e.g., 'welcome:') and any actions/subcommands

Sample Application Component Tree

The component outline sidebar on this  page shows a sample dataflow application - the dflibg™ dfirefly SDK. Indeed, this SDK/browser extension is itself a dflibg-based dataflow application! [The peer supporting modules embedded in the library of the 'inspected' application are also dataflow application modules. Communication between modules in this SDK and those peer modules also occurs via 'dataflow'!].

This SDK also makes extensive use of reusable dataflow modules – called Dynamic Server Modules. These modules drive the menu bars, the navigation bars, the hierarchical expand-collapse displays, and a host of other features. Hence, the SDK is a treasure trove of examples.

The next page includes credits and notes; following that are the individual dfTool introductions.

Legal Notes

All trademarks present in these pages are property of their respective owners. Use or inclusion of any is solely for the purpose of identification, and to illustrate how use of this SDK may integrate with the respective product or service.

A partial list: Google Chrome® is a registered trademark of Alphabet, Inc.; Firebug® is a registered trademark of the Mozilla Foundation; Graphviz developed at AT&T Bell Laboratories/AT&T Research, is now community supported.

In addition, dflibg™ library, dflibg dfirefly™ dataflow developer tools, and dflibg emerald™ dataflow software development kit, are trademarks of Chrysalis Systems (NJ) (supported at dflibg.org).

General Notes

The dflibg library, the dflibg dfirefly dataflow developer tools and dflibg emerald SDK are works-in-progress. While stable and most documented features fully functional, there are a few exceptions where a feature may not be complete. Attempts to note such exceptions are made here or in the documentation and/or sample demonstrations at dflibg.org. Unlike many projects, dflibg documentation is written before/during the software as a roadmap to development.

The following pages are the individual dfTool introductions.

Examine

Is the primary 'inspection' tool. Instead of inspecting scripts/functions, document elements, etc., it inspects the components of dataflow –

• nodes – where computation occurs – and
• their datastreams – where data and events flow between nodes and the outside world

In a dflibg-based dataflow application, related nodes are grouped into unnamed dataflow engines, dfes, and named dfe's, i.e., modules. Every application also includes a _global dfe for nodes not instantiated as part of a dfe or module. Use the Navigation Frame to browse these dfe's and modules to select a node and examine its properties and datastreams.

Navigation Frame

In the list of dfe's and modules, locate a node in the same way as you would drill-down a file directory tree to locate a file. The list in the examine tool's Navigation Frame is the Project View of the application.

In a Project View, nodes in a module or dfe are organized into several groups –

• parameter nodes – constants and references
• injector nodes – where data enters – and,
• computational nodes

When a node is selected, its detail properties, implementation function and statistics, datastreams (and their current values), etc., are shown in the detail frame. In addition, a node summary (name, type, current value) are shown in a header in the watch, debug, and console tools. Once selected, use the command buttons to a) addt the node to dfirefly's dataflow watchlist, b) add the node to the dataflow debugger's dataflow breakpoint list, or c) at any later point, refresh the properties and values display.

Watch

Monitors data in the executing dataflow machine; where an imperative tool 'watches' a variable for changes in value, dfirefly watches datastreams. Use watch to monitor specific datastreams that flow –

• into the dataflow engine – an input datastream to an injection node
• to other nodes – a node's output datastreams, and
• to document objects – a node's object datastreams

To monitor a datastream, we monitor to node that generates the datastream. To monitor an input datastream, we monitor the injection node to which it is connected.

So, to monitor data on a datastream, select a node from any navigator (examine panel, watch panel, debug panel, etc.). The node and its current output are displayed in the node header. [Complete node details are shown in the examine panel.] Click on the [ watch] button to add the selected node and its output datastream to the watchlist.

Selecting an injection node is a convenient way to monitor input events from document elements, dataflows from other windows, extensions, modules, threads, etc.

Other Ways to Watch Data

When a dataflow breakpoint is set, via any [ break] button, the node is added to the breakpoint list displayed in the debug panel, This list also monitors the value of the node, i.e., the value of its output datastream. The datastreams in the breakpoint list are always being watched, even if you have hidden their display. Further, if you disable an individual breakpoint in the breakpoint list, its output is still being monitored and (a summary of) its value will continue to be updated as that value is updated.

In addition, for most places where a node name is being displayed, clicking on that name queries dflibg in the running application. The node and its output datastream(s) are displayed in the watch panel's node header. Every node header (and the command bar in the examine panel) includes a [refresh] button, [] Click on it to requery for the node's current value(s).

Debug

is a full-featured dataflow debugger tool. In imperative Javascript, a debugger allows us to break at a particular instruction, inspect the 'machine state', and step through instructions to try to find where data manipulation went wrong.

In a dataflow debugger we follow the data – the dataflow events are the breakpoints. A breakpoint occurs when a node is about to be fired because because data became ready. Hence, we can

• set a breakpoint for a node,
• observe its inputs (and from whence they came)
• step through the transformation phases of data in the node, and then
• follow (step next, in, out, etc.) how data flows through the application.

Dataflow Threads

A completely new debugger strategy has being implemented using Dataflow Threads. Every external input – whether it is from a document event, dataflow from another thread module, window, or extension, an XMLHTTP response, etc. – creates a new dataflow thread. The thread begins with

• the injection node where the data/event was recognised,
• the proxy node where an AJAX response/acknowledgement was returned,
• the proxy node where a server/client's promise has been fullfilled and a response was returned.

The tail of the thread is the current node, the breakpoint. Please check the debug panel overview page for more details on the dataflow debugger.

A Dataflow Thread is completely independent of the Javascript stack! Hence, BOTH dfirefly AND the browser's Developer Tools may be used at the same time

Linkage

Good application design often dictates a modular approach – grouping together functions (in this case, nodes) that are highly cohesive, i.e., belong together. They may operate on the same data or stream of data as it flows through the dataflow engine (informational or sequential cohesion). They may focus on implementing a particular task or feature (functional cohesion). By its inherent nature, dataflow design is typically a combination of these.

Once functions (nodes) have been grouped (often as per the dataflow graphs or subgraphs) (dflibg-speak 'clusters'), best practices prefer that they be loosely coupled. One group (module, dfe, thread, ...) is unaware of the internal design of another. They 'share' information via well-defined, formal interfaces.

dflibg™ objects provide .client() and .source() methods to establish datastream linkages between such groups or modules, between a dfe in the current window and one in another, to/from a thread, an in-line frame, or even an extension component; between document events and 'listening' modules/nodes.

In addition, stream objects facilitate the ultimate loosely coupled interfaces - a negotiated datastream.

The linkage dfTool is used to identify and inspect those datastreams. For negotiated datastreams, it identifies the dataflows and the 'connection' state., etc.

Network

Network, shmetwork! Programmers using dflibg dataflow will never code another XMLHTTPrequest block, ever. All AJAX (Asynchronous Javascript and XML) communication occurs as the result of dataflow and defined behaviours for nodes. Nodes where data flows to/from external servers are Proxy Nodes. That is, they 'stand in for' or represent a dialogue with the external server.

The network facility provides the ability to monitor

• the datastreams flowing into the proxy node,
• the resulting output (the 'request' to GET, or the 'data' to POST, as applicable) from the proxy node to the server
• the response or acknowledgement from the server, and
• identify the distribution/propagation of the response/acknowledgement

Operation

For the network tool, the sidebar will show both true Proxy Nodes (those that may GET or POST and propagate the response to other nodes and/or clients) and Restricted Proxies (a node whose server response only flows to any attached document elements – not to other nodes and/or clients).

Select a Proxy Node to monitor and the click on [start] in the activity menu bar.

Dataflow Console

The dataflow console provides two features in separate panels. The first panel, the dflibg console log, captures dataflow log messages and exceptions issued by the dflibg library in the window for which dflibg dfirefly was activated.

I - dataflow console log

Tracking down just what caused runtime exceptions in any programming system is often fraught with utter frustration. The dflibg library-issued exception messages, catch the exceptions and map them to their origin. Not only is the exception identified, but where it occurred and why.

That is, dflibg-generated exception messages identify the specific node (and module) in whose implementation function the exception occurred. The messages also identify the origin of the error – the input to the dataflow engine and the node where that input was injected.

console log history

It is an unfortunate fact that the dflibg library, in the window/tab to be inspected, will have already issued many log and exception messages before dflibg™ dfirefly can be opened. Not to worry, all messages are cached and can be retrieved at any later time.

II - dataflow console command panel

The second console panel, the dataflow console command panel Is used to directly manipulate the dataflow nodes and application, much as typical developer tools' consoles allow inspection of javascript objects, invocation of functions, etc., by direct entry of javascript code.

While javascript can not be entered at a command prompt,

• data can be injected into the running dataflow engine,
• node configurations/properties may be modified
• selected library methods, module, node methods may be invoked.

Lint and the dflibg Dynamic Linker

dflibg™ lint does for dataflow – nodes, datastreams, modules and their graphs – what the Programmer's Workbench lint(1) did, what modern GCC compilers do for C/C++, and what some Javascript Lints do for imperative Javascript:

It identifies inconsistencies in the dataflow relationships, possible issues to investigate or verify design intention, etc.

Where traditional lint(1) identifies dead and unreachable code, mismatched call parameters, ambiguous usage, etc., dflibg lint identifies

• orphans, nodes with no inputs
• dead-end dataflows, nodes whose output goes nowhere, attaches to no document element, is never forwarded to another module, server, etc.
• inconsistent number of input datatreams, inappropriate or misused node types
• dataflow cycles and loops, recombinant paths

When the dflibg Dynamic Linker is activated, names and references unresolved at load/instantiation are resolved on-the-fly – when modules, CBF and document sections, etc., are loaded at some later time. dflibg unresolved may be invoked at any time to examine what references remain to be resolved.

For more information, see dflibg...Volume II - The Programmer's Desk Reference

Dataflow Analysis Toolkit

Features of the Dataflow Analysis Toolkit assist in the evaluation and improvement of both the static dataflow design and its performance. Tool panels include -

• Map - dataflow locality (natural clusters and mapping)
• Graph - visualization of the dataflow design
• Profile - performance / operational statistics

Profile

Execution statistics are continuously being accumulated when nodes in a dataflow application are being dispatched. Individual statistics are reported as part of a node's detail inspection in the Examine tool panel. Use the Profile tool to analyze this statistics and provide a relative comparison of what is occuring in the application

Map/Graph

While dfe and modules form the static design of a dataflow application, the dataflow design – the dataflow graph of the paths data follows from its injection into the dataflow engine until it is exported or written to a document – is analyzed via map/graph

map

analyzes the dataflow in the application, unrolling the dataflow to create

• a list of the dataflow clusters
• a report on how those clusters overlap

graph

derives the graph – nodes and datastreams/arcs. Currently, a dot> language representation of the DFE(s) is generated and displayed in a separate window.

Until the javascript port of Graphviz has been integrated, copy the output and feed to dot(1), on your Unix™ or Linux system. If the graph visualization suite is not already on your Mac™ OS X, the download is available on the Mac App Store.

Reference Set

In the future, this will have links to dfTools topics in the dflibg Desk Reference. For now, you'll just have to buy the book, make a donation, or open a subscription.

www.dflibg.org/support