Content ▶ Edition ▶

User: Password: Log in | Register

Plotting tools for networks, part I

In the first two installments in this series on plotting tools (which covered gnuplot and matplotlib), we introduced tools for creating plots and graphs, and used the terms interchangeably to refer to the typical April 15, 2015 scientific plot relating one set of quantities to another. In this article we use the term "graph" in its This article was contributed mathematical, graph-theory context, meaning a set of nodes connected by edges. There is a strong family by Lee Phillips resemblance among graph-theory graphs, flowcharts, and network diagrams—so much so that some of the same tools can be coerced into creating all of them. We will now survey several mature free-software systems for building these types of visualizations. At least one of these tools will likely be useful if you are ever in need of an automated way to diagram source-code interdependencies, make an organizational chart, visualize a computer network, or organize a sports tournament. We will start with a graphical charting tool and a flexible graphing system that can easily be called by other programs.

Flowcharting with Dia

A flowchart is a diagram of a process, algorithm, workflow, or something similar. Flowcharts for different fields often employ a specialized graphical language of symbols that represent entities common to the field. For example, a circuit diagram is a type of flowchart that uses special symbols for diodes, resistors, and other circuit elements. There are flowchart languages for logic circuits, chemical engineering, software design, and much more.

Dia, a free (in all senses) diagram editor for Linux and other systems, comes with symbol libraries encompassing all of these examples, plus many others, both common and exotic. And, if that's not sufficient, the program allows you to make your own symbols.

Dia is a GUI program that uses the GTK+ libraries. You use it somewhat like Inkscape or other drawing programs. However, to make effective use of the program you should remember that you are not creating a drawing, but, rather, defining a set of relationships between entities. These relationships are represented by lines and curves (perhaps with arrowheads or labels), and the entities take the forms of the various symbolic shapes we mentioned above, often with their own text labels.

The trick to defining these relationships through the graphical interface is to make the connections in the right way. Since it takes a while to extract these techniques from the documentation, we'll outline the steps here.

After selecting the desired shape from the panel and dragging it out in the canvas to the approximate size you think it should be, immediately press Return and type the text label for the shape. The label will be properly centered, the shape will grow as required to accommodate it, and the label will be permanently attached to the shape and move with it. To connect two entities with a line, draw the line between the centers of the entities; you know you've hit the correct spot when the shape glows yellow.

To attach a text label to a line (such as the "yes" and "no" labels in the screenshot) you need to follow a different procedure: with object snapping turned on, create a text entity using the text tool that looks like a "T", then drag it by its handle, connecting it to the line's attachment point. This point is indicated by a small "x" and is at the center of the line. A red glow will signal that you've made the attachment.

If you've defined all your labels, entities, and connections using these techniques, then you'll be able to move the nodes around at will on the canvas until the chart is neat and easy to follow. The topology of the graph, which carries the actual information in the flowchart, won't change but, by moving things around, you can change a tangle of crossed lines into a neat diagram where the flow is clear.

Dia saves your work in an XML file (a compressed one by default, though there is also an option to save it uncompressed), and can export it into a wide variety of image formats, including vector formats such as SVG.

The program should be available in your package manager. Development is steady but moves at a slow pace, so it's likely you'll get the current version even from a conservative distribution. If you need to, however, you can download sources or binaries from Dia headquarters.

Graphviz: infrastructure for graphs

Dia is a useful and versatile tool for creating and laying out a graph by hand. Sometimes, however, we begin with a (possibly large) set of data that we want to visualize as a network-style graph or flowchart. We may also want to experiment with different types of visualizations or to produce different graph styles that present the same data for different purposes.

Graphviz solves these problems by providing a declarative language, called "dot," that represents nodes and the connections between them as text. The dot language can accommodate a large set of visual and logical attributes of many types of nodes, their relationships, and their interconnections. Nevertheless, it's intuitive, with an easy-to-remember and readable syntax. Here is an almost-minimal example of a dot file that defines a simple graph:

strict digraph "example" { A -> {B C}; D [shape = box]; C -> D; D -> C [color = blue]; }

The keyword strict at the beginning means that no redundant edges are allowed; a digraph is a directed graph (meaning that the edges have a direction, often represented by an arrowhead at the end and, perhaps, at the beginning). The second line says that node A is connected with both nodes B and C, in the direction starting from A. The next line declares a new node called D and defines an attribute that specifies how D should be drawn. Then, we declare that C is connected to D, and that D is connected back to C. This last edge has an attribute specifying its color.

The Graphviz infrastructure also comes with several layout engines that interpret dot files and produce the actual graphs. Some of the engines are for directed graphs, some are for undirected graphs, and some handle both types. The problem of taking a graph specification— with perhaps thousands of nodes—and producing a usable visual representation is not trivial, and is the subject of continuing research. Each of Graphviz's engines has a mathematical theory [PDF] behind it, and each will generate a different type of graph.

For simple directed graphs such as the one represented in the dot file above, the engine called "dot" is usually best. We invoke it on the command line:

dot -o simpledot.png -Tpng simpledot.dot

This generates a PNG output file (one of many choices), using simpledot.dot as the graph specification. If we store the code snippet above into this file, we get the output shown here:

It's clear how the definitions of nodes and edges have been translated into a picture. If we apply a different layout engine to the same dot file, for example fdp:

fdp -o simpledot.png -Tpng simpledot.dot we get the same information, but depicted in a different style:

A brief summary of the various layout engines that come with the system is provided in the dot man page. The dot engine produces a simple, hierarchical layout, whereas fdp, sfdp, and neato all treat the edges as springs. That is, they attempt to arrive at a neat arrangement by starting with a random layout and allowing the system to relax to a minimum energy configuration. The different engines will produce distinct results, as they are all based on different algorithms.

The use of a language to define graphs means that Graphviz can serve as the graphical engine for other systems or programs; they merely need to format their output in the dot language. There are many examples of this. Snakefood is a program that analyzes Python programs and determines their dependencies. You can point it at a directory of Python files and it will return the interdependencies among the files and to external modules, in its own format, which is a collection of Python tuples.

This output can be piped to a Snakefood utility that translates it into dot, which can then be processed by any of the Graphviz engines that can handle directed graphs—usually the dot engine is the best choice. Here is the result of applying the process to a directory holding the files for the bsddb package, an interface to the Berkeley DB library:

The dot file corresponding to this graph is available for further exploration.

You can also use Graphviz without using the dot language directly, by using one of its programming language interfaces. For example, the pygraphviz library for Python allows you to define a graph object, add nodes and edges to it, and create a graph image by calling the draw() method on the object. The Graphviz layout engine is selected with an argument passed to the draw() method.

We've barely scratched the surface of what Graphviz can do. Facilities for subgraphs, record nodes, adding labels in HTML, and more make this a general-purpose powerhouse for any type of automated graph creation. Graphviz is free software (as is Snakefood) and should be available through your package manager.

Stay tuned

In part 2 of this article we'll continue our survey of network graphing tools by looking at two sophisticated libraries. One is used within the LaTeX document-processing system to allow you to embed diagrams into your books and papers. The other is used with Python to allow you to explore the properties of general networks and plot the results.

(Log in to post comments)

Plotting tools for networks, part I Posted Apr 15, 2015 22:36 UTC (Wed) by [email protected] (subscriber, #100408) [Link]

When creating network diagrams manually, I find using yEd very useful, convenient, and easy to use. The tons of autolayout options are really nice.

While not as pretty as yEd generated graphs, using the Navy CORE network simulator for linux to create a network diagram for a IP network is pretty neat as well, and lets you actually simulate your network afterward :- )

While it is not really suitable for network diagrams, the freemind tool allows a certain class of information association.

Plotting tools for networks, part I Posted Apr 16, 2015 0:00 UTC (Thu) by jlargentaye (subscriber, #75206) [Link]

After dicking around with GraphViz in college, I finally realized its power (and the power of the Dot language) when I discovered the "gvpr" tool, which is basically AWK for graphs.

If Graphviz's layouters don't do what you want, you can try writing a GVPR script to do so. The really powerful thing is that you can process your initial Dot source via a pipeline of GVPR (or other!) scripts that augment the layout (by adding to dot source), then pass that into a layouter which will preserve your parameters. Check "man gvpr".

Plotting tools for networks, part I Posted Apr 16, 2015 0:30 UTC (Thu) by tonyblackwell (subscriber, #43641) [Link]

Pity yEd is proprietary. Despite this, worth downloading (currently free, think they make their money from back- end tools) just to see the functionality. Takes columns of objects with their overlapping connectors, then their e.g. 'organic' layout transform gives beautiful diagrams. Allows grouping. There must be some elegant programming gone into this tool.(No, I've nothing to do with them - found it when my wife wanted better summaries of our affairs!)

Plotting tools for networks, part I Posted Apr 15, 2015 23:09 UTC (Wed) by tbielawa (guest, #72509) [Link]

Graphviz is by far one of my favorite tools for generating graphs. I think that the two graphs I'm most proud of are:

1. A graph showing the XMPP (Jabber) state transitions during the client connections process. It's quite hefty and utilizes some of the extra features described in this article (such as subgraphs). PAD-XMPP: Graphs [PNG]

2. A graph of class inheritance in a Python project I maintain. The graph data is automatically generated in two phases. First by scanning the source tree and building up references, then parsing those references and constructing a proper dot-language file. Taboot: Graphs [PNG]

What makes #2 so fun (I'm a geek, graphs are fun, ok?) to me is that each time you run the make_graph script sub-classes receive a random color to help illustrate where major changes from parent classes are introduced. You could run it several times and end up with varying colored charts. Same structure, just different node colors.

Plotting tools for networks, part I Posted Apr 17, 2015 9:13 UTC (Fri) by lacos (guest, #70616) [Link]

The state transitions diagram looks awesome. Impressive.

Plotting tools for networks, part I Posted Apr 15, 2015 23:12 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link] also take a look at blockdiag.com It's python and similar to Graphviz, and has some sub-tools for specialized items sequence diagrams action diagrams network diagrams rack layouts packet/memory diagrams

Distributed algorithms Posted Apr 15, 2015 23:27 UTC (Wed) by yhvh (subscriber, #99655) [Link]

Are there any tools that can visualize distributed algorithms easily, with effects like messages animating along edges?

Distributed algorithms Posted Apr 15, 2015 23:58 UTC (Wed) by gerdesj (subscriber, #5446) [Link]

>>Are there any tools that can visualize distributed algorithms easily, with effects like messages animating along edges?

Yes

Cheers Jon

Distributed algorithms Posted Apr 16, 2015 0:04 UTC (Thu) by gerdesj (subscriber, #5446) [Link]

>>Are there any tools that can visualize distributed algorithms easily, with effects like messages animating along edges?

No

Cheers Jon

Distributed algorithms Posted Apr 16, 2015 0:05 UTC (Thu) by gerdesj (subscriber, #5446) [Link]

>> Are there any tools that can visualize distributed algorithms easily, with effects like messages animating along edges?

Details?

Cheers Jon

Distributed algorithms Posted Apr 16, 2015 1:09 UTC (Thu) by yhvh (subscriber, #99655) [Link]

I'm implementing distributed algorithms for a final year project, like this one for example. These algorithms operate by passing messages between nodes in a graph. I'd like to show how messages propagate through the graph as the algorithm is running, since it's not always obvious from the resulting (or intermediate) node state how it all happened, (which node effected this change?).

I'd like the messages to appear as some shape (say a circle), travelling along the edge between the nodes at various timestamps, from information gleaned from logs on the various nodes (simulated or otherwise).

Thanks, Will

Distributed algorithms Posted Apr 18, 2015 5:25 UTC (Sat) by wmfa (subscriber, #37197) [Link]

Just an idea: instantiate a black-and-white dot graph, then render individual frames using color for your messages. For example, fill in a node while the message is in a node being processed, and color the edge while it is in transit to the next node. Multiple colors might help elucidate some aspect of your algorithm.

The rendered frames can then be stacked into an animated GIF using ImageMagick. example: https://www.tjhsst.edu/~dhyatt/supercomp/n401a.html

Distributed algorithms Posted Apr 20, 2015 15:24 UTC (Mon) by cry_regarder (subscriber, #50545) [Link]

I think you might want to search for "petri net visualization open source".

Maybe https://github.com/zamzam/PetriNetSim might do the trick for you?

Cry

Plotting tools for networks, part I Posted Apr 16, 2015 0:16 UTC (Thu) by pflugstad (subscriber, #224) [Link]

Strongly recommend you look at Gephi as well. I've been using it on some flow graphs lately and it works very well.

On that note, does anyone know of a graphing tool that does good 3D? I.e. that lets you rotate the resulting graphic on all axis (Gephi can sort-of do this, but doesn't let you rotate the view very well)?

Plotting tools for networks, part I Posted Apr 16, 2015 0:27 UTC (Thu) by gerdesj (subscriber, #5446) [Link]

>> Plotting tools for networks, part I ... In the first two installments in this series ...

RLY?

Dia, GaphViz n dot border on genius. None of them are particularly pretty but each one is capable of delighting the studious user and generating sheer beauty. They all take a bit of work though.

Dia has a GUI and is great for a quick err diagram. It's not quite as good as Visio in some ways but it is free.

GV and dot are excellent. Simply point your language of choice at them and they deliver in absolute spades.

Plotting tools for networks, part I Posted Apr 22, 2015 9:14 UTC (Wed) by paulj (subscriber, #341) [Link]

+1 on Dia being a great tool.

It has (had?) some rough edges. Stay away from those and it's a powerful little tool for doing diagrammes (with a much simpler UI than Inkscape, because of that focus).

Plotting tools for networks, part I Posted Apr 16, 2015 9:08 UTC (Thu) by wodny (subscriber, #73045) [Link] xdot is a useful tool letting the user explore a .dot graph like a vector image, so there is no need to manually re- render a bitmap. It has zooming, panning and automatic refresh. It highlights the tail vertex of the selected edge.

Plotting tools for networks, part I Posted Apr 16, 2015 16:34 UTC (Thu) by tbielawa (guest, #72509) [Link]

I had never heard of this xdot before (it's python-xdot in the fedora repos if anyone is interested). That's really really cool! The zooming is really smooth.

Plotting tools for networks, part I Posted Apr 16, 2015 9:25 UTC (Thu) by robinst (subscriber, #61173) [Link]

Another very good library for programmatically visualizing things is d3js.

See examples (warning, big page): https://github.com/mbostock/d3/wiki/Gallery

Plotting tools for networks, part I Posted Apr 16, 2015 15:00 UTC (Thu) by dashesy (subscriber, #74652) [Link]

I recently used darw.io and I enjoyed its simplicity and integration with Google Drive, software should take on new ways I guess.

Plotting tools for networks, part I Posted Apr 20, 2015 15:54 UTC (Mon) by cry_regarder (subscriber, #50545) [Link]

not open source

Plotting tools for networks, part I Posted Apr 20, 2015 16:27 UTC (Mon) by dashesy (subscriber, #74652) [Link]

No, my point was that new paradigm is the web, open source can find a new home there, and maybe even make some money through social engagement like Pateron. I have used Dia before, really neat but desktops will soon be developers-only, end users will be on the web, it is a shame many users will never see it, nor even Inkscape.

Plotting tools for networks, part I Posted Apr 16, 2015 22:27 UTC (Thu) by jnareb (subscriber, #46500) [Link]

Very nice tool that makes use of GraphViz is gprof2dot.py to visualize call graph (not only from gprof, but also e.g. callgrind and other profilers). Yet another is description-driven PlantUML tool.

I guess that by "one is used within the LaTeX document-processing system" you mean PGF/TikZ (take a look at http://www.texample.net/), isn't it? Or was it rather specialized FeynMF package?

Plotting tools for networks, part I Posted Apr 16, 2015 22:43 UTC (Thu) by boog (subscriber, #30882) [Link]

Or maybe Asymptote

http://asymptote.sourceforge.net/ http://asymptote.sourceforge.net/gallery/

which includes a "flowchart" module.

Plotting tools for networks, part I Posted Apr 16, 2015 23:28 UTC (Thu) by leephillips (subscriber, #100450) [Link]

Asymptote seems awesome, although I've never used it. It's not really used within LaTeX, though, is it, even thought it uses LaTeX for typesetting its text and labels?

Plotting tools for networks, part I Posted Apr 18, 2015 20:33 UTC (Sat) by boog (subscriber, #30882) [Link]

You can embed asymptote code in a LaTeX document, in which case it automatically picks up some of the attributes of the document, which is really nice for visual consistency. However, most people probably use it in a stand alone manner (often then including the figures produced into a LaTeX document).

Plotting tools for networks, part I Posted Apr 17, 2015 11:22 UTC (Fri) by zenk (guest, #99503) [Link]

The dot file translated from callgrind file by gprof2dot has wrong cost percentage (at least when I used it). So I wrote my callgrind2dot (https://github.com/zenkj/callgrind2dot) using lua, which is much faster than gprof2dot and works fine for me.

Count me among the Graphviz fans Posted Apr 17, 2015 4:11 UTC (Fri) by gwolf (subscriber, #14632) [Link]

I am in the final stages of editting a textbook on operating systems (in Spanish, available at http://sistop.org/sistemas_operativos.pdf ). I used three tools for practically all of my diagrams (for which you can see the sources at https://github.com/gwolf/sistop/tree/master/fig ):

• Graphviz • Ditaa • Gnuplot

I achieved some interesting medium-level wizardry, mostly with Graphviz. I have taken on to using Graphviz to many different tasks — Currently I'm working on using it to model a ~1000-node trust network evolving through time. Very good results so far.

Plotting tools for networks, part I Posted Apr 17, 2015 16:24 UTC (Fri) by karkhaz (subscriber, #99844) [Link]

Another extremely useful tool that spits out dot graphs: [1] shows the dependency graph of a makefile. I find it invaluble when initially trying to comprehend large makefiles written by other people (or even myself).

Also, a +viewer+ for dot files. Normally one would run dot -Tpng or Twhatever to get an image, but zgrviewer [2] reads in a dot file directly and allows you to zoom, pan etc. It has some very useful features for huge graphs where the edges are all mixed up, for example you can `follow' an edge around a graph without getting lost.

[1] https://github.com/lindenb/makefile2graph [2] http://zvtm.sourceforge.net/zgrviewer.html

Plotting tools for networks, part I Posted Apr 17, 2015 17:42 UTC (Fri) by leephillips (subscriber, #100450) [Link]

ZGRviewer looks amazing, although early in development. A tool to graph the dependencies in a Makefile would be genuinely useful, thanks for mentioning it.

Plotting tools for networks, part I Posted Apr 23, 2015 9:03 UTC (Thu) by ssokolow (guest, #94568) [Link]

Some other good tools in a similar vein are: erd (A graphviz-based tool for generating entity resource diagrams from textual descriptions) https://github.com/BurntSushi/erd mscgen (A graphviz-inspired tool for generating message sequence charts from textual descriptions) http://www.mcternan.me.uk/mscgen/

Copyright © 2015, Eklektix, Inc. Comments and public postings are copyrighted by their creators. Linux is a registered trademark of Linus Torvalds