The Uprising of Data Driven Documents

Many developers have the feeling, that with D3, Mike Bostock’s concept for Data Driven Documents, a new area of data visualization for developers has begun. At Sclable we have already been using D3’s predecessor protovis years ago, and are following the development of D3.js – Bostock’s javascript library for manipulating documents based on data – very closely.

 

Data Visualization – History Repeating

As with many approaches in current technological innovations, the D3 community wanted to go back to the roots of data visualization. While William Playfair’s “Statistical Breviary” from 1801 is considered to be the milestone for statistical charts, Florence Nightingale’s “A contribution to the sanitary history of the British army during the late war with Russia” from 1855 and her famous “polar area diagram” (also “rose charts”, “nightingale rose” or “coxcombs”) is where data visualization began.

Data Visualization. Example of polar area diagram by Florence Nightingale (1820–1910). This "Diagram of the causes of mortality in the army in the East" was published in Notes on Matters Affecting the Health, Efficiency, and Hospital Administration of the British Army and sent to Queen Victoria in 1858. This graphic indicates the number of deaths that occurred from preventable diseases (in blue), those that were the results of wounds (in red), and those due to other causes (in black). The legend reads: The Areas of the blue, red, & black wedges are each measured from the centre as the common vertex. The blue wedges measured from the centre of the circle represent area for area the deaths from Preventable or Mitigable Zymotic diseases, the red wedges measured from the centre the deaths from wounds, & the black wedges measured from the centre the deaths from all other causes. The black line across the red triangle in Nov. 1854 marks the boundary of the deaths from all other causes during the month. In October 1854, & April 1855, the black area coincides with the red, in January & February 1856, the blue coincides with the black. The entire areas may be compared by following the blue, the red, & the black lines enclosing them. Image: Florence Nightingale, CC

Example of polar area diagram by Florence Nightingale (1820–1910).
This “Diagram of the causes of mortality in the army in the East” was published in Notes on Matters Affecting the Health, Efficiency, and Hospital Administration of the British Army and sent to Queen Victoria in 1858. This graphic indicates the number of deaths that occurred from preventable diseases (in blue), those that were the results of wounds (in red), and those due to other causes (in black).
Image: Florence Nightingale, CC

Touch the Data

Just like in those days, today’s data visualizers come from interdisciplinary contexts. One of the vanguards of contemporary data visualization techniques is Martin Krzywinski, who developed a number of amazing visualization for the Human Genome Project and developed CIRCOS. With the advance of better web browsers and techniques such as two-way data-binding, a collective dream of every information aesthete is about to become true: Data visualizations with rich interactions and animations you can touch.

Data visualization. Have a look at interactive version Nightingale’s famous polar area diagram built upon Bostock’s D3.js from Frisco based developer Athan Kgryte. Athan did this D3 implementation as a rework of Mike Bostock’s very first implementation of the polar area chart based on protovis based on a quite thorough analysis dealing with the “mathematics of the coxcombs”. Image: Florence Nightingale, CC

Have a look at interactive version Nightingale’s famous polar area diagram built upon Bostock’s D3.js from Frisco based developer Athan Kgryte. Athan did this D3 implementation as a rework of Mike Bostock’s very first implementation of the polar area chart based on protovis based on a quite thorough analysis dealing with the “mathematics of the coxcombs”.
Image: Florence Nightingale, CC

Business Value in Data Driven Documents

Business Applications are striving for reports and control dashboards. Most concepts of reporting applications lack the fundamental research the Data Driven Documents community has been doing. As a result, Business Leaders are presented with visualizations they simply do not understand or are presented with a varying “Lie Factor” (see Tufte 1983). Today’s data visualization research tightens the binding of data and visualization: The graph presented should not be result of some computation logic elusive to the beholder. Rather the visualization should be a real representation of the nature of the data and any event the has an impact on this very nature must have an impact on any visualization bound to the data.

Data visualization. This graphic was originally published by the NY Times. It tries to show the mandated fuel economy standards for autos set by the US Department of Transportation. The standard required an increase in mileage from 18 to 27.5, an increase of 53%. The magnitude of increase shown in the graph is 783%, which results in a lie factor of 14.8! The “Lie Factor” is a value to describe the relation between the size of effect shown in a graphic and the size of effect shown in the data. Edward Tufte, Prof. at the Yale University, defined the “Lie Factor” in his book “The Visual Display of Quantitative Information” in 1983. Image: NY Times, CC, InfoVis-Wiki

This graphic was originally published by the NY Times. It tries to show the mandated fuel economy standards for autos set by the US Department of Transportation. The standard required an increase in mileage from 18 to 27.5, an increase of 53%. The magnitude of increase shown in the graph is 783%, which results in a lie factor of 14.8!
The “Lie Factor” is a value to describe the relation between the size of effect shown in a graphic and the size of effect shown in the data. Edward Tufte, Prof. at the Yale University, defined the “Lie Factor” in his book “The Visual Display of Quantitative Information” in 1983. Image: NY Times, CC, InfoVis-Wiki

Beyond Interactivity

Let me explain, what we mean by “any event” when it comes to Business Application Development with Sclable. Interactivity isn’t a bad thing for exploring data. Most interactions are limited to data filtering though. Let’s think of other events such as data being modified by a business application’s user or automations while you are viewing a visualized representation of it. Or imagine the impact of extremely volatile “viewing permissions” for users of a business application on reports or control dashboards.

 

Sclable Visualization Development Environment

At Sclable we are about to start a research and development project within the next few weeks with the prospective outcome of a concept along with a prototypical implementation for a complete Visualization Development Environment built upon the Sclable Platform. Our goal is to make it a breeze for developers to build rich and lie-factor free, accurate and pleasing visualisations. Just as fast you can build up your business domain model along with your application with the Sclable Platform.

 

New features for Sclable’s Core Engine

There are a number of ground breaking features we will implement into Sclable’s Core Engine during our R&D project. Just to name a few: Aggregates will let you bind aggregation computations to the domain model. A complete refactoring of Calculated Attributes and Calculation Functions using the event system, already built inside Sclable since the early beta of version 2.0, will give you the freedom of any computation detached from the model. Selections will get an overhaul as they will get used extensively with data visualizations.

 

A Glimpse on what is Upcoming

The upcoming release of the Sclable Platform’s Domain Designer component will feature some conceptual Ideas: While manipulation your business application’s domain model, rich visualizations are shown to ease getting the big picture on large and complex models. Model subsets such as workflow and relation graphs are written in D3 bound to the data of the Sclable meta model domain via angular’s two-way data-binding.

Data visualization. Sclable's Domain Designer Component from the curren version 2.0 release of the Sclable Platform

Sclable’s Domain Designer Component from the current version 2.0 release of the Sclable Platform features some of the technical stuff which will be the starting point of Sclable’s research and development project on “Data visualizations upon evolving domain models”. Image: Sclable

Outlook on more Insight

The world of Business Application Development is changing a lot right now. At Sclable, our prediction is that data visualization as a topic will become one of the next big things soon. We have been seeing the “Big Data” hype along with a lot of misconceptions and misunderstandings around it. What is needed urgently within the field of business application is insight. Sclable is on the way to give you the tools to create such insight.

LEARN MORE ABOUT DATA VISUALIZATION AT SCLABLE

Research Project: Data Visualization upon evolutionary developed Domain Models

Blogpost: 3 Top Business Data Visualization Challenges and how we adress them at Sclable