Supporting businesses with data visualizations is undoubted one of the most crucial factors for detecting and understanding the challenges every business has to master to be successful. But data visualization involves a number of heterogenous skills within the scientific fields of economics, mathematics, geography, data analysis, programming and design. And it isn’t a one-off operation.

Most companies struggle when it comes to create simple graphical dashboards or visualizations for a standard executive report.

CHALLENGE 1: ASSURING DATA QUALITY

Business data quality often is quite poor. Data sources are missing constraints, or even correct data types. Interestingly enough, legacy systems are better than younger ones, because in the early days, precise data models were crucial for disk usage and performance.

For many data visualization professionals it is still shocking how much effort companies put into operations called “normalization”, “weighting”, “data cleansing”, “sanitization” instead of guarding the accumulation of quality data in the first place. Usually you can see more creativity in finding euphemisms for data manipulation and truncation than in dealing with the root of the problem.

Image: “The costs of poor data quality”, Anders Haug, Frederik Zachariassen, Dennis van Liempd University of Southern Denmark

Image: “The costs of poor data quality”, Anders Haug, Frederik Zachariassen, Dennis van Liempd
University of Southern Denmark

CHALLENGE 2: UNDERSTANDING & SHAPING DATA

The second challenge is tightly intertwined with the data quality issue. Bad data quality often goes hand in hand with insufficient structurization of the underlying data architecture. Poorly structured business domains are extremely difficult to understand and shape. Without thorough domain expertise, data wranglers won’t be able to provide correctly aggregated information for a visualization.

To be able to create graphical representations from your data, you need to take it to a higher level: The simplest form is to aggregate columns of tabular data, a task you can do with any spreadsheet application. Add more dimensions and you’ll quickly end up with concepts like online analytical processing (OLAP). Adding niftier aggregation functions than counts and averages will increase complexity.

Image: OLAP Data Cube From “A Brief Tutorial on Database Queries, Data Mining, and OLAP”, Lutz Hamel, Department of Computer Science and Statistics University of Rhode Island.

Image: OLAP Data Cube From “A Brief Tutorial on Database Queries, Data Mining, and OLAP”, Lutz Hamel, Department of Computer Science and Statistics University of Rhode Island.

Challenge 3: Displaying Meaningful Results

Getting valuable and meaningful insights from business data visualizations depends foremost on a deep understanding of the origins of the data, the aggregation procedures and the audience that will interpret the visualization. A common mistake is that visualizations are used without much of a context. Poor or absent context is exactly what renders data visualizations meaningless.

A data visualization example with poor context, from: “The Visual Display of Quantitative Information”, Edward Tufte 1983

A data visualization example with poor context, from: “The Visual Display of Quantitative Information”, Edward Tufte 1983

 

In general, most visualizations do not provide enough context. It is essential that a data graph is linked to the aggregation matrix. It would be ideal if the aggregation is linked to the aggregation function as well as the underlying data set. How many dashboards or reports provide such a deep linked context?

HOW WE ADDRESS THESE ISSUES AT SCLABLE

At Sclable we aim to combine the latest scientific findings, data wrangling methods, programming techniques and design approaches to get all the major obstacles out of the way of data visualization creators.

Data Quality & Integrity

The Sclable Platform’s application architecture guarantees the highest possible data quality and integrity on every architectural level. You can define your own data types to meet your exact needs, exploit uniqueness and requiredness concepts and create your own regular expression rules for data validation.

Data Consolidation and Aggregation

In Sclable you don’t just select and aggregate data. Instead you are modelling calculations, aggregations, selections and data filters. The way your business data is managed and how it can be analyzed is both part of your business domain model.

Never lose context

Since data accumulation as well as data aggregation and analysis is based on your domain model, you’ll never be off context with your data visualizations. Sclable allows you to create “Data Driven Documents” directly from your business domain. Your data visualizations are more than just a way to visualize information: Your graphs allow you to explore your business domain’s data interactively: you can feel your data behind the graph.

Feel the data behind the graph: A data visualization upon D3.js providing an aggregation matrix showing interactive small multiples along with a geo map, made with using dc.js,  Crossfilter, and Leaflet.

THE POWER OF THE DOMAIN MODEL

As with any other complex task in business application development, our goal at Sclable is to provide platform capabilities, tools and best practises that are making it easy for you to achieve your data visualization goals. You can develop and implement advanced algorithms if you want. But you don’t have to:

Having your model at hand, the possibilities of what eventually could be visualized meaningfully shrink drastically, since a lot of decisions and rules for your data are already described within the model. The approach we are taking is to pre-analyze the domain model your application is running on. This gives a lot of useful input for pre-selecting the right “recipe” for application developers. These recipes resamble the latest and greatest research in the fields of mathematics, data science, UX and UI engineering. Applying them should be a matter of minutes for developers.

 

Learn more about Data Visualization at Sclable

Research Project: Data Visualization upon evolutionary developed Domain Models

Blogpost: Domain Driven Data Visualization