Showing posts with label advanced. Show all posts
Showing posts with label advanced. Show all posts

Thursday, January 7, 2016

Visualizing Big Data with d3.js

You're creating a dashboard or other visualization of Big Data in `d3.js`. You've heard that SVG-based visualizations can experience a performance hit as they render more and more elements.
Perhaps you've even hit the object threshold where you are starting to notice significant lags. Where to now? How do you build your visualization to ensure that it scales?


  • Simplify your data/dashboard by aggregating. Are all of the elements that you are rendering necessary? Make sure that each pixel counts and that an aggregate data view wouldn't more effectively convey your point. For example, Google Maps successfully manages to visualize a large quantity of data. However, they don't just throw all of their data at you -- depending on the zoom level that you choose, you will be presented with a certain amount of map detail. This allows for a better and more responsive visual experience than if they did not have aggregated data as part of their visualization. 
  • Simplify your data/dashboard by sampling. Try sampling a subset of your dataset and visualizing that -- that is often enough to perceive the general shape of your data. 
  • Try to move some of the rendering server-side. Normally, the bulk of rendering is done client-side with `d3.js`. However, you can generate an HTML document of part of your visualization, save it in your server's memory, and then inject it into your client's visualization, where the chunk that your server rendered appears together with the rest of the visualization that your client renders. For more details, check out a tutorial here.
  • Try a hybrid canvas/svg approach. This gives you some of the benefits of the SVG, while still avoiding the performance constraints of too many SVG objects.
  • Change tools. At some point, the performance hit of creating your DOM and manipulating will be significant enough that the other optimization techniques will not be sufficient. The weakest point in the `d3.js` stack is your browser -- change your rendering process by hooking up `d3.js` to Google BigQuery. You can check out this link for more info.


Do you have other tips and tricks that you've used to visualize big data with `d3.js`? Have another stack that you recommend? Tell us about it in the comments below!

Tuesday, January 5, 2016

To scroll or not to scroll -- changing the rules of the game for dashboard UX

I was rereading Few's Information Dashboard Design recently and was struck by one of his "13 key mistakes in dashboard design":

 Exceeding the Boundaries of a Single Screen
My insistence [is] that a dashboard should confine its display to a single screen, with no need for scrolling or switching between multiple screens...
-- Few, Information Dashboard Design, Ch. 3.1

First, what does this mean in a post-mobile revolution world? Many devices no longer have screens big enough to fit a traditional dashboard. Refusing to make online applications mobile compatible is simply no longer an option given the ubiquity of devices. What are we left with? We need to find a way to provide meaningful data for our dashboard viewers without necessitating side-by-side visual comparison. We need to use dynamic, flexible layouts such that when viewers are on a smaller device, their screen renders content in a way that best uses their screen real estate. Finally, we need to make sure that we are presenting data in ways that allow for meaningful comparisons to be made. Given the plethora of data manipulation libraries, we have no excuse not to slice, dice, and aggregate the underlying data to facilitate viewers' analysis.

What about the cases where viewers are using devices that do have sufficient screen real estate to make a meaningful one-page layout? It used to be the case that many well-designed web pages also followed this steadfast rule: keep your content to a single page to avoid the burden of having your user scroll down the page. Additionally, if we had too much content on one page, it took a long time for content to load which created a sub-par experience.

Now that we have the ability to widely make asynchronous requests (e.g. with Ajax), this is not as much of an issue. Users are also used to web applications that make use of infinite scrolling -- scrolling is a more natural part of the online user experience than it was even 5 years ago. To put it in the language that Donald Norman from The Design of Everyday Things uses: the nature of the browser environment's affordances have changed. Browser users have always been able to scroll through a web application. Nowadays, though, browser users are aware that they can scroll easily, and use this as a primary method to interact with web applications, whereas before it was a secondary method.

Does this mean that we should let our dashboards grow into multi-page mammoths? No. The point of a dashboard should still be to succinctly provide informative data. There is more justification, though, to think about extending the dashboard boundary beyond the edge of the page when it is necessary.


Sign up to receive data viz talk tips and updates via email. We're low traffic and take privacy very seriously.