I'm Shirley Wu, freelance data visualization engineer, and I create highly interactive data visualizations with D3.js and React.js, and fun pop-culture visualizations as the other half of data sketch|es. Ask me anything about dataviz, D3.js+React.js, freelancing, and more!

Abstract

Hi there! My name is Shirley, and I love creating highly interactive visualizations. My most successful thus far is An Interactive Visualization of Hamilton, which was tweeted about by Lin-Manuel Miranda himself 😍. I'm the other half of data sketches along with my great friend Nadieh Bremer, and we visualize a wide variety of topics - movies, the Olympics, music, culture, community - and write about our whole process from start to finish. It's got some great in-progress screenshots and wacky canvas bugs, so make sure to check out our write-ups!

Before I took the plunge to freelance, I architected complex visual tools for a security company as a software engineer, and I wrote about my experiences using D3.js with React.js there. I've since taken those and many more lessons, and compiled them into a comprehensive talk about React.js+D3.js best practices. I'm excited about seeing more sophisticated visualizations built for the web, about coupling dataviz best practices with UI best practices, and about further exploring the potential of highly interactive data visualizations.

In the past year of doing data sketches, I've also realized how important it is to make creating data visualization more accessible. On top of our detailed process write-ups, I taught an Introduction to D3.js workshop on Frontend Masters, and I've started live-coding and interviewing others in dataviz+tech on Twitch (past episodes archived on my Youtube). I hope to show that building data visualizations isn't hard, but just takes lots of practice, patience, and perseverance.

Finally, I quit my job cold turkey last May and started freelancing. It's been a hard, stressful, rewarding, amazing year and a half full of ups and downs (and some travel) but I've learned a lot about myself and I'm hoping to continue for as long as I can.

Ask me anything about the value of fun data visualizations, storytelling, D3.js in the bay, D3.js+React.js, live-streaming on Twitch, freelancing, and my thoughts on the Elijah "dataviz is dead" topic - I'll be back at noon PST to answer all your questions!

Update 11:22AM PST: here's proof that it's actually me (though idk why anyone would wanna pretend to be me...). Thank you for such great questions so far please keep them coming~ I can't wait to start answering in exactly 38 minutes!

Update 12:00PM PST: getting started on them questions while munching on some delicious lunch :D

Update 4:09PM PST: wow I am a tremendously slow answerer. I'm going to go take a break, flounce around my apartment, stretch out my back, roll around in bed. I'll be back in an hour-ish, thank you for all the questions so far!

Update 6:30PM PST: by 1hr I totally meant 2.5hrs πŸ˜… I'm back to (hopefully) finish answering all your questions (:

Update 12:00AM PST: wow how poetic, I just finished answering all the questions after 12 hours (I took lots of breaks haha). Thank you for a great day of questions!

What is one thing you think everyone should spend more time on when creating a data visualization?

ostedog

Digging into the data.

I used to be really guilty of not spending much time with my data and jumping straight to the visualization, especially because I usually use APIs and automated scripts to get relatively large datasets. I think the visualizations I made were alright, but they lacked the little insights and details that really draw in an audience. They were more broad-stroke sweeping statements and that's only interesting on a very shallow level.

This past year I've had the pleasure of getting a really deep look into Nadieh's process, and how much time she spends cleaning her data and going through it in R and really trying to comb through it for interesting trends and statements she can make. She builds her visualizations around those, and I think those very personal touches, annotations, and details really draw in the audience and make them stay for longer. I also attended a great talk by Mimi Onuoha at Eyeo about the importance of data collection, and that's really changed my outlook on data gathering and visualization also.

I think digging into data is really important and probably pretty obvious when we have a specific dataset, but I think it's just as important when we're building for product and working with constantly changing data. Even if they're constantly changing, the shape of the data should always be similar; we should dig through whatever datasets we can get our hands on, and see if we can find patterns and edge cases. I think that's one of the things I really wish I had been better at when I was at my previous job, because I'd probably been able to help build much more interesting visualizations.


Hi Shirley! I'm a huge fan of your work, and really admire you as a role model, especially as a student who hopes to get into similar work. I have a few questions:

  1. Can you speak to your process when creating something like the 'Interactive Visualization of Hamilton'? Did you gather the data yourself? Do you use pen and paper for initial ideas? Do you ever use Sketch or Illustrator (or something like that) to validate your ideas? How do you know which visualization types you want to use?

  2. What do you think the future of data visualization is? Whats the purpose of your work (not to sound cynical at all, I'd just love to hear what you have to say about this!)

  3. What types of projects do you work on as a freelancer? Is there a general intent that companies have with wanting data viz stories? (If there is a link of past projects I'll snoop around!).

ratt03

Thank you so so much! That is incredibly kind, and it makes me happy every time I hear from a student that they've gotten interested in dataviz (:

  1. I actually have the whole process written up over on data sketches if you want the detailed account, and my code is also all open source (though idk if you wanna dig through all that lol). But yes, I do go get my own data, either online via scripts, or manual entry. For Hamilton in particular, I switched back and forth between sketching with pen and paper to think through my design and actually coding with the data. The dataset was on the bigger side, so it was really helpful to see it all on the screen as I was iterating through my design. And as for how do I know which visualization types...it really depends on the data. Sometimes I know right away (very rarely) and most of the times I'm just experimenting and iterating as I explore the data more; I'll think of something that I think might show the data in a good way, and it won't work out, so I'll try something else. But at every single one of those iterations, I learn something new/interesting about the data that I can then use in my final piece.

  2. I think that data visualization will continue to be important as companies accrue more and more data that they need to get heads or tails out of, but I think we're getting to a point where the field is contracting. What I mean by that is, companies are starting to hone in that what they want aren't fancy visualizations for the sake of fancy, but rather on visualizations that fit their use case (though I think we're still far from companies actually really knowing what to do with their data and how to visualize them). This isn't based on any empirical data though, just my gut feelings and observations haha. But for me personally, I want to make fun visualizations that inspire; I want more of the general public - especially the younger audiences - to know what visualizations can look like, and get more and more used to reading them. That's more and more the goal of my work.

  3. Here's [my portfolio](sxywu.com), but I work on a variety of projects. Sometimes it's as straightforward as they already have a mock-up and I just have to implement, but oftentimes they come with a specific dataset that they want to visualize and communicate a purpose. I do wish I can work with more companies on visualizations within their product, but that's difficult to outsource to a freelancer that won't be around for the long-term.


What are your favourite techniques for retaining information richness and sparking self-exploration, without overwhelming or scaring off the audience?

nllho

What a great question! I think I'm far from great at this, but I'll try my best to explain my thoughts on it.

First, I love scrollytelling (and thus I love the brilliance that is pudding.cool) and that's because in this age of internet and information overload, the idea of slowly peeling apart the data and revealing all the interesting parts of it and the only thing the user has to do is scroll - that's genius. It means even if the information is dense and rich, they can go at their own pace, and they can explore at their own pace.

Having said that, I'm also a big believer of letting the audience get out of a visualization what they want. I try to put layers in my work; if all a reader wants is to come in and scroll through and see pretty colors pass them by on their screen - that's great because at least then they got exposed to a visualization that they otherwise wouldn't have. If they just want to read my analysis and what I've found and bounce, that's also great. But if they want to then use the exploratory tool that I built to do my analysis and go and do their own analysis, that's absolutely superb. But I'm happy at any spectrum of that, and I try to make it easy to enjoy at any of those levels (though I'm definitely not always successful).


Why do you think so much data visualization is just really terrible charts that shows interesting data? Do you think the work we do when we make fancy data visualization is just "dressing up the data" and really what's important is interesting data or do you think something else leads to these highly upvoted, incredibly popular but terribly ugly data visualizations of important data?

elijahmeeks

I think that's a really interesting question, and I wonder if it stems from the fact that data visualization takes such a broad spectrum of skill sets. To be able to find interesting data, be able to recognize that it's interesting data, and to accurately pull out the parts that are interesting, that takes very specific training (data cleaning+analysis+statistics?). But to be able to then take that interesting data and "dress it up", that takes design (HCI/UI/UX) training, not to mention programming background if we don't want to use pre-built software. That's a lot to ask for, especially if all a person wants to do is communicate an interesting dataset they found. I've never met someone that was perfect at executing every single of those steps.

Having said that (and maybe I'm idealistic), I think both interesting data and a great visualization is important. A fancy visualization without a great dataset is just a fancy visualization for the sake of being fancy. An interesting dataset without the appropriate visualization falls short of communicating everything that makes it interesting.

But we can't possibly ask every single person to be perfect at every single one of those skills; perhaps what we should work towards is easier access to people with skills we don't have. But that's just something lofty I'm throwing out there with no idea of execution πŸ‘


Any books or online classes you'd recommend for an intermediate user of D3.js that wants to take his skills to the next level?

Julien_Assouline

I've heard great things about Elijah's D3.js in Action though I've never read it. I think Scott's 2nd Edition of Interactive Data Visualization also goes into a lot more detail than his previous edition did.

Having said that, I think if you're already intermediate - and I'd say that's if you intimately intuitively understand how selections work underneath the hood and can pick up most of the other parts of the library by reading the documentation - then the best way to take your skills to the next level is to just build. Find something you're curious about, go out and get the dataset, and build out a visualization with D3.js. Or take an existing dataset and try to rebuild the visualization from scratch, using the documentation and bl.ocks as reference (though never copying). You'll learn so much more doing that than with any class (the goal is understanding) AND you'll have something to show for it after. But I do caution you to start small, with very attainable goals, and slowly build up to bigger things.

I think the thing that really took my skills to the next level was actually understanding SVG and how SVG paths work. By understanding the underlying technologies, it really helped me understand why D3.js is organized and architected the way it is, and how and when to best use it.


I visualize data at work. I went to a few D3 meetups in the bay but I didn't feel motivated to try to become part of the seemingly tight-knit community that ya'll have created. It seems part of freelancing is "branding yourself" but I have no interest in that. How do you manage doing visualization all day for work but then also spending your free time socializing visualization irl and online?

nthitz

I've actually never thought of it that way! I'm definitely one of those people that has to do what they love for work; I actually did an internship one summer at a finance/investment bank, and the work definitely didn't gel with me. I went through this dramatic negative cycle where I got more and more depressed because I wasn't enjoying my work and wasn't putting in my full effort, and because I wasn't putting in my full effort, I couldn't be proud of my work, and that made me even more depressed πŸ˜… so I vowed to find something I really love and can be excited about working on, in and out of work.

So that's probably why I love the D3 community I've gotten to know, because I get to geek out with them about the thing I love doing. I suppose it's kind of like I'd get just as excited if I made friends that like the same Studio Ghibli movies I do, kpop songs, Harry Potter movies, anime/manga...it's just one of my interests that I love bonding with other people about, that just also so happens to pay me.

Having said that, I do get burnt out sometimes (it is work afterall), but even then I get burnt out by the coding and the creating and I still love talking about it. I think it's just all about priorities and personalities and you shouldn't feel the need to force yourself to make friends or join a community; only do it if you find that you're enjoying yourself :D


Only use 3D graphs or never use color ever again?

ostedog

Never use color ever again. There's some gorgeous black and white visualizations from before color was widely available, and I'd find it to be an interesting and creative challenge to create visualizations with no color. It'll allow me to explore different textures, and also to really consider the core of what I'm trying to convey (the black and white visualizations I've seen were always elegantly simple and straightforward).

If you're interested in black and white or more historical visualizations, definitely follow RJ (@infowetrust) he has a great piece on most iconic visualizations of history and just got his hands on a really great book on business visualizations (I think).


What is the first question you ask yourself (about the project/data/etc) when starting a new visualization?

samhinshaw

What am I trying to learn from this dataset? What do I hope to communicate, and who am I communicating to?

(I realize that's 3 questions hahaha).


What do you think about Tableau and their data visualization software?

reverse_lrigwoc

I can't make much comment since I've never used it! I think it's like any great tool, it really depends on how it's used; I'm sure it gives great power and insight if the person using it know their end goal (what questions do they want to ask the data, what are they trying to learn, etc.), but I'm sure it's just as easy to abuse and funnel in data that doesn't make sense with the visualization, etc. My worst fear is if a company has Tableau or any other similar software but doesn't use it properly and get jaded about data visualization in general and write it off from their future altogether.


What is the best, and worst, thing about being a data visualization freelancer?

ostedog

Thank you I really like this question (:

The best definitely has to be the range of projects I get to work on. My previous job, I was working on the same project for 2.5 years, and I really loved that project, but towards the end I really wanted variety. I've been able to get that, and even more importantly, I've gotten to talk to a lot of different people and companies and learn their motivations and challenges and philosophies. That's really important to me, that I can learn from a breadth of people, and it's really informed my own world views and priorities and what I want to do with my life in the short-erm. It's also opened up my eyes to possibilities - things I can do with my skill sets - that I previously would have never even dreamed of.

The worst thing for me is the loneliness. I'm quite introverted so I don't need to be around people 24/7, but when I'm at home days on end and the only person I've seen face to face is my boyfriend (no matter how cute his face is)...sometimes I just really want to be able to turn around and talk to someone and bounce ideas. I really miss that about being in an office (though not much else tbh).


What are your thoughts on the live-coding and interviews on Twitch? What have you learned from either of the experiences? Any tips for people that would like to start using live streams?

susielu

Thank you Susie πŸ’–

It's been SUPER fun so far, and I'm really enjoying it. I've even made a few friends with the people that come back regularly (: For the live-coding, I know I'm good with D3.js and SVG and such, but there's a lot of other parts that I lack - things like ES2016 (or whatever the latest is now...ES7? I'm so confused) and Promises and I learn a lot from the people that join. I've also gotten help debugging problems that I wouldn't have been able to figure out for hours. And it's also just really fun to talk to people about what I'm working on and how their day was, etc.

And as for interviews, I think all the people I'm lucky to be friends with and get to interview have such great work and thoughts/philosophies and outlooks on life. I love learning from them (including you!) and that in turn forces me to synthesize what I learn and internalize it into my own thoughts and outlooks (there's a theory about creativity that I've been mulling over through past #shirleychats). I also really find it fun seeing what kind of questions people ask during the interviews.

As for tips, I'm still too early to give any really good ones...but I think having the face cam helps people see my expressions and connect with what I'm saying. I read that I should be talking the whole time, so I try to talk through my whole thought process and once in a while I step back and talk about my motivations about why I'm doing something a certain way. I think sometimes it gets really draining (I'm not used to talking for 2 hours straight hahaha) but I try to make it fun and have fun with it.

Oh and of course a gif of a cute cat catching a heart is very necessary for every time someone follows me πŸ‘


What's your favorite charting technique or coding trick that you've learned since last May?

1wheel

HI Adam πŸ‘‹ thank you for the question!

And of course it's GOOEY BAR CHARTS lol jkjk. The thing that I've learned in the past year that's had the biggest impact on my work is Greensock. I took Sarah Drasner and Val Head's two-day workshop on web animations and it's hands down the best career investment I've made recently. Prior to the workshop, animations were an afterthought, not because I didn't find them important, but rather I just didn't know what to do with them. But their workshop is great at both teaching us the practical of how to implement with specific animation libraries, and also the theory behind animations and how to best use them.

That really opened up my mind to the possibilities of what we can do with animations in data visualization beyond just transitioning from one state to another. I'm experimenting more and more with how to use animation in storytelling, how to use it to dramatically reveal points, use it to teach how to read a graph, etc. I started experimenting with the Google project and I'm working on something now with a great dataset and D3+Greensock that I'm really excited to share in the coming months (:


Hey Shirley, thanks for everything you do! Watching your live coding is worth so much when learning--getting to see your thought process, troubleshooting, etc, and not just a perfectly polished tutorial.

What is one principle of viz theory that you wished everyone consuming data visualizations understood?

samhinshaw

Thank you so much for joining my live-streams - please give me a shoutout next time you join!

Not as much about visualization theory, but I think it's important for consumers to understand how human data and data visualizations can be. Data tends to get portrayed as this perfect, robotic source of truth, that whatever the data says that must be the right conclusion. But one of the most important things I learned at Mimi's Eyeo talk is how imperfect data collection is, how systemic bias seeps into the collection process (for example, surveys of poorer neighborhoods aren't taken because they're harder to access or the survey takers lack the resources to do so, or more disadvantaged groups aren't taken into account when collecting datasets of the "average" population, etc.). And even if we're using machine generated data (user interactions with Facebook or a shopping site, etc.) that takes out the bias of a human collector of data, the person doing the visualization has their own biases. We all grow up in a society and none of us are immune to prejudice, so no matter how hard we try to be unbiased, it'll still subconsciously seep into our work.

So I really wish that when a person sees a datapoint or a data visualization, that they don't take whatever claims it's making for granted and really dig in for themselves.


I’m favoriting this sub to read later. I’m a data programmer who really wants to improve my visualization game. I’m already impressed with some of your work. What would be your recommendations for this area as someone starting from scratch (from the visual perspective ).

greatduelist

From a visual perspective, I'd suggest really exposing yourself to all the visualization work that's out there and internalize what works and what doesn't about them. I love going through the Information is Beautiful Awards, I subscribe to Flowing Data and Andy Kirk's RSS feeds (I especially love Andy's Little of Visualization Design that points out subtle details of a visualization that helps make it clearer/more readable, etc.), I follow Jan Willem Tulp, Nadieh, Studio Terp's Pinterest boards...

The reason why I suggest all these is bc if you already program, it's probably not that hard to pick up D3.js or some equivalent to build your visualization. The part that you need, then, is the design part, and to get good at design, you just need to look at lots of examples and being able to pick out the good parts and being able to use them later yourself in the right situations.


How does your design and development process change (if at all) when the data source is static vs dynamic (API driven)?

steveg3003

Mmmm I find it a lot harder to design and code for dynamic content. When I have a static data set, I can really dig into it and I know that whatever interesting thing I find I can design around it and it'll never change. It also makes my coding easier since I can cheat and use magic numbers and absolutely position things like annotations because I know the positions of my data will never change (though I'll have to adjust for mobile).

But when it comes to dynamic data sources, I know that something I find may or may not always persist. I know that the layout from one set to another can dramatically change, so I have to write extra code that can be flexible (not that that's bad, oftentimes it's quite fun, but it is a lot more work and brain usage). And perhaps worst of all, I know I can never catch all the edge cases before I push it out to production because I can't possibly predict all the ways that the data can change.

In that sense, I tend to think of static datasets to be a great visualization challenge (how do I best highlight this dataset) whereas dynamic datasets are more of an architectural/engineering challenge (how can I best write my code to cover all my cases). In that sense, dynamic datasets probably also take more iteration and more maintenance and are more common within product or as an internal tool.


Ahh, I saw your tweet shared before by LMM and it inspired me to look at more visualizations you made. I also really like that you have the books part of your page/the videos because sometimes it shows how making one visualizations isn't a such a simple process, and it's really nice to see how people organize and set them up.

You mentioned UI best practices. What would be a good way to start learning more? (or just in general, lol.) I entered data viz as a scientist, and our focus was on presenting the data in a way that made it the clearest, but never focused so much on UI of it.

sarahbotts

Yes! I'm so glad, thank you so much for checking out my work (: By books/videos...do you mean my blog posts and talks I've given? Or do you mean our write-ups on datasketches?

Now that you mention it...my education has been quite ad hoc since I'm self-taught when it comes to frontend so I don't have one definite source to point you to. There's probably a lot of really great talks out there about UI/UX, I remember really enjoying Miles McCrocklin's talk a few years back about Interaction Design for Dataviz and I learned a lot from that. I usually just try to pay attention to the websites I go to and pick out interactions they did that I liked and didn't like. When I was trying to learn more web design a year back, I found a lot of Jonathan Z White's articles really helpful (I especially liked the ones about developing eye for design and color schemes in kpop). I also like the talks that come out of the CSS/JSConfs, it gives me a great overview of what people are currently doing with the web (:

Sorry that wasn't much of an answer, I'm not really as great with UI practices as I am with dataviz, though I'm trying to learn and be better!


Your work and venture is super inspiring! As a freelancer, how do you find the right balance between client work, learning new things and making sure clients find you? How is the overlap among these things? What were some ups and downs you mentioned, esp. those that took you by surprise?

monfera

Thank you Robert! When I first started freelancing, I really concentrated on my portfolio. I tried to prioritize client projects that I can publish and put into my portfolio - so that's why I did the visualization about viewer brainwaves, Hamilton, and Google. I had actually turned down some clients right before those projects (ones that I really liked the people) because I knew either that they weren't the skillsets I wanted to develop or that they were in a private product that I couldn't really publicly talk about. It really helped to keep in mind my priorities and the kinds of experiences/skillsets I wanted to gain, so that I didn't feel the need to jump at every single request that came by (having money in savings also helped).

Of course, datasketches helped the most in building my portfolio - it's a really fun way to show people what I can do and it lets potential clients really get to know my style. When it comes to client projects, they're coming to me for my expertise and not as much for me to experiment/learn new tech, so we use data sketches as our way to try new things and experiment. I've gained an incredible amount of technical skill working on them and I feel that I understand my dataviz process much better now. It also gives me the sandbox to try out things I wouldn't with a client, and I've definitely had my hits and misses but the feedback has been valuable.

As for ups and downs, I think I never realized the full emotional range that stress could take on hahaha. There's such a subtle but rich spectrum, from holy crap I'm not getting any clients stress to holy crap suddenly I'm getting so many I don't know what I'm doing stress. And there's am I going to be able to make my rent this month stress and there's also happy I'm getting any clients at all stress. That whole range of emotion has taken me by surprise hahaha.


Question from a friend: Can you remember a time where the use of statistics dramatically changed your opinion on something? A scenario where the stats disproved many of your preconceived notions about a topic?

zonination

I don't know about dramatically, but I remember when the BART strikes were happening the Bay Area 4 years ago, Ian and a few others from the D3 meetups put together a hackathon. I went into that dataset with lots of questions, because the media was reporting that BART employees were striking even though they were making 80k a year in salary, and that they were asking for 20% in raises.

I made this visualization (one of my very first I published), where I found that most unionized employees averaged 50-60k in base salary and that the 80k figure came from averaging their salaries AND benefits packages together. I also learned that the union employees were asking for 20% over four years (if I remember correctly), and that the actual dollar impact on their salaries were minimal (and they were asking for it bc they had been withheld raises for the 4 or 5 years beforehand due to recession).

I wouldn't have known those if I hadn't looked into the data (though I didn't use much statistics) and if I wouldn't have felt it so keenly if I hadn't visualized the data. I think that was the moment I realized how powerful dataviz can be.


What would you consider to be the best example of a good data visualization? What about the worst?

zonination

I don't have just one example of either, though I do have lots of visualizations that I really appreciate. For the worst...I guess any visualization that misleads/misinforms the reader about the data.

I will say that my most memorable visualization is from Aaron Koblin's TED talk. It was the first time I had seen anything more than a bar/pie chart and I didn't know how to take it. But the moment the bars spiked up to say Happy New Years, I felt chills. I rewatched that talk over and over. I wanted to do what he did. Looking back to what I've been exposed to since, it's not the most technically or even aesthetically impressive visualization I've seen, but it had tremendous impact.


What sparked your interest in the data visualization field, and how did you realize it was the right career path for you? ✨

samhinshaw

Funny I just mentioned it in my last answer! The moment I became aware of data visualization was back in 2010 or 2011 when I came across Aaron Koblin's TED talk. I was a junior in college and I was feeling melodramatically lost, not sure if I wanted to do finance/investment banking after college. I came across his talk and it gave me chills and I kept on watching and I thought to myself I wanted to do what he did.

When I got to my first full-time job, my manager asked if I wanted to work with D3.js. I said yes, and a few years later I was in so deep I didn't know how to get back out haha. I fought it for a while, I was like there's so many other things I want to learn too! but then I realized that data visualization was a perfect combination of art, code, and math and that's basically all my loves and that's when I accepted it ✌️

Having said that, I want to emphasize that dataviz is my interest and my career path right now but I'm hoping for a long life ahead of me and there's still a lot of things I want to learn and get good at and do and make.


What has been the hardest part of doing a project like Data sketches? Some of the resulting data visualization has been amazing by both you and Nadieh!

ostedog

Thank you so much! The hardest part has to be the pressure I put on myself to make something I'm proud of every single time. I'm not competitive with others (I used to be as a kid and around middle school I realized how counterproductive that was hahaha) but I'm very competitive with myself. I make it a goal to do better and improve as much as I can every single time...except that's exhausting at the scale and pace we had set for ourselves.

I'm lucky that I found a partner that's as understanding (and damn inspiring) as Nadieh, and she's been really great about letting me take it at a pace that's sustainable for me. That's why the projects have slowed down since summer, but we're definitely still working on them!


Who do you look up to in the data visualization field?

Who do you go to for inspiration? Both in dataviz and perhaps other fields?

ostedog

There's so many!

I'll start with the easiest: Nadieh :D I really respect her because she's so great at the parts of dataviz I find I'm lacking - like her data analysis and the creativity of her visualizations that really bring out the dataset. Sometimes she comes out with a datasketches or a client project and my breadth is taken away.

I've also always found Giorgia Lupi's work really inspiring. Her Eyeo talk from 2014 (or 2015?) really made me realize how unique and beautiful visualizations can look. I don't always agree with her choice of visual (I always appreciate complexity but sometimes I find them overly complex) but I always find them inspiring and eye-opening from an aesthetics perspective.

I also love Moritz Stefaner and Jan Willem Tulp's work, always something I can take away from there too. And of course, Pudding has been putting out such great visual essays.

That's definitely not a comprehensive list, just the ones off the top of my head. There are so many people whose work I really admire, that I always learn from, and strive to be as good as.

As for non dataviz, Sarah Drasner and Amy (@sailorhg...I realized I have no idea what her last name is) are my heroes. Sarah for her super beautiful codepens and thoughtful talks (especially the ones on animations) and just overall being a really great human being and mentor figure. And Amy for her bubblesort zines, where she empowers younger audiences (especially girls!) by drawing computer science topics in super cute cartoon form. And just overall her spreading of positivity and cute tshirts and pins and really embracing her femininity which can be hard in tech.


My job is something like data analysis or business intelligence or data science + Shiny development (I don't even know anymore), but I'm super inspired by your work. I really want to learn D3.js and someday even React too, and maybe focus more on dataviz.

Where do you get your inspiration for projects? Or do you find inspiration comes easily to you? I find I have a hard time doing personal projects because I have no idea where to even begin.

Also: favorite kpop group?

triple_dee

It was definitely harder when I started, because even if I had something I was curious about I wouldn't know how to start and what to do with it. So I think I tried to work on other people's ideas and their curiosities (I don't think most of them went anywhere since I just wasn't experienced enough), but after a few years I started to be able to recognize when I was curious about something, where to potentially get that data, what might be interesting angles...I did have the luxury of doing mostly viz work at my full-time jobs also, so even when I wasn't actively working on a side project I was still gaining some skill.

Nowadays, ideas come to me almost unbidden. I'll see something or someone will mention a topic and I'll start thinking about the data I can go get and what might make it interesting. I have to actively filter through and prioritize the ones I find the most interesting (sometimes my boyfriend pitches them too haha). Right now, I'm pretty damn excited about my tswift viz where I'm looking at the colors of her past MVs.

Whatever you decide to do, it doesn't have to be grand. It doesn't have to be full fledged. It's great to start small, if you don't have something you're interested in, see if the people around you have something small they're interested in that you can do for them. Once you get started and do a few, I think it might get hard to stop :D

AND THANKS FOR ASKING THE MOST IMPORTANT QUESTION! I actually actively followed kpop in the mid-2000s, so my tastes are old but HANDS DOWN DBSK. I'm so sad they split apart they were just so so good. I really like Big Bang too, I stopped listening to kpop ~2009 when they were really getting traction. And not that you asked, but my fav jpop band is Arashi (:


Big, big fan of you wonderful, funny and gorgeous work! :) What are some of the most valuable lessons that your learned while doing data sketches? Things you think you might not have discovered without the project?

nbremer

I've finally gotten to your questions! I'm a huge fan of you too hehehehe <3 (this probably grosses everyone else out hahah)

For data sketches, it might actually be the realization that my skills have gotten to a point that I can get any dataset and build any visualization I set my mind to. I might really struggle, and spend lots of my time banging my head, but I know that with perseverance and the internet I can figure it out. That's really empowering, and gives me the confidence to pursue other related interests, like Processing or WebGL or even art in the physical world. It just makes me excited about all the possibilities.

I think I'd have eventually gotten here, but data sketches really accelerated it.

(Oh, and of course to always sketch to smooth out my thinking errors 😜...and for that matter, to ask strangers in bars my math problems 😜😜😜)


Do you have examples of things that you changed on a data visualization because a client asked you to do so, but that from a data or visual best practise standpoint it didn't make that much sense, or it was a breach of consistency?

nbremer

Not...off the top of my head actually. I'm not sure if that's just because I've had great creative control over my client projects...or if I'm just not as aware of issues of accuracy? I've definitely had instances where the client has asked me to do something that'd make a part of the visualization inconsistent, but I've always been able to point it out and convince them otherwise. I try to get to the bottom of what they really actually want when they ask for a change, and working towards a solution that does as much justice to the data as I can while fulfilling their priorities.


Hi Shirley, SVG or CANVAS ? When and where ??

gokatz

SVG if you have less than ~2000 elements and you have high interactivity. Canvas if you have more than that amount or if you're doing some heavy animations but you'll have to roll out your own interactions (click/hover events etc.).


Is there an area you are looking to improve in or learn more about for future work?

campkels

SO MANY PLACES. I want to get better at the actual data analysis, and I also want to get better at optimizing for browser performance (and mobile for that matter...).

I also want to teach myself Processing because I want to make beautiful generative art, WebGL because I want to explore what I can do there, I want to play more with Arduinos and LEDs and solder...so many things I want to code and create.


How do you go about selling work? Do you create dashboards and then walk away, leaving the client with an ETL process and someone who then owns the dashboard?

I've always avoided consulting companies in this space because the 1-2month turnover seems annoying and doesn't give enough time to really understand the data and there create good vizualizations

sharpchicity

I actually haven't created a single dashboard for clients, and I rarely get asked to do them (I can remember maybe one email asking me to do a dashboard).

I haven't, however, figured out how to work with clients on visualizations for their product or internal tooling. It's like you said - as a freelancer, I'm pretty short-term, and that doesn't give me enough time to be able to go in, fully understand their data, and work with their eng team to prototype and implement something that they can then take over. I don't know if it just means I need to have a longer contract with them, or if it's just not something for freelancers and it's only something full-time employees can work on (because they need to be able to keep implementing updates and new features and maintain it).

Most of my projects have been based around a specific dataset and I create a bespoke visualization for them, usually for narrative/marketing purposes. My clients usually find me through my portfolio or the talks I give.


Hi Shirley,

I'm a senior studying computer engineering. I want to be a full stack engineer and I'm fascinated by your work.

What steps should I take to develop the skills necessary to be able to develop good data visuals?

Thank you.

Ilovepoopies

There are some really great resources on the internet; Nadieh has this blog post on learning dataviz, as well as this really great dataviz book reviews.

For D3.js, Ian has this great post The Hitchhiker's Guide to D3.js. Overall, follow people's work, their twitters, watch talks from conferences like Openvis Conf for inspiration and to understand what's the latest. Really immerse yourself in it :D


Two parter:

1: Where do you see data viz going with regards to annual reports of listed companies. I see some are dipping their toes into infographics but its still in infant stages. Some stick to the ol' excel for EVERYONE!

  1. I consider myself intermediate with Illustrator, Indesign and PS. Thats where I create most of my infographics. Where can I go to for the best training of D3, React etc?
TeargasTimmy

Oh do you mean the Year in Review type things? I don't know if I've really seen examples of data visualization in those (mostly infographic like you said) but I also haven't really paid attention to them. It'll be really cool if companies start doing them as full on visualizations!

As for where to go for training, I'm pretty biased but I really like Frontend Masters. I have an intro D3 course on there, and Brian Holt has a great React workshop. It also has lots of Javascript courses if you haven't touched JS before. But in terms of picking up D3+React, they're at their core programming languages, so it's good to get some CS fundamentals in there (just basic intro CS classes that you could probably find on Coursera) and that should help make D3/React/any other libraries make more sense. For the actual visualization, knowledge of SVG/CSS would also be really great.


I'm currently steeped in the beginning stages of a couple analytics projects for a couple companies but eventually I will get to the point where most of the data pulling and cleaning are automated. I'll have to dig into properly presenting insights/findings in a couple months. I know, for a fact, I'm awful at data viz. As a result, I currently don't have questions but I will undoubtedly have some in the near-ish future. What is the best place to reach you for future questions?

theonlyonedancing

I'm the most responsive on twitter @sxywu! Sometimes people email me but I'm actually much slower with email πŸ˜… .


Why do you build visualizations from the perspective of focusing on a single topic as opposed to building a framework for a user to go in and easily build out what they want to see?

I realize this is not an easy problem to solve, but it would be ideal if a user could go to a site, add in their data source, maybe do a query / filter on it, then send it to various visualizations that are interconnected (such as dc.js / crossfilter.js).

I've been building visualization apps in d3js and started branching into dashboards more recently over the years, and I'm trying to figure out the best way to make a data agnostic product.

Paddington_the_Bear

That's a really great question and the answer is really simple: I have more fun working on single topics than building frameworks. There's already lots of really great solutions that people much smarter than me have created or are working on (like Density Design's Raw Graphs or Kiln's Flourish or...Tableau or many more that I've never even heard of). I feel it takes much more engineering effort than I could handle by just myself, and I don't feel the need to join in on something that other people already have a great handle on.

Even more importantly, the thing that I really enjoy about dataviz is finding that something fun in a dataset. I especially love datasets about human behavior and relationships and using data to understand people's motivations. There's something really satisfying about finding those, and then sharing them via a visualization. It's also where I think I can contribute more.


Hey ! Amazing works, I saw some of them in the past but never looked for the author to be honest

So, is there any course I should follow to learn about story-telling ? I think it's the most important part for a unique experience :)

Thank you !

kentricks

Thank you so much! I don't know any courses on storytelling (perhaps schools of journalism might have them?) but I just try to follow the great work being down at NYT, WaPo, Pudding...I also have been finding myself paying a lot more attention to story structure in movies. There's some great Youtube channels that cover film plots and story devices, and I remember having a great conversation about film storytelling with Erik Hazzard.

I absolutely agree that it's the most important part; I'm increasingly convinced that great narratives are what resonate with people and get talked about. It's definitely something I'm working on and trying to get better at!


  1. What made you decide to go with D3, and what other options have considered?
  2. What do you recommend for print, specifically for making figures that go into a LaTeX document? I'm eager to learn if there is something better than Asymptote, which I find to be capable but difficult.
Bromskloss

My answer is pretty boring haha, my first full-time job asked me to look into and learn D3.js, so I did. I haven't really tried to use any other options, just because I ended up really loving the freedom and flexibility of D3.js so I'm really not great of a person to talk about how different charting libraries compare.

As for your 2nd question...I unfortunately have no idea because I have 0 experience with LaTeX documents and have never heard of Asymptote.


Hey man, I've been getting into D3, and like it a lot sofar. I've been worried however that I wouldn't find any demand in the market and I'd be wasting my time. In what market do you find your customers and how do you approach them. Do they approach you?

ConceptoftheLiving

Well it's weird because D3.js is still quite niche, but it's one of the most starred libraries on Github (though that's probably just because anybody that has tried to make a graph tried to do it with D3.js). I think the important part isn't to fixate on D3.js, because it's a great tool but it's a tool nonetheless, but rather to pick up more data visualization principles. That's what people usually find me for, because they have a dataset they want to understand and they want a visualization to help them understand it. They come from a lot of different industries (though mostly tech), but basically...anybody with data will eventually realize they want to do something with it. Most if not all of my clients have approached me because they've seen my work somewhere or heard my talk or somehow found my portfolio and want something similar.


Hi Shirley, how do you deal with raster data, for example a matrix of values representing light intensity or something (like from a CAT scan).

I feel like a lot of the data viz community ignores raster data because it's not very neat. For example, D3.js doesn't even support it. Do you have any recommendations for that?

caks

I'm really sorry to say that I've never worked with raster data either ):


What limits for numbers of data points displayed have you hit, and why, or alternatively what techniques do you use to manage a large set of points required to interact with? E.g relational tensions over 50k+points graphed in spheres then filtered interactively.

humpherman

I work in the browser, so when I use SVG the upper limit I hit is ~3000 elements (and that's if there's no repainting of them, so no animations, very little interactions, etc.). I can push that up higher in canvas though I've never tried to find the upper bound. I really like showing every single data point, but after a certain amount that's really not feasible. After ~1000 data points it's also about balancing showing everything vs. visual clutter, and how to best aggregate/filter the data. (And that really depends on the data).

I'm starting to try and pay more and more attention to performance optimization in browsers, though I'm nowhere near knowledgeable enough.


Hello Shirley! Thanks for this AMA. You would have come across these #beginners question.

1) What will be your suggestion/advice for a beginner in data viz?

2) Where to start as a data visualization developer?

3) What will be your recommendation on the learning resources for D3 for beginners apart from the official GitHub docs ;) ?

4) is D3 alone enough for dataWiz?

gokatz

Thank you for all the questions! I actually answered pretty much all of those in this Medium article/quick interview especially the part about learning resources for those just starting out and advice for those just starting out (:

As for if D3 alone is enough, yes and no. If you just want to make something standard that D3 ships with (like all the layouts and geo modules) then D3 is more than enough. If you want to do more than that, other web tech might come more into play. (I most often use React, but that's only because I build really interactive visualizations where I often have to manage data state, and I've been using more and more Greensock for aniation). It's really just about identifying what you want to do, and finding the tools (including D3) that does the job.


What 10 npm modules do you use most in React+D3 projects?

pabloneruda

Mode modules?


Favorite drama at the moment and why? O, and favorite Ghibli movie? And favorite Disney/Pixar animation while we're at it. If you're really feeling up to it, favorite celebrity crush? ;)

nbremer

I'm not sure about favorite, but I've been watching King's Woman and holy crap it's beautiful. The wardrobes, the sets, the colors, the compositions. I'd say...don't pay too much attention to the plot haha, but it's definitely giving me SO MUCH aesthetic inspo right now, I just want to take their color schemes and make visualizations out of them πŸ’–

Pixar/Disney Up will always have my heart. And Toy Story 3, because I watched it in a Berkeley auditorium with ~700 other college kids that grew up with Toy Story and we laughed and cried and standing ovation-ed together and that's just a memory you don't forget.

Favorite celebrity crushes: Gal Gadot, Mila Kunis, I also LOVE Daniel Henney and Matsumoto Jun 😍 and more recently Liu Yi Hao and Aaron Yan (:


Additional Assets

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.