Hi! I am Andy Kirk, editor of visualisingdata.com and dataviz freelancer doing design consultancy, teaching, writing and dataviz for Arsenal FC - I am delighted to invite you to AMA/Ask Me Anything!

Abstract

Hello everyone. My name is Andy Kirk and I am a UK-based freelance data visualisation specialist. I do dataviz design consultancy, run training workshops, write books, give talks, undertake research work, lecture at Imperial College and I am the editor of visualisingdata.com. I also provide data visualisation services to the Arsenal FC performance team. You can find me on the web, on Twitter, Facebook and Instagram.

I launched visualisingdata.com at start of 2010 to continue learning, research and writing about the subject. It won (gold, 2015) and lost (silver, 2016) awards at the last two Kantar Information is Beautiful awards event. I tend to be known for my list-making, with my ‘Best of…' monthly and ’10 most significant developments’ posts quite popular as well as my ‘Little of visualisation design’ #LittleVis series. I also try to compile useful data resources for folks trying to make sense of all the options out there, such as dataviz tools, the chartmaker directory and dataviz books.

Since I became a freelance professional in 2011 I have focused, primarily, on providing data visualisation consultancy and training workshops – of which I have delivered over 210 public and private training events across the UK, Europe, North America, India, South Africa and Australia. You can see my past clients listed here. In July 2016, I released my second book entitled “Data Visualisation: A Handbook for Data Driven Design”, published by Sage.

So that's me in text form here's proof that I am actually me.

** Update @ 6:30pm (BST): I'm back, let's do this **

Hey Andy! Big fan of your LittleVis series!

My question: ...did he really block you? (kidding!)

Real question: Your books are very inspiring and insightful. But it is almost as encouraging just to see you have time to do all that you do (books, blog, freelancing, workshops all over the place, and so much else).

How do you find time for it all? Are there things you want to do but are too busy, and which would be the first to go if you could replace it with something else?

Geographist

Thanks very much. Yes he did, then I blocked him in probably the lamest ‘this’ll show you!’ counterattack ever :) Anyway, thank you for the kind words about the book and all the other activities. Yes, I do get involved in a fair variety of stuff. Though I have, at times, overcommitted and found myself utterly drained (this happened after my book last summer and at the start of June this year where I’d had an absolutely relentless 5 months) I feel it is important for my sensibilities to maintain a variety of different pursuits. It keeps me fresh and avoids me getting to a stage where I’m just phoning it in. I also find the different things I do are part of a coherent system of activities – the book and blog are my marketing, working on a book helps me fine-tune my teaching/training materials, my design/consultancy experiences feedback in to my teaching by way of sharing stories from the front line, travelling to do workshops helps me meet people from around the field to broaden my horizons etc.

I am getting better at saying no to things, which is a vital stage to get for any freelancer, but this is so hard to do, especially when it saying ‘no’ to something in order to create space to breathe rather than a straight ‘no’ because you’ve just got other things on. Overall I feel I’ve a good balance going – helped by not having staff and overhead concerns like that – I do work all sorts of non-standard hours but mainly because I still enjoy it.

I would love to have more time to just do viz design work – either passion projects or more client work – that’s the big thing I’d seek to do more of if the time permitted alongside simply learning more tools through practice. I hate to say it publicly, in case it is listening, but I think probably maintaining the blog is the thing that I would love to handover to somebody else! Doing 7.5 years of monthly best of posts certainly weighs heavily in terms of time committed but I have to just realise it keeps the footfall going.


Hi Andy! Can you tell us folks who are just starting out in this field purely because of interest how to move forward?

Bluethf

Hi Blue, can I call you Blue? It's a good question that I'm going to answer initially by batting back to you - can you tell me what you're aims are, are you looking to move into this professionally/commercially so your goal is career focused OR more just looking to develop competency almost in a hobby capacity? Thanks


Question from a friend: Can you remember a time where the use of statistics dramatically changed your opinion on something? A scenario where the stats disproved many of your preconceived notions about a topic?

zonination

I really want there to be a great, Hollywood-esque response to this, but I can’t summon one right now! I’m going to dwell on this and see if I can find a suitable answer. I fear it will end up being something tragically dull and childlike "Well, one day I was convinced it was going to rain but then saw a man on television showing the weather forecast and saying it would be warm and then I decided to leave my umbrella and coat at home and then the forecaster was right and it was really warm and I was really happy to have been proved wrong". Leave it with me!


Hi Andy, What´s your "by default" process for dataviz regarding software? I mean do you trat data in Tableau, style in Illustrator and then program with d3? do you have templates ou reuse, etc? thanks!

Chodrolas

Thanks for the question. I am a 20 year Excel guy so that will always play a strong role in my workflow, VBA is my only competent area of programming and for much of the work I get involved in for clients often there is a need to create some sort of automation process to accompany the datavis solutions I’m proposing (you know, to make them viable for reproduction). This is a big factor certainly in my work with Arsenal. After Excel, Tableau definitely is my go-to tool next for playing about with initial charting ideas and – particularly for data that is unfamiliar to me – exploring its qualities/properties.

I am not competent in d3 or other front end libraries so if I’m developing interactive work I will collaborator with others who have this skillset. As you mention though, for static work, Adobe Illustrator is always the place I end up in to fine-tune the appearance of all my charts/graphics that I will bring in to Ai as vector images. I use http://rawgraphs.io/ a lot also, for non-standard charts. Re. re-using templates, yes, I do find occasion to do this but as my projects tend to be quite diverse I don’t usually find much efficiencies exist as the shape of the data changes their scope considerably.


Have you any thoughts on how 3D texhnology like VR/AR can be used in the data visualization fiels?

ostedog

I have experienced both and feel there are different types of potential with each. It is a conversation I’ve had with a few people recently. The main thing is that both clearly create new ways of experiencing physical (virtual, real) space and each brings 3D visual into play more effectively so than is reasonably possible with most screen interfaces/interactions given you can offer greater flexibility for modifying your field of view.

I think AR has more practical/widespread scope than VR for smaller experiences and could be a fascinating way to consume visualisations that require more space than a single mobile screen window provides. What I mean by this is in terms of mass experience and access. VR feels more of a 'boutique' solution, albeit with huge potential and power. With VR you need the kit (realistically, phone and cardboard viewer as minimum through to high end viewers) and the development of a deep/rich world in which to explore is going to be more expensive and more involving with VR. AR, on the other hand, feels much more achievable to create smaller solutions for more people to experience just using an App on their phones. The AR visual worlds required to be created feels smaller and probably easier for more people to be able to create a solution (I’m thinking here the increased accessibility of AR authoring kits). There is also the issue with VR of the ‘you just look a bit of an idiot’ factor which clearly had an impact on Google Glass. So again I feel VR will be more for highly-intensive, more complex, greater exploratory experiences.


Hi Andy! As a Data Analyst, I put a lot of focus into the presentation layer of my job (visualization), and as such, I've studied the subject in depth. In learning more about visualization and reading books from Few, Tufte, yourself and others, I've begun to put a lot of emphasis on best practices. However, I sometimes find myself so worried about following "Best Practice" guidelines, that I often forget about the exploration of my data. This results in me creating a Dashboard/Charts that completely follow "Best Practices", but don't actually say anything about the data.

In your career/teachings, have you ever run across this problem? Do you feel that sometimes the pressure for Best Practices causes analysts to forget to analyze?

Visualmodelofdata

Thanks for your question. Let me start off by saying don't worry, it is an entirely common thing that I do indeed find amongst the majority of my delegates, that’s in part a reason why they are there in the first place!

I would say, firstly, that your visuals don’t ALWAYS have to say something – some tasks can’t and shouldn’t result in a key message(s) because there may not be a message or to determine what is a relevant/signicant finding is at the mercy of one’s interpretation and that will vary from one reader to the next depending on their individual needs. What I find though amongst some delegates is the pressure doesn’t necessarily come from the best practices so much as the desire to impress, to emulate the creative and innovative techniques they see getting good hits, tweets, likes, upvotes etc. One must always try to be led by the data and use the visuals to enlighten our readers about this content. I'm not going to get knee-deep into the 'does form follow function' debate but in a sense you can separate your thinking between the content and the way it is communicated. I always stress the importance of editorial thinking in my teaching/writing as the most influential differentiator between the average and the good/great in this field: Having a discerning eye for what analysis you are going to visually portray. The question of how comes later…

Overall, I think you're simply going through the natural stage-by-stage progression we all go through. The second part of my answer is that your observation chimes with this notion of a diagram I pulled together to try to explain the shape of my own journey and one I feel many others go through - http://www.visualisingdata.com/images/DevelopingConvictions.png. In summary, we begin by finding, reading, listening and observing the rules/principles we find from the ‘experts’. We hang off them because they offer a learnable and reliable framework. At some point, we develop a sense that there are other ideas out there worth considering, we start to discover doubts that these ‘always do that, never do that’ rules don’t necessarily apply to ‘always’. So we lose a sense of clarity for a while and we have to redouble our efforts to find our independent convictions, to try things, to make mistakes, to learn from them but – ultimately – develop our own voice.


Hi Andy, love your Chartmaker Directory. I am involved in performance analysis in Sport and I find visualising data for squad players of all aptitudes in sport very difficult. Have you created a team-wide viz that covers the main key numbers of a match or player for Arsenal? Did you have to interact with the Arsenal management/players to understand their ability to understand data/graphs?

harrytimes

Thank you Harry. As you say there are so many dimensions to track in any sport and football is no different. For Arsenal, yes I do create a number of different outputs that look to show all players achievements/contributions about a given topic in a single viz. I am often playing the role of translator from the big-brain data people and the people more on the football/coaching/specialist function side so I’m trying to find visual solutions that will answer their key questions in as accessible form as possible but, in doing so, often have to push back against the (understandable) desire for too many things being pushed in to (let’s say) a 1-page single view. Naturally, this can lead to a certain risk of losing the synthesis between different perspectives of interest that have to be presented in distinct, separate ‘places’ but the alternative is that you put too much in to one thing that serves nobody.

I do produce things for the management/players but don’t personally have contact with the players so there is a need to educate the initial recipients and get them to impart that understanding onwards. One of the key things they are embracing though is that working out how to read a frequently received visualisation product (generated after each match, for example) will become clearer and clearer through repeated exposure. There are some very smart operators at the club, though, and I have found a huge amount of buy-in so far.


This AMA is being permanently archived by The Winnower, a publishing platform that offers traditional scholarly publishing tools to traditional and non-traditional scholarly outputs—because scholarly communication doesn’t just happen in journals.

To cite this AMA please use: https://doi.org/10.15200/winn.150591.15436

You can learn more and start contributing at authorea.com

redditWinnower

Thank you Reddit Winnower. I know you're not a real person but doesn't mean to say I shouldn't express my thanks.


Hi Andy,

What are the most common visualizations that the Arsenal Management look at? I would imagine you create a lot of heat maps looking at player's positioning.
Also what are some of the most common Metrics they look at to judge performance?

zlataniskillingit

Hi Zlatan, thanks for the question. You’re not hear to pass on secrets to Mourinho are you? :) Firstly, I haven’t created a single heatmap of player position, actually! (There are others doing analysis so me not doing it doesn’t mean it is not done). I would suggest that for most of the analysis into that kind of issue video analysis/clips are more utilised than the more abstract visualisations.

Unfortunately, regardless of whether you are a sleeper agent acting on behalf of Man Utd, I can only discuss this work in broad terms. And actually, the word ‘broad’ is a good fit because the club do genuinely look at everything. When I say ‘the club’, I refer to the sports and data scientists within and working with the club who do the heavy lifting, gathering, managing, analysing and researching all the data you can imagine they have. As it moves up the chain, each level of management and each area of specialism is largely responsible for taking the detail and distilling upwards the key messages, the most salient issues. I understand Mr W is very hungry for all the detail but realistically it isn’t sustainable to dump everything at his desk. What we are getting better at, slowly but surely, is bringing previously disparate analysis subjects together without overloading matters (as per the answer above).


Hi Andy! Is there some set of data or type of information, real or hypothetical, that has not yet been visualised, that you believe would be really useful to the public if it were visualised?

DangerSuit

That’s a great question. If you look around the web/reddit for visualisations and infographics, there is genuinely a full spectrum of topics covered. I can only think that the answer of what is not yet visualised that would be valuable lies somewhere in the realm of a complex system, perhaps a qualitative, political or scientific context. A graph showing life expectancy in a country going up is great, but what does this line going up mean in terms of its influence/impact on other ‘things’, like a health system’s capacity to look after an aging population, like the consequences of fewer young people etc. I fear that as a society we see things – or are presented things – too much in isolation and too simplified. Off-topic but the Brexit vote was never a binary thing, offering a yes/no option over simplified a hugely complex and hugely nuanced issue. Cause and effect is the holy grail of all analysis and that is something that you only can grasp when you have a true understanding of a system. Anyway, back to the question, I think the single biggest issue facing us is climate change so, rather than just seeing co2 and sea level visuals I’m sure there has to be more that can be potentially done to show a clearer window into the cause/effect of our individual choices as consumer and as citizens.

(From that grandstanding, preachy ‘we need more climate science visualisations, people’, watch this for a huge gear shift)

Another answer to your question concerns one of the projects I’m working on – and hope to start in December now that all the data is collected – which is to visualise the rhythm and architecture of Seinfeld. My brother in law has watched all ~180 episodes to manually quantify and categorise all sorts of moments and observations so that I have a brand new, hand-curated digital data set to play with and visualise. Whether this is useful to the public, well, who knows!


Welcome to /r/dataisbeautiful!

What would you consider to be the best example of a good data visualization? What about the worst?

zonination

My criteria for a good data visualisation is: 1) Trustworthy - can I have faith in the data being shown and trust there are no deceptions at play (in the data or the visual) 2) Accessible - does the visualisation design help me to access the understanding available from the data that could not be achieved in its raw/table form and in a way that removes unnecessary/wasted thinking - doesn't mean simplified, doesn't mean quick, doesn't even mean easy - I want the rewards to exceed my efforts 3) Is it elegant - hard to describe other than you know when it is missing.

So, an example of a viz that satisfied these criterion for me was by Alvin Chang of Vox that I talked about in this post http://www.visualisingdata.com/2015/12/12757/.

By extension, the opposite of these are (1) Misleading, (2) Confusing and (3) Ugly and the list of such works that demonstrate any of these 'qualities' is endless so I'll just randomly pick anything from WTF viz: http://viz.wtf/post/124510524590/landmarks


What would be the one thing that people don't think about, but could really make a data visalization better?

ostedog

Well, I don’t think this is necessarily something people DON’T think about but I just feel not enough people think ENOUGH about it – colour choices. Colour is such a powerful visual cue to associate with quantitative and categorical values, to emphasise and de-emphasise, to organise content into a visual hierarchy. You can do loads of good but also loads of bad with ineffective colouring because everything you leave on a screen/page possesses a colour attribute – even if that ‘colour’ choice is blank/emptiness. In my experience, most people could make the biggest improvement to their work with better, more careful colour choices.


How do you attack a new visualization project? And do you have any projects that you never get to finish?

ostedog

Unfortunately, an answer to the first part of your question has, previously, required me to write a whole book in response! Let me try not to be obtuse though and say, aside from establishing practical matters about requirements, timescales, what data, for whom (audience), format etc. the most important thing I get into discussions (if client) or thinking about (if just my own work) is to clarify what is the overriding curiosity driving this work. This can be a very specific, clearly articulated question or a more ambiguous statement. It might be something that changes as you get more familiar with the potential of your data, as your clients finally decide what they are pursuing or as you get more au fait with a subject, but anything that helps you as early as possible to establish some clarity about what you’re seeking to accomplish (in terms of facilitating understanding) and, by extension, what you are NOT, can only be helpful.

For the second part of your question I’m going to sound pedantic but I’m going to create a distinction between finished and completed. There are lots of personal projects that I have not finished and hope to revisit some of them (and have lost interest in others). I would say though that every piece I have finished is incomplete. What I mean by this is you have to finish work because you run out of time/scope but there is always, with any creative work, a burning sense of incomplete – I always look back and wish I had more time/capability to do X, Y or Z to a piece.


Hi Andy, as someone who is just starting out in the world of data/visualisation/analysis, I wondered what the one piece of advice you wish you'd had when you were starting out was? And also, do you need an assistant? (Kidding, kind of)..

the_freckly_penguin

Hi Freckly, thanks for your question. I actually probably do need an assistant but I'm such a control freak that I've yet to ever consider expanding my enterprise to a second member of the team! It is a great question and I'm going to cheat and offer three themes: be patient (because if you demonstrate quality it will be discovered/rewarded eventually), work hard (similar reason, hard work will also be reward) and be yourself (because you can't talk, write, teach, do anything for long pretending to be someone or something you are not). Hope that helps and isn't a cop out!


Hi Andy! Your website and blog are a great learning resource I often refer my employees to. What are your top tips for promoting better visualization practices in a large organization that resists change by default?

data_rockstar

Thanks so much, really appreciate that! The kind of resistance you mention is common to a lot of clients I deal with. Let me just bounce some general ideas of the different ways I try to start the ball rolling: 1) Disrupt the norm: Demonstrate how ineffective existing methods are, eg. why a given report is not of sufficient visual quality to be effective (so wasted effort, wasted $$$) 2) Whet the appetite: Demonstrate how effective methods are often achievable just by fresh thinking, new attitudes - its rarely about investing big bucks into tools/talent 3) Set the bar: Look what others are doing, why would you want to not emulate demonstrable good practice in other industries/organisations (especially helpful if you can show a rival doing good stuff) 4) Be curious: If a company is going to be data driven then it needs to have a culture of question asking, people need to be encouraged to ask questions about how things are operating, succeeding, functioning etc. Data driven answers that serve these curiosities show that this is the lubricant to get things moving more effectively and efficiently.


Which of your own projects are you most proud of and why?

ostedog

Maybe I'm a Fassbender type robot but I don't necessarily feel pride in any of my work! The only project I've worked on that I do have genuine pride in would be the second/most recent book because I know I had nothing left to give upon completion of that and I know that I put SO much effort in to it - it was my maximum. But I already have now moved on, know more things, have different ways of expressing things that if I was to read the book now I'd be thinking "Oh, I really want to change that". Other projects I've worked on will also leave with me a sense of 'if only I could have done this, if only I'd had more time to do that, if only I had the skill to do that cool idea I had in my mind...'. I've had this discussion with many other people and the nice way they've expressed this feeling to me is that if you are always progressing you will always look at your previous work with a desire to do things differently and better.


Hi Andy, love your work! I've followed you for years.

We've all seen bad data visualizations; often in the TV news media, magazines and tabloids - which also happen to be some of the most read and circulated media sources. Two questions:

Does this worry you? That biased representation of data is misinforming the public / what can we do about this? (Potentially impacting people's views, voting habits, beliefs etc.)

And, simply put, what is the worst/funniest data visualization you have seen recently?!

Thank you

LucasCu90

Hi Lucas, thank you for the long term following, really appreciate that! It hugely worries me. There is some dreadful mis-use of data, of information, of statistics, of visuals in all walks of life but perhaps especially surrounding politics. The field is often focused (understandably) on the making side of visualisation but I feel the biggest single difference maker comes from teaching readers. Teaching them to be more sophisticated and challenging as consumers of visualisation, pitfalls, lies, deceptions, how to read different charts, what things to look for, when to be cool with 'gist', when to demand details etc. Alberto Cairo is tackling this in part with his Visual Trumpery tour, there are many more articles and discussions about what to look out for when it comes to lying visuals but I think the bigger factor is how we get this type of education into schools - get kids, students, graduates to be far more informed about this vital literacy.

This was a subject I was keenly involved looking into during 2014-2015 when I collaborated on a research project called ‘Seeing Data’ (http://seeingdata.org/), funded by the Arts & Humanities Research Council and hosted by the University of Sheffield. This project explored data visualisation literacy amongst the general public (UK) to learn about some of the human factors that influence engagement and understanding amongst readers of visualisation works. There's a long way to go but I'm keen to continue looking at this side of the coin.

As for the worst dataviz I've seen recently... I don't have links to hand but there were some TRULY APPALLING visualisations of the recent hurricanes especially in respect of the colour choices. Otherwise, I love little dumb things like this https://mobile.twitter.com/evergreendata/status/842037059007193088 or annoying things like this https://mobile.twitter.com/UniversitiesUK/status/900983957545680896 (the map is the LEAST INTERESTING thing to base your visual around, chart the numbers!!)


Hi Andy,

My initial question to you is regarding your educational background? Are you a self-taught person or did you attend design and coding courses :P?

Materazzi23

Hi Materazzi, thanks for your question. My educational background should begin at school (as a kid) when I was into Art and Maths. I wanted to be an architect but bombing at Physics prevented that route so at undergraduate level I did a BSc in Operational Research at Lancaster University. From there I my working career largely involved a range of business analysis, performance analysis and information management positions at organisations including CIS Insurance, West Yorkshire Police and the University of Leeds. Soon after starting my job at the Uni of Leeds (as an Information Manager, not an academic) I started a Masters programme - that was 10 years ago last week. This was a Masters by Research M.A programme designed for members of staff to study workplace-relevant subjects, at an academic level, via a large self-direct study programme. I did a research project around data visualisation, which completely unlocked a passion for the subject and began my journey into this field. Following graduation I launched visualisingdata.com at start of 2010 to continue learning, research and writing about the subject and out of this profile, work opportunities emerged. For a while I was living a dual existence of full-time day job and a busy night and weekend job until I took the decision to go freelance (initially part-time but a few months later full time).


Hi Andy, long time fan and just realized you do viz for my club. Hence, sports viz questions:

1) Do you think football (US-soccer) needs more data viz? I have a feeling it trails most american sports in data viz department; I mean, the amount of analysis accompanied with meaningful visuals usually isn't as common as in most american sports (eg: not too much of football in https://flowingdata.com/tag/sports/, https://flowingdata.com/tag/sports/).

2) Do you have plans to publish your sports visualizations?

3) pls create a viz with Bellerin on the left leading to a disaster /s

djama

Thanks very much!

1) I think it does need it and I feel it does trail other sports. The data availability and the sophisticated knowledge of key/relevant metrics that have influence is increasing, so the visual techniques should follow suit. I see lots of great stuff being done online by hobbyist analysts/blogger, so the techniques are being demonstrated but next is to see these reaching the print media, the broadcast media and of course the clubs themselves.

2) I don't sadly because all my work is purely for internal usage and naturally it is therefore tied up with the confidentiality issues

3) Ha ha, I think this visual does the best job for that - https://www.theanfieldwrap.com/uploads/2017/08/170827-146-Liverpool_Arsenal.jpg - full declaration, I'm a LFC season ticket holder :)


What do you think of pie charts?

AndriPi

Despite what some prominent voices say, I believe pie charts, like any chart, has a role to play in visualisation so long as the circumstances are right.

1) A pie chart shows how the quantities of different constituent categories make up a whole so if you are using them to show anything else, you're using it incorrectly. 2) If the total percentage aggregate is greater than or less than 100% the chart is corrupted. 3) The whole has to be meaningful – often people just add up independent percentages but that is not what a pie chart is about. 4) The category values must represent exclusive quantities; nothing should be counted twice or overlap across different categories.

I find the role of a pie chart is primarily about being able to compare a part to a whole rather than necessarily, easily comparing one part to another part so I feel they therefore work best when there are only two distinct parts (something vs. something else). The use of local labelling for category values can be useful but too many labels can become overly cluttered, especially when attempting to label very small angled sectors. Colouring of the categories is best achieved with different hues and not colour lightness/darkness to maximise the visible difference. Positioning the first slice at the vertical 12 o’clock position gives a useful baseline to help judge the first sector angle value. The ordering of sectors using descending values or ordinal characteristics helps with the overall readability and allocation of effort.

Do not consider using gratuitous decoration (like 3D, gradient colours, texture effects or exploding slices). 3D will distort the reading of the values, the other effects don't distort so much, more that they are just a bit outdated and look like shit.


Additional Assets

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.