Human-Centered Design in Data Science

Pablo Hernández

2020-10-01 15:39:15
Reading Time: 6 minutes

Unless you have being living under a rock for the last several decades, you are probably aware of the fact that AI and machine learning are here to stay. They are revolutionising our lives not only technologically, but socially as well. This is why countries such as USA, Canada, Japan and even the Vaticanhave started to publish roadmaps and guidelines to ensure the safe and ethical development of AI. Not wanting to be left behind, the EU has since published a document called “Ethics Guidelines for Trustworthy AI“. One sentiment, in particular, stood out to me from the document: “AI systems need to be human-centric...”. This concept of AI specifically being human-centered can be found in most ethical guidelines published by other countries. What exactly is human-centered design and how important is it to data science and AI?

Why do we need human-centered design?

I was first introduced to human-centered design a few years ago through the research of the famous UX researcher Don Norman. In his research, Don mentioned that our problems with push/pull doors have more to do with bad design rather than incompetence. For our brains, seeing a handle on a door instantly triggers a subconscious assumption. When handles are installed onto push doors, awkward situations are bound to occur as that subconscious assumption comes into play. This story didn’t really resonate with me at first — it didn’t seem like anything more than a curiosity that you might share with your friends or partner. Later, deeper into my work, I began to understand just how powerful this story was. In my opinion, we engineers (as an aerospace engineer myself) tend to overlook two important points in our designs. First, we tend to ignore the fact that, in most cases, what we create does not operate in a void and will involve a human operator at some point. Second, when we do account for the human operator, we tend to assume that everything that we know is also obvious to them. A lack of user knowledge can create great potential risks. For example, on Air France flight 447, the pilot’s lack of knowledge of how the onboard computer worked during emergency situations was one of the main causes of the accident.

I believe data scientists also suffer from these flaws — this is most present in AI solutions development processes. A survey made in 2019 by NewVantage Partners concluded that about 92% of Fortune 1000 companies are increasing their investments in Big Data and AI and 92% mentioned that investment in AI is required to stay both agile and competitive. There is no doubt of the power and importance of AI. With some claiming that “data is the new oil” and the great Andrew Ng himself saying that “AI is the new electricity“, it may pertinent to call to mind the old saying: “All that glitters is not gold”.

By Gartner and VentureBeat AI estimates, about 80% of AI solutions fail to deliver and never make it to production. Why is this? Among the main reasons we can find, there are several typical and expected ones: not having enough data, data quality issues or wrong algorithms. But one of the most recurrent roadblocks to production, and the one most underrated by data scientists, is reluctance of management or users to adopt the technology. However, we tend to see this as inevitable and an extension of ignorance of how the technology works. In an interview about human-centered designer Peter Bull, the co-founder of rightly described that the success of a solution depends on it being desirable enough for someone to want to use it while also being feasible from a technological perspective and viable from a business perspective. Only in purely  academic circles would the technical feasibility of AI be the only element necessary to address as research projects try to use the latest and most modern techniques available. Most data scientists are aware that their solutions must not only be technically feasible but also translatable to solutions for real business problems. Lots of work is put in by data scientists to understand these types of business problems before getting to work. But understanding a problem and knowing what users need to solve that same problem are two very different things. This may be why many analytics or AI projects end up in solutions, systems or dashboards that are rarely used by anyone, despite perhaps offering great insights. This is when we blame the reluctance of management or users to adopt the technology. In these types of scenarios, according to Peter Bull, we may still be missing one of the three legs for a successful solution: “Desirability!”. The cultural impact these new technologies can have on users is largely underestimated during the design phases. To ensure success, it is important to take into account user knowledge and the skills needed to increase their acceptance and engagement. This is where human-centered design centers may help your AI solutions achieve a 100% (or close) success rate.

How does human-centered design work?

Image source:

Human-centered design is basically just a framework by which we create a design process. We can define it as a problem-solving process that is primarily focused on understanding the user perspective of the problem and the development of solutions that really meet their needs. Human-centered design is closely related to other popular design processes in this field such as Design Thinking or the Double Diamond model.

This is why you may find definitions of Human-centered design that vary a little, although the overall concept is always the same. In this blog, I’ll use the definition of Human-centered design process used by, one of the leading organisations in this area. Human-centered design is a process that consists of three phases: Inspiration, Ideation and Implementation.

Phase 1: Inspiration

This first phase is all about understanding how the user works and what the user needs. You shouldn’t worry about data availability or possible AI techniques you could use. Your only objective should be to drop any possible preconceived notions and expected outcome you may have and truly understand and internalize the needs, wants and challenges of the user. This would be done through interviews, surveys and direct or indirect observation. For example, say we want to create a tool to help automate  part of an ATCO‘s job. You would not jump straight into trying to automate the complex process to safely manage the flow of air traffic. Instead, you would try to understand the different processes and actions a ATCO performs throughout the day and identify those areas in which an automated system could help him. The idea here is that you have a higher likelihood of success in providing a system that adapts and aligns itself with the decision-making process of an ATCO than with providing a system that totally upends this process and makes ATCOs the ones who have to adapt to it.

Phase 2: Ideation

This is the phase where we would start working with data. This starts with the Exploratory Data Analysis and entails visualizing, better understanding the domain and brainstorming any potential solutions to the problems identified in phase 1. You should not exclude any idea, as impossible or silly as it may seem (you never know!). One key aspect of this phase is prototyping. It is important that you create several low-fidelity prototypes (sketches, powerpoints, mockups) to test your ideas and gather real user feedback and insights. The ideation phase is characterised by these iterative cycles of prototypes and feedback gathering. This iterative process would be maintained even when transitioning to higher-fidelity prototypes to help ensure that the results always align with user needs and also helps prevent wasted time, effort and money. Going back to our ATCO example, in this phase, we would come up with different areas to automate in the ATCO’s decision-making process and create examples of future scenarios where these ideas would be implemented. We would then present these different scenarios to the ATCOs and collect feedback on how useful they see each idea — if it is something they need, if they would actually use it or if they may foresee possible limitations.

Phase 3: Implementation

Phase 3 is the last and usually the longest phase. Once you have a sound prototype that you and the users are happy with, it is time to build a pilot. You’ll go from a high-fidelity prototype to a pilot solution that will be tested with a limited release. Working with the stakeholders, we would select a relevant scenario as well as group of users to test the prototype solution. Similar to the previous phase, this one will not have a linear development and there would also be different iterations. Feedback and data would be gathered from these tests in order to evaluate the solution’s effectiveness, efficiency and usefulness. If needed, changes would be made to the pilot solutions and tested again. In the end, we would arrive at a prototype ready for production. Congratulations! Also during this phase, or earlier if possible, you would create the business model for the solution and prepare for the roll-out into the real world. With all this, and if done correctly, you will have covered the three areas mentioned by Peter Bull. To finish with our ATCO example, in this phase, we would test our prototypes within controlled groups either in simulations or in shadow-mode at airport towers or ACCs . Once we have evaluated its technical feasibility, we would create our business plan, searching for potential partners and interested clients. Finally, we would take our solution to ATCOs around the world to improve safety and efficiency in ATM.

While it’s true that no process is perfect, the principles of a Human-centered design process can bring lots of benefits to the world of data science and AI. Each project is different — no size fits all — so you should take these principles and adapt them to the needs and limitations of your project. All told, human-centered design will surely help increase your chances of success as most solutions being developed today will ultimately involve a human user in some way or another. Finally, human-centered design has the benefit of involving users from the beginning and all along the way. This helps to create a greater bond between the user and the solution, seeing it as theirs, as well as creating greater levels of trust and confidence. This will be a key stepping stone in developing a trustworthy AI. I hope you have enjoyed the blog and don’t forget to visit for more interesting posts!