The Challenges of Being a Data Scientist
Business Problem Solving with Data Science — Framing the Business Problem — 1 of 3
One problem I face when talking to a lot of aspirant data scientists is their focus on machine learning techniques, rather than the Business problems that Data science can solve. The CXO doesnt care for your accuracy score metric, he really wants some very different kind of problems to be solved with Data Science for his company! Here is an attempt to help people think about Business problem Solving with Data Science in a structured way!
An international restaurant aggregator company, YumEats! has decided to expand their operations and wished to enter the Indian sub- continent. YumEats! allows the users to select food from restaurants and order to their homes. Like every aggregator in the space, there are mainly 2 channels- B2C and B2B. The B2C revenue comes from a special rewards program or exclusive membership program that would allow the consumers to prescribe to a monthly or annual membership. The B2B revenue comes from collecting a portion of the fee per order collected as commission from the restaurants. Hence, the company has to acquire both consumers as well as other businesses.
YumEats! has set up a data science wing in their company and hired you as a data scientist on the team. They want you to analyse data and help the company expand in terms of both growth and revenue. This would be possible with both more Daily active users on the platform as well as lots of restaurants with a wide variety of cuisines that would further drive traffic to the app. They have already run a whirlwind campaign with deep discounts that are piling the losses and putting a lot of strain. You have been given a mandate — to inform where to cut the spends with minimal damage and also simultaneously find ways to increase the revenue stream with data driven insights.
You know the basics of Machine Learning and understand the basic nuts and bolts of the algorithms. Great! Awesome! But before you apply the algorithm, you need data. And before you collect data, you need to be clear about the actual business problem you are solving. At the end, businesses look to data science teams to give insights and help solve problems. The journey from a business problem to a data science problem is not so straightforward, and hence in the next posts, I will make an attempt to demystify the process. The process of constructing a data science solution to a business problem is often represented as the following path
While I will focus a lot on how to define the problem and setting the objectives in this post, we will also briefly touch upon the other steps and show how to solve a business problem. The first step of the path — defining the problem — contains tasks such as understanding business needs, scoping a solution, and planning the analysis. However, while translating a business problem into a data science model is a process, it is not linear. Each step in the process usually needs to be revisited multiple times in order to arrive at an analytically sound, maintainable, and scalable solution. The “define the problem” box in the simple linear diagram above can actually be exploded into a much more nuanced process:
It is very common to initially define a business need, but then, as you proceed to more fully scope the problem, realize that an entirely different need is more pressing. Likewise, it is common to scope a solution, only to realize later that data access limitations or engineering constraints require a change in that scope. Often, these changes in plans will even occur after you think you have left the “define the problem” stage of the process. For example, it is common for model tuning issues to raise scalability concerns which may require a substantial re-evaluation of what problems you are trying to solve and how you plan to solve them.A large amount of a data scientist’s work takes place away from the computer: data scientists must work with non-technical co-workers to define the goals and scope of their projects. A well-understood business problem and a well- designed plan of action will lead to better results, less wasted effort, and happier stakeholders.
Define the problem
An international restaurant aggregator company, YumEats! has decided to expand their operations and wished to enter the Indian sub- continent. YumEats! allows the users to select food from restaurants and order to their homes. Like every aggregator in the space, there are mainly 2 channels- B2C and B2B. The B2C revenue comes from a special rewards program or exclusive membership program that would allow the consumers to prescribe to a monthly or annual membership. The B2B revenue comes from collecting a portion of the fee per order collected as commission from the restaurants. Hence, the company has to acquire both consumers as well as other businesses.
YumEats! has set up a data science wing in their company and hired you as a data scientist on the team. They want you to analyse data and help the company expand in terms of both growth and revenue. This would be possible with both more Daily active users on the platform as well as lots of restaurants with a wide variety of cuisines that would further drive traffic to the app. They have already run a whirlwind campaign with deep discounts that are piling the losses and putting a lot of strain. You have been given a mandate — to inform where to cut the spends with minimal damage and also simultaneously find ways to increase the revenue stream with data driven insights.
The above problem is a business problem that is presented in the most raw form — which is to somehow improve revenue and cut losses by analysing the data. Before any form of analysis can be performed, a thorough understanding of the problem is required. To understand the problem in its entirety, you first need to talk to the respective stakeholders.
Data scientists always work with stakeholders. Stakeholders are people who have a say in how the business operates and in what goals the business needs to prioritize. Stakeholders could be managers and executives, but they could also be individual contributors who have responsibilities over specific aspects of marketing, engineering, sales, finance, operations, or any of the facets of a business enterprise. Different stakeholders have different requirements.
For example, in the case study just presented, because of the direct revenue impacts and cuts on spending needed, the stakeholder could be a leadership level person on the Marketing Team or Sales Team CMO or Director, Sales). Instead, if it was a product manager, then the person is more likely to be focused on customer-facing features. A stakeholder who is an engineering manager is more likely to be concerned with the maintainability of a product, and aim to minimize the extent to which changes will create unanticipated work for his or her team. An executive stakeholder (like the one we are working with) is more likely to be focused on the “bottom line” — he or she won’t care too much about the product’s maintainability or about specific features as long as he or she can be assured that revenue will increase, or a client’s business will be retained, or expenses can be cut.
When faced with a scenario like the one given above, the first instinct of most data scientists is to consider different methods they could use to achieve the desired result. That is almost always the wrong reaction to this kind of situation. Here are some of the things to consider:
present and drive the required change. Understand the constraints (if any) that you need to operate under. Align with their agenda and make your objective as close as possible to theirs.
Most of the time, however, people who ask for data science help do not know how to ask questions in a way that are data-science ready. While most data scientists are used to thinking about analysis in terms of method, features and variables, and data transformations, most stakeholders are used to thinking about analysis in terms of spreadsheets or other tools they are familiar with. When they confront a business problem, they think “how would I solve this if I had to do it myself?” They are asking the best question they know how to ask, but that question needs to be translated and shaped into something a data science can act on. For YumEats!, the CxOs are asking the right questions –
As a data scientist you must translate these questions into actual data science problems that need to be solved. When asked to use “data science” to solve a problem, your first task is to think of ways you can ensure that you understand the problem. You need to meet their problem on their terms, not yours.
Preparing Data
It’s rare that data ever comes perfect or clean, causing Data Scientists to spend nearly 80% of their time cleaning and preparing data, according to Forbes. The title of this article is called ‘Most Time-Consuming, Least Enjoyable Data Science Task‘, so you can imagine why 13.2% of Data Scientists are looking for new jobs. If most of your time is spent improving the quality of data, making it accurate and consistent before doing any analysis on it; it can become very draining, mundane, and very time-consuming.
A way to reduce the amount of time spent on preparing data and the lack of motivation of your Data Scientists, adopting emerging AI-enabled data science technologies such as Augmented Analytics. Augmented Analytics automates manual data cleansing and preparation tasks, allowing data scientists to be more productive and spend time with other tasks such as analysis that they prefer to do and enjoy.
Jack of all trades
There is a big misconception in many organisations that Data Scientists can do everything. As a Data Scientist, you are required to collect and clean data, build models and analyse the data. As mentioned above, 80% of Data Scientists’ time is spent on cleaning and preparing the data, so you can imagine how much effort is put in trying to achieve building models and analysing the data after.
This is a lot to ask from Data Scientists and will cause any data team to function ineffectively. They will lose motivation, they will feel overwhelmed by the workload, and probably want to quit. Other colleagues need to understand each Data Scientists’ role, strengths, and weaknesses. Also, this misinterpretation that Data Scientists are so clever, they can do everything and they are capable to do anything assigned.
Is Business Analytics the Future?
There is no doubt that the future will be data-led. From mundane activities to complex business ideas, everything will be completely data-driven. Already, businesses that have implemented data science and analytics have shown a competitive advantage over their close competitors.
According to research, 47% of the organizations firmly trust that data analytics have significantly transformed their respective industry. 40% of the businesses have already made it their top priority to fetch insights from their raw, unstructured data.
When such widespread adoption is happening in all industries, it is imperative to say that the next industrial revolution will happen based on data insights. Only those companies who put business intelligence into use will reap greater benefits than their competitors. Are you on the list?
Sources:
https://medium.com/@shwedoshi/business-problem-solving-with-data-science-a76cf3f0fc7
https://www.kdnuggets.com/2022/02/data-scientist-challenges.html
https://www.datatobiz.com/blog/data-science-process-solve-business-problem/