Hello everyone, My name is Kenshin Fujiwara and I am the CEO and founder of HACARUS Inc.
Through this series of blogs, I will discuss a wide range of topics of AI, from the history of AI to practical tips for successful application of AI projects. I hope my blog posts will help you gain a better understanding of AI and solve your business issues.
In the last blog, we studied the first step for processing with an AI project, problem identification through three case studies. Today, let’s continue to discuss other important factors when implementing issue setting.
Other than PDCA (Plan-Do-Check-Act) Cycle and a field-driven problem-based approach, which we studied before, it is also important to keep holistic optimization in mind as you gain experience with AI development and begin to set goals and identify key issues. In cases where the POC phase is successful, management will often be asked to either approve or reject pilot testing based on the results. At that time, management will take a fresh look at the AI implementation focusing on the company-wide cost-effectiveness of the plan and its management impact.
In fact, it is not uncommon for an organization to reject the implementation of an AI system if it only solves a small localized problem. To prevent this, the initial introduction of the AI should be planned. This includes considering how the project will affect the overall workflow and company processes. If the project is localized to one department, it is still important to show how the results of the project will trickle out to the other departments. By considering the larger picture, you can move forward with the project while avoiding cases where improvements are made to a single process but company-wide benefits aren’t realized.
What are the AI Evaluation Indicators
Another important factor to consider when implementing issue setting is what the AI evaluation indicators will be and how to set the numerical targets for them. Several examples of evaluation indicators include, “the percentage of correct answers”, “the conformance rate”, and “the reproducibility rate”.
Looking at these three examples, it is easy to understand the importance of the correctness rate. However, when working with inspection AI, it is also important that you do not miss any defects. For this reason, the reproducibility rate, or how many defects the AI correctly identifies, may be a higher priority than the correctness rate. There are a variety of evaluation metrics that can be used, and while I won’t go into further detail, it is important to choose a metric that matches the use case of your project.
How to Set the Numerical Targets
Next, let us consider some numerical targets using autonomous driving as an example. If we are aiming for a 100% correctness rate for fully autonomous driving, we are still many years away from realizing this goal for practical use. However, if we plan on using AI to merely support the human driver, then a correctness rate of 90% might be sufficient. While the human driver is still responsible for the majority of the driving operations, the AI would greatly reduce the burden placed on the driver and make driving safer for everyone.
This concept also applies to the introduction of AI in the business world. If you are waiting for AI to solve 100% of the problem, you will never get around to implementing it. For example, if you find that X% of the problem can be solved using AI, the remaining percent can be covered by human input. This way, you can still gain benefits in terms of management impact and cost-effectiveness. In this way, it might be worth implementing an AI system.
In fact, in my experience, if the AI accuracy is less than 100%, the possibility of realization is quite high. However, the process of increasing the accuracy towards 100% will drastically increase the time and resources needed.
Set the Realistic Goals
Once an AI project is started, it is important to carefully monitor the relationship between target indicators and cost-effectiveness. Based on this, trial-and-error is repeated to find the most appropriate indicators. In this environment, skills and work experience are of the essence when it comes to goal setting.
In some cases, it is difficult to set reasonable goals in the beginning, but I encourage you to perform on-site interviews and other methods to set realistic goals.
In the next blog, I will introduce the following steps for proceeding with an AI Project: the proof of concept and pilot testing. For your updated knowledge and insight about AI technology, subscribe to our newsletter or visit HACARUS website https://hacarus.com.