The 5 Steps for Proceeding with an AI Project: How to Successfully and Effectively Manage AI Projects (Part 4)

The 5 Steps For Proceeding With An AI Project: How To Successfully And Effectively Manage AI Projects (Part 4)

Hello everyone, My name is Kenshin Fujiwara and I am the CEO and founder of HACARUS Inc. 

Through this series of blogs, I will discuss a wide range of topics of AI, from the history of AI to practical tips for successful application of AI projects. I hope my blog posts will help you gain a better understanding of AI and solve your business issues.

In the last blog, we studied about step 2. proof of concept (POC) and step 3. pilot testing. In today’s blog, we will continue to discuss the last two steps for proceeding with an AI project. However, before studying the following steps, let’s take a look at the difference between the POC and pilot stages first.

The Difference between the POC and Pilot Stages

One difference between the POC and pilot stages is the data collection methods. While the POC runs on data collected in advance for verification and training purposes, the pilot uses actual data generated in real time from the field. This allows the pilot to function in an environment as close as possible to the production operations. 

Test pilots are extremely important. In the case of AI, unlike ordinary IT systems, changes in the nature of input data can alter the performance confirmed during the POC. Therefore, I strongly recommend conducting a pilot in a real-world environment before full-scale operation. 

If these results between the POC and the pilot are different after running the test with real data, it might be necessary to go back to the POC and rebuild the model. In other cases, the model might be fine-tuned to improve the accuracy instead. 

Similar to the POC phase, the development schedule for a pilot varies case-by-case, but in general, it takes between 3 to 6 months in the manufacturing sector. In this phase, it is recommended to verify the system’s usability as well as validate it with realistic data, and modify it as necessary.

There are other concerns that we must address during this phase. First, we must avoid a situation where a well-performing AI system isn’t integrated due to a poor user interface or difficult-to-use software. At the same time, it is important to ensure that the operation flow and procedures are still similar to the original that workers use.

If there are any cases where the integrated AI makes a misjudgment, it is crucial to thoroughly check the cause. In addition, data should be stored in a database whenever possible. This way, it can be used for retraining the AI model after full-scale implementation. 

In any case, one of the key points of this pilot is to make users aware that AI can make mistakes and to help them understand how to respond to errors. This way, employees can seamlessly transition to using the AI system without affecting their work. 

Step 4: Full-Scale Implementation

Once you have confirmed that the AI system can be integrated into your operations and produce a certain level of results, it is time for full-scale implementation. Whether it is for in-house production or for end-user product implementation, there are other factors to consider besides the performance of just the AI. 

One difficulty with AI projects is that they often cannot use the same quality control methods that have been developed for regular software programs. There are also no easy-to-understand best practices that can be applied to AI products. 

Another thing that should be done during this phase is to define a method for monitoring the performance and updating the AI during operation. Even though verification with real data has already been conducted during the pilot phase, it is assumed that more complex data will be used during the full-scale implementation.

Monitoring or Updating AI’s Performance

Therefore, it is necessary to monitor whether the introduced AI is working properly and to develop countermeasures in case performance degradation is observed. Whether monitoring or updating, it is relatively easy to do so when the AI is provided in the cloud, and there are countless tools available to assist in this process.

On the other hand, if the system is used in a closed environment, such as in a hospital or factory, it is necessary to collect monitoring information within that closed environment and build a separate system to alert the user via e-mail when a problem occurs. Since this area also involves the IT department, I recommend that the monitoring process and system are put in place prior to operation. 

Industry-Specific Requirements 

Another important thing is to obtain approval from the Ministry of Health, Labor, and Welfare when working on a project that involves medical devices. This applies to not only the medical field, and industry-specific requirements should be checked prior to product launch. These precautions should be addressed from the pilot testing phase through the full-scale implementation.

Step 5: Project Operation

Honestly, AI project operation is no different from the operation of a normal software system. This is assuming that the systems and processes for performance monitoring and updating methods have been adequately prepared in advance. During this phase, the main tasks are to conduct regular monitoring and address problems as they arise. 

For AI systems, the main problems are typically performance degradation and system errors caused by abnormal outputs resulting from unexpected inputs. Solutions to these problems may include software updates, AI relearning, and fine-tuning of the model. 

Retaining a Solid Operation Team

Since the operation of the AI will continue as long as the system is in use, it is not possible to specify an exact lifetime of the system. However, I recommend that companies retain a solid operation team for at least one year after the introduction of the AI system. 

If the system is related to business operations, the characteristics of the input data will usually remain relevant for around one year, or one cycle. For customer service applications, consumer trends will have an impact on the data, so the number of operational resources will vary depending on the type of service. 

So far, we have studied the steps for AI implementation. In the next blog, I will introduce the points of focus and methods for approaching an AI project. For your updated knowledge and insight about AI technology, subscribe to our newsletter or visit HACARUS website

Subscribe to our newsletter

Click here to sign up