About the technology stack of the AI product development team that combines software and hardware

About The Technology Stack Of The AI Product Development Team That Combines Software And Hardware

Hello, I’m Ippei Usami, a data scientist in the Industry Division. In this article, I would like to introduce the technology stack we use. 

While there have been articles in the past that introduced the overall technology stack of HACARUS, this time I will focus on the technology stack specific to the Industry Division to which I belong.

Industry Division Operations

First, before diving into the technology stack, let me briefly introduce what kind of business we are engaged in to help you visualize it easily. For more detailed information, you can refer to the description of the duties and responsibilities of our division, but in simple terms, we develop a product that performs visual inspection of final products using a robot arm equipped with advanced cameras and lighting. Additionally, we also develop software that enables inspection without the need for a robot, as long as images and cameras are provided.

Therefore, broadly speaking, our development involves:

  • Creating machine learning models (such as developing proprietary algorithms)
  • Providing a user interface (UI) for customers to perform tasks such as model training, inspection configuration, and execution
  • Developing the hardware configuration and control aspects, including robots, cameras, lighting, and other equipment
  • One distinctive aspect of the Industry Division within HACARUS is the involvement of hardware. Consequently, the chosen technologies may differ from those used in other divisions.

Technology stack used

We utilize C++, C#, and Python as programming languages in our development.

C++ is used for implementing machine learning algorithms. Initially, the implementations were done in Python due to its ease of use. However, considering the permissible computation time on the production line, which is in the order of milliseconds, we needed to reevaluate the computational methods and aim for optimization. To achieve this, we started using C++ and incorporated libraries such as Eigen and Intel Math Kernel Library for faster processing. The details of these optimizations could be quite extensive, so we can discuss them on another occasion. Nevertheless, we have prepared wrappers using platform invoke and pybind11, enabling seamless integration and achieving equivalent results when called from C# or Python.

C# is primarily used for UI creation and backend processing other than machine learning algorithms. We employ WPF for UI development, and C# is also used for image processing and hardware control. We are utilizing the .NET 6.0 framework. Initially, due to compatibility with the camera’s SDK, we had to use .NET Framework 4.8. However, with recent SDK updates, we were able to update the framework to .NET 6.0. The update process itself posed some challenges, which we could discuss in a separate article in the future.

We rely on several external libraries, including OpenCvSharp4, Caliburn.Micro, and Serilog. These libraries assist us with tasks such as image processing, data binding with XAML, and logging. However, it’s worth mentioning that OpenCvSharp4 requires careful resource management to avoid potential memory leaks.

Python is not directly used in the product itself. However, we employ it for tasks such as performance validation of machine learning models and verification of HACARUS Check functionality using customer-provided data.

Our code is hosted on GitLab, and we utilize GitLab CI/CD for continuous integration and deployment. With each push to the remote repository, the CI pipeline runs, executing MSTest. Furthermore, when a release tag is applied, a button press on GitLab CI triggers a release build using the tagged commit. This process creates a release on GitLab and uploads the installer to a specific folder on Google Drive. The sales team then distributes this installer to customers. We would be interested in writing a separate article detailing our implementation of CI/CD.

Docker is used for building C++ code and Python wrappers. However, since the product itself is based on WPF, it is not containerized.

Furthermore, although it is not directly related to the technology stack, our development follows the Scrum methodology. We employ two-week sprints to determine and develop specific features. For more details, please refer to this article.

Lastly,

If there are those who have already viewed HACARUS’ recruitment guidelines or past technical introduction articles, the information presented here may have been quite surprising. That’s because within HACARUS, various businesses are being developed.

If you have ambitions such as developing your own product using C# (.NET), creating a new visual inspection device using robots, or designing high-speed algorithms in C++ that can withstand inspections on production lines, we encourage you to take a look at our recruitment page and come talk to us.

Of course, we also develop products in fields such as healthcare and labor safety support, so if you have an interest in those areas, you are also welcome to join us.

Subscribe to our newsletter

Click here to sign up