ITOCHU Techno-Solutions Corporation (hereinafter, CTC) is a comprehensive IT services company that solves various customer challenges with advanced IT solutions. Regarding infrastructure, the company supports multi-cloud environments, combining public clouds, SaaS, and other technologies with its own data centers, and provides integrated data center services that also incorporate network and operational monitoring. The service provides a portal site called "SERVICE GATEWAY" for user companies, which compiles everything from checking information about contracted services to inquiry forms. Providing the necessary information on the portal site requires data integration with multiple internal systems, but as the number of systems and processes increases, operation and management becomes more complex. While building a new system specialized in data integration, the company came across DataSpider. By introducing DataSpider, they were able to realize a system that allows for efficient development. Based on their track record and know-how in using DataSpider internally, the company is also considering providing DataSpider services to customers. We spoke with the company about this.
Customer Issues
data integration processing between multiple systems has become more complex, and we want to improve it so that it can be developed and managed more efficiently.
Benefits of implementation
Compared to scratch development,
Development time
Significant reduction
Detailed design documents, specifications, etc.
Documentation creation
No need
In-house
Leveraging our track record of implementation,
Expanding services to customers
Consider
data integration processing on customer portal sites reaches its limits
CTC offers a variety of IT solutions, including its own proprietary data centers and cloud infrastructure. CTC offers multi-cloud services that combine on-premise, private cloud, public cloud, and even SaaS to meet customer needs. The portal site "SERVICE GATEWAY" for customers who use these services has been in operation for the past six years, providing a one-stop shop for everything from information on contracted services, to confirmation of various deliverables and manuals, service operation status, and the reception of questions and inquiries. In order to provide various information on "SERVICE GATEWAY," it is essential to data integration with multiple systems, including the company's internal contract management system, ticket management system, and data center rack information management system. Because we build and utilize systems suited to each requirement, the number of connections and processes has increased, making things complicated (Yokobori).
Manager of the System Development Section, Service Infrastructure Management Department, DC/DX Business Promotion Headquarters
Masato Yokobori
Previously, data integration processing had been incorporated into a system called "DSAS," which served as an internal portal, but the company was beginning to feel limitations in its operation. Because a proprietary mechanism for connecting systems individually had been developed, the more systems there were, the more complicated management became. In batch processing of data integration, an error in one area would affect the next process, and the concentration of user portal access processing and data integration API calls could cause delays throughout the system. "Multiple functions coexist in one system, and changes to one area would have a wide impact, which made maintenance issues more apparent. Until now, we had focused on development efficiency and consolidated functions into "DSAS," but we decided to separate data integration function and build a separate system (Yokobori)."
Although the design was already underway, with the assumption that it would be developed from scratch, the decision was made to implement DataSpider.
There are two types of data integration processing: API processing, which acquires data from other systems in real time, and batch processing, which synchronizes large volumes of data. API processing was implemented using an API Gateway product, while batch processing was originally planned to be developed from scratch. "We had actually progressed to the design stage, when someone in the company introduced DataSpider to us, saying, 'Maybe we can use this,'" (Mr. Yokobori) Scratch development of batch processing involves formatting data acquired via SQL or file integration, and registering it in the linked database using SQL. Development must take into account differences between each database, but DataSpider can automate the entire process. "I was surprised to find such a tool, but I felt that if we included the testing process, we could reduce development man-hours by 30 to 40 percent" (Mr. Yokobori)
Normally, tools are not introduced once a project has progressed to the design phase. However, after the development engineers actually tried it out, they determined that DataSpider would improve development efficiency. They quickly decided to proceed with verification for its introduction. "If the tool was not usable, there would be a risk that we would have to redevelop it, but with scratch development, the process is clear and it is more reliable. Nevertheless, we prioritized the decision of the engineers who wanted to use DataSpider," says Yokobori.
A carefully crafted tool that allows for efficient development and is highly user-friendly
Consideration began in May 2020, and after basic operational testing, actual development began in July. The first process took time to create, as they had to check log output methods and how to handle errors, but as they progressed to the second and third processes, development became faster. "We can explain which items are linked to which other items on the DataSpider screen, so it's easy to share information and hand over between members. Detailed design documents are no longer necessary, and specifications can also be output, which has significantly reduced the time and effort required to create documents. Error handling, which we previously had to create one by one, is now pre-prepared in DataSpider. We're still verifying the finer points, but we hope that it will be enough to handle the situation," says Takezawa.
DC/DX Business Promotion Headquarters Service Infrastructure Management Department System Development Section
Daisuke Takezawa
Management issues were also resolved with DataSpider's functions. As with general system development, there are development, staging, and production environments, and each environment has a different connection destination. When scripts are copied from the development environment to the staging environment, and from the staging environment to the production environment, the connection destination information must be replaced, but doing it manually raises the risk of making a mistake. "I thought this was a common case, so I contacted support wondering if there was some kind of solution... and they told me about a function called 'Type,' which allowed me to easily switch the connection destination," says Takezawa.
DataSpider's ease of use has also been highly praised. "The help system is very detailed, so even if you have trouble using it, you can usually refer to it and solve the problem yourself. It's convenient to be able to solve the problem on the spot without having to go through the trouble of contacting support every time.
The table mapping screens are easy to understand, and the menus and screen transitions work just as we expected. I feel it's a carefully crafted tool (Takezawa)."
In addition to expanding to other systems, we are also considering expanding it as our own in-house service.
Currently, about half of the 20 processes have been completed, with a focus on data integration with MySQL. Aiming to put it into production by the end of the year, construction of the production environment is also underway in parallel. Going forward, they plan to consider utilizing REST and file integration, and hope to utilize functions such as scheduled execution to serve as a hub for operational management jobs that go beyond data integration.
This data integration also connects private clouds, public clouds, and SaaS, and there is growing demand from customers for such multi-platform solutions. "As we move to multi-platform solutions, the need for data integration will inevitably increase, and we feel that DataSpider is an effective tool in this regard." DataSpider is able to meet these customer needs, and the company aims to provide services based on its own implementation track record and know-how.
ITOCHU Techno-Solutions Corporation
As a comprehensive IT services company, we provide total support from consulting to design, development, operation and maintenance. We provide optimal services in a wide range of fields by combining IT solutions and cloud services that are suited to each. Our mission is to "challenge the possibilities of IT that will change tomorrow and contribute to the realization of a prosperous society full of dreams," and we pursue cutting-edge technology, improve the quality of our services, and contribute to our customers and society.
Sales partner: CTC SP Co., Ltd.
- The content of this case study is current as of the time of the interview. The content of this case study may change without notice.