Last Update:
Jan 22, 2024 9:35:30 PM
Updated By:
Created date:
Jul 17, 2020 11:42:25 AM
The Big Iron, or as most of us call it the Mainframe. Yes, those legacy systems of the past that we always hear about in movies that control some devious or super powerful weapon. That very well could be true, but they also control a lot in our daily lives. Every time we go to an ATM or make a purchase at a grocery store, and that one time a year we all love…yes, tax season.
It is amazing how these systems that we now only read about or hear about in movies still run much of our lives. You check your balance on that smart phone…yes, more than likely your request got processed by a mainframe. We simply cannot live without them yet the thought of having to interact with one can seem like a fate worse than death.
How do we get data from these mythical machines? Or better yet, how do we get data in? This is simply not something that is done by firing up your IDE and typing out some code. Let’s also not forget about the decades of business logic that is in those Cobol programs. What is Cobol you ask? I will let you Google that to investigate.
Enter OpenLegacy, a company that has made the integration to the mainframe and many other applications (i.e. VSAM, IMS, AS400, ADABAS) as easy as connecting to an API. The OpenLegacy product can scan your Cobol applications and provide an API that will allow any application to seamlessly and easily communicate with your backend legacy systems. This API gives a lot of advantages that I wish I’d had many years ago when I was a young developer working at the banks.
Back then, I would have to get in contact with someone working on the mainframe that knew the data I wanted to work with, and then hope that they had a program already built that would extract all the data I required. Then we had to come up with how they would get the data to me. Now, you just need to have a bit of code from OpenLegacy installed on the system and you can go about your business.
The architecture is very simple. Utilizing the Talend Component Kit (TCK) along with OpenLegacy SDK, the OpenLegacy team was very quickly able to create components that can interact with the OpenLegacy Microservices/APIs to integrate with the backend legacy system. They first utilized the OpenLegacy discovery service to get a list of available services (i.e. Cobol applications, etc.) and present that to the user. The user will then select which service they want to interact with, and the OpenLegacy platform generates a metadata structure. That structure is what will be used with Talend to communicate back to the OpenLegacy services to then process the data we want from the legacy system.
The second component in this Job is the parser. It will take this raw Cobol program and the execution path and turn them into the metadata that will then be utilized when communicating to the OpenLegacy service on the legacy backend server. In the screenshot below, you can see a snippet of the metadata that is being generated.
The Job in the image below shows how, by taking the metadata from the previous task, you can then pass in the record id. When called against the mainframe system the legacy backend server will return that record from the mainframe to Talend. You can use this component, along with another system (RDBMS, Application, etc.), to get a list of all the accounts that you want to read from your legacy system. Those accounts can then be joined to the legacy record and produce a current view of the data in real-time and not a long delayed nightly extract that would be sent over to you.
You would now be able to keep your Cloud applications/data warehouses in sync with your back end legacy systems in real-time with just a few steps in your Talend Data Fabric. This will remove the burden on your legacy system and developers, as well as your front end or application developers working on their next generation system for your business.
It is amazing how these systems that we now only read about or hear about in movies still run much of our lives. You check your balance on that smart phone…yes, more than likely your request got processed by a mainframe. We simply cannot live without them yet the thought of having to interact with one can seem like a fate worse than death.
How do we get data from these mythical machines? Or better yet, how do we get data in? This is simply not something that is done by firing up your IDE and typing out some code. Let’s also not forget about the decades of business logic that is in those Cobol programs. What is Cobol you ask? I will let you Google that to investigate.
Enter OpenLegacy, a company that has made the integration to the mainframe and many other applications (i.e. VSAM, IMS, AS400, ADABAS) as easy as connecting to an API. The OpenLegacy product can scan your Cobol applications and provide an API that will allow any application to seamlessly and easily communicate with your backend legacy systems. This API gives a lot of advantages that I wish I’d had many years ago when I was a young developer working at the banks.
Back then, I would have to get in contact with someone working on the mainframe that knew the data I wanted to work with, and then hope that they had a program already built that would extract all the data I required. Then we had to come up with how they would get the data to me. Now, you just need to have a bit of code from OpenLegacy installed on the system and you can go about your business.
Talend / OpenLegacy Architecture
The architecture is very simple. Utilizing the Talend Component Kit (TCK) along with OpenLegacy SDK, the OpenLegacy team was very quickly able to create components that can interact with the OpenLegacy Microservices/APIs to integrate with the backend legacy system. They first utilized the OpenLegacy discovery service to get a list of available services (i.e. Cobol applications, etc.) and present that to the user. The user will then select which service they want to interact with, and the OpenLegacy platform generates a metadata structure. That structure is what will be used with Talend to communicate back to the OpenLegacy services to then process the data we want from the legacy system.
Talend Job with OpenLegacy
As of this writing, the Talend and OpenLegacy component has two distinct processes: first there is the Design Time, and then the Run Time. To help make this more visual, we have broken these out into two Talend Jobs that show how this works.Design Time
This Job works with the OpenLegacy APIs to show how they take a Cobol program along with its execution path, and then translate this into the metadata that will be used when making the call to extract data. In the image below, we have manually taken a Cobol program and entered it into our component, along with the execution path of ITEMDET.The second component in this Job is the parser. It will take this raw Cobol program and the execution path and turn them into the metadata that will then be utilized when communicating to the OpenLegacy service on the legacy backend server. In the screenshot below, you can see a snippet of the metadata that is being generated.
Run Time
The Job in the image below shows how, by taking the metadata from the previous task, you can then pass in the record id. When called against the mainframe system the legacy backend server will return that record from the mainframe to Talend. You can use this component, along with another system (RDBMS, Application, etc.), to get a list of all the accounts that you want to read from your legacy system. Those accounts can then be joined to the legacy record and produce a current view of the data in real-time and not a long delayed nightly extract that would be sent over to you.
You would now be able to keep your Cloud applications/data warehouses in sync with your back end legacy systems in real-time with just a few steps in your Talend Data Fabric. This will remove the burden on your legacy system and developers, as well as your front end or application developers working on their next generation system for your business.
