How to handle the new columns added to the existing excel source file over the time
Hi All,
Currently we have a process where the job read the data from excel, apply some business logic on the data and load them into the destination table. However yearly once, the vendor sends a new file with few more columns gets added at the end of the existing columns and the new format stays for a year.
Right now, Yearly once, we reload the input (Excel sheet)& output (database table schema) metadata to the new format, the job works without any issues.
Now we are looking for a permanent solution where the job need to detect the new columns both from the excel and also in the destination table and work without any issue.
Right now all the columns are integer data type and future new columns also integer data type.
