Below is my requirement, for which I need your help.
I need to extract the files from EC2 (a remote server) using the tscp component and zip all the files into one common folder and place zip into target. I don't see any archive components for scp.tscpfilelist, like tfilelist, does not have the currentfile and currentfilepath variables.
Can anyone help me with this design? How can I achieve this using the SCP component?
I've already done it this way, but the client does not require it because the job server is not on EC2. Job server is running on premise. I can't retrieve it into a folder because, again, those will get placed on the job server, which we don't want. We wanted to directly pull the files and zip and place them into the target.
OK, well another way to go with this is to SSH into the EC2 system (using the tSSH component), zip them there and then download the zip files. Or you could find another location to load the files to, zip them there and then do the rest. You could also try to build your own process (using Java) to zip them in flight. Unfortunately there isn't a component that is built-in to the product that will do this....but you can build one.
There is no reason that I know of. You have to keep in mind that to have a component to achieve every single use case on its own, would require an inordinate number of components. The components that are supplied enable you to achieve most use cases and there is also the ability for you to write your own components.
Requiring that a set of files are zipped not at source and not at target is not a common use case. The zipping process requires that files are all present in a location so that every set of common bits can be analysed to identify sections that can essentially be replaced with a smaller combinations of bits. Why can't the files not be copied to a location on the source or target, zipped, the copies deleted and then the zipped files be moved?
