topic Re: [resolved] tLoop with java.util.zip.ZipEntry in Talend Studio

[resolved] tLoop with java.util.zip.ZipEntry

Anonymous — Sat, 16 Nov 2024 13:49:52 GMT

Hi,
I am trying to process zip files "on the fly" which means that I want to get the files in the zip file one by one and process them without unziping the full archive. This means that I need to loop trough the zip file.
I used the tLoop component and set us the following values:
1.Declaration:"java.util.zip.ZipInputStream z = new java.util.zip.ZipInputStream(new java.io.BufferedInputStream(new java.io.FileInputStream("D:/myfile.zip"))); java.util.zip.ZipEntry e = z.getNextEntry()"
2.Condition:"e!=null"
3.Iteration:"e = z.getNextEntry()"
The loop seems to be working fine because when I print in a tJavaFlex component ((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) it loops exactly as the number of files in the zip file.
My problem is now how to get the ZipEntry value e and process it further (create temporary file, process that file and thzen delete it before moving to the next entry in the zip file).
The tLoop component provides another out value ((Integer)globalMap.get("tLoop_1_CURRENT_VALUE")), but it is of type Integer and I need to get a ZipEntry as an output.
Can someone please help me with some ideas?
Thanks
Magi

Re: [resolved] tLoop with java.util.zip.ZipEntry

Anonymous — Thu, 13 Aug 2009 06:35:04 GMT

Hello
The tLoop componnent use the java logic: for loop or while loop. In you case, it is better to create a routine to exract the files from a zip file and write them into a folder, then use a tFileList to iterate each file, process them, finally, use a tFileDelete to delete the files.
Here I create a routine which can be used extract files from a zip file to a folder.

package routines;
import java.io.*;
import java.util.zip.*;
public class forum7756 {
	static final int size = 4096;
	public static void extractingFileFromZip(String outputFolder) {
		String path = outputFolder;
		try {
			BufferedOutputStream dest = null;
			FileInputStream fis = new FileInputStream("D:/ttt.zip");
			ZipInputStream zis = new ZipInputStream(
					new BufferedInputStream(fis));
			ZipEntry entry;
			while ((entry = zis.getNextEntry()) != null) {
				System.out.println("Extracting: " + entry);
				int count;
				byte buffer[] = new byte;
				// write the files to the disk
				String filePath = path + entry.getName();
				File outFile = new File(filePath);
				if (!outFile.exists()) {
					outFile.getParentFile().mkdirs();
				}
				FileOutputStream fos = new FileOutputStream(outFile);
				dest = new BufferedOutputStream(fos);
				while ((count = zis.read(buffer)) != -1) {
					dest.write(buffer);
				}
				dest.flush();
				dest.close();
			}
			zis.close();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

After creating a routine, you can call it on tJava like this:
forum7756.extractingFileFromZip("d:/myFolderPath/");
Best regards

shong

Re: [resolved] tLoop with java.util.zip.ZipEntry

Anonymous — Thu, 13 Aug 2009 07:00:24 GMT

Thank you. This is exactly what I wanted to avoid (it is may old solution). I cannot aford to extract all the files because of the nature of the project (too many files, too often and there is not enough space to extract and then process).
Isn't there a way to just get the content of the ZipEntry from the loop?
Thanks
Magi

Re: [resolved] tLoop with java.util.zip.ZipEntry

Anonymous — Thu, 13 Aug 2009 07:06:51 GMT

Hello

Isn't there a way to just get the content of the ZipEntry from the loop?

the current value or the current iterate of tLoop are Integer. So, it is not the correct way to do that with tLoop component.
If you just want to return a ZipEntry object, we can return it in routine.
Best regards
shong

Re: [resolved] tLoop with java.util.zip.ZipEntry

Anonymous — Thu, 13 Aug 2009 13:07:32 GMT

Actually, I made it work with tSetGlobalVariable, tJava, tLoop, tFixedFlowInput, tJavaRow ...
So, it is possible to unzip on the fly. Just set 1 global ZipFile, 1 global ZipEntry array and a counter and it works.