How to filter lines with Regex from an input directory and write to Kafka?
I'd like to read files from a directory and write the lines in each file that satisfy a regular expression into Kafka. I need to include the file name in the line written to Kafka. For e.g. if the file is /exp/talend/first.txt and it has a line first then the line to Kafka should be /exp/talend/first.txt:first. Now I use the tFileList component and connect it to tFileInputRegex component with an iterate connection. However when I give the context variable for the file name as "file Name Stream: ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))" the red error on the regex component does not go away and the task does not run. The documentation for the tFileList shows this as a global variable during flow execution.
Im not sure but you are escaping \" in your regex, make sure you escape properly because a \ slash is also an escape char in regex, so ... \\\ , three slashes to escape one slash, not 100% sure, but google a bit on escaping double quotes in java and regex
