Bypass header record parse

[Lloyd3]

We are using tFileInputDelimited to parse a flat file. The file has 5 record types as defined in the first two characters of the record, i.e. H1, H2, H3, D1, T1. I only need data from the D1 records and I have that schema defined in the File Delimited Metadata. In the tMap I have a filter that only selects D1 records. However, the parse fails because the H and T records are still being parsed and the fields in those records have different data types so I get data type mismatch errors. Because of the way multiple files may be concatenated, these H and T records can appear anywhere in the file.

How do I bypass and discard those records?

BTW this issue is also having a negative effect on other delimited files that have a header that does not have a record type but has some clearly identifiable feature that should enable us to discard the record. As above, the header might be in the file or may not.

[Anonymous]

Hi

tFileInputDelimited can't filter data, we usually filter the data using other component such as tMap, tFilterRow. Can you show us an example of data? the record type 'H1' is contained in the value? If so, read the data with string type and do the filter, then convert string type to other data types if needed.

Regards

Shong

[Lloyd3]

Yes, we could read the data as string but there many fields of different types (date, number, etc) that it seems to be a hack to do it that way rather than just having a simpler way to discard a record without throwing a parse error. I was not clear on the filtering I am using, it is in tMap, not fFileInputDelimited. I was not able to get the tFilterRow to work; not sure if it would go between tFilterInputDelimited and tMap or just before tFilterDelimited.

The schema for the input file is based on the D1 record. The H1, H2 and H3 records need to be discarded. Records H2 and H3 are failing because the text in the column does not match the D1 schema.