Field length with dynamic schema columns

[Anonymous]

I'm currently attempting to use the Dynamic schema settings for columns (in a tFileInputDelimited) to read in a dynamically structured file and then add it into a staging table in an MS SQL Server DB before further processing.
This seems to work fine as it is but I'm losing rows through truncation whenever a field is over 100 characters. Is there a setting somewhere that I can override this 100 character limit?

[Anonymous]

Hello @chanaka 

Could you please let us know if this related topic helps?

https://community.talend.com/t5/Design-and-Development/resolved-Generic-Talend-Job/m-p/92354

Best regards

Sabrina

[Anonymous]

Sorry to bump this, but does anyone have a simple solution to this?  I've read the various links and posts and nothing is helping me.  I am a SQL developer, not a Java developer and I am trying to work with a CSV that has over 500 columns I have received from a client.  I need to build the schema dynamically from the column headers, and then dump the data into a SQL table, so that the rest of my job can then select only the required columns to do the rest of the work I need.

It seems ridiculous that you cannot overwrite the column length for dynamic schema settings.  VARCHAR in SQL can handle 8000 characters.  Limiting this to 100 makes it almost useless. 

[Anonymous]

You can manually set the column length of dynamic columns.

Consider the following code snippet

Dynamic dyn = row1.dynamic_column;

for(int i = 0; i < dyn.getColumnCount(); i++){
	DynamicMetadata meta = dyn.getColumnMetadata(i);
	meta.setLength(5000);
}
row2.dynamic_column = dyn;

This will set them all to 5000

[Anonymous]

@evansdar 

FOrgive my ignorance, but where would I use that code? As I said in my post above, I'm not a java developer, I'm a SQL developer.  My talend job only has 4 components

DBConnection >>>  tFileInputDelimited >>> tMap >>> tDBOutput

tFileInputDelimited schema has one "Dynamic" type column.

[Anonymous]

You'll want to put it in a tJavaRow after whichever component the defined Dynamic Schema is in (looks liketFileInputDelimited in your job)

The name of the row connection is how the data will be passed into and out of the component.

[Anonymous]

@evansdar 

Brilliant.  Thank you so much.  That's all sorted now.