Not able to single insert 10k rows with 20 column each to Redshift

kaps1602 — Sat, 16 Nov 2024 02:21:45 GMT

I have around 2 million rows to be inserted to Redshift and each row has 20 columns. I have configured number of rows per insert 10000 and commit every 10000. Job get failed with error : java.sql.SQLException: [Amazon](500310) Invalid operation: there is no parameter $-32768;. Exception trace mentioned at the end.

Than I reduced number of rows per insert and commit every to 1000. It started working fine, but will takes lots of time to insert 2 million record.

How shall I quickly insert my data to Redshift?

Exception in component tRedshiftOutput_1 (CSV_RemoveNewLineFromElement_CSV)
java.sql.SQLException: [Amazon](500310) Invalid operation: there is no parameter $-32768;
at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source)
at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source)
at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source)
at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source)
at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source)
at com.amazon.redshift.client.PGMessagingContext.getParameterDescription(Unknown Source)
at com.amazon.redshift.client.PGClient.prepareStatement(Unknown Source)
at com.amazon.redshift.dataengine.PGQueryExecutor.<init>(Unknown Source)
at com.amazon.redshift.dataengine.PGDataEngine.prepare(Unknown Source)
at com.amazon.jdbc.common.SPreparedStatement.<init>(Unknown Source)
at com.amazon.jdbc.jdbc41.S41PreparedStatement.<init>(Unknown Source)
[statistics] disconnected
at com.amazon.redshift.core.jdbc41.PGJDBC41PreparedStatement.<init>(Unknown Source)
at com.amazon.redshift.core.jdbc41.PGJDBC41ObjectFactory.createPreparedStatement(Unknown Source)
at com.amazon.jdbc.common.SConnection.prepareStatement(Unknown Source)
at com.amazon.jdbc.common.SConnection.prepareStatement(Unknown Source)
at tcpa_data_project.csv_removenewlinefromelement_csv_0_1.CSV_RemoveNewLineFromElement_CSV.tFileList_1Process(CSV_RemoveNewLineFromElement_CSV.java:4487)
at tcpa_data_project.csv_removenewlinefromelement_csv_0_1.CSV_RemoveNewLineFromElement_CSV.tFileInputDelimited_6Process(CSV_RemoveNewLineFromElement_CSV.java:1019)
at tcpa_data_project.csv_removenewlinefromelement_csv_0_1.CSV_RemoveNewLineFromElement_CSV.runJobInTOS(CSV_RemoveNewLineFromElement_CSV.java:6198)
Caused by: com.amazon.support.exceptions.ErrorException: [Amazon](500310) Invalid operation: there is no parameter $-32768;

Re: Not able to single insert 10k rows with 20 column each to Redshift

kaps1602 — Mon, 25 May 2020 12:04:21 GMT

Anyone has anything on this issue.

topic Re: Not able to single insert 10k rows with 20 column each to Redshift in Talend Studio

Not able to single insert 10k rows with 20 column each to Redshift

Re: Not able to single insert 10k rows with 20 column each to Redshift