topic Re: When we use the tMap function is allowed 100 rows at the time? in Talend Studio

When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Sat, 16 Nov 2024 09:00:25 GMT

When we use the tmap Function is allowed 100 rows in the one CSV file upload the data in the MySQL database on the particular table, when we use that table of one column is (int "Auto increment") column. How to solve this insert data in database, i have attached the sample screen shot.

We need to insert the All records into the database.

Re: When we use the tMap function is allowed 100 rows at the time?

TRF — Tue, 28 Nov 2017 07:34:41 GMT

Hi,

You probably have a join or a filter in the tMap.

That could be the reason why you have less rows on the output side than on the input side.

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 07:54:41 GMT

Yeah we use the JOIN in tmap, i attached the Screen flow but we create in the database in that particular table we use "Auto increment " in on column, only 100 rows inserted into that table.

Can you guide me the further process,

In Database , we use the one column structure

| salesorderid | int(11) | NO | PRI | NULL | auto_increment |

Re: When we use the tMap function is allowed 100 rows at the time?

TRF — Tue, 28 Nov 2017 08:00:36 GMT

As you can see from your 1st image, only 100 rows come out from the tMap which means 45 rows don't match with the join condition.

In this case, these rows are rejected and only 100 are pushed to the database.

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 08:57:07 GMT

Its maybe after get 100 rows its auto commit , then after 45 rows is reject when check the database .

But we used commit after the "OnComponentOk" only .

Re: When we use the tMap function is allowed 100 rows at the time?

TRF — Tue, 28 Nov 2017 09:07:18 GMT

Once again, you have only 100 rows on the tMap output flow which means 45 rows are rejected by the inner join in your tMap.

No other reason to look after.

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 10:07:27 GMT

Yes i accept that , but first time we need insert the all records 145 rows in database,and lookup i check the every row record in the database table . Now no data in that particular table so we insert the all data ,but after every 100 rows data is rejected . why reject after 100 Rows data ,i don't understand that flow ??

I check the another data file also same thing happen.

First time only we insert the all data . can you guide me why correctly 100 rows only accept and other data is reject .

we checked only database in lookup . can you guide me

Re: When we use the tMap function is allowed 100 rows at the time?

TRF — Tue, 28 Nov 2017 10:17:02 GMT

What's the content of the temp_salesorder_list table?

What's the content of the OD_FILE file?

What's the configuration of the tMap?

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 10:43:17 GMT

Content of the temp_salesorder_list table:

QUERY :

"SELECT distinct(salesorderNo) FROM temp_salesorder_list where salesorderNo='"+((String)globalMap.get("OrderNumber"))+"' and fileName ='"+((String)globalMap.get("FileName"))+"'"

content of the OD_FILE file:

SAMPLE DATA:

1115|In - INDIA|0000001115|5100408707|20171021|124540|ZOR1|1000|02|0006490025|TEST NAME|SP 7/1476, OUYH|PLCADE|691581|11|INDIA|1115|606|TS41|INDIA|010001|Interior|0100010172|GOLD|000000017210101000|10 L WHITE|10.000|1.000|1975.00

5100408707 - salesorderNo

configuration of the tMap :

below 100 row data is insert successfully,

Re: When we use the tMap function is allowed 100 rows at the time?

Anonymous — Tue, 28 Nov 2017 10:59:10 GMT

Hi can you try to unckeck this box (highlighted with yellow) in advanced setting please.

SGV

Re: When we use the tMap function is allowed 100 rows at the time?

TRF — Tue, 28 Nov 2017 11:09:59 GMT

@Ashok200792, this is not exactly the same case.
You have no rows in the input table so every records from the file are rejected by the inner join, catched on the output flow and inserted into the output db.
Also, you have only 99 records in the input file, not 145 as in the previous example.

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 11:49:03 GMT

i don't know how to figure that , i try another file for example also

@sgv

I try that its return only 2 rows .

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 12:39:55 GMT

mysql> select * from temp_salesorder_list\G
Empty set (0.24 sec)

(100 below OR equal 100 rows ) before we Lookup the database (We check monitored), check 0 rows return

(100 rows )After we Lookup the database (We check monitored) ,check 1 rows return

I share the before pictures return 1 rows from the Lookup (temp_salesorder_list) ,it database we never insert the exit data.

Empty set (0.24 sec) --- No data

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Tue, 28 Nov 2017 13:52:23 GMT

We used tMysqlinput everytime check the row in database(initially zero record in the table )
its allowed 100 rows after that every data is rejected,

After that 100 rows completed i will checked the row is database ,its not commit return ( Empty set (0.24 sec) --- No data)

can you explain why that other record or rejected ??

Any alternative solution for this?? can you guide me . we struck in couple of days .

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Wed, 29 Nov 2017 05:35:21 GMT

Its rejected , but in MySQL database Empty set (0.24 sec) --- No data in that table, but we get the 1 rows to check after 100 rows so its goes to rejected flow.

Can you guide me.

Re: When we use the tMap function is allowed 100 rows at the time?

Ashok200792 — Fri, 29 Dec 2017 09:27:52 GMT

Finally we get the solution for this topic , but i don't know how the data after 100 rows its goes rejected without commit the mysql , matching data from Mysql database we added in the query one more where condition to pass that time its working very fine .

Thanks for support team,

By
ASHOK M