Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
badri-nair
Contributor
Contributor
‎2019-02-07 07:44 AM

Hadoop hdfs block size

I am unloading data from oracle database into Hadoop.

landing the file as as HDFS file. But the unloaded file is a single file 750 MB in size.

Can anyone please help me understand why its acting so. I would like to have each part file 128 MB in size.

where do i have to update the settings ?


Found 2 items
-rw-r--r-- 3 hdfs supergroup 0 2019-02-07 07:37 /var/lib/hadoop-hdfs/lake/itineraryitem2/_SUCCESS
-rw-r--r-- 3 hdfs supergroup 730087184 2019-02-07 07:37 /var/lib/hadoop-hdfs/lake/itineraryitem2/part-00000

 

Labels (4)
Labels
294 Views
0 Likes
1 Reply
manodwhb
Champion II
Champion II
‎2019-02-07 07:53 AM

@badri-nair ,check below link will help you to understand.

https://www.talend.com/blog/2018/04/12/apache-spark-performance-and-tuning-blog/

294 Views
0 Likes