Help on Partial Load

[priyarane]

Hi Community,

Can some one help one partial for the below example.

Out of thre only one table will effect each run.

Table1:
LOAD * INLINE [
Key1, Value1
abc, 200
def, 234
];



Table2:
LOAD * Inline [
Key1, Value1
D, 100
E, 200
F, 200
];



Table3:
LOAD * INLINE [
Key1, Value1
G, 100
H, 200
I, 100
];

_priya

[priyarane]

new records should flow in Table1 and old data should go awaly like replace

[avinashelite]

then every time do a fresh/full reload on the table 1.....so that every time it gets the values which are presently in that table ...

[priyarane]

as of now there more than 10 tables we are loading in the datamodel for one of 10 updation not every thing.

so we thought of do the partial load(encourage any other model as well)

Note: same column names across tables

[avinashelite]

In that case do a resident load of the master table and delete the content of table 1 each time and freshly load the new data every time . In order to identify the data (since every column is same it get merged into 1 table ) add a source Identifier like

Table1:

LOAD *,

'Table1' as Source;
LOAD * INLINE [
Key1, Value1
abc, 200
def, 234
];

Table2:

'Table2' as Source;
LOAD * Inline [
Key1, Value1
D, 100
E, 200
F, 200
];

Table3:

'Table3' as Source;
LOAD * INLINE [
Key1, Value1
G, 100
H, 200
I, 100
];

NoConcatenate

Master:

LOAD *

resident

Source<>'Table1';

Drop table tablename;

Table1:

ADD only

LOAD *,

'Table1' as Source;
LOAD * INLINE [
Key1, Value1
abc, 200
def, 234
];