
Anonymous
Not applicable
2010-09-17
11:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Metadata driven ETL - a future "Talend ETL Architect" tool
I believe that meta-data driven ETL is the next wave. Products that embrace it will thrive.
I love Talend's flexibility, but feel that a meta-data definition, configuration management (connections to physical elements), business rules and a transformation repository, is needed. This is all part of meta-data mangement that an architect would do, NOT an ETL developer. These structures would be defined in a higher layer - I.E. an "Talend ETL Architect" (new product, hint hint). This layer resolves all configuration, and physical connections to databases, files, flat files, etc, inside it. Now, the development layer is decoupled from the physical layer (read that again).
Once done, the actual ETL developer does not need to know about the underlying physical layer - that is done by the architect at that higher level. Now, the developer can use the eclipse visual development, and draw on components that connect to the higher level meta-data elements ONLY. This decoupling of the "Architect" layer from the "Developer" layer will lead to much higher level developer productivity, and the ability for the architect to work with a business analyst at a higher level to define the meta-data layer in a "Talend ETL Architect tool".
This could be huge for Talend, in my opinion - I hope you consider it.
Dave
I love Talend's flexibility, but feel that a meta-data definition, configuration management (connections to physical elements), business rules and a transformation repository, is needed. This is all part of meta-data mangement that an architect would do, NOT an ETL developer. These structures would be defined in a higher layer - I.E. an "Talend ETL Architect" (new product, hint hint). This layer resolves all configuration, and physical connections to databases, files, flat files, etc, inside it. Now, the development layer is decoupled from the physical layer (read that again).
Once done, the actual ETL developer does not need to know about the underlying physical layer - that is done by the architect at that higher level. Now, the developer can use the eclipse visual development, and draw on components that connect to the higher level meta-data elements ONLY. This decoupling of the "Architect" layer from the "Developer" layer will lead to much higher level developer productivity, and the ability for the architect to work with a business analyst at a higher level to define the meta-data layer in a "Talend ETL Architect tool".
This could be huge for Talend, in my opinion - I hope you consider it.
Dave
162 Views
0 Replies
