How to fetch a file from Box and load it on the DB which is in the Local System????

[ankit7359]

Hi,

I am bit confused about a scenario - 

i have uploaded a file on BOX and now i would like to download the file and do some operations like updating the file etc.. and then i would like to insert into DB. 

I do have a rough idea like - 

tBOXLIST(Details to be given here) -------->>>>LOCAL_DIRECTORY(FILE TO BE DOWNLOADED)---------->>>>>Transformations Logics(tmap or other components)--------->>>>>>>To be loaded to DB(DB_OUTPUT_Component).

Can anyone help me on this ???

Are there any videos/docs such that i can refer to -> for loading data from cloud to DB..

Pls reach out,if necessary.

Thanks,

Ankit.

[Anonymous]

Hi Ankit,

     Your thought process is right.

      You can bring multiple files from Box to local as first stage. Then use tFilelIst and other usual components to process each file and load them to DB.

The best place to look for examples is always help.talend.com

https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/hj2FnNv1btiSRqPm2wWjrQ?section=sect-components...

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[ankit7359]

Hi @nthampi,

Thanks for your reply...

But.. i came across another workaround like using HTTP request.. if i were to use HTTP request then how should i design and are there any precautions i must be aware of.

Thanks,

Ankit

[Anonymous]

Hi @ankit7359,

     Could you please provide more details about the workaround using http mode?

Warm Regards,

Nikhil Thampi

[ankit7359]

hi @nthampi,

Thanks for the reply.. 

well i was thinking what if we use tHttprequest component and loading the component with box details like access token and etc..  and then using get method to pull files from the box and then placing those files in remote FTP server and then taking the file and loading to DB.

tHttprequest(BOX-link)------>>>COMPONENT(GET METHOD)*-------->>>>PLACING THE FILES IN THE REMOTE FTP SERVER(TFTPCONNECTION)------->>>>TFTPGET(PULLING THE CERTAIN FILES AS PER FILEMASK)*------>>>>TFILELIST(ITERATING THE FILES AS PER FILEMASK)------>>>>TFILEINPUT(AS PER FILEMASK)------>>>LOADING TO DB...

I am not sure of the above workaround.. but i am tryin to explore what are the other ways it could be achieved and keeping performance parameters aside.

Pls do suggest and give your opinions on the above.

Thanks,

Ankit.

[Anonymous]

Hi Ankit,

      End of day, the Box components of the Talend are also making API calls to move the data between Box and your local server. The method you are using above is also doing the same stuff but you are calling the Box APIs through generic HTTP request components instead of Box native calls shared by Talend.

     If you notice, you are making multiple hops for same files in above methodology. So my recommendation would be to use the native Box components of Talend and complete the processing.

      Simple, easy and widely used by many Talend customers 🙂

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[ankit7359]

hi @nthampi,

Thanks again for the reply.

I do have a query for this.. for example.. what if we use box component but my admin user says that we cant give the access tokens and other credentials as they hold sensitive data...???

What are your suggestions on this Hypothetical scenario.... the only reason i am asking these many questions is i want to cover all the gaps.. like a workaround solution for the every scenario...

Pls do give your suggestions on this..

Thanks,

Ankit

[Anonymous]

Hi Ankit,

    If we have to connect to a service like Box, we need to definitely give the connection parameters shown below.

We cannot escape from that option. You need to safely keep the credentials and pass them upon request by Talend job.

Seems we are deviating from the original topic 🙂

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[ankit7359]

hi @nthampi,

the reason why i asked about using httprequest is because.. not always we get access from admin team and even if we get it then it is limited.. keepin that in mind.. if we were to build this scenario without box componnets how would we do it.. ??

this is the reason, i was asking question which arent straight but around the loop..

Thanks,

Ankit

[Anonymous]

Hi Ankit,

     I believe it is a Box support related query. Even if you are using http request my understanding is that you will still come across the point where you will have to pass the user credentials. In that case also I believe your admin team has to provide the credentials to connect to Box.

     But if you still want to create a component of your own, you will have to build them using the Box reference guide shown below. I would suggest to contact Box team to provide the best practices to you in this area.

https://developer.box.com/reference

      The aim of Talend is to remove the underlying complexities of connecting a component. But developers who are interested in coding part are welcome to create there own components using custom components or by creating routines from their end. In this case, they will have to design and develop the components according to their specific need as Talend do not have any control on the flow.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂