How to copy data from S3 bucket to Redshift database

[Anonymous]

Hi
Which component I should use to copy the uploaded files on s3 bucket to the redshift database.If possible please share screenshot.

[Anonymous]

Hi ashish.s,
So far, there is no Amazon S3 component in latest product(5.3.0). Please see the related forum https://community.talend.com/t5/Design-and-Development/Talend-to-Amazon-S3/td-p/108299 and vote for the jira issue Support to integrate with Amazon S3 (Simple Storage Service)
Best regards
Sabrina

[Anonymous]

Hi sabrina
i got s3 component through this http://www.labdecisionnel.com/index.php?option=com_content&view=section&layout=blog&id=12&Itemid=144...
Ysance_ComposantsS3.zip file I installed it in my studio but i am not finding any component by which we can write copy command like( copy events
from 's3://$MY-BUCKET/PATH/TO/FILES/FOR/UPLOAD'
credentials 'aws_access_key_id=$ACCESS-KEY;aws_secret_access_key=$SECRET-ACCESS-KEY'
delimiter '\t' as per the AWS doc to load data in redshift database
Thanks
Ashish.s

[Anonymous]

Hi,

i got s3 component through this http://www.labdecisionnel.com/index.php ? mitstart=5
Ysance_ComposantsS3.zip file I installed it in my studio but i am not finding any component by which we can write copy command like( copy events
from 's3://$MY-BUCKET/PATH/TO/FILES/FOR/UPLOAD'
credentials 'aws_access_key_id=$ACCESS-KEY;aws_secret_access_key=$SECRET-ACCESS-KEY'
delimiter '\t'wink as per the AWS doc to load data in redshift database

It neither is a talend component nor is talend custom component. So have no idea about that. I have made a search on talend exchange portal and found some custom components about S3 http://www.talendforge.org/exchange/index.php?page=2&product=tos&nav=1,1,1, perhaps it will give a help to you. However, Components available in Talend Exchange are developed by the Community thus are not available natively in the Studio, you need to install them.
Best regards
Sabrina

[Anonymous]

Hi
Solved the problem through using tpostgresqlrow component and added copy command in query.

Thanks
Ashish.s

[Anonymous]

Hi,
And by the way, you have the tRedshiftRow which allows to do the same thing.
Cheers,

[Anonymous]

Hi
Solved the problem through using tpostgresqlrow component and added copy command in query.

Thanks
Ashish.s

hey how did you write the copy query.did u write through the query or the copy command in cmd

[Anonymous]

Thanks it worked

[Anonymous]

!SESSION 2013-08-27 09:26:57.443 -----------------------------------------------
eclipse.buildId=unknown
java.version=1.7.0_25
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments: -os win32 -ws win32 -arch x86
This is a continuation of log file C:\Talend\TOS_DI-Win32-r104014-V5.3.1\workspace\.metadata\.bak_0.log
Created Time: 2013-08-27 09:27:00.579
!ENTRY org.eclipse.osgi 2 0 2013-08-27 09:27:00.579
!MESSAGE The following is a complete list of bundles which are not resolved, see the prior log entry for the root cause if it exists:
org.osgi.framework.BundleException: Exception in org.eclipse.core.resources.ResourcesPlugin.start() of bundle org.eclipse.core.resources.
Root exception:
org.eclipse.core.internal.dtree.ObjectNotFoundException: Tree element '/SAS_STOCKCHECK/process/SFTP_ETL_MIGRATION_0.1.screenshot' not found. at org.ecli

To see the whole post, download it here
OriginalPost.pdf

[Anonymous]

I am running into trouble when I am running TOS on server. On local host it is running fine..Can anyone look into it and let me know the solution soon or should I re install it
!ENTRY org.eclipse.osgi 4 0 2013-08-27 09:29:47.387
!MESSAGE Application error
!STACK 1
java.lang.RuntimeException: The service org.talend.core.model.migration.IMigrationToolService has not been registered.
at org.talend.core.GlobalServiceRegister.getService(GlobalServiceRegister.java:148)
at org.talend.rcp.intro.Application.start(Application.java:84)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:619)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:574)
at org.eclipse.equinox.launcher.Main.run(Main.java:1407)