Streaming Data from Azure Blob Storage using Cloud Engine
My use case is a need to take an Excel or AVRO file from Azure Blob Storage and load it into a SQL Database. In the past, when working with a Talend Remote Engine, I would use a tAzureStorageGet component to download the file locally. Then a separate component to open the local file and stream it into my destination. However, I have a new requirement to use a Talend Cloud Engine rather than a Talend Remote Engine. As such, I don't believe I have access to any local file system (please correct me if I'm wrong). Is there a component or technique to stream the data from an Azure Blob Storage object without first downloading it locally?
Hi
You can still access local file system when using Talend Cloud Engine, such as '/tmp/', and you can delete the file after it is processed in the end of job. Can you try and let me know if it works?
