Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I'm trying to do a drop and recreate all tables in Compose but the option is grayed out. We had a production failure on the backend and I'm trying to reload the tables using Compose.
Thanks,
Jessica S.
@jmsmith0507 try validating once and that should enable the 'drop and create' table option.
The other option is to use the Reset project options in Project settings and unselect everything and choose 'Storage table and files'
Let me know if you are still seeing the issue.
Thanks,
Nanda
Hi Nanda,
Thanks, doing the validate did not enable to the 'drop and recreate' option for me.
I tried your suggestion to reset project with 'storage tables and files' selected. I did a Create. And then I generated instructions for the task and validated the storage zone. However, after all of that it doesn't look like the Compose project is complete based on the green bar. Did I miss a step?
Thank you!
@jmsmith0507 Sorry for the delay.
From the screenshot, it looks like you are able to run the task for Full load, but the CDC task is giving you the "ETL task was not been generated" error, correct?
Please click on the clear cache button on the task command for both Full and CDC tasks, generate instructions and then 'Run' the task. Let me know how it goes.
Thanks,
Nanda
@jmsmith0507 It sounds like some kind of caching issue. I like to try one last thing before I ask you to open a case with support.
On the project menu under the setting could you please enable 'Generate DDL scripts but do not run them', clear the landing cache, and then see if that lets you generate the instructions for CDC and run it.
If that doesn't resolve the issue, I'll have to get on a call with you to get to the bottom of this issue. So, please open a case with support and the team should be able to find the root cause and fix it.
Thanks,
Nanda
No, unfortunately it did not work. At the moment, this isn't a high priority for my group. I'll open a case if this becomes a priority in the future. Thanks anyway.
Jessica S.
Hi @jmsmith0507
The other option is use Reset project options in Project settings and choose the following option/applicable options:
But this step will remove all the entities as well and we will have to add all of the tables again.
I'm assuming you are using Replicate as the source and if we do a full load on the Replicate it should trigger Full load on Compose as well and there is no need to drop and recreate the tables.
Thank you,
Hi,
Thank you for your response. I'm not very familiar with Replicate or Compose, but I see there is a Full load tab and a Change Processing tab for the task in question. I've attached the screenshot of it. The Full load was last done in 2021 and we update the tables in real time using the Change Processing from what I understand. I'm trying to get the Change Processing working. The only way I'm familiar with getting it back to working is to do a reload in Compose. If I follow your reset option, how can I add the tables back? My goal is to be able to do a drop and recreate all tables.
Thank you!
Jessica S.
The Replicate tasks is in error state as per the screenshot. If you click the dropdown next to Run, you will find the option Reload Target which will trigger a Full load on the Replicate. I'm assuming in Compose Landing connection we have checked the option 'Associate with Replicate' and selected the task. If this is the case once the Replicate completes the Full Load the Compose will initiate it's Full Load operation where it will truncate all the existing table data and do a fresh load.
On your question on getting the tables back, on the Storage Zone -> Metadata -> Discover and select all the required tables and continue further.
I think we can better assist you if you could open a support case and we can have a meeting and walk you through the steps.
Thank you,
@jmsmith0507 try validating once and that should enable the 'drop and create' table option.
The other option is to use the Reset project options in Project settings and unselect everything and choose 'Storage table and files'
Let me know if you are still seeing the issue.
Thanks,
Nanda
Hi Nanda,
Thanks, doing the validate did not enable to the 'drop and recreate' option for me.
I tried your suggestion to reset project with 'storage tables and files' selected. I did a Create. And then I generated instructions for the task and validated the storage zone. However, after all of that it doesn't look like the Compose project is complete based on the green bar. Did I miss a step?
Thank you!
@jmsmith0507 Glad to know resetting the project with 'storage tables and files' worked for you.
Your Compose project will not show a completely filled green bar until you run the task that you just generated the instructions. So, please go ahead and run the task and that should get you the complete project.
Let me know how it goes.
Thanks,
Nanad
Hi Nanda,
Thanks for your response. It says the storage zone task cannot be run due to the following reason(s): ETL task was not been generated...
Jessica S.
@jmsmith0507 The run option will not get enabled until you build the instructions. So, I think something else is going on with this project.
Could you send me the screenshots?
1. Project main page - this will have the 'landing and storage connection' and 'storage connection
2. Manage data storage task - this will have task details like, create new task, duplicate, generate, run, etc
Thanks,
Nanda
Please find all the screenshots attached. It appears for one task the Run option is no longer grayed out for some reason. The CDC shows Run as grayed out.
Thanks for any insights.
Jessica S.