Qlik Talend Cloud: Dynamic Engine Removal Issues

rpooya · Nov 4, 2024 12:58:35 AM

When attempting to upgrade/reinstall a Dynamic Engine (DE), you may encounter an issue where you cannot deploy a new set of manifest files to the cluster. When applying the manifest files, even though it may apply all the namespaces and pods were created, a check of the DE using the "kubectl wait --for=condition=Available=true -n qlik-dynamic-engine deployment/engine-operator-dp --timeout 60s" command may fail with the following message:

Error from server (NotFound): deployment.apps "engine-operation-dp" not found

Cause

In many cases, the issue arises when some or all pods and namespaces may have been deleted, but the previous setups and definitions remain on the cluster. If these remain, they may hinder the successful application of manifest files, as these files will only generate new setups and definitions, rather than replacing the existing ones on the cluster.

Resolution

To remove any previous setups and definitions, please follow these steps:

The environment setup may still be deployed on the cluster if it hasn't been removed already. To verify this, please execute the following command:
```
kubectl get qlikruntimes.qlik.com
```
If a custom resource still exists, it should return the name similar to this, "qlik-processing-env-<id>". Copy the value down and delete the environment-related resources and associated namespaces with the following commands:
```
kubectl delete qlikruntimes.qlik.com remote-engine-<id>
```
```
kubectl delete namespace qlik-processing-env-<id>
```
After removing the environment setup, the next setup to go is the infrastructure environment. Users can run the "kubectl get qlikruntimeinfras.qlik.com" command to show the resources with that name.

Once the name that returns has been copied, delete the infrastructure setup and associated namespace with the following commands:
```
kubectl delete qlikruntimeinfras.qlik.com qlik-dynamic-engine-<id>
```
```
kubectl delete namespace talend-infra
```
```
kubectl delete namespace qlik-dynamic-engine
```
The final step is to delete the custom resource definitions created by the initial DE manifest file. These definitions can be removed with the following commands:
```
kubectl delete crd qlikruntimes.qlik.com
```
```
kubectl delete crd qlikruntimeinfras.qlik.com
```

Once removed, download a new set of Manifest files from the TMC of the Dynamic Engine object and follow the steps in the Documentation to apply them.

Requesting Assistance from Support

If these steps are followed and the DE is unable to connect to TMC or establish a link with the assigned environment, kindly contact Support to investigate and verify the current issue.

When reaching out, the following items should be provided:

Screenshots of the following items:
- The output generated by the "kubectl wait --for=condition=Available=true -n qlik-dynamic-engine deployment/engine-operator-dp --timeout 60s" command, to validate the error output shown.
- All deployed (and failed) pods and namespaces on the cluster, via the "kubectl get pods --all-namespaces" command.
- The DE Object and assigned workspaces in TMC, to confirm the status of the connection and pairing.
- The subscription page in TMC allows Support can confirm the Domain Name and ID.
A zipped copy of the manifest files that were being applied, for Support to review and validate.

Environment

Talend Data Integration 8.0.1

Qlik Talend Cloud: Dynamic Engine Removal Issues