Can we run a simple query using source endpoint

[adbdkb]

I am just starting to explore Qlik Replicate.  My first task is replicate a DB2 table to a Kafka topic.  So, I am trying to set-up the source and target endpoints.  

I do not know the DB2 database - it belongs to a different team.  So, one of things I would like to do is run simple SQL SELECT queries on different tables in the schema I am going to work on to understand the database a little.

Is there a way to run queries on a connection?  If so, how can I do it?

Thanks

[Steve_Nguyen]

@adbdkb from following the thread, this thread is solve?

 any new questions should be open new thread. 

[adbdkb]

Sure.  But, no new question has been asked - but clarifications to the questions / comments related to the original question have been posted.

Regards

[PGN]

Well, that's why many organizations have test or dev systems with mocked up data which should be used to discover and identify those capabilities. As far as retrieving a password from a vault, that's just part of good security and although it may be a hassle, it's a necessary one.  I'm a DBA and have to use a special secure server account when accessing PHI data.  

[Steve_Nguyen]

@adbdkb

if you are referring to :

The only point I was making there was, to the business person, if they don't know the differences / capabilities of different tools, they can consider it to be a hassle to keep on entering the password - which they also have to retrieve from a vault.

This is currently by design. you can submit feature request:

https://community.qlik.com/t5/Ideation/ct-p/qlik-product-insight

[adbdkb]

Thanks @Steve_Nguyen .  I shall do that.

[KellyHobson]

Hey @adbdkb 

A quick way to test a sample of the output is to point to a file target, as you have mentioned. Additionally, if you have some background on the table you can set a filter on a column (such as an id or date value) so you get a subset of the table.

Task designer-> Table Settings -> Filter

Ex.

$id < 5

ex. 

$id > 5 and $id <8

You can test with these filters in the expression builder.

All the best,

Kelly 

[adbdkb]

Well, this IS a Dev system, not even UAT :).  And I do understand the gates put in place for security - and reasons for those.  My reason for asking the question - was before looking for other options, it would be better to know the tool's capabilities. 🙂

[adbdkb]

Thanks @KellyHobson  for the detailed explanation.  I will use this approach to get the information.

[Steve_Nguyen]

@adbdkb , was all questions answer on your topic ?

[Heinvandenheuvel]

>> If I am asked to replicate a table, how is it not a valid reason to try to understand the same data?  

It depends on your organization, and your place in that. If that organization for the right or wrong reasons thinks you can do that knowing the table name only then so be it.

>> So, I wouldn't consider knowing the data as mere curiosity.

Then explain exactly that. Not to us, but to you management structure. We'll try to give you good arguments.

Even with the best intentions those application designers my not know enough about the data or the Replicate capabilities and restrictions to do a good job and you can probably help.

There are many examples: where good Replicate skills can help a lot and knowledge about the (meta) data is needed: LOBs, Identity Columns, Calculated values, Materialized Views, Transformations.

I do not disagree that you can do a better job knowing the (Meta) data.  For myself I find it an absolute requirement fully knowing the source (meta) data to do a good job replicating. I don't want to work in an environment where I'm not trusted with that information.  Actual production data being the tricky part.

'They' should realize, or be explained, how they are essentially going to give one the keys to the kingdom with the ability to define Replicate tasks. I personally insist on heavy, verified auditing when there could be any doubt.

Anyway, all I strongly disagree with is the notion to allow Replicate maintained credentials to be used for anything but a Replicate task.

Most potential Replicate issues should be determined in a DEV environment.

Some issues can only be seen in production, notably the performance/volume related issues, but sometimes also things like  (referential) integrity, data validation,  and data bytes level issues like a 'funky' character in a customer name never tested in DEV.

>> Well, this IS a Dev system, not even UAT :).  

Thank you for clarifying, I was just about to ask. That makes that DBA's decision seemingly more silly NOT to give you alternative access methods. You may have to prove them wrong, but in principle just knowing the table names is enough to create a task so you'll need a better explanation.

>>  you can define a task where you can have the target endpoint as a file, where you can see the data. 

For sure. Easy Peazy.  But if you did that in my organization with the clear intent to bypass security policies in place, even with the intent to just help, no malice, I would still ask you to be fired on the spot. Plausible deniability!

Btw... another good source for meta data definitions, possibly explaining replication issues is the output from  REPCTL dumpmetadata

Good luck!

Hein