topic Re: Python SSE: how to go through same RequestIterator multiple times in Integration, Extension & APIs

Python SSE: how to go through same RequestIterator multiple times

adamlopuch — Sat, 16 Nov 2024 05:42:34 GMT

Hi Qlik SSE gurus,

The Python SSE is working like a charm, however in one call I'd like to go through the *same* request payload *twice*... is this possible?

My goal:

FIRST ITERATION: go through each row and extract all parameters
Parse all distinct parameter values in entire request
SECOND ITERATION: go through each row AGAIN and yield result with distinct parameters

My issue:

I suspect that the (simple) approach I am using fully consumes the request iterator the first time I go through it. (Step 1). When I go through it a second time (Step 3), there is nothing to iterate.

Example code:

# STEP 1: Go through each row and extract all parameters all_params = [] for bundled_rows in request: for row in bundled_rows.rows: all_params.append([d.strData for d in row.duals]) # STEP 2: Get all values for parameters across entire request distinct_params = something_with(all_params) # STEP 3: go through each row again and use the distinct parameters for bundled_rows in request: response_rows = [] for row in bundled_rows.rows: duals = iter([SSE.Dual(numData=something_with(distinct_params)]) response_rows.append(SSE.Row(duals=duals)) yield SSE.BundledRows(rows=response_rows)

Re: Python SSE: how to go through same RequestIterator multiple times

Nabeel_Asif — Thu, 30 May 2019 01:05:58 GMT

Hi @adamlopuch ,

Your suspicion is correct; the request is a generator object which can only be consumed once in Python. One way around it is to first store the request in a different data structure such as a list that you can step through multiple times.

# Get a list from the generator object so that it can be iterated over multiple times request_list = [request_rows for request_rows in request]

In my SSE I use a function that can then take this list and convert it to a pandas dataframe which is usually a more convenient structure for Python:

def request_df(request_list, row_template, col_headers): """ This function takes in a SSE request as a list together with a row template and column headers as lists of strings. Returns a Data Frame for the request. e.g. request_df(request_list, ['strData', 'numData', 'strData'], ['dim1', 'measure', 'kwargs']) """ rows = [row for request_rows in request_list for row in request_rows.rows] outer = [] for i in range(len(rows)): inner = [] for j in range(len(row_template)): inner.append(getattr(rows[i].duals[j], row_template[j])) outer.append(inner) return pd.DataFrame(outer, columns=col_headers)

Finally, at the end of your function you'll need to structure the output as SSE.Duals inside SSE.Rows inside SSE.BundledRows 😂.