PowerBI, Rockset and Pagination

HI, i have a little question, i have pagination implemented in power bi without query lambdas, using directly a query. In the same dashboard for example i have two different process that retun for example 400k rows. The problem is that sometimes the two process return all the data, but sometimes one of the process just return 100k (the limit) and dont execute properly the pagination.

Any idea? we have disable in powerbi the option two make the process in parallel mode, is a secuencial process, but it seems that when you try to process two pagination process in a few second between the end and the start of the other, appears the problem.

Thanks

Hello!
I am having the same problem:
I am using pagination to get data from 100.000 to 100.000. The total of rows is around 800000. When I use pagination, sometimes I am getting 100000 and sometimes I am getting the total of rows (around 800000).

Any idea?

Thank you very much!

Hi @dserna and @ljimenez !

Thanks for reporting this! We’re currently investigating. Please give us some time, and we’ll update this thread.

Best,
nadine

1 Like

Hi,

I’m still working on investigating this issue. A bit more information on how you use pagination would be helpful. So let’s say you run a sql query that should generate 400,000 documents and your initial initial_paginate_response_doc_count is 1,000. Are you saying that sometimes that the initial response returns 400,000 when you expect 1,000. Or is it that when after running the sql query you fetch the paginated results but only find 1,000 when there should actually be 400,000 in total? Or is it something else?

Thanks!

Hi!

With same sql query, and same initial_paginate_response_doc_count as 100000, and same total of rows as 400000; the initial response always returns 100000, but I sometimes get the next_cursor indicator and sometimes I do not get it.

Thanks.

@ljimenez Can you please see if your request is missing a /

We’ve found that missing a / in the query like below would result in issues.

The first version was:

let Json = Json.Document(Web.Contents(BaseUrl, [Headers=[Authorization="ApiKey " & ApiKey, #"Content-Type"="application/json"], RelativePath=queryId & "/pages/" & next, Query=[docs="100000"]]))

The correct version is:

let Json = Json.Document(Web.Contents(BaseUrl, [Headers=[Authorization="ApiKey " & ApiKey, #"Content-Type"="application/json"], RelativePath="/" & queryId & "/pages/" & next, Query=[docs="100000"]]))

If you don’t put the “/” sometimes work but sometimes it doesn’t work. With the / always works because force always the relative path.

Thank you for helping us out with this @dserna and thank you for giving us more info ljimenez!

2 Likes

Hi Yash! Yes, I was working on it with @dserna until discovering the problem.

Thank you very much by your response.

2 Likes