E9.2 Consuming REST API with multiple pages

[Glenn McClenny]

A 3rd party API returns data in pages of 100 records, meaning if there would be 500 records in a json response, the rest API would have to be called 5 times passing the appropriate next page number. When the API is call, the end of the json response has the page info in it.

"resultInfo": {
"page": 1,
"perPage": 100,
"totalPages": 27,
"count": 100,
"totalCount": 2633,
"previousPage": null,
"nextPage": "/api/XXXX?PageSize=100&page=2"
},

How can I "loop" through the pages to update the appropriate JDE tables with all the records?

 

[DaveWagoner]

Easiest would be to process 1 page of records at a time. You then build a "simple" recursive orch where you take 1 page of api results, process those in JDE, and then check the results to see if nextPage is populated. If it is, then call the same orch recursively.

The recursive call needs to be a local AIS REST call rather than an orch component for it to work properly in P98220W and other places incl export/import. At least on my version Also you'll want to put in a hard count limit on your recursions just in case you get into an infinite loop or process more recs than you want.

A bit more difficult but not crazy would be to recursively call the API while "accumulating" your dataset, and then once the recursion is done you process the entire dataset once. It's a bit more dangerous IMO.

I'll be presenting at blueprint including a recursive orch that I'll be giving away if you're going, otherwise I'll hand it out here after conference.