• Introducing Dark Mode! Switch by clicking on the lightbulb icon next to Search or by clicking on Default style at the bottom left of the page!

Speed gain using cache to write lines to flat file

fellipesg

Member
Hello guys.
I would like to know if using a cache structure, to write lines to a flat file in it, would be faster than stopping the application flow to wirte one line to flat file and get back to the application to process it again. So, if I "cache" the lines within the Do Section, in my report, and in the End Section get the results back and write then to the file at once, will the report finish earlier than the ordinary way?

Att
 

BOster

Legendary Poster
In that exact scenario probably no, in fact its just extra processing so it might even be slower. The file i/o (the actual C api that is ultimately used) should all ready have some sort of buffering/caching in place for performance. If you are seeing performance issues it is probably NOT with the actual flat file i/o operation.
 

fellipesg

Member
Hi, 78aspide78
I have a report wich have a bussiness view with a join between F03B14 and F03B11. For each day in an specifc month, I need to count how many payments were made in that day, according to another custom table, wich have KCO, DCT and SDCT. So, for each KCO in my custom table, I need to check if in the BSVW are payments made with the same DCT and SDCT in my custom table. After calculating payments for each row in my custom table(for each day in a month), I need to write one line to a flat file with the results. I have close to 500 lines In my custom table to compare with the payments(a lot). My report is taking 6 hours to finish in PY and, believe me, 24 hours to finish writting in PD. I would like to know if I can make it faster by caching the results.
 
Discover how to build no-code data integrations and business process automations.

Larry_Jones

Legendary Poster
If you're really running into a performance constraint with your current method (must be a LOT of data), you'd be better off writing to / reading from a database "work" table than JDE cache in my opinion.
 

sselman

Well Known Member
Depends on the report. But i would start by get a log and determine where you are losing the time.

We have also wrote reports to call a SQL SP, populate a temp table, and then display the data from that table in the report output to reduce runtimes of very large jobs.
 
Last edited:

Larry_Jones

Legendary Poster
Sounds like a program design / approach issue rather then something that can be fixed by using cache or a work table.
 

BOster

Legendary Poster
Sounds like a program design / approach issue rather then something that can be fixed by using cache or a work table.
Agreed. Either one of these will probably just add time. Your bottleneck is probably NOT the actual writing of the flat file to disk but rather with all the logic that determines what to write to the file. I would start with the SQL that is being executed.
 

Adrian_Chimirel

Legendary Poster
No Felipe, it is NOT normal.
How many millions records are you processing, from how many dozens of tables?

PS A full conversion from one release to another may take over 12 hours to complete, but that runs once in five or more years.
 
Top