Improve Queue Efficiency

Shrikanthn

Well Known Member
Hi List
Currently we have 5-6 different job queues set up in JDE. However I believe that they might not be running optimally. I infact wanted to increase the number of queues so that users don’t have to wait even the little they do now. However I just read that one can work on multi-threaded queues etc. Which i took to mean simultaneous processing of jobs rather than sequential processing. So wanted to know more about it. Would be grateful if somebody can give a non-technical explanation of multi-threading. Also wanted to know what is the resource constraint when considering number of queues and multi-threading, is it number of CPUs or RAM? We have an enterprise server that has 2GB RAM, Two 200GHz CPUs. My IT colleagues tell me that the system is not utilizing the RAM fully, which leads me to believe we could implement multi-threading if it depends on RAM.

Best regards,
Shrikanth
 
Multithreading as implemented by JDE is the ability to process multiple batch programs simultaneously within the same logical batch queue. OK, this is really multitasking.

Lets say you currently have only one "QB7334" batch queue. If you have a batch program coded to run in QB7334, and a user submits a very long running job to this queue, users submitting afterwards will wait in line for their job to process if their job is also coded to run in QB7334. Adding "threads" will reduce queue wait time for those additional users. The runqueue process works on a cycle, configurable by your CNC, which tells the runqueue how long to sleep before waking up and picking up the next job. Each batch program is an individual process, running on a specific CPU managed by the Operating System.

Multithreading in the more technical sense is the individual "threads", or instructions, of a process being delegated to a processor or multiple processors as dictated by the Operating System. This isn't a very technical explanation, but it shows you the difference between JDE speak and rest of the world speak. (Don't get me started on the definition of "Asynchronous" and "Synchronous".)

Setting the sleep time too low can consume extra CPU cycles you might want for your active batch jobs. Setting the sleep time too high may cause batch jobs to wait in the job master table longer than necessary. A good setting is between 10 and 15 seconds. I've set it as low as 5 seconds on a fast server.

JDE batch multithreading is both CPU *and* MEMORY limited. Resources are even more limited if you are running your database on the same server as the Enterprise server software. If you are a lucky one and have a separate server, the old JDE rule of thumb has been "one UBE per processor". With the latest and greatest hardware, this is really not the case. With only two processors, you are limited by what you can do due to server OS overhead, but if the batch programs you want to run are not logic intensive, in other words if they spend more of their time waiting on I/O in the database, you may be able to run two or three UBE's per processor. That jives with what you say you have configured, around 6. You might be able to squeeze in another couple of processes, but it is best to experiment on a non production system. With a set of 14 875 MHz RISC processors, I'm able to submit up to 75 simultaneous batch processes. Some quick math will show you this is more than 5 batches per processor, but in my particular case many of the programs are not long running processes and they aren't logic or application server intensive processes. Your mileage may vary, but I've seen batch processes consume between 10 and *gasp* 200MB of RAM each - it really depends on the requirements of the particular job.

You must be careful that post jobs which can collide with each other due to data selection are not running at the same time, therefore check the documentation for your specific circumstances to see if multithreading is appropriate for you. The R09801 used to be single-threaded, but ESU's have been delivered in the past couple of years to allow the GL post job to run in multithreaded mode. *note* Don't take my word for it, check your release documentation and then with the vendor or an experienced CNC consultant before implementing anything!

In order to implement this solution, you setup X number of "threads", with XE/ERP 8.0 this means you spin up additional queue processes at startup time. This is done by editing the JDE.INI for Windows servers. If you were running UNIX, it would be the RunOneWorld.sh. Releases >=8.9 have a new interupt driven queue processing system and a new kernel type definition. This allows those users to use program P986130 to add, remove and tweak job queues per server and port.
 
Thanks a ton charles for such a detailed explanation. So it seems that in my case processors might be the big constraint. I get what you are saying. will check with our local consultants. Thanks again
Shrikanth
 
Back
Top