OneWorld - Multithreading

jgersic64

Active Member
Since I have started with my current employer, there have been complaints
about the speed of the system and the time that is required to complete
reports. Granted, the reports are large, and the comparison being made was
from a large AS/400 system where some of the users had come from (compared
to NT). So I have setup a separate logic (applications) server to separate
the JDE logic from the SQL7 database. What I have found is disappointing. It
appears that the Oneworld UBE process is not truly multi-threaded. When I
kick off a report, I can see it executing on the box on a single processor.
That processor is being used 80-100%, but none (it is a 4-way) of the
processors are being touched. I have played with NT, prioritization, and the
jde.ini file, and have yet to get any more performance out of the ube
system. If this is the case, then throwing a UBE from a single to a four-way
will not help at all, only in the case when you are running multiple UBEs at
the same time where each UBE may be running on a different CPU.
I have read up in the JDE books, and it just notes that the 'developer
can select for the UBE to be multithreaded' but I don't see any
options/properties in the RDA for this..All I can assume is that they mean
the business functions and c source code behind it, is written to be
multi-threaded.
Does anyone have experience with the above and "forcing" the code to use
more than one processor, or have any other performance ideas? I suspect this
is a limitation of the actual foundation code, and how it was ported to NT,
but wanted to see what the list thought. At this point, I am frustrated and
management is pushing to possibly move the platform to AS/400.
Any ideas?

John

The logic server is a 4-way 550mhz P3/Xeon w/1GB of ram.


Xe, SP16/Update2, NT4/SP6a, SQL7/SP3, WTS/Metaframe 1.8




----------------------------------------------------------------
John Gersic
Financial Systems DBA
703.744-6057
1861 International Drive
McLean, Virginia 22102
MicroStrategy-Best in Business Intelligence
http:/www.microstrategy.com/ <http:/www.microstrategy.com/>
-----------------------------------------------------------------
 
John :

As far as I have seen, an UBE is tied to the CPU that executes it.
OneWorld can't or doesn't know how to spread a single UBE process
across several CPUs.
Multiprocessor NT UBE servers are useful if :
a) You plan to run several simultaneous UBEs there.
OR
b) You run UBEs and MSSQL/Oracle on that box (UBE Server plus Database)

Sebastian Sajaroff



B7321 to Xe, NT/W2K/SQL
JAS, Interoperability
MCDBA,MCP+I,MCSE,Citrix Admin
[email protected]
Grupo ASSA - Application Software SA
 
Hello John,

I think your desire for UBE's to be multi-threaded is reasonable but not possible given the generic nature of UBE's. Even on the AS/400 I believe (correct me I am wrong) that each batch job only runs on 1 processor at any given moment. I think when you see the term multi-threaded UBE mentioned in JDE documents, it is referring to jobs that can have multiple copies running at the same time without causing data corruption. A "multi-threaded queue" is a job queue that can execute more than 1 job at the same time.

Single-threaded jobs are those like the GL Post (R09801) that should usually have only one copy executing at a time.

When you ask for multi-threaded UBE's I think you are really asking for is a UBE whose code excution has been broken down into units that can run in parallel. This is not generally possible with a generic runtime environment such as the OW foundation. I don't think it is realistic to expect the OW runtime to analyze any UBE that might be passed to it, figure out which pieces of the code can be run in parallel, submit those pieces to individual processors and then consolidate the results.

Products like Oracle Parallel Server are written to allow parallel processing of SQL queries in certain situations. SQL by it nature is a much more limited universe of possible commands and execution paths. UBE's could contain just about anything. With any multi-purpose language I have used, I have had to build parallel execution into my programs on a case-by-case basis.

There is a way to do parallel execution in OW by using UBE interconnect command and spawning worker UBE's in asychronous mode. If your job queues are set up for multiple simultaneous jobs and you have the processors to support the load you could get the result you want. You would have to build your parallel UBEs . A few base JDE UBE's work this way out of the box but not many.

The above would take a fair amount of effort. I think you will get more performance results by looking at the database and tuning at that level. Make use of your other CPU's by allowing multiple jobs to execute in 1 or more queues.

Regards,

Justin Miller
[email protected]

working with B7332 and XE on AS/400, NT, Solaris and AIX
 
I am not sure of the mechanics but I am nearly 100% certain it is NOT a
limitation of the foundation. The reason I know this is because we run MRP
multi-threaded. The primary MRP UBE spawns multiple MRP jobs that are in
fact threaded thru the use of semaphores.






JD Nowell
OW: B7332
ES: AS400 V4R4 CO: DB2/400 SP: 11.2
Users: 250 TSE Users: 100
 
I think we need to get a few facts straight about UBE's and how they work. In effect, you are ALL right !

First of all, a single process can only operate on a single processor. This is true for any computer program - including Oracle, JD Edwards and Windows ! A process can certainly have multiple threads all running at the same time - but unless they are spawned off as an autonomous process, they will always run under the parent process, hence will be constrained to a single processor.

To change a program to run in parallel - ie across multiple processors - then we need a "loader" to spawn these threads into multiple autonomous processes and be able to collate the information at the end.

JD Edwards UBE's are designed through the RDA - the CASE tool - and each process if designed correctly should be able to be run autonomously. However, JDE do not themselves (except for ONE example) design the programs to run as a multi-process procedure. Instead, the customer has to determine if it is possible to run multiple UBE's in parallel to speed up the process. An example of this could be Sales Update - where a customer doesn't like their 50 branches taking 12 hours to process under a single job - therefore splits the process into 50 smaller processes, that (conceivably) could take 15 minutes to process in parallel. This architecture is called "Parametric processing" - ie the ability to take certain procedures and utilize a parallel methodology to increase the speed of tasks.

However, when I was at JDE and showed Parametric processing to the VP's - to demonstrate Moonshot - Ed came back with "but you cannot do that with MRP" - and he was right. The parametric procedure demanded a simple process that could be "chopped" into smaller parts, but MRP and DRP will process information that has been found using a previous process - in effect a hierachial process :

X - level 1
|
---------
| | |
A B C - level 2
|
-----
| | |
a b c - level 3

(I hope the above doesn't look like a mess !)

In the above diagram, level 3 must be processed before level 2 and again, this level must be processed before level 1.

This is how MRP and DRP operate - and why Ed thought it not possible to run the above in parallel. Well, development took some of my architecture designs and Diane Webster developed a brilliant product that is called the "New" MRP - R3482 I believe.

This R3482 has a processing option to "spawn" subsystem processes which actually perform the work. The 1st process will be a "master" process - which spawns the job information for level 3. In our example, let us suppose that 2 processes are being spawned. Process 1 will pick up this job information for level 3, item #a and Process 2 will process level 3, item #b. When one of these processes finish - they will automatically move to level 3, item#c. If process 2 grabs item#c - when process 1 finishes it realizes there is no more data to process - so it goes into a "sleep" mode. When process 2 finishes - it realizes it was the last process active - so it spawns level 2's job information and picks up item#A. Process 1 will "wake up" because it realizes theres work to do, and pick up item#B. This goes on until the task is completed.

As you can see - this method is truly ground-breaking in the ERP world - with 4 processes running on an AS400 in OneWorld - MRP/DRP runs as quickley as World (thats CASE logic versus native RPG !) - with 12 processes, it could conceivably run 3 times faster ! Thats darn quick.

Because of the distributed processing model of JDE - I did try to ask development to create an associated "list" of application servers that the master process could choose from - which would make MRP/DRP one of the most parallel business processes available - but JDE development said they had other things to do !

By the way - Oracle works in the same way. If a single user performs a command against the database - the logic inside Oracle will determine that such a command will take a long time to finish - it will then spawn parallel processes with instructions to perform on a smaller data selection - and will return with information for the master listener to collate.

Its a little more complex than just expecting it to run "parallel" no matter what !!!!



ERP Sourcing
http://www.erpsourcing.com
[email protected]
 
Thanks for the replies, I think my understanding is pretty clear now
of JDE's definition of 'multithreaded'. One other question, and this may
how my naivety of NT..but when a UBE is processing, why does the
runbatch.exe only take a few percent of the available cpu resources? I
have put the priority to 'high' through the task manager, but runbatch
still only take a few percent. If there was a way to ensure that the
runbatch program would take full advantage of a cpu it would decrease
processing time dramatically. Now that I am playing with the logic
server I can see that the run batch job is muddeling about at maybe 20%
of a cpu, and SQL is taking 3-20% on the database server. Is there any
other tuning I can perform, or am missing? Lastly, has anyone migrated
from SQL7--> SQL2k and seen a performance boost?

John

Xe - SP16/Update2, NT/SP6a, SQL7/SP3, WTS/Metaframe 1.8




Xe, Update2, SP16
NT/SP6a, SQL7/SP3, WTS/Metaframe 1.8
 
Re: RE: OneWorld - Multithreading

When a job is using only 20% of a CPU is usually means that it is doing something other than pure computation. Most UBE's are asking the database for something, executing some logic, looping within a section and then asking the database for something else.

UBE's like MRP will use a lot more CPU. I don't know if one exists, but, a UBE that optimized the placement of different size boxes on a pallet or shelf would also be mostly CPU. (Like playing a giant 3D version of Tetris thousands of times).

One of the reasons that I mentioned DB tuning in my previous post is that in my experience a lot of performance issues with UBE's are not because a shortage of CPU capacity but rather database bottlenecks. These bottlenecks can be caused by things like a lack of good disk hardware, poor database file placement, inefficient SQL being generated by the OneWorld middleware, and other things like a lack of key indexes.

In answer to your SQL7 to SQL2000 performance questions. Yes, I have done this upgrade and noticed some improvement. It was not as drastic as what I saw between 6.5 and 7 but still noticeable.

Justin Miller
[email protected]

working with B7332 and XE on AS/400, NT, Solaris and AIX
 
To Justin and Gerd:


THIS IS EXACTLY WHAT MRP DOES.

Jon is correct in saying that you have to program this into the UBE you want
to run multi-threaded. It just so happens that MRP is coded this way and
that there are very few others that are also coded this way.


Here are the server INI settings that turn parallel processing (aka
multithreading) on.

[UBE]
UBESubsystemLimit=6
UBEParallelList=R3483






JD Nowell
OW: B7332
ES: AS400 V4R4 CO: DB2/400 SP: 11.2
Users: 250 TSE Users: 100
 
Re: RE: OneWorld - Multithreading

ok - first of all, lets look at what could conceivably slow down a UBE process. There are 4 factors, usually, that tell us how fast a UBE is going to perform in an Application Server environment :

1. CPU speed (single processor)
2. Network speed (bendwidth & latency)
3. Database Access Speed (disk access time)
4. Bus speed for all machines involved

A process will run only as fast as the slowest factor in the above. If you are watching the CPU on the app server - and its running low - then one of the other 3 items could be slowing down the process

If you are seeing up to 20% CPU on the database server - then it cannot be the bottleneck. Watch the drives on the database server - if they are being "thrashed" then you have your bottleneck. If not, then it has to be the network between the application server and the database server.

Lastly, if the CPU, Disk and Network are all running optimally - then the bus speed could be the bottleneck - ie you are running as efficiently as possible for the computer type you are using !

The above is true pretty much for any process - providing the code is written somewhat efficiently (no "wait" statements !)

Jon Steel
Xe Upgrade Specialist

ERP Sourcing
http://www.erpsourcing.com
[email protected]
 
Back
Top