E9.0 JDENET_n JDENET_k eventual kernel and CPU core allocation

[JohnDanter2]

Hi folks

Forgive the very high level questions here, but my understanding of JDENET_n is the listener that's looking for all work requests into and out of E1 and JDENET_k is the system that recognises and classifies each request and passes control to a specific kernal (security, scheduler, BSFN call, UBE launch etc)
The goal of these questions is to try and help my company maximize throughout of parallel processing jobs (be they BSFNs or UBEs/subsystems)

Qu 1 Where do these engines reside and function? On each server or just one? If so which one?

Qu 2 UBEParallelList and UBESubsystemLimit these seem to be dictating the number of like subsystems you can have on your system running at anyone time....?

Qu 3 How does the JDENET_k know that a subsystem is using a CPU core? And how many can a CPU core actually process as online docs differ in their answer

Qu 4 If we start using AIS server to process requests, is it too linked to the number of CPU cores and the max jobs it can throughput at anyone time?

Qu 5 Are the number f BSFNs processing at anyone time also linked to the # of CPU cores? - If I pushed 'work' into BSFN calls vs UBE subsystems, could I get more jobs processing at the same time?

Qu 6 If I had 100 SHPNs to process, which would be the fastest way to process them.
a) 100 subsystem requests to process 1 SHPN each (I am seeing a bottleneck of only 3 executing at anyone time)
b) 100 BSFN calls to process 1 SHPN each
c) 100 AIS /ORCH calls to processes 1 SHPN each

Some light reading reading on a few of the subjects above

https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=816350608437348&id=626115.1&_afrWindowMode=0&_adf.ctrl-state=1cygz81atk_4

Thanks

John

 

[jean_driscoll]

What type of server are you asking about, each server processes differently. JDE software sits on top of the operating system.

1) The 'engines' jdenet_n (or Network) and jdenet_k are programs that run on the enterprise server. The security, scheduler, BSFN call, UBE launch etc are jdenet_k kernels, the jdenet_n (Network) kernel monitors them.

2) UBEParallelList should only be used for specific UBE's like R3483 - manufacturing execution. UBESubsystemLimit does specify (or the UBE definitions themselves) how many subsystem jobs can be executed at once.

3) no. again each server is different so that is why you may be finding different answers, but the kernels do not know how much CPU they are using. On some platforms you can manage that from the operating system side, not the JDE side.

4) Yes, bsfn's called by AIS eventually gets processed on the enterprise server

5) BSFN calls and UBE subsystems are working on the same server using the same set of CPU's, divide the work as you see fit.

6) not sure about your question, they all come down to processing bsfn's. For same BSFN, Processing UBE's using subsystem is the fastest, as the job stays in memory, without having to create new job constructs.

 

[JohnDanter2]

Love it Jean, thank you

AIX server

It all seems to boil down to the fact that E1 can only handle 1-2 UBEs running on the batch server per CPU core allocated to it.
I was hoping there was a setting to control that and there doesn't seem to be one. My take on it it, the batch server is actually busy doing a whole host of other things (as many kernels run there) and only a few clock cycles per 'instruction set' are dedicated to UBE processing. Hence the limit on 2 UBEs

Does anyone know how many BSFNS can be handled per CPU core, or all the requests just multiplexed through for each running BSFN?

 

[Tom_Davidson]

Johnd,

" E1 can only handle 1-2 UBEs running on the batch server per CPU core allocated to it.", it depends on the platform I have an IBMi with 7 virtual procs, I easily run 30-50 UBEs simultaneously along with the kernels from 3 versions of JDE.

Platform matters, but so also does your class of machine, the skill of your staff, and what types of work you run in JDE.

Tom

 

[JohnDanter2]

Johnd,

" E1 can only handle 1-2 UBEs running on the batch server per CPU core allocated to it.", it depends on the platform I have an IBMi with 7 virtual procs, I easily run 30-50 UBEs simultaneously along with the kernels from 3 versions of JDE.

Platform matters, but so also does your class of machine, the skill of your staff, and what types of work you run in JDE.

Tom

Thanks Tom

That quote isn't mine by the way It's coming from numerous E1 white papers and SARs etc where people ask the same thing and this is the statement. Which was even given to myself quite recently.

We are on AIX, as for class of the machine, no idea. All we want to know is why when I have written a process to call 100 F986113 SBS wake up requests to a certain SBS version, only 1 processes at any one time.
I've suggested breaking the work down into like versions (one per DCTO) but even then the like DCTO calls queue. We just want more to run in parallel

 

[Tom_Davidson]

I think the issue is in your subsystem jobs. A subsystem job simply removes the overhead of starting/stopping a UBE. I have always used a third party scheduler for my UBEs due to requirements that the prior job in a stream of jobs (such as EOD or ME processing) continue or not based on the prior job completing normally.

There are others here that use subsystem jobs extensively, that should be able to help.

Tom

 

[JohnDanter2]

No worries, thanks Tom

What we tested was launching the same subsystem version 3 times, based on UBESubsystemLimit = 3
Yes it ran 3 times and jobs ran in parallel but they took slower per SHPN order than when actually ran with just one. My guess is this is why JDE advise to link that to the number of CPU cores available.
I did the same test with launching 30 UBEs to process the same data. Each UBE run normally takes 40 seconds when ran one at a time. But....Load all 30 up to run all at the same time and it moves out to 6 minutes per UBE! Clearly the CPU is struggling, but I'm not sure what metric to show my managers that this is the case. Time should be enough.

I'm struggling, despite this clear evidence to persuade management we need more cores on the ENT to get the best out of E1. We've only got 1 shared core!

 

[Neal Scarr]

Hi John.

Hope you are keeping well.

Unless I'm missing something, If you run the 30 UBE's one after another in a single threaded job queue that will take 30 x 40 seconds = 1200 seconds = 20 minutes. If you run the 30 jobs in a multi threaded job queue, all 30 jobs will have finished after 6 minutes, a saving of 14 minutes.

Cheers,

Neal

 

[brother_of_karamazov]

No worries, thanks Tom

What we tested was launching the same subsystem version 3 times, based on UBESubsystemLimit = 3
Yes it ran 3 times and jobs ran in parallel but they took slower per SHPN order than when actually ran with just one. My guess is this is why JDE advise to link that to the number of CPU cores available.
I did the same test with launching 30 UBEs to process the same data. Each UBE run normally takes 40 seconds when ran one at a time. But....Load all 30 up to run all at the same time and it moves out to 6 minutes per UBE! Clearly the CPU is struggling, but I'm not sure what metric to show my managers that this is the case. Time should be enough.

I'm struggling, despite this clear evidence to persuade management we need more cores on the ENT to get the best out of E1. We've only got 1 shared core!

Are you trying to run a combined UBE/BSFN E1 Enterprise Server on a single core server?

 

[brother_of_karamazov]

No worries, thanks Tom

What we tested was launching the same subsystem version 3 times, based on UBESubsystemLimit = 3
Yes it ran 3 times and jobs ran in parallel but they took slower per SHPN order than when actually ran with just one. My guess is this is why JDE advise to link that to the number of CPU cores available.
I did the same test with launching 30 UBEs to process the same data. Each UBE run normally takes 40 seconds when ran one at a time. But....Load all 30 up to run all at the same time and it moves out to 6 minutes per UBE! Clearly the CPU is struggling, but I'm not sure what metric to show my managers that this is the case. Time should be enough.

I'm struggling, despite this clear evidence to persuade management we need more cores on the ENT to get the best out of E1. We've only got 1 shared core!

Are you trying to run a combined UBE/BSFN E1 Enterprise Server on a single core server?

 

[JohnDanter2]

Are you trying to run a combined UBE/BSFN E1 Enterprise Server on a single core server?

Yes, except it's not even a single core, it's 0.7. And people are asking me why multithreading isn't really working
I launch the jobs all at once fine, it's just not far off the same speed as running them one by one.

I a talking about the batch server anyway. No idea about the ENT yet

 

[Tom_Davidson]

Have you tried giving them the supermarket example? Use a white board, only open one lane, and have them draw out multithreading. I've done that with C level folks in the past.

 

[brother_of_karamazov]

Yes, except it's not even a single core, it's 0.7. And people are asking me why multithreading isn't really working
I launch the jobs all at once fine, it's just not far off the same speed as running them one by one.

I a talking about the batch server anyway. No idea about the ENT yet

Yeah, that is probably not going to work well, but I have the feeling that you already know that.

 

[JohnDanter2]

Yeah, that is probably not going to work well, but I have the feeling that you already know that.

Yep

Trying to explain it is awkward as to me it's simple and plain as day and shouldn't really need explaining. I've used a juggler analogy. Balls are jobs, jugglers are cores lol