Moonshot and the R47011

[epost]

The R47011 job creates a tremendous amount of redo, it even updates address
book. A lot of the redo could be eliminated if the F42UI11 and F42UI12
tables could be converted to global temporary tables. If the system crashed
the EDI job would need to be reprocessed so losing the information in those
tables would not likely be catostrophic. Anyone ever tried such
foolishness? The data in the global temp tables is session specific so if
session A inserts dataset A then spawns session B to do some work, session B
will not have access to session A's data. Not sure if the R47011 works this
way or not.

Thanks,
Ethan
http://www.geocities.com/epost1


------------------------------------------------------------------------------
This e-mail is intended for the use of the addressee(s) only and may contain privileged, confidential, or proprietary information that is exempt from disclosure under law. If you have received this message in error, please inform us promptly by reply e-mail, then delete the e-mail and destroy any printed copy. Thank you.

==============================================================================

 

[altquark]

Hello Ethan

Wow - you got me intrigued. I actually wrote Moonshot and managed the project team - and still keep in contact with those who actually put together the Functional applications.

It seems to me as if your thread started in the middle of someone elses thread. Could you email me the full thread ? Maybe I can help you some more !

Jon Steel
erpSOURCING LLC
http://www.erpsourcing.com

ERP Sourcing
http://www.erpsourcing.com
[email protected]

 

[epost]

Not sure what I wrote in the prior thread but here is the gist as it stands
now. We performed a bunch of testing on Sunday. Results show on 12 CPU box
running Oracle and the Enterprise Server that we could obtain 100,000 sales
order lines per hour running about 30 simultaneous R47011's with different
data selections. We have put a trigger on the F47011 and F47012 that throws
the data into the 30 different batches as the data comes in by using one of
the unused fields. The initial points of contention where F4211, F42UI11,
F42UI12. We increased freelists dramatically and that fixed everything.
Our configuration is good, redo, datafiles etc...are configured optimally.
There are a few tweaks we can still make to get a little more speed out of
the system but the problem is the Advanced Pricing module.

When we activated Advanced Pricing we took about a 60% hit and immediately
became CPU bound. Why? Well one look at user calls and a trace file tells
the answer. One SQL statement in the trace file makes 95,000 executes and
150,000 fetches to return 55,000 rows from the F4092 table. This is related
to just one of the batches that was processing less than 500 lines! It is
also possible that the code is rebinding for each execute. This SQL
statement is just one of numerous SQL statements related to Advanced Pricing
that behave this way. With 29 other processes doing exactly the same thing
and user calls near 1,000,000 per minute it is no wonder we are CPU bound.

All of the documents I have found regarding Moonshot would confirm that the
results we got are actually better than expected. However, we have a third
party involved that has simply asserted that this machine is plenty powerful
enough to get to 150,000 lines per minute with Advanced Pricing activated.
At this point I am looking for the magic bullet because I don't think it can
be done with current hardware configuration. I am sure I could play with a
few parameters and tweak another 10-20 thousand records per minute out of it
but we are 110,000 records away from the clients goal at this point. I see
no way of reducing CPU usage unless we can reduce the number of user calls
which means rewriting the application code.

The real culprit here is the code. Well written code could easily handle
150,000 lines per hour on a much smaller machine. This stuff is just
terribly inefficient. Thanks for taking time to respond. Really appreciate
the help in this matter.

Thanks,
Ethan Post
http://www.geocities.com/epost1


------------------------------------------------------------------------------
This e-mail is intended for the use of the addressee(s) only and may contain privileged, confidential, or proprietary information that is exempt from disclosure under law. If you have received this message in error, please inform us promptly by reply e-mail, then delete the e-mail and destroy any printed copy. Thank you.

==============================================================================

 

[altquark]

Re: RE: Moonshot and the R47011

Wow ! Sounds like Moonshot all over again !

First of all, it seems to me that you are running both the OneWorld logic and Oracle on the 12 CPU box. Because OneWorld becomes CPU bound very easily, this means that you are not taking advantage of Oracle's better CPU handling. In all of the Moonshot exercises - and the key to scalability here - the OneWorld logic is partitioned away from the Database server. Your 12 CPU Oracle machine will easily handle a larger number of transactions if there were, say, 20 CPU's in front of it performing the OneWorld logic as Application servers.

Advanced Pricing traditionally showed us a 50% decrease in the number of transactions just by turning on the module - no matter how much logic was associated with the lines themselves. It seemed to us that Advanced Pricing was going through each order line twice for some reason.

One of the most important messages from the Moonshot project is to explain that OneWorld can (almost) linearly scale based on the number of application servers in front of a well configured Database server. Three tier technology is the key to scalability (please take the time to peruse my website for more information on scalability and oneworld) due to the method of queueing the user requests inside the application beforehand.

OneWorld is a CASE tool - it often generates the most gawdawful SQL code imagineable - you have picked up an example in the latest trace you mentioned. However, there are other reasons why the CASE tool is the prefereable method of generating the SQL statements - and certainly reasons why there are no triggers in OneWorld. From a DBA's perspective - it can often be confusing, but from a larger perspective of marketing the product, upgrading the product etc etc - it starts to become clear. It is also important that JDE OneWorld remains as independent as possible from the Hardware and Database platforms for future support (not all of us wished to be under Larry Ellisons rule in the future !)

Because we discovered that at the time it was impossible for us to change the application code - you also mentioned the fact - we discovered that we had to find a way around the CPU bound processes - hence the 3rd Tier Application Servers. Moonshot showed that the 3rd tier was extremely efficient to manage - and since moving away from JD Edwards, I have seen more and more customers moving to a 3 Tier Architecture.

Lastly, one has to realize that your goal of 250,000 transactions per hour (I believe that is the number you are trying to gain) is incredibly high. Purely from a Distribution company's perspective - that is almost double Amazon.COM's daily transaction load in one hour ! It is also certainly double GM's hourly transaction rate. My guess is that your customer is a heavily utilized retailer. I am not sure where your 3rd party could possible make the recommendation of "...simply asserted that this machine is plenty powerful enough to get to 150,000 lines per minute with Advanced Pricing activated" without moving to 3 tier technology. In my estimation, the size of transactions you are looking at would require a pretty heavily loaded configuration - using HP equipment as an example :

1 x HP9000 V2600 - preferably loaded with 16 processors
5 x HP9000 N class - 8 Way Application Servers providing 40 processors in front of the database server

With advanced pricing tuned on, and using my preduction of ~4,000 transactions per hour per Application CPU with Advanced Pricing - you should be able to easily achieve 160,000 transactions per hour. You will need to increase the number of R47011 processes to at least 40 (preferably 50) to achieve the results

I hope that helps. Please do not hesitate to involve me if you need me.


Jon Steel
Xe Upgrade Specialist
Chief Technology Officer
erpSOURCING LLC
10694 East Powers Drive
Englewood
Colorado 80111
Work phone : (303) 224 9468
Cell phone : (303) 883 9168
Fax Phone : (303) 224 9467
Email (main) : [email protected]
Website : http://www.erpsourcing.com


ERP Sourcing
http://www.erpsourcing.com
[email protected]

 

[WhippingBoy]

Re: RE: Moonshot and the R47011

Well, before you get too flustered with Advanced Pricing lets see your setup. I'm wagering that you've got too much for what you're trying to do.

You refer to the 4092, that's group definition stuff. Now, if your setup is full of groups, dynamic groups, and too many preference entries, you're going to get screwed. . . and FAST.

There are a lot of pieces of information for pricing and as many as 21 different search methods. If you're trying too many methods there's no doubt about the results you've seen. The system is flexible enough to let you hang yourself if you're not careful.

Show us your groups, hierarchy, schedule, and adjustment definitions. Then some representative data.

I've got 10 years of Advanced Pricing between myself and an employee.

I also wrote most of the Pricing system for World and led the redesign for OneWorld. Granted, that was a few years back, but it's still pretty much the same stuff.

Darren Ricciardi - OneWorld Whipping Boy

Looking for work in OR NEAR Amsterdam THE NETHERLANDS