Performance problems in R42800, R47011, R47071, R43800, R5847001, R097021, R007031

First post in jdelist. I've worked with JDE for 5 days, so please be kind.

We are:
Oracle JD Edwards EnterpriseOne,
E8.0 sp24, ES Sun, Oracle DB 10.2

Most of my experence is in Oracle database. When we SQL trace the above reports, we see huge waits in 'SQL*Net message from client'. We do a top command on the apps server and we see huge amount of CPU usage for the app server process. We do a truss command on the apps process and it appears to be in a loop but it slowly makes requests to the database. There is seems to be no performance problems an the database side. It's huge waits for the apps to process.

Has anyone seen a similar problem / solution?

Found a similar post http://www.jdelist.com/ubb/showthreaded.php?Number=22914

We have done and reviewed statistics.

Re: Performance problems in R42800, R47011, R47071, R43800, R5847001, R097021, R007031

The SQL*Net message from client wait simply means that the client (JDE) is doing something other than requesting data from Oracle. The UBEs you mention operate in a loop against a driving query. For each record fetched there is an enormous amount of business logic applied. This BSFN execution accounts for the high CPU against the runube process on the app server.

The behavior you are seeing doesn't surprise me for the jobs like Sales Order update. They can be long running and CPU intensive.

What tuning goal are you working towards related to these jobs? Are you trying to shrink your overnight batch window?

Re: Performance problems in R42800, R47011, R47071, R43800, R5847001, R097021, R007031

Justin,
Thank you for your post.
Yes, we are trying to reduce the batch processing window. When I look at the Oracle performance, 75% of the time is waiting for the client to respond. The Oracle db looks great. Let's focus on R04701. It runs for 20 hours. What can be done on the application side to reduce the client side waits? Are there are apps server parameters to reduce the number of network calls. (Pass a larger number of rows per network call). Seems like there should be some type of change to be made in the application to make the calls more efficient. In Oracle programs, this would be the array size.

Re: Performance problems in R42800, R47011, R47071, R43800, R5847001, R097021, R007031

For an integrity report like R04701 not sure how much optimization can be done , if you see no activity on the DB side.

But for update jobs like R42800 , R47011 if you are running against a large volume of data , splitting into concurrent runs that go against mutually exclusive data sets would be something you should consider. All these jobs tend to run longer when running against larger data sets (has to do with internal caching etc). I have tracked the throughput of these jobs (no of orders processed in a minute for example) , and found that it starts to reduce , the longer the job runs

Have a look at Oracle doc - 748333.1 on the support site which talks about UBE performance and tuning

Re: Performance problems in R42800, R47011, R47071, R43800, R5847001, R097021, R007031

What kind of volumes are we talking about? (Number of Sales Order Lines, Invoices, etc). If you're dealing with tens of thousands of transactions your timings may not be totally out there - but 20 hours is incredible to me.

Here's some things to check / consider:

1. What frequency are these jobs run at? If weekly or monthly consider upping the frequency to daily / weekly to reduce the transaction load into smaller "bites". For example we run R42800 every night, R4701* runs weekly.

2. Check Free Disk space on Enterprise Server.

3. Check JDE.INI on Enterprise Server to ensure that debug logging is turned off. If turned on it has a DRAMATIC impact on performance.
Section [DEBUG]
Output=NONE

4. How old is your server hardware? Has the business grown dramatically while the hardware hasn't been upgraded?

Re: Performance problems in R42800, R47011, R47071, R43800, R5847001, R097021, R007031

WOW! Ice Cube's post was right on the mark. Four of the reports are specifically mentioned in the Oracle Tech note. This outlined how UBEs work too.
Big thanks to all that replied.