Performance - trying to improve on 2.5 seconds response time

Michael L.

Well Known Member
Morning List. I am posting this with the hopes of taking our interactive response times from 2.5 to 3.5 seconds to sub second if that’s at all possible with our existing iseries 400, J.D. Edwards configuration.

I know performance is one of those intangibles that is difficult to identify. I’m hoping that any OS/400 shops out there can review our subsystem/shared pool configurations below and perhaps make a recommendation or two.

Our configuration is as follows, J.D. Edwards 8.11 SP1 (no ASU’s applied) - Tools release 8.98.3.3 - iSeries Power5 - OS/400 V5R4M5 - WAS 6.1.x. I do physically have three racks with seven partitions of which one I have recently upgraded to OS i7.1 on our DV LPAR. I’m doing PY next week and Production sometime in July. I mention this because perhaps i7.1 will be our magic bullet??

We have implemented all relevant performance recommendations in the most recent tuning doc (January 2008) by Jim Denton as well as the excellent doc from Collaborate by both Colin Dawes and Jim Denton.

IBM has done a through performance review of our Database/logic server as well as our Web server (using Job Watcher / iDoctor) and reported back that they see no issues on either box. Our Infrastructure/Network team report that bandwidth/throughput is faster than fast.

Our desktops/laptops are IE 8 with 2GB of ram or less, i5 2.4GHZ and Windows XP SP3.

Any suggestions on how to reduce the performance would be much appreciated. Thanks in advance, Michael

Production Database/Logic server

Machine Pool = 7GB
*BASE = 12GB
*INTERACT 450MB, yes MB
JDE811 in a shared pool with 12GB
QUSRWRK in a shared pool with 54GB
QBATCH in a shared pool with 21GB

Production Web Server (7 JVM’s)

Machine Pool = 3GB
*BASE = 18GB
*INTERACT 2GB
QWAS61 in a shared pool with 40GB
 
[ QUOTE ]
Any suggestions on how to reduce the performance would be much appreciated.

[/ QUOTE ]

Michael,
Lots of ways to reduce performance
grin.gif


Sorry, I really don't have any input for you on your issue.

Good luck.
 
What do you define as "interactive response time". Is this the screen to screen time or the time it takes to load an application?

What is the fix current date on 8.11 SP1? Did you do a fix current?

The part you're missing for the shared pool is the activity level and faulting rates..........critical part of the picture.

How much CPW is assigned to each LPAR? What is the min, desired and max processors? How many virtual processors per LPAR?

What fix pack of WAS are you on? Are you using the latest jt400.jar?

How many users are we looking at here and what modules?

When you're looking at performance is this for a specific application or is it "across the board".

For the movement from V5R4 to V7R1 there are quite a lot of things to do to prepare here.

Where the IBM consultants you brough in AS400 skilled, JDE skilled or both? Pretty hard to optimize performance of a ERP system if you don't have experience and knowledge of the ERP system.

Lots of things that Jim and I couldn't cover in the 2 x 1 hour presentations that we did at Collaborate that either of us (or both) could be engaged to look at (did you attend and receive both presentations?).

Colin
 
[ QUOTE ]
Morning List. I am posting this with the hopes of taking our interactive response times from 2.5 to 3.5 seconds to sub second if that’s at all possible with our existing iseries 400, J.D. Edwards configuration.

[/ QUOTE ]

What specific application do you want to run sub-second ? Is there a transaction you're trying to get to work that is currently 2.5 to 3.5 seconds long ? Is this a sales order entry line - or a login ? Is this a Locally (LAN) connected user or someone on a WAN ?

If you already have a specific transaction in mind (or several in mind) for performance improvements, then that is definitely something that can be worked upon - trying to "generically make everything faster" is a little more obscure !!!
 
You may want to try segregating your QSQSRVR jobs into a separate memory pool. Segregation of work does wonders, as someone said though, faulting is your enemy, paging isn't necessairly as long as you aren't faulting. I try to keep my DB pages/fault ratio > 10 and Non-DB at 2.

I see you are running WAS, WAS (actually any java app), HATES faults, they play havoc with garbage collection, so make sure your WAS pool isn't faulting hardly at all.

What you are asking probably requires someone pretty knowledgable of JDE and performance on the IBM i.

Tom Davidson
 
Colin, I have attached a Word document trying to answer all your questions as best as possible, particularly this one - The part you're missing for the shared pool is the activity level and faulting rates..........critical part of the picture. [I’ve included several ‘print screens’ in the Word doc.]

Certainly don’t expect you to spend much time on this but if were missing something obvious that comes to mind and when you have some time and patience it would be great to hear your thoughts.

Jon, The retrieving of records once the application has opened good. We’re not trying to optimize a particular transaction but are trying to make the menus’ and applications open much faster than they do today. This issue is for both our Lan and Wan users.

Tom, thanks very much and I will run this by my Manager.

Michael
 

Attachments

  • 169824-Production config.doc
    52.5 KB · Views: 173
So was 811 SP1 ever made Code Current? Like on the initial install?


So how many ESU's total have been installed? If there never was a Code Currency done then give up now. No amount of tunin will solve the issues. Throwing more hardware at it just masks the issues.

Can you post the SBSD details for QUSRWRK & QSYSWRK? Is the entire subsystem running in a different pool or just certain jobs under the subsystem? What is in the routing entries and prestart job configs?

Also post the prestart jobs for QSQSRVR and QZDASOINIT details.

What is the Web Server config? How maany instances of WAS? How many profiles? How many Application Servers per profile? Is there custering configured in WAS or with a hardware load balancer?

Have you configured the HTTP server for keep alives, threads, etc?

Have you configured caching and compression at the HTTP level?

Is IE tuned on the client end? ie are you using the default of 1 thread or have you moved this up to 20 threads yet?

Colin
 
I am having a bit of deja vu as we had similar problems when we went live....there is a setting change in the java shareclass which you need to ensure is set correctly. It is in Jim Denton and Colin's presentation now but you might not have spotted it...
We were seeing contention on the JVM shareclasses caches and just had to configure one per JVM. We also configured for a garbage collection policy of gencon which more effectively manages the needs of both short lived temporary and tenured heap objects. This involves just adding this to the JVM's generic arguments where jde1cache is replaced by a unique name for each of the JVMs/cluster nodes: -Xshareclasses:name=jde1cache,groupAccess,nonFatal -Xscmx50M -Xgcpolicy:gencon -Xmns256m -Xmnx1024m .

The other thing I would suggest is to get Colin (or someone) to set you up an intel JAS server (it only takes a day for these consultants to create one for you). Then put a bunch of users on it and see what they are experiencing. This takes the iSeries JAS out of the equation and you will know whether the issue is at logic / system end or with the JAS server. Then you will be able to focus your efforts more effectively.
Rich
 
Yep...it's in mine and Jim's presentation.

Basically the more JVM's you have with WAS the slower the performance!

Unless of course you give each JVM or group of JCM's their own "cache". By default all JVM's under WAS share the same cache.

Also check for "bad" code trhat may be looping or causing memory errors. These errors have a larger affect on the AS400 than other platforms.

Also are you using the internal virtual NIC for the Web Server to Enterprise Server communications?


Colin
 
Back
Top