XML 'Specs' in 8.12/TR8.96

jgersic

jgersic

Reputable Poster
XML \'Specs\' in 8.12/TR8.96

I was curious, is anyone using the new xml based 'specs' functionality in 8.12+TR8.96? I am quite interested in it as it seems to really simplify the entire package build process (really eliminates it) - but is anyone successfully using this? If this works, it would make a good adittional argument to push to 8.12 from our current release.

-John
 
Re: XML \'Specs\' in 8.12/TR8.96

John,

The XML specs in 8.12/8.96 do not actually eliminate the package build process. Package builds are still required and in fact take longer to run than their non-XML counterparts on the same hardware. Package specs are stored in table form in the associated Central Objects data source. So for instance, the ER Spec Detail table for package PD812FA will be stored in a table called F98741PD812FA.

What is not strictly required is the JAS generation process. The JAS server can JITI objects from the current package and use the XML objects to build serialized java objects on the fly. However there is a noticeable delay in doing this. For a production system you are going to probably want to run JAS generation.

Another thing to keep in mind is that UBEs read their specs from the database each time they run. This introduces an increased launching overhead as compared to non-XML specs. For a quick running report a large portion of the runtime will be initializing the specs. For short, high-volume reports XML specs will change the number that can be run in a given amount of time. I am going live with 8.12 in two sites. We have yet to do full volume testing of picklists and other high-volume reports but based on some initial time tests I suspect that we may need to use subsystem jobs to get the needed throughput where we previously did not require them.

One a positive note it seems that the JAS gen process is faster and more reliable. I don't know if this is the XML specs and XML spec APIs making things faster or an improvement in the latest 8.96 tools release. I am now consistently getting through generations with 5 languages without crashing. I am hoping another positive thing that XML specs will give us is the elimination "spec corruption".
 
Re: XML \'Specs\' in 8.12/TR8.96

Thankyou Justin for that insight. I thought that 812 would not really eliminate any of the package build processes - just make it "optional" for those that don't really worry about performance (!)

I'm positive that the "spec corruption issue" is absolutely taken care of, since realistically the reason the TAM files would become corrupt is because OneWorld really didn't have a method of locking individual TAM records while updating them - hence multiple users could theoretically update the same TAM record (and end up corrupting it). Because all of the supported database types have locking mechanisms, I think that this has been taken care of. Of course, this is probably the SOLE reason they went to "XML specs" - although we'll have to see what some of the advanced toolset developers (Pastuhov for example) can come up with.

I'm concerned that JDE have completely gone around in a circle when it comes to UBE performance once again. Certainly long jobs like MRP or Sales Update won't be affected by performance - but absolutely high-frequency jobs like pickslips and invoices will indeed be affected by reading from the database. I'm certainly interested in hearing when you go full-volume, Justin.
 
Re: XML \'Specs\' in 8.12/TR8.96

There are still some TAM files around, DD ones specifically.

Plus, there's nothing to prevent JDE from saving corrupt specs. The actual save would work perfectly, because of the DB locking, but the end result would hardly be different.

We really have to wait and see.

I have tested specs consistency in one spec table, all rows, and found no problems in a fresh install, so this is promising, but I cannot say that there's no way to corrupt the specs now.

In fact, this new technology introduces dependency on external software for compression algorithms and so an accidental update to one of those packages can render all specs unusable to the rest of the machines.

And the specs are now bigger (when not compressed) and slower to read (i.e.: a byte would now be stored as a some-20-character-long string), plus it takes some CPU to [de-]compress them.

And the package builds now dump the specs _TWICE_, so although each time it's faster than the old build process, overall it may well be comparable.

It does open some additional possibilities for tools development, of course, like a specs editor tool I'm now designing, so I'm not really complaining ;-)
 
Re: XML \'Specs\' in 8.12/TR8.96

"Another thing to keep in mind is that UBEs read their specs from the database each time they run."

Not true in more recent 8.96 releases - they read the xml and build a spec cache where each individual UBE has a tiny and independent tam file. If you change the objects, the metadata kernel will clear the cache for the object and regenerate it. UBE performance is about the same, at least 8.96.E1 and later.

As far as picks and invoices, you run them in subsystem mode as always, and they only ever read specs once anyway so this is a moot point.
 
Re: XML \'Specs\' in 8.12/TR8.96

Thanks for the correcting my information. My UBE performance study was done prior to 8.96E1. At the time I was informed of the potential performance issue by an Oracle field consultant who works with the performance and benchmarking group. I am glad to hear that they have worked around the performance issue by creating a TAM cache. I have recently put down F1 and will run my testing again. I am sure the performance will be about the same as pre XML specs as you stated.

I wonder if TAM locking is still done on these cache files. I would hope that once cached the files would be opened in read mode and allow for multiple instances of the same UBE/version to read from the file simultaneously. I'll have to test for that.

As for subsystems I'll take the "open to discussion" definition of moot and maintain that it is a choice as to whether to run picklists, invoices in a subsystem or not. Unless the long-term stability of subsystems has somehow improved I believe it is still something to be done only after some thought. Other than numerous negative personal experiences with subsystems my premise for this is:

JDE code can leak memory either out-of-the-box or due to customizations. (Leaks can be anywhere in the stack - application code, runtime engine, middleware) A UBE is subject to memory leaks.

A subsystem job is a UBE that runs in a loop and may leak memory in each iteration. Over time this may cause the job to behave upredictably or crash altogether.
 
Re: XML \'Specs\' in 8.12/TR8.96

"I wonder if TAM locking is still done on these cache files. I would hope that once cached the files would be opened in read mode and allow for multiple instances of the same UBE/version to read from the file simultaneously. I'll have to test for that."

Locks are there, but each ube is given a unique set of tam with only the objects it requires, so there is no tam lock contention -- even for multiple runs of the same report/version. Those 'runtime tams' are separate from the main cache, which is used to copy from. They are deleted at the end of the job, if I recall rightly.
 
Re: XML \'Specs\' in 8.12/TR8.96

But wouldn't this mean that it's still read (and written) for each UBE run and so is still slower than before XML?
;-)
 
Re: XML \'Specs\' in 8.12/TR8.96

No, cause it's copied from the already generated spec cache for everything but data selection and sequencing. So the rest of the data is already there. In practice, it really is about the same speed once the cache is established.

The first time you run one, it can take awhile.
smirk.gif
 
Back
Top