UNICODE poll - do you use it, are you planning to...?

[rgreensl]

Hi everyone,
We are going live on 8.12 next year and I need to decide whether to go unicode or not. I know there is a 3 ->7% performance benefit but also that it will approximately double our database and impact on backups and increase our line utilisation to our HA machine. As well as that it will be an extra step on the upgrade weekend. From my point of view I'd rather not do it.

My manager is of the opinion that we must go unicode as this is what JDE is written / tested with, and therefore support may be compromised. This was also the message given at a previous open world (2007 I think).

This year I attended open world and Oracle were of the opinion that you don't need to convert at all. I couldn't find anyone who was on Unicode that didn't need it for multi language support.

As a result I thought I'd do a quick poll to find out who's on unicode or planning to be.

I can then use the results to persuade the powers that be to make the correct decision.

thanks,
Rich

 

[Sebastian Sajaroff]

Hi Warwick,

Here are some comments from my experience on Unicode

1. Your database will grow by ~40% (not all fields grow
2x, only alphanumeric ones do).

2. Disk is not as expensive as it used to be a couple
of years ago. In fact, you'll spend much more money
on the upgrade project itself (customs, retrofit,
CNC and Developer hours, functional analysis, etc)
than on a bunch of hard disks.

3. JDE is definitely going on the Unicode direction,
so : the sooner you take that ship, the cheaper will be
the ticket!

4. What platform are you planning to run E812 on?

5. If you're running any 3rd party or made-in-house
apps with JDE, then check ASAP for their Unicode support.
Typical examples : RPG programs (iSeries), custom PL
scripts (Oracle), Visual Basic apps (Windows), T-SQL
code, Access and Excel macros, etc.

6. If you're planning to support multi-languages, then
Unicode will make your life easier.
Far East, Cyrillic, Arabic, Greek, Hebrew, etc...users
are so much easily handled by Unicode than by dealing
with DBCS, and several CCSIDs and pagecodes.

 

[rgreensl]

Hi sebastian, thanks for your comments. I have responded as follows:
1 I tried a test database and it grew from 25->35Gb so that sizing is probably about right
2 it's not so much the disk size it's more the backup overhead and that of replicating data changes to our High availability machine. The cost of disk could perhaps be better used in prc / memory...?
3 I just don't see many people on unicode at the moment. Every presentation in san fran had customers on non-unicode dbs. Our UK user group has no-one (as far as I'm aware) on unicode
4 we'll be running on iseries, all on i most likely.
5. We're well aware that we'll have to update our 3rd party interfaces with JDE files. We're revisiting anyway due to table changes in 8.12. It would be less work if we didn't change though
6 We're not planning to support multi languages. Just a UK only company.
Rich

 

[altquark]

Don't be one of the few non-unicode customers out there.

All new installations of 8.9 are unicode - so only the upgrades are non-unicode. Many people are performing fresh, parallel installations - and migrating data/code across rather than performing the upgrade process - so the majority of upgrades/installs are going to be unicode based.

The only platform users which seem to have issues with this are AS/400 users. Is it because they historically viewed DASD as being significantly more expensive on that platform ? I'm not sure why I continue to hear that "disk is expensive" - each time I get a quote, it seems to be very similar to PC and Unix pricing. You're going to need disk - so might as well add 40-50% more for unicode.

Believe me, you'll be left behind if you don't convert to unicode. The only reason NOT to go unicode is 3rd party support - and so far I haven't heard any partner products NOT being able to support unicode....

 

[Bill Dotson]

It's true, historically iSeries disk drives have been somewhat more expensive than their Windows and Unix counterparts. That price gap has narrowed over the past several years, although the high-speed, high-capacity drives for the iSeries still seem to lag behind Windows and Unix. Regardless, I would rather pay the price for the disks than pay the performance penalty for not converting to Unicode.

At some point, not converting to Unicode is going to bite you in the butt.

 

[cmanderson]

...and for those how don't know, by now I would think most do, but for the newbies...

If you're running EnterpriseOne 8.9 and above, you're running a database with Unicode schemas (System, Code, etc.)

It is the customers choice to convert Business Data/Control Tables after the upgrade...

 

[list6654]

Re: UNICODE poll - -> !UNICODE==(Poor Performance)

If you do not convert to unicode, JDE app performance is degraded because the db and the jde kernel code has to translate to/from unicode.

 

[Alex_Pastuhov]

Re: UNICODE poll - -> !UNICODE==(Poor Performance)

This is what Oracle claimed, but Unicode will actually likely make your system slower: ~40% on-disk data size increase will directly translate into a proportional performance problem.

It's faster to convert strings to or from Unicode in memory, than to read and write more from/to disk.

This also means that you'll need more memory.

So, you should rephrase this question as
"!UNICODE==(Better Performance)".

 

[cmanderson]

Re: UNICODE poll - -> !UNICODE==(Poor Performance)

I'm sure Oracle would love to sell you some Oracle Advanced Compression for 11g - that would, in theory, get performance back on par with non-Unicode Xe. ;-)

 

[list6654]

Re: UNICODE poll - -> !UNICODE==(Poor Performance)

My experience has shown me this that the runtime Unicode translation is not insignificant. I have seen performance in both situations on the same business data. The app was much slower in non-Unicode than in Unicode. I have seen this on more than 1 installation. My users basically validated for me what JDE recommended.

In most JDE installs with Unicode, you are talking about UTF-8. UTF-8 encodes up to 4-byte octets for any given character. For character set ISO Latin databases, it just so happens that for the standard code page (Latin), the first octet uses the same binary character representation as ASCII. And, for standard ASCII characters, Unicode uses the same number of bytes as ASCII (1 byte). The other 3 octets are zero's (unless you are using a character whose encoding is greater than 127). Most RAID controllers use some form of internal compression while processing the disk I/O. The 3 high octets would be ignored internally. Unicode's affect on RAID performance is influenced by the volume of decimal data.

What you are pointing out really pertains to disk management in general. If you have improperly sized disk devices or a lack of free growth space for the database disk files, performance will be degraded. But this is true even if you are not using Unicode.

The JDE Unicode translation, on the other hand, is dependent on kernel code to do the translation on the fly. If you have system data in Unicode and Business Data in non-Unicode then you have a problem. On dissimilar databases if you do a key comparison or resequence the data in memory, then you have to do a translation on the fly. I suspect that this is the performance cost that I observed.

I would be curious to see what data you have that says there is no performance impact from Unicode (assuming that suitable storage is in place). I stand by my statement that !UNICODE==(Poor Performance).

 

[cdawes]

Wow seems like a lot of FUD comming from some supposed experts.

Here's the part line coming from JDEdwards directly.

1. JDE is not forcing Unicode now or in the forseeable future. There is a very specific document on when to go unicode or not. In fact JDE has reversed their position on Unicode and saying only "do it if it makes sense". Check with Clayton Seeley on this on is you have doubts.

2. Unicode increases disk space by 40 to 80% which also increases yout back up window. I like a small backup window...what do you guys like?

3. Hardware gets cheaper over time so why go to unicode now if you don't have to? Buy cheaper disk in the future if you decide to go unicode.

4. Is non-unicode SLOWER than unicode? The answer is YES here but by how much? IBM's tests on the AS/400 indicated that there was only 3% difference. Other system tests on other platforms showed up to a 5% difference. CPU is cheap, disk is expensive.

I can't believe the FUD that is out there coming from the talented people on the list? What benchmark tests are you quoting? I'm quoting the ones done by the VENDOR's themseleves. Where's your evidence from a controlled test?


Colin

 

[list6654]

CPU is cheap, disk is expensive.

If your expert opinion is that CPU is cheap and disk is expensive, then I am going to have to call "B_______!".

You will have no problem googling the relative cost of disk over time vs. the cost of database-class servers. You will find very quickly that the cost of disk has been going down steadily relative to servers for many years.

When was the last time you added a processor to an existing box? Other than AS/400's I know of no companies doing this. My experience with AS/400's is that companies don't buy processors alone - they have to buy memory in tandem too. And, AS/400 processors are not cheap.

You say that "CPU is cheap" but the cost of the CPU isn't the issue. I do not believe most server vendors even leave a socket open any more. Basically you are talking about adding a server _or_ even worse, rehosting a database from one box onto another more powerful one. The cost comes in the migration - building up the box, migrating the database, etc. Those costs are huge relative to resizing a partition or migrating a disk file.

 

[altquark]

[ QUOTE ]

1. JDE is not forcing Unicode now or in the forseeable future. There is a very specific document on when to go unicode or not. In fact JDE has reversed their position on Unicode and saying only "do it if it makes sense". Check with Clayton Seeley on this on is you have doubts.


[/ QUOTE ]

I think this one statement represents the confusion that most customers see all the time coming from JDE and GSS. JDE do not seem to be able to consistently come up with a decision one way or another where some sort of choice is concerned.

Oracle/JDE needs to come up with an announcement that explicitly states whether customers who are upgrading should go unicode or not. My information was current as of 3 months ago, where GSS explicitly stated that it would be "preferable" from a support standpoint to implement a unicode database during an upgrade - and this statement has been carried forward since prior to 8.9 was released. Therefore, my guess is that the statement from a specific member, while certainly based on performance information as far as JDE in concerned, is in contradiction to what customers/support people and everyone in the industry has heard for the past 5 years.

REMEMBER, however, that FRESH INSTALLS of EnterpriseOne will REQUIRE you to implement the business database as Unicode - so the statement that "JDE is not forcing Unicode now or in the forseeable future" is completely incorrect as far as new installations are concerned. They ARE forcing unicode - and as such, it is wise to go with what new customers are seeing. I'd rather BE unicode today in line with all the new customers, not lagging behind because there are a number of "upgraded" customers. But thats my personal opinion, and nothing to do with what JDE states (in fact, everything I've ever stated regarding best practices has been personal opinion - not necessarily what JDE states).

As someone else has already pointed out, the system data and data dictionary, central objects etc are always set to unicode under 8.9 an above (again, JDE forces that). Why mix unicode and non-unicode ? It just doesn't make much sense to me.

Personally, I've never stated that Unicode is a performance increase or decrease. I've always viewed the fact that Unicode provided about an equal performance compared to non-unicode as far as JDE was concerned. Certainly, other factors are a far larger concern when it comes to performance. Just remember, if you are using multi-language packs - then you really should be converting your business data to unicode.

However, unicode DOES increase the size of your database about 50% or so. Which therefore increases the time it takes to perform a backup. Of course, really LARGE customers that need 24/7 and utilize hardware solutions to provide their backup really don't see much of an impact to a larger database being backed up. The time it takes to write a snapshot image is still a few milliseconds or so !

Just my 2c