Data-only Upgrade questions

Hi everyone, I need your advice:

We are doing a E811 -> E900 (Win/Oracle, E900 is on TR8983) data-only Upgrade, the PD900 is already installed and functional there, with all the tables. We are going to do a test run first to estimate time req's and then the final upgrade.

Now, the questions are:

1) Do we need to drop all target E900 PRODDTA/PRODCTL tables before we start the Upgrade Plan, or is it supposed to clear them on the fly as needed?

2) Are data-only Upgrades known for doing any changes to the source tables? (because it's live PROD data in E811)

TIA!

1) No, the upgrade plan will create new tables, reindex everything that has new indexes, alter any tables that have a new structure, and leave the rest alone.

When a table has a new structure, it usually creates a table with a temporary name, moves the old data into the temporary table, creates a table with the original name but the new structure, then moves the data back, along with any other logic needed to fill the new fields.

2) For the above reason, you do not do a data-only upgrade to Production until go-live. All our timing tests usually happen by copying PD data down to PY, and running the data-only upgrade plan for PY.

Ken,

Many thanks!

So, 1a) you are saying that the existing data in the target tables will be preserved - this would not be very good in this case. Maybe we should at least truncate them first?

And 2a) I know, it's just that PY had been Upgraded before and we had major changes in the system since then (different hardware, etc.), so the client really wants to do this twice in PD now. Plus, of course, all the target Environments were created by difefrent people at different times using different techniques, so there's really no telling that what works in PY would work in PD just the same. And, of course, not the least issue is that what's in PY E900 now, contains some task & UDC customizations, which we will need to then bring across to PD after the final Upgrade, so wiping PY data with another Upgrade would mean that we will first have to copy it down to DV.

And actually 2b) if we do this twice in PD, would the second run be just the same, or does JDE remember anywhere that this has happened before and try to skip any steps?

TIA!

1a) You want to have a PD environment with 9.0 table structures, but otherwise empty of data? Just want to make sure that I am understanding what you wrote ...

2a) Yes, it is typically the case that the application consultants have staged the soon-to-be Production UDCs and menus in PY, and we wind up copying them up to PD after the data-only upgrade plan has run.

2b) I have found that running an upgrade plan to an environment where one has already been run will result in many lines of the conversion plan wanting to go to status 70 (Duplicate), and that I have to make sure they are all set to 30 so they will run again.

1a) I want the data to come from E811, so whatever is already in the target tables, I do not want that to be preserved.

As I said, in the current E900 PD, there are all the tables with data there already. Plus, if doing the Upgrade twice in PD, what with the data that the first Upgrade would bring across from E811? - it would make it the same as this current starting point all over again, with all the target tables containing data.

So the question is: should it be all truncated (or indeed dropped?) before each Upgrade Plan?

2b) Ok, that was to be expected, thanks for the confirmation. But once set back to 30, the actual upgrade TC's would not be skipping anything once started, right? - this is what I don't know for certain. I know that they generally shouldn't, but then again, it's JDE, so who knows.

As you noticed, I'm not doing upgrades frequently any longer ;-)

Many thanks for your advice again, much appreciated!

And the other question that I had: it's not going to change anything in the source E811 PROD DB during an upgrade, is it? I have seen XE Upgrades wiping source OMW config clean, so I'm a bit suspicious about what it might be doing in the source DB...

1a) OK, if I understand correctly, the 9.0 PD database was loaded with some data from previous 9.0 installation, and the 8.11 PD data is in a separate place?

If that is the case, then I'm not sure will happen. I have only been doing "in place" upgrades the last few years, and I don't remember what happens if a data upgrade goes between 2 different databases. I think that you will need to have the PD Business Data for 9.0 be empty, as you are wanting to do.

You should check the data sources and the OCM in JDEPLAN before you do this. In order for it to work, you should probably have PD900 owning the mappings for Business Data - PROD and Control Tables - Prod, and those 2 data sources pointing to the new database, while the OCM mappings for PD811 should have different data source names for Business Data and Control Tables, with those data sources pointed to the old 8.11 database.

Hopefully this is making sense ...

2b) Once you reset the status to 30, and Save it after making the changes, then you will be good to start. If you want to make sure, just go back into the Conversion Workbench and check the statuses again after you Save them.

Other) No, in the scenario I think you are doing, the original data will not be touched.

Yes, that's right.

I'd prefer an in-place upgrade too, but in this case, the source data is still non-Unicode, so I cannot do this: not with the same DSN.

Plus, the Upgrade reads the DSN off the old System E811 in the beginning of the plan, so there's no way to set it to anything else in advance - it will be lost. How do you do that, I wonder? - a fake System DSN?

In this case it's not an issue: it will read the Business Data - PROD DSN definition straight off the old System - E811, so I don't have to do anything, it would know where to get the old data from by itself.

Ok, empty sounds good. But to what extent? - truncate the existing tables, or drop them entirely? I think JDE may not create new tables for what it does not need, so the end result with an empty schema in the beginning, would probably be an incomplete DB, I'm not sure. But on the other hand, some TC's may balk at touching pre-existing tables, even if empty.

Excellent, thanks again for the confirmation!

Just so I understand this correctly..........

System A is 811 on seperate hardware and System B is 9.0 on seperate hardware?

At somepoint you brought up 9.0 on the new hardware and converted the data?

At somepoint you did a "test go-live" into PD900 or are you actually running a live system on PD900?

I've done 15+ 8/12/9.00 upgrades - 1/2 AS400 and 1/2 Oracle and in all cases we always do a test go-live into the PD812/PD900 environment.

Let me know the exact setup and I should be able to describe a good methodology of precise steps.


Colin

Yes, correct: separate systems entirely.

PD900 now has a copy of the data that was upgraded from PY811 to E900, it is not live, but working in terms of we can test it and sign in into PD900.

In other words: there was a new E900 setup, then new PS900 was created, DV was Upgraded from DV811 (copy of PD811 data), then PY was Upgraded (same and again, not by me) and a PD900 was setup in E900 with the specs and data taken off PY900. So, while PD900 has a good upgraded DB, the Upgrade never took place in PD900.

Thanks!

The purpose of reading the old System data source is to bring across things like Location, OCM mappings from the previous release, etc.

Since that has already been done, there is no need to do it again. I get it to skip that part, so the old System data leaves the modified data sources and OCM mappings alone, and you'll be able to convert the data between the two databases.

Unfortunately, I don't remember the details of exactly what it does when you are not doing an in-place data upgrade. I vaguely remember having a set of empty tables in the target data source, but I don't remember if the table conversions recreated the tables anyway.

Maybe Colin has run into this situation more recently, and can tell you what will happen.

Thank you!

Hi,
Maybe I made a mistake, tell me if it's the case.
My steps for a data-only upgrade from 8.11 to 9.0 :
1. export PRODDTA and PRODCTL from 8.11
2. drop schemas PRODDTA and PRODCTL in 9.0
3. import PRODCTL and PRODDTA (8.11) into 9.0 database
4. create and run your data only upgrade plan.
Delphine

Okay so all you want to do is re-convert production ........PD900, time and time again. You can do this as many times as you want (I do this atleast twice for major upgrades).

The short answer is that you should simply drop all of the tables in the E900 PRODDTA and PRODCTL. Leave the PD900 schema where it is. Import PRODDTA and PRODCTL from E811 and drop it into the E900 database. What you end up with is a PD900 environment with data in 811 format.

1. Ensure F04571/F04572 are clear on 811
2. Run all end of day processing on 811
3. Stop 811 and export PRODDTA and PRODCTL
4. Import PRODDTA and PRODCTL to the E900 system (PD)
5. Set up your E900 system for upgrade processing (INI files, reset passwords, truncate F00950 and restore from Planner, etc.)
6. Run a select distinct on the F984052 to see whih ESU's you'll need to reapply after the upgrade (only need to apply those that do something other than an R98700)
7. Delete the TC History for the target environment in P984052 for any revious UPGRADE plan
8. Delete the Table and Index Creation history for the environment in the P984072 (EVERYTHING)
9. For step #6 delete all ESU history except the R98700 records in the P984052.
10. Clean up Media Objects, shut down Server Manger, Peoplebooks, yada yada yada
11. Create your PD900 upgrade plan
12. Execute your upgrade plan.
13. Confirm the Upgrade and do the Post Upgrade Manual TC's
14. Reapply Update Update 1/2
15. Reapply ESU's identified in step #6
16. Compare UDC's in the newly upgraded PD900 to your staging PY900 and fix the differences
17. Look for OCM differences if the Planner ESU changed between D900 and PY900 upgrades
18. Copy setup tables from PY900 to PD900
19. Change system to LIVE processing from UPGRADE processing
20. Restore security table

I'm off to Ottawa tomorrow for a 9.0 go-live and this is pretty much the summarized version of my 500+ line plan.


Colin

Colin,

Thank you very much! It's a very comprehensive plan with things I wasn't even aware of! Much appreciated!

Your ##'s 6, 7, 8 (especially) & 9 are exactly what I was looking for!

Can you please elaborate on ##'s 5/19? - are you talking about the [DSPWD] stanza here? Or are you also saying the JDE password must be "JDE" for the entire process? Or something else I don't know?

There is also a small issue here, though: what you are describing is an in-place upgrade, but my old E811 data is still non-Unicode, so the original DSN is non-Unicode and the target DSN is Unicode. Do you know if this would work? I.e.: if I copy the data from E811 and keep the BD-PD900 DSN as Unicode, then JDE may well be confused with what it finds.

Maybe I should stick with running this Upgrade across the two DB's, right off the live E811 data? - this is the major question I am trying to find a definitive answer for...

Please refer to Colin's plan, it's very detailed.

Is your E811 data in Unicode format already?

Believe it or not but the password sshould go back to default for the upgrade. If you go from pre 8.9 to 8.11 or later there are a few TC's that look for the default passwords and fail. At last check it was only 2 but they are big conversion chains (I can look up the exact ones).

So this means resetting all the schema password s to default, JDE password to JDE, etc.

I'm not a big fan of unicode unless you're going double byte. I would just set the data source flags to non-unicode on "Business Data - Prod" and "Control Tables - PROD" and then if you really want to then do the unicode conversion after the upgrade.

Just make sure you don't have some environments as unicode and others as non-unicode. JDE actually works and doesn't really care what format the table is in but it just looks kinda weird. I was actually amazed to see that it worked this way (found it at a client audit). I'm still dumb struck for an explanation on this.

When you do the upgrade it does not copy the data from database A on server X to database B on server Y. The upgrade of data is IN PLACE.

The upgrade of objects does however copy across databases and data sources, The tought process here is that JDE delivers 99% of the code and the client modifies 1%. With the data JDE delivers 1% and the client puts in the other 99%.

So overall you can insert the unicode convertion anywhere between steps 11 and 20.


Colin

Ok, thanks, yes, I suspected as much.

Yes, I feel the same about Unicode, but I think E900 does not really give you this choice anymore, except that the configuration options are of course still there. So this _is_ actually an option, especially if there are confirmed cases of sites using it.

That will be exactly the case here, if we go non-Unicode for PROD, because PS, DEV & CRP are Unicode already ;-)

Interestingly enough, I noticed that even if your DSN says non-Unicode, JDE can still read Unicode data through it - it logs an error, but seems to be correcting it on the fly.

Ok, that's news to me, I thought I could do a cross-DB upgrade.

But then when you create a Plan, what would you use as the source DSN? - the same BD-PROD as the target? I was actually under an impression that it would be using the old BD-PROD, read off the old System - E811, which of course it knows about and can connect to. Or does this imply setting F0092 Initial Task to Completed before you kick it off?

After creating your plan, but before running it through Installation Workbench, check on the data sources (Planner - 900, but it wouldn't hurt to check on the existing System and Server Map too).

I normally disable the step of bringing over the System data on the later passes of data upgrades because there is usually nothing from the old System that you want overwriting anything in the new System.

This leaves me free to make sure (and modify if necessary) the data source definitions so that they are pointing to the database that I want to upgrade, and not the current PD811 on the old server.

Thanks!

I remember that this gets overwritten by the Installation Workbench (itself, not by any UBE it would subsequently run) about 1 min in. Unless this has changed since when I did this last.

I clearly remember watching for it after kicking off a Plan, to fix it quickly before it's being used later on.

BTW, do you have any sites running E900 with non-Unicode CTL & DTA DSN's?

I have a ton of sites running E900 with non-unicode.

Rememebr that an upgrade by default is non-unicode if it's from pre 8.9.

I also have a few clients who were net new or migrations who did not want unicode.

All you need to do is drop all the tables, change the data source prperties and use the R98403 to copy the tables from planner and voila --> non-unicode!

Typically for any World migration it's default non-unicode but it's a pain with your custom RPG so I just get rid of it.

Data volume stays down, disk price stays down, backup window stays down.


Colin