AS/400 HARD Crash

scott_parker

Reputable Poster
Greetings.

Yesterday we had a power outage at about 2:50pm. Our UPS kicked in but for some reason it failed at 2:54pm. (so much for the backup power)
Anyway, we restarted the AS/400 at about 3:30. Once back up at about 3:45 or so we did a controlled IPL (needed to completely disconnect it from the UPS) and then went on our merry way.

A problem that was discovered was that about 3000 sales order lines had gotten set BACK 1 or 2 Status. Lines that had been canceled were no longer canceled. Lines that were Ready for Invoicing were back to Ready for Ship Confirm.
Lines that were Ready for Ship Confirm were now Ready for Picking.

We looked at the Sales Ledger file and the records in that file SHOWED a particular order was at ready for invoicing. But the F4211 showed Ready for ship confirm. With these 2 files being so out of sync we had/have a small nightmare on our hands.

What we have done so far is to set all the status back to what the last Sales Ledger record shows.

(we also plan on doing a cycle count of any and all items that were picked/shipped for the last 2 days.)

Calls to JDE and one to IBM has not gotten us anywhere. Neither have any clue as to what could have caused this.


Anyone out there ever see something like this happen before???


(the UPS repairman is on his way)



Scott Parker
Grote Industries, LLC.
WorldSoftware Version 8.1.2 AS/400 V4R5
 
Check system value 'QUPSDLYTIM' make sure that you are allowing enough time
to shut your system down properly.

Example: If the value is 300, then you only have 5 minutes before the
system shuts down, the value is in seconds.

You can bump this value up accordingly. If this value is ok, then you may
want to search logs in system service tools for additional information in
order to isolate the problem to either the AS/400 or UPS system...

Hope this helps

Ernest Givens
System Administrator
SCB Enterprise Solutions
480-539-2224
 
Scott

It sounds like the F4211 was pretty heavily cached in memory, and the ledger
history was not. Thats a pretty wild situation. Did any other associated
files have problems (ie. cardex, loc/lot inventory, etc)?

Wow.

Chuck
<P ID="edit"><FONT SIZE=-1>Edited by ekempter on 2/16/01 02:06 PM.</FONT></P>
 
Re: RE: AS/400 HARD Crash

That value is set to *NOMAX
We do NOT want to shut the system down just because it went on the UPS.
We have a job that monitors for the message about the UPS kicking in and after 10 mins it starts shutting things down. This job had started its "Count down" but due to the fact that the UPS just died it did not finish.
In this case though the UPS Failed and acutally burned out.
So what happened is basicly the AS/400 got unplugged.

(just found out we have to purchase a new UPS.)


When fully restored again, the AS/400 seemed to have reverted back to a Sales File (F4211) that was about 1/2 a day old. I have looked in the History Log and can see when the UPS kicked in, and then when the IPL of us re-starting the box. There is about a 1/2 hour gap of no messages from when the box was actually down.


Scott Parker
Grote Industries, LLC.
WorldSoftware Version 8.1.2 AS/400 V4R5
 
We have at the moment only found 3 records from any other files that dont seem to be correct. F41021 (inventory file) not in sync with the Cardex. Cardex is correct.

I knew that a program would "Cache" records that are to be written but I thought that once that program ended it would actually write the records.

As far as I can tell we were not doing any "Batch" updates to the multiple of records that seem to have been effected. If I was seeing only 1 "Set" of errors then I would really suspect that. but because there are different "Sets" of Status codes that are messed up I do not know of any one program that could have been running.

Picking, Ship Confirm, Invoicing. There is no one program that does ALL that.

Doesn't the AS/400 Immediatly Write/Update records to the file when a program is completed?



Scott Parker
Grote Industries, LLC.
WorldSoftware Version 8.1.2 AS/400 V4R5
 
Have you checked QRECOVERY for journal receivers? They will/should show the
before and after pictures of records that were in use at the time of the
crash. At least I have found this to be true and work for us when we had a
hard crash.
Liz Abernathy
Smith Fiberglass Products
Little Rock, AR 72209
 
Scott

I am not positive that the AS/400 immediately writes records when a program
completes. There are several factors which determine how this occurs. File
parameter FRCRATIO (on creation of a file), can influence how records get
forced to storage. Journalling also would be an important factor, as
journals do get forced to storage, and upon IPL recovery, update the files
as necessary. (pretty sure of that, anyway). I wonder if there is some
object damage. Although, if there were object damage, I would assume the
F4211 would be unreadable...

My assumption is that when the 400 goes down hard, like it did, that all
bets are off, and you wander into "unpredictable results may occur"
territory.

At least its Friday...
<P ID="edit"><FONT SIZE=-1>Edited by ekempter on 2/16/01 02:04 PM.</FONT></P>
 
I think your right one the unpredictable results thing.
and no I am not glad its friday. This means I will probally be working tomorrow because we need to install a new UPS.

hehe isnt IS work fun.

Scott Parker
Grote Industries, LLC.
WorldSoftware Version 8.1.2 AS/400 V4R5
 
Re: RE: AS/400 HARD Crash

Scott,

This sounds like an issue outside of JDE. You may want to float this question
on the midrange-l list.

Phil

<P ID="edit"><FONT SIZE=-1>Edited by ekempter on 2/16/01 02:04 PM.</FONT></P>
 
Re: RE: AS/400 HARD Crash

I know very Little about journals.
How do I "Check" one?



Scott Parker
Grote Industries, LLC.
WorldSoftware Version 8.1.2 AS/400 V4R5
 
I have been through several AS/400 hard crashes (in the days before UPS was
affordable or considered necessary for non-sales oriented business). I have
never had this problem, and was always on JDE. Sorry I can't be of more
help, but thought I'd at least share that. In my experiences, only 1 or 2
records (at most) are not written/updated.

Regards,
Sally White
<P ID="edit"><FONT SIZE=-1>Edited by ekempter on 2/16/01 02:03 PM.</FONT></P>
 
Doesn't the AS/400 Immediatly Write/Update records to the file when a
program is completed?
Yes.


Thanks,
Elizabeth (Liz) Hill
JDE National Practice
IBM Global Services
Client Office 281-335-4467 T/L 260-4467
Home Office 281-459-2044
 
RE: RE: AS/400 HARD Crash

Well I am not well versed either, but if you go to the QRECOVERY lib, take
option 12 to open you will see if there are any Journal Receivers. If I
recall I simply did a GO CMDRCV and found all of the commands need to view
and/or accept the change. We a had a tornado 2 years ago during production
hours and lost power as well as UPS support due to lightning. So..it has
been awhile but I am sure you can get help from IBM IF you do have a journal
receiver in your QRECOVERY lib.
Hope this helps.
Liz
 
RE: RE: AS/400 HARD Crash

Been exremely happy with APC 3000NET WITH POWER CHUTE SOFTWARE.

Don Bryant, MIS Director, mailto:[email protected]
Amalie Oil Company
1601 McCloskey Blvd.
Tampa Fl, 33605



Don Bryant, MIS Director
Phone 813-248-1988 ext 124
Fax 813-248-1488
 
Re: RE: AS/400 HARD Crash

Scott

You probably do not have journalling on your files in your production data
environment. If you did, I would be very surprised if you had the data
phenomenon you had. To see if a file is journalled, use the DSPFD command,
TYPE(*ATR) (as journalling is part of a files base attributes), check the
last parameters of the file. It will probably show "File is currently
journalled *NO". If the file is journalled, you will see additional
parameters which indicate to which journal object the file is journalled,
whether you have "before and after" record images, or just "after" images,
and so on.
<P ID="edit"><FONT SIZE=-1>Edited by ekempter on 2/16/01 02:01 PM.</FONT></P>
 
Back
Top