Very Unusual Security Issues

[peterbruce]

JDEList,

We have an unusual problem. The security kernels stop writing to the logs on our enterprise server. Additionally the P98OWSEC doesn’t seem to work properly. We have LDAP enabled, so form W98OWSECI (security detail revisions) requires you to tick a box to allow change changes to a specific detail. However, when attempting to re-enable a user id I found that the tick boxes were disabled, but the detail entry fields were editable – just like it is without LDAP enabled. I changed the status from ‘02’ to ‘01’ and click OK, but the user detail was not updated. I did the same thing on both the web client and the fat admin/dev client, with the same results. I had the debug log on the fat client and it showed that the LDAP_isLDAPAuthenticationEnabled business function returned a ‘0’ instead of a ‘1’ – meaning that LDAP wasn’t enabled, but it is enabled!

This is a recent behaviour going back about 6 weeks or so to mid-February 2017. About this time we had a firewall upgrade. I don’t know how a firewall upgrade would stop the security kernels from writing to the logs.

We have two installations that are completely separate. One for production and the other for testing and development. The problems are occurring in both environment.

Our installation configuration is: E9.1 TR9.1.2.1, Enterprise Server: Sun; Database Server: Sun; Oracle DB: 11g, Weblogic; Create!form 7

Any help would be gratefully accepted.

 

[Alex_Pastuhov]

Generally speaking, AD/LDAP are generally somewhat unreliable, so the client software would often need to wait and sometimes retry. I think I remember some old posts here saying the JDE security kernels can sometimes get locked up as a result. And if that is what you are seeing, then it would probably explain why it stops writing to the logs and it could conceivably misreport its status as a result too. I guess if you reboot the security server, it would probably start working again.

And there are multiple different ports it can communicate over, plus potentially multiple DC's, so firewalls can cause comms issues. Is there anything in the logs?

 

[peterbruce]

Alex,

I appreciate your response.

I'm not sure about the Enterprise Server OS logs and the Firewall logs, but there is virtually nothing in the JDE logs. The only reference was from a JDE Net process saying it did not get a response from the security kernel.

As far as I can see the issue doesn't involve the LDAP connection (at least directly), depending on exactly how the LDAP_isLDAPAuthenticationEnabled business function operates.

 

[ice_cube210]

HI Peter , have you tried putting the security kernel in debug mode when this issue happens and see what's being written ? The security kernel has this limitation that if it looses connection to the LDAP server even for a split second then it will remain looping on that error of not being able to connect to LDAP , even though the LDAP server may be up and running fine. The only way to recover from this situation is to kill the affected security kernel (or restart services ) , this what I had to do at one of my customer sites where JDE would loose connectivity to LDAP server often.

Since the issue is happening on both of your JDE instances I think the LDAP server is definitely involved , assuming they are both pointing to the same LDAP server. Is the LDAP server virtual ? If it is part of a vmware vMotion setup check that it wasn't being moved around when the issue was observed?

 

[peterbruce]

Ice,

Thanks for a very interesting reply!

I was not aware that security kernels can be put into debug mode. Is this a JDE function? Or is it an operating system function? How do you do it? All our JDE servers are unix/sun/zones (sun's virtual machine) with the exception of the Deployment server of course. I wasn't certain of the involvement of the LDAP server (unix/sun) in our issues. Users can still log on and off without issue. Though there may be an occasional problem with users becoming disabled in JDE before the maximum password attempts is reached.

The same LDAP server and firewall are involved in both the Production and test/Development installations.

 

[ice_cube210]

HI Peter,

Putting the kernel in debug is a JDE function and can be done using Server Manager (Or using SAW back in the old days) . Under the Runtime Metrics of your Enterprise Server in Server Manager click on process detail and go into a Kernel , you will see the option to turn on debug on the kernel.

Check out this Server Manager guide for tools 9.1 available online. I have linked the section that talks about this

https://docs.oracle.com/cd/E24902_01/doc.91/e18837/view_runtime_metrics.htm#log_file_configuration

 

[peterbruce]

Ice,

Thanks very much for the information on the kernel via server manager. I'll look at that tomorrow.

 

[gopal122]

Hi Pete,

Is the issue still there? Please let us know your findings.

 

[peterbruce]

Gopal,

I raised an SR with Oracle Support, who were very helpful. They eventually provided a POC which I have installed in our test/development instance. It has been a couple of weeks now, but so far it looks good.