ORA-03137: TTC protocol errors --> JDE connection issues

kororia1

Member
Hi All,

We've went live recently and are starting to see periodic instances of the following in our Database Server Logs


ORA-03137: TTC protocol internal error : [12271] [] [] [] [] [] [] []

Its quite random in time and frequency and not something we've been able to replicate at will. What we do know is that when we see these DB log entries we may have a minor / major issue in JDE. For example the Scheduler kernel may be affected and stop submitting jobs or possibly a Call object kernel affected and hence a user transaction fails. What we do know is that its directly related to stability issues in JDE.

Now, what we do know is that the statements that are throwing this issue are always related to spec table lookups and its been reported by Oracle from the logs we provided that the bind variables are being corrupted or badly formed. It always seems to be a problem with the object name field i.e this value contains unreadable characters or has a value that's not the correct length (ASCII code is too short) This results in the oracle client issuing a SQL to the DB that then throws this Error (and leads on to DB connection issues for that kernel). In some cases we have to restart the enterprise services to address this.


Sample statements (others but all package spec related, none related to transaction tables)


SELECT * FROM PY910.F98740PYF150525 WHERE ( ELPRDTYP = :KEY1 AND ELOBNM = :KEY2 ) ORDER BY ELPRDTYP ASC,ELOBNM ASC,ELVERS ASC,ELFMNM ASC,ELCTRLID ASC,ELWEVENT ASC,ELERID3 ASC;
SELECT * FROM PD910.F98743PDF150722 WHERE ( DTOBNM = :KEY1 )
SELECT * FROM PD910.F98760PDF141124 WHERE ( RTOBNM = :KEY1 AND RTVERS = :KEY2 ) ORDER BY RTOBNM ASC,RTVERS ASC,RTTEXTID ASC,RTLNGP ASC,RTSY ASC ;

Now,

to me the object reference being used is in fact either wrong or getting corrupted by the kernel thats making the request but i dont know how i can prove that, i've checked OL and cant see any corrupt / dodgy object names.

if it was a general issue with the Oracle client i would expect to see various issues reported referencing transaction tables when logic / UBEs run and have issues. We only see issues from the spec tables. That leads me to believe something is requesting object code/specs but passing a specific "rogue" entry. At this stage i cant see how its a DB client / server BUG ...

We've also set a few DB Params but none have made any difference, namely DISABLE_OOB=ON and some changes to disable peak binding. None of these have made a difference.

Its possible that we have a NW Issue that's causing some packets to drop but again, we would expect our logic and UBE servers to have this problem for all types of SQL traffic.

We've been working with Oracle for several weeks and not much to go on, as usual they want full DB trace / Client debug logs for an issue that we cannot repeat and only experience in a busy production system..We'v shared our configu and there is no suggestion of Bugs/Oracle client patches/Tools Upgrades etc as the solution..


With that in mind has anyone seen anything like this before? I'm up for any suggestions on how we can trap this.



Database Server: Oracle RAC 12.1.0.1.0, OEL v6.6
JDE Application: JDE 9.1, Tools Release 9.1.5.2
JDE Enterprise Server: OEL v6.5, Oracle Database Client 12.1.0.1.0
 
Top