Large Grid Imports

I have a client that is upgrading from XE to 9.0. Their standard practice on the fat client in XE is to create very large journal entries by copying from excel and pasting into the grid. Over 3000 records is fairly common and sometimes they can be much larger...well over 10000 records. The are used to and accept the time it can take to do this process.

Obviously this is causing some headaches on the web in 9.0. We've had to tweak various settings and timeouts in order to make everything work in an acceptable manner, but I still continue to and have since day one explained my position that large copies or imports such as this should not be done in an interactive app and can have extremely negative side effects on the web instances - out of memory, crash, slow response, etc.

So, I come to you all asking how are you handling this type of function - what's the best practice? They are not happy with my answer that it should be a batch function when it's that large. They were doing this via grid import, but found that the copy/paste did not hold them to the limit set in the ini...but really, these two functions are basically doing the same thing.

And so I don't get chastised:
EOne 9.0 Update 1 plus a gazillion ESUs, TR 8.98.3.3, AIX6.1, Oracle 11.2.0.1, OAS 10.1.3.5

habarric,

In 'Many' cases, you can 'Fat-Enable' the application, and load your spreadsheets through cut/paste. If the application is not a PowerForm and does 'Not' have Web-Only logic - it might be an opportunity to go 'old-school'.

I don't recall the row-limit for most grids.

F9860.ANSIF - Change from "W" to <blank>

Let me/us know if this will work for you, if you need a different solution or get the issue resolved.

(db)

I had thought about that, and this particular grid would probably work(P0911-Add). Although, I'm not real comfortable opening that can of worms or presenting that to them as a "best practice". Also, I'm not real sure they would be happy having to flip over to a fat client for a certain portion of their job function.

It is a good thought and I appreciate the feedback.

habarric,

We are in a similar situation, having upgraded within the past year from Xe to 9.0. Our users have been breaking entries up into smaller chunks than they would otherwise have done in Xe (and they are complaining bitterly about it). However, even in Xe it was not a very efficient process.

We are now looking to have them use a macro in excel to create a text file, use R47002C to copy from the text file into the F0902Z1 and then use R09110Z to take it from there to the GL. Sounds like a lot of steps, but it still should be faster than what they're doing now (or what they did in Xe).


Oracle published Doc ID 882403.1 using the work order interface table as an example, but you should be able to extrapolate to GL processing.

Thanks a bunch for the info...we are presenting them with similar options(and they're complaining bitterly as well!).

[ QUOTE ]
I have a client that is upgrading from XE to 9.0. Their standard practice on the fat client in XE is to create very large journal entries by copying from excel and pasting into the grid. Over 3000 records is fairly common and sometimes they can be much larger...well over 10000 records. The are used to and accept the time it can take to do this process.

Obviously this is causing some headaches on the web in 9.0. We've had to tweak various settings and timeouts in order to make everything work in an acceptable manner, but I still continue to and have since day one explained my position that large copies or imports such as this should not be done in an interactive app and can have extremely negative side effects on the web instances - out of memory, crash, slow response, etc.

So, I come to you all asking how are you handling this type of function - what's the best practice? They are not happy with my answer that it should be a batch function when it's that large. They were doing this via grid import, but found that the copy/paste did not hold them to the limit set in the ini...but really, these two functions are basically doing the same thing.

And so I don't get chastised:
EOne 9.0 Update 1 plus a gazillion ESUs, TR 8.98.3.3, AIX6.1, Oracle 11.2.0.1, OAS 10.1.3.5

[/ QUOTE ]


Pasting to grid is improved in 9.0. Check out the "Import from Clipboard" function.

http://blog.karamazovgroup.com/2010/05/enterpriseone-tools-release-8983-notes.html

Brother,

In 9.0 - what might be the limitation of Paste-to-grid? Is there a limit to the number of rows (or a performance issue, that needs to be considered)?

Thoughts?

(db)

[ QUOTE ]
Brother,

In 9.0 - what might be the limitation of Paste-to-grid? Is there a limit to the number of rows (or a performance issue, that needs to be considered)?

Thoughts?

(db)

[/ QUOTE ]

When we were in Denver discussing this new functionality I recall being told "be careful with large numbers of records" and we talked about how pasting into a grid is probably not the best way to place a large number of records into E1.

Seriously, we have so many ways now to move data around, isn't pasting from Excel a bit Neanderthal?

Brother/Jeff,

What are the Non-Neandrathal ways to move data from a Spreadsheet (where they 'fixed' all the numbers) to E1?

Habarric,

Would your users be open to a UBE that grabs the ginormous CSV and loads it into a Temp Application (where they can Review it in the grid, Edit - then Post)? Ideally, that 'might' be the best solution (we'll see if there is another non-Neanderthal answer).
- save the spreadsheet as a CSV to an IT Defined folder
- Run the UBE, passing in the CVS's File Name
- After UBE completes, User Reviews/Edits the uploaded values
- At this point, there still hasn't been any impact to "Live Data"
- When users determines the data is good, they "Post" to "Live Data"

Not a terribly difficult change from what they are doing, now? Maybe a little better?

An easy to install solution, can be built for the right price....

(db)

[ QUOTE ]
Brother/Jeff,

What are the Non-Neandrathal ways to move data from a Spreadsheet (where they 'fixed' all the numbers) to E1?

[/ QUOTE ]

Why a UBE that grabs the ginormous CSV and loads it into a Temp Application (where they can Review it in the grid, Edit - then Post) of course.

habarric,

Sorry for the late reply.

We have a 500 line limit on pasting journals into the jde web GUI (E8.11sp1). For larger journals we import them through the 'Z' table much as Dan describes:

[ QUOTE ]

Would your users be open to a UBE that grabs the ginormous CSV and loads it into a Temp Application (where they can Review it in the grid, Edit - then Post)? Ideally, that 'might' be the best solution (we'll see if there is another non-Neanderthal answer).
- save the spreadsheet as a CSV to an IT Defined folder
- Run the UBE, passing in the CVS's File Name
- After UBE completes, User Reviews/Edits the uploaded values
- At this point, there still hasn't been any impact to "Live Data"
- When users determines the data is good, they "Post" to "Live Data"



[/ QUOTE ]

Copy and paste is their preferred method for now. Obviously, we think it's a bit neanderthal as well (hence why I came here to ask all of you neanderthal's your thoughts!).

So far, they haven't been real responsive to anything that involves a step of running a ube that loads either a temp app or the Z files. That may be changing soon.

Also, we are looking at various other options such as process improvements around why they are having to create these manual entries in the first place, but that's over the head of this lowly CNC consultant.

Thanks for all the feedback...the real goal was to show them that this is a problem that everyone is dealing with and that has been achieved.