Why I created a blog

Its been many years since I first created this blog. It has remained true to Essbase and related information over those years. Hopefully it has answered questions and given you insight over those years. I will continue to provide my observations and comments on the ever changing world of EPM. Don't be surprised if the scope of the blog changes and brings in other Hyperion topics.


Tuesday, April 26, 2011

Why server file loads are quicker

Most of us know that if the data file exists on the server (especially in the database directory) and we do a data load it is faster than if the file is on the client. But what do EAS and MAxL actaully do when the file is on the client machine? If you look at an EAS data load,  when you start it, you see the box:
image
When the dialog hits 100%, in the message area of EAS, you will see a statement something like:
User selected data file ‘C:\temp\data.txt’ is uploaded to server for execution at location d:\Oracle\Middleware\user_projects\epmsystem1\tmp\tempfile9706446338807355961.txt
(your path will vary depending on what version of Essbase is installed)
EAS uploads (FTPs) the file onto the Essbase server then processes it. When the process is complete, it deletes the file.  What is the implication of this. Aside from being slower that having the file on the server, it is possible that aborted loads are leaving dead files in the temp directory. I had a system where the number of files and space consumed became significant.
Talking about slower, at a client I’m at, I tested the differences in load time. I’m connected over a VPN connection and have a data file that is 65 meg. For each test, I loaded into the same empty ASO database.  I’m running the web version of EAS so it is on my local machine.  Running the data load process with a client file took about 36 minutes.
Running the same file on the server in the database directory took less than 1 minute. 35 minutes of savings which is significant when you are waiting for processes to run.
To improve my time overall since the files were on my local machine, I would zip them up, copy them to the server, unzip them and then process them. In all I saved about 20 minutes off EAS’s client time. This showed me that the upload does not do any file compression when it copies the file.

Update note:
I've talked with a few people that have read this post and need to clearify. When I talk about a server file, I mean the file should be in the application/database directory on  the server. This way EAS recongnizes it as a server file. The easy way to check if the system thinks it is a server file is to open up the load utility in EAS and select find file. Click on the server tab. IF the file pops up, it is in the right place (It should be saved as a txt file to make life easier).


While this will not affect most people as they are doing their processing on the server or in house on fast networks, it does help to understand what EAS and MaxL do when you have a client file to load.
This is just a single tidbit of the type of information you can get at Kscope 11. I’ll be presenting a couple of sessions. One on Advanced Essbase Studio tips and another one on the exception to the rules on outline design. I think both are pretty good sessions. Come see for yourself and let me know. I’m sure I’m going to get sucked into some other sessions , like a panel or two on Hyperion.
If you want to save $100 off of registration,  enter the three character code IRC   (which stands for interRel consulting). I’m not sure how long the code is good for, so don’t wait too long.

9 comments:

Cameron Lackpour said...

Glenn,

Were the aborted files the result of:
1) The EAS client getting terminated
2) A crashed EAS service

I can't think of a clean way within EAS to kill a data load save going to MaxL or another EAS session and killing the session. Am I not applying enough imagination to this?

Regards,

Cameron Lackpour

GlennS said...

Cameron,
The aborted files were the result of both things you mentioned, dataloads getting terminated and network issues. It seems every time you upload a file it uses a new name for it so aborted ones don't get deleted

Michael Sterling said...

The real question is: Did your client customer get billed for the 1 minute test or the 36 minute test? :)

And you didn't think I was paying attention.

GlennS said...

They got billed for all of it as it wasn't a test, but rather me working on site and through VPN remotely at their request. By me zipping the file and putting it the server, I ended up saving them loading time.

sam yarborough said...

If only I had the kinds of file servers Google or yahoo have then I wouldnt be having speed issues and tweaking this directory or other lol

tisco said...

Hello,

This has an issue though, that you cannot upload .csv files...

Regards and thanks for your posts!

Samantha said...

If you have a great connection, I cant tell any difference between upload times.

GlennS said...

While this might be true for a small file large files will be much slower

Anonymous said...

Can we configure the TEMP folder path on v9?