Most of us know that if the data file exists on the server (especially in the database directory) and we do a data load it is faster than if the file is on the client. But what do EAS and MAxL actaully do when the file is on the client machine? If you look at an EAS data load, when you start it, you see the box:
When the dialog hits 100%, in the message area of EAS, you will see a statement something like:
User selected data file ‘C:\temp\data.txt’ is uploaded to server for execution at location d:\Oracle\Middleware\user_projects\epmsystem1\tmp\tempfile9706446338807355961.txt
(your path will vary depending on what version of Essbase is installed)
EAS uploads (FTPs) the file onto the Essbase server then processes it. When the process is complete, it deletes the file. What is the implication of this. Aside from being slower that having the file on the server, it is possible that aborted loads are leaving dead files in the temp directory. I had a system where the number of files and space consumed became significant.
Talking about slower, at a client I’m at, I tested the differences in load time. I’m connected over a VPN connection and have a data file that is 65 meg. For each test, I loaded into the same empty ASO database. I’m running the web version of EAS so it is on my local machine. Running the data load process with a client file took about 36 minutes.
Running the same file on the server in the database directory took less than 1 minute. 35 minutes of savings which is significant when you are waiting for processes to run.
To improve my time overall since the files were on my local machine, I would zip them up, copy them to the server, unzip them and then process them. In all I saved about 20 minutes off EAS’s client time. This showed me that the upload does not do any file compression when it copies the file.
Update note:
I've talked with a few people that have read this post and need to clearify. When I talk about a server file, I mean the file should be in the application/database directory on the server. This way EAS recongnizes it as a server file. The easy way to check if the system thinks it is a server file is to open up the load utility in EAS and select find file. Click on the server tab. IF the file pops up, it is in the right place (It should be saved as a txt file to make life easier).
While this will not affect most people as they are doing their processing on the server or in house on fast networks, it does help to understand what EAS and MaxL do when you have a client file to load.
This is just a single tidbit of the type of information you can get at Kscope 11. I’ll be presenting a couple of sessions. One on Advanced Essbase Studio tips and another one on the exception to the rules on outline design. I think both are pretty good sessions. Come see for yourself and let me know. I’m sure I’m going to get sucked into some other sessions , like a panel or two on Hyperion.
If you want to save $100 off of registration, enter the three character code IRC (which stands for interRel consulting). I’m not sure how long the code is good for, so don’t wait too long.
ODI in the hybrid database world – Amazon Redshift – AWS CLI
-
Written on June 5, 2023 by Rodrigo Radtke de Souza Hi all, probably this is
the last post of this series on how to load data from on-premises databases
to ...
1 year ago
9 comments:
Glenn,
Were the aborted files the result of:
1) The EAS client getting terminated
2) A crashed EAS service
I can't think of a clean way within EAS to kill a data load save going to MaxL or another EAS session and killing the session. Am I not applying enough imagination to this?
Regards,
Cameron Lackpour
Cameron,
The aborted files were the result of both things you mentioned, dataloads getting terminated and network issues. It seems every time you upload a file it uses a new name for it so aborted ones don't get deleted
The real question is: Did your client customer get billed for the 1 minute test or the 36 minute test? :)
And you didn't think I was paying attention.
They got billed for all of it as it wasn't a test, but rather me working on site and through VPN remotely at their request. By me zipping the file and putting it the server, I ended up saving them loading time.
If only I had the kinds of file servers Google or yahoo have then I wouldnt be having speed issues and tweaking this directory or other lol
Hello,
This has an issue though, that you cannot upload .csv files...
Regards and thanks for your posts!
If you have a great connection, I cant tell any difference between upload times.
While this might be true for a small file large files will be much slower
Can we configure the TEMP folder path on v9?
Post a Comment