Why I created a blog

Its been many years since I first created this blog. It has remained true to Essbase and related information over those years. Hopefully it has answered questions and given you insight over those years. I will continue to provide my observations and comments on the ever changing world of EPM. Don't be surprised if the scope of the blog changes and brings in other Hyperion topics.


Friday, August 28, 2009

New Essbase book Review - Oracle Essbase 9 Implementation Guide

Recently, I was asked by Packit Publishing to review a new book on Essbase by Joe Gomez and Sarma Anantapantula titled Oracle Essbase 9 Implementation Guide. I jumped at the chance. There are so few resources available out there for someone new to Essbase, it’s a shame. I understand it’s a small audience, but since the Oracle acquisition it is growing. I would like to thank the authors for spending their time, energy, sweat and tears to create the book. I know it is a difficult task, for I have a hard enough time just updating my blog from time to time. So my hat is off to them for their undertaking.

Before my review, I feel it only proper that I give a couple of disclaimers before I review the book.
1. I am not the intended audience for it as they say it is for the IT professional who wants to start working with Essbase. I’ve been doing the way too long, about 14 years now (I think)
2. The company I work for puts out a competing book. Look smarter than you are with Essbase.(I am not the author nor an editor or reviewer of it).
3. I realize this is the first printing of the book and a tried to overlook errors like typos, obvious misstatements and wrong graphics for the text. I tried to concentrate on the subject matter.

With that said, I came into the assignment with an unbiased and open mind. I attempted to read the book from a new IT professional’s point of view while using my knowledge to insure the material was accurate.

I was excited when I got the book last week and using time I don’t have, began to go through it. I have to say, I really wanted to like the book as another source of information would be invaluable to the Essbase community. I am sorry to say I feel this book falls very short of being a good guide or reference. There were some good points, but the problems out weighted those few glimmers of insight. There are a numbers of troubles I had with the book.

First the good. As bright notes, the book talks about Cube Preview in EAS. It is a useful tool for administrators that no one ever talks about. In addition, they spend time talking about report scripts and even go through some of the syntax. Everyone seems to ignore this topic thinking report scripts are dead. I thank the authors for reminding us that they can still be useful. Even though the classic add-in is dying, they spend a bit of time on Query designer. This feature can be very useful and they mention some of the high points of it. I also enjoyed the introduction to data warehousing. It was interesting, although there was not really applicable to the subject of the book.

What did I not like about the book, I’m afraid more than I liked. I won’t go through everything, but give you a number or items I had difficulty with.

First, the book was hard to follow. While in the middle of a subject, the authors would veer off to talk about something related, but minor. In other cases it appeared pages might be missing or thought processes were incomplete. I would get into a subject and it would just end.. As an example of being hard to follow, when talking about dimension building, they went into a detailed discussion of MaxL. With Data loads, they did it again and then for Calc scripts veered off to Esscmd, this while having a whole chapter on automation and not really adding value to the topic. Another item that made it hard to follow was extreme detail would be presented on what buttons and options meant of various screens, but important functionality would be glossed over. For example, they described the load screen extremely well, but gloss over, the two different types of joins, selection and rejection criteria, adding text, etc, giving no examples at all. Finally, the examples were not cohesive. I never really got the overall understanding of what the database they were building looked like. It would morph into different dimensionality without explanation.

Second, there was not a cohesive flow to the examples. It would have been nice to have examples that built on each other. There was talk about the dimensions of the outline, and one example of how to build it, but I would have liked to build on prior steps. Perhaps first create the dimensions manually and build the dimension members manually, then exercises to add the members of other dimensions through load rules. Once flaw here (and with the Look Smarter book line) is there should be a source for sample files used to do dimension builds, data loads, and results from calc scripts that would allow the reader to easily follow the examples.

Third, unknown or inaccurate terminology was used. For instance, I’ve never heard of a parent dimension. Does this imply that there are child dimensions? Throughout the book the terminology was inconsistent. Dimensions were called all sorts of things. I know this next one is trifling, but in the Calculations section, they call the set commands functions. They are not functions but commands. There is a difference and if you are writing you need to be accurate with your information.

Fourth, this is a book that is supposed to be based on System 9. In the install section you get the most basic of installs, Essbase, EAS and the Excel Add-in. No mention of Provider servers, external authentication, FR etc. They talk about Esscmd (a lot). As a dying interface, why spend the time on it. I have to admit the authors do warn you in the automation chapter that Esscmd is being phased out. Get new users using MaxL, since it is the direction of the future. Yes you can mention that Esscmd exists, but don’t waste the readers time teaching it. I have the same comment about the classic Excel Add-in. While I love the add-in, new readers should be learning Smartview. I guess that would have required the author to discuss how to install and configure APS which would have made that section longer and more difficult.

Finally, I could live with all of the above and still be relatively happy, but there are too many inaccuracies in the book that would either confuse or mislead readers. In some cases the book would give completely wrong information. Here are a very few of the many examples I found.

Did you know that Smartview costs extra? I didn’t but according to the authors it does.

When talking about attribute dimensions, they use color as an example and put it on their product dimension. Their product dimension level zero members are Car models (like Sedan). This makes the reader believe you could have different colors (attribute dimension members) for the same model. No where do they tell you that a base member can only have one Attribute from an attribute dimension associated with it. Attributes on dense dimensions? I don’t know they never tell you if you can, but allude that it’s possible.

A second example is their discussion of Two Pass. It is stated that Two pass is only allowed on the Accounts dimension and only on Dynamic calc or Dynamic calc and store members. While only on the accounts dimension was applicable on version 3 of the software, it has not been the case ever since Dynamic calc came into effect. “Only on Dynamic calc (and store)”, If this were the case, what is there a command called Calc TwoPass. Further in the section on Two Pass, they give an example and lead the reader to believe that if you perform aggregations on the database in different orders of dimensions, your results will be different. The example shows data that is added up to parents. The last time I checked addition and subtraction were commutative. I surely hope that adding up my database in different orders will not affect the results. Had they used the typical example of a ratio supplying different results if you add the sums or sum the adds, I could understand, but this example makes me scratch my head.

A third example of misinformation is in the section on Calc scripts. They give you the following calc script in an example
Fix(@IDescendants(“Calendar Periods”))
“Gross Sales” = “Sales” – “Discounts”;
Endifix
While this calculation is not incorrect, It is no different than just running the calculation on the entire database. In the discussion on Fix, it is stated that items left out of a fix statement exclude them, in truth, dimensions left out of a fix statement are all included at all levels. Like the above, a lot of examples are not well thought out. For Sumrange, the use @Descendants(Products) Based on the outline example they give, you would be summing together multiple levels, parents and children. I doubt that would give the answer one would want. I thought it interesting that they showed examples of calculations, but never showed the starting point or the result. The book claims that after reading the chapter on calculations, you would not need to take a calculation course. I believe the opposite is the truth. After reading their explanations I would need a course more than ever. If I knew nothing about calc scripts, I would have come out of that chapter more confused than when I went in it.

I don’t remember a section on security and I was not impressed with the section on optimization. As you can tell by now, I was rather disappointed with the book. As I said earlier, I applaud the authors’ intentions but I think this book falls much too short to be useful.

Thursday, August 13, 2009

Upcoming Webinar

On Tuesday August 18th (2009) I'll be giving the weekly interRel webinar. It will be on little known features of Essbase. IT is basically the same presentation as one I gave at Kaleidoscope in June. If you missed it then, try to attend this week. You can register for it at www.interrel.com
I was scheduled to repeat the webcast on Thurs Aug 20th, but I'll be unavailable, so my co-worker and friend Cameron Lackpour will be giving it then. Heck sign up for both and see who does it better. At any rate, I think it a worthwhile topic and everyone I know that has seen it has picked up a few tidbits (or more)

Monday, August 10, 2009

4th addendum to Data Export

Well, I guess the team at Oracle was listening to someone. I just looked at the release notes for EPM Fusion Edition 11.1.1.3 and there plain as day is the bug fix

Data Extraction. While exporting data to a relational database, DATAEXPORT does not create thedelimiters if there are one or more missing values in the last column. [8507606]

Another problem I was getting at a client was when I would remote onto a server using the consloe option, when I logged of the Essbase application would hang. That apparently has been fixed as well

Agent. When Essbase is running as a Windows service and the domain user logs off the machine,
the Essbase applications hang, which in turn causes Essbase Server to hang. [8279377, 8464004]


While I worked around both of these problems, Its nice to know I won't have to in the future

Monday, July 13, 2009

3rd addendum to DataExport

My last blog post talks about dropping columns when exporting to relational using the dataexport command.
I found the answer looking through the knowledgebase. Turns out you have to set
DEXPSQLROWSIZE 1 to get it to work. I don't know if I like tihs answer as it requires you to basically turn off bulk insersion, but at least it works and I can continue with my development withput having to asdd jexport to the mix.

Friday, July 10, 2009

2nd Addendum to Data Export

OK, I'm starting to see why people don't like Data Export. I'm trying to export a particular set of data to a relational table and for some reason, one of the dimensions is not showing up in the table. If I change the export to write to a file, the column is there. It is interesting that the order of the row members is different in the flat file than in the relational export. What appears to be happening is it is taking one of my two dense dimensions which shows up as the last row member before my data vailes in my flat file and is moving(or perhaps overlaying) what is my second row member in the flat file. I know that is clear as mud so to show you an example a row my flat file export looks like:

Actual,r1,aaa1,bbb1,ccc1,ddd1,eee1,Tier1,123456.789

In the relational it looks like:
Actual,Tier1,aaa1,bbb1,ccc1,ddd1,eee1,blank,123456.789

so it is shifting or overwriting the R1 with Tier1 and where tier1 should be is blank.

If anyone has figured a workaround for this other than to export to a file and load it into relational, let me know. In the meantime I'm opening a SR with Oracle and se eif they have a fix

Tuesday, July 7, 2009

Addendum to Data Export post

As an addendum to my post on Data export, John Goodwin reminded me of a work around I had to do. There is a problem when you are exporting to a relational table and your columns dimension does not have values in the trailing members (for example you have Jan-Dec in columns and you only have data in Jan –Mar) the data export will fail because the record columns don’t match the table columns. . In order to get around it, I set :
DataExportDynamicCalc On
And
DataExportLevel ALL

Then in my fix statement, I had to make sure that the last column of the load would always have data. Testing it by sending it to a flat file confirmed that if the last column had data, the intermediate columns would have something as well. In Sample Basic if Years is your column dimension, in your fix statement you could fix on Jan:Dec and Year. Since Year is a dynamic calc member if any month has data so will Year. Note, this could be be a problem if you have time balance accounts without skip missing turned on). Because you need to set all levels, you also have to make sure you fix on the level zero members of your other dimensions as well. I set the other dimensions to @relative (dimension name, level zero). Since the last column had data it would load to the end.

In talking with others, they have had problems with Data export, So far, I’ve not found anything I could not work around.

Monday, July 6, 2009

Dataexport is great

I’ve had the need to work with the newish dataExport command (I did it in 11.1.1.2) recently and thought I would share some things I’ve found with it and with Sql interface in load rules. I think you will find my musings interesting.

First, I was using data export to export date to a flat file to act as an audit report for some users. It worked like a charm. Some of the things I found are if you specify to export level zero, no matter what you put in your fix statement it will only look at level zero blocks. Using DataExportDynamicCalc on allowed me to export dynamically calculated members as well. For my 20K rows it did not seem to slow the export down, ut I don’t know the impact on a big data set. I could also specify DataExportDimHeader ON to get column names. Using DataExportColHeader "Dense dimension name" I could specify the dense dimension I wanted as the column. It would have been nice if I could put a single member of a sparse dimension there, but I understand why I can't.

Next I needed to back up some of the static information from my cube. Static in that it is user input for my testing and I didn’t have an easy source to reload from. I set up a fix statement and used the binfile option (DATAEXPORT "Binfile" "fileName"). It created a file that I could reload to my db. I can see the usefulness of this on a production database where you need to save off something like budget, clear the database and reload it with source actual and the budget. It’s much easier than the old export and reload and much quicker. In addition, you can be selective of what you export instead of exporting everything.

Finally, I needed to load data populate a Sql database with data from Essbase, modify it and load it back. Yes there are some things that Sql can do better than Essbase. In this case, It was to take two disassociated sets of data and merge them together. It needed to join them on the one common dimension member and basically turn the two 3k row tables into about 1.5 million records that get loaded back into Essbase. I set up the ODBC driver with no problem and exported the data into flat files to see their structure. I then created tables that matched the structures. I will say that there is where I had minor difficulty. If the columns don’t match exactly, the export fails with little information (Just the typical messages in the log that tell you “you screwed up”). I played around with the table figuring out that I miscounted the columns and fixed it and it worked fine. I defined the amount columns as float and found that for #missing values Essbase stuck -2e-15 in the columns that were once #missing in Essbase. A quick stored procedure and I converted them to null.

Oh but wait, how could I run the Stored procedure. I could run an OSQL but the instance I was working on the tools are not working right. I could get into Sql Server, but could not run OSQL or Bulk insert. So thinking swiftly, I thought of load rules. A load rule is just supposed to take Sql commands, so how could I get it to run a stored procedure. I know I can put statements like a union in a Sql statement, So I tried something like:

Getdate()
Exec myStoredprocedure.

I clicked ok/retrieve, entered my id and password, and lo and behold, I got the system date back into my load rule. I checked my tables and the bad characters were converted to nulls. Wow it worked. Who would have thought? I figured I could use the load date to update the alias of a member to set the last time the calculation was run. Another suggest I had was to use rejection criteria to reject the row and load nothing. I used this technique to run another stored procedure that truncated and populated a table from the tables I loaded, so my next step was to create a load rule and bring the data back in. Everything was done without having to resort to anything but MaxL statements.

I’ve since added a custom defined function that will run the SQL statement directly from the calc script. I got this from Touifc Walkim the development manager for Smartview, a very nice guy and CDF guru. Some clients don’t like the idea of CDFs so I have my origina method available when necessary. IF this works, you can get the CDF here. Note, I had to remove the link to the CDF as after 15 years blogger has decided it is is some sort of spam or virus instead of something to help other developers.



Adding DATAEXPORTENABLEBATCHINSERT TRUE to my config file made the process run faster as it allows the dataexport to use a batch insert method(when the ODBC driver allows it).

As I use Dataexport more, I’ll comment on my findings, but I have to say I’m impressed so far. I have asked for enhancements to the command, and they have been added to the enhancement list. I’m interested to see if or when they are implemented. I was very happy when I was told they were added to the list since it appears that Oracle is open to hear what people recommend. Some of the things I recommended were:
Add an append option to the file export so you could have multiple fix statements write to the same file
Add a debug option to the SQL export to see the SQL statement generated to make it easier to debug when you have problems
Allow aliases to be exported instead of file names.

Let’s see if these get implemented.