I Would Now Like to Turn the Call Over to Mr. Oliver Mayer. Please Go Ahead
Total Page:16
File Type:pdf, Size:1020Kb
SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 1
SAP AMERICA
August 7, 2003 10:00 a.m. CDT
Moderator Ladies and gentlemen, thank you for standing by and welcome to BW
Know How Network conference call. At this time all participant lines are
in a listen-only mode. Later there will be an opportunity for questions and
instructions will be given at that time. As a reminder, the conference is
being recorded.
I would now like to turn the call over to Mr. Oliver Mayer. Please go
ahead.
O. Mayer Thank you, Rita. Good morning, everybody, or as the case may be if
you’re dialing in from Europe, good evening. I’d like to welcome
everybody to today’s call, which will cover the all-important BW OLAP
cache. So, again, thanks everybody for taking their time out of their busy
day to attend today’s call. I think we’ll have lots of good information. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 2
So without further ado, I will turn it over to Klaus Werner, our speaker for
today.
K. Werner Hello, everybody. Let’s get started right away. Let’s have a look at slide
number two, which is a content slide. What we will cover today is an
overview on the OLAP cache. The entire call will be on the global OLAP
cache, so first of all, an overview, what do we mean by OLAP cache and
what is it about. Next thing is the usage during query execution. Then we
have a comparison between the OLAP cache and aggregates. We’ll talk
about the global OLAP cache maintenance, monitoring, and we also give
some recommendations at the end.
So we start with the overview on the OLAP cache. The OLAP cache
buffers query (result set) data in order to improve performance of
subsequent query executions. So if somebody runs a query, a result set is
retrieved by the OLAP processor and is delivered, first of all, to the front
end and is also buffered in the so-called OLAP cache.
It comes in two flavors – the old cache that has always existed, the local
cache, which is session dependent, and in which the current query data is
stored in memory. That is available since BW 1.2B. Now we have a new SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 3 thing, which is new with BW 3.x, which is a so-called global OLAP cache. This one is session and user independent, and as I said, is new with
BW 3.x.
Since this is quite new, we have ongoing enhancements, and the information in this presentation refers to at least support package 13, support package 7, respectively, for BW 3.0B or 3.1C. The focus of this presentation is clearly the global OLAP cache, so whenever we refer to
OLAP cache, we mean the global OLAP cache, which is new with BW
3.0.
Go to the next slide – location of the global OLAP cache. The global
OLAP cache is stored, first of all, in shared memory. There is a relatively new SAP memory segment, the shared memory, whose size is defined with this parameter given here, and the number of objects that can be stored in there at the same time is the max objects parameter. This is, of course, application server dependent, because the memory itself is application server dependent. We’re talking about the location, so where is data stored in case of an overflow of this cache. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 4
There are three possibilities that, first of all, something is removed from the cache, which is the least recently used. The next possibility is to swap into a file, so move data from the memory to a file or to a cluster table, and then load it back in case it is needed again. The next possibility is file or cluster table, as we’ve just mentioned for swapping, and it is also possible to immediately use the file or the cluster table instead of the memory.
This is possible, again, in two flavors – application server dependent or independent. You can see this below in this screenshot, where it is possible to define these settings for each single query. So in the query properties you can specify, first of all, that the cache is inactive, so caching is not used at all. That will be a situation similar to what you have in the BW 2.0 system, for example. Then the setting number 1 main memory cache without swapping, so it will be stored in main memory, but not swapped to a file or across the table.
The next entry is with swapping, so store in memory and swap out to a file or cluster table in case the memory is full. The third option is to immediately write to a cluster table or a flat file for each individual application server or do the same thing across application servers, so SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 5 independent of the application server. In this case we have a central file or have a central cluster table, and write the query result set to this cluster table or file.
The next section is about usage during query execution. On the slide “query execution order of data availability check” we can see how and in which order the data availability is checked during a query execution. The first thing that would happen if you run a query or a navigational step is that it will be checked if the data is still available in the local cache. That means, for example, if you have a very large query and then you restrict to a fixed value within that query, it would use the local cache because that data is already available, and it has just to restrict from that data to give a sub-selection of that data in your new navigation step. That will be taken out of the local OLAP cache.
The next step will be to check if this is not available, is it available maybe in the global OLAP cache. Has this query already been run by some other user or in some other session with those selection criteria? Is the data available in the global OLAP cache? If yes, data is retrieved from there.
If no, in case of an InfoCube, say, as an InfoProvider, it will be checked SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 6 do I have aggregates available for this query. If no, I have to go through the InfoProvider itself on the database, in this example a basic InfoCube.
On the next slide we see the tradeoff between different ways of accessing data, getting data from up to an offline portal cache, pre-calculation and so on. The important part here is where does the OLAP cache come in, somewhere between pre-calculation and aggregates. Pre-calculation means that you calculate query results and save them somewhere in the cluster table and retrieve those pre-calculated data, which is static.
The OLAP cache, one level underneath is more flexible so it’s more an online feature, not an offline pre-calculated feature located mainly in memory or, again, in cluster tables or files. Underneath the OLAP cache there are the aggregates. So OLAP cache is somewhere between the aggregates and pre-calculation in terms of performance, as well as in terms of reusability. Offline will usually give me the best performance.
Accessing the InfoCube will usually give me the worst performance.
Somewhere in between we have located the OLAP cache with major performance benefits compared to aggregates or InfoCubes, but on the other hand, less reusability compared to aggregates or InfoCubes. That’s SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 7 the idea of this slide, so you have a tradeoff between performance and reusability.
Let’s go to the next slide. We have a comparison of the global OLAP cache and aggregates. Why is that? Because, first of all, probably everybody attending this call knows aggregates. The global OLAP cache and aggregates have some similarities and some differences. For the understanding, it’s better to really compare these two and see how do they work together or how are they somehow competing. Global OLAP cache and aggregates both store redundant data in order to improve query performance.
One stores the data on the database. One stores the data mainly in memory, but how do they work together? It’s not like they are in some kind of competition. They are defined on different levels of the architecture and complement one another. That’s important to understand.
As an example, the global OLAP cache is filled and the data is retrieved, for example, from an aggregate.
On the other hand, when you can use the global OLAP cache, it takes load off the aggregate because not everybody has to use those aggregates, SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 8 because of subsequent execution of the query will go to the OLAP cache.
So it will take load off your database. But it’s important to understand there is no competition, and it’s mentioned here explicitly again this is not a guideline to decide which one is better because there is no better or worse. It’s just that you have to have both for best performance, both in aligned coexistence.
We go the slide – “properties of global OLAP cache and aggregates”. The physical location is as we’ve seen for the OLAP cache in the main memory file or cluster table. For the aggregates it’s on the database. So in regards to server dependency we have partially application server dependent OLAP cache, and totally application server independent aggregates. The query dependency is fully query dependent for the OLAP cache and independent of a query for the aggregates.
InfoProviders: the aggregates, as you probably know, can only be defined on basic InfoCubes. For the global OLAP cache all InfoProviders can be used. In terms of performance, we have a rather stable, very good performance for the global OLAP cache, and for the aggregate it really depends on the size of the aggregates of your data model, how your SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 9 queries match the aggregates and so on, so there is more to take into consideration when talking about performance of the aggregates.
Availability of data in the OLAP cache, of course, depends on previous query executions because the OLAP cache only stores data if a query has been executed, and the frequency of invalidations. We’ll talk about invalidations in detail later on. The availability of data in the aggregates is always given. You always have the data, the currently reportable data, available in the aggregates.
Granularity of data, this is a very interesting aspect. The global OLAP cache is pretty close to the query definition. It really stores the summarized data according to the query, according to the query navigations. Also, currency conversions are already performed. For the aggregate you probably know you can filter or summarize by characteristics, navigational attributes, hierarchy levels. This is independent of the query.
Sub-queries or selections, like restricted key figures, put different selections through the OLAP processor. Both have the same behavior.
Within one query you can use more than one aggregate, as you probably SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 10 know, and you can also use more than one global cache entry. If you have several selections in your query, several query cache entries can be used.
Go to the next slide – global OLAP cache maintenance. First of all, let’s talk about the global OLAP cache relevant parameters. As we’ve already seen, there is a memory segment in the SAP basis, and there is a basis profile parameter that determines the size of this memory segment and the maximum number of objects that can be stored in that particular area. On the BW side we have customizing of the size of the local OLAP cache and of the global OLAP cache.
We can determine whether the global OLAP cache is globally active or inactive in the system. We can specify a persistence mode, which means here you can determine whether a file or a cluster table should be used in case of swapping or in case of writing queries directly to a persistent data store. And you can specify a flat file name, which is then used in case of using a flat file as persistent mode. All of those parameters can be set by a transaction RSRCACHE or RSRT.
Now let’s talk about invalidation of global OLAP cache, which is the next slide. Those entries in the OLAP cache are a snapshot of query data, when SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 11 the query was run basically. Over time things will change. For example, an InfoProvider will get new data, and then the cache must be invalidated or those entries must be invalidated that no longer refer to the current state. So invalidations will happen based on time stamps and time stamps are given for InfoProviders.
When has the last data been loaded? When was the last deletion of data as examples. The report itself, the query could be changed, so there is a query generation time that comes in. There may be pre-calculated value sets, so-called buckets. They may also change or changes of independent
InfoProviders may occur. Then there is the change run. Changes in navigational attributes, changes in hierarchies will certainly influence query results.
So also in case of a change run, you will have different query results and invalidations take place. The same is true for currencies because currency conversions are already pre-calculated in the OLAP cache. So if currency conversion rates, for example, change, then the entries based on the old currency conversion rates are no longer valid and must be invalidated. So query results are, in general, invalidated if one component of this query has changed. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 12
The next item here is about a frequently asked question. How about change of key dates? Just to clarify this: The key date is basically stored in a variable. There will be a new cache entry if this key date changes, but the old entry is not invalidated, because somebody else might still go back to the old key date in the query and report on the old data so this is not a case of invalidation. This is a case of a different variable value. That’s important to understand. There’s also the possibility of manually invalidating the cache, which basically means you delete all the cache entries.
The next slide talks about switching off the global OLAP cache. The global OLAP cache can be switched off, first of all, globally for the entire system. It can also be switched off for an InfoProvider. If you do this, it just means that this is the default for new queries. So whenever you define a new query on that particular InfoProvider, for this query the cache will not be used. It will not affect the old queries on this InfoProvider.
For old queries you would have to change the setting manually at the query definition or the query properties. For a query, you can individually set the OLAP cache on or off. For one query execution for testing, you SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 13 can do the same when you run transaction RSRT. RSRT is a transaction where you can run queries within the normal SAP GUI and then debug.
There you can switch off the usage of the cache for testing purposes.
Reasons for switching off is, first of all, performance testing, but also performance tuning. If you have a very fast query, you may not want to have the cache overhead, because there is certainly some overhead for checking if an entry is available in the cache or not, and if the query is fast anyway, why would you want to use the cache. You may want to avoid unnecessary swapping in the cache as well.
If you have a lot of queries that are, for example, fast and that unnecessarily caused swapping in the cache, you may want to switch it off for those. So queries to be switched off are first of all queries that have large result sets, for example just used for batch printing. Why would you ruin your OLAP cache and fill your memory unnecessarily with such a query, so that would certainly be a candidate for switching off.
Queries with frequent invalidations: If you have an InfoProvider to which you load data every five minutes, you would probably not want to have the cache active for those queries, and an InfoProvider with a lot of ad hoc SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 14 queries maybe. If you create new queries, just for one time usage, you wouldn’t want to have them cached either.
On this next slide we’ll talk about pre-calculation. Whenever a query is executed, the global OLAP cache is filled. So whenever you use the
OLAP processor basically, independent of whether you use it through the
Bex Analyzer, a Web application, third-party front ends, the reporting agent, or for example, RSRTRACE. Whenever you execute a query, this
OLAP cache is filled. You can make use of that fact by pre-calculating query results and having them stored automatically in the global OLAP cache.
This is, for example, possible with the reporting agent. It’s not what that reporting agent has been designed for, but you can kind of abuse the reporting agent to pre-calculate also the global OLAP cache. So you don’t have to use any specific option for this cache filling. You can, for example, use the pre-calculation of Web templates to fill this global OLAP cache.
You can run those queries after, for example, data loading or change runs are finished. You can kick off the reporting agent and pre-calculate your SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 15 queries. This could be used for very important and performance critical queries. Again, it’s not a specific setting for the global OLAP cache. It’s just a regular reporting agent that is used for also filling the OLAP cache.
That happens automatically when you specify settings in their reporting agent, and you run those. So as you maybe know, you can specify that you pre-calculate users specifically. You can choose roles or individual users for pre-calculating their query results as they would see them, depending on their authorization. Or you can also use the control query to pre-calculate your query results, your navigation steps basically.
Running the queries with user authorization or specific selections with the control query has advantages and disadvantages. A disadvantage may be that you have more cache entries, more different ones. But on the other hand, your cache entries are more specific. You have improved performance because it’s more specific smaller result sets.
On the other hand, you may have more redundancy, especially if you have similar authorizations for different users. That may mean also more maintenance effort for you. If you run a query with kind of all authorizations or general selections, then there are fewer cache entries, SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 16 less specific, worse performance, but on the other hand, also easier maintenance.
We go to slide number 18 now, the content slide. “Monitoring”, and go to
19 immediately. First of all, let’s talk about the structure. That’s important to understand in order to understand the monitor itself. As already mentioned before, the global OLAP cache is query dependent. So the first level parent of all entries is the query. Within this query you use variables. You use presentation hierarchies. That’s the next level.
You may use different presentation hierarchies, for example. That would result in different entries on this next level. Then we come to the selections, currencies. Currency conversion is already pre-calculated in the OLAP cache. The next level is the data itself. That’s the structure of the global OLAP cache. The last two – global data selections, currency and data – are summarized in one entry in the monitor of the global OLAP cache.
On the next slide we see screenshots of what the monitor transaction looks like now. I mean support package 13 or higher. You may still remember an older screenshot. It looked different before. So we have those buttons SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 17 basically on the left upper side, and if you press those buttons you see what happens. If you press the button cache parameters, you will see the local cache size, the global cache size, whether the cache is active and also the persistence mode.
If you press the main memory button, then you will see, again, the maximum cache size, which is the sizing of specified as a parameter, the current cache size, the current swap size, and what is used of the current cache of the maximum the cache could grow to. In this example we would have 250 megabytes customized, but only 414 kilobytes used, so that’s close to zero percent and that’s why it gives you zero percent here.
The overall number of cache entries, 634, is also given here. We have 634 cache entries.
The next section is interesting. The buffer poll time will just tell you how old this information is that is currently displayed. It’s by default updated once in five minutes. The next number, the 32% refer to the maximum usage of the cache, taking into account the size, as well as the number of objects. So here at the moment the limiting factor is the number of objects. We see that we have 634 current cache entries, out of 2,000 we have specified in the parameter. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 18
You cannot see the 2,000 here in this slide, but that’s what you can see in the next slide. So 32% of the cache is used. Not the cache size in this case, but the maximum number of objects. The next “buffer setting cache” gives you the information that at the moment in this system 100% of this basis memory area is currently used by the OLAP cache, so nobody else is at the moment using this segment, this SAP shared memory segment. That’s this information. There may be other components also using it and then you would see a low number here.
The next thing you see, if you press on the button, for example, application server, then you see the current cache size there. You see the current swap size, which doesn’t make any sense because it’s already swapping this is talking about. So it’s basically the current swap size itself that is displayed here in the first row, and in the second row it would say swapping of the swapping, but there is no swapping of the swapping.
So in the next support package you will no longer see this line. It doesn’t make any sense. You will see the current cache entries, which means basically the currently swapped entries on an application server level. The same information you get for cross-application servers if you press the SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 19 second to last button. Whenever you press buffer objects, underneath one of those buttons you will get a view of the current buffered objects.
In this example we pressed not this last button, it’s just for display in this slide. We pressed the button on the main memory because the others are empty anyway. There you see the query name first. You remember the structure of the global OLAP cache. The first level is the query. The next level is hierarchies and variables, and then summarized the selections, currency conversions, and the data. If you double-click on this hierarchies and variables, you can also see which hierarchies and variables are taken into account here.
On the next slide, “cache monitor basis” you see a basis view, a basis transaction that some of you may know, the ST02, which is a view on the memory that SAP allocates. This is not BW specific, but basically Web
Application Server specific.
Here we have the last line, export/import shared memory, and that’s where you get the basis view on this memory segment that is used for the global
OLAP cache. You see a hit ratio here. You get a size, and the last figure you see here in this screenshot is the number of entries that you can have SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 20 in this cache, so this is the 2,000 I have referred to in this slide before.
That’s the 2,000, out of which currently 634 entries were used and that’s why it gives you the usage of 32% in this previous slide.
On the next slide, the query properties, you see a screenshot of transaction
RSRT. You specify a query, and press the button “technical information”, and within the technical information you have a lot of data about the query
– performance relevant data and also data like generation time, internal technical names, and things like that. One section of that is the cache- relevant data. It specifies whether the query can use the cache.
It usually says yes; it rarely says no. If it says no, it tells you why it cannot use that cache. You see the query generation time and you see the last data change of the underlying InfoProvider. If you press this properties button, then you will see, first of all, the read mode. You all probably know the read mode, and on the same level where you can specify query specifically read mode, you can now specify also the cache mode. That’s exactly that little screenshot we had in our of the first slides where you can specify for one individual query whether the cache is active or not, and whether you use main memory with, without swapping, or cluster table or flat file application server dependent or independent. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 21
On the next slide we have given some exceptions, in which case the OLAP cache will or can at the moment not be used. It’s currently not used for queries containing variables with replacement path, sometimes also referred to as pre-query. It’s at the moment not yet used for most recent reporting, but that has been implemented with support package 15 so this is the latest information.
Another restriction is that large objects are not buffered in memory, so if you chose memory buffering and you have a very large query result set that exceeds about something between a fourth and a fifth of the export/ import buffer, then the result will not be written through the global OLAP cache.
On this next slide or second slide, we give some recommendations for setting the parameters. We will investigate that more and we will give you more detailed information that will be published in the SAP Service
Marketplace, but this is for now what we usually recommend. 80% recommendation: Say for 80% of the systems it should be fine, that you use 200 megabytes for the global OLAP cache, and for the export/import buffer set memory 100 megabytes. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 22
This last value is not the default setting. That’s a thing you’d have to change because the default is only four megabyte and you could run out of these four megabyte easily. So you should check in your system the usage of this segment in transaction ST02 and increase the size if it is too small.
Different aspects need to be considered when setting up those parameters, and we will elaborate on this a little more in the next slide. Bear in mind that you can decide for each query to set the cache to active or inactive, and to use the memory or use flat file or the cluster table.
So we’ll go to this next slide. There is a matrix giving you some aspects you may take into consideration when deciding for a single query or your entire system which kind of caching option to use. What is important is certainly the frequency of changes of data. Like if you, for example, load every five minutes to an InfoProvider, the caching will not be that efficient because invalidations will take place frequently. Also, certainly the amount of active users and of query navigations that are performed in your system, the amount of different queries.
Remember that the OLAP cache is dependent on queries, so it’s certainly interesting whether you have many queries or you only have a few queries to determine how efficient the cache can be. Also, the number of ad hoc SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 23 queries is important. If you create a new query, this new query will certainly not be cached at first execution because you have just created it.
If you usually do ad hoc querying, the cache is also not that good.
The size of the result set may influence your decision on whether you use the memory or not for caching those query result sets. If you have very large result sets, it may make sense to immediately write them to a file or a cluster table. Also, the load on InfoProviders on database tables is interesting. If you have a high load on the database on the InfoProvider tables, it certainly makes sense to use one of those caching options.
Low query performance is, of course, another indicator. That’s what it’s all about: increasing the performance of queries. So if you have a low query performance you would certainly want to use this global OLAP cache. User groups assigned to application servers, that’s interesting to decide whether you want to use a cache across application servers or application server specific. If you assign your user groups to different application servers, user groups doing similar kind of reporting, it certainly makes sense to have the cache application server dependent. The speed of your I/O is certainly also interesting. Cache will, of course, have SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 24
less I/O impact than retrieving the data from an InfoCube or from
aggregate.
We’ll go to the next slide. There we have some links. More information
can be found in the online documentation. There will be a documentation
enhancement soon available in SAP Service Marketplace, especially about
the OLAP cache. There will be some conferences, BI conferences in
Berlin, and Tech Ed in Las Vegas and in Basel, and also the ASUG BITI
forum in Dallas.
That’s the end of the presentation. You now have time to ask questions.
Moderator Our first question is from Radu Drasta with Colgate-Palmolive. Please go
ahead.
R. Drasta Is there a method to keep a certain report in the cache or how would you
ensure to keep a given report, and perhaps some of it’s navigation steps
over a longer period of time? I’m thinking for the use of certain executive
reports that do not refresh that often, perhaps year-to-date summaries,
things like that. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 25
K. Werner If I get your question right, it’s about keeping query results for a long
time. You can keep them kind of forever if you use the swapping options
or if you use the cluster table and the flat file. They will never be removed
unless they’re invalidated, and they will certainly be invalidated if you
load new data. So if you have a management report, a very important one,
it will usually also have some current data in there and you will load new
data maybe daily or say at least monthly. Then the cache entry will
always be invalidated. But if it’s not invalidated, it will stay in the cluster
table or flat file if you use swapping or you immediately write it there.
R. Drasta Granted, let’s say that they only change monthly. Let’s say that the cache
is filling up and this report, since it hasn’t been refreshed, so perhaps it
isn’t run that often as the other ones, may be getting at the end of queue
and the algorithm would throw it out of the cache. In that situation should
we set up the reporting agent that runs this all the time, or would there be
some other way of making sure say we run this report, don’t ever take it
out of the cache until the data changes behind it? Is there such a way of
doing it?
K. Werner No, it’s not necessary. This swapping or this removing of the cache out of
the cache will only happen if you choose this first option, main memory SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 26
without swapping. With all other options for this particular query, all
other options will guarantee that it will never be removed from the OLAP
cache. Never.
R. Drasta I see. So unless the results change, it will never be swapped The answer
is there should be a select few queries like that in the set with that
parameter going to the OLAP cache?
K. Werner Yes. You can use several queries. You can use all queries like that. It
will not be removed. Only the cache size will, of course, maybe increase
unnecessarily. But for the important queries, you should make sure that
you use one of those options, two to four, and not the first one.
Moderator Ravi Lakkaraju with Joann Stores.
R. Lakkaraju Does the OLAP cache get invalidated with a cycling of the system, too?
K. Werner With a …?
R. Lakkaraju Say you cycled the system, shut it down, and bring it up for daily
maintenance. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 27
K. Werner If you shut it down and bring it back up again, the memory cache will be
certainly removed and the swapping is removed, but the file on the cluster
table will remain available.
R. Lakkaraju So is there a way for it to read through the file and bring it back into
cache? Is there a way to do that?
K. Werner No. There is no such option at the moment.
Moderator Andreas Peters, UBS.
A. Peters I have a question concerning the filling using the reporting agent. Is there
a difference if I fill this cache using web templates or if we did it the last
test using the print agent, which does not print any lines?
K. Werner It’s the same whatever you use. Only I think for the Web templates you’ll
have some more options like doing it by authorizations or control query.
I’m not sure about this, but there is no difference whatsoever. It only
depends on what kind of query you use, what kind of navigation step you SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 28
perform there, and the only important thing is that it is performed no
matter what you do with the data afterwards.
Whether you use it in batch printing or not use it in batch printing, or put it
into a Web template, put the data somewhere or put the HTML
somewhere. Independent of that, it’s only important that the query gets
executed in the way you would like to have it executed.
A. Peters The next question is if I create a drilldown, which has a lot of lines, let’s
say 64,000 lines, which is indicated in the RSRT, the query result is
incomplete due to the large number of lines. If I now create a reporting
agent job, which creates such a large number of lines, does that mean that
the cache is not completely filled will all the lines or is it there all in there
even if it’s not all printed?
K. Werner First of all, you should not have queries with 64,000 lines. Second, the
restriction comes from Microsoft Excel, not from SAP. If you don’t use
Excel, for example, batch printing, then there is no such restriction.
A. Peters The last question would be, if I have one query which runs and which
makes drilldown into characteristic one, and have a second drilldown into SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 29
characteristic two, now the second user comes. He first drills down into
characteristic one and immediately into characteristic two. Is the cache
used or not or do I have to create a set with both characteristics drilled
down?
K. Werner Let me introduce and hand over to Stefan Miller, who is the developer in
that area, who has just joined. Let me hand over to him for the answer.
S. Miller The cache would be used. There is only one problem. If they do it at the
same time, really at the same time, then it could be that the cache data has
to be calculated by each user. But if they do it with a time difference, it
will be used. Is that okay?
A. Peters Yes.
Moderator Steve Rudolph, Monsanto.
S. Rudolph Thank you for the conference call. My question is whether there is any
correlation between OLAP cache and BW statistics. Does it require or
make use of BW statistics to make decisions on use of the OLAP cache? SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 30
K. Werner At the moment, unfortunately, there is no direct relation indicating that the
OLAP cache has been used in the BW statistics. That’s still something
that we have to develop, that we have to do. There is an indication for it if
no data records are selected from the database and transferred to the
application server. You may know those fields QDBSEL and
QDBTRANS in the table RSDDSTAT or the corresponding InfoObjects
in the BW statistics indicating how many records have been read from the
database and how many records have been transferred to the application
server.
This will always be zero if the OLAP cache was used, but it may be zero if
the OLAP cache was not used, just because there was no data available,
but that should be an exceptional case. So usually if you look for those
entries with no records selected from the database, you will have those
entries that refer to usage of an OLAP cache.
S. Rudolph Conversely, it sounds like from what you said, there is no direct
requirement to have statistics active in order to use OLAP cache. Is that
correct? SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 31
K. Werner Yes. That’s certainly correct. There is no dependency of using BW
statistics on the OLAP cache. Maybe I misunderstood your question.
They are two totally different things. So statistics are always collected
independent of what kind of settings you have with your query, but the
thing at the moment is that you cannot see directly into statistics whether
the OLAP cache was used or not. Only by this guessing from QDBSEL
and QDBTRANS is equal to zero.
S. Rudolph I understand. Thank you.
Moderator Eloy Meira with Rich’s Foods.
E. Meira I have two questions. Is there a possibility to swap the cache into the
cluster table? I also have a second question. Is the cluster table faster for
caching or swapping that cache than the file?
K. Werner Your first question maybe refers to the existing cache and memory of
somehow getting it into the cluster table. Then the answer is no. But for
swapping you can choose the persistence mode cluster table or you can
choose in the query properties to immediately write to that cluster table all
the data that that query delivers. Maybe you can put your question in
another way so I understand what your question really is, the first one. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 32
E. Meira It’s just to ensure that if we decided, for example, to use buffing or
swapping technique for attaching the cache, the OLAP memory, that the
cluster table is going to be an efficient way of doing it. I will assume that
also that’s related to my second question, that the cluster will be faster on
the file, but that was my second question.
K. Werner I understand. So you can use the cluster table for swapping. That’s for
sure. You choose the persistent cluster table and you specify for your
queries the option main memory with swapping. That’s the first thing.
Performance really depends on many factors, like how fast is the
communication between your application servers or application servers
and server you have the file on. May be cluster table may be faster or it
may be slower. I think both are possible. Unfortunately, we have not
done any detailed measurements on that, but we will do so probably by the
end of the year, but I cannot give you a reliable answer right now. But it
will be published some time this year in the SAP Service Marketplace.
Moderator Troy Mosser, Molex. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 33
T. Mosser We are using this … the data-mining tool for populating an object from a
query. Will that use the cache …?
K. Werner Could you repeat what you are using? I didn’t get that part.
T. Mosser Yes. The transaction code is RSBNWB. It’s data mining. Basically, the
data-mining tool is calling the transaction for CRM analytics.
K. Werner This is a very good question. Unfortunately, I do not have the answer
because I’m not a CRM expert. So I don’t know what kind of interface
they’re using. If they use the APO interface, the answer is no. If they use
the normal OLAP processor, the answer is yes.
T. Mosser I’m not sure because sometimes you have both the options. Sometimes
you can do it … transaction or sometimes it uses the OLAP, because you
can use encryption aggregation and calculator … and so on. In that case it
has to go through the OLAP processor.
K. Werner Yes. In that case it would be cached. In the case where it directly access
the database, it will certainly not be cached. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 34
T. Mosser If a query has a very high OLAP time like in … caching will have,
because it will install data after calculation. Is that correct?
K. Werner That’s correct.
Moderator There is a question from Robby Lakkaraju with JoAnn Stores. Go ahead.
R. Lakkaraju When we run a query from …, you go into say ST02 or ST04 and start
monitoring that query, you’ll see it a lot of different SQLs being
performed at the database level. Now each one could be, let’s say, if the
main query was on a multi-cube, then each one of the sub-queries, so to
speak, would be at the InfoProvider level. So does the cache store the
results at the main query level or does it store it at the sub-query level?
K. Werner It has different levels so, first of all, it is totally dependent on the query
and the query, again, is totally dependent on the InfoProvider because the
query is always defined exactly on one InfoProvider. It may be a
MultiProvider, but it certainly is related to exactly one InfoProvider. The
cache entry is thus also directly related to an InfoProvider via the query
because the cache entry is dependent on the query. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 35
Within the query, there are different selections maybe, like restricted key
figures for different selections to the OLAP processors to the database.
These different selections are stored as separate entries on the lower level.
If you remember that slide, we have the queries. Then we have
hierarchies and variables, and then we have the selections, currency
conversions, and so on. So always exactly related to the InfoProvider the
query is defined on, but different selections on a lower level of that
InfoProvider.
R. Lakkaraju Let’s say my query contains a non-cumulative key figure as one of the key
figures. I’m reporting for a certain week, but when I actually run that
query and I look at the SQL statements that are being generated with, let’s
say, ST05 traces or something, I will see that it’s actually going back and
calculating the non-cumulative based on its corresponding cumulative
delta values.
So even though my query contains only the non-cumulative key figure in
the results set, the sub-queries behind the scenes has actually gone ahead
and calculated the non-cumulative values based on its corresponding
deltas from the cumulatives, and then posted in the result sets. Does the SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 36
cache contain the non-cumulative calculated value or does the cache
contain the delta sets from the cumulatives?
K. Werner It contains calculated non-cumulative value.
Moderator Richard Oxley, Kenakore.
R. Oxley My question concerns the shared memory segment, and ST02 shows what
is using in that shared segment. What other functionality uses that shared
memory segment, other than the OLAP cache?
K. Werner At the moment in the BW system, I think nothing. No other component is
currently using it, but there will probably be in the future, so that’s why
we have prepared already this number, how many percent of the segment
is used by the OLAP cache. That will usually be 100% now, but in the
future it will be lower than 100%. So there will be other components
coming in.
Moderator Walter Froehlicher with IBM.
W. Froehlicher I have a question regarding free characteristics. … we have about a
couple free characteristics in there and we want to give the user, the SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 37
community, give the community the rate that it was obviously cache
independent major characteristic it drills into. Could we achieve it by
running the reporting agent by having or expanding all the characteristics
it runs, and would then the cache be used independent of which
combination of those characteristics they would select?
K. Werner Yes. The developer looks a little suspicious here, so I may wait. Just a
moment and let me just clarify it. The answer is yes.
W. Froehlicher I have a second question; it’s the second to last one actually. It’s
regarding this play … noticed it today. We actually run a pre-calculate
query, which could drilldown in a characteristic, which has to be …
attached. Now the user reperforms or re-executes the same query, the
same read on the cache actually is considered, but now if the user then
inactivates the display hierarchy, the community … database and it’s from
the cache.
K. Werner That’s correct. You remember this second level? The first level of the
global cache structure was a query. The next level was hierarchy and was
referred to presentation hierarchies or display hierarchies and variables. If SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 38
this key is wrong, the presentation hierarchy is no longer available; the
cache entry will not be used. That’s correct.
W. Froehlicher … data would be somewhere in the cache memory of it?
K. Werner No. It’s summarized up to the presentation hierarchy level basically, and
that’s why this is a key of this structure.
Moderator Lonnie Luehmann with Nike.
L. Luehmann I have two brief questions. The first one was a clarification on the
information on slide 25 about the recommendations. You mentioned 80%.
How should I apply that to determining the size of the global cache that
we would use here. We have a CIDB and three application servers in our
production environment. How do I apply that to arrive at the efficient
definition for the global cache size?
K. Werner I would start out with this values given here for every application server
basically, and then also maybe take into consideration what is told on the
next slide, where they have a special situation with maybe a lot of ad hoc
querying or whatever. Then watch the size of the cache. If the size is SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 39
much smaller than what you really need, then you can decrease those
parameters. If it is bigger, if there is a lot of swapping going on, you may
want to increase it. Start with the current recommendations given on that
slide and watch those numbers, like percentage of usage and ST02 and all
this kind of stuff.
L. Luehmann One other quick question. Is it possible to cache data from a remote cube?
K. Werner Yes, it is. There is a validity period you can specify here for the queries.
You can specify this in transaction SPRO, where you can also set
InfoProvider defaults like the read mode, you may know, and others, also
the cache mode. Additionally, for remote cube there will be a validity
period, I think in seconds, where you can specify how long they should be
valid because, of course, you can never know when the underlying data
structure changes in the remote system. So it’s just a value you have to
specify yourself, and you have to be aware of the fact that the data may be
a little outdated say, for example, by five minutes if you specify 300
seconds.
Moderator Next, Radu Drasta, Colgate-Palmolive. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 40
R. Drasta I have another question regarding the parameters of the basis level. There
is a buffer size, I understand, that’s in your recommendation. That’s the
100-megabyte?
K. Werner Correct.
R. Drasta The number of objects, is that number of queries or number of all those
different objects?
K. Werner It’s a number of all entries. Not the number of queries, but the number of
all entries of all selections, of everything what you can see as the lowest
level item in the query monitor. So you may also want to increase this
size; that’s correct. That’s one thing I forgot to mention, so you may also
want to increase from the 2,000 default to some higher level because that
may be the limiting factor.
R. Drasta There are all by application servers as I understand?
K. Werner Yes. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 41
R. Drasta The last one is the estimated size of the largest object. Again, that could
be like a result set, could be like a data package. Right?
K. Werner Yes. But at the moment this parameter is not really used for our purposes,
so you can ignore this parameter at the moment in our discussion here
about the global OLAP cache.
R. Drasta I’m sorry. I don’t understand.
K. Werner This parameter’s not used at the moment.
R. Drasta The large object size?
K. Werner Yes. Has no meaning for our OLAP cache here.
Moderator Walter Froehlicher, IBM.
W. Froehlicher Let’s assume we build a query with a structure containing various
selections, for instance, with restricted key figures and we perform
drilldowns after a filter, so filter and drilldown. What we observe now is
that every time everything is selected for the entire query. Is that correct
or is that not the case? If that is the case, then we can just do one filter and SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 42
drilldown, and store the data for all structure elements in the cache with
that.
K. Werner We have some problems understanding this question. Is what you’re
saying is if you filter?
W. Froehlicher Filter and drilldown.
K. Werner But then you say the filter is kind of ignored in the cache.
W. Froehlicher Yes. I would say from what we have observed from the measurements on
how long it takes to select it, that it doesn’t matter on which row I filter
and drilldown. It always takes the same amount of time to execute the
query, and so also to fill the cache. So I assume that in the background we
do not filter on a specific selection, but we do a drilldown for all the
members of the structure, and just display at the very end the one element
of the structure, which is filtered on which we have put … the filter.
K. Werner But what you say is about filling the cache. Filling the
cache is not really a factor in the overall query performance if you don’t
use the cache, but you read from some underlying data structure. So SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 43
filling of the cache you wouldn’t see much difference there, depending on
the results that are written to the OLAP cache. Otherwise, we would
really be in trouble and the OLAP cache would ruin our query
performance.
So filling the cache is not a performance issue here and it will only buffer
the data that is really retrieved. So if you have a filter in that query, it will
not retrieve any additional data and write it to the OLAP cache. That will
be very unusual. Did I misunderstand your question?
W. Froehlicher … happening for let’s just say if you do a navigation on a query or right
now … say filter and drilldown to. It seems that in a structure it does in
the background the selection, the retrieving of the data out of whatever for
all the elements of the structure. It’s that drilldown and not just for a
specific selection, as you would assume since you were doing a filter and
drilldown. It should be much faster to do a filter and drilldown on a
specific row of the structure than it is to do it on all elements.
K. Werner I absolutely agree, but this is not a problem of the OLAP cache. This
would be a problem of the query execution. And if that is the case, if that
is really true and if you can certainly have a look at the database statement SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 44
that is handed down to the database, if the OLAP cache is not used, then
please open an OSS message and we have a severe problem there. That
would be important to know, but that’s not a problem whatsoever of the
OLAP cache. That would be a problem of a query execution and query
performance.
W. Froehlicher In relation to the OLAP cache, if we want to … opportunity to do it, we
just have to do one of those and not for every specific filter drilled onto it.
So that’s why I asked the question there.
K. Werner I should hope no. I should hope you’re wrong about that. If you’re right,
please do open an OSS message.
Moderator We have no further questions at this time.
O. Mayer Since we have no further questions, I’d like to thank everybody for dialing
in today and taking the time out to attend the call. Of course, many thanks
go to Klaus for putting this presentation together and for Stefan for
dropping in and helping with the question and answer session. So, again,
I’d like to thank everybody for attending today’s call. Then we’ll wrap
this call up until two weeks from now. SAP AMERICA Host: Margaret Anderson August 7, 2003/10:00 a.m. CDT Page 45
Moderator This does conclude your teleconference for today. Thank you for your
participation. You may now disconnect.