Using the Create Base View From Query Option Revision 20200810 NOTE This document is confidential and proprietary of Denodo Technologies. No part of this document may be reproduced in any form by any means without prior written authorization of Denodo Technologies. Copyright © 2021 Denodo Technologies Proprietary and Confidential Using the Create Base View From Query Option 20200810 2 of 15 CONTENTS 1 GOAL...............................................................................3 2 INTRODUCTION.................................................................4 2.1 INTERPOLATION VARIABLES...........................................................4 2.2 WHEREEXPRESSION.......................................................................5 3 BEST PRACTICES...............................................................7 3.1 PROS............................................................................................7 3.2 CONS AND SOFTWARE LIMITATIONS................................................7 4 EXAMPLES OF USE............................................................9 4.1 ANALYTICAL FUNCTION (RANK)......................................................9 4.2 ANALYTICAL FUNCTION (NTILE)....................................................12 Using the Create Base View From Query Option 20200810 3 of 15 1 GOAL Denodo allows to create base views on top of JDBC and ODBC data sources that, when queried, will execute the SQL query provided in the definition of the base view. This document explains when and how to use this option. The limitations of this functionality will also be explained. Using the Create Base View From Query Option 20200810 4 of 15 2 INTRODUCTION Developers have two options to create base views in Denodo from JDBC and ODBC sources: ● Graphically: this is the usual way of creating JDBC/ODBC views. After creating the data source, the user has to select the tables to create base views. ● SQL Sentence: by using the option ‘Create From Query’, the user specifies a SQL query to be executed when querying the base view. When writing this SQL query, consider the following: ○ This SQL query has to use the syntax of the queried database and not the syntax of Denodo. ○ The query may have interpolation variables, which allow the SQL query sent to the database to be parameterized according to the conditions of the query executed over the base view. ○ The SQL query should specify an alias to all the columns of the query that are the result of an expression that has one or more input parameters and at least one is an interpolation variable. For instance: SELECT ABS( ROUND( @value ) ) AS value_abs_round… If an alias is not defined, Denodo will automatically generate a name for the corresponding column in the created base view, which can be counter-intuitive and change from one data source to another. ○ If the SQL query invokes a PL/SQL function using an interpolation variable, you also need to assign an alias to the return value. For instance: SELECT function(@INPUTVAR) AS value FROM sales This document is focused on the ‘SQL Sentence’ option. The following sections explain pros and cons of using this option and some advice on when and how to use it. 2.1 INTERPOLATION VARIABLES Interpolation variables allow the SQL query sent to the database to be parameterized according to the conditions of the query executed on the base view. By using these variables, you do not have to provide the value of a field in an expression at design time (when creating the base view). Instead, you will provide the values of these variables at runtime, when you query the view. For instance, suppose you have a SALES table in a database and you want to create a base view with a SQL Sentence. You could create a base view SALES_VDB by writing a query like this one: SELECT Function(amount) as AVERAGE FROM SALES WHERE product_id = @product_id and then execute it with different values for the product_id field. Note that the query uses a function in the SELECT clause, this is a typical example of the use of the ‘Create Base View From Query’ option when using a database-proprietary function. This way, if we execute the following query on the base view in Denodo: Using the Create Base View From Query Option 20200810 5 of 15 SELECT * FROM SALES_VDB WHERE product_id = 2 The Server will substitute the variable @product_id with the value from the WHERE clause of the query and execute: SELECT Function(amount) AS average FROM SALES WHERE product_id = 2 Each interpolation variable must be related with an attribute belonging to the generated base view, so that the variable can obtain its value at run time. If the SQL query used to access the data source returns any field with the same name as the variable, then the variable will be associated to the corresponding base view attribute. If the SQL query does not return any field with the same name as the variable, Denodo will add a new attribute to the base view, with the same name. 2.2 WHEREEXPRESSION Denodo provides a predefined interpolation variable called WHEREEXPRESSION that simplifies the process of creating a base view from an SQL query. At runtime, the Server will replace WHEREEXPRESSION with the condition sent to the base view. If we create a base view named SALES_VDB with the following SQL query: SELECT sales_id, product_id, customer_id, profit_margin as margin, StoredProcedure(amount), sales_date FROM sales WHERE @WHEREEXPRESSION we will be able to execute queries like the following on the base view in Denodo: SELECT * FROM SALES_VDB WHERE margin > 0 AND getyear(sales_date) > 2010 Note that the user has to specify the actual name of the fields for the aliases used in the SELECT clause of the query. For instance, the SQL query defines the margin alias for profit_margin. Therefore, the user must specify the profit_margin value for the attribute named margin in the VQL query. The Server will substitute the variable WHEREEXPRESSION with the condition of the WHERE clause of the query: SELECT sales_id, product_id, customer_id, profit_margin as margin, StoredProcedure(amount), sales_date FROM sales WHERE profit_margin > 0 AND extract (year from sales_date) > 2010 Note that the alias name is translated to its original name when the query is finally executed. Sometimes, you may need to filter the results of the SQL query using fields that are not projected and, therefore, will not appear in the base view schema. Denodo provides a way to do this. A typical example happens when the SQL query used to Using the Create Base View From Query Option 20200810 6 of 15 create the base view uses GROUP BY. For instance, suppose the SALES table has a field store_country that is not included in the schema of a new virtual view SALES_VDB2 defined as: SELECT product_id, customer_id, sum(profit_margin) as total_margin FROM sales WHERE @WHEREEXPRESSION GROUP BY product_id, customer_id but you want to filter the results of the SALES table by store_country, you can execute queries like this over the Denodo base view: SELECT * FROM SALES_VDB2 WHERE store_country = ‘US’ and they will be translated to queries like this executed on the data source: SELECT product_id, customer_id, sum(profit_margin) as total_margin FROM sales WHERE store_country = ‘US’ GROUP BY product_id, customer_id In order to make this work, you have to specify STORE_COUNTRY in the option ‘List of field names not included in the above list that can appear in the delegated conditions’ when you create a base view with WHEREEXPRESSION. To avoid forcing the queries to always have the WHERE clause, Denodo provides the interpolation function ExecuteIfIsNotNull. For more information about this interpolation function and the WHEREEXPRESSION variable, you can check the section Using the WHEREEXPRESSION Variable of the Virtual DataPort Administration Guide. Please note that the interpolation variable WHEREEXPRESSION cannot be used with ODBC data sources. Only JDBC data sources are allowed. Using the Create Base View From Query Option 20200810 7 of 15 3 BEST PRACTICES This section will explain the main pros and cons of using the ‘Create from query’ option and some recommendations about when and how to use it. 3.1 PROS You should use this option when: 1. You have complex queries already built. Development process can be faster if you want to import the results of a complex query you have already written (for instance, because you use it for another application, as some BI tool for reporting). You will only have to copy & paste it instead of replicating the hierarchy of views in Denodo. 2. You have to use database-proprietary functions or constructions. ○ If the reason to create a view with a SQL Sentence is because you want to use a function not included in Denodo, and you are going to use it in several views, it is better to create a delegable custom function instead (when possible). Read the section Developing Custom Functions of the Developer Guide to learn how to develop custom functions. 3. You have an SQL query that has been tuned to perform better than the query that Denodo will execute. For instance, you may want to provide a certain optimization hint to the database using its proprietary syntax. 4. You have to execute a stored procedure of the database or a query that uses a stored procedure, for databases where this is not graphically supported (currently, only Oracle, SQL Server and DB2 are supported). Keep also in mind that, when possible you should enable the option ‘Delegate SQL sentence as subquery’(enabled by default starting from Denodo 6.0), located in Options > Search methods > wrapper source configuration in VDP Admin Tool, for the base views created using this option in order to let Denodo delegate more queries to the data source (as shown in Figure 4 and Figure 5). One limitation on this option is explained in the following section. 3.2 CONS AND SOFTWARE LIMITATIONS 1. Using a SQL sentence may be harder to maintain and reuse: ○ For instance, suppose that you have many views that use the same subquery in their definitions; if the subquery changes, you will need to manually modify it in every view.