WORKING WITH SUBQUERY IN THE SQL PROCEDURE Lei Zhang, Domain Solutions Corp., Cambridge, MA Danbo Yi, Abt Associates, Cambridge, MA ABSTRACT blood pressure at each visit recorded by doctors in ® variables PATID, VISID, VISDATE, MDIABP and Proc SQL is a major contribution to the SAS /BASE MSYSBP. The source SAS codes to create the three system. One of powerful features in SQL procedure data sets are listed in the appendix. is subquery, which provides great flexibility in manipulating and querying data in multiple tables simultaneously. However, subquery is the subtlest SUBQUERY IN THE WHERE CLAUSE part of the SQL procedure. Users have to understand the correct way to use subqueries in a We begin with WHERE clause because it is the variety of situations. This paper will explore how most common place to use subqueries. Consider subqueries work properly with different components following two queries: of SQL such as WHERE, HAVING, FROM, Example 1: Find patients who have records in table SELECT, SET clause. It also demonstrates when DEMO but not in table VITAL. and how to convert subqueries into JOINs solutions to improve the query performance. Proc SQL; Select patid from demog INTRODUCTION Where patid not in (select patid from vital); One of powerful features in the SAS SQL procedure Result: is its subquery, which allow a SELECT statement to be nested inside the clauses of another SELECT, or PATID inside an INSERT, UPDATE or DELETE statement, ----- even or inside another subquery. Depending on the clause that contains it, a subquery can select a 150 single value or multiple values from related tables. If more than one subquery is used in a query- expression, the innermost query is evaluated first, then the next innermost query, and so on. PROC Example 2: Find patients who didn’t have their SQL allows a subquery (contained in parentheses) blood pressures measured twice at a visit. to be used at any point in a query expression, but user must understand when and where to use Proc SQL; uncorrelated/correlated subqueries, non-scalar and scalar subqueries, or the combinations. This paper select patid from vital T1 will discuss the subquery usage with following where 2>(select count(*) sections: from vital T2 · Subquery in Where clause where T1.patid=T2.patid · Subquery in Having clause and · Subquery in From clause T1.visid=T2.visid); · Subquery in Select clause · Subquery in SET clause Result: · Subquery and JOINs The examples in this paper refer to an imaginary set PATID of clinical trial data, but the techniques used will ----- apply to other industries as well. There are three 110 SAS data sets in the example database: Data set 140 DEMO contains variables PATID (Patient ID), AGE, and SEX. Data set VITAL contains variables PATID, In the example 1, The SELECT expression in the VISID (Visit ID), VISDATE (Visit date), DIABP and parentheses is an un-correlated subquery. It is an SYSBP (patient’s diastolic/ systolic blood pressure inner query that is evaluated before the outer (main) which is measured twice at a visit). Data set query and its results are not dependent on the VITMEAN contains patient’s mean diastolic/ systolic values in the main query. It is not a scalar subquery This solution is using a double negative structure. because it will return multiple values. The original question can be put in another way: Find each patient for whom no visit exists in which In the example2, the SELECT expression in the the patient concerned has never had. The two parenthesis is a correlated subquery because the nested subqueryies produce a list of visits for whom subquery refers to values from variables T1.PATID a given patient didn’t have. The main query and T1.VISID in a table T1 of the outer query. The presents those patients for whom the result table of correlated subquery is evaluated for each row in the the subquery is empty. SQL procedure determines outer query. With correlated subqueries, PROC SQL for each patient separately whether the subquery executes the subquery and the outer query together. yields no result. The subquery is also a scalar subquery because aggregate function COUNT(*) always returns one In the WHERE clause, IN, ANY, ALL, EXISTS value with one column for each row values from the predicates are allowed to use any type of subqueries outer query at a time. (see example 1 and 3 above); However, only scalar subqueries can be used in relation operators (>, =, Multiple subqueries can be used in the SQL <, etc), BETWEEN, and LIKE predicates. One way WHERE statement. to turn a subquery to into a scalar subquery is using aggregate functions. Consider Example 4 below, Example 3: Find patients who have the maximum which two scalar subqueries are used in the number of visits. BETWEEN predicate. Proc SQL; Example 4: Find patients whose age is in the average age +/- 5 years. Select distinct patid Proc SQL; from vitmean as a Select patid Where not exists from demog (select visid from vitmean where age between Except (select Avg(age) Select visid from vitmean as b from demog)-5 Where a.patid=b.patid); and (select avg(age) Result: from demog)+5; Result: PATID ----- PATID 120 ------ 110 Below is a nested correlated subquery solution to example 3. SUBQUERY IN THE HAVING CLAUSE Proc SQL; Users can use one or more subqueries in the Having clause as long as they do not produce multiple Select distinct patid from vitmean as a values. It means that the most outer subquery in the Having clause must be scalar subquery, either where not exists uncorrelated or correlated. The following query is (select distinct visid an example, from vitmean as b where not exists Example 4: Find patients who didn’t have maximun (select visid number of visits. from vitmean as c Proc SQL; where a.patid=c.patid Select patid and From vitmean as a b.visid=c.visid)); Group by patid Having count(visid) from vital <(select where visid>'1') as a, max(cnt) (select distinct patid, from visid, visdate (select from vital count(visid) where visid<'4') as b as cnt from vitmean where(input(a.visid,2.)-1 group by patid) =input(b.visid,2.)) ); and a.patid=b.patid order by a.patid, a.visid; Result: Result: PATID ------ PATID VISID INTERVAL 100 ---------------------------- 110 100 2 7 130 100 3 7 140 120 2 8 120 3 6 The scalar subquery in this command counts the 120 4 34 maximum number of visits first, and then main 130 2 35 query find the patient whose number of visits is not equal to the maximun value calculated by the 130 3 38 subquery. For a correlated scalar subquery in the 140 2 9 Having clause, see example 6 in next section. In the example 5, two subqueries are joined together SUBQUERY IN THE FROM CLAUSE to form an in-line view in the From clause. The first one has all required patient visit data with visit ID Subqueries can be used in the SQL FROM clause to >1, and the second one has all required patient visit constitute in-line views in the SQL procedure data with visit ID <4. They are joined by PATID and because subqueries can be regarded as temporary VISID to calculate the interval between this visit and tables. You will notice the usage in the example 4, in previous visit. The result is shown above. which subquery in the From clause is actually an in- line view. In-line views are views in the FROM Example 6 below is more complicated case, in clause. They are extended temporary tables which which two subqueries are joined together by visid are constructed by UNION, INTERSECT, EXCEPT, and patid to from an in-line view. The results from and JOINs based on tables and subqueries. In-line the in-line view are then used by main (outer) query view can not correlate with its main query, but outer to obtain the list of patients who have exact same (main) query can refer any values in the tables and visits. subqueries which constitute the in-line view. The Example 6: Find patients who have exact same main advantages of using in-line view is improving visits. the SQL performance and reducing the SQL statements. Consider Example 5. proc sql; Example 5: Calculate the interval between this visit create table vital as and previous visit for each patient visit. select distinct patid, visid proc sql; from vital; select a.patid, a.visid, select T1.PATID as PATID, intck('day', b.visdate, a.visdate) T2.PATID as PATID1 as interval FROM from (select patid, visid from VITAL) (select distinct patid, AS T1 visid, visdate inner join (select patid, visid from VITAL) Result: AS T2 on T1.visid=T2.visid PATID VISID DIFF and T1.patid < T2.patid 100 1 0 Group BY T1.PATID, T2.PATID 100 2 0 Having count(*)=(Select count(*) 100 3 0 From vital as T3 110 1 0 Where 110 3 0 T3.PATID=T1.PATID) 120 1 0 and 120 2 34 count(*)= (Select Count(*) 120 3 7.5 From Vital as T4 120 4 0 130 1 -0.5 Where 130 2 -0.5 T4.PATID=T2.PATID); 130 3 -0.5 RESULT: 140 1 0 140 2 0 PATID PATID1 ----------------- 100 130 The subquery in the SQL clause is a correlated scalar subquery. It will calculate average systolic blood pressure based on VITAL table for each SUBQUERY IN THE SELECT CLAUSE patient visit from table VITMEAN. The main query then use the result to calculate the difference between recorded systolic blood pressure and SAS SQL can use a scalar subquery or even a calculated systolic blood pressure for each patient correlated scalar subquery in the SELECT clause.