Most of the time in SQL, you can simply join tables or views to one another to get the result you want. Often you add inline views and scalar subqueries to the mix, and you can soon create relatively complex solutions to many problems. With analytic functions, you really start to rock ‘n’ roll and can solve almost anything.
But it can happen from time to time that you have, for instance, a scalar subquery and wish that it could return multiple columns instead of just a single column. You can make workarounds with object types or string concatenation, but it’s never really elegant nor efficient.
Also from time to time, you would really like, for example, a predicate inside the inline view to reference a value from a table outside the inline view, which is normally not possible. Often the workaround is to select the column you would like a predicate on in the inline view select list and put the predicate in the join on clause instead. This is often good enough, and the optimizer can often do predicate pushing to automatically do what you actually wanted – but it is not always able to do this, in which case you end up with an inefficient query.
For both those problems, it has been possible since version 12.1 to solve them by correlating the inline view with
apply, enabling you in essence to do your own predicate pushing.
Brewery products and sales
In the application schema of the Good Beer Trading Co, I have a couple of views (shown in Figure 1-1) I can use to illustrate inline view correlation.
Figure 1-1 Two views used in this article to illustrate lateral inline views
It could just as easily have been tables that I used to demonstrate these techniques, so for this article, just think of them as such. The internals of the views will be more relevant in later articles.
brewery_products shows which beers the Good Beer Trading Co buys from which breweries, while view
yearly_sales shows how many bottles of each beer are sold per year. Joining the two together in Listing 1 on
product_id, I can see the yearly sales of those beers that are bought from Balthazar Brauerei.
SQL> select 2 bp.brewery_name 3 , bp.product_id as p_id 4 , bp.product_name 5 , ys.yr 6 , ys.yr_qty 7 from brewery_products bp 8 join yearly_sales ys 9 on ys.product_id = bp.product_id 10 where bp.brewery_id = 518 11 order by bp.product_id, ys.yr;
Listing 1 The yearly sales of the three beers from Balthazar Brauerei
This data of 3 years of sales of three beers will be the basis for the examples of this blog:
BREWERY_NAME P_ID PRODUCT_NAME YR YR_QTY Balthazar Brauerei 5310 Monks and Nuns 2016 478 Balthazar Brauerei 5310 Monks and Nuns 2017 582 Balthazar Brauerei 5310 Monks and Nuns 2018 425 Balthazar Brauerei 5430 Hercule Trippel 2016 261 Balthazar Brauerei 5430 Hercule Trippel 2017 344 Balthazar Brauerei 5430 Hercule Trippel 2018 451 Balthazar Brauerei 6520 Der Helle Kumpel 2016 415 Balthazar Brauerei 6520 Der Helle Kumpel 2017 458 Balthazar Brauerei 6520 Der Helle Kumpel 2018 357
At first I’ll use this to show a typical problem.
Scalar subqueries and multiple columns
The task at hand is to show for each of the three beers of Balthazar Brauerei which year the most bottles of that particular beer are sold and how many bottles that were. I can do this with two scalar subqueries in Listing 2.
SQL> select 2 bp.brewery_name 3 , bp.product_id as p_id 4 , bp.product_name 5 , ( 6 select ys.yr 7 from yearly_sales ys 8 where ys.product_id = bp.product_id 9 order by ys.yr_qty desc 10 fetch first row only 11 ) as yr 12 , ( 13 select ys.yr_qty 14 from yearly_sales ys 15 where ys.product_id = bp.product_id 16 order by ys.yr_qty desc 17 fetch first row only 18 ) as yr_qty 19 from brewery_products bp 20 where bp.brewery_id = 518 21 order by bp.product_id;
Listing 2 Retrieving two columns from the best-selling year per beer
For the data at hand (where there are no ties between years), it works okay and gives me the desired output:
BREWERY_NAME P_ID PRODUCT_NAME YR YR_QTY Balthazar Brauerei 5310 Monks and Nuns 2017 582 Balthazar Brauerei 5430 Hercule Trippel 2018 451 Balthazar Brauerei 6520 Der Helle Kumpel 2017 458
But there are some issues with this strategy:
- The same data in
yearly_salesis accessed twice. Had I needed more than two columns, it would have been multiple times.
- Since my order by is not unique, my
fetch firstrow will return a random one (well, probably the first it happens to find using whichever access plan it uses, of which I have no control, so in effect, it could be any one) of those rows that have the highest
yr_qty. That means in the multiple subqueries, I have no guarantee that the values come from the same row – if I had had a column showing the profit of the beer in that year and a subquery to retrieve this profit, it might show the profit of a different year than the one shown in the yr column of the output.
A classic workaround is to use just a single scalar subquery like in Listing 3.
SQL> select 2 brewery_name 3 , product_id as p_id 4 , product_name 5 , to_number( 6 substr(yr_qty_str, 1, instr(yr_qty_str, ';') - 1) 7 ) as yr 8 , to_number( 9 substr(yr_qty_str, instr(yr_qty_str, ';') + 1) 10 ) as yr_qty 11 from ( 12 select 13 bp.brewery_name 14 , bp.product_id 15 , bp.product_name 16 , ( 17 select ys.yr || ';' || ys.yr_qty 18 from yearly_sales ys 19 where ys.product_id = bp.product_id 20 order by ys.yr_qty desc 21 fetch first row only 22 ) as yr_qty_str 23 from brewery_products bp 24 where bp.brewery_id = 518 25 ) 26 order by product_id;
Listing 3 Using just a single scalar subquery and value concatenation
The scalar subquery is here in lines 16–22, finding the row I want and then selecting in line 17 a concatenation of the values I am interested in. Then I place the entire thing in an inline view (lines 11–25) and split the concatenated string into individual values again in lines 5–10.
The output of this is exactly the same as Listing 2, so that is all good, right? Well, as you can see, if I need more than two columns, it can quickly become unwieldy code. If I had been concatenating string values, I would have needed to worry about using a delimiter that didn’t exist in the real data. If I had been concatenating dates and timestamps, I’d need to use
to_timestamp. And what if I had LOB columns or columns of complex types? Then I couldn’t do this at all.
So there are many good reasons to try Listing 4 as an alternative workaround.
SQL> select 2 brewery_name 3 , product_id as p_id 4 , product_name 5 , yr 6 , yr_qty 7 from ( 8 select 9 bp.brewery_name 10 , bp.product_id 11 , bp.product_name 12 , ys.yr 13 , ys.yr_qty 14 , row_number() over ( 15 partition by bp.product_id 16 order by ys.yr_qty desc 17 ) as rn 18 from brewery_products bp 19 join yearly_sales ys 20 on ys.product_id = bp.product_id 21 where bp.brewery_id = 518 22 ) 23 where rn = 1 24 order by product_id;
Using analytic function to be able to retrieve all columns if desired
This also gives the exact same output as Listing 2, just without any scalar subqueries at all.
Here I join the two views in lines 18–20 instead of querying
yearly_sales in a scalar subquery. But doing that makes it impossible for me to use the
fetch first syntax, as I need a row per brewery and
fetch first does not support a partition clause.
Instead I use the
row_number analytic function in lines 14–17 to assign consecutive numbers 1, 2, 3 … in descending order of
yr_qty, in effect giving the row with the highest
yr_qty the value 1 in rn. This happens for each beer because of the
partition by in line 15, so there will be a row with
rn=1 for each beer. These rows I keep with the
where clause in line 23.
The effect of this is that I can query as many columns from the
yearly_sales view as I want – here I query two columns in lines 12–13. These can then be used directly in the outer query as well in lines 5–6. No concatenation needed, each column is available directly, no matter the datatype.
This is a much nicer workaround than Listing 3, so isn’t this good enough? In this case it is fine, but the alternative with correlated inline views can be more flexible for some situations.
Correlating inline view
Listing 5 is yet another way to produce the exact same output as Listing 2, just this time by correlating an inline view.
SQL> select 2 bp.brewery_name 3 , bp.product_id as p_id 4 , bp.product_name 5 , top_ys.yr 6 , top_ys.yr_qty 7 from brewery_products bp 8 cross join lateral( 9 select 10 ys.yr 11 , ys.yr_qty 12 from yearly_sales ys 13 where ys.product_id = bp.product_id 14 order by ys.yr_qty desc 15 fetch first row only 16 ) top_ys 17 where bp.brewery_id = 518 18 order by bp.product_id;
Listing 5 Achieving the same with a lateral inline view
The way this works is as follows:
- I do not join
yearly_salesdirectly; instead I join to the inline view
top_ysin line 8.
- The inline view in lines 9–15 queries
yearly_salesand uses the
fetch firstrow to find the row of the year with the highest sales. But it is not executed for all beers finding a single row with the best-selling year across all beers, for line 13 correlates the
- Line 13 would normally raise an error, since it would not make sense in the usual joining to an inline view. But I placed the keyword lateral in front of the inline view in line 8, which tells the database that I want a correlation here, so it should execute the inline view once for each row of the correlated outer row source – in this case
brewery_products. That means that for each beer, there will be executed an individual
fetch firstrow query, almost as if it were a scalar subquery.
- I then use
cross joinin line 8 to do the actual joining, which simply is because I need no on clause in this case. I have all the correlation I need in line 13, so I need not use an
Using this lateral inline view enables me to get it executed for each beer like a scalar subquery, but to have individual columns queried like in Listing 4.
You might wonder about the
cross join and say, “This isn’t a Cartesian product, is it?”
Consider if I had used the traditional join style with a comma-separated list of tables and views and all join predicates in the
where clause and no on clauses. In that join style, Cartesian joins happen if you have no join predicate at all between two tables/views (sometimes that can happen by accident – a classic error that can be hard to catch).
If I had written Listing 5 with traditional style joins, line 8 would have looked like this:
... 7 from brewery_products bp 8 , lateral( 9 select ...
And with no join predicates in the
where clause, it does exactly the same that the
cross join does. But because of the lateral clause, it becomes a “Cartesian” join between each row of
brewery_products and each output row set of the correlated inline view as it is executed for each beer. So for each beer, it actually is a Cartesian product (think of it as “partitioned Cartesian”), but the net effect is that the total result looks like a correlated join and doesn’t appear Cartesian at all. Just don’t let the
cross join syntax confuse you.
I could have chosen to avoid the confusion of the
cross join by using a regular inner join like this:
... 7 from brewery_products bp 8 join lateral( 9 select ... 16 ) top_ys 17 on 1=1 18 where bp.brewery_id = 518 ...
Since the correlation happens inside the lateral inline view, I can simply let the on clause be always true. The effect is exactly the same.
It might be that you feel that both
cross join and the
on 1=1 methods really do not state clearly what happens – both syntaxes can be considered a bit “cludgy” if you will. Then perhaps you might like the alternative syntax
cross apply instead as in Listing 6.
SQL> select 2 bp.brewery_name 3 , bp.product_id as p_id 4 , bp.product_name 5 , top_ys.yr 6 , top_ys.yr_qty 7 from brewery_products bp 8 cross apply( 9 select 10 ys.yr 11 , ys.yr_qty 12 from yearly_sales ys 13 where ys.product_id = bp.product_id 14 order by ys.yr_qty desc 15 fetch first row only 16 ) top_ys 17 where bp.brewery_id = 518 18 order by bp.product_id;
The output is the same as Listing 2 like the previous listings, but this time I am using neither
join, but the keywords
cross apply in line 8. What this means is that for each row in brewery_products, the inline view will be applied. And when I use apply, I am allowed to correlate the inline view with the predicate in line 13, just like using
lateral. Behind the scenes, the database does exactly the same as a lateral inline view; it is just a case of which syntax you prefer.
cross distinguishes it from the variant outer apply, which I’ll show in a moment. Here cross is to be thought of as “partitioned Cartesian” as I discussed in the preceding text.
Note You can use the
outer applynot only for inline views but also for calling table functions (pipelined or not) in a correlated manner. This would require a longer syntax if you use lateral. Probably you won’t see it used often on table functions, as the table functions in Oracle can be used as a correlated row source in joins anyway, so it is rarely necessary to use apply, though sometimes it can improve readability.
Outer joining correlated inline view
So far my uses of
apply have only been of the
cross variety. That means that in fact I have been cheating a little – it is not really the same as using scalar subqueries. It is only because of having sales data for all the beers that Listings 1-2 to 1-6 all had the same output.
If a scalar subquery finds nothing, the value in that output column of the
brewery_products row will be null – but if a
cross join lateral or
cross apply inline view finds no rows, then the
brewery_products row will not be in the output at all.
What I need to really emulate the output of the scalar subquery method is a functionality like an
outer join, which I do in Listing 7. In this listing, I still find the top year and quantity for each beer, but only of those yearly sales that were less than 400.
SQL> select 2 bp.brewery_name 3 , bp.product_id as p_id 4 , bp.product_name 5 , top_ys.yr 6 , top_ys.yr_qty 7 from brewery_products bp 8 outer apply( 9 select 10 ys.yr 11 , ys.yr_qty 12 from yearly_sales ys 13 where ys.product_id = bp.product_id 14 and ys.yr_qty < 400 15 order by ys.yr_qty desc 16 fetch first row only 17 ) top_ys 18 where bp.brewery_id = 518 19 order by bp.product_id;
Listing 7 Using outer apply when you need
outer join functionality
In line 14, I make the inline view query only years that had sales of less than 400 bottles. And then in line 8, I changed
cross apply to outer apply, giving me this result:
BREWERY_NAME P_ID PRODUCT_NAME YR YR_QTY Balthazar Brauerei 5310 Monks and Nuns Balthazar Brauerei 5430 Hercule Trippel 2017 344 Balthazar Brauerei 6520 Der Helle Kumpel 2018 357
f I had been using
cross apply in line 8, I would only have seen the last two rows in the output.
So outer apply is more correct to use if you want an output that is completely identical to the scalar subquery method. But just like you don’t want to use regular outer joins unnecessarily, you should use
cross apply if you know for a fact that rows always will be returned.
An outer apply is the same as a
left outer join lateral with an on 1=1 join clause, so outer apply cannot support right correlation, only left.
There are cases where an outer join lateral is more flexible than outer apply, since you can actually use the on clause sensibly, like in Listing 8.
SQL> select 2 bp.brewery_name 3 , bp.product_id as p_id 4 , bp.product_name 5 , top_ys.yr 6 , top_ys.yr_qty 7 from brewery_products bp 8 left outer join lateral( 9 select 10 ys.yr 11 , ys.yr_qty 12 from yearly_sales ys 13 where ys.product_id = bp.product_id 14 order by ys.yr_qty desc 15 fetch first row only 16 ) top_ys 17 on top_ys.yr_qty < 500 18 where bp.brewery_id = 518 19 order by bp.product_id;
Listing 8 Outer join with the lateral keyword
Since I use lateral in the left outer join in line 8, the inline view is executed once for every beer, finding the best-selling year and quantity, just like most of the examples in the article. But in the on clause in line 17, I filter, so I only output a
top_ys row if the quantity is less than 500. It gives me this output, which is almost but not quite the same as the output of Listings 1-2 to 1-6:
BREWERY_NAME P_ID PRODUCT_NAME YR YR_QTY Balthazar Brauerei 5310 Monks and Nuns Balthazar Brauerei 5430 Hercule Trippel 2018 451 Balthazar Brauerei 6520 Der Helle Kumpel 2017 458
Normally the on clause is for the joining of the two tables (or views) and shouldn’t really contain a filter predicate. But in this case, it is exactly because I do the filtering in the on clause that I get the preceding result. Filtering in different places would solve different problems:
- If the filter predicate is inside the inline view (like Listing 7), the problem solved is “For each beer show me the best-selling year and quantity out of those years that sold less than 400 bottles.”
- If the filter predicate is in the on clause (like Listing 8), the problem solved is “For each beer show me the best-selling year and quantity if that year sold less than 500 bottles.”
- If the filter predicate had been in the where clause right after line 18, the problem solved would have been “For each beer where the best-selling year sold less than 500 bottles, show me the best-selling year and quantity.” (And then it shouldn’t be an
outer join, but just an
In all, lateral and apply (both in cross and outer versions) have several uses that, though they might be solvable by various other workarounds, can be quite nice and efficient. Typically you don’t want to use it if the best access path would be to build the entire results of the inline view first and then hash or merge the join with the outer table (for such a case, Listing 4 is often a better solution). But if the best path would be to do the outer table and then nested loop join to the inline view, lateral and apply are very nice methods.
In this article I’ve shown you some workarounds to some problems and then given you examples of how to solve the same using correlated inline views, so you now know about
- Using keyword lateral to enable doing a left correlation inside an inline view
- Distinguishing between cross and outer versions of joining to the lateral inline view
- Applying the
outer applyas alternative syntax to achieve a left correlation
- Deciding whether a correlated inline view or a regular inline view with analytic functions can solve a problem most efficiently
Being able to correlate inline views can be handy for several situations in your application development.