When do we use WITH clause, and what are main benefits of it?
C

5

5

I was working on task about optimization queries. One of the improvement ways was using WITH clause. I notice that it did very good job, and it lead to shorter time of execution, but i am not sure now, when should I use WITH clause and is there any risk of using it?

Here is one of the queries that I am working on :

WITH MY_TABLE AS 
    (   SELECT PROD_KY,
               sum(GROUPISPRIVATE) AS ISPRIVATE,
               sum(GROUPISSHARED) AS ISSHARED
        FROM
             (
                SELECT GRP_PROD_CUSTOMER.PROD_KY,
                       1 as ISPRIVATE,
                       0 as ISSHARED
                FROM CUSTOMER
                JOIN GRP_CUSTOMER ON GRP_CUSTOMER.CUST_KY = CUSTOMER.CUST_KY
                JOIN GRP_PROD_CUSTOMER ON GRP_PROD_CUSTOMER.GRP_KY = GRP_CUSTOMER.GRP_KY                                                                                                                                                                                                                                                                                                            
                GROUP BY GRP_PROD_CUSTOMER.PROD_KY
             ) 
   GROUP BY PROD_KY
)
SELECT * FROM MY_TABLE;
Coset answered 22/8, 2019 at 13:41 Comment(4)
Are you really sure the wrapping it in a sub-query factoring clause (WITH) in that query did anything as it appears to be useless?Vanettavang
Yes, I am sure. I also worked on more complex queries(with more sub-queries) where the difference is much bigger.Coset
I'm not saying that other queries may not have benefits in using a sub-query factoring clause but the particular query in your question wrapping it in WITH offers zero benefits in that instance (it also shouldn't hurt the performance but may make the query less readable as it has useless additional syntax for a developer to parse). Perhaps you need to pick a better example for your question?Vanettavang
I think most experienced Oracle developers will believe you are mistaken. There is no reason why the query you posted should benefit from the WITH clause. Be aware, Oracle does a lot of caching. If you run the query without the WITH and it is slow and then immediately run it with the WITH and it is fast, it does not necessarily mean that the second query was better. It's more likely that the second query is benefiting from the fact that the 1st one did all the hard work of reading the relevant blocks from disk. Don't measure walk-clock time -- measure consistent gets.Dusty
O
6

is there any risk of using it?

Yes. Oracle may decide to materialize the subquery, which means writing its result set to disk and then reading it back (except it might not mean that in 12cR2 or later). That unexpected I/O could be a performance hit. Not always, and usually we can trust the optimizer to make the correct choice. However, Oracle has provided us with hints to tell the optimizer how to handle the result set: /*+ materialize */ to um materialize it and /*+ inline */ to keep it in memory.

I start with this potential downside because I think it's important to understand that the WITH clause is not a silver bullet and it won't improve every single query, and may even degrade performance. For instance I share the scepticism of the other commenters that the query you posted is in any way faster because you re-wrote it as a common table expression.

Generally, the use cases for the WITH clause are:

  1. We want to use the result set from the subquery multiple times

    with cte as
      ( select blah from meh )
    select * 
    from t1
         join t2 on t1.id = t2.id
    where t1.col1 in ( select blah from cte )
    and   t2.col2 not in ( select blah from cte)
    
  2. We want to be build a cascade of subqueries:

    with cte as
      ( select id, blah from meh )
      , cte2 as 
       ( select t2.*, cte.blah
         from cte
              join t2 on t2.id = cte.id)
      , cte3 as 
        ( select t3.*, cte2.*
          from cte2
               join t3 on t3.col2 = cte2.something ) 
       ….
    

This second approach is beguiling and can be useful for implementing complex business logic in pure SQL. But it can lead to a procedural mindset and lose the power sets and joins. This too is a risk.

  1. We want to use recursive WITH clause. This allows us to replace Oracle's own CONNECT BY syntax with a more standard approach. Find out more

  2. In 12c and later we can write user-defined functions in the WITH clause. This is a powerful feature, especially for users who need to implement some logic in PL/SQL but only have SELECT access to the database. Find out more

For the record I have seen some very successful and highly performative uses of the second type of WITH clause. However I have also seen uses of WITH when it would have been just as easy to write an inline view. For instance, this is just using the WITH clause as syntactic sugar ...

with cte as
  ( select id, blah from meh )
select t2.*, cte.blah 
from  t2
      join cte on cte.id = t2.id

… and would be clearer as ...

select t2.*, cte.blah 
from  t2
      join ( select id, blah from meh ) cte on cte.id = t2.id
Overstuffed answered 22/8, 2019 at 14:10 Comment(7)
Why do you view that as a risk? Oracle will do that if the results of common table expression are needed in multiple plan steps. And materializing it doesn't necessarily mean "to disk".Dusty
The risk is not that Oracle may decide to materialize the subquery; the risk is that Oracle's decision to do so may be wrong. The optimizer decides whether to materialize the subquery, and it should do so only when it's optimal to do so. The risk is that the optimizer may be wrong. With that said, the same risk exists with many other transformations, not just with the WITH clause.Megilp
@mathguy I agree most with the last part of your comment. Avoiding WITH for that reason would be like saying I am not going to use views because Oracle may decide inappropriately to push a predicate down into it. Maybe the point is that WITH can complicate the CBO's job, just like views can. Both make the query more organized at the expense of optimization difficulties?Dusty
@MatthewMcPeak - And materializing it doesn't necessarily mean "to disk". I've always assumed - and it is an assumption - that the SYS generated temporary tables we see in explain plans - e.g. SYS_TEMP_0FDA22B05_93A119B8 - were basically global temporary tables and so written to disk. Can you explain what else might happen? ThanksOverstuffed
@mathguy - so many M's commenting! That's what I get for posting half an answer to avoid the three vote closure I saw on the horizon. Anyway. I don't think we shouldn't use WITH clause syntax; I am a great fan of it used properly. But I do think we should only use it when it's beneficial to do so.Overstuffed
@Overstuffed I was referring to in-memory cursor duration temp tables. Oracle will store them in PGA memory instead of disk, but if you are not on 12.2 or later, you'll have to wait. On 12.2, you'll see plan steps like "LOAD AS SELECT (CURSOR DURATION MEMORY)" in your plan.Dusty
@MatthewMcPeak - ah right. I was on 12cR1 for a couple of years but right now I'm back on 11g. Thanks for the clarificationOverstuffed
A
2

WITH clause is introduced in oracle to match SQL-99 standard.

The main purpose is to reduce the complexity and repetitive code.

Lets say you need to find the average salary of one department and then need to fetch all the department(d1) with more than average salary of that department(d1).

This can make multiple references to the subquery more efficient and readable.

The MATERIALIZE and INLINE optimizer hints can be used to influence the decision. The undocumented MATERIALIZE hint tells the optimizer to resolve the subquery as a global temporary table, while the INLINE hint tells it to process the query inline. Decision to use the hint is purely depends on logic that we are going to implement in query.

In oracle 12c, declaration of PL/SQL Block in WITH clause is introduced.

You must refer it from oracle documents.

Cheers!!

Adowa answered 22/8, 2019 at 14:32 Comment(0)
P
1

Your query is rather useless in terms of WITH statement (aka Common Table Expression, CTE)

Anyway, using the WITH clause brings several benefits:

  • The query is better readable (in my opinion)
  • You can use the same subquery several times in the main query. You can even cascade them.
  • Oracle can materialize the subquery, i.e. Oracle may create a temporary table and stores result of the subquery in it. This can give better performance.
Potage answered 22/8, 2019 at 14:16 Comment(0)
V
1

The WITH clause may be processed as an inline view or resolved as a temporary table. The SQL WITH clause is very similar to the use of Global temporary tables. This technique is often used to improve query speed for complex subqueries and enables the Oracle optimizer to push the necessary predicates into the views.

The advantage of the latter is that repeated references to the subquery may be more efficient as the data is easily retrieved from the temporary table, rather than being requeried by each reference. You should assess the performance implications of the WITH clause on a case-by-case basis.

You can read more here:

http://www.dba-oracle.com/t_with_clause.htm

https://oracle-base.com/articles/misc/with-clause

Vogt answered 22/8, 2019 at 14:23 Comment(0)
C
1

one point to consider is, that different RDBMS handle the with clause - aka common table expressions (CTE) aka subquery factoring - differently:

So depending on the RDBMS you use and its version your mileage may vary.

Contortionist answered 23/8, 2019 at 10:6 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.