Doctrine - How to print out the real sql, not just the prepared statement?

Question

We're using Doctrine, a PHP ORM. I am creating a query like this:

$q = Doctrine_Query::create()->select('id')->from('MyTable');

and then in the function I'm adding in various where clauses and things as appropriate, like this

$q->where('normalisedname = ? OR name = ?', array($string, $originalString));

Later on, before execute()-ing that query object, I want to print out the raw SQL in order to examine it, and do this:

$q->getSQLQuery();

However that only prints out the prepared statement, not the full query. I want to see what it is sending to the MySQL, but instead it is printing out a prepared statement, including ?'s. Is there some way to see the 'full' query?

Lauer

Best way I've found to see full query is described in this answer: https://mcmap.net/q/56129/-how-to-show-the-last-queries-executed-on-mysql – Dudgeon

You can take advantage of the work done by Doctrine (the profiler is displaying a runnable query). See my answer below for details – Phi

Answer 1

Doctrine is not sending a "real SQL query" to the database server : it is actually using prepared statements, which means :

• Sending the statement, for it to be prepared (this is what is returned by $query->getSql())
• And, then, sending the parameters (returned by $query->getParameters())
• and executing the prepared statements

This means there is never a "real" SQL query on the PHP side — so, Doctrine cannot display it.

Heilner

Pascal: you shouldn't say it's not a "real SQL query" because prepared statement are real SQL query, it's just that the parameters as sent out separately. This wording could confuse people (e.g. olivierpons.fr/2014/03/22/symfony-2-avantages-et-inconvenients). – Krak

$query->getParameters(); will NOT return parameters in correct order, as they should appear in prepared query statement – Worcestershire

I think here author of question did not care what doctrine sends or not. What user and I wanted to know is how to get query which we can copy paste and run without having to manually replaces question marks with parameters. Like in codeigniter. I think I had found this in symfony debugger, but I still cannot find when I run script from command line. – Wuhsien

Answer 2

A working example:

$qb = $this->createQueryBuilder('a');
$query=$qb->getQuery();
// SHOW SQL: 
echo $query->getSQL(); 
// Show Parameters: 
echo $query->getParameters();

Axletree

Whilst it works as variable assignments, you might want to consider this: print $query->getSQL(); foreach ($query->getParameters() as $param) { print "{$param->getName()} -> {$param->getValue()}\n"; } as you'll get a more readable output – Exceptional

it give littel benefit. When I copy the sql, I still have search wichi parameter where to insert manually, it takes ton of time. We want a query with inserted parameterrs, why we cannot find it so long? Even in codeigniter framework as far as I remember, in the profiler you could copy the query and run instantly without manually. We need same on symfony. – Wuhsien

Answer 3

You can check the query executed by your app if you log all the queries in mysql:

http://dev.mysql.com/doc/refman/5.1/en/query-log.html

there will be more queries not only the one that you are looking for but you can grep for it.

but usually ->getSql(); works

Edit:

to view all the mysql queries I use

sudo vim /etc/mysql/my.cnf 

and add those 2 lines:

general_log = on
general_log_file = /tmp/mysql.log

and restart mysql

Edit 2 In case you dont find the mysql config (it can be in many places), just set those variables from mysql command line.

mysql -u root -p 

SHOW VARIABLES LIKE 'general_log_file';
SHOW VARIABLES LIKE 'general_log';

SET GLOBAL general_log = 'on';
SET GLOBAL general_log_file = '/tmp/mysql.log';

//view the queries
sudo tail -f /tmp/mysql.log

The life of those settings is until MySQL is restarted. Or the laptop. So they are not permanent - which is great in my opinion - I just need them when I debug and I dont need to worry to edit the config then to remove them. If you dont remove the logging, it might grow too much if you forget about it.

Reprove

Answer 4

I have created a Doctrine2 Logger that does exactly this. It "hydrates" the parametrized sql query with the values using Doctrine 2 own data type conversors.

<?php


namespace Drsm\Doctrine\DBAL\Logging;
use Doctrine\DBAL\Logging\SQLLogger,
    Doctrine\DBAL\Types\Type,
    Doctrine\DBAL\Platforms\AbstractPlatform;
/**
 * A SQL logger that logs to the standard output and
 * subtitutes params to get a ready to execute SQL sentence

 * @author  [email protected]
 */
class EchoWriteSQLWithoutParamsLogger implements SQLLogger

{
    const QUERY_TYPE_SELECT="SELECT";
    const QUERY_TYPE_UPDATE="UPDATE";
    const QUERY_TYPE_INSERT="INSERT";
    const QUERY_TYPE_DELETE="DELETE";
    const QUERY_TYPE_CREATE="CREATE";
    const QUERY_TYPE_ALTER="ALTER";

    private $dbPlatform;
    private $loggedQueryTypes;
    public function __construct(AbstractPlatform $dbPlatform, array $loggedQueryTypes=array()){
        $this->dbPlatform=$dbPlatform;
        $this->loggedQueryTypes=$loggedQueryTypes;
    }
    /**
     * {@inheritdoc}
     */
    public function startQuery($sql, array $params = null, array $types = null)

    {
        if($this->isLoggable($sql)){
            if(!empty($params)){
                foreach ($params as $key=>$param) {
                    $type=Type::getType($types[$key]);
                    $value=$type->convertToDatabaseValue($param,$this->dbPlatform);
                    $sql = join(var_export($value, true), explode('?', $sql, 2));
                }

            }
            echo $sql . " ;".PHP_EOL;
        }
    }

    /**
     * {@inheritdoc}
     */
    public function stopQuery()
    {

    }
    private function isLoggable($sql){
        if (empty($this->loggedQueryTypes)) return true;
        foreach($this->loggedQueryTypes as $validType){
            if (strpos($sql, $validType) === 0) return true;
        }
        return false;
    }
}

Usage Example:; The following peace of code will echo on standard output any INSERT,UPDATE,DELETE SQL sentences generated with $em Entity Manager,

/**@var  \Doctrine\ORM\EntityManager $em */
$em->getConnection()
                ->getConfiguration()
                ->setSQLLogger(
                    new EchoWriteSQLWithoutParamsLogger(
                        $em->getConnection()->getDatabasePlatform(),
                        array(
                            EchoWriteSQLWithoutParamsLogger::QUERY_TYPE_UPDATE,
                            EchoWriteSQLWithoutParamsLogger::QUERY_TYPE_INSERT,
                            EchoWriteSQLWithoutParamsLogger::QUERY_TYPE_DELETE
                        )
                    )
                );

Hyphenated

Does not work when parameters are date strings like '2019-01-01' – Wuhsien

Answer 5

There is no other real query, this is how prepared statements work. The values are bound in the database server, not in the application layer.

See my answer to this question: In PHP with PDO, how to check the final SQL parametrized query?

(Repeated here for convenience:)

Using prepared statements with parametrised values is not simply another way to dynamically create a string of SQL. You create a prepared statement at the database, and then send the parameter values alone.

So what is probably sent to the database will be a PREPARE ..., then SET ... and finally EXECUTE ....

You won't be able to get some SQL string like SELECT * FROM ..., even if it would produce equivalent results, because no such query was ever actually sent to the database.

Guinna

Answer 6

getSqlQuery() does technically show the whole SQL command, but it's a lot more useful when you can see the parameters as well.

echo $q->getSqlQuery();
foreach ($q->getFlattenedParams() as $index => $param)
  echo "$index => $param";

To make this pattern more reusable, there's a nice approach described in the comments at Raw SQL from Doctrine Query Object.

Shonna

I know this is an old post, but both of your links lead to a 404 page. Can you update your answer please? I am asking, because i am not sure what you mean with $q. It doesn't seem to be the query nor the query builder. – Auerbach

I'm afraid I can't find the more reusable code. $q in this case is a Doctrine 1 query. You may be using Doctrine 2, in which case you'll want something like $qb = $this->createQueryBuilder('a'); $q = $qb->getQuery(); $sql = $q->getSQL(); $params = $q->getParameters(); Hopefully that helps! – Shonna

Answer 7

My solution:

 /**
 * Get SQL from query
 * 
 * @author Yosef Kaminskyi 
 * @param QueryBilderDql $query
 * @return int
 */
public function getFullSQL($query)
{
    $sql = $query->getSql();
    $paramsList = $this->getListParamsByDql($query->getDql());
    $paramsArr =$this->getParamsArray($query->getParameters());
    $fullSql='';
    for($i=0;$i<strlen($sql);$i++){
        if($sql[$i]=='?'){
            $nameParam=array_shift($paramsList);

            if(is_string ($paramsArr[$nameParam])){
                $fullSql.= '"'.addslashes($paramsArr[$nameParam]).'"';
             }
            elseif(is_array($paramsArr[$nameParam])){
                $sqlArr='';
                foreach ($paramsArr[$nameParam] as $var){
                    if(!empty($sqlArr))
                        $sqlArr.=',';

                    if(is_string($var)){
                        $sqlArr.='"'.addslashes($var).'"';
                    }else
                        $sqlArr.=$var;
                }
                $fullSql.=$sqlArr;
            }elseif(is_object($paramsArr[$nameParam])){
                switch(get_class($paramsArr[$nameParam])){
                    case 'DateTime':
                             $fullSql.= "'".$paramsArr[$nameParam]->format('Y-m-d H:i:s')."'";
                          break;
                    default:
                        $fullSql.= $paramsArr[$nameParam]->getId();
                }

            }
            else                     
                $fullSql.= $paramsArr[$nameParam];

        }  else {
            $fullSql.=$sql[$i];
        }
    }
    return $fullSql;
}

 /**
 * Get query params list
 * 
 * @author Yosef Kaminskyi <[email protected]>
 * @param  Doctrine\ORM\Query\Parameter $paramObj
 * @return int
 */
protected function getParamsArray($paramObj)
{
    $parameters=array();
    foreach ($paramObj as $val){
        /* @var $val Doctrine\ORM\Query\Parameter */
        $parameters[$val->getName()]=$val->getValue();
    }

    return $parameters;
}
 public function getListParamsByDql($dql)
{
    $parsedDql = preg_split("/:/", $dql);
    $length = count($parsedDql);
    $parmeters = array();
    for($i=1;$i<$length;$i++){
        if(ctype_alpha($parsedDql[$i][0])){
            $param = (preg_split("/[' ' )]/", $parsedDql[$i]));
            $parmeters[] = $param[0];
        }
    }

    return $parmeters;}

Example of usage:

$query = $this->_entityRepository->createQueryBuilder('item');
$query->leftJoin('item.receptionUser','users');
$query->where('item.customerid = :customer')->setParameter('customer',$customer)
->andWhere('item.paymentmethod = :paymethod')->setParameter('paymethod',"Bonus");
echo $this->getFullSQL($query->getQuery());

Monk

very nice. works with normal queries but I have got a query with regexp and looks like does not support $qb = $this->createQueryBuilder('r') ->innerJoin('r.profile', 'p') ->addSelect('p') ->where('REGEXP(:fileNamePattern, r.fileNamePattern) = 1') ->andWhere('p.incomingLocation = :incomingLocation') ->setParameters([ 'fileNamePattern' => $fileName, 'incomingLocation' => $location ])->getQuery(); – Gerlach

Does not work with all queries. When I had this ->setParameters(array( 'insuranceCarrier' => $insuranceCarrier, 'dateFrom' => $dateFrom->format('Y-m-d'), 'dateTo' => $dateTo->format('Y-m-d'), )) those were left with ? marks in sql. – Wuhsien

Answer 8

You can easily access the SQL parameters using the following approach.

   $result = $qb->getQuery()->getSQL();

   $param_values = '';  
   $col_names = '';   

   foreach ($result->getParameters() as $index => $param){              
            $param_values .= $param->getValue().',';
            $col_names .= $param->getName().',';
   } 

   //echo rtrim($param_values,',');
   //echo rtrim($col_names,',');    

So if you printed out the $param_values and $col_names , you can get the parameter values passing through the sql and respective column names.

Note : If $param returns an array, you need to re iterate, as parameters inside IN (:?) usually comes is as a nested array.

Meantime if you found another approach, please be kind enough to share with us :)

Thank you!

Polacre

Answer 9

More clear solution:

 /**
 * Get string query 
 * 
 * @param Doctrine_Query $query
 * @return string
 */
public function getDqlWithParams(Doctrine_Query $query){
    $vals = $query->getFlattenedParams();
    $sql = $query->getDql();
    $sql = str_replace('?', '%s', $sql);
    return vsprintf($sql, $vals);
}

Nikolos

$query->getFlattenedParams(); not exist – Dottie

@Dottie for newer version of Doctrine you can replace getFlattenedParams() by getParameters(). Also one may find more useful getSQL() instead of getDql(). – Inmost

Answer 10

Maybe it can be useful for someone:

// Printing the SQL with real values
$vals = $query->getFlattenedParams();
foreach(explode('?', $query->getSqlQuery()) as $i => $part) {
    $sql = (isset($sql) ? $sql : null) . $part;
    if (isset($vals[$i])) $sql .= $vals[$i];
}

echo $sql;

Triumvirate

Answer 11

Solution:1
====================================================================================

function showQuery($query)
{
    return sprintf(str_replace('?', '%s', $query->getSql()), $query->getParams());
}

// call function  
echo showQuery($doctrineQuery);

Solution:2
====================================================================================

function showQuery($query)
{
    // define vars              
    $output    = NULL;
    $out_query = $query->getSql();
    $out_param = $query->getParams();

    // replace params
   for($i=0; $i<strlen($out_query); $i++) {
       $output .= ( strpos($out_query[$i], '?') !== FALSE ) ? "'" .str_replace('?', array_shift($out_param), $out_query[$i]). "'" : $out_query[$i];
   }

   // output
   return sprintf("%s", $output);
}

// call function  
echo showQuery($doctrineQueryObject);

Yardage

Answer 12

You can use :

$query->getSQL();

If you are using MySQL you can use Workbench to view running SQL statements. You can also use view the running query from mysql by using the following :

 SHOW FULL PROCESSLIST \G

Noreen

Answer 13

TL;DR

$qb = ... // your query builder
$query = $qb->getQuery();
// temporarily enable logging for your query (will also work in prod env)
$conf = $query->getEntityManager()->getConnection()->getConfiguration();
$backupLogger = $conf->getSQLLogger();
$logger = new \Doctrine\DBAL\Logging\DebugStack();
$conf->setSQLLogger($logger);
// execute query
$res = $query->getResult();
$conf->setSQLLogger($backupLogger); //restore logger for other queries
$params = [
  'query' => array_pop($logger->queries) //extract query log details
  //your other twig params here...
]
return $params; //send this to your twig template...

in your twig files, use Doctrine's twig helpers filters:

// show raw query:
{{ (query.sql ~ ';')|doctrine_replace_query_parameters(query.params)
// highlighted
{{ (query.sql ~ ';')|doctrine_replace_query_parameters(query.params)|doctrine_pretty_query(highlight_only = true) }}
// highlighted and formatted (i.e. with tabs and newlines)
{{ (query.sql ~ ';')|doctrine_replace_query_parameters(query.params)|doctrine_pretty_query }}

Explanation:

The other answers mentioning that Prepared statement are actually "real queries" are right, but they don't answer the obvious asker's expectation... Every developer wants to display a "runnable query" for debugging (or to display it to the user).

So, I looked into Symfony profiler's source to see how they do it. The Doctrine part is Doctrine's responsibility so they made a doctrine-bundle to integrate with Symfony. Having a look at the doctrine-bundle/Resources/views/Collector/db.html.twig file, you will find out how they do it (this might change across versions). Interestingly, they created twig filters that we can reuse (see above).

For everything to work we need to enable Logging for our query. There are multiple ways to do this and here I use DebugStack which allows to log queries without actually printing them. This also ensure that this will work in production mode if this is what you need...

If you need further formatting, you will see that they include some CSS in a style tag, so simply "steal" it ^^:

.highlight pre { margin: 0; white-space: pre-wrap; }
.highlight .keyword   { color: #8959A8; font-weight: bold; }
.highlight .word      { color: #222222; }
.highlight .variable  { color: #916319; }
.highlight .symbol    { color: #222222; }
.highlight .comment   { color: #999999; }
.highlight .backtick  { color: #718C00; }
.highlight .string    { color: #718C00; }
.highlight .number    { color: #F5871F; font-weight: bold; }
.highlight .error     { color: #C82829; }

Hope, this will help ;-)

Phi

This helps a lot ! I just take your code and put it in a function executeAndReturnRealQuery($query). The only changes I made is I directly use an instance of DoctrineExtension, then return $doctrineExtension->replaceQueryParameters($params['query']['sql'], $params['query']['sql']); – Atory

Answer 14

I wrote a simple logger, which can log query with inserted parameters. Installation:

composer require cmyker/doctrine-sql-logger:dev-master

Usage:

$connection = $this->getEntityManager()->getConnection(); 
$logger = new \Cmyker\DoctrineSqlLogger\Logger($connection);
$connection->getConfiguration()->setSQLLogger($logger);
//some query here
echo $logger->lastQuery;

Filtration

Answer 15

I made some research for this topic, because i wanted to debug a generated SQL query and execute it in the sql editor. As seen in all the answers, it is a highly technical topic.

When i assume that the initial question is base on dev-env, one very simple answer is missing at the moment. You can just use the build in Symfony profiler. Just click on the Doctrine Tab, Scroll to the query you want to inspect. Then click on "view runnable query" and you can paste your query directly in your SQL editor

More UI base approach but very quick and without debugging code overhead.

Monroy

Just found also how to use this with API calls. Just look at the response header for x-debug-token-link, it's the URL of the associated profiler page :) – Alimony

This solution is poor... ti's better to get the real full query from code. – Mothball

Answer 16

Modified @dsamblas function to work when parameters are date strings like this '2019-01-01' and when there is array passed using IN like

$qb->expr()->in('ps.code', ':activeCodes'),

. So do everything what dsamblas wrote, but replace startQuery with this one or see the differences and add my code. (in case he modified something in his function and my version does not have modifications).

public function startQuery($sql, array $params = null, array $types = null)

{
    if($this->isLoggable($sql)){
        if(!empty($params)){
            foreach ($params as $key=>$param) {

                try {
                    $type=Type::getType($types[$key]);
                    $value=$type->convertToDatabaseValue($param,$this->dbPlatform);
                } catch (Exception $e) {
                    if (is_array($param)) {
                        // connect arrays like ("A", "R", "C") for SQL IN
                        $value = '"' . implode('","', $param) . '"';
                    } else {
                        $value = $param; // case when there are date strings
                    }
                }

                $sql = join(var_export($value, true), explode('?', $sql, 2));
            }

        }
        echo $sql . " ;".PHP_EOL;
    }
}

Did not test much.

Wuhsien

Answer 17

$sql = $query->getSQL();
$obj->mapDQLParametersNamesToSQL($query->getDQL(), $sql);
echo $sql;//to see parameters names in sql
$obj->mapDQLParametersValuesToSQL($query->getParameters(), $sql);
echo $sql;//to see parameters values in sql

public function mapDQLParametersNamesToSQL($dql, &$sql)
{
    $matches = [];
    $parameterNamePattern = '/:\w+/';
    /** Found parameter names in DQL */
    preg_match_all($parameterNamePattern, $dql, $matches);
    if (empty($matches[0])) {
        return;
    }
    $needle = '?';
    foreach ($matches[0] as $match) {
        $strPos = strpos($sql, $needle);
        if ($strPos !== false) {
            /** Paste parameter names in SQL */
            $sql = substr_replace($sql, $match, $strPos, strlen($needle));
        }
    }
}

public function mapDQLParametersValuesToSQL($parameters, &$sql)
{
    $matches = [];
    $parameterNamePattern = '/:\w+/';
    /** Found parameter names in SQL */
    preg_match_all($parameterNamePattern, $sql, $matches);
    if (empty($matches[0])) {
        return;
    }
    foreach ($matches[0] as $parameterName) {
        $strPos = strpos($sql, $parameterName);
        if ($strPos !== false) {
            foreach ($parameters as $parameter) {
                /** @var \Doctrine\ORM\Query\Parameter $parameter */
                if ($parameterName !== ':' . $parameter->getName()) {
                    continue;
                }
                $parameterValue = $parameter->getValue();
                if (is_string($parameterValue)) {
                    $parameterValue = "'$parameterValue'";
                }
                if (is_array($parameterValue)) {
                    foreach ($parameterValue as $key => $value) {
                        if (is_string($value)) {
                            $parameterValue[$key] = "'$value'";
                        }
                    }
                    $parameterValue = implode(', ', $parameterValue);
                }
                /** Paste parameter values in SQL */
                $sql = substr_replace($sql, $parameterValue, $strPos, strlen($parameterName));
            }
        }
    }
}

Jessjessa

Answer 18

$stmt = $this->connection->prepare($sql);

var_dump($stmt->getWrappedStatement());

Handed

Answer 19

$sql = $query->getSQL();

$parameters = [];
    foreach ($query->getParameters() as $parameter) {
        $parameters[] = $parameter->getValue();
    }

$result = $connection->executeQuery($sql, $parameters)
        ->fetchAll();

Turnery

You should add some text to your answer explaining what the code does. – Squama

Answer 20

You can build an sql string by combining the sql prepared statement with bindings like this way:

$sql = str_replace_array('?', $query->getBindings(), $query->toSql())

str_replace_array(string $search, array $replacement, string $subject): string

PHP's str_replace_array function replaces each instance of $search in $subject with values from $replacement array sequentially.

Sheathbill

str_replace_array is not a standard PHP function – Darbies

Answer 21

To print out an SQL query in Doctrine, use:

$query->getResult()->getSql();

Nahum

do not forget to add description with your Answer? Just one liner without description, not acceptable. – Pollute

To print out sql query in Doctrine use $query->getResult()->getSql(); Thanks – Nahum

instead of adding commnet, Edit your answer – Pollute