Laravel Fluent Query Builder Join with subquery

Question

Okay after hours of research and still using DB::select I have to ask this question. Because I am about to throw my computer away ;).

I want to get the last input of a user (base on the timestamp). I can do this with the raw sql

SELECT  c.*, p.*
FROM    users c INNER JOIN
(
  SELECT  user_id,
          MAX(created_at) MaxDate
  FROM    `catch-text`
  GROUP BY user_id
 ) MaxDates ON c.id = MaxDates.user_id INNER JOIN
    `catch-text` p ON   MaxDates.user_id = p.user_id
     AND MaxDates.MaxDate = p.created_at

I got this query from another post here on stackoverflow.

I have tried everything to do this with the fluent query builder in Laravel however with no success.

I know the manual says you can do this:

DB::table('users')
    ->join('contacts', function($join)
    {
        $join->on('users.id', '=', 'contacts.user_id')->orOn(...);
    })
    ->get();

But that is not helping much because I do not see how I could use a subquery there? Anyone who can light up my day?

Begrudge

From Laravel 9 at least, there is a joinSub() method that allows you to build a sub-query using all the tools of the query builder, give the sub-query a name, then join it to the outer query however you like. – Smack

Answer 1

Ok for all of you out there that arrived here in desperation searching for the same problem. I hope you will find this quicker then I did ;O.

This is how it is solved. JoostK told me at github that "the first argument to join is the table (or data) you're joining.". And he was right.

Here is the code. Different table and names but you will get the idea right? It t

DB::table('users')
        ->select('first_name', 'TotalCatches.*')

        ->join(DB::raw('(SELECT user_id, COUNT(user_id) TotalCatch,
               DATEDIFF(NOW(), MIN(created_at)) Days,
               COUNT(user_id)/DATEDIFF(NOW(), MIN(created_at))
               CatchesPerDay FROM `catch-text` GROUP BY user_id)
               TotalCatches'), 
        function($join)
        {
           $join->on('users.id', '=', 'TotalCatches.user_id');
        })
        ->orderBy('TotalCatches.CatchesPerDay', 'DESC')
        ->get();

Begrudge

This package is supposed to provide a PHP tool set to build queries, yet we need to write down the query and inject it by selectRaw statement! Did you have any luck to pass a fluent query object to the query, instead of query string? – Combined

This becomes impossible if you have a parameter in the subselect - see @ph4r05 answer for example, but I'm not sure if it could be simplified to avoid those ugly toSql and addBinding – Exceptional

Thanks! The Group By portion of the subquery is important. I was trying a similar query and my calculated values were off depending on the number of results for each join. I tried a Union ALL but that was unmanageable. This solution works perfectly. – Mccool

Your statement needs to alias the subquery ...) as TotalCatches – Savoyard

@Combined you can do something like DB::raw( '('. Model::select('id')->modeFiltering->toSql() . ') AS tableAlias' ) – Savoyard

Answer 2

I was looking for a solution to quite a related problem: finding the newest records per group which is a specialization of a typical greatest-n-per-group with N = 1.

The solution involves the problem you are dealing with here (i.e., how to build the query in Eloquent) so I am posting it as it might be helpful for others. It demonstrates a cleaner way of sub-query construction using powerful Eloquent fluent interface with multiple join columns and where condition inside joined sub-select.

In my example I want to fetch the newest DNS scan results (table scan_dns) per group identified by watch_id. I build the sub-query separately.

The SQL I want Eloquent to generate:

SELECT * FROM `scan_dns` AS `s`
INNER JOIN (
  SELECT x.watch_id, MAX(x.last_scan_at) as last_scan
  FROM `scan_dns` AS `x`
  WHERE `x`.`watch_id` IN (1,2,3,4,5,42)
  GROUP BY `x`.`watch_id`) AS ss
ON `s`.`watch_id` = `ss`.`watch_id` AND `s`.`last_scan_at` = `ss`.`last_scan`

I did it in the following way:

// table name of the model
$dnsTable = (new DnsResult())->getTable();

// groups to select in sub-query
$ids = collect([1,2,3,4,5,42]);

// sub-select to be joined on
$subq = DnsResult::query()
    ->select('x.watch_id')
    ->selectRaw('MAX(x.last_scan_at) as last_scan')
    ->from($dnsTable . ' AS x')
    ->whereIn('x.watch_id', $ids)
    ->groupBy('x.watch_id');
$qqSql = $subq->toSql();  // compiles to SQL

// the main query
$q = DnsResult::query()
    ->from($dnsTable . ' AS s')
    ->join(
        DB::raw('(' . $qqSql. ') AS ss'),
        function(JoinClause $join) use ($subq) {
            $join->on('s.watch_id', '=', 'ss.watch_id')
                 ->on('s.last_scan_at', '=', 'ss.last_scan')
                 ->addBinding($subq->getBindings());  
                 // bindings for sub-query WHERE added
        });

$results = $q->get();

UPDATE:

Since Laravel 5.6.17 the sub-query joins were added so there is a native way to build the query.

$latestPosts = DB::table('posts')
                   ->select('user_id', DB::raw('MAX(created_at) as last_post_created_at'))
                   ->where('is_published', true)
                   ->groupBy('user_id');

$users = DB::table('users')
        ->joinSub($latestPosts, 'latest_posts', function ($join) {
            $join->on('users.id', '=', 'latest_posts.user_id');
        })->get();

Rouvin

JoinCluse doesn't have an addBinding method for me, Laravel 5.2, any other suggestion? – Compressor

I found the new 5.6 way to do very nice to use (with the joinSub, leftJoinSub, etc ...) but I have an issue, my subquery doesn't take into account the ON condition to give results to the join. I have something like $userInfoSubquery = DB::table('user_info') ->select() ->orderBy('info_date', 'DESC') ->limit(1); as my subquery. But if the newest info does not belong to the user, it won't return something ... So I need to be able to do something like : ->where('user_id', '=', userid) somehow in the subquery ... – Goa

You can use query builder directly in the sub-query, or you can use eloquent models if that helps with scopes and relationships that you may already have in the models. – Smack

Answer 3

I think what you looking for is "joinSub". It's supported from laravel ^5.6. If you using laravel version below 5.6 you can also register it as macro in your app service provider file. like this https://github.com/teamtnt/laravel-scout-tntsearch-driver/issues/171#issuecomment-413062522

$subquery = DB::table('catch-text')
            ->select(DB::raw("user_id,MAX(created_at) as MaxDate"))
            ->groupBy('user_id');

$query = User::joinSub($subquery,'MaxDates',function($join){
          $join->on('users.id','=','MaxDates.user_id');
})->select(['users.*','MaxDates.*']);

Utrecht

Answer 4

I am on Laravel 7.25 and I don't know if it supports on previous versions or not but Its pretty good.

Syntax for the function:

public function joinSub($query, $as, $first, $operator = null, $second = null, $type = 'inner', $where = false)

Example:

Showing/Getting the user ID and the total number of posts by them left joining two tables users and posts.

        return DB::table('users')
            ->joinSub('select user_id,count(id) noOfPosts from posts group by user_id', 'totalPosts', 'users.id', '=', 'totalPosts.user_id', 'left')
            ->select('users.name', 'totalPosts.noOfPosts')
            ->get();

Alternative:

If you don't wanna mention 'left' for leftjoin then you can use another prebuilt function

    public function leftJoinSub($query, $as, $first, $operator = null, $second = null)
    {
        return $this->joinSub($query, $as, $first, $operator, $second, 'left');
    }

And yeah, it actually calls the same function but it passes the join type itself. You can apply the same logic for other joins i.e. righJoinSub(...) etc.

Photophore

Answer 5

Query with sub query in Laravel

$resortData = DB::table('resort')
        ->leftJoin('country', 'resort.country', '=', 'country.id')
        ->leftJoin('states', 'resort.state', '=', 'states.id')
        ->leftJoin('city', 'resort.city', '=', 'city.id')
        ->select('resort.*', 'country.name as country_name', 'states.name as state_name','city.name as city_name', DB::raw("(SELECT GROUP_CONCAT(amenities.name) from resort_amenities LEFT JOIN amenities on amenities.id= resort_amenities.amenities_id WHERE resort_amenities.resort_id=resort.id) as amenities_name"))->groupBy('resort.id')
        ->orderBy('resort.id', 'DESC')
       ->get();

Contingence

Answer 6

I can't comment because my reputation is not high enough. @Franklin Rivero if you are using Laravel 5.2 you can set the bindings on the main query instead of the join using the setBindings method.

So the main query in @ph4r05's answer would look something like this:

$q = DnsResult::query()
    ->from($dnsTable . ' AS s')
    ->join(
        DB::raw('(' . $qqSql. ') AS ss'),
        function(JoinClause $join) {
            $join->on('s.watch_id', '=', 'ss.watch_id')
                 ->on('s.last_scan_at', '=', 'ss.last_scan');
        })
    ->setBindings($subq->getBindings());

Emogeneemollient

Answer 7

You can use following addon to handle all subquery related function from laravel 5.5+

https://github.com/maksimru/eloquent-subquery-magic

User::selectRaw('user_id,comments_by_user.total_count')->leftJoinSubquery(
  //subquery
  Comment::selectRaw('user_id,count(*) total_count')
      ->groupBy('user_id'),
  //alias
  'comments_by_user', 
  //closure for "on" statement
  function ($join) {
      $join->on('users.id', '=', 'comments_by_user.user_id');
  }
)->get();

Iconostasis

Answer 8

For my case, I found another solution, it is to specify the query with a sub-query, to avoid ambiguity.

Property::joinSub(Agency::select('id', 'subscription', )->getQuery(), 'agencies', 'agencies.id', 'properties.agency_id')

Nucellus