Disable Doctrine 2 lazy loading when using JMS Serializer?

Question

Im using Doctrine 2 ORM in my Zend project and need to serialize my Entities to JSON in several cases.

ATM i use the Querybuilder and join all tables i need. But my serializer causes doctrine to lazy load every associated Entity which results in pretty huge data amounts and provokes recursion.

Now im looking for a way to totally disable Doctrines lazy loading behavior.

My way to select data would be the following:

$qb= $this->_em->createQueryBuilder()
            ->from("\Project\Entity\Personappointment", 'pa')
            ->select('pa', 't', 'c', 'a', 'aps', 'apt', 'p')
            ->leftjoin('pa.table', 't')
            ->leftjoin('pa.company', 'c')
            ->leftjoin('pa.appointment', 'a')
            ->leftjoin('a.appointmentstatus', 'aps')
            ->leftjoin('a.appointmenttype', 'apt')
            ->leftjoin('a.person','p')

I would like my resultset to only contain the selected tables and associations.

Any help would be greatly appreciated.

Gascony

If you use JMS Serializer, rely on my Answer. If you want to avoid JMS Serializer altogether, rely on Exanders Answer (especially the comments). – Gascony

Answer 1

After having looked for the answer in Doctrine, my team figured out that the JMS Serializer was the "problem". It triggered the use of Doctrine Proxies automatically. We wrote a Patch for JMS Serializer to avoid the Lazy Loading.

We implemented our own DoctrineProxyHandler which just doesn't trigger Doctrines lazyloading mechanism and registered it within our SerializationHandlers Array.

class DoctrineProxyHandler implements SerializationHandlerInterface {

public function serialize(VisitorInterface $visitor, $data, $type, &$handled)
{
    if (($data instanceof Proxy || $data instanceof ORMProxy) && (!$data->__isInitialized__ || get_class($data) === $type)) {
        $handled = true;

        if (!$data->__isInitialized__) {

            //don't trigger doctrine lazy loading
            //$data->__load();

            return null;
        }

        $navigator = $visitor->getNavigator();
        $navigator->detachObject($data);

        // pass the parent class not to load the metadata for the proxy class
        return $navigator->accept($data, get_parent_class($data), $visitor);
    }

    return null;
}

Now i can simply select my table, join the associations i need - and my JSON will contain just the data i selected instead of infinite depth associations and recursions :)

$qb= $this->_em->createQueryBuilder()
        ->from("\Project\Entity\Personappointment", 'pa')
        ->select('pa', 't', 'c', 'a')
        ->leftjoin('pa.table', 't')
        ->leftjoin('pa.company', 'c')
        ->leftjoin('pa.appointment', 'a')

JSON will just contain

{  
  Personappointment: { table {fields}, company {fields}, appointment {fields}}
  Personappointment: { table {fields}, company {fields}, appointment {fields}}
  Personappointment: { table {fields}, company {fields}, appointment {fields}}
  .
  .
}

Gascony

How do you enable a custom proxyhandler? – Finkelstein

Be careful, we figured out that there are several more places where lazy loading is triggered, although these are used less often. Can lead to strange behavior if ignored. – Gascony

Specifically when collections of data are loaded (Doctrine\ORM\PersistentCollection). – Bowling

Thanks for is answer, which I strongly inspired to build this component: github.com/alcalyn/serializer-doctrine-proxies which spots also another place where there is autoloading (see class DoctrineProxySubscriber) – Clemenceau

Answer 2

In the latest version of JMSSerializer, the place you should look at is

JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber

instead of

Serializer\Handler\DoctrineProxyHandler

To override the default lazy load behavior, one should define his own event subscriber.

In your app/config.yml add this:

parameters:
    ...
    jms_serializer.doctrine_proxy_subscriber.class: Your\Bundle\Event\DoctrineProxySubscriber

you can copy the class from JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber to Your\Bundle\Event\DoctrineProxySubscriber and comment out the $object->__load(); line

public function onPreSerialize(PreSerializeEvent $event)
{
    $object = $event->getObject();
    $type = $event->getType();

    // If the set type name is not an actual class, but a faked type for which a custom handler exists, we do not
    // modify it with this subscriber. Also, we forgo autoloading here as an instance of this type is already created,
    // so it must be loaded if its a real class.
    $virtualType = ! class_exists($type['name'], false);

    if ($object instanceof PersistentCollection) {
        if ( ! $virtualType) {
            $event->setType('ArrayCollection');
        }

        return;
    }

    if ( ! $object instanceof Proxy && ! $object instanceof ORMProxy) {
        return;
    }

     //$object->__load(); Just comment this out

    if ( ! $virtualType) {
        $event->setType(get_parent_class($object));
    }
}

Arlynearlynne

Thanks. This was helpful. However PersistentCollection objects are still being loaded. Any solution to disable this ? – Allocution

@nkobber, Hi I too am facing same problem, did you find solution to this ? – Manumit

Answer 3

This may very well be called an ugly crutch, but you could just select() the data that you really need, then hydrate the result to an array using the getArrayResult() method of the Query object...

Simplex

We tried this method at first. But we have a pretty large DB Model and use Doctrines Mapped Superclass. So this simple Method works as long as we dont select multiple Entities which have equal named fields. For Example: We are using Doctrines "Version" Field, which exists in nearly every Entity. I would have to manually use "select as" entityname_version which is - as you said - an ugly crutch :( – Gascony

I'm not sure, but won't it actually return an array of arrays, where each second-level array represents a hydrated object?.. Which means there won't be any field name clashes if you select by tablename.* – Simplex

Doesn't the data of associated entities get stored in a separate array, which is, in turn, stored under the approriate key in the array representing the object cobntaining the association (although I'm not too sure about bidirectional associations). – Simplex

Oh, and I meant selecting "entityName1, entityName2 etc." , not "entityName1.*, entityName2.*". Where entityName2 may actually be entityName1.someAssociatedEntity1 – Simplex

Although I don't actually remember if the resulting return value will differ based on whether you select entityName.* - or just entityName... – Simplex

In fact, im using some kind of JMS Serializer, which is (and thats a pity) not capable of putting the "Entities name." in front of the field. "TCompany.Version" field is getting reduced to "Version" :-/ – Gascony

Do you actually need to use that serializer when you're working with arrays, not with entities? – Simplex

thx to you too. Yes, i need the serializer cause i want JSON - and the PHP json_encode() method is useless since it wont work with private/protected doctrine properties. Anyway, i recommend everyone not to use Doctrine when there is JSON serialization needed. – Gascony

err...it seems I'm missing something. If you're hydrating the result to an array, where will the private properties come from when you're serializing the array to JSON?.. – Simplex

You are right too. I tried it and the hydrated array just contains the tables i joined manually. So we could dispose of JMS Serializer altogether. How could i be so blind? – Gascony

the other drawback of hydrating to an array is if you're also using a serialization subscriber. So far, I've not found a way to configure one of those to hook into an array, only an object. – Epigraphy

Answer 4

When using Doctrine's query builder, you can't disable lazy loading of linked model classes. If you want to bypass such behavior, you better have to request data with Doctrine's DBAL.

Don't use \Doctrine\ORM\QueryBuilder but \Doctrine\DBAL\Query\QueryBuilder.

$qb = new QueryBuilder($this->_em->getConnection());
$expr = $qb->expr();

$qb->select('pa.*', 't.*', 'c.*', 'a.*', 'aps.*', 'apt.*', 'p.*')
   ->from('person_appointment', 'pa')
   ->leftJoin('pa', 'table', 't', $expr->eq('pa.table_id', 't.table_id'))
   // put other joints here
   // ...
   ->leftjoin('a', 'person', 'p', $expr->eq('a.person_id', 'p.person_id'));

Dessalines

Hi! Thanks, this one looked good at first - but it seems that the DBAL qb is just capable of building SQL statements that i could execute with PDO afterwards.. All ORM functionality is missing. – Gascony

The problem is that what you want is to ignore ORM functionalities! You can't load a model class with Doctrine ORM without loading related model classes. – Dessalines

Maybe im approaching the problem from the wrong side, and should start modifying the serializer. But the Serializer cant know if the association it calls is allready there or lazy loaded when called. My wish would be that an association that was not mentioned in a JOIN is just NULL instead of being lazy loaded.. – Gascony

I awarded your answer with the bounty since you made one point clear: No Doctrine ORM without Lazy Loading. thx – Gascony

Thanks. I think you don't approach your problem the right way. Class loading isn't the issue here. The performance drop is pretty small: if you load 10 classes more it's nothing compared to the 50 (maybe more) loaded by Doctrine or even your application framework. – Dessalines

chuber50: "No Doctrine ORM without Lazy Loading" - not true. doctrine supports both lazy loading and eager loading, by default lazy loading – Intramuscular

Answer 5

Case you want pragmatically use your or default subscriber,

@DavidLin answer:

you can copy the class from JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber to Your\Bundle\Event\DoctrineProxySubscriber and comment out the $object->__load(); line

<?php

/*
 * Copyright 2013 Johannes M. Schmitt <[email protected]>
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

namespace Your\Bundle\Event;

use Doctrine\ORM\PersistentCollection;
use Doctrine\ODM\MongoDB\PersistentCollection as MongoDBPersistentCollection;
use Doctrine\ODM\PHPCR\PersistentCollection as PHPCRPersistentCollection;
use Doctrine\Common\Persistence\Proxy;
use Doctrine\ORM\Proxy\Proxy as ORMProxy;
use JMS\Serializer\EventDispatcher\PreSerializeEvent;
use JMS\Serializer\EventDispatcher\EventSubscriberInterface;

class AvoidDoctrineProxySubscriber implements EventSubscriberInterface
{
    public function onPreSerialize(PreSerializeEvent $event)
    {
        $object = $event->getObject();
        $type = $event->getType();

        // If the set type name is not an actual class, but a faked type for which a custom handler exists, we do not
        // modify it with this subscriber. Also, we forgo autoloading here as an instance of this type is already created,
        // so it must be loaded if its a real class.
        $virtualType = ! class_exists($type['name'], false);

        if ($object instanceof PersistentCollection
            || $object instanceof MongoDBPersistentCollection
            || $object instanceof PHPCRPersistentCollection
        ) {
            if ( ! $virtualType) {
                $event->setType('ArrayCollection');
            }

            return;
        }

        if ( ! $object instanceof Proxy && ! $object instanceof ORMProxy) {
            return;
        }


        //Avoiding doctrine lazy load proxyes
        //$object->__load();

        if ( ! $virtualType) {
            $event->setType(get_parent_class($object));
        }
    }

    public static function getSubscribedEvents()
    {
        return array(
            array('event' => 'serializer.pre_serialize', 'method' => 'onPreSerialize'),
        );
    }
}

And initialize the serialize like this:

$serializer = JMS\Serializer\SerializerBuilder::create()
    //remove this to use lazy loading 
    ->configureListeners(function(JMS\Serializer\EventDispatcher\EventDispatcher $dispatcher) {
        $dispatcher->addSubscriber(new Your\Bundle\Event\AvoidDoctrineProxySubscriber());
    })  
    // !remove this to use lazy loading 
    ->build();

//and serialize the data with/without Lazy

serializer->serialize($data, 'json');

Boric