Getting id of associated child records in factory_boy

Question

I have a function with a number of parameters, then a specialized instantiation of that function, with some settings for each of the function's parameters. So I have a structure like the following:

class Function(models.Model):
    name = models.CharField()

class FunctionParameter(models.Model):
    function = models.ForeignKey(Function)

class FunctionInstantiation(models.Model):
    function = models.ForeignKey(Function)

class ParameterSetting(models.Model):
    function_instantiation = models.ForeignKey(FunctionInstantiation)
    function_parameter = models.ForeignKey(FunctionParameter)

In FunctionFactory I can use factory.RelatedFactory to create the parameters.

But in FunctionInstantiationFactory I can't use factory.RelatedFactory(ParameterSetting) to create ParameterSettings, because I don't have access to the parameter objects created within FunctionFactory, so I can't set parameter_setting.function_parameter_id.

How can FunctionInstantiationFactory look up the parameter_id of parameters created in FunctionFactory? Can I get at them from the return value of RelatedFactory(FunctionFactory)? Or do I need to look at the database?

Doublehung

Answer 1

This is Xelnor's answer, but fixes the bug so that only one function_instantiation is created, rather than one for each parameter/parameter_setting pair.

class FunctionFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.Function
    name = factory.Sequence(lambda n: "Function %d" % n)


class FunctionParameterFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.FunctionParameter
    function = factory.SubFactory(FunctionFactory)


class FunctionInstantiationFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.FunctionInstantiation
    function = factory.SubFactory(FunctionFactory)


class ParameterSettingFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.ParameterSetting

    function_instantiation = factory.SubFactory(FunctionInstantiationFactory)
    function_parameter = factory.SubFactory(FunctionParameterFactory,
        function=factory.SelfAttribute('..function_instantiation.function'))


class FunctionToParameterSettingsFactory(FunctionInstantiationFactory):
    class Meta:
        model = models.FunctionInstantiation

    # This overrides the function_instantiation created inside
    # ParameterSettingFactory, which then overrides the Function creation,
    # with the SelfAttribute('..function_instantiation.function') syntax.
    parameter_setting_1 = factory.RelatedFactory(ParameterSettingFactory, 
        'function_instantiation')
    parameter_setting_2 = factory.RelatedFactory(ParameterSettingFactory, 
        'function_instantiation')

The following demonstrates the solutions to a few other problems anyone using this pattern will probably encounter, such as overriding related objects' values, and links to other tables, themselves linked. It draws largely from techniques Xelnor introduced in his answer.

class FunctionFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.Function
    name = factory.Sequence(lambda n: "Function %d" % n)


class FunctionParameterFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.FunctionParameter
    name = factory.Sequence(lambda n: "Function %d" % n)
    function = factory.SubFactory(FunctionFactory)


class ParameterSettingFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.ParameterSetting
    name = factory.Sequence(lambda n: "Function %d" % n)

    function_instantiation = factory.SubFactory(FunctionInstantiationFactory)
    function_parameter = factory.SubFactory(FunctionParameterFactory,
        function=factory.SelfAttribute('..function_instantiation.function'))


class DatasetAnd2ColumnsFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.Function
    dataset = factory.SubFactory(DatasetFactory,
        name=factory.Sequence(lambda n: "Custom dataset %d" % n))
    column_1 = factory.SubFactory(ColumnFactory, dataset=dataset,
        name=factory.Sequence(lambda n: "Column 1 %d" % n))
    column_2 = factory.SubFactory(ColumnFactory, dataset=dataset,
        name=factory.Sequence(lambda n: "Column 2 %d" % n))


# I found it neater not to inherit in the end, due to needing quite a lot of
# additional complexity not included in my original question.
class FunctionToParameterSettingsFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.FunctionInstantiation

    name = factory.Sequence(lambda n: "Custom instantiation name %d" % n)
    # You can call Sequence to pass values to SubFactories
    function = factory.SubFactory(FunctionFactory, 
        name=factory.Sequence(lambda n: "Custom function %d" % n))

    parameter_setting_1 = factory.RelatedFactory(ParameterSettingFactory, 
        'function_instantiation',
        # Note the __ syntax for override values for nested objects:
        parameter__name='Parameter 1',
        name='Parameter Setting 1')
    # Possible to use Sequence here too, and makes looking at data easier
    parameter_setting_2 = factory.RelatedFactory(ParameterSettingFactory, 
        'function_instantiation',
        parameter__name=factory.Sequence(lambda n: "Param 1 for fn %d" % n),
        name=factory.Sequence(lambda n: "Param Setting 1 for fn %d" % n))

I now need to create a dataset with some columns of data, and join the parameter_setting records with those columns. To do so, this goes at the end of FunctionToParameterSettingsFactory:

@factory.post_generation
def post(self, create, extracted, **kwargs):
    if not create:
         return

    dataset = DatasetAnd2ColumnsFactory()
    column_ids_by_name = 
        dict((column.name, column.id) for column in dataset.column_set.all())

    # self is the `FunctioInstantiation` Django object just created by the `FunctionToParameterSettingsFactory`
    for parameter_setting in self.parametersetting_set.all():
        if parameter_setting.name == 'age_in':
            parameter_setting.column_id = column_ids_by_name['Age']
            parameter_setting.save()
        elif parameter_setting.name == 'income_in':
            parameter_setting.column_id = column_ids_by_name['Income']
            parameter_setting.save()

This is admittedly a bit hacky. I tried passing column=column_1 in the RelatedFactory calls, but that triggered creation of multiple datasets, each column linked to a different one. I tried all sorts of acrobatics with SelfAttribute and LazyAttribute, but you can't use either in a RelatedFactory call, and you can't create something with SubFactory(SelfAttribute()) and then pass it into RelatedFactory, as that breaks SelfAttribute (see my other question).

In my real code I had several more models with a foreign key to dataset and it all tied up fine.

Doublehung

My edits were rejected, sorry @Xelnor, I tried to keep the answer credit with you. – Doublehung

There is a bug in the last code sample. dataset is repeatedly created so column_1 and column_2 get different datasets (both with the overridden name field). If I create a column_3 with dataset=factory.LazyAttribute(lambda col: col.factory_parent.dataset), it works fine. If I then add column_3 to a ParameterSettingFactory call, I get AttributeError: The parameter dataset is unknown. and factory_boy finds only ParameterSettingFactory locals, and no factory_parent. – Doublehung

I've solved the above issue, described here, and updated the answer. – Doublehung

Answer 2

The factory.SubFactory is intended to follow a ForeignKey; if you want to use it the other way around, you should use a RelatedFactory instead.

For your example, I'd go with the following factories:

class FunctionFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.Function
    name = factory.Sequence(lambda n: "Function %d" % n)


class FunctionParameterFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.FunctionParameter
    function = factory.SubFactory(FunctionFactory)


class FunctionInstantiationFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.FunctionInstantiation
    function = factory.SubFactory(FunctionFactory)


class ParameterSettingFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = models.ParameterSetting
        exclude = ['function']

    # We'll need a FunctionFactory; this field is part of 'exclude',
    # thus available while building the factory but not passed to the
    # target Django model
    function = factory.SubFactory(FunctionFactory)

    # Use the function from our Factory for both
    # function_instantiation and function_parameter
    function_instantiation = factory.SubFactory(FunctionInstantiationFactory,
        function=factory.SelfAttribute('..function'))
    function_parameter = factory.SubFactory(FunctionParameterFactory,
        function=factory.SelfAttribute('..function'))

And you can add an extra factory, FunctionWithParametersFactory, that creates parameters along:

class FunctionWithParametersFactory(FunctionFactory):
    parameter1 = factory.RelatedFactory(ParameterSettingFactory, 'function')
    parameter2 = factory.RelatedFactory(ParameterSettingFactory, 'function')

Calling that factory will perform the following:

1. Create a Function object (through FunctionFactory)
2. Call ParameterSettingFactory, pointing it to the created Function object
3. Call ParameterSettingFactory a second time, still pointing it to the same Function object
4. Return that Function object.

Esperanzaespial

Sorry, just seconds ago I updated my question with RelatedFactory instead of SubFactory, while you were typing. Was just reading up on SelfAttribute but was hopelessly stuck along the lines of function.parameter[0].id. Will take me a few minutes to understand your code so bear with me before I mark as the answer. Huge thanks, as you can tell I'm new to factory_boy and its ilk. – Doublehung

Some notes, self-evident to anyone familiar with factory_boy. The factory.SelfAttribute('.. syntax tells the SubFactory (FunctionInstantiationFactory) to get the named object (function) from the calling factory (ParameterSettingFactory) scope. exclude stops factory_boy from trying to save function as a foreign key on the ParameterSetting object (which doesn't have that FK). FunctionWithParametersFactory, by inheriting from FunctionFactory, starts by creating Function, then uses ParameterSettingFactory to create the other three. Huge thanks @Esperanzaespial this taught me a lot. – Doublehung

From the docs I think function=factory.LazyAttribute(lambda function_parameter: function_parameter.factory_parent.function) would be an alternative (inferior) way of implementing the second function=factory.SelfAttribute('..function'). @Xelnor's solution is nicer and neater in every way, just noting a related discovery. – Doublehung