django: guidelines for speeding up template rendering performance [closed]

Question

How would I go about speeding up Django template rendering? My template takes about 1-2 seconds or so to render, after the view function fully computes whatever it needs to.

I've already attempted to perform all database access in the view, such that the template only hits RAM and not the DB engine.

I do have a lot of includes - could there be an issue there?

Durian

Answer 1

I just spent a good deal of time optimizing my django templating code. Below are optimization guidelines that worked for me, but depending on your setup, you may not get as significant of a speedup.

• Mark Variables as safe: Django's automatic security measures are really nice, but they do come with a small performance hit. If you're using many variables in your template, and you know that it's safe, then be sure to mark it as such.
• Cache Compiled Templates: When you call render_template_from_string, django pulls the template, compiles it, and then renders it. Django makes it easy to cache the first two parts of this process and store them in memory. All you need to do is make one small change in your settings.py file to add cached.Loader to your TEMPLATE_LOADERS. More is explained in the Django documentation.
• Fix or Don't Use endless pagination: I found that the endless pagination plugin slowed things down extremely. That's because it has to load a new template for every single page number. I went in and hacked at it until I got it doing what I wanted, but before you do that, try removing it and see what type of performance improvement you get.
• Don't Use Fancy Templating Features: Inheritance, nested blocks, for loops, and includes are great, but they come with a performance cost. Be very careful when using them and try not to do multiple levels of inheritance. For loops may be better handled with client side rendering.
• Client Side Rendering: An easy trick to speed up server side rendering is to just do it on the client. One strategy is to embed a json structure in the django template, and then have javascript build the HTML. This obviously defeats the purpose of django server side rendering, but it will result in a speedup. This is especially useful if you have to render content below the fold (i.e., the user does not need to see it immediately when the page loads).

Doing the above cut my rendering time of a complex page on a GAE instance from about 1.0S to 250ms, but again, your mileage may vary.

Unseat

+1 for removing includes. I used includes inside for loops. Removing them resulted in huge speed up of rendering time (close to 2 times faster) – Neuroma

In the current version of Django (1.10 as of this writing), cached.Loader will be enabled automatically when DEBUG is False. – Swallowtail

Do you have more info on how to fix endless pagination (like an issue or the reason) – Palp

The problem is that endless pagination puts an include statement within a for statement, which causes lots of loading and of templates and can be quite slow. Can't remember how I fixed it. – Unseat

What is the django alternative to Don't Use Fancy Templating Features? I suspect that you intended Client Side Rendering to dovetail with the previous recommendation. – Lowercase

Enabling 'django.template.loaders.cached.Loader' in TEMPLATE -> loaders helped my case quite a lot. Thanks for the optimization tips. – Lymphoma

Answer 2

I would recommend as Step 0 adding to your Django Debug Toolbar an extra panel called Django Toolbar Template Timings. It told me exactly how much time was being spent in each template (block, etc), and how much of that time is in SQL. It's also extra validation that this is your problem.

Here's how to add a panel to Debug Toolbar. http://django-debug-toolbar.readthedocs.org/en/latest/configuration.html#debug-toolbar-panels

Choosy

Unfortunately this package (and all others recommended for this purpose) is no longer maintained. I am looking for an alternative. – Expostulate

@Expostulate any updates on finding an alternative? – Autunite

This one is nice, but is not completely the same : pypi.org/project/django-debug-toolbar-template-profiler – Tieratierce

Answer 3

Use ManifestStaticFilesStorage to serve your static files. The performance boost I've witnessed relative to using CachedStaticFilesStorage with the default LocMemCache is immense. The difference being no hashes ever need to be calculated at runtime.

I don't quite know why the difference is as huge as it is - while it's true that CachedStaticFilesStorage would initially need to calculate hashes and fill the cache, once the cache is filled I wouldn't expect a significant performance penalty relative to the manifest method. But it is massive, and the documentation also recommends using ManifestStaticFilesStorage for performance.

Quillan

Answer 4

Another trick I found: I had to display an HTML table of 755 rows * 15 columns (filling a total of 11,325 data).

That page used to delay for about 25/30 seconds loading and then the page was a bit laggy. What I did was setting the table with display:none and after the page was fully loaded, changed the CSS property with JavaScript.

After all of that, the page is loading in max 6 seconds. I suppose that Django spends much less time rendering non-visible elements.

I do not know if it only works in my case, but it seems to be.

Ostwald

Gave this a shot, it was an interesting idea, but didn't seem to do anything for me. – Broadcaster

"I suppose that Django spends much less time rendering non-visible elements." Using CSS like that does not make any difference on the Django side. What you are speeding up is the rendering on the browser side. – Yearning

Answer 5

If you're looking to reduce load time in your localhost then for most people using cached.Loader will significantly cut the load time.

Note: you cant use APP_DIRS: True when loaders are defined.

By default (when DEBUG is True), the template system reads and compiles your templates every time they’re rendered. While the Django template system is quite fast, the overhead from reading and compiling templates can add up.

You configure the cached template loader with a list of other loaders that it should wrap. The wrapped loaders are used to locate unknown templates when they’re first encountered. The cached loader then stores the compiled Template in memory. The cached Template instance is returned for subsequent requests to load the same template.

This loader is automatically enabled if OPTIONS['loaders'] isn’t specified and OPTIONS['debug'] is False (the latter option defaults to the value of DEBUG).

Add this to your TEMPLATES['OPTIONS'].

"loaders": [
    (
        "django.template.loaders.cached.Loader",
        [
            "django.template.loaders.filesystem.Loader",
            "django.template.loaders.app_directories.Loader",
        ],
    ),
],

Now your TEMPLATES settings will look like this.

TEMPLATES = [{
    'BACKEND': 'django.template.backends.django.DjangoTemplates',
    'DIRS': [BASE_DIR / 'templates'],
    'APP_DIRS': False,
    ...
    'OPTIONS': {
        'context_processors': [...]
        'loaders': [
            ('django.template.loaders.cached.Loader', [
                'django.template.loaders.filesystem.Loader',
                'django.template.loaders.app_directories.Loader',
                'path.to.custom.Loader',
            ]),
        ],
    },
}]

---

btw this option is already mentioned with some other options in the speedplane answer.

Biquarterly

Answer 6

An useful tool to profile your django templates is the django-extension library - RunProfileServer.

This profiler will capture calls made inside your django templates which will help you identify the bottlenecks.

If you have installed django-extensions library, you can run:

python3 manage.py runprofileserver --use-cprofile --prof-path=/tmp/results/

for a profiling version of the local server.

Responding to requests will now generate .prof files in your directory that you can visualize using snakeviz or other visualizing packages.

Example using snakeviz:

Novah