Tridion and PDF (high-volume) content management

Question

We have 5,000 PDFs that should, in total, be no larger than 200gb. They are likely to be required to be updated throughout the year in batches of around a 1,000.

As I see, there are two main routes...

1) Publish the PDF and associated metadata through Tridion 2) Import directly into the delivery environment and manage the PDF metadata in Tridion

A compelling (business) reason for putting these PDFs through the CMS is the route to getting them on production - CMS=Easy - non-CMS=Not easy at all and the control it gives directly to the business.

We certainly would prefer to manage metadata directly associated with the binary item and also take advantage of component linking (to track where used etc.) rather than mapping components (for the metadata) with "links" to a non-CMS controlled binary item - so it seems to me through the CMS would make more sense.

Now - there's the question of bloating the database / blocking the publishing queue...

Some of these items may need go through workflow (if we batch upload through WebDAV I presume we could define specific cartridges for specific folders and therefore associate different schema?). However - using WebDAV would presumably mean the PDFs (and historical versions of) would be stored in the database which could be problematic.

So... we could link these in Tridion as external link components but I presume this would then mean we couldn't use WebDAV (or could we still use WebDAV with externally_linked PDFs - seems like it doesn't make sense?)

I'm sure large quantities of binaries being managed in (or around) the CMS is something many of us have come across and would be very interested in hearing how others have approached this dilemma?

Thanks

Jornada

Answer 1

Just to answer your point:

So... we could link these in Tridion as external link components but I presume this would then mean we couldn't use WebDAV (or could we still use WebDAV with externally_linked PDFs - seems like it doesn't make sense?)

I'm not sure if this is possible with WebDAV but if you do decide to go down the external multimedia route you could write a simple application to create the multimedia components based on the storage directory/directories for the pdf files.

I have seen an implementation where a specific publication was used to just allow users to put the files and publish through Tridion. The publication published to a shared area, that published shared folder, was mapped into the required presentation websites (virtual directory in IIS).... a custom schema field was used to help users select how to embed the pdfs into content. I know, this is a VERY strange solution, but it solved a lot of problems in terms of, easy security and publishing configuration and the pdfs were not replicated in the db via blueprinting / localization.

Duque

Thanks John - we'll talk more on this approach and how it could be considered and I'll update the post after that. – Jornada

Answer 2

I think you (or someone in your team) must have asked this in the SDL Tridion Forum. If it is not, it is a giant coincidence, and take a look there at the suggestions.

I will let others chime in with their experiences, but if you want to use WebDAV it will always use the uploaded version of a Multimedia Component. You should think of External Binaries as a read only source which the CMS has no control over.

As I said in the other forum, I would suggest that you stick with uploading them to the CMS if you need workflow and to manage user permissions on who can edit the binaries. If you were talking Terra-bytes I would be more concerned, but I don't anticipate a big problem for your data size.

Consider running the purge tool quite often to delete very old versions of the binaries, and make sure to allow some time when publishing them all. Also publish them in small batches, as publication transaction over 500Mb are notoriously memory intensive.

Ursine

Cheers Chris. I'd deliberately added the post (updated) here to share the discussion in a more open forum and extended on that discussion. The biggest business requirement is to enable them to be uploaded in batches rather than the current method of one at a time - which is why I lean towards WebDAV where I can as it's out of the box (and enabled) functionality. Some good points to consider though. thanks. – Jornada