Connect to the renderfarm

The tutorial shows you how to integrate das element with your render farm.
In this example AWS Deadline is used. The concept however applies to every other render farm manager as well.

Watch this video tutorial to get started!

Follow these steps

• create shared resources folder on the server

  • add environment variable DASELEMENT_RESOURCES to point to that shared folder

• add pre & post render hooks (needed for the job dependencies)

• adjust transcoding templates

  • copy and paste the "executable" and "parameters" from the example files
    can be found on the top of each Python files

• add new Path Values for the width and height

Example files

The latest example files for renderfarm jobs can be found here:

https://github.com/das-element/resources

Python hooks

For each transcoding task, the Python hook for ‘pre_render’ (pre_render.py) and ‘post render' (post_render.py) gets the Job ID from the render farm. If the main task isn’t finished then there is nothing to render for the proxy tasks and they will probably fail.

It is why it is important to set a dependencies on the proxy tasks to the main task.

Pre Render Hook

The input is the resolver data dictionary which will be used to resolve the path pattern. You can modify it here before the transcoding task gets processed.

You will need to return the same dictionary with your changes included.

In the example below, in order to resolve a path pattern like <custom.dependecy> you need to add some value for the custom data. If that is not provided the resolve would otherwise fail if you re-render the proxies and there is no main task to provide the dependency which we normally would get from the post render hook after sending the task to the render farm.

Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/hooks/examples/deadline/pre_render.py

Post Render Hook

The input is the output of the process called by the custom command task.

You will have to return a Dictionary which will be added as custom to the resolver data and can later be accessed in the Path Builder. return {'dependency': job_id} can later be resolved with <custom.dependency>

Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/hooks/examples/deadline/post_render.py

Transcoding tasks

Examples for Nuke

Here are some example scripts and templates on how to use The Foundry Nuke for transcoding elements.

Check out the example script here:
https://github.com/das-element/resources/tree/main/scripts/custom/examples/nuke

Transcoding task: main element

Simple command line

# exec:
# Linux:
cp
# Windows:
copy

params:
\"<source.path>\" \"<path>\"

Python script

The source file(s) can be a single file, like a single frame or a movie file or a sequence of files.
That makes stuff a little bit more complicated if you want to have a generic file to deal with all these cases.

Find below an example script which was tested on Linux and Windows with Deadline.
Please make sure to install the fileseq Python package. ❤ Thank you to the developers!

Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/copy_main.py

Transcoding task: thumbnail

In this example we use FFmpeg to create the thumbnail.

Simple command line

It shows you a way how to do it without any custom scripting. However this might be not very practical in production, be cause e need to create two transcoding tasks. One for single image files and another for movies/sequences. They have different command lines because movies/sequences need to know which frame is the thumbnail.

# exec:
/opt/Thinkbox/Deadline10/bin/deadlinecommand

# params:
# command line for single images
-SubmitCommandLineJob -executable "/usr/bin/ffmpeg" -arguments "-i "<path>" -y -vframes 1 -an -s <width>x<height> "<path_thumbnail>"" -frames 1 -chunksize 1 -priority 50 -name "[das element] <name> - thumbnail image" -prop BatchName="[das element] <name>" -prop JobDependencies="<custom.dependency>"
# command line for movie files/sequences  (the additional time flag (-ss) is defined)
-SubmitCommandLineJob -executable "/usr/bin/ffmpeg" -arguments "-i "<path>" -y -vframes 1 -an -s <width>x<height> -ss <frame_first> "<path_thumbnail>"" -frames 1 -chunksize 1 -priority 50 -name "[das element] <name> - thumbnail movie/sequence" -prop BatchName="[das element] <name>" -prop JobDependencies="<custom.dependency>"

Python script

Creating proxy thumbnail can also be done with a custom python script. FFmpeg is used to render the proxy files in this example.

There is a parameter that describes the type of media (<media_type>) this is either image, movie or sequence. This helps you to determine how to deal with the different file paths types. It can be included in the Custom Commands Parameters.

Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/create_thumbnail.py

Transcoding task: proxy movie

In this example we use FFmpeg to create proxy movie file.

Simple command line

# exec:
/opt/Thinkbox/Deadline10/bin/deadlinecommand

# params:
-SubmitCommandLineJob -executable "/usr/bin/ffmpeg" -arguments "-i \"<path>\" -y -r 24 -vf \"scale=<width>:<height>:force_original_aspect_ratio=decrease,pad=<width>:<height>:(ow-iw)/2:(oh-ih)/2\" -vcodec libx264 -crf 23 -preset faster -tune film -pix_fmt yuv420p -framerate 24 -timecode 00:00:41:17 -acodec copy \"<path_proxy>\"" -frames 1 -chunksize 1 -priority 50 -name "[das element] <name> - proxy movie" -prop BatchName="[das element] <name>" -prop JobDependencies="<custom.dependency>"

Python script

Creating proxy movie files can also be done with a custom python script. FFmpeg is used to render the proxy files in this example.

There is a parameter that describes the type of media (<media_type>) this is either image, movie or sequence. This helps you to determine how to deal with the different file paths types. It can be included in the Custom Commands Parameters.

Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/create_proxy.py

Transcoding task: filmstrip

In this example we use FFmpeg to create a filmstrip for movie files and sequences of images.

Python script

This can be done with a custom python script. FFmpeg is used to render the filmstrip in this example.

There is a parameter that describes the type of media (<media_type>) this is either image, movie or sequence. This helps you to determine how to deal with the different file paths types. It can be included in the Custom Commands Parameters.

Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/create_filmstrip.py

List of useful tags

The keys get resolved during ingest/proxy generation time and will be replaced with the actual value.
You can use the post render hook to add custom values.