Following the release of Shotgun’s new descriptor workflow along with its Bootstrap API I’ve finally got round to updating our pipeline to take advantage of these new features. It has been a big improvement over the old Toolkit pipeline configuration deployment approach and has definitely aided our development and deployment processes.

There has been one hurdle we’ve had to overcome; when bootstrapping in to a Shotgun environment on rendernodes with multiple slaves, it appears that the bootstrap process fails if there are concurrent threads all trying to copy the pipeline config to a shared location.

Thankfully there was a simple solution which involves setting up a custom GlobalJobPreLoad.py which appears to have resolved the issue.

  1. Create a file called GlobalJobPreLoad.py and save it to your deadline repos /custom/plugins directory. This file will be run before every job that’s loaded by each slave.
  2. Copy the following code in to the GlobalJobPreLoad.py file :


from System.IO import *
from Deadline.Events import *
from Deadline.Scripting import *
import os
def set_slave_specific_shotgun_home_path(deadlinePlugin):
'''
In situations where you have multiple deadline Slaves on a single rendernode
Shotgun may fail to bootstrap a pipelineconfig if more than one slave
attempts to bootstrap at the same time. This is because by default each
slave will use the same location to store it's shotgun pipeline configuration so
when more than one slave is bootstrapping, there may be two processes attempting to
copy files to the same location at the same time, thus causing an error.
Setting a slave-specific directory per slave prevents this clash from occurring.
:param deadlinePlugin: The deadlinePlugin class passed from __main__
:return: None
'''
deadlinePlugin.LogInfo(". Setting-up environment")
slavename = deadlinePlugin.GetSlaveName()
deadlinePlugin.SetProcessEnvironmentVariable("DEADLINE_SLAVENAME", slavename)
SHOTGUN_HOME = os.path.join('c:\\', 'studio','shotgun','cache',slavename)
deadlinePlugin.LogInfo(". setting SHOTGUN_HOME : %s" % SHOTGUN_HOME)
deadlinePlugin.SetProcessEnvironmentVariable("SHOTGUN_HOME", SHOTGUN_HOME)
def __main__(deadlinePlugin):
# Set SHOTGUN_HOME environment variable to slave-specific location
set_slave_specific_shotgun_home_path(deadlinePlugin)

What this script does is set a slave-specific SHOTGUN_HOME location(in this case c:\your_studio\shotgun\cache\slaveName ensuring that each slave can bootstrap in to its own folder. Problem solved!

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Skip to toolbar