Following the release of Shotgun’s new descriptor workflow along with its Bootstrap API I’ve finally got round to updating our pipeline to take advantage of these new features. It has been a big improvement over the old Toolkit pipeline configuration deployment approach and has definitely aided our development and deployment processes.
There has been one hurdle we’ve had to overcome; when bootstrapping in to a Shotgun environment on rendernodes with multiple slaves, it appears that the bootstrap process fails if there are concurrent threads all trying to copy the pipeline config to a shared location.
Thankfully there was a simple solution which involves setting up a custom GlobalJobPreLoad.py which appears to have resolved the issue.
- Create a file called GlobalJobPreLoad.py and save it to your deadline repos /custom/plugins directory. This file will be run before every job that’s loaded by each slave.
- Copy the following code in to the GlobalJobPreLoad.py file :
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from System.IO import * | |
| from Deadline.Events import * | |
| from Deadline.Scripting import * | |
| import os | |
| def set_slave_specific_shotgun_home_path(deadlinePlugin): | |
| ''' | |
| In situations where you have multiple deadline Slaves on a single rendernode | |
| Shotgun may fail to bootstrap a pipelineconfig if more than one slave | |
| attempts to bootstrap at the same time. This is because by default each | |
| slave will use the same location to store it's shotgun pipeline configuration so | |
| when more than one slave is bootstrapping, there may be two processes attempting to | |
| copy files to the same location at the same time, thus causing an error. | |
| Setting a slave-specific directory per slave prevents this clash from occurring. | |
| :param deadlinePlugin: The deadlinePlugin class passed from __main__ | |
| :return: None | |
| ''' | |
| deadlinePlugin.LogInfo(". Setting-up environment") | |
| slavename = deadlinePlugin.GetSlaveName() | |
| deadlinePlugin.SetProcessEnvironmentVariable("DEADLINE_SLAVENAME", slavename) | |
| SHOTGUN_HOME = os.path.join('c:\\', 'studio','shotgun','cache',slavename) | |
| deadlinePlugin.LogInfo(". setting SHOTGUN_HOME : %s" % SHOTGUN_HOME) | |
| deadlinePlugin.SetProcessEnvironmentVariable("SHOTGUN_HOME", SHOTGUN_HOME) | |
| def __main__(deadlinePlugin): | |
| # Set SHOTGUN_HOME environment variable to slave-specific location | |
| set_slave_specific_shotgun_home_path(deadlinePlugin) |
What this script does is set a slave-specific SHOTGUN_HOME location(in this case c:\your_studio\shotgun\cache\slaveName ensuring that each slave can bootstrap in to its own folder. Problem solved!