Render Farm Distribution with Distributed Rendering

I manage a small render farm that uses Thinkbox’s Deadline for render management and primarily VRay for Maya for most significant renders. We are adjusting our render farm to include VRay distributed rendering slaves. The techniques I’m discussing in this article should work with any distributed rendering slave that does not mind the slaves being interrupted mid-render, and any render farm software that allows for job interruption based on priority (for the Loose option).

Note that none of these techniques will speed up any renders when the farm is very busy. We will still get the same number of renders in the same amount of time. There is a slight advantage for Limited in getting a smaller subset more quickly, which I will discuss later.

I have thought through two methods of handling distributed rendering on a farm, and I have labeled them Limited and Loose. I will explain the difference between the two and why I believe Limited is better for most situations.

Let’s first understand the base case without using distributed rendering at all. In the farm, each render node will receive a job and render it by itself. It does not ask for help from any other render nodes, regardless of how busy (or not busy) the farm is. This means if we have 12 render nodes, we are at peak performance only if there are at least 12 jobs to be rendered. If there are less than 12 jobs, the effectiveness drops directly with the number of nodes not being used. Only 3 jobs to render? 9 of our render nodes are ineffective.

Now to add distributed rendering! Adding distributed rendering is always an improvement, and will never slow down render times (excluding a small bit of overhead). The only thing it adds is more complexity to our farm.

The Limited distributed rendering option will label each render node as one of two categories: a Render Master or a Render Slave. The Render Masters will exist in the render farm software, handling scene files as they are received from the users. When a job is received and started, the Render Master will ask for a limited number of Render Slaves to help with the render job (the Render Master can help do the work as well, assuming the software can support it and the machine is powerful enough). This will apply the power of only a section of the farm onto each rendered image. With a Limited farm, we are at peak performance when we have at least as many jobs as we have Render Master nodes. If the farm has less jobs than Render Masters, the efficiency goes down at a rate of (# of unused Render Masters)/(# of all Render Masters).

In the Loose distributed rendering option each render node is considered equal. Every node will be set up to have a low-priority job that is interruptible (by the render farm software). This job will run the a Distributed Rendering Slave (DRS for short). When a job from a user is received and started, whichever node picks up the job will interrupt the DRS job and ask all other render nodes with the DRS job running to help with the user’s job. This will apply the power of all available render nodes onto the rendered image. With a Loose farm, we are at peak performance starting at 1 job, however we lose performance as soon as a busy farm starts to free up.

At first glance, it sounds like Loose is better than Limited. Why wouldn’t we want to always throw all of our rendering power behind every image? There are two problems with Loose and one with Limited. Limited’s problem has already been discussed: only allowing a subsection of the farm to be used on each render. Loose’s problems include: 1) only speeding up the render time of the first user job to get all of the DRS jobs, and 2) having no improvement over a farm without distributed rendering when the farm is very busy.

Loose’s problem 1) can be fixed or avoided if the distributed rendering software will continue to query for helping slaves after the render has started. In my case, VRay does not ask for help after the initial query has been sent. This means that if I sent two jobs to our farm of 12 nodes using Loose, the first job will get 10 helpers and the second job will have to render by itself for the entire duration. I cannot, however, come up with a solution for Loose’s problem 2) because of the nature of the system.

Limited’s problem and Loose’s problem 2) are similarly caused by the nature of their design, but the Limited problem has a slight advantage on a very busy farm: we will get smaller groups of images quicker using Limited than using Loose. If all 12 nodes are busy in a Loose farm, we must wait the full duration of rendering a single image to get our 12 images all at once. If we the same jobs on a Limited farm with 3 Render Masters, each with 3 Render Slaves (12 total), we will get smaller groups of images (3 at a time) in 1/4 the time. This allows for faster feedback for the user, which may result in jobs being canceled and resubmitted sooner.

Therefore, using a Limited distributed rendering system would be the superior of the two options.

Please let me know in the comments if there is an alternative, or there is some advantage that I have missed.

Super Simple Python – Function and Variable Names

One of the best ways to keep your code clean and understandable is to use well defined function and variable names. Take, for example, the following bit of code:

def get_stuff(p, dirs=False):
    return_list = []
    for a, b, c in os.path.walk(p):
        return_list.extend(c)
        if dirs:
            return_list.extend(b)
    return return_list

It takes a lot of effort to figure out what is going on here. We can tell from the function name get_stuff that it’s used to get something, but what is it getting? The arguements p and dirs are very ambiguous. The p argument says nothing about what it is, and the dirs argument implies that it is a list of directories, but it’s actually a boolean! The a, b, and c variables have completely useless names. At least return_list tells us that it’s intended to be returned… though a good comment can do that much clearer!

Here’s the same function re-written with much more understandable function and variable names.

def get_file_names(path, include_directories=False):
    """
    Gets a list of all file names in the given path, including files in subdirectories.
    """
    list_of_file_names = []
    for root, dirs, file_names in os.path.walk(path):
        list_of_file_names.extend(file_names)
        if include_directories:
            list_of_file_names.extend(dirs)
    return list_of_file_names

For those people who are worried about the extra time they’re saving by using short names, think again! If you find a bug in the code or need to modify it in the future, trying to decipher the cryptic function will more than make up the difference in time. In addition, all modern IDEs (and even some text editors) have code completion, letting you only need to type a couple letters to fill out an entire variable name.

Super Simple Python – Input

One of the ways to make your program more interesting is to have some kind of input from the user. There are many ways to have input, including a GUI (or Graphical User Interface), and reading from a file (which we covered in a previous SSP). Another way is to get input directly from the user on the command line using the input() function.

Luckily, input() is one of the easiest ways to get information from the user. We simply assign the function call to a variable, type our message when the prompt comes up, and Python takes care of the rest.

>>> user_input = input()
>? Hello world!
>>> print(user_input)
Hello world!

input() returns whatever the user types into the console as a string. It also takes an optional argument for a text prompt, which lets us tell the user what they should type.

 >>> full_name = input("What is your first and last name?")
What is your first and last name?
>? Frank Sinatra
>>> first_name, last_name = full_name.split()
>>> print("Hello {0}!".format(first_name))
Hello Frank!

This works as long as the user gives valid information. If they also give their middle name, we’re in trouble!

>>> full_name = input("What is your first and last name?")
What is your first and last name?
>? Frank Albert Sinatra
>>> first_name, last_name = full_name.split()
ValueError: too many values to unpack (expected 2)

A common way to account for this is to continue to ask the user for input until they give something that is valid. We can accomplish this by using a while loop.

first_name = ""
last_name = ""
while not (first_name and last_name):
    full_name = input("What is your first and last name?")
    split_name = full_name.split()
    if len(split_name) == 2:
        first_name, last_name = split_name
    else:
        print("Invalid input. Please give your first and last names only!")
print("Hello {0}!".format(first_name))

This will give us some interaction like the following:

What is your first and last name?
>? Frank Albert Sinatra
Invalid input. Please give your first and last names only!
What is your first and last name?
>? Torel Twiddler
Hello Torel!

Super Simple Python – Constants

A constant is a value in your code that doesn’t change during execution. They are best used in places that share values throughout your code, or when someone may change their mind about the value itself.

An example of this would be for some text that you wanted to display in multiple locations throughout your program. Normally, you would just copy the the text everywhere it would be used. That means you need to do a lot of extra typing (or copy-paste a bunch). Even worse, if you need to change the value of it, you will have to find all of the occurrences in your code!

import some_modules

class MyClass:
# A bunch of methods here...
    def do_something(self):
        # ... some code
        print("This is my awesome message!")
        # ... more code

    def do_something_else(self):
        # ... some code
        print("This is my awesome message!")
        # ... more code

There is an easier way!

Constants are identified by being in all caps with underscores between words, and are usually put at the top of your .py file just after the imports. If we replace the strings throughout the code that will be the same value with our constant, we can quickly and easily adjust the message as we like!

import some_modules

INFO_MESSAGE = "This is my awesome message!"

class MyClass:
    # A bunch of methods here...
    def do_something(self):
        # ... some code
        print(INFO_MESSAGE)
        # ... more code

    def do_something_else(self):
        # ... some code
        print(INFO_MESSAGE)
        # ... more code

Now in this example, we only need to change INFO_MESSAGE to update our code. This also works well for values that will never change, due to the nature of the value. For example, we may use a short approximation of pi as 3.14. That way, we don’t have to remember the value of pi each time we use it:

import some_modules

INFO_MESSAGE = "We're using pi!"
PI = 3.14

def get_circumference(radius):
    print(INFO_MESSAGE)
    circumference = 2 * PI * radius
    return circumference

def get_area(radius):
    print(INFO_MESSAGE)
    area = PI * PI * radius
    return area

Super Simple Python – Classes Pt. 2: Methods

Methods are the workhorses of classes. They are where you turn to get things done.

They work just like functions, except they work on the class object. Let’s look at the example from the previous lesson – Point:

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def get_location(self):
        """Returns the location of the point as a tuple."""
        return self.x, self.y

The Point class has two variables, x and y, and one method, get_location().

This get_location() function doesn’t do much here, simply returning the x and y attributes. We could make something a bit more interesting, however:

import math

class Point:
    # ... Fill in the other code here
    def difference(self, other):
        """
        This takes another point (other) and finds the difference
        between this point and the other.

        The formula for the difference (or distance) between
        two points is D = sqrt(dx*dx + dy*dy)
        ref: http://www.mathopenref.com/coorddist.html
        """
        dx = self.x - other.x
        dy = self.y - other.y
        return math.sqrt((dx * dx) + (dy * dy))

The difference method, when executed on a point and given another point, returns the distance between those two points.

>>> a = Point(0, 3)
>>> b = Point(4, 0)
>>> a.difference(b)
5.0

Horray! Pythagoras was a pretty smart, and so are we!

Now let’s try some methods on our Human class as well. (Note: you should probably write this into a file first, rather than directly into the Python console, for sanity.)

class Human:
    def __init__(self, name, hair_color, shoe_size=10):
        # Here we set the attributes passed from creation
        self.name = name
        self.hair_color = hair_color
        self.show_size = shoe_size

        # Here we set the attributes that are the same
        # for each Human
        self.age = 0
        self.position = Point(0, 0)

    def walk(self, dx, dy):
        """Walks in the given dx and dy direction."""
        self.position.x += dx
        self.position.y += dy

    def talk(self, sentence):
        """Speak the given sentence aloud."""
        print(sentence)

    def chew_bubblegum(self):
        """Chew some bubblegum. We assume that
        the Human has some bubblegum already."""
        print("*smack* *smack* *pop*")

Now that we have several methods to play with!

>>> bob = Human("Bob", "brown")
>>> bob.talk("Howdy")
Howdy
>>> bob.chew_bubblegum()
*smack* *smack* *pop*
>>> bob.position.get_location()
(0, 0)
>>> bob.walk(3, 7)
>>> bob.position.get_location()
(3, 7)

Super Simple Python – Classes Part 1: Creation and Attributes

Time for class! Other programming lessons are second class! This lesson is in a class of it’s own!

Ok, I’ll stop.

A class is a programming object that groups together common functions (called methods) and variables (called attributes) that effect and define that object. Here is a very simple class for a point on a grid:

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def get_location(self):
        """Returns the location of the point as a tuple."""
        return self.x, self.y

In our Point class, first we give it a name, then define how to create one using the __init__ method, in which we give it two attributes, x and y. Finally we add an action function that can be used later (the get_location method).

For our example, let’s make a class called a Human. Our Human will have several methods such as walk, talk, and chew_bubblegum. It will also have several attributes such as name, age, hair_color and shoe_size. In this lesson, we’ll deal with creation of the class and it’s attributes.

There are a few things we must do to define a class. First, we define our class with a name:

class Human:

Next, we need to show how our Human is created using the __init__ method (which stands for initialize). The __init__ method is a special function that classes will always run first when creating an instance of the class.

class Human:
    def __init__(self, name, hair_color, shoe_size=10):

In our __init__, we need to initialize all of our attributes (or variables attached to our object). We define __init__ just like any other function, except indented for the class. The variables in the __init__ method will be used to create an object from the class.

There is one extra argument, called self, that gives the method a reference to the object that is being created. In other words, the self variable is a reference to the current instance of the class. self must always be the first argument in a class method.

class Human:
    def __init__(self, name, hair_color, shoe_size=10):
        # Here we set the attributes passed from creation
        self.name = name
        self.hair_color = hair_color
        self.show_size = shoe_size

        # Here we set the attributes that are the same
        # for all Humans
        self.age = 0

Here, we’ve defined our class Human. Now let’s create a couple Humans!

>>> bob = Human("Bob", "brown")
>>> bob
<Human object at 0x02A8EB30>
>>> joe = Human("Joe", "black", shoe_size=13)
>>> joe
<Human object at 0x02A8EF30>

Awesome! We have our Human objects! The hexidecimal number (i.e. 0x02A8EF30) included in with each object shows where the object is in memory, which we tend to not worry about. Now to use the attributes of our objects, we use the format object_name.attribute:

>>> print(bob.name)
Bob
>>> print(bob.shoe_size)
10
>>> print(bob.age)
0
>>> print(joe.name)
Joe
>>> print(joe.shoe_size)
13
>>> print(joe.hair_color)
black

We now have the shell of a Human! It only has attributes so far, so next time we will give our Humans something to do!

Super Simple Python – Objects

You may have heard of the term Object Oriented Programming (OOP) when hearing about some programming languages, such as Python, Java or C++. In this lesson, I’m going to explain what OOP means, what it really means, and what it REALLY really means.

OOP – What it means

Nearly everything in Python (with similar concepts in Java, C++, etc.) is derived from object, the base class for everything. A string, an integer, a dictionary, a function, a class; all are sub-classes of object.

What is object you ask? Well, it’s a thing. Very similarly to how a Widget is an abstract idea of some product in Economics class – an object is the abstract idea of some programming thing.

OOP – What it really means

An object by itself is fairly useless until you do something with it. Lists, which are made from objects, are very useful because they have extra stuff attached that make it unique from other objects. The same goes for strings, dictionaries, and numbers. This extra stuff comes in two types: methods and attributes.

Methods are functions attached to an object. They usually do something with the object or to the object that they are attached to. For example, a list is an object:

>>> my_list = [3, 1, 2]
>>> my_list
[3, 1, 2]

This list can be sorted with a call to a method called sort.

>>> my_list.sort()
>>> my_list
[1, 2, 3]

The method sort is a function that sorts the list that it is attached to. Any function attached to an object in this way is called a method. Here’s some other examples we’ve already seen:

>>> my_list.index(2) # index() is a method
3
>>> my_dictionary = {'bananas': 34, 'apples': 27}
>>> my_dictionary.get('bananas') # get() is a method on dictionaries
34

Attributes are variables attached to an object. Attributes work just like methods, except while methods hold a function to be called, attributes hold some data for that object.

We’ll talk more about attributes and their uses when we discuss classes in another lesson.

OOP – What it REALLY really means

Because everything is an object, we can abstract away difficult concepts by compartmentalizing our code. Anything that has to do with dogs goes on the Dog object; say dog.make_noise(), which barks. Anything that has to do with cats goes on the Cat object; say cat.make_noise(), which meows. That allows us to then call my_pet.make_noise() without knowing what kind of pet we have. That is the true power of OOP.

Don’t get discouraged if Object Oriented doesn’t make much sense yet. It is one of the most difficult concepts to grasp in programming. Having a Computer Science minor, I didn’t truly understand it until several months of working as a programmer. If you’re interested in a more in-depth study on abstraction of objects, check out Code Complete by Steve McConnel.

Next lesson, we’ll do some practice with objects and abstraction by creating our own class! Until next time!

Super Simple Python – Basic String Formatting

String formatting is the process of making your strings flexible and pretty.

Let’s say we want to print out a bunch of information about some numbers.

----------------
1. Number: 2
2. Odd?: False
3. x*2: 4
4. Factorial: 2

Now if we were to try to create this string by adding all the parts together, it would be a giant mess – something like the following. (We’re using the math module in this code bit, so first run a import math to make it available.)

import math

for number in range(5):
    output_string = "----------------n"
    output_string += "1. Number: " + str(number) + "n"
    output_string += "2. False?: " + str(True if number {f528e267df4df4a9788ca8d563bbe1691493a122d7c723196cd7a72052137914} 2 else False) + "n"
    output_string += "3. x*2: " + str(number * 2) + "n"
    output_string += "4. Factorial: " + str(math.factorial(number)) + "n"

    print(output_string)

It can be difficult to sort out the difference between the code and the string using this method. Remember that being able to read and understand your code in the future is one of the most important aspects of programming!

There is, thankfully, a much cleaner way to do this. We can build a template string with keys to identify where we want variables to go.

template_string = """----------------
1. Number: {number}
2. Odd?: {is_odd}
3. x*2: {times_two}
4. Factorial: {factorial}"""

Now that we have a template string with keys like {number} and {is_odd}, we can use the format function to add our variables to the keys. This will, for example, replace the key {number} with the argument number that we pass to the format function.

import math

for number in range(5):
    is_odd = True if number {f528e267df4df4a9788ca8d563bbe1691493a122d7c723196cd7a72052137914} 2 else False
    times_two = number * 2
    factorial = math.factorial(number)

    formatted_string = template_string.format(  # Replaces keys with keyword arguments
            number=number,
            is_odd=is_odd,
            times_two=times_two,
            factorial=factorial)
    print(formatted_string)

This easier-to-read bit of code will give us a very nicely structured print statement.

You can also use string.format() with positional arguments instead of keyword arguments like so:

>>> greeting = "It's {0}, {1}! It's {0}!".format(  # Replaces keys with arguments
...     "Tuesday", "James")
>>> print(greeting)
It's Tuesday, James! It's Tuesday!

The keys {0} and {1} indicate the arguments passed to the format method. The first argument "Tuesday" will replace anywhere with a {0}, and the second argument "James" replaces {1}.

The string.format() method is the current prefered method for string formatting. With a bit of practice, manipulating strings will become second nature! There are even more interesting things you can do with string formatting, which we’ll look at in another episode!

Super Simple Python #16 – Reading Files

Last lesson we learned about writing to a file, this time we will read from it as well.

If this is your first time here, I recommend looking at the first lesson. Starting there and going through the rest of the lessons will prepare you to go through this more advanced lesson!

For our lesson, let’s create a file with some information in it. Create a file somewhere (in this example, it’s at C:/Users/Salvatore/Documents/raven.txt) and type in a few lines of information that we can retrieve.

Once upon a midnight dreary, while I pondered weak and weary,
Over many a quaint and curious volume of forgotten lore,
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
"'Tis some visitor," I muttered, "tapping at my chamber door -
Only this, and nothing more."

Just as we opened the file for writing last time, we need to open it for reading. We use the 'r' mode to access the file in read mode.

>>> f = open('C:/Users/Salvatore/Documents/raven.txt', 'r')

Now that we have the file open, there are a few methods that we can use on f to get the data from the file.

read

The first method is the most straightforward. The f.read() method reads and returns the entire contents of the file as a string.

>>> raven = f.read()
>>> print(raven)
Once upon a midnight dreary, while I pondered weak and weary,
Over many a quaint and curious volume of forgotten lore,
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
"'Tis some visitor," I muttered, "tapping at my chamber door -
Only this, and nothing more."

One thing to be wary of when using read is that the entire contents of the file is read and put into memory. That means if you are reading a very large file (say multiple gigabytes), it will load it all into memory, which could lead you to very slow performance.

Another thing to note when using any of the read methods (or write methods) is that there is a pointer index that remembers the location to which the file has been read (or written). To see this, try reading from the file again:

>>> f.read()
''

It now returns an empty string because the index is at the end of the file, with nothing left to read! If we want to read from the beginning again there are two options: 1) close and open the file again, or 2) use the f.seek(0) method. The integer passed to f.seek indicates where to move the index to, 0 being the character at the beginning of the file, 1 being the character after the first, 2 the character after that, etc.

readline and readlines

The next commonly used methods of reading a file are f.readline() and f.readlines(). While read returns a single string of the entire file, readline returns a string of the first line separated by the newline character (‘\n’). Subsequent calls to readline will return the next line in the file.

>>> f.seek(0)  # To return to the beginning of the file
0
>>> f.readline()
'Once upon a midnight dreary, while I pondered weak and weary,\n'
>>> print(f.readline())
Over many a quaint and curious volume of forgotten lore,

Similarly, readlines returns a list of the same strings, also separated by newline characters.

>>> lines = f.readlines()
>>> for line in lines:
...     print(line)
...
While I nodded, nearly napping, suddenly there came a tapping,

As of some one gently rapping, rapping at my chamber door.

"'Tis some visitor," I muttered, "tapping at my chamber door -

Only this, and nothing more."

Note that we have an extra blank line between each phrase. This is because readline and readlines include the newline character at the end of each string, and using print adds a newline character itself. To avoid this issue, call the strip() method on each of the strings. This removes any extra whitespace characters from either end of the string.

>>> f.seek(0)
>>> for line in lines:
...     print(line.strip())
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
"'Tis some visitor," I muttered, "tapping at my chamber door -
Only this, and nothing more."

Using f.readline() with large files has the side effect over f.read() or f.readlines() of not loading the entire contents of the file into memory.

Super Simple Python #15 – Writing to Files

This week we will learn how to write to files! Writing to files is one of the simplest and most basic ways to communicate information, either to a user or to another program (or to the program itself!).

If this is your first time here, I recommend looking at the first lesson. Starting there and going through the rest of the lessons will prepare you to go through this more advanced lesson!

This first part is easiest to do in an interactive console.

Opening a file to either read or write is very simple: use the open built-in function. To use open, we must pass two arguments: the path of the file as a string (e.g. 'C:/directory/path/file_name.txt'), which you can get from using your computers file explorer (either Windows Explorer or Finder on a Mac); and an access mode which tells the program how to open the file. There are several access modes, but the most common are 'r' for read the file, 'w' for write to file, and 'a' for append to the end of the file. We’ll be writing in our examples this lesson.

This is how we use open with a path and an access mode:

>>> f = open('C:/test_file.txt', 'w')

A couple of things happen when we execute this line.

The first thing is that when we use 'w' to gain write access, Python will lock the file path that we’ve given it, so that it has exclusive rights to the file. Until we close the file or close Python, that file cannot be modified or deleted by any other program.

The next thing to note is that we get a file object back from open. This file object is what we will use to write stuff to the file.

To close our open file, we call the close method on the file object. This will allow other programs to get access to the file. In practice, you should only keep a file open as long as you need access to it.

>>> f.close()

Note that you cannot write to a file if it has been closed! You must re-open to access it again.

There are two common methods for writing to a file: write and writelines. The first method, write, takes any string you give it and writes it directly to the file.

>>> f.write("This is a string.")
>>> f.write("Here is another string.")

If you close the file using f.close() and look at it in a text editor (PyCharm will work! Just drag it in from your file browser), you will notice that they are on the same line:

This is a string.Here is another string.

If you use the write method, you need to handle new-line characters, or \n, yourself.

>>> f = open('C:/test_file.txt', 'w')
>>> f.write("This is a string.\n")
>>> f.write("Here is another string.\n")
>>> f.close()

You should see this in the file:

This is a string.
Here is another string.

The other method, writelines, handles the \n‘s for you, but you need to pass it a list of strings.

>>> lines_to_write = ["This string will be used with 'writelines'!", "This one will too!"]
>>> f = open('C:/test_file.txt', 'w')
>>> f.writelines(lines_to_write)
>>> f.close()

Now if we check the file:

This string will be used with 'writelines'!
This one will too!

That is really all there is to writing files. The only other interesting things that are used often are the read mode 'r', and the append mode 'a'. Append mode works just like write, except when you use write or writelines, it appends the text to the end of the file, which is very useful for logging information. You should play with it! Try running the same commands in this lesson with append mode 'a' instead of write mode 'w'.

We will handle read mode next lesson.