treyharrell.com

Renderman for Maya / Renderman Studio displacement hints

September 1st, 2009

I’m about a month into using Renderman Studio v2, and I’d like to share a few random hints about how to get things working how you’d expect them to. This list is generally for people who are used to the mental ray/Maya way of doing things.

This first collection is about one of PRMan’s strongest points: displacements. It’s absurdly fast due to how the REYES architecture handles micropolygons, and it barely has any RAM footprint at all.

1. Object scale is incredibly important when working with displacements in RFM. Your displacement map is calculated in world space, so if you scale up your mesh in Maya, it’ll do weird things to a 32-bit displacement, and you’ll end up having to tune your alpha gain & offset to match the scaling ratio.

Freezing your transforms doesn’t work, because it’s reading the map in world space.

The proper way to fix this issue is to scale your mesh in Maya, freeze the transforms, export it out as an .obj, then import it at the lowest subdivision level into your sculpt, recalculate all your maps, then export the mesh again from ZBrush (UVs might smooth or shift, always re-export your .obj after calculating displacements).

After that, reimport it into Maya (make sure you check “Merge UVs” under the export options!). This way ZBrush is calculating your displacements at actual scale, and you can use the magic numbers: alpha gain of 2.2, offset of -1.1 on your map. Remember to turn off quadratic filtering on your image as it softens your displacements.

Note: Alpha gain / offset of 2.2 / -1.1 are for versions of ZBrush prior to 3.5. From 3.5 on, use 1 / -0.5.

2. Your shading rate affects how much of the displacement map is actually used, and you have to set this multiple places depending on your scene setup (subsurface render passes, renderradiosity render passes as well as the “quality” tab under render globals). In general, you can think of a “1″ setting for shading rate as production quality, but it’s possible to go higher (more on this below).

Worth clarifying that you don’t necessarily need to set this to the *same* value if you’re, say, doing an environment light with color bleeding. There’s a strong possiblity that a 64-sampled renderradiosity map will look totally fine and have enough detail with a sample rate of 20(!!!) and save you a ton of map calculation time.

3. Is your displacement, hair or other micro-detailed feature not sharp enough? Try cranking your sample rate all the way down to .1. — zero isn’t an option, it’ll set it to 1 automatically.

4. Getting weird blobby surface artifacts when rendering displacements (or subsurface effects)? It’s likely due to PRman’s credo of avoiding raytracing at all costs.

What you’re seeing is the result of your pointclouds or raytraced shadows being calculated against your un-displaced model, then just the points that invoke raytracing are comped over your render, and unless you force Renderman to calculate displacements against everything, you’ll end up with an ugly, blobby, posterized mess.

Click on your object, then go to the attribute editor and manage attributes. You need to add a “trace displacements” attribute to your object. (Attributes->Renderman->Manage Attributes).

Be aware, however, that if you’re using lots of rays (for raytraced shadows, for instance) that each of those rays is getting calculated a bunch of times and it can all add up in your render times.

5. While not strictly a displacement tip, it applies due to the way “trace displacements” is handled: when doing pointcloud passes for environment lights (RenderRadiosity etc), make damned sure that raytracing is disabled for your pointcloud passes or your render will quite likely never complete in your lifetime.

6. Getting a ghost of your mesh’s wireframe when using displacements and raytraced shadows? The fix is the same as tip #3 — turn your shading rate way way way down. Generally .1 is the magic setting to fix this. If you’re using other passes (subsurface, radiosity etc) make sure to set your sample rate there as well if you’re seeing artifacts on those passes as they preview render. This is a brute-force fix that looks good without tweaking any other settings.

There’s another method to fix this issue that isn’t quite so brute force: adjust your trace bias up by small amounts until the self-shadowing (that’s what those ghosts are) go away. Render times will likely be much quicker with this method, but shadows won’t likely be quite as detailed or accurate.

7. Magic numbers for 32-bit ZBrush displacements in Slim:

Assuming you’re using a combine node:

Do displacement: checked
Use shading normals: checked
Base: -0.50
Kb: 2.2

Note: Base / Kb of -0.50 / 2.2 are for versions of ZBrush prior to 3.5. From 3.5 on, use -0.50 / 1

Add a bump, make it an image file with 0 filter, set to a value of 1.

Pretty much the same thing as Maya… the math’s the same, even if the numbers are different, so it might trip some folks up.

–

More hints to come in the not-too-distant future.

Entry Filed under: CG, Geek, Tutorial