treyharrell.com

Optimizing Multiple Huge Maya/Mental Ray Displacements

January 5th, 2010

Recently I’ve been working on some R&D for stereo 3D environments and pipeline issues.

One big issue is that cheats that you become accustomed to in production such as projections and bump maps break in an awful lot of circumstances when viewed in stereo, so very fine displacements will come in quite handy.

Since ZBrush 3.5 (and earlier if you knew how to break Multi-Displace’s cryptic code), it’s quite easy to generate single channel gray (1×32 bit) displacements instead of traditional (3×32 bit) maps… essentially giving you the same amount of detail in 1/3 the disk space and network throughput overhead… and RAM overhead if the entire texture needs to remain in memory.

PRMan in Maya handles single channel gray float TIFFs just fine, but Mental Ray (as of Maya 2010) doesn’t. The old swiss army knife imf_copy won’t recognize the files either to convert them to another friendly format, such as .map if you’re having RAM issues.

More after the break…

Long story short, I realized that I’m creating an enormous RAM and network bottleneck when working with multiple 8k displacements in a single scene, and the problem gets multiplied with the more render nodes that you’re using.

Example: 1×32 8k displacement = ~263mb while a 3×32 8k displacement = ~787mb. If you think about the implications there, it doesn’t take many of those maps to bring your RAM and network down to a crawl.

Why don’t we just combine huge displacement maps 3 at a time: one map per RGB channel of a 3×32 .tiff? That gives us most of the benefits of using the much smaller grayscale maps: smaller disk & ram footprint, significantly faster network spools, and the exact same quality. It just takes a few minutes to set this up.

Open 3 different displacement maps (all at the same resolution). One by one, copy it and paste into a single channel of a new 32-bit per channel RGB Photoshop file. You do this my clicking on your channels palette then clicking on each channel individually.

Once you’re done, save out the file as a 32-bit TIFF, uncompressed.

Now we come into Maya and build our network.

The idea is you take your single texture file node, shove it into a Luminance node per channel, and it’s in the format that the displacement shader and shading groups require.

Click the image above to enlarge the an example network.

Basically given a file node, three luminance nodes and three displacement nodes in hypershade, hook them up like this:

(Texture)File1.outColorR -> Luminance1.valueR
(Texture)File1.outColorR -> Luminance1.valueG
(Texture)File1.outColorR -> Luminance1.valueB
Luminance1.outValue -> displacementShader1.displacement

Then set File1 color gain and color offset to 1 gain V,-0.5 in offset V (ZBrush 3.5) or 2.2 in gain V, -1.1 in offset V (Earlier ZBrush) by clicking on the color swatch next to the offsets, switching to HSV and entering a number. Yep, you’re setting the gain and offset on the color, not the alpha like usual.

Rinse and repeat with File1 G and B channels:

(Texture)File1.outColorG -> Luminance2.valueR
(Texture)File1.outColorG -> Luminance2.valueG
(Texture)File1.outColorG -> Luminance2.valueB
Luminance2.outValue -> displacementShader2.displacement

(Texture)File1.outColorB -> Luminance3.valueR
(Texture)File1.outColorB -> Luminance3.valueG
(Texture)File1.outColorB -> Luminance3.valueB
Luminance3.outValue -> displacementShader3.displacement

The other cool bit about this is that you can convert it into .map format and it still works, however .map format generally adds about 1/3 of the filesize as overhead when converting to a pyramid lookup. Depending on your file sizes vs. RAM vs. network throughput it may or may not be worth converting to .map.

Here’s an example scene using the tutorial setup, just with much smaller maps. Maya 2010 .ma format.

http://www.treyharrell.com/downloads/thDisplacementSplit.zip

I’m sure I’m not the first to use this particular technique, but it came in super handy tonight — it saved 4-5 minutes network spool per render slave — about 18 minutes total per frame — plus about 2gb RAM overhead on an R&D project.

Entry Filed under: CG, Geek, Tutorial