SDB2 shader bundles
Shaun White Skateboarding ships its Direct3D 9 shaders as bundles beside the executable: ekshadersPChigh.bin, ekshadersPCmed.bin and ekshadersPClow.bin for the three quality levels, three ...compress.bin siblings and ekshadersinitpccompress.bin. The three named bundles are uncompressed and carry both the original HLSL source of the effects and the compiled Shader Model 3 programs. Each ...compress.bin sibling is its quality bundle compressed, byte for byte, and ekshadersinitpccompress.bin is a small fourth bundle of the same kind (see Compressed bundles). Proven · SWS[the four compressed files decoded and compared with the three uncompressed ones]
The executable takes its shaders from these bundles only, never from the forges, and checks the magic SDB2 and the version 39 when it opens one. Observed · SWS[the executable’s loader, read]
From d3dx9_41.dll it imports only D3DXCreateTextureFromFileInMemory, D3DXCreateBuffer and D3DXGetShaderConstantTable, and from d3d9.dll only Direct3DCreate9: nothing that compiles or assembles a shader. Proven · SWS[the executable’s import table] The HLSL sources in the bundles are therefore not compiled at run time through an imported function; every program the game draws with comes precompiled. Inferred · SWS
Container
Section titled “Container”All three uncompressed bundles start with the same 24-byte header. Proven · all three quality files
Little-endian. Identical in the three quality files except for the record count.
| Offset | Size | Type | Field | Confidence | Notes |
|---|---|---|---|---|---|
| 0x0 | 4 | char[4] = SDB2 | magic | Proven | |
| 0x4 | 4 | u32 = 39 | version | Proven | |
| 0x8 | 4 | u32 = 62 | source count | Proven | |
| 0xC | 4 | u32 | record count | Proven | 6,456 in the high and medium files, 3,919 in the low file |
| 0x10 | 4 | u32 = 24 | first source | Proven | |
| 0x14 | 4 | u32 = 304563 | table offset | Proven | 0x4A5B3 |
The sources run from offset 24 to the table at 304,563 with nothing between them; the table follows, then the compiled payloads, which tile the rest of the file to its end (35,094,609 bytes high, 34,181,857 medium, 16,310,862 low). Observed
Little-endian.
| Offset | Size | Type | Field | Confidence | Notes |
|---|---|---|---|---|---|
| 0x0 | 64 | char[64] | file name | Observed | NUL-padded |
| 0x40 | 4 | u32 | CRC-32 | Observed | of the exact source bytes, line endings included |
| 0x44 | 4 | u32 | length | Observed | |
| 0x48 | 16 | u32[4] | dependency identity | Observed | 128 bits with exactly one bit set, a different bit per source |
| 0x58 | length | bytes | source text | Observed | HLSL |
The compiled table is laid out for 8,192 records of 36 bytes (nine u32) and only the active records are stored, followed by the payloads; the unused part is zero. Each record is a seven-word key, then the payload’s offset and length. Observed
Records
Section titled “Records”A record’s payload begins 0xBEEF. Five kinds exist; the loader dispatches on the kind: Observed
| Kind | Content | Count in the high file |
|---|---|---|
| 4 | An effect pass: vertex and pixel program with state and sampler tables | 6,446 |
| 5, 7 | A utility-stage program | 1 each |
| 6 | A pixel program whose only use is a presence flag | 1 |
| 8 | A pass of render state only, with no programs | 7 |
Fifty-four kind-4 records of the high file have length zero: they name keys that were never compiled. They hold 6,393 vertex and 6,394 pixel programs, 26,921 sampler bindings, 59,876 state pairs and 39,938 state-variable names. The medium file has the same records with 25,298 sampler bindings; the low file has 3,919 records (3,909 of kind 4), none missing, 3,910 vertex and 3,911 pixel programs, 13,026 sampler bindings, 36,657 state pairs and 23,429 names. Each file tiles exactly to its end with these counts. Observed
Little-endian. The bytecodes follow for their exact declared lengths, then the state, sampler and name tables.
| Offset | Size | Type | Field | Confidence | Notes |
|---|---|---|---|---|---|
| 0x0 | 2 | u16 = 0xBEEF | marker | Observed | |
| 0x2 | 1 | u8 | literal state count | Observed | |
| 0x3 | 1 | u8 | save and restore state count | Observed | |
| 0x4 | 1 | u8 | state variable name count | Observed | |
| 0x5 | 1 | u8 | sampler group count | Observed | |
| 0x6 | 1 | u8 | second sampler group count | Observed | 0 in the PC files |
| 0x7 | 1 | u8 = 0 | reserved | Observed | |
| 0x8 | 4 | u32 | vertex program length | Observed | |
| 0xC | 4 | u32 | pixel program length | Observed | |
| 0x10 | 8 | u64 | vertex stage cache key | Unknown | no hash algorithm is established |
| 0x18 | 8 | u64 | pixel stage cache key | Unknown | |
| 0x20 | 16 | bytes = 0 | reserved | Observed |
After the programs come the literal state pairs, the save-and-restore pairs (each a Direct3D render-state number and a value, both u32), the sampler groups and the NUL-terminated names of the runtime state variables. A sampler entry is the sampler name and the texture parameter name (each NUL-terminated), a packed address u32, a count u32 and that many pairs of sampler-state kind and value; on PC the packed address is 0x00800000 or’d with the pixel sampler slot. There is no padding between the strings and integers. Observed
The second state array holds placeholders that the loader fills: on entry the pass reads the current device states into them, applies the first array and, on exit, restores the saved values. Observed
The pixel programs of the high file are ps_3_0 and use 22 token kinds, 20 of them executable (MOV, ADD, MAD, MUL, RCP, RSQ, DP3, DP4, MIN, MAX, EXP, LOG, LRP, FRC, POW, NRM, TEXKILL, TEX, CMP, DP2ADD), all in straight lines with no IF, LOOP or CALL. Unique programs number 589 vertex and 3,752 pixel by byte hash. Observed
The compiled programs
Section titled “The compiled programs”The effect passes of the three uncompressed bundles hold 7,306 distinct programs by their bytes, 1,099 vertex and 6,207 pixel (588 and 3,751 in the high bundle, 538 and 3,273 in the medium, 543 and 2,062 in the low; most repeat across bundles). The utility records add two more: kind 5’s vertex program (300 bytes) and kind 6’s pixel program (136 bytes), the same in all three bundles; kind 7’s 136-byte pixel program is also one of the effect programs. That makes 7,308 distinct programs, 1,100 vertex and 6,208 pixel, and the init bundle adds none. Every one is Shader Model 3.0: the version token is 0xFFFE0300 for a vertex program and 0xFFFF0300 for a pixel program, and none is Shader Model 1 or 2. Proven · SWS · every record of the four bundles
The counts and checks below are over the 7,306 effect programs. Together they hold 63,679 vertex and 601,220 pixel instructions. A program has no control flow: no if, loop, rep, call, break or predicate. Observed · the three uncompressed bundles
| Stage | Opcodes used (besides dcl and def) |
Confidence |
|---|---|---|
| Pixel | add cmp dp2add dp3 dp4 exp frc log lrp mad max min mov mul nrm pow rcp rsq tex texkill |
Observed · the three uncompressed bundles |
| Vertex | add dp3 dp4 exp frc log lrp mad max min mov mova mul nrm pow rcp rsq sge sincos slt |
Observed · the three uncompressed bundles |
The vertex programs address the float constants relative to the address register in 4,170 source operands, always as c[a0.x], c[a0.y], c[a0.z] or c[a0.w] plus the register number, in mul (1,038), mad (3,114), mov, max and min (6 each), after a mova a0, r (344 writes of all four lanes and 12 of x alone). Observed · the three uncompressed bundles
This is the form of a matrix-palette skinning loop, a mova of a scaled index vector followed by reads of the bone matrices from the constant file; that these programs skin is concluded from the form. Inferred
Pixel source operands carry no modifier besides negate (68,301), absolute value (4,171) and negated absolute value (6); vertex operands only negate (4,603). Saturation is on 27,515 pixel results and 656 vertex results; no result is partial precision. Observed · the three uncompressed bundles
Microsoft’s D3DDisassemble lists the same instructions, in the same order, with the same registers, swizzles and modifiers, as a reader of the tokens: all 1,099 vertex programs and all 6,207 pixel programs, line for line. Observed · the three uncompressed bundles
That reading depends on four details of the token format that these programs use. A register’s type is five bits, the low three at
bits 28 to 30 of the operand token and the high two at bits 11 and 12, so the colour outputs oC0 and oD0 and the samplers
(types 8 to 10) decode only with both parts; bit 31 of an operand token is a marker set on every operand, not part of the type.
Relative addressing is bit 13, and the address-register token follows the operand. Every sincos has a destination and one
source. A dcl token’s second word is a usage and an index (usage in bits 0 to 3, index in bits 16 to 19), or for a pixel
sampler the texture type in bits 27 to 30. Observed · the three uncompressed bundles
No vertex program declares a sampler. The pixel programs declare samplers s0 to s10 and, above a gap, s14 and
s15: s0 in 6,042 of the 6,207, s15 in 2,631 and s14 in 1,544, and none s11 to s13. A pixel program declares at
most 12, and 3,559 of them declare a set that is not a run starting at s0. Observed[the declarations of the three
uncompressed bundles’ programs]
| Sampler | s0 |
s1 |
s2 |
s3 |
s4 |
s5 |
s6 |
s7 |
s8 |
s9 |
s10 |
s14 |
s15 |
Confidence |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pixel programs declaring it | 6,042 | 5,471 | 4,658 | 3,488 | 2,118 | 1,037 | 389 | 117 | 31 | 6 | 1 | 1,544 | 2,631 | Observed |
On the reference rasterizer (D3DDEVTYPE_REF, software vertex processing) mova rounds to the nearest integer with a half going away from zero (-2.5 gives -3, 0.5 gives 1), a relative constant read at an index from 256 up to 4096 returns zero, and slt and sge are the IEEE comparisons (a NaN operand gives 0). Observed · a vertex program run on the reference rasterizer
Compressed bundles
Section titled “Compressed bundles”A ...compress.bin file is one compressed stream of the SDB2 bytes: a byte 0xF5 and the decoded size, then an Anvil block header with no chunk table, then the chunks, each with a header of its own. It uses the magic, version, algorithm and checksum of the forges’ data blocks, but not their layout: the forges’ blocks list all chunk sizes up front in 16-bit fields, while here each chunk carries 32-bit sizes before its bytes. Proven · SWS[all four compressed files]
Little-endian. The chunks run to the end of the file.
| Offset | Size | Type | Field | Confidence | Notes |
|---|---|---|---|---|---|
| 0x0 | 1 | u8 = 0xF5 | unknown | Unknown | the same in all four files |
| 0x1 | 4 | u32 | decoded size | Proven | the length of the decoded SDB2 bundle |
| 0x5 | 8 | u64 = 0x1004FA9957FBAA33 | magic | Proven | |
| 0xD | 2 | u16 = 1 | version | Proven | |
| 0xF | 1 | u8 = 2 | algorithm | Proven | LZO2A |
| 0x10 | 4 | u32 = 32768 | largest chunk | Inferred | no chunk decodes to more |
| 0x14 | variable | chunks | chunks | Proven | each: u8 flag (1), u32 stored size, u32 decoded size, u32 Adler-32 (from 0) of the stored bytes, the stored bytes |
Every chunk of the four files has the flag 1 and is LZO2A-compressed (none is stored raw), passes its checksum and decodes to its stated size; the decoded sizes add up to the size at offset 1, and the chunks end exactly at the end of the file. Proven · SWS[all four compressed files]
| File | Size | Chunks | Decoded | Decodes to |
|---|---|---|---|---|
ekshadersPChighcompress.bin |
8,772,168 | 1,072 | 35,094,609 | ekshadersPChigh.bin, byte for byte |
ekshadersPCmedcompress.bin |
8,205,957 | 1,044 | 34,181,857 | ekshadersPCmed.bin, byte for byte |
ekshadersPClowcompress.bin |
3,746,524 | 498 | 16,310,862 | ekshadersPClow.bin, byte for byte |
ekshadersinitpccompress.bin |
86,059 | 19 | 609,323 | an SDB2 bundle with no uncompressed copy |
Proven · SWS[all four compressed files]
The init bundle has the same header (version 39, 62 sources) with 16 records, 13 of kind 4 and one each of kinds 5, 6 and 7, and its table at 302,216. Its sources have the names, in the same order, of the quality bundles’ sources, but six differ in content: ekcustomsemantics.h, eklighting.hlsl, ekcommonps.hlsl, eklightingps.hlsl, platform.inc.fx and ekgeneric.fx. Its records hold 29 programs, 21 distinct (11 vertex, 10 pixel), every one of them also in the quality bundles. Proven · SWS[the decoded init bundle] Which of the two copies of a quality bundle the game opens, and when it loads the init bundle, is not traced. Unknown · SWS
Selecting a record
Section titled “Selecting a record”A material does not name its program. The material class sets a key from its stored switches (see the material body) and the pass looks the key up; the key words are not the global define numbering. The generic colour passes of the effect ekgeneric.fx all use the shader configuration value 2, and the high bundle holds no stored colour permutation with configuration 0: those records are the shadow and depth passes. Observed
The layered diffuse equation
Section titled “The layered diffuse equation”The pixel source ekgeneric.fx computes a surface’s diffuse colour from the vertex colour, the mask and up to three diffuse layers. With the mask sample defaulting to 1 and an absent vertex colour reading as 1, let f = COLOR0 · mask. The base layer is lerp(tex1.rgb, tex1.rgb · tint1.rgb, f.r). The second layer, tex2 · tint2, is blended over it by f.g, multiplied by its own alpha when its UseTextureAlpha flag is set. The third layer is blended likewise by f.b. Neither a plain product with the vertex RGB nor alpha as the only weight reproduces the compiled program; the compiled pixel program of one record agrees with this equation. Observed · SWS[a compiled pixel program disassembled; reference rasteriser runs]
A pixel program run on a Microsoft reference rasteriser with controlled constants and textures, for 246 colour values of the layered equation, matched this computation in 234 exactly and 12 within 1e-5. Observed
The products in this equation are taken on the texture’s stored values, even for a texture whose TextureMap is flagged sRGB. None of the 6,399 effect and state records of the high-quality bundle sets sampler state 11 (D3DSAMP_SRGBTEXTURE), the pixel programs contain no gamma conversion, and in a runtime trace of one level all 304 calls that set sampler state 11 in a window of 40,001 SetSamplerState calls set it to 0. Observed · SWS[the bundle’s records; one level, one call window at run time]
A tint therefore multiplies sRGB-encoded texels as numbers, so in linear terms it acts as if it were itself sRGB-encoded. Inferred Materials of this kind (an sRGB diffuse with a non-white tint) are 327 of 773 in level 01 and 124 of 273 in level 05. Observed · SWS
The equations above give the base colour of the restored state only. Per-frame light constants, the fog and the tone controls are bound by frame jobs that are not described. Unknown