STFS

From Free60
Jump to: navigation, search

STFS (Secure Transacted File System) is the file system used by the Xbox 360 for all packages created and downloaded by the system. It is protected using a series of SHA1 hashes and a RSA signature. STFS is commonly found in Xbox 360 Content Packages (XContent), but is not limited to those only as the PEC (Profile Embedded Content) files employ STFS. The two known categories for STFS are read-only and writeable. Read-only content packages are found with a PIRS/LIVE signed header and writeable content packages are console signed (CON).

STFS

The 360 uses packages to transfer saves/content/games/pictures and more. Most packages start with the strings PIRS, LIVE or "CON ", some are even embedded inside gamer profiles (using the PEC format described below). All of these are STFS content packages which hold the real files along with metadata that the dashboard reads like the title, the licenses and the RSA signature which is used to verify the package.

The acronym STFS stands for Secure Transacted File System, which shows how the packages are secure (signature and hashes) and transacted (multiple file / directory revisions)

LIVE and PIRS files come from Xbox Live, these are signed using a private key that only Microsoft has. The console uses a public key which is hardcoded inside it to verify the package and make sure the person is allowed to use it. CON files are created by the console for saves and profiles. The console uses its own private key to sign CON files. Many editors are available for saves and profiles which can be used with no modification to the console.

Throughout an STFS package, there is a series of SHA1 hashes used to verify the package, and help with downloads (if a block isn't valid, it can be redownloaded). The hashes are located at certain parts of the file, a way of calculating where is (will be!) down below.

Volume Descriptor

There are 2 types of volume descriptor a content file may have, one for STFS packages and another for SVOD

STFS

Offset Length Type Information
0x00 0x01 byte Volume descriptor size (0x24)
0x01 0x01 byte Reserved
0x02 0x01 byte Block Seperation
0x03 0x02 signed short File Table Block Count
0x05 0x03 signed int24 File Table Block Number
0x08 0x14 byte[] Top Hash Table Hash
0x1C 0x04 signed int Total Allocated Block Count
0x20 0x04 signed int Total Unallocated Block Count

SVOD

Offset Length Type Information
0x00 0x01 byte Volume descriptor size (0x24)
0x01 0x01 byte Block Cache Element Count
0x02 0x01 byte Worker Thread Processor
0x03 0x01 byte Worker Thread Priority
0x04 0x14 byte[] Hash
0x18 0x01 byte Device features
0x19 0x03 uint24 Data block count
0x1C 0x03 uint24 Data block offset
0x1F 0x05 byte[] Padding/reserved

Content Packages

Header

Offset Length Type Information
0x0 0x4 ascii string Magic

The magic can be one of the following:

Signature Type Information
"CON " Signed by a console. Found on many files such as cache files, profiles, saved games.
PIRS Signed by Microsoft. Found on files delivered by Microsoft through non-Xbox Live means such as system updates.
LIVE Signed by Microsoft. Found on files delivered over Xbox Live such as items from the Marketplace like themes.

The following is used for Console Signed ("CON ") packages, and is used by the Xbox as the format for signatures generated by it:

Offset Length Type Information
0x4 0x2 bytes Public Key Certificate Size
0x6 0x5 bytes Certificate Owner Console ID
0xB 0x14 ascii string Certificate Owner Console Part Number
0x1F 0x1 byte Certificate Owner Console Type (1 for devkit, 2 for retail)
0x20 0x8 ascii string Certificate Date of Generation
0x28 0x4 bytes Public Exponent
0x2C 0x80 bytes Public Modulus
0xAC 0x100 bytes Certificate Signature
0x1AC 0x80 bytes Signature

and for remotely signed (LIVE/PIRS) packages:

Offset Length Type Information
0x4 0x100 bytes Package Signature
0x104 0x128 bytes Padding

The Package Signature is generated using the value at 0x32C (Content ID/Header SHA1).

Metadata

Offset Length Type Information
0x22C 0x100 license entries (see below) Licensing Data (used to check package owner)
0x32C 0x14 bytes Header SHA1 Hash (from 0x344 to first hash table)
0x340 0x4 unsigned int HeaderSize
0x344 0x4 signed int Content Type (see below)
0x348 0x4 signed int Metadata Version (see below)
0x34C 0x8 signed long Content Size
0x354 0x4 unsigned int Media ID
0x358 0x4 signed int Version (system/title updates)
0x35C 0x4 signed int Base Version (system/title updates)
0x360 0x4 unsigned int Title ID
0x364 0x1 byte Platform (Xbox 360 = 2/PC = 4)
0x365 0x1 byte Executable Type
0x366 0x1 byte Disc Number
0x367 0x1 byte Disc In Set
0x368 0x4 unsigned int Save Game ID
0x36C 0x5 bytes Console ID
0x371 0x8 bytes Profile ID
0x379 0x24 Volume Descriptor File System Volume Descriptor
0x39D 0x4 signed int Data File Count
0x3A1 0x8 signed long Data File Combined Size
937 4 int32(STFS = 0, SVOD = 1) Descriptor type
941 4 int32 Reserved
0x3B1 0x4C bytes Padding
0x3FD 0x14 bytes Device ID
0x411 0x900 (each 0x80 = different locale) unicode string Display Name
0xD11 0x900 (each 0x80 = different locale) unicode string Display Description
0x1611 0x80 unicode string Publisher Name
0x1691 0x80 unicode string Title Name
0x1711 0x1 byte Transfer Flags (see below)
0x1712 0x4 signed int Thumbnail Image Size
0x1716 0x4 signed int Title Thumbnail Image Size
0x171A 0x4000 (thumbnail size) image Thumbnail Image
0x571A 0x4000 (title thumbnail size) image Title Thumbnail Image

Version 2

If the Metadata Version field is set to 2, the format is slightly changed:

Offset Length Type Information
0x3B1 0x10 bytes Series ID
0x3C1 0x10 bytes Season ID
0x3D1 0x2 signed short Season Number
0x3D5 0x2 signed short Episode Number
0x3D5 0x28 bytes Padding
0x171A 0x3D00 (thumbnail size) image Thumbnail Image
0x541A 0x300 (each 0x80 = different locale) image Additional Display Names
0x571A 0x3D00 (title thumbnail size) image Title Thumbnail Image
0x941A 0x300 (each 0x80 = different locale) image Additional Display Descriptions

License Entries

For every entry in the license data field:

Offset Length Type Information
0x0 0x8 signed long License ID (XUID / PUID / console id)
0x8 0x4 signed int License Bits
0xC 0x4 signed int License Flags

Content Types

Value Description
0xD0000 Arcade Title
0x9000 Avatar Item
0x40000 Cache File
0x2000000 Community Game
0x80000 Game Demo
0x20000 Gamer Picture
0xA0000 Game Title
0xC0000 Game Trailer
0x400000 Game Video
0x4000 Installed Game
0xB0000 Installer
0x2000 IPTV Pause Buffer
0xF0000 License Store
0x2 Marketplace Content
0x100000 Movie
0x300000 Music Video
0x500000 Podcast Video
0x10000 Profile
0x3 Publisher
0x1 Saved Game
0x50000 Storage Download
0x30000 Theme
0x200000 TV
0x90000 Video
0x600000 Viral Video
0x70000 Xbox Download
0x5000 Xbox Original Game
0x60000 Xbox Saved Game
0x1000 Xbox 360 Title
0x5000 Xbox Title
0xE0000 XNA

Transfer Flags

Value Description
0x00 DeviceID and ProfileID Transfer
0x20 Move Only Transfer
0x40 DeviceID Transfer
0x80 ProfileID Transfer
0xC0 None

These are bit flags, dunno why everyone treats like enum.
Bit0: None(Yet?)
Bit1: None
Bit2: Deep Link Supported
Bit3: Disable Network Storage
Bit4: Kinect Enabled
Bit5: Move Only Transfer
Bit6: Device Transfer
Bit7: Profile Transfer

Profile Embedded Content (PEC)

When extra security is needed for content which is already using STFS, PEC files may be used to add an extra layer on top. PEC files use the STFS descriptor and algorithms, but has no similarity with content packages.

PEC files are only currently used for avatar items/clothing.

Block 0 starts at 0x3000, and hash table 0 at 0x1000/0x2000.

Header

Offset Length Type Information
0x0 0x228 Console Security Certificate Signed
0x228 0x14 bytes SHA1 hash from 0x23C-0x1000
0x23C 0x8 signed long Unknown
0x244 0x24 Volume Descriptor (STFS) File System Volume Descriptor
0x268 0x4 signed int Unknown
0x26C 0x8 bytes Profile ID
0x274 0x1 byte Unknown
0x275 0x5 bytes Console ID

File Listing

The value at 0x37E (File Table Block Number on the structure above) determines where the file table begins. As it is a block number, you will have to convert it to an offset. Here is some code in C# for converting:

       internal int BlockToOffset(int xBlock)
       {
           int xReturn = 0;
           if (xBlock > 0xFFFFFF)
               xReturn = -1;
           else
               xReturn = (((MetaData.HeaderSize + 0xFFF) & 0xF000) + (xBlock << 12));
           return xReturn;
       }
       internal int ComputeDataBlockNumber(int xBlock)
       {
           int xBlockShift;
           if(((MetaData.HeaderSize + 0xFFF) & 0xF000) == 0xB000)
               xBlockShift = 1;
           else
               if((MetaData.Descriptor.BlockSeperation & 1) == 1)
                   xBlockShift = 0;
               else
                   xBlockShift = 1;
           
           int xBase = ((xBlock + 0xAA) / 0xAA);
           if(this.Header.Magic == XContent_Header.Header_Magic.CON)
               xBase = (xBase << xBlockShift);
           int xReturn = (xBase + xBlock);
           if (xBlock > 0xAA)
           {
               xBase = ((xBlock + 0x70E4) / 0x70E4);
               if (this.Header.Magic == XContent_Header.Header_Magic.CON)
                   xBase = (xBase << xBlockShift);
               xReturn += xBase;
               if (xBlock > 0x70E4)
               {
                   xBase = ((xBlock + 0x4AF768) / 0x4AF768);
                   if (this.Header.Magic == xBlockShift)
                       xBase = (xBase << 1);
                   xReturn = (xReturn + xBase);
               }
           }
           return xReturn;
       }

Each embedded file starts at a 4096 byte boundary. The optional space between embedded files is filled with null bytes.

The file listing consists of entries which have the format below. The listing ends with an entry consisting of only null bytes.

Offset Length Type Information
0x0 0x28 ascii string File name, null-padded
0x28 0x1 byte Length of file name, plus flags
0x29 0x3 signed int24 Number of blocks allocated for file (little endian)
0x2C 0x3 signed int24 Copy of 0x29
0x2F 0x3 signed int24 Starting block number of file (little endian)
0x32 0x2 signed short Path indicator (big endian)
0x34 0x4 unsigned int Size of file in bytes (big endian)
0x38 0x4 signed int Update date/time stamp of file
0x3C 0x4 signed int Access date/time stamp of file

Byte 0x28 also has two flags: bit 6 and bit 7. If bit 6 is set it means that all of the blocks in the file are consecutive. Bit 7 indicates that the file is a directory.
The first 6 bits 0-5, are the length of the filename.

        public System.Boolean IsDirectory//bit 7
        {
            get
            {
                return (Flags & 128) == 128;
            }
            set
            {
                if (value != IsDirectory)
                {
                    Flags ^= 128;
                }
            }
        }
        //public System.Boolean IsUnknown//bit 6
        //{
        //    get
        //    {
        //        return (Flags & 64) == 64;
        //    }
        //    set
        //    {
        //        if (value != IsUnknown)
        //        {
        //            Flags ^= 64;
        //        }
        //    }
        //}
        public System.Byte NameLength//first 6 bits
        {
            get
            {
                return (System.Byte)(Flags & 63);
            }
            set
            {
                Flags ^= (System.Byte)(value ^ NameLength);
            }
        }

The path indicator indicates the path of the file. -1 (0xFFFF) means that the file is in the root directory, any other value V refers to the (sub)directory which is listed as the Vth entry in the listing (counting from 0). Directories can nest.

The FAT format is used for the date/time stamps of the files.

Hash Tables / Block Offsets

A block is 0x1000 (4096) bytes and the first block is located at 0xC000. Every 0xAA (170) blocks there is a block that contains a hash of each block and every 0xAA*0xAA (0x70E4) blocks there is another table (presumably hashing the 0xAA table blocks). This means that when using a block number from a file listing you need to account for the hash tables which are not included in the block numbering system. For example block 171 is actually located where you would expect block 172 to reside ((171 + int(171/170)) * 0x1000 + 0xC000).

Offset Length Type Information
0x0 0x14 byte Hash(SHA1)
0x14 0x1 byte Status byte
0x15 0x3 Int24 Next block

The status byte is very important when injecting files into a package. Here are the meanings of them.

Value Meaning
0x00 Unused Block
0x40 Free Block (previously used) 0x80 Used Block
0x80 Used Block
0xC0 Newly Allocated Block

Files are not always stored in consecutive blocks, you have to find the corresponding hash table record and read the next block number (similar to cluster maps in FAT).

Here is some C# code for converting a block to it's hash table position(it may not work perfectly!):

       internal int ComputeLevelNHashBlockNumber(int xBlock, int xLevel)
       {
           int xBlockShift;
           if (((MetaData.HeaderSize + 0xFFF) & 0xF000) == 0xB000)
               xBlockShift = 1;
           else
               if ((MetaData.Descriptor.BlockSeperation & 1) == 1)
                   xBlockShift = 0;
               else
                   xBlockShift = 1;
           int[] xBlockStep;
           if (((MetaData.HeaderSize + 0xFFF) & 0xF000) == 0xB000)
               xBlockStep = new[] { 0xAC, 0x723A };
           else
               if ((MetaData.Descriptor.BlockSeperation & 1) == 1)
                   xBlockStep = new[] { 0xAB, 0x718F };
               else
                   xBlockStep = new[] { 0xAC, 0x723A };
           int xReturn;
           int xBase;
           int xStep = xBlockStep[1];
           if (xLevel == 0)
           {
               xReturn = (xBlock / 0xAA);
               xStep = (xReturn * xBlockStep[0]);
               if (xReturn != 0)
               {
                   xReturn = (xBlock / 0x70E4);
                   xBase = (xReturn + 1);
                   if (this.Header.Magic == XContent_Header.Header_Magic.CON)
                       xStep += (xBase << xBlockShift);
                   else
                       xStep += xBase;
                   if (xReturn == 0)
                       return xStep;
                   else
                   {
                       if (this.Header.Magic == XContent_Header.Header_Magic.CON)
                           return (xStep + (1 << xBlockShift));
                       else
                           return (xStep + 1);
                   }
               }
               else
                   return xStep;
           }
           else if (xLevel == 1)
           {
               xReturn = (xBlock / 0x70E4);
               xBase = (xReturn * xStep);
               if (xReturn != 0)
               {
                   if (this.Header.Magic == XContent_Header.Header_Magic.CON)
                       return (xBase + (1 << xBlockShift));
                   else
                       return (xBase + 1);
               }
               else
                   return (xBase + xBlockStep[0]);
           }
           else if (xLevel == 2)
           {
               return xStep;
           }
           else
           {
               throw new Exception("Level Unknown: " + xLevel.ToString());
               return 0xFFFFFF;
           }
       }

Tools

An (old) tool (Python 2.5 required) to analyze and extract these archive files is available at extract360.py (2008-08-03, 23056 bytes, MD5 = 3aa517c83d01c618927b78d0ca665d02)

wxPirs 1.1 can extract from LIVE/PIRS files fine, but as it doesn't use hash tables properly it doesn't work well with CON files.

A newer tool was released by DJ Shepherd called Le Fluffie, which can create and extract from CON/LIVE/PIRS files (but it has some problems with creation, some prefer to use XLAST)

XLAST inside the Xbox 360 SDK can create LIVE/PIRS packages, but it is illegal to share it.

A new python library py360 can read STFS files