Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
These are global settings applied to the entire VFX. Parameters such as Revised Scale and Revised Color can be used to adjust the size and color of an entire VFX without having to manually change the values of each individual component.
Also of note are the DrawLayer parameter, which controls when the VFX is drawn (such as BeforeSky or BeforeCloud), and the Clipbox settings.
Emitters, as their name suggests, emit objects (either other emitters, or particles). They are triggered through timeline items, and can be moved in 3D space just like any other object. Unlike particles, emitters themselves are not visible. Emitters can also trigger a sound to play.
When an emitter creates another object, it can influence the properties of its child (such as color, scale, or position). Whether or not an emitter will influence its children is determined by parameters such as InfluenceCoordScale.
There are several types of emitters:
Point
Cone
Cone Model (like a cone, but is subdivided across its axes)
Sphere
Sphere Model
Model (any model, where objects can be emitted on its vertices)
Welcome to the XIV Dev Wiki. The idea is for this to be a single place where everything is located in a easy to access and searchable manner. Meta discussion and questions should be via issues on the GitHub repo that this documentation is pulled from: https://github.com/xivdev/docs/.
Final Fantasy XIV © 2010-2022 SQUARE ENIX CO., LTD. All Rights Reserved. We are not affiliated with SQUARE ENIX CO., LTD. in any way.
Particles are the visible component of a VFX, and are created exclusively by emitters. They are similar to emitters in that they can be positioned in 3D space.
There are several types of particles:
Decal
Decal Ring
Disc
Laser
Light Model
Line
Model
Parameter
Polygon
Polyline (line made of triangles)
Powder (used for small particles, such as the dust which some weapons create)
Windmill
Quad (a 1x1 square)
Particles can have textures mapped onto them using several different types of nodes (covered later).
Sometimes, a particle does not require complex animations, and can use the SimpleAnimations node instead (in order for it to work, the SimpleAnimationsEnabled attribute must be True) This is often used for Powder particles, which often have very basic animations due to their size and number.
There are several types of texture nodes which can be applied to a particle. Most texture nodes will have a TextureIndex. This determines which texture will be used as a color, normal map, etc. TextureColor1 is especially important as it has MaskTextureIndex, which determines which parts of the particle will be transparent.
Most texture nodes will also have a UVSetIndex (UV Sets are discussed later).
UV Sets are contained within particles, and are used to manipulate textures. This primarily involves scaling, rotating, and offsetting a texture. For example, most aura or fire VFXs will use a UV Set to animate a texture so that it "scrolls" across the particle, creating the illusion of a moving flame or energy aura.
Every VFX has exactly 1 Scheduler.
Schedulers control when Timelines are started, and have 2 components: a list of items, and a list of triggers. A scheduler can have an arbitrary number of items, but must have 12 triggers. Both items and triggers follow the same format:
What the triggers represent is not understood, but they are probably each for a specific type of action, such as sheathing a weapon, getting hit, etc. Unused triggers have TimelineIndex = -1.
Items function like triggers, but will start automatically without any additional input. If you are having trouble viewing a VFX, such as one which was originally used for a cutscene, it can often be useful to disable all the triggers, and instead exclusively use items to start timelines.
Documentation for the VFX system used by FFXIV
(TODO)
A timeline consists of a list of items, and a list of clips.
Items represent the creation of an emitter, while clips represent other actions, such as deleting particles. Although clips are not well understood, the are probably used for things such as hiding a vfx when a weapon is sheathed.
Clips must be triggered from items using the ClipNumber parameter. Usually, items which trigger clips will have EmitterIndex = -1.
Items are also where emitters are attached to binders and effectors using BinderIndex, EmitterIndex, and EffectorIndex. Items which were originally used for a cutscene or background will usually have BinderIndex = -1, which will cause them to not be rendered on player characters, so creating a binder and setting BinderIndex = 0 can fix this.
Various open source resources and projects
This is a list of various open source resources and projects related to FFXIV. Please leave an issue if your project is listed here and you'd like to remove it.
link
language
C#
A popular custom launcher with extra features. Good reference for the login flow (including Steam/Free Trial accounts) and the patching system.
link
language
C#
A plugin framework and API, meant to be used with XIVLauncher. Almost all official plugins are open source.
link
language
C#
Library for reading and interacting with FFXIV game files and Excel data.
link
language
C#
Another library for reading and interacting with game files/Excel data. Provides community maintained metadata on Excel data types.
link
language
Rust
Yet another library for game data, but in Rust.
link
language
TypeScript
Another library for game data, in TypeScript.
link
language
C#, Python
A C# library containing community-maintained structs in the client, and Python scripts to help reverse engineer the game binary in IDA/Ghidra.
link
language
C#
A command line interface for SaintCoinach. Features commands to export raw game files, the UI images, and music.
link
language
C#
A GUI interface for browsing Excel data and viewing territories, powered by SaintCoinach.
link
language
Java
A GUI for browsing the game files. You may be interested in this fork adding more features.
link
source
An API for game versions and patch information.
link
source
A list of file paths in the game files, crowdsourced via a Dalamud plugin.
link
source
A general-purpose API for FFXIV. Most useful if you can't embed a library into your application or need Lodestone information.
link
language
C++
A server emulator. Not always up to date, but still a great reference on many things.
link
language
C#
A general library for packet capture with specialized FFXIV functionality.
link
language
Go, C#
A set of tools to capture network data and store it into an SQLite database.
link
language
Python
A library to parse ACT logs.
This page contains the file structure itself. What each of the structures listed here are used for can be on other pages.
AVFX files are organized as a nested series of blocks, with each block following the format:
The contents of a block can further contain other blocks. It is also worth noting that the name of a block is reversed from its actual meaning. For example, the "top-level" block for an AVFX file is "XFVA"(AVFX reversed). Names which are fewer than 4 character are also padded out to 4 bytes. Blocks are also padded out to be 4-aligned, so even if Sizeis 3, there will be an extra 00 before the next block starts.
Note: the names of the blocks are reversed for readability. Also, the notation TmLn[], for example, simply means several sequential TmLn blocks, one directly after the other. There is no "list" structure in AVFX files.
The structure is also abridged for brevity, the full list of parameters can be found at the link below.
Schd)The way Item and Trgr blocks are organized is somewhat unintuitive. Both of them have an identical structure:
However, each subsequent Item/Trgr will have the data of the previous appended to it. For example:
For this reason, the easiest way to read a list of Item is to take the last one, and split it into 3-block chunks.
TmLn)Timeline items have this structure, and are similar to scheduler items in that each item contains the data of the previous ones as well.
Timeline clips are different in that the data they contain is not organized into blocks, but is rather one continuous 164-byte:
Emit)The Data block contains information relevant to the emitter's type (specified in the EVT parameter). Depending on the type, the Data block may not exist at all (structures).
ItEm and ItPr blocks share the same basic structure, but follow the same pattern as items and triggers within schedulers, where previous items' data is appended to each subsequent one. However, in an emitter, all of the ItEm data is included in the ItPr data. As an example:
Ptcl)Like with emitters, the contents of the Data block depends on the particle type (parameter PrVT ). Some particle types do not contain a Data block (structures).
The Smpl block contains 2 unique parameters: Cols and Frms. Cols is 16 bytes, where each 4 bytes represent an rgba color (each byte is one channel). Frms is 8 bytes, where each 2 bytes is an integer.
Efct)As with emitters and particles, the Data block (structures) depends on the type of effector, and may not exist.
Bind)Data is, once again, dependent on the type of binder, and may not exist (structures). Each of the binder properties have this structure.
Tex)Texture blocks are simply paths to atex files within the game's internal file structure.
Modl)The embedded models have 4 possible blocks within them: VNum , VDrw , VEmt, and VIdx . All 4 of the blocks can be in the same model, however VNum and VEmt are always paired, as are VIdx and VDrw .
VIdxThis is a list of 2-byte integers, where each 3 integers represents a triangle in the model. The integers themselves are the indexes of vertices within VDrw .
VDrwVDrw is a list of vertices, where each vertex is 36 bytes long with the following format:
VEmtThis is a list of 28-byte vertices, with the following format:
VNumThis is a list of 2-byte integers, where each integer corresponds to a vertex in VEmt (so the length of elements in VNum always equals the number in VEmt).
Curves have the following structure:
The name of a curve varies, but some examples are X ,RGB , SclA , etc. They are used to animate motion over time. Keys is a single block which contains the information on the shape of the curve. Every 16 bytes in Keys corresponds to a single keyframe, with the following format:
Life controls how long a particle lives, and can be somewhat counter-intuitive. Life = -1 means that the emitter lives forever, so it's animations will only trigger once. If you want an emitter to continuously loop (that is, die and get re-created), set it a value greater than -1.
Documentation for files located within the games internal filesystem.
Excel is the relational data storage system that the game uses to store information, similarly to a relational database. The structure of EXL, EXH and EXD files are documented here.
Binders determine where a vfx should be created, and its tracking behavior (should it follow a moving character).
There are several types of binders:
Point (the most common, attach to a character, for example)
Camera
Spline
Linear
Binders can be attached to caster or target, and have several "property" fields. Properties have the attribute BinderPointId, which allows a vfx to be bound to a specific point on a model (such as at the hilt of a sword). Binder points are specified in the skeleton's .mdl file, although this is not well-understood.
Effectors apply miscellaneous effects to emitters, particles, or the camera. In order to be used, they must either be triggered through a timeline item, or an emitter.
There are several types of effectors:
Point light
Directional light
Radial Blur
Black hole
Camera quake (if you've ever been annoyed by the screen shake from Holy, this is why)
Contains information about files that are specific to each character.
Data files are xxx.dat files generally located in the XIV client folder and sub folders at <..>\Documents\My Games\FINAL FANTASY XIV - A Realm Reborn\.
The FFXIV Character Creator saves presets, when saving a character appearance at the end of character creation, in a file called FFXIV_CHARA_XX.dat where XX is the preset slot number from 01 to 40.
All data related to characters that have been used on the current machine is saved in a folder called FFXIV_CHRxxxx where xxxx is a unique ID for the character.
This folder contains *.dat files for character settings, hotbars and UI configuration, macros and so on, as well as a log folder containing the chat logs.
*[Last updated FFXIV 6.58]
This folder, called FFXIV_CHRxxxx where xxxx is a unique ID for the character, contains all of the client side data related to one or another of the characters played.
Following is a list of the data files found in this folder.
Note: most of the information in here is taken from this reddit post without doublechekcing for now.
TODO: figure out the format for each file.
ACQ.DAT: The list of recent /tell contacts. Binary format
ADDON.DAT: This stores your UI settings such as HUD layouts and window sizes. Binary format
COMMON.DAT: This stores your Character Settings. Plain text format.
CONTROL0.DAT: This stores Keyboard/Mouse Settings. Plain text format.
CONTROL1.DAT: This stores Gamepay Settings. Plain text format.
GEARSET.DAT: This contains your Gear sets. Binary format.
GS.DAT: Unknown. Binary format.
HOTBAR.DAT: This contains the contents of your Hotbars, but not their positions or such. Binary format.
ITEMFDR.DAT: Item search index, used when searching for similar items. Binary format.
ITEMODR.DAT: This contains Inventory and Retainer item sort positions. Binary format.
KEYBIND.DAT: This contains your Keybinds. Binary format.
LOGFLTR.DAT: This contains your Chat log filters. Binary format.
MACRO.DAT: This is your Macros. Binary format.
UISAVE.DAT: This contains timers such as the retainer venture timer, possibly other stuff. Binary format.
"log" folder : The game writes your chat history to disk each time its internal buffer overflows, which potentially allows you to retrieve chat history in longer play sessions.
The character creator saves it's presets in binairy files called FFXIV_CHARA_XX.dat where XX is the preset slot number. These files are located in the <..>\Documents\My Games\FINAL FANTASY XIV - A Realm Reborn\ folder.
Note: The investigation into this format has been made in a very cursory manner. There is no garantie of accuracy.
[Last updated: FFXIV 7.0 Benchmark]
The file is read in little endian format.
The file contains 4 sections:
A 16 bytes header
A 32 bytes data block
A 40 bytes/characters comment section
A 124 bytes of 0x00 at the end of the file
The 16 bytes of file header are split into 4 sections:
A 32 bits file header @0x00, in orange. Always 0x2013FF14.
A 32 bits version ID @0x04. Only the LSB seems to be used. Here in green, 7(0x00000007).
A 32 bits assumed checksum value @0x08. Here in red, 0x77225B1E.
A 32 bits padding of 0x00000000 @0x0C, in purple.
Knowing the following data sizes (See full pattern file for the full enum values):
Notes:
FaceFeature is a bitfield where each bit is considered a boolean for a specific feature. For exemple bits 6 (0x40) and 5 (0x20) represent left and right tattoos for Hyur and will be 1 when the tattos is visible.
EyeShape, LightDarkColor and FacePaint use the 7th bit (MSB, 0x08) of the u8 as a boolean to for exemple, switch between "light" or "dark" color, or "reversed" facepaint.
These are all of the data saved in the file:
Sections containing match (...){} are single bytes that take on different meanings depending on, usually, the character's Race.
From 0x3000 to 0x5008 included are the 40 characters (bytes) available as a comment when saving the preset.
The rest of the file is empty space. No usage has been found yet for it.
Everything SqPack: indexes, dat files
SqPack(s) are formed from the following concepts, in which order 'matters':
Data Files
Repositories are essentially a collection of categories and there's not much to know here. As of writing, there's 4, one for the base game (ffxiv) and one for each expansion (ex1, ex2, etc.). Consider the following directory structure:
Each folder inside sqpack/ is it's own repository. As an aside, when the game tries to access files, it distinguishes which repository to find a file in by parsing the file path and pulling out 2 segments, the repository name and the category.
For example:
The first two paths explicitly define their repository in their file path, and it's always the second segment - if it's omitted it defaults to the ffxiv repository. Category is always the first segment and must be present for a path to resolve.
Categories are just logical separations of game data. The following categories exist:
Every game path will start with one of the above names and it defines which index to search to find a file.
Indexes and regular data files are effectively SqPack files and subsequently have a common header, SqPackHeader. It looks like the following:
Indexes (and data) starts at size so you want to seek to the value of size before you read anything out of the file.
The index data is located directly after the header and depends on which variant of index file you load. The retail client ships with both variants of index files, which we'll mostly refer to as index and index2 to make the difference obvious. Contrary to the retail client shipping with both index variants, benchmarks only ship with index2 files. The reason is unknown.
Immediately following the SqPackHeader there's a SqPackIndexHeader (which is only present in index files):
The actual SqPackIndexHeader is 0x400bytes large, but for the purposes of this, we're only interested in the first 16 bytes. From the indexDataOffset and indexDataSize, you can determine where to start reading from and how many index elements exist inside an index. indexDataOffset is an absolute offset to where the index data is located, and indexDataSize is the collective size of every IndexHashTableEntry that's in a file. This entry is slightly different in the case of index2 files, so we'll generally cover the two with a focus on index1 files for now.
There's a couple notable differences between the C++ and C# version, so we'll just explain the C++ version and the latter will make sense too.
The hash is a u64 that contains two u32s: the lower bits are the filename CRC32, the higher bits are the folder CRC32.
Generally speaking, calculating a hash works like this:
Convert the path to lowercase
Find the last instance of / and split the string with the last / existing in the first group. The filename needs to have no directory separators
Calculate the CRC32 of both path segments
Join both CRC32s into a u64, eg. directoryHash << 32 | filenameHash
The dataFileId is to identify which file (on disk) contains the file. Larger categories are split across multiple files (each is capped at 2,000,000,000 bytes, or 2 GB), so this is used to distinguish between 020000.win32.dat0 and 020000.win32.dat1 for example, where dataFileId would be 0 and 1 respectively for files located in either dat.
The offset is the absolute number of 8 byte aligned segments that the file is located at within a specific dat file. In a given dat file, a file is located at offset * 0x8 which gives you the absolute offset to start reading a file from.
The main difference between index and index2 is that the entire path is encoded into one CRC32 hash and does not split the path by folder and filename. Outside of that, everything is identical to index.
Here is a sample file, displayed in hexadecimal, from the 7.0 Benchmark Character Creator: The colors represent the data sections related to this other view: The mechanisms to understand this view are detailed in the next sections.
The actual preset data follows. Using 's , here is the actual character data, in order. The full pattern parser file for imHex is available and can be opened in any text editor or ImHex.
bg/ex3/01_nvt_n4/twn/n4t1/bgparts/n4t1_a1_chr03.mdl
ex3
bg
music/ex2/bgm_ex2_system_title.scd
ex2
music
chara/weapon/w0501/obj/body/b0018/vfx/texture/uv_cryst_128s.atex
ffxiv
chara
0
common
Contains basic data like fonts, vulgar words dictionary, shader input textures
1
bgcommon
Contains textures, models, environments and collision that are shared between territories
2
bg
Contains layouts, definitions, collision, models and textures for specific territories
3
cut
Contains cutscene animations and definitions
4
chara
Contains models, textures and definition files for all humans/demihumans/monsters
5
shader
Contains compiled shaders
6
ui
Contains UI layouts and textures
7
sound
Contains sound effects
8
vfx
Contains textures and VFX definition files(AVFX)
9
ui_script
No index/dat in retail client, likely leftover from silverlight
A
exd
B
game_script
Contains compiled Lua scripts (5.1) for quests, cutscenes and battles
C
music
Contains BGM
12
sqpack_test
Category missing in retail client/no files.
13
debug
Category missing in retail client/no files.
Excel is a binary data format for storing relational data. Contains info for EXL, EXH and EXD files.
Excel is a binary data format for storing relational data. It's composed of 3 different files:
Excel List (.exl) is used to provide the excel index for the game, and allows certain sheets to be accessed by an ID rather than a name.
Excel Header (.exh) is used to define the data layout, variant, segmentation and available languages in the event of localisation being present.
Excel Data (.exd) is the raw data itself and just contains an offset table and then data for each row. The structure of this file changes depending on which variant is defined in the header.
Probably the simplest format in the entire game given it's just a text file.
The first line of the file is its 'header' and defines its type, EXLT and it's version following that. Each sequential line after the 'header' is a path, relative to the exd/ category, and its immutable ID. The immutable ID is optional and in cases where not relevant (eg. quest dialogue files) it's value is -1.
For example, here's an example of the content of a list file:
From these entries, you can build something like the following:
209
Achievement
exd/achievement.exh
4
Action
exd/action.exh
content/DeepDungeon2Achievement
exd/content/deepdungeon2achievement.exh
content/DeepDungeon2Gacha
exd/content/deepdungeon2gacha.exh
From here, the only thing you can do is parse headers as you will not be able to accurately figure out data paths without first reading the header due to language and row segmentation.
As a note, any excel entry in this file with an ID set get their header pre-cached by the game on startup.
The Excel Header defines the schema and how you read data files. The first structure in this file is it's header which contains information you'll need to parse the rest of the file.
This file in its entirely is in big endian. You will need to convert this file to little endian on applicable systems.
For example, in C#, BitConverter.IsLittleEndian will return whether or not a conversion needs to take place. See the Lumina source code for this file for a working example.
The magic is always EXHF. If it's not, the file is probably not the file you're trying to read.
DataOffset isn't relevant to this file at all, but is required when loading certain data from Excel Data files such as strings. It points to the end of the fixed size data of a row. For example, a row can be made of a bunch of integral types which have a known length at compile time, however a string length is variable. This offset allows you to then seek to the end of the row and access any additional data that may be on the end. If this is weird, don't worry, because it'll make sense later.
ColumnCount is how many columns exist within the file and is the first thing you'll read after the header. In other words, you'll need to read sizeof( ColumnDefinition ) * ColumnCount immediately after the header.
Offset is relative to the row offset, which you don't have until you read the offset table from a data file.
Type is the type of data stored. Most don't have any special handling outside of String and PackedBoolX but we'll cover that later.
PageCount is the number of 'pages' an excel data sheet is split into. Many sheets only have a single page and every row will exist in a single page, however larger sheets such as Quest or Item have many pages. This information is another structure and immediately follows the column definition data.
startId is the row id where a page starts. You'll need this to build file paths for data files.
rowCount is how many parent rows a sheet contains. As a quick example before we cover it more in depth later is that given a row id, you can calculate which page a row is on with row >= startId && row < startId + rowCount - 1.
Finally, the last thing you'll need to read out of a header file is the languages. These are needed to generate paths to data files along with the paging information.
You can read out the languages as is.
Used when reading data. Default requires no extra processing and you can just iterate over the offset list inside a data file. SubRows makes each row contain it's own rows. As a better example, it's like having a compound key on a database table. Instead of one column being the identifier for a row, you have two instead.
This is the total count of all rows across every page. The game uses this field when it internally queries for a row count.
Once you have the 3 pieces of critical information from the headers, namely:
The column count
The page count
The languages
You have everything you need to find data files! How exciting. The files follow one of 2 path formats, which makes this pretty easy. Make sure not to discard the data from the header, there's still some other information you'll need from it.
If the no language is set, your paths follow the following format:
Otherwise, if one or many languages are set (as in, not None), the following format applies:
This should be relatively obvious, but here's a few examples:
Item
0
English (en)
exd/item_0_en.exd
Item
10000
Japanese (ja)
exd/item_10000_ja.exd
Mount
0
French (fr)
exd/mount_0_fr.exd
Quest
65536
German (de)
exd/quest_65536_de.exd
The above also applies for quest sheets and so on which exist in subfolders.
The data file contains a single page of sheet data and as mentioned before, you will need the header file to read data correctly.
Similarly to Excel Headers, excel data files are entirely in big endian and will need to be converted to little endian on applicable systems.
The value of magic should always be EXDF.
indexSize is how large the row offset index is, in terms of total size. To convert that to a number of entries, you'd do indexSize / sizeof( ExcelDataOffset ).
Immediately following the ExcelDataHeader, the root row offsets are stored. The reason it's called the 'root row offset' is because on variant 2 sheets (or sheets with subrows), this offset won't point to data that can be read following the column data.
rowId is the absolute row id, so a simple way to map these in whatever you're doing is to convert the list of data offsets in this file to a key value map, where the key is the rowId and the value is the offset. Then you can directly index rowIds on any given data page.
offset is the absolute offset to where the data is located in the file. It can be used as is.
Once you seek to a row offset by following the offset list after the header, the first thing you'll encounter is the row header.
The dataSize is the entire size of the row, including any data for subrows (if they exist). You can use this field to copy out the exact amount of data for a row (or subrows) to then later parse if you choose not to do it in place.
rowCount is always 1 on variant 1 sheets and you can ignore that field entirely if you choose to. However, on variant 2 sheets, the rowCount is how many subrows belong to a row.
Therefore, on variant 1 sheets, immediately after the row header is your row data. You can then read columns directly out of the data given a column offset that you read from the header.
On the other hand, variant 2 sheets, following the header is the first subrow, or subrow 0. You can calculate the offset to n subrow with the following:
To clear up a few things:
The rowOffset is the offset from the offset list.
rowOffset + 6 is the skipping the size of the row header. You could just store the offset once you read the row header and use that.
header.dataOffset points to the end of the raw data of the row, or is basically the 'fixed' size of a row, not including strings. This lets you seek to the end of a row and immediately go to the next subrow.
There are some intricacies to reading certain types of columns which we'll cover in the next section.
Reading data out of a row depends a bit on the type of the column that you're reading. Most don't require any additional logic and you can read them out of the row and use them as is.
All of the types are as follows:
As mentioned previously, variable length data is stored past the end of the fixed length data segment of a row. So to access a string, you need to read a uint32 where there string column is. You can then obtain the offset of the string by doing the following:
Where rowOffset is the offset of the first column of a row, the dataOffset from the header and the uint32 that you read out from the column.
Packed bools are always 8 bits/1 byte long. To each type is basically which bit you need to bitwise and against to read the correct bool out.
A simple way of doing this for all of them is something like the following:
Alternatively, you can handle each one indivdually, but it's better to be lazy and do it the lazy way instead.
Read and use it as is. It just works™ - no magic required. Just make sure the endianness is correct.
(TODO)
ZiPatch(.patch) is a file format used to update online games made by SQEX.
We'll only cover version 3 for now.
This patch format has been extant since FFXIV 1.0, but has had a few modifications since then, especially from 2.0 onward.
A patch file name generally starts with either H or D, for HIST and DIFF, respectively, followed by a version string, such as 2013.08.09.0001.0000. This is optionally followed by a lowercase letter (starting at 'a') in case the patch file corresponding to that version string would be larger than 1.5GB for HIST files. e.g.: H2013.08.09.0001.0000a.patch, D2019.05.29.0000.0000.patch.
Patch files follow the following naming convention:
Where D|H indicates that a patch is a delta patch or a history patch respectively.
Some example patch names are as follows:
Byte order, unless explicitly mentioned, is big endian order. That is, 0x00AB_CDEF in hexadecimal will be encoded as 00 AB CD EF in file - not EF CD AB 00 which is little endian.
All patch files start with 91 5A 49 50 41 54 43 48 0D 0A 1A 0A.
The ZiPatch signature is followed by a contiguous array of chunks with the following structure
Where Header.Size indicates the size of Chunk.Payload, and Header.Name is a 4-character string indicating the type of the chunk, e.g., "FHDR", "APLY", and "SQPK".
Following Header, there's a Chunk describing the chunk contents, which will be detailed in the following pages. Following the chunk payload, there is a CRC32 of Header.Name with Payload.
This contiguous array of chunks terminates with a chunk of Name "EOF_".
FHDR is always the first chunk in the file, containing a few unused fields and some information about the current patch file (such as number of commands, etc.)
FhdrChunk.Name has been observed to be HIST for history patches, or DIFF for delta patches.
The two currently known Options, Ignore Missing and Ignore Mismatch are normally set to false via two APLY chunks set right after the FHDR. This means that the official patcher fails if a file it expects there is missing or some data it expects to be there is not.
Creates directory at Path. Path is relative to the game folder plus the patch type being applied (boot or game).
e.g.: If Path = "movie\ex1", and we are patching 'game' at "C:\Games\SquareEnix\FINAL FANTASY XIV - A Realm Reborn\", we would create the folder "ex1" at location "C:\Games\SquareEnix\FINAL FANTASY XIV - A Realm Reborn\game\movie\"
Similar to ADIR, but the folder at Path is deleted instead. It only deletes empty folders.
Indicates no chunks should be processed after this.
Internal ping calculation
An IPC message is sent by the client which includes the timestamp encoded as a 32-bit integer value.
An IPC message is sent back by the server with the same timestamp.
The client generates a new timestamp, divides it by 10,000, and then computes the difference between them to get the RTT for the connection.
SqpkChunk is a 'container' chunk which executes one of several operations in SqpkOperation. These will be detailed in the following subsections.
Important to note that any fields prepended with "Block" will mean that the number is in blocks, or units of 128-bytes. e.g., BlockOffset means that the offset in bytes would be calculated by BlockOffset << 7 or BlockOffset * 128.
These operations often contain fields indicating which file the operation will apply to. These adhere to the following structure:
When referring to block headers, the following struct will be relevant:
This operation writes SqpkAddData.Data to the dat File, at BlockOffset. It will then write BlockDeleteCount blocks of empty (0x00) 128-bytes.
This operation writes BlockCount empty (0x00) 128-byte blocks at BlockOffset, with the first of those blocks containing a BlockHeader struct with:Size = 128, Type = FileSize = UsedBlocks = 0, TotalBlocks = BlockCount
If Delete Data encounters EOF while writing or seeking to BlockOffset, it will fail.
Expand Data follows the same structure as Delete Data, but the behaviour differs. When Expand Data seeks past the end of file or writes past the end of file, it does not fail, since this is its intended use.
Notes:
FilePath is a null terminated utf-8 string. Length of the string is designated by FileHeader.FilePathSize
Notes:
If blkHeader.Compressed is true then the payload is a deflated stream.
If blkHeader.Compressed is false then the payload is a raw bytes stream.
Yes, AlignedBlockSize is very dodgy. Apparently they are rounded up to next multiple of 128 except they're not.
There are no good ways to determine how many blocks are actually encoded in SqpkFileBlocks.
Usually the last block won't be compressed but keeping track of payload.remaining() and stop if it's all read will be more robust against a forged file.
This is currently no-op.
This is currently no-op.
Game packet structures
Game packets are composed of 3 distinct parts, the packet header, the segment header and a (sometimes) optional IPC header.
Only present when the parent segment type is set to SEGMENTTYPE_IPC.
Decoding packets is reasonably simple and assuming you have a buffer that you write data in to for each connection, it's something like the following:
A lot of detail is omitted for brevity, but it's generally pretty straightforward.
A more comprehensive example of packet parsing can be found in Sapphire:
The client uses to generate an arbitrary timestamp, which it converts roughly into milliseconds by dividing it by 10,000.
(and some container-level parsing)
magic
A magic value that identifies a packet.
This is FF14ARR if you read it both in its hexadecimal and ASCII representation at the same time.
timestamp
The number of milliseconds since the unix epoch when the packet was sent.
size
The size of the entire packet including its segments and data.
connectionType
The connection type. This will be 1 for zone channel connections and 2 for chat. This is only sent on the initial connection now, previously this was sent with every packet but that is no longer the case.
unknown_20
Alignment, most likely.
isCompressed
Whether the segments + remaining data is compressed. The header is always uncompressed. This data is compressed with zlib and there is no header - default compression settings.
unknown_24
Alignment, most likely.
size
The size of this segment and its data (if any).
source_actor
The actor ID of the actor who effectively caused this packet to be sent.
For example, if another player casts an action, the source_actor field will contain their actor ID.
target_actor
The actor ID of the actor who is affected by the packet.
This isn't used consistently, but the same logical rules apply as source_actor.
type
The type of segment, see below for more detail. Based on the value of this field indicates what kind of data you'd expect to find after the segment header.
paddding
Alignment.
SEGMENTTYPE_SESSIONINIT
Used to login to a world or chat server.
The packet that has a segment that has a type set to this will contain a correct connectionType set in the packet header. Use this to record what kind of connection it is.
Example implementation is in Sapphire.
SEGMENTTYPE_IPC
Used for segments that contain data that should be handled by the packet router for the associated channel.
Chat messages, using actions and etc. will always be sent via this segment type and there will be a FFXIVARR_IPC_HEADER immediately following the segment header.
SEGMENTTYPE_KEEPALIVE
TODO: can't remember where this is actually used - lobby? As a note, world (and chat?) use IPCs to ping/pong. Because reasons.
SEGMENTTYPE_ENCRYPTIONINIT
Used to initialize Blowfish for the lobby channel. The client sends a packet to lobby with it's key phrase which is then used to """secure""" a lobby session (spoiler alert: it's not secure).
reserved
type
This will contain the opcode of the packet which identifies which hanadler the packet data following this packet should go.
padding
Potentially data here and not padding but it's probably not that important. Right?
serverId
TODO: write about retail server architecture.
timestamp
A Unix timestamp in seconds since the epoch. Not really sure why this exists here but it does and that's what it has in it.
padding1
Alignment.
(TODO)
Animation lock is an internal timer that player has to wait certain amount of time before they are allowed to use any actions.
Animation lock is an internal timer that player has to wait certain amount of time before they are allowed to use any actions.
Timer counts toward zero. (i.e. it represents remaining time)
Many GCDs have 0.1s animation lock. This timer is while casting. It is resumed after player finishes casting on client side.
oGCDs and instant skills have 0.5s initial animation lock.
Server can also set animation lock in response to actions used from the client. (usually by the packet commonly called "effect packet") However, it simply discards any previous timer running. Therefore, game does not compensate for player's ping. (i.e. time already spent during initial animation lock)
You can't target ground while animation lock is active.
When those three above combined, it can greatly deter game-play experience in certain condition. This will be discussed below.
Some actions like limit break, foods and duty actions may lock for different amount of time. For example, movement skill locks for 0.8 seconds and items have 2.0 seconds.
Since animation lock from the server overwrites curent animation lock to new value, it is this reason why double weaving between GCDs extremely unliable when player is connected to the datacentre across continent.
If the client fail to receive response during initial animation lock, client is allowed to use next action. TODO.
While Swiftcast is one of notorious offender, this property is not limited to Swiftcast only. For example, Red Mage's Dualcast just works same as hardcasting Swiftcast.
TODO: rescue lb3
TODO: it is possible to cancel oGCD 0.6s animlock by letting GCD overwrites it on high ping environment. (something something related to hardcasting swiftcast)
In most zones, weather is pseudorandom and can be derived from the current time. Each zone has a weather table, which contains the possible weather states for the zone and the stacked probability of that weather being active.
The value that the pseudorandom number generator produces is referred to as the target value. This value can be calculated deterministically for any given instant. It will always be in the range [0, 99].
Given a UNIX timestamp unixSeconds:
Once the target value has been calculated, the WeatherRate row for the current TerritoryType is retrieved. The rates of each entry in the row are accumulated until the target value is less than the current value of the rate accumulator. At this point, the current weather rate entry is selected, and the corresponding weather ID denotes the Weather row that should be used.
Territories can be queried by name using their PlaceName attribute. Territories have a WeatherRate that describes the weather rate row they use. Using the process described above, the current Weather being used can be determined.
Some zones, such as the original version of Diadem, use the WeatherChange message to control what weather the player experiences. It is not known what zones currently use this message for weather.
RSF is very similar to RSV except it works on file level.
The procedure to generate a RSF file is as follows:
RSV is a placeholder value used for masking the actual value until the player is actually zoned in.
The packet sent from the server upon entering the instance is as follows:
Few things to note:
Just like Super Mario 64, y component of coordinate represents altitude. Think z component as depth as in Z-buffer.
(TODO)
Upon launching the game, the retail launcher passes handful of information (e.g. login token, client language, etc...) to the game that usually looks like this:
Encrypt UTF-8 plain text using Blowfish; you need to pad the buffer with zero as needed
Key is GetTickCount() & 0xFFFF_0000
Block cipher mode of operation is ECB (yes, you read it right.)
Convert encrypted bytes to Base64URL
Format it as //**sqex0003{base64}{checksum}**//
