chunked_file.hpp File Reference

The chunk framework, vocabulary and engine in one header. More...

#include <meta>
#include <array>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <format>
#include <optional>
#include <span>
#include <type_traits>
#include <vector>
#include <welder/vocabulary.hpp>
#include <wowlib/core/buffer.hpp>
#include <wowlib/core/client_version.hpp>
#include <wowlib/core/error.hpp>
#include <wowlib/formats/common/annotations.hpp>
#include <wowlib/formats/common/entity_reflect.hpp>
#include <wowlib/formats/common/fourcc.hpp>
#include <wowlib/formats/common/string_block.hpp>
#include <wowlib/formats/common/validation.hpp>

Include dependency graph for chunked_file.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
struct	wowlib::formats::UnknownChunk
	A chunk the entity does not model, preserved verbatim for round-trip. More...
struct	wowlib::formats::JournalEntry
	One chunk encounter in file order — the write path replays the journal to reproduce the original byte layout exactly. More...
struct	wowlib::formats::ChunkExtras
	Round-trip bookkeeping common to every chunked entity: the encounter journal, unmodeled chunks, and stray trailing bytes. More...
struct	wowlib::formats::ChunkedFile< Derived >
	The serialization face of a chunked entity, mixed in CRTP-style: an entity struct E : ChunkedFile<E> carries the ChunkExtras bookkeeping and gains the read()/write() methods every file representation shares. More...
struct	wowlib::formats::ChunkBlob
	An unparsed chunk payload, preserved verbatim for round-trip. More...
class	wowlib::formats::Repeated< T, N >
	Storage for a chunk that may appear up to N times in one entity (MOTV texcoord sets, MOCV vertex-color layers). More...
struct	wowlib::formats::detail::RepeatedTraits< T >
	Trait: is T a Repeated<U, N> (a repeats-annotated member)? More...
struct	wowlib::formats::detail::RepeatedTraits< Repeated< U, N > >
class	wowlib::formats::detail::ChunkWriter
	Append-only chunk emission with size backpatching. More...

Namespaces
namespace	wowlib
namespace	wowlib::formats
namespace	wowlib::formats::detail

Concepts
concept	wowlib::formats::ChunkedEntity
	A type the chunk serializer can read and write: carries the round-trip bookkeeping and knows which client version it is laid out for.
concept	wowlib::formats::SelfSerializing
	A member type that owns its chunk-payload encoding: the serializer hands it the raw payload on read and the output buffer on write (StringBlock, ChunkBlob).

Functions
consteval std::uint32_t	wowlib::formats::detail::chunkMagicOf (std::meta::info member)
	The chunk() fourcc of member member, or 0 if it carries none.
template<typename E>
consteval auto	wowlib::formats::detail::writeOrder ()
	The member indices (into membersOf<E>) in the order fresh entities emit their chunks.
std::unexpected< Error >	wowlib::formats::detail::chunkError (ErrorCode code, std::uint32_t fourcc, std::size_t offset, std::string_view what, FourCCEndian endian=FourCCEndian::Reversed)
	Build a chunk-scoped error value.
template<typename E>
consteval FourCCEndian	wowlib::formats::detail::unknownFourccEndian ()
	The id layout to display for chunks the entity has no spec for (garbage magics, unknown chunks): the entity's declared UnknownFourccEndian when present (the forward-magic M2 file family), else the reversed common case.
template<typename M>
Result< void >	wowlib::formats::detail::readValue (M &dst, std::span< const std::byte > payload, std::uint32_t fourcc, std::size_t offset, FourCCEndian endian)
	Read one chunk payload into member dst, dispatching on the member kind (nested entity, self-serializing, array, raw binary struct).
template<typename M>
Result< void >	wowlib::formats::detail::writeValue (const M &src, FileBuffer &out)
	Append member src's chunk payload to out (payload only — the caller emits the chunk header).
template<ChunkedEntity E>
bool	wowlib::formats::detail::entityEngaged (const E &entity)
	Does a never-journaled nested entity hold anything worth a chunk?
template<typename M>
bool	wowlib::formats::detail::engaged (const M &value)
	Is the member's chunk written when it never appeared in the journal?
template<ChunkedEntity E>
Result< void >	wowlib::formats::detail::readEntity (E &entity, std::span< const std::byte > data)
	Deserialize data into entity — the engine behind ChunkedFile::read(); see that method for the full contract.
template<ChunkedEntity E>
Result< void >	wowlib::formats::detail::writeEntity (const E &entity, FileBuffer &out)
	Serialize entity, appending to out — the engine behind ChunkedFile::write(); see that method for the full contract.
template<typename E>
consteval std::int32_t	wowlib::formats::detail::chunkMemberIndex (std::uint32_t magic)
	The flattened member index (a JournalEntry::member value) of the chunk member carrying magic, or -1 when no member does.
template<ChunkedEntity E>
std::vector< JournalEntry >	wowlib::formats::detail::freshJournal (const E &entity)
	The journal a FRESH write of entity would produce: one entry per engaged (or required) chunk member in canonical write order, repeated/ repeating members expanded to one entry per element, no unknown-chunk or trailing entries.

Detailed Description

The chunk framework, vocabulary and engine in one header.

Vocabulary: the round-trip bookkeeping every chunked entity carries (ChunkExtras), the ChunkedFile mixin that gives an entity its read()/write() serialization methods, and the member types the serializer dispatches on beyond plain structs and vectors — ChunkBlob and Repeated (StringBlock lives in its own header).

Engine: the reflection-driven chunk serializer behind ChunkedFile::read()/write(). detail::readEntity / detail::writeEntity walk a chunked entity's annotated members (template for over the reflected member list) and map them onto the fourcc+size chunk stream. Reading is chunk-order-independent — chunks dispatch by fourcc into the matching member — while every encounter is journaled so writing can replay the original order and reproduce the file byte for byte, unknown chunks and trailing bytes included. Fresh entities (empty journal) write their members in declaration order, which each entity declares in canonical client order.

Definition in file chunked_file.hpp.