Ethereum Virtual Machine — Storage layout

Introduction

In the earlier part 1 post, we describe EVM as a stack-based virtual machine and how Solidity compiles down to ByteCode which is a set of opcodes that will be executed by EVM. This post will describe how EVM stores data in storage and memory.

Type of data location

There are in total 3 types of data locations: memory, calldata and storage.

1. Memory

This is used to hold temporary values and only exists within the scope of the function call. However, they are mutable within the function call.

2. Calldata

Like memory, this is used to hold temporary values and erased between function calls. However, it is immutable within function call and is slightly cheaper in gas cost as it skipped copying the values into memory sload and sstore and read directly from call data (cheaper in gas, see calldataXxx opcodes).

Calldata can only be specified in function argument.

function limit(calldata string issue) {
}

3. Storage

Each smart contract maintains its own storage. This persistent storage is basically a key-value mapping with 2 ²⁵⁶ keys mapped to each value of 32 bytes. Smart contract can read or write from any particular slot. These data will be permanently stord on blockchain and retrievable on subsequent operation.

Credits to https://noxx.substack.com/p/evm-deep-dives-the-path-to-shadowy-3ea for the image

All state variables are stored in storage. Example below

Contract HelloWorld {
  uint256 a;  // in storage
  addresss b; // in storage
}

Where is storage stored?

State variables are arranged in a compact continuous manner and if possible (within 32 bytes value), they can occupy the same storage slot. For example, if there are continuous bool variables (bool variable only occupies 1 byte), they could all occupy the same slot.

// one slot can slot 32 bytes of value and solidity will try to pack
Contract HelloWorld {
  uint256 apple;                         // in slot 0
  address pear;                          // in slot 1
  mapping(address => uint256) banana;    // in slot 2 
  bool xx1;                              // in slot 3 - 1 byte
  bool xx2;                              // in slot 3 - 1 byte
  bool xx3;                              // in slot 3 - 1 byte
  uint8 xx4;                             // in slot 3 - 1 byte 
  bytes16 xx5;                           // in slot 3 - 16 bytes
  uint128 xx6;                           // in slot 4
}

Here a high level of storage size for each type

// uint
uint8   -> 1 byte
uint16  -> 2 byte
...
uint128 -> 16 bytes
uint256 -> 32byte

// bytes
bytes1  -> 1 byte
bytess. -> 2 bytes
...
bytes32 -> 32 bytes

// bool and address
bool    -> 1 bytes
address -> 20 bytes

What about string?

It really depends if the string is less than 32 bytes in length.

Short string (31 bytes and below):

contract StringDemo {
  uint256 totalSupply;    // slot 0
  string name = "Jeremy"  // slot 1
}

When a string is 31 bytes and below, the entire string will only occupy one slot. Thus slot id 1 will contain this value 0x4a6572656d79000000000000000000000000000000000000000000000000000c where
4a6572656d79 → Jeremy and c → 12 represent the length of the bytes.

Long string (above 31 bytes)

pragma solidity ^0.8.0;

contract StringDemo {
  uint256 totalSupply;    // slot 0
  string name;            // slot 1

  constructor() {
        name = "JeremyJeremyJeremyJeremyJeremyJeremyJeremyJeremyJeremyJeremyJeremyJeremy";
    }
}

When a string is above 31 bytes, it will span into multiple slots. The storage layout would map to the below

{
 "0xd8b934580fcE35a11B58C6D73aDeE468a2833fa8": {
  "0xea7809e925a8989e20c901c4c1da82f0ba29b26797760d445a0ce4cf3c6fbd31": {
   "key": "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf7",
   "value": "0x72656d794a6572656d794a6572656d794a6572656d794a6572656d794a657265"
  },
  "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6": {
   "key": "0x0000000000000000000000000000000000000000000000000000000000000001",
   "value": "0x91"
  },
  "0xb32787652f8eacc66cda8b4b73a1b9c31381474fe9e723b0ba866bfbd5dde02b": {
   "key": "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf8",
   "value": "0x6d794a6572656d79000000000000000000000000000000000000000000000000"
  },
  "0xb5d9d894133a730aa651ef62d26b0ffa846233c74177a591a4a896adfda97d22": {
   "key": "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6",
   "value": "0x4a6572656d794a6572656d794a6572656d794a6572656d794a6572656d794a65"
  }
 }
}

• slot 1 (0x91 -> 145) would contain the length of the string * 2 + 1 which is 72 * 2 + 1
• the first slot of the string would be in
  keccak256(slotId) → keccak256(0000000000000000000000000000000000000000000000000000000000000001) → b10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6
• The next slot would be +1 of the previous slot
• 3 slots to store the value, eg. the value of the 3 keys from f6 to f8
0x4a6572656d794a6572656d794a6572656d794a6572656d794a6572656d794a65 → JeremyJeremyJeremyJeremyJeremyJe
0x72656d794a6572656d794a6572656d794a6572656d794a6572656d794a657265 → remyJeremyJeremyJeremyJeremyJere
0x6d794a6572656d79000000000000000000000000000000000000000000000000 → myJeremy

What about other types like arrays, structs, or mapping?

These will always occupy a new slot because the size is unknown until it's assigned later in your contract and they cannot be stored with their data in between other state variables. Instead, they are assumed to take up 32 bytes, and the elements within them are stored starting at a separate storage slot that is computed using a Keccak-256 hash.

Arrays

Let's use the below as an example

pragma solidity ^0.8.0;

contract ArrayDemo {
    uint256 totalSupply;                    // slot 0
    address[] users;                        // slot 1

    constructor() {
        users.push(0x33A3FFd50C5805eF071380bDEbe76aEa8DFE248C);
        users.push(0x75cdA57917E9F73705dc8BCF8A6B2f99AdBdc5a5);
    }

    function getUsers(uint256 _index) public view returns (address) {
        return users[_index];
    }
}

You can do the

1. Copy the above MappingDemo over, compile and eploy
2. Try calling getUsers(0)
3. Click the debug button beside the call. You will see a debug window and one of it is Full Storage Changes — this will list down the current contract’s storage state.

// Copied into JSON
{
 "0xf8e81D47203A594245E36C48e151709F0C19fBe8": {
  "0xea7809e925a8989e20c901c4c1da82f0ba29b26797760d445a0ce4cf3c6fbd31": {
   "key": "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf7",
   "value": "0x75cda57917e9f73705dc8bcf8a6b2f99adbdc5a5"
  },
  "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6": {
   "key": "0x0000000000000000000000000000000000000000000000000000000000000001",
   "value": "0x02"
  },
  "0xb5d9d894133a730aa651ef62d26b0ffa846233c74177a591a4a896adfda97d22": {
   "key": "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6",
   "value": "0x33a3ffd50c5805ef071380bdebe76aea8dfe248c"
  }
 }
}

When it comes to the array, Solidity stores the following

1. at the slot id of the array variable — store the length of the array
2. For first value in the array, store it as below: keccak256(slot_number)
   keccak256(0000000000000000000000000000000000000000000000000000000000000001) →
   b10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6this would be the slot id for the first element in the array.
   To get the second element, simply add 1 to the slot id.
   b10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6 + 1 →
   b10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf7

Mapping

Let’s use the below as an example

pragma solidity ^0.8.0;

contract MappingDemo {
    uint256 totalSupply;                    // slot 0
    mapping(address => uint256) approvals;  // slot 1

    constructor() {
        approvals[0x33A3FFd50C5805eF071380bDEbe76aEa8DFE248C] = 10;
        approvals[0x75cdA57917E9F73705dc8BCF8A6B2f99AdBdc5a5] = 20;
    }

    function getApprovals(address _address) public view returns (uint256) {
        return approvals[_address];
    }
} 

You can do the same as above — using remix, deploy and run the getApprovals function. The debug window should open up and the Full Storage Changes should display the current slot -> value in the contract’s storage.

// copied into JSON
{
 "0xd8b934580fcE35a11B58C6D73aDeE468a2833fa8": {
  "0x1e4e158014a9a1febc73ef5d19b66d843f6aa90ea6118e3657216f786fb47c9a": {
   "key": "0x4ab1da6bef620abdbfad2d7e5ea513e8b321d569c98e4e1c6e49203dfaa7be78",
   "value": "0x0a"
  },
  "0x3b5ff7fac7f97ce24ae48ba5b7ddb8cc01435fdf36c473226001b4b4a89f8a31": {
   "key": "0x131570861e5a8150e79ed6fb6473ca441e9f33013ec9bf95a91b2dec64802cdc",
   "value": "0x14"
  }
 }
}

We are focusing on the 2 keys -> values in there. You should notice the 2 values:0x0a → 10 and 0x14 → 20. Let’s start with exploring how the first key is derived: 0x4ab1da6bef620abdbfad2d7e5ea513e8b321d569c98e4e1c6e49203dfaa7be78

Step 1: lower case and pad the approval address
33A3FFd50C5805eF071380bDEbe76aEa8DFE248C (lowercase) →
33a3ffd50c5805ef071380bdebe76aea8dfe248c (pad to 32 bytes) →
00000000000000000000000033a3ffd50c5805ef071380bdebe76aea8dfe248c

Step 2: Append with the storage slot of the mapping variable
00000000000000000000000033a3ffd50c5805ef071380bdebe76aea8dfe248 (append) → 00000000000000000000000033a3ffd50c5805ef071380bdebe76aea8dfe248c0000000000000000000000000000000000000000000000000000000000000001

Step 3: Keccak256
00000000000000000000000033a3ffd50c5805ef071380bdebe76aea8dfe248c0000000000000000000000000000000000000000000000000000000000000001 → 4ab1da6bef620abdbfad2d7e5ea513e8b321d569c98e4e1c6e49203dfaa7be78

And here’s how you derive the storage slot of a mapping! Try doing the same for the other address as an exercise.

Structs

For struct, the slot index where it is declared will be reserved for the first value, the next slot for the second value, and so on. An example as below: You can see both owner and fee and pack in the same slot. This are usually gas optimisation you can apply as Solidity will load the entire struct usually from storage and it will thus cost lesser gas if lesser slots are loaded.

contract StructDemo {
  struct Item {
    uint256 price; // first slot -- 32 bytes
    address owner; // second slot -- 20 bytes
    uint8 fee;     // second slot -- 1 bytes
  }
}

Closing thoughts

I hope this give you an idea of Solidity storage layout and helps you in your gas optimization journey!