Data Migration

When the chain needs to be upgraded after its runtime logic is modified, it may involve changes in the data storage structure. In this case, users need to define a data migration logic to store the old data in the new structure.

Storage migration

Storage migrations are user-defined, one-time functions that allow developers to rework existing storage in order to convert it to conform to updated expectations. For instance, imagine a runtime upgrade that changes the data type used to represent user balances from an unsigned integer to a signed integer - in this case, the storage migration would read the existing value as an unsigned integer and write back an updated value that has been converted to a signed integer. Failure to perform a storage migration when needed will result in the runtime execution engine misinterpreting the storage values that represent the runtime state and lead to undefined behavior.

Framework implementation

FRAME storage migrations are implemented by way of the OnRuntimeUpgrade trait, which specifies a single function on_runtime_upgrade. This function provides a hook that allows runtime developers to specify logic that will run immediately after a runtime upgrade but before any on_initialize function has executed.

The logic and complexity of each migration vary in the needs:

  1. When mitigating deprecated storageMaps, you need to define generate_storage_alias! macros for the old storageMaps as follows: 

generate_storage_alias!(
		Audit,
		UnVerifyProof<T: Config> => Map<
            (Blake2_128Concat, T::AccountId),
            BoundedVec<OldProveInfo<T>, T::ChallengeMaximum>
        >
);

Audit is the name of the pallet, UnVerifyProof is the name of the Storage, and the Map corresponds to the StorgaeMap type. The generic parameters in the Map are the old primary keys and values.

  1. Use version control to make migration processing more secure. Declare the version in Pallet as follows:

const STORAGE_VERSION: StorageVersion = StorageVersion::new(1);

Then use the #[pallet::storage_version()] macro to declare the version for the current pallet:

#[pallet::pallet]
#[pallet::storage_version(STORAGE_VERSION)]
#[pallet::generate_store(pub(super) trait Store)]
pub struct Pallet<T>(_);

Through version comparison, call the corresponding migration logic:

pub fn migrate<T: Config>() -> Weight {
	let version = StorageVersion::get::<Pallet<T>>();
	let mut weight: Weight = 0;

	if version < 1 {
        log::info!("Audit version 1 -> 2 migrations start!");
        weight = weight.saturating_add(v2::migrate::<T>());
        StorageVersion::new(2).put::<Pallet<T>>();
	}

	weight
}

  1. The order of migration execution will in the order in which the pallets appear in construct_runtime! macro. The order in which the upgrade occurs is in reverse order.

  2. Declare the final migration object in runtime and execute it in the next upgrade:

pub type Executive = frame_executive::Executive<
	Runtime,
	Block,
	frame_system::ChainContext<Runtime>,
	Runtime,
	AllPalletsWithSystem,
	(here is implemented migration structure)
>;

Testing Migrations

It is important to test storage migrations and a number of utilities exist in Substrate to assist in this process:

  1. The Substrate Debug Kit includes a Remote Externalities tool that allows storage migration unit testing to be safely performed on live chain data.

  2. The fork-off-substrate script makes it easy to create a chain specification that can be used to bootstrap a local test chain for testing runtime upgrades and storage migrations.

In the OnRuntimeUpgrade trait, two functions pre_upgrade and post_upgrade are defined for the use of testing.

For a more specific approach, please refer to the official Substrate migration examples.

PreviousSmart ContractNextOffchain Worker