Operator Service

Learn how to configure the Operator Service to become a node operator on StakeWise V3.

All node operators of StakeWise V3 must run Operator Service in order to successfully register validators on the platform. The below 4-step guide walks you through the initial setup of Stakewise Operator Service.

Prerequisites

Execution client

Ensure your execution node is fully synced and running. Any execution client that supports ETH Execution API specification can be used:

• Nethermind (Ethereum, Gnosis)

• Besu (Ethereum)

• Erigon (Ethereum, Gnosis)

• Geth (Ethereum)

• Reth (Ethereum)

Special configuration for Erigon and Reth

Reth

The required setting in the reth.toml is:

[prune.segments.receipts_log_filter.0x6B5815467da09DaA7DC83Db21c9239d98Bb487b5]
before = 18470089

Erigon

The required flags for Erigon: erigon --prune=receipts --prune.to=18470089

Consensus client

Ensure your consensus node is fully synced and running. Any consensus client that supports ETH Beacon Node API specification can be used:

• Lighthouse (Ethereum, Gnosis)

• Nimbus (Ethereum, Gnosis)

• Prysm (Ethereum)

• Teku (Ethereum, Gnosis)

• Lodestar (Ethereum, Gnosis)

Vault

You must have a deployed Vault. You can create a new Vault or use an existing one. To create a new Vault:

1. Go to Operate page.

2. Connect with your wallet in upper right corner, then click on "Create Vault".

3. Process vault setup step by step.

4. Once vault is deployed go to its page.

You can find the vault address either in the URL bar or in the "Contract address" field by scrolling to the "Details" at the bottom of the page. The vault address is used in the following sections.

Install Operator Service

Operator Service can be run via a binary, docker image, deployed on a Kubernetes cluster using the Operator Helm Chart, or built from source. Decide on your preferred method and follow the respective instructions below.

Binary

Head to the releases page to find the latest version of Operator Service. Identify the binary file specific to your node hardware, download and decompress it.

You will execute Operator Service commands from within the v3-operator folder using the below format (note that the use of flags is optional):

./operator COMMAND --flagA=123 --flagB=xyz

Install script (Linux and macOS)

To install a binary for the latest release, run:

curl -sSfL https://raw.githubusercontent.com/stakewise/v3-operator/master/scripts/install.sh | sh -s

The binary will be installed inside the ~/bin directory. Add the binary to your path:

export PATH=$PATH:~/bin

If you want to install a specific version to a custom location, run:

curl -sSfL https://raw.githubusercontent.com/stakewise/v3-operator/master/scripts/install.sh | sh -s -- -b <custom_location> vX.X.X

Docker Image

In the context of running an Operator Service through Docker, it is crucial to ensure that the --data-dir flag is used in all calls. This flag provides a directory path where essential data for the Operator Service is read from or written to, allowing for a persistent data storage mechanism across Docker container instances.

Pull the latest docker operator docker image:

docker pull europe-west4-docker.pkg.dev/stakewiselabs/public/v3-operator:v2.2.0

You can also build the docker image from source by cloning this repo and executing the following command from within the v3-operator folder:

docker build --pull -t europe-west4-docker.pkg.dev/stakewiselabs/public/v3-operator:v2.2.0 .

You will execute Operator Service commands using the format below (note the use of flags are optional):

docker run --rm -ti \
-u $(id -u):$(id -g) \
-v ~/.stakewise/:/data \
europe-west4-docker.pkg.dev/stakewiselabs/public/v3-operator:v2.2.0 \
src/main.py COMMAND \
--data-dir=/data \
--flagA=123 \
--flagB=xyz

Source Files

Build requirements:

• Python 3.10+

• Poetry

Clone this repo and install dependencies by executing the following command from within the v3-operator folder:

poetry install --only main

You will execute Operator Service commands from within the v3-operator folder using the below format (note that the use of flags is optional):

PYTHONPATH=. poetry run python src/main.py COMMAND --flagA=123 --flagB=xyz

Kubernetes (advanced)

A separate guide runs through the set-up of Operator Service via Kubernetes, designed to run large numbers of validators (up to 10,000). Visit the Kubernetes setup for more details.

Prepare Operator Service

In order to run Operator Service, you must first create keystores and deposit data file for your Vault's validators, and set up a hot wallet for Operator Service to handle validator registrations.

Operator Service has in-built functionality to generate all of the above, or you are free to use your preferred methods of generating keystores and deposit data file, such as via Wagyu Keygen, and your preferred tool for generating the hot wallet, such as MetaMask or MyEtherWallet.

Note, the deposit data file must be created using the Vault contract as the withdrawal address. You can find the Vault address either via the URL bar of your Vault page or in the "Contract address" field by scrolling to the "Details" section at the bottom of the Vault page.

The below steps walk you through this set-up using Operator Service:

Step 1. Create mnemonic

Run the init command and follow the steps to set up your mnemonic used to derive validator keys. For example, if running Operator Service from binary, you would use:

./operator init

Example output

Enter the network name (mainnet, holesky) [mainnet]:
Enter your vault address: 0x3320a...68
Choose your mnemonic language (chinese_simplified, chinese_traditional, czech, english, italian, korean, portuguese, spanish) [english]:
This is your seed phrase. Write it down and store it safely, it is the ONLY way to recover your validator keys.

pumpkin anxiety private salon inquiry ....


Press any key when you have written down your mnemonic.

Please type your mnemonic (separated by spaces) to confirm you have written it down

: pumpkin anxiety private salon inquiry ....

done.
Successfully initialized configuration for vault 0x3320a...68

Step 2. Create validator keys

Next, run the create-keys command to kickstart the deposit data and validator keystores creation process, making sure you have your newly created mnemonic to hand:

./operator create-keys

Example output

Enter the vault address: 0x3320a...68
Enter the number of the validator keys to generate: 10
Enter the mnemonic for generating the validator keys: pumpkin anxiety private salon inquiry ....
Creating validator keys:    [####################################]  10/10
Generating deposit data JSON    [####################################]  10/10
Exporting validator keystores    [####################################]  10/10

Done. Generated 10 keys for 0x3320a...68 vault.
Keystores saved to /home/user/.stakewise/0x3320a...68/keystores file
Deposit data saved to /home/user/.stakewise/0x3320a...68/keystores/deposit_data.json file

You may not want the operator service to have direct access to the validator keys. Validator keystores do not need to be present directly in the operator. You can check the remote signer or Hashicorp Vault guides on how to run Operator Service with them.

Remember to upload the newly generated validator keys to the validator(s). For that, please follow a guide for your consensus client. The password for your keystores is located in the password.txt file in the keystores folder.

Step 3. Create hot wallet

Run the create-wallet command to create your hot wallet using your mnemonic (note, this mnemonic can be the same as the one used to generate the validator keys, or a new mnemonic if you desire).

./operator create-wallet

Example output

Enter the vault address: 0x3320a...68
Enter the mnemonic for generating the wallet: pumpkin anxiety private salon inquiry ...
Done. The wallet and password saved to /home/user/.stakewise/0x3320a...68/wallet directory. The wallet address is: 0x239B...e3Cc

Note, you must send some ETH (or xDAI for Gnosis) to the wallet for gas expenses. Each validator registration costs around 0.01 ETH with 30 Gwei gas price. You must keep an eye on your wallet balance, otherwise validators will stop registering if the balance falls too low.

Upload deposit data file to Vault

Once you have created your validator keys, deposit data file, and hot wallet, you need to upload the deposit data file to the Vault. This process connects your node to the Vault. Note, if there is more than one node operator in a Vault, you first need to merge all operator deposit data files into a single file (use the merge-deposit-data command). Uploading the deposit data file can be achieved either through the StakeWise UI or via Operator Service and can only be done by the Vault Admin or Keys Manager.

StakeWise UI

1. Connect with your wallet and head to the Operate page.

2. Select the Vault you want to upload the deposit data file to.

3. In the upper right corner, click on "Settings" and open the "Deposit Data" tab. The "Settings" button is only visible to the Vault Admin or Keys Manager.

4. Upload the deposit data file either by dragging and dropping the file, or clicking to choose the file via your file browser.

5. Click Save and a transaction will be created to sign using your wallet. The Vault's deposit data file will be uploaded when the transaction is confirmed on the network.

Operator Service

If for some reason uploading deposit data using UI is not an option. You can calculate deposit data Merkle tree root with the following command:

./operator get-validators-root

Enter the vault address: 0xeEFFFD4C23D2E8c845870e273861e7d60Df49663
The validator deposit data Merkle tree root: 0x50437ed72066c1a09ee85978f168ac7c58fbc9cd4beb7962c13e68e7faac26d7

Finally, upload the Merkle tree root to your Vault contract by calling setValidatorsRoot. Below shows the steps to do this via Etherscan, but the same can be achieved via CLI if you prefer ( using eth-cli and eth contract:send for example). Note, the ABI of the contract can be found here.

1. Head to your Vault's contract address page on Etherscan in your browser (e.g. replacing 0x000 with your Vault contract address: https://etherscan.io/address/0x000).

2. Select the Contract tab and then Write as Proxy. If you don't have Write As Proxy option, click on the Code tab, then More Options, Is this a Proxy?, Verify, Save. Now you should have Write As Proxy option.

3. Connect your wallet to Etherscan (note this must be either the Vault Admin or Keys Manager).

4. Find the setValidatorsRoot function and click to reveal the drop-down.

5. Enter your Merkle tree root returned from the command and click Write.

6. Confirm the transaction in your wallet to finalize the deposit data upload to your Vault

Run Operator Service

You are ready to run the Operator Service using the start command, optionally passing your Vault address and consensus and execution endpoints as flags.

If you did not use Operator Service to generate hot wallet, you will need to add the following flags:

• --hot-wallet-file - path to the password-protected .txt file containing your hot wallet private key.

• --hot-wallet-password-file - path to a .txt file containing the password to open the protected hot wallet private key file.

If you did not use Operator Service to generate validator keys, you will need to add the following flag:

• --keystores-dir - The directory with validator keys in the EIP-2335 standard. The folder must contain either a single password.txt password file for all the keystores or separate password files for each keystore with the same name as keystore, but ending with .txt. For example, keystore1.json, keystore1.txt, etc.

If you did not use Operator Service to generate deposit data file, or you use combined deposit data file from multiple operators, you will need to add the following flag:

• --deposit-data-file - Path to the deposit data file (Vault directory is default).

Binary

You can start the operator service using binary with the following command:

./operator start \
    --vault=0x000... \
    --consensus-endpoints=http://localhost:5052 \
    --execution-endpoints=http://localhost:8545

Docker

For docker, you first need to mount the folder containing validator keystores and deposit data file generated into the docker container. You then need to also include the --data-dir flag alongside the start command as per the below:

docker run --restart on-failure:10 \
-u $(id -u):$(id -g) \
-v ~/.stakewise/:/data \
europe-west4-docker.pkg.dev/stakewiselabs/public/v3-operator:v2.2.0 \
src/main.py start \
--vault=0x3320ad928c20187602a2b2c04eeaa813fa899468 \
--data-dir=/data \
--consensus-endpoints=http://localhost:5052 \
--execution-endpoints=http://localhost:8545

Source Files

PYTHONPATH=. poetry run python src/main.py start \
--vault=0x000... \
--consensus-endpoints=http://localhost:5052 \
--execution-endpoints=http://localhost:8545

Congratulations, you should now have Operator Service up and running and ready to trigger validator registrations within your Vault!

Other commands

Operator Service has many different commands that are not mandatory but might come in handy.

Add validator keys to Vault

You can always add more validator keys to your Vault. For that, you need to generate new validator keys and deposit data as described in Step 2. Create validator keys and upload the deposit data file to your Vault as described in Step 3. Upload deposit data file to Vault. Note, uploading a new deposit data file will overwrite the existing file and consequently overwrite previously un-used validator keys. It can be done at any point, but only by the Vault Admin or Keys Manager.

Validators voluntary exit

The validator exits are handled by oracles, but in case you want to force trigger exit your validators, you can run the following command:

./operator validators-exit

Follow the steps, confirming your consensus node endpoint, Vault address, and the validator indexes to exit.

Example output

Enter the comma separated list of API endpoints for consensus nodes: https://example.com
Enter your vault address: 0x3320ad928c20187602a2b2c04eeaa813fa899468
Are you sure you want to exit 3 validators with indexes: 513571, 513572, 513861? [y/N]: y
Validators 513571, 513572, 513861 exits successfully initiated

Update Vault state (Harvest Vault)

Updating the Vault state distributes the Vault fee to the Vault fee address and updates each staker's position. If an ERC-20 token was chosen during Vault creation, the Vault specific ERC-20 reprices based on the rewards/penalties since the previous update and the Vault fees are distributed in newly minted ERC-20 tokens.

By default, each Vault state gets updated whenever a user interacts with the Vault (deposit, withdraw, etc.), with a 12 hours cooldown. Vault state can also be updated by the Vault operator(s) by passing the --harvest-vault flag to the Operator Service start command. Harvest occurs every 12 hours and the gas fees are paid by the hot wallet linked to the Operator Service.

Harvesting the Vault rewards simplifies the contract calls to the Vault contract and reduces the gas fees for stakers, for example, the Vault does not need to sync rewards before calling deposit when a user stakes.

Merge deposit data files from multiple operators

You can use the following command to merge deposit data file:

./operator merge-deposit-data

Recover validator keystores

You can recover validator keystores that are active.

Make sure there are no validators running with recovered validator keystores and 2 epochs have passed, otherwise you can get slashed. For security purposes, make sure to protect your mnemonic as it can be used to generate your validator keys.

./operator recover

Example output

Enter the mnemonic for generating the validator keys: [Your Mnemonic Here]
Enter your vault address: 0x3320ad928c20187602a2b2c04eeaa813fa899468
Enter comma separated list of API endpoints for execution nodes: https://example.com
Enter comma separated list of API endpoints for consensus nodes: https://example.com
Enter the network name: goerli
Found 24 validators, recovering...
Generating keystores [####################################] 100%
Keystores for vault {vault} successfully recovered to {keystores_dir}

Optional extras

Environment variables

Operator Service can be configured via environment variables instead of CLI flags. Copy this example file and save it as .env. Run through the file, adjusting as and where necessary based on your node configuration.

Remember to load the environment variables before running Operator Service, for example:

export $(grep -v '^#' .env | xargs)

You can check the environment variables are set and loaded correctly by running env.

Max gas fee

To mitigate excessive gas costs, operators can pass the --max-fee-per-gas-wei flag when starting Operator Service (or configure this variable via Environment Variables) to set the maximum base fee they are happy to pay for both validator registrations and Vault harvests (if Operator is started using the --harvest-vault flag).

Reduce Operator Service CPU load

--pool-size can be passed as a flag with both start and create-keys commands. This flag defines the number of CPU cores that are used to both load keystores and create keystores. By default, Operator Service will use 100% of the CPU cores.

Setting --pool-size to (number of CPU cores) / 2 is a safe way to ensure that Operator Service does not take up too much CPU load and impact node performance during the creation and loading of keystores.