Written by
John Sloboda
Published on
April 28, 2024
Copy link

How to Set Up a Solana Validator

How to Set Up a Solana Validator

This post will focus on getting a Solana mainnet validator up and running from a technical perspective. We will set up tools and configs to make ongoing operations easier, but more advanced validator operations will be out of scope. Also, the economics of running a validator, including topics like attracting stake and applying for subsidies, will be out of scope for this post. As a brief aside, those interested in running a validator should use Cogent Crypto's Validator Profit Calculator to estimate a validator's earnings for a given hypothetical scenario.

This post assumes familiarity with basic system administration on Linux. You don’t have to be a programmer or a systems administration expert to run a validator, but you need to be comfortable running commands in the terminal, basic shell scripting, and working with configuration files.

If you don’t know much about Linux systems administration but would like to learn more, there are many great starting points, such as Linux Journey.

Terminology: What is a Solana Validator?

The Solana CLI tool suite contains the solana-validator binary, which can:

  • Connect to a Solana cluster, and sync the current cluster state
  • Produce, receive, and share snapshots and other cluster data to stay up-to-date
  • Vote to verify new blocks as part of the cluster consensus
  • Process incoming transactions and produce new blocks
  • Expose RPC and event-driven APIs to allow the cluster data to be queried and new transactions to be submitted

A Solana validator can also be called the client (since it connects to the network of validators and requests data), server (since it handles requests from and provides data to other validators), or node.

An RPC node is an unstaked, non-voting Solana validator that keeps track of all the network’s information. This means that they only respond to data requests and do not participate in consensus.

The term “Solana Validator” can also refer to an entity within the Solana ecosystem that runs a staked, voting Solana validator and participates in the consensus.

More information about the distinction between RPCs, RPC providers, and validators can be found here.

Starting Off: Get the Server Hardware

You will need a beefy computer dedicated solely to operating the validator and a second one if you want to run validators in both testnet and mainnet. Currently, the minimum recommendation is 12 cores/24 threads, 256GB RAM, 2x 1TB SSD disks (ideally w/ RAID0), and a 10GB internet connection. The hardware requirements can be found in more detail here.

If you are tempted to save costs by only running in mainnet, be aware that you must run a testnet validator to qualify for the Solana Foundation Delegation Program (SFDP).

You can purchase and operate the hardware yourself or rent hardware from a data center provider (we recommend TeraSwitch). You can also rent from a cloud computing provider, but we do not recommend this since it is often cost-prohibitive and more prone to performance issues. Solana Foundation Server Program aims to make it easier for validator operators to rent hardware from data center operators. Find out more here.

To help decentralize the network, you may want to consider ASNs and cities that do not already have a lot of Solana nodes or stake. Some staking pools and the SFDP will reward you based on decentralization by ASN and city.

The top current data centers can be found here.

Install the Solana CLI Locally

Having a local install of the tools is highly recommended, so you can generate all the necessary keys locally and keep the withdrawal key away from the validator server. Instructions on how to set up the Solana CLI locally can be found here.

Once the tools are available locally in your command line, configure them to point to testnet by default:


$ solana config set --url https://api.testnet.solana.com

For mainnet, you can point to the default endpoint here:


$ solana config set --url https://api.mainnet-beta.solana.com

The public mainnet endpoints are often oversaturated with requests. If you are not satisfied with the performance, you can sign up for Helius and use your Helius endpoint like so:


$ solana config set --url https://mainnet.helius-rpc.com/?api-key=<YOUR-HELIUS-API-KEY>

Setting up and bootstrapping the validator will require executing transactions, so you must have a wallet set up with some SOL in it. If you’re just getting set up with testnet, you can create a default wallet and airdrop yourself some SOL with the following commands:


$ mkdir -p ~/.config/solana
$ solana-keygen new --outfile ~/.config/solana/id.json
$ solana airdrop 1 ~/.config/solana/id.json

It is highly recommended to use different wallets for testnet and mainnet. Since the key format is the same, the same keys can be used on both, but resist the temptation to do this. If you use the same keys for funded wallets on testnet and mainnet, it is easy to execute commands intended for testnet on mainnet accidentally.

Read more about generating keys here, along with some more secure key-generation methods.

Set up Vote Account

Your validator’s vote account is created using the solana create-vote-account command and three keypairs:

  • Identity Keypair - The validator uses this key pair to identify itself to the network and submit transactions such as votes. The private key will need to live on the server, so the best security is to not keep any more SOL in it than is needed for voting transactions (roughly 1 SOL/day).
  • Vote Account Keypair - This key pair refers to the vote account itself, and is used to look up the vote account and delegate stake against it. The create-vote-account command only needs the private key to prove that you own the requisite key. After the vote account is created, only the public key is ever needed. This cannot be changed.
  • Withdraw Keypair - This key pair acts as a master key for the vote account, and is used to withdraw rewards, change the identity or withdraw key, and other account operations. This key is by far the most sensitive of the three and should not be generated on or copied to your server. For mainnet, consider using a paper wallet, a hardware wallet, or a multi-sig for this key.

Create the Keypairs

Create the keypairs for testnet as follows:


$ solana-keygen new -o identity.json
$ solana-keygen new -o vote.json
$ solana-keygen new -o withdraw.json

Optional: Use a Paper Wallet for the Withdraw Key

A paper wallet is a 12 (or 24) word phrase written down on a sheet of paper, then entered via keyboard whenever the key is needed to sign something. If you are not planning on setting up a multi-sig, the added security this provides is appropriate for the mainnet withdrawal key since it gives root access to your vote account. Run this command:


$ solana-keygen new --no-outfile

Write down the phrase that is shown on a piece of paper. You may also want to note the pubkey, the exact command you used to create the key, and the solana-keygen --version. Be aware that different schemes for converting the passphrase into a private key exist. Before you put the sheet away, verify that you can use it to access the key:


$ solana-keygen verify <PUBKEY> ASK

NOTE: as of this writing, the ASK and prompt:// placeholders have different key decoding behavior. Make sure you are aware of which one works for your key.

Absolutely do not lose the piece of paper with the key written down, as there is no other way to recover it. Don’t forget to clear the key phrase out of your terminal buffer.

Optional: Grind for a Vanity Key for Vote Account

Some validators have vanity pubkeys for identity or vote account keys. For example, the Helius validator has HEL1USMZKAL2odpNBj2oCjffnFGaYwmbGmyewGv1e2TU for our identity key, and he1iusunGwqrNtafDtLdhsUQDFvo13z9sUa36PauBtk for our vote account. You make these with the grind sub-command as follows:


$ solana-keygen grind --starts-with PREF1X:1

This sub-command will generate a key whose pubkey begins with PREF1X, which can be any valid base58 string you specify. This works by generating keys until it comes across one that matches the criteria, so it takes an exponential amount of time based on the length of the prefix you want.

The vote account key cannot be changed, especially once you advertise it and people start delegating stake to it, so if you want a vanity key, now is the time to do it.

Create the Vote Account from the Keypairs

Once the keypairs are ready to go, create the vote account itself as follows:


$ solana create-vote-account --fee-payer ~/.config/solana/id.json vote_account.json identity.json withdrawal.json

Here, vote_account.json is the vote account keypair, identity.json is the validator identity keypair, withdrawal.json is the withdrawal authority keypair, and ~/.config/solana/id.json is a wallet with SOL in it that will be used to pay for the vote account creation.

If you used a paper wallet for the withdrawal keypair, your command could look like this:


$ solana create-vote-account --fee-payer ~/.config/solana/id.json vote_account.json identity.json ASK

You can check on the vote account (or any vote account) with this command:


$ solana vote-account <PUBKEY>

At this point, the identity keypair will be needed to run the validator server, the withdrawal keypair will need to be kept secret and used only for vote account withdrawals and changes, and the vote account keypair will no longer be needed (just don’t forget the pubkey).

More info on vote accounts can be found here.

Optional: Set the Commission

The create-vote-account takes the commission as an optional parameter, but by default, it is 100%, implying your validator is private. If you want to attract external stake to your mainnet validator, you will likely need to choose a lower commission.

If our vote account pubkey was <VOTE_ACCT_PUBKEY>, we wanted to pay for the change from our id.json funded wallet, we had a paper wallet withdrawal key, and we wanted to set the commission to 8%, we could do it like so:


$ solana vote-update-commission --fee-payer ~/.config/solana/id.json <VOTE_ACCT_PUBKEY> 8 ASK

Optional: Add a Multi-sig

A multi-sig is a special type of wallet where the signatures of one or more other wallets are required in order to execute transactions, and can be used to share custody of the vote account between multiple people. Even if you manage the validator yourself, you may still want to consider a multi-sig to add others to the vote account later easily.

Squads is a multi-sig tool for Solana that comes with functionality specifically for managing shared validators. To set up the validator in Squads:

  1. Set up your browser with a supported Solana wallet plugin, such as Backpack
  2. Navigate to the Squads dApp
  3. Configure a squad with the desired people and parameters you’d like for your validator
  4. Navigate to Developers > Validators, and go through the flow to add the validator
  5. You will be asked to paste in the withdrawal private key, this will be used to change the existing withdrawal key over to the squad’s shared withdrawal key
  6. Verify the updated withdrawal pubkey using solana vote-account

More on using Squads for shared validator management can be found here.

Configure your Machine as a Validator Server

Your steps will vary depending on the starting point of your machine, but the goal is to end up with:

  • Ubuntu’s latest LTS version, installed and up-to-date
    • Other Linux distributions (i.e., Debian-based) may work as well, but all examples here assume Ubuntu; the further you go from Ubuntu, the more you will have to modify the commands
  • User-level SSH access configured
  • sol service user set up
  • System tuning and hard disks set up

System and User Setup

Make sure your system is up-to-date, and create the sol service user:


$ sudo apt update
$ sudo apt upgrade
$ sudo adduser sol

You may also want to give sudo access (this trades off some security for convenience):


$ sudo adduser sol sudo

System Tuning

Add the recommended settings from Solana Labs to sysctl and systemd to increase the limits on how many file descriptors, memory-mapped files, etc., are allowed. When your validator is the current or upcoming block leader, the entire network will be trying to open connections and send it transactions, so the file descriptor limit is particularly important.

The latest system tuning settings recommended by Solana Labs can be found here.

Hard Drive Configuration

The current recommendation from Solana Labs is that you should have 2 physical 1TB+ SSDs, one for accounts data and one for ledgers data (the O/S can also be on this disk). They do not recommend having accounts and ledgers on the same disk due to high IOPS.

Here is their guide on how you set up the 2 drives.

An alternative way of leveraging multiple drives to increase the available IOPS is to configure them together in a single RAID0 volume and let the RAID controller optimize which stripes are stored on what drive to maximize the IOPS capacity of both drives. This also makes it possible to add additional drives to the RAID to increase IOPS further if needed. We have not yet had any performance issues running a validator in this configuration.

If you are using a single RAID0 volume, all you need to do is set up the validator data directories on your volume, which can be done directly under the sol user home:


$ sudo su sol
$ mkdir -p /home/sol/accounts
$ mkdir -p /home/sol/ledger
$ mkdir -p /home/sol/snapshots
$ mkdir -p /home/sol/logs

Install the Solana CLI on the Validator Server

Repeat the steps above to install the Solana CLI, except this time on the validator server as the sol user. When installing to the validator server, building from source is highly recommended.

If you use the Solana install tool, you can use the solana-install command to perform future updates to new versions (which you must do to continue participating in the cluster).

Optional: Install the Jito Validator Client

Jito Labs has forked the Solana validator and released its own version, adding Maximal Extractable Value (MEV) capability. MEV is the idea of adding or reordering transactions within a block you are creating, in order to profit. Jito has created a market for people who want to pay a premium to do this (”searchers”). If you run the Jito validator, they will send you MEV transaction bundles for your validator to include in the blocks it creates in exchange for an extra tip.

The Jito validator adds some complexity and latency to your validator operations, but adds MEV tips as an additional revenue source. The configuration is mostly the same, except you will have to look into additionally:

  • Collecting and managing MEV tips
  • Latency from the Jito block engine
  • Possibly running a Jito relayer

Instructions to install Jito’s validator client can be found here.

Create Validator Run Scripts

Create validator.sh Shell Script

First, copy your validator identity keypair to the remote server:


$ scp identity.json remoteuser@your.validator.host:/home/sol

The standard way to manage the configuration of your validator is by wrapping it in a shell script. Create a script as a starting point and make it executable:


$ sudo su sol
$ cat >/home/sol/validator.sh <<EOF
#!/bin/bash
PATH=/home/sol/.local/share/solana/install/active_release/bin:$PATH
exec solana-validator \
--identity /home/sol/identity.json \
--vote-account <VOTE_ACCOUNT_PUBKEY> \
--known-validator 5D1fNXzvv5NjV1ysLjirC4WY92RNsVH18vjmcszZd8on \
--known-validator 7XSY3MrYnK8vq693Rju17bbPkCN3Z7KvvfvJx4kdrsSY \
--known-validator Ft5fbkqNa76vnsjYNwjDZUXoTWpP7VYm3mtsaQckQADN \
--known-validator 9QxCLckBiJc783jnMvXZubK4wH86Eqqvashtrwvcsgkv \
--only-known-rpc \
--log /home/sol/logs/solana-validator.log \
--accounts /home/sol/accounts \
--snapshots /home/sol/snapshots \
--ledger /home/sol/ledger \
--rpc-port 8899 \
--dynamic-port-range 8000-8020 \
--entrypoint entrypoint.testnet.solana.com:8001 \
--entrypoint entrypoint2.testnet.solana.com:8001 \
--entrypoint entrypoint3.testnet.solana.com:8001 \
--expected-genesis-hash 4uhcVJyU9pJkvQyS88uRDiswHXSCkY3zQawwpjk2NsNY \
--wal-recovery-mode skip_any_corrupted_record \
--limit-ledger-size
EOF
$ chmod +x /home/sol/validator.sh

These settings are appropriate to get up and running quickly and safely on testnet. Once everything is set up, you will need to come back to this script and run solana-validator --help to see the full list of configuration options and customize it.

Try bringing up the validator using the script:


$ ./validator.sh

The first time it comes up, it will need to catch up to the current state of the cluster.

Verify the Validator is Running Correctly

You can follow your (or anyone’s) externally visible progress from anywhere in the cluster using:


$ solana catchup <IDENTITY_PUBKEY>

You can see its progress internally from a session as the sol user with:


$ solana-validator --ledger /home/sol/ledger monitor

More info for checking on whether your validator is connecting to the cluster properly can be found here.

Set Up Validator as a Daemon Process

Create a systemd unit file to have it managed as a daemon:


$ sudo cat >/etc/systemd/system/sol.service <<EOF
[Unit]
Description=Solana Validator
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=sol
LimitNOFILE=1000000
LogRateLimitIntervalSec=0
ExecStart=/home/sol/validator.sh

[Install]
WantedBy=multi-user.target
EOF

Make sure your earlier ./validator.sh invocation is no longer running, then bring up the validator as a daemon:


$ sudo systemctl daemon-reload
$ sudo systemctl enable --now sol

Set Up Monitoring and Security

Disable Password-based Authentication

In your /etc/ssh/sshd_config file, the SSH daemon can be set to not accept password or challenge-response login methods:


...
PasswordAuthentication no
ChallengeResponseAuthentication no
...

Don’t forget to reload the sshd daemon with the new config:


$ sudo systemctl reload sshd

Add fail2ban and ufw

fail2ban will work out of the box to ban any connections that fail to authenticate multiple times:


$ sudo apt install fail2ban

ufw is a firewall that comes with the system and can be set up with a few commands (assuming your final validator config uses the default ports and port ranges and hosts SSH on port 22):


$ sudo ufw allow 22/tcp
$ sudo ufw allow 8000:10000/tcp
$ sudo ufw allow 8000:10000/udp
$ sudo ufw enable

Validator Identity Keypair

In practice, most validators keep their identity key on the validator server so the validator script can run and be restarted without human intervention. However, it is possible to use ASK or prompt:// to pass in the identity key to the validator so it does not have to be kept on the validator server’s file system. Doing this increases your operational overhead and risk, since a human now has to stand in for systemd and manually manage the daemon process, but it’s possible.

The best strategy for securing your identity keypair is to fund only with the minimum amount of SOL needed to cover a few days of voting costs.

Shinobi Systems has put out some tooling to help manage vote accounts, including automatic balance transfers out of the vote account without using the withdrawal key every time.

See the Solana Labs page on validator security for more tips.

Set Up Monitoring with Watchtower

solana-watchtower is included in the Solana CLI toolkit, and can be used to alert on issues with your validator and the broader cluster. Configuring and running it can be done similarly to the validator itself by making a watchtower.shscript.

Here is an example where it uses PagerDuty for alerting and writes its logs to watchtower.log:


$ cat >watchtower.sh <<EOF
#!/bin/bash
PATH=/home/solana/.local/share/solana/install/active_release/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

exec &gt;>watchtower.log \\
env PAGERDUTY_INTEGRATION_KEY=<PAGERDUTY_KEY> \\
solana-watchtower \\
--validator-identity <IDENTITY_PUBKEY> \\
--monitor-active-stake \\
--interval 20 \\
--minimum-validator-identity-balance 3 \\
--url https://api.testnet.solana.com
EOF

Solana Labs also has an example of how to set it up using Telegram for alerts.

Setting up watchtower on the same machine as your validator is not recommended because you will not get notifications if the entire machine goes down, but you may be able to get away with that if you have multiple forms of monitoring in place and can monitor for those cases separately.

Unlike the validator, watchtower is not resource intensive, and can be run from a cloud or serverless compute environment such as those offered by AWS or GCP.

Optional: Community-based Monitoring

There are many people within the Solana community that also monitor the cluster and are willing to provide alerting based on similar conditions offered by watchtower.

Stakewiz offers Telegram-based monitoring on Solana validators. For example, if you wanted alerting based on the Helius validator, you could navigate to its page, then click “+ Create Alert”.

Note that community-based monitoring is designed for people to be aware of issues with validators they stake with. As a validator operator you should not rely solely on others to monitor your services.

Optional: Custom Monitoring via getHealth RPC Method

If you would like to go straight to the source of your validator’s health, your validator’s JSON RPC interface has a method called getHealth that you can poll about whether your validator is healthy, and info about any problems.

Optional: Third Party Monitoring Solutions

Once your validator is scaled up and you have a revenue stream to protect, you may want to consider commercial-grade SaaS monitoring solutions such as Datadog or Splunk. These types of solutions usually work either by running a daemon process on your server, which monitors logs and metrics locally and exports them to the provider, or by making the getHealth or a similar status endpoint available and having the provider poll your validator remotely.

Add Stake to the Validator

The Solana network reaches consensus by a proof-of-stake (PoS) voting process, which means a validator’s voting weight is proportional to how much SOL stake is delegated to it, similar to how a stock corporation works if you think of staked SOL as shares. In addition to voting, the frequency at which your validator is the block leader is proportional to the amount of SOL staked to it.

There are many wallets, dApps, and other tools that make it easy to stake your SOL to existing validators, but here are the steps to do it manually with the CLI:

  1. Create 3 keypairs using solana-keygen new
  2. Create a stake account from the keypairs using solana create-stake-account
  3. Transfer some SOL into the stake account (can be done with solana aidrop for testnet)
  4. Delegate the deposited stake to a validator using solana delegate-stake
  5. Await the next epoch for the stake to activate, and check with solana stake-account

Create Keypairs and Stake Account

The process of creating the keypairs and stake account parallels creating the vote account, where the 3 keypairs are as follows:

  • Stake Authority Keypair - allows operational tasks on the stake account such as delegating, undelegating, splitting, merging
  • Stake Account Keypair - the public key identifying the stake account itself (the private key is not needed after the stake account is created)
  • Withdrawal Keypair - the master keypair for withdrawing stake and resetting authority keypairs on the stake account. Treat with care and consider a hardware wallet, paper wallet, or multi-sig

It is highly recommended to have separate keys for testnet and mainnet for the same reasons as the vote account. However, you can reuse a single stake authority and withdrawal authority keypair for multiple stake accounts within that cluster. Using the same keys for stake accounts is a prerequisite for merging them.

Here is an example of creating a stake account with 1 SOL on testnet using file-based wallets for the keypairs, funding it from your id.json wallet, and delegating that stake to your new validator:


$ solana-keygen new -o stake_auth.json
$ solana-keygen new -o stake_acct_1.json
$ solana-keygen new -o stake_withdrawal_auth.json
$ solana airdrop 1 ~/.config/solana/id.json
$ solana create-stake-account --from ~/.config/solana/id.json stake_acct_1.json 1 --stake-authority stake_auth.json --withdraw-authority stake_withdraw_auth.json --fee-payer ~/.config/solana/id.json
$ solana delegate-stake --stake-authority stake_auth.json <STAKE_ACCT_1_PUBKEY> <VOTE_ACCT_PUBKEY> --fee-payer ~/.config/solana/id.json

NOTE: <VOTE_ACCT_PUBKEY> refers to the address of the vote account, not the validator identity!

Now you can check up on it to see if the stake is activating and what epoch it will be active at:


$ solana stake-account <STAKE_ACCT_1_PUBKEY>

Once the next epoch begins and the stake is active, your validator’s votes will start counting, and you can view the recent votes as part of the solana vote-account command. If you only added a few SOL like in the example, do not expect your validator to become the block leader.

Voting costs about 1-2 SOL per day, so at the time of writing, it would cost your validator about $200-300/day just to vote on mainnet.

(More info on stake delegation and managing stake accounts).

Publish the Validator’s Information

If you browse the common Solana validator directories, such as validators.app, you’ll notice that all the validators have a name, description, logo, and other metadata. This data is published on-chain for all the registered validators in each cluster, and you can view it by running solana validator-info get.

Once your validator is up and running, you may want to publish your info, particularly once you are running on mainnet and trying to attract stake.

Here is a basic example of how to publish your metadata:


$ solana validator-info publish "My Awesome Validator" \
--website "https://awesome-validator.xyz/" \
--icon-url "https://awesome-validator.xyz/icon360x360.png" \
--keypair validator_identity.json \
--details "The best validator in the world!"

When the metadata record is created, it will have its own key, which can be updated later with the --info-pubkey argument. Note that the icon URL can be a maximum of 80 characters in length. More info on publishing validator info and metadata can be found here.

Congrats!

Congratulations! If you’ve followed along and made it this far, you should be up and running with your own Solana validator. Welcome to the Solana Validator Community!

Additional Resources