Managing Multiple SSH Keys on a Single Linux System

Scenario

A single Linux machine needs to handle multiple Git accounts or code hosting platforms simultaneously, each requiring its own SSH key pair for authentication.

Generating Multiple SSH Key Pairs

Create distinct key pairs for different accounts. For example, generating keys for three seperate identities:

$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):/root/.ssh/id_rsa_work
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa_work.
Your public key has been saved in /root/.ssh/id_rsa_work.pub.

When prompted for the key save location, specify a custom path rather than accepting the default id_rsa. This prevents overwriting any existing default key. Repeat this process to generate additional key pairs for personal projects, client work, or different hosting services.

Starting the SSH Agent

Check if the SSH authentication agent is running:

$ ssh-add -l
Could not open a connection to your authentication agent.

The error indicates no agent is active. Initialize the agent:

exec ssh-agent bash

To clear all keys from the agent (when needed for a fresh start):

$ ssh-add -D

Registering Keys with the Agent

Add each private key to the running agent:

$ ssh-add ~/.ssh/id_rsa_work
$ ssh-add ~/.ssh/id_rsa_personal
$ ssh-add ~/.ssh/id_rsa_client

Deploying Public Keys to Remote Servers

Upload the contents of each .pub file to the corresponding code hosting service. On GitHub, navigate to Settings → SSH and GPG keys → New SSH key. On other platforms, locate the SSH key management secsion in account settings and paste the full public key content.

Configuring SSH Host Aliases

Create or edit the ~/.ssh/config file to map connections to their respective keys:

Host work-github
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_rsa_work

Host personal-oschina
    HostName git.oschina.net
    User git
    IdentityFile ~/.ssh/id_rsa_personal

Host client-bitbucket
    HostName bitbucket.org
    User git
    IdentityFile ~/.ssh/id_rsa_client

Each Host entry defines an alias used in git operations. The HostName specifies the actual server, while IdentityFile points to the corresponding private key.

Using the Configuration

Clone repositories using the configured aliases instead of direct URLs:

git clone git@work-github:organization/repo.git
git clone git@personal-oschina:username/project.git
git clone git@client-bitbucket:clientteam/repository.git

All configured key pairs now operate concurrently, with SSH automaticalyl selecting the correct key based on the host alias used in the repository URL.

Tags: ssh Linux Git Authentication SSH Keys

Posted on Thu, 17 Sep 2026 16:48:27 +0000 by RynMan