Connecting to a server over SSH: first steps, and a key instead of a password

When you buy a Linux server you are given three things: an IP address, a username (usually root) and a password. This guide goes from the first connection all the way to disabling the password entirely.

The first connection

On Windows 10/11, macOS and Linux an SSH client is already installed — no extra program is needed:

ssh [email protected]

On the first connection you are asked to verify a fingerprint. When you type yes, that fingerprint is saved to ~/.ssh/known_hosts and is verified silently on later connections.

If a fingerprint warning appears on a later connection, stop. Either the server has been reinstalled or your connection has been intercepted. If you do not know why, do not connect.

If the port is different

ssh -p 2222 [email protected]

SSH keys: leave the password behind

Connecting with a password means constant guessing attempts at a door open to the internet. An open port 22 starts attracting automated attempts within hours. A key removes this problem entirely.

1. Generate a key (on your own computer)

ssh-keygen -t ed25519 -C "you@yourcomputer"

Choose ed25519 — it is shorter and stronger than RSA. When you are asked for a passphrase, do not leave it empty: if your key file is stolen, that passphrase is the only protection you have left.

2. Copy the key to the server

ssh-copy-id [email protected]

If ssh-copy-id is not available, do it by hand:

cat ~/.ssh/id_ed25519.pub | ssh root@IP "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

3. VERIFY that you are connecting with the key

ssh [email protected]

If you got in without being asked for a password, the key works. Do not skip this step — the next step turns the password off, and if the key is not working you lose access to the server.

4. Disable password login

In /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password

Then:

sshd -t && systemctl reload sshd

sshd -t tests the configuration before it is applied. If you skip this command and load a broken configuration, the SSH service will not come up.

Do not close your current session

After applying the change, try connecting from a new terminal without closing the session you already have open. If the new connection fails, you can undo the change from the old session that is still open. This single habit removes the most common cause of losing access to a server.

A useful shortcut

Add this to ~/.ssh/config:

Host datafex
    HostName 185.137.98.29
    User root
    Port 22
    IdentityFile ~/.ssh/id_ed25519

From now on ssh datafex is enough.

Next step

The first job once the connection is established is hardening the server: server security: the basic steps

If you are using a Windows server: connecting to a Windows server over RDP