Using kops and AWS Bastion Hosts Correctly

You have correctly provisioned your AWS infrastructure using AWS Bastion Quickstart or with kops and want to connect to your private instances using the bastion hosts.

First some principles:

• Terminate your bastion host after using it (set autoscaling to 0).
• Do not use your bastion host as a store, especially not for anything security relevant
• Specifically never copy any private key to the bastion host
• Close down the security group to your specific IP
• After using the bastion host, close down the security group completely

This may all sound paranoid, but even if some threats are theoretical at best, your security compliance group will come hunting for you if you don’t follow the steps.

To connect to any server in the private subnets, you can use ssh proxying. Add the following snippets to your ~/.ssh/config file

Host bastion
HostName <fqdn of your bastion-elb or bastion-host>
IdentityFile ~/.ssh/bastion_rsa # your bastion private key
User ec2-user

Host private-server
User ubuntu
HostName <private IP of your server in private subnet>
ProxyCommand ssh -q -W %h:%p bastion
IdentityFile ~/.ssh/privateserver_rsa # your bastion private key

Try connecting to your bastion host first

ssh bastion

Then after succeeding, log out and try

ssh private-server

If you are following the principles above, you will retrovision your bastion host (set autoscaling to 1) every time you need it you will run into an issue: ssh will not connect to the server since you kept the name but the server’s identity has changed. This is a security measure that prevents an intruder to redirect traffic to a new server while keeping the name. You can safely delete the old server’s identity from ssh by using:

ssh-keygen -R <bastion-elb>