Using knife ec2 plugin to create VM in VPC private subnet

Question

Although I've written a fair amount of chef, I'm fairly new to both AWS/VPC and administrating network traffic (especially a bastion host).

Using the knife ec2 plugin, I would like the capability to dynamically create and bootstrap a VM from my developer workstation. The VM should be able to exist in either a public or private subnet of my VPC. I would like to do all of this without the use of an elastic IP. I would also like for my bastion host to be hands-off (i.e. I would like to avoid having to create explicit per-VM listening tunnels on my bastion host)

I have successfully used the knife ec2 plugin to create a VM in the legacy EC2 model (e.g. outside of my VPC). I am now trying to create an instance in my VPC. On the knife command line, I'm specifying a gateway, security groups, subnet, etc. The VM gets created, but knife fails to ssh to it afterwards.

Here's my knife command line:

knife ec2 server create \

    --flavor t1.micro \

    --identity-file <ssh_private_key> \

    --image ami-3fec7956 \

    --security-group-ids sg-9721e1f8 \

    --subnet subnet-e4764d88 \

    --ssh-user ubuntu \

    --server-connect-attribute private_ip_address \

    --ssh-port 22 \

    --ssh-gateway <gateway_public_dns_hostname (route 53)> \

    --tags isVPC=true,os=ubuntu-12.04,subnet_type=public-build-1c \

    --node-name <VM_NAME>

I suspect that my problem has to do with the configuration of my bastion host. After a day of googling, I wasn't able to find a configuration that works. I'm able to ssh to the bastion host, and from there I can ssh to the newly created VM. I cannot get knife to successfully duplicate this using the gateway argument.

I've played around with /etc/ssh/ssh_config. Here is how it exists today:

    ForwardAgent yes

#ForwardX11 no

#ForwardX11Trusted yes

#RhostsRSAAuthentication no

#RSAAuthentication yes

#PasswordAuthentication no

#HostbasedAuthentication yes

#GSSAPIAuthentication no

#GSSAPIDelegateCredentials no

#GSSAPIKeyExchange no

#GSSAPITrustDNS no

#BatchMode no

   CheckHostIP no

#AddressFamily any

#ConnectTimeout 0

    StrictHostKeyChecking no

    IdentityFile ~/.ssh/identity

#IdentityFile ~/.ssh/id_rsa

#IdentityFile ~/.ssh/id_dsa

#Port 22

#Protocol 2,1

#Cipher 3des

#Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc

#MACs hmac-md5,hmac-sha1,[email protected],hmac-ripemd160

#EscapeChar ~

    Tunnel yes

#TunnelDevice any:any

#PermitLocalCommand no

#VisualHostKey no

#ProxyCommand ssh -q -W %h:%p gateway.example.com

    SendEnv LANG LC_*

    HashKnownHosts yes

    GSSAPIAuthentication yes

    GSSAPIDelegateCredentials no

    GatewayPorts yes

I have also set /home/ubuntu/.ssh/identity to the matching private key of my new instance.

I notice the following in the bastion host's /var/log/auth.log:

May  9 12:15:47 ip-10-0-224-93 sshd[8455]: Invalid user  from <WORKSTATION_IP>

May  9 12:15:47 ip-10-0-224-93 sshd[8455]: input_userauth_request: invalid user  [preauth]

Answer 1

While specifying my internet gateway, I was missing the username. Username must be specified for both cases, not just --ssh-user.

knife ec2 server create \

    --flavor t1.micro \

    --identity-file <ssh_private_key> \

    --image ami-3fec7956 \

    --security-group-ids sg-9721e1f8 \

    --subnet subnet-e4764d88 \

    --ssh-user ubuntu \

    --server-connect-attribute private_ip_address \

    --ssh-port 22 \

    --ssh-gateway ubuntu@<gateway_public_dns_hostname (route 53)> \

    --tags isVPC=true,os=ubuntu-18.04,subnet_type=public-build-1c \

    --node-name <VM_NAME>

To make you understand it clear, let me show the line which has been updated

--ssh-gateway ubuntu@<gateway_public_dns_hostname (route 53)>