SSH tunnel port forwarding with python (sshtunnel library)

Let's imagine that we have following situation: we have API/website/some service that is not reachable from our PC/server we are currenly connected to. This might be caused by for example firewall/routing rules. But we have ssh access to another server that CAN connect to the target server that is running the service we want to interact with.

Terms:
LOCAL CLIENT - our PC/some server we are connected to already
REMOTE SERVER - bastion node that we can ssh to from LOCAL CLIENT
PRIVATE SERVER - target server that is running the service we are interested in, this server is reachable from REMOTE SERVER and unreachable from LOCAL CLIENT

+----------+               +----------+                +---------+
|  LOCAL   |               |  REMOTE  |                | PRIVATE |
|  CLIENT  | <=== SSH ===> |  SERVER  | <=== HTTP ===> | SERVER  |
+----------+               +----------+                +---------+

Example application: we have VPC (Virtual Private Cloud) on Amazon AWS, some services are reachable from the internet, but for our own use we have additional RESTful Flask API that is gathering metrics about our users. This API is private and not open publicly. What we can do is to setup special server within the VPC (bastion node) that will allow only SSH acces for my specific user via RSA key. This bastion node will then have access to the private API on some other private server within VPC. We can then use port forwarding from our PC to bastion node and let bastion contact API on private server. We will get the API response on our PC.

We will implement such port forwarding solution with Python sshtunnel library (https://pypi.org/project/sshtunnel/). Article and code is based on these two related posts: https://blog.ruanbekker.com/blog/2018/04/23/setup-a-ssh-tunnel-with-the-sshtunnel-module-in-python/, https://medium.com/cemac/ssh-tunnel-port-forwarding-87b054724876.

Example scenario: we will connect from our linux VM on PC into a remote server that this web runs on and then we will contact www.example.com website that is running on some random server. We should get back some simple html from that website.

LOCAL CLIENT - VM on my PC
REMOTE SERVER - tcoil.info server accessible from VM via RSA key
PRIVATE SERVER - web service running on www.example.com

Copy your public key to remote server:

coil@coil-VM:~/.ssh$ ssh-copy-id -i ~/.ssh/id_rsa.pub ubuntu@tcoil.info
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/coil/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
ubuntu@tcoil.info's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'ubuntu@tcoil.info'"
and check to make sure that only the key(s) you wanted were added.

coil@coil-VM:~/.ssh$

Test connectivity:

coil@coil-VM:~/Desktop$ ssh ubuntu@tcoil.info
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-135-generic x86_64)
$

Install python packages on our PC:

coil@coil-VM:~/Desktop$ pip3 install sshtunnel requests

Then we can call from our PC following python script to get to get response from "private server" that was accessed via bastion node:

from sshtunnel import SSHTunnelForwarder
import requests

remote_user         = 'ubuntu'
remote_host         = 'tcoil.info'
remote_port         = 22
local_host          = '127.0.0.1'
local_port          = 5000
private_server      = 'http://example.com'
private_server_port = 80

try:
    with SSHTunnelForwarder(
         (remote_host, remote_port),
         ssh_username=remote_user,
         ssh_private_key='/home/coil/.ssh/id_rsa',
         remote_bind_address=(local_host, local_port),
         local_bind_address=(local_host, local_port),
         ) as server:

        server.start()
        print('server connected')

        r = requests.get(f'{private_server}:{private_server_port}').content
        print(r)

except Exception as e:
    print(str(e))

We got some html from example.com which is exactly what we expected.

coil@coil-VM:~/Desktop/sshtunnel$ python3 ssh_tunnel.py 
server connected
b'<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset="utf-8" />\n    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n    <meta name="viewport" content="width=device-width, initial-scale=1" />\n    <style type="text/css">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href="https://www.iana.org/domains/example">More information...</a></p>\n</div>\n</body>\n</html>\n'
coil@coil-VM:~/Desktop/sshtunnel$

Such approach is convenient for interacting with RESTful APIs that we would not be able to reach otherwise.

Sources:
https://pypi.org/project/sshtunnel/
https://medium.com/cemac/ssh-tunnel-port-forwarding-87b054724876
https://blog.ruanbekker.com/blog/2018/04/23/setup-a-ssh-tunnel-with-the-sshtunnel-module-in-python/