Jacky's Treasure Box: Building web server of multiple docker instances with ssl (https) protection in AWS lightsail

Background

Continue from last tutorial of creating http server with ssl protection, this time I want a bid more advanced, here is the situation.

I have a take away web application, which consists of 5 servers, they are frontend server powered by react, middleware server powered by loopback (nodeJS server), mongoExpress for monitoring mongoDB in UI, mongoDB database and a ftp server for updating the menu and meal information. System architecture is as shown as follow

So there exists 5 docker containers, running in the same local network, I want to put them altogether as a web application to serve my client, but the problem is the SSL cert, I want security transaction and need to register and deploy the certificate for my react front-end and middleware loopback server, so how can I deploy all of them (with same internal 80 port) to the Internet?

Problems

The following are the requirements & problems needed to solve

1. Getting a signed certificate from trusted party

2. Allow "api.goodmaneat.com" to be surfed by front-end server to acquire middleware functionalities through port 443 (https) (which co-exists with the front-end container)

3. Is the cert being shared by *.goodmaneat.com and goodmaneat.com?

4. The following is what needed to achieve

- www.goodmaneat.com -> goodmaneat.com

- www.goodmaneat.com/admin -> goodmaneat.com/admin

- http://goodmaneat.com -> https://goodmaneat.com

So basically, I want the server to strip the www prefix and force redirect to https

Solutions

The below items are what I have done to tackle the problems

1. Create a lightsail AWS instance of type Amazon Linux 2, which is good if you have any service needed to use aws cli

2. Install docker and docker compose to the Amazon Linux (https://gist.github.com/npearce/6f3c7826c7499587f00957fee62f8ee9)

- Note: Logout / restart the instance after installing or docker cannot function properly

3. Assume you deploy your docker images in AWS, configure your login credential first before accessing Amazon registry (https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html)

4. Git clone or upload all files to the server and use docker compose to build up the docker images to containers, I have the following directories for making the whole application functions. Check all 5 containers are up and run.

Docker-compose file reference: (https://docs.google.com/document/d/1SPWOBeLL23E75W_D9U38jZACVNR9jZVwbZqtljIWX2I/edit?usp=sharing)

Note for some important points for the docker-compose yml file

- Loopback container's port setting should be 8082:80 (host:container), we cannot have 80:80 as frontend has already occupied the port 80

- Add 443:443 port settings to frontend container to serve for https connection

- Add nginx and letsencrypt folder to store nginx configuration (which we will deal with in later steps) and certificates (Note: the whole /etc/letsecrypt folder should be mounted for https to work properly)

- Update the dockerFile of the frontend container as the following (https://docs.google.com/document/d/1hJyFyWSazhE_qx9G2dBGc0of9BlDuBgdk0VdKGRRGwk/edit?usp=sharing), here, we install certbox for obtaining certificates

5. docker exec into frontend container, follow step 2 to step 6 to complete the retrieval of certificate process (https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-using-lets-encrypt-certificates-with-wordpress)

Note:

- Here we assume you already have a domain name registered and have full control as you need to add TXT records to complete the letsencrypt challenges, you should have also configured the route 53 record (assume you are using AWS as your domain name service provider) to add the domain name and IP mapping of "www.goodmaneat.com", "goodmaneat.com" and other sub domain name required to the route 53 record table.

6. Still in frontend container, assuming you are using nginx as web server, head to /etc/nginx/conf.d/nginx.conf (https://docs.google.com/document/d/18LuvdXzsE1qsyBP3fFpP1A1v528MLiYA_Ime-_yptfY/edit?usp=sharing), update the configuration file as the specified URL to

- Locate the certificate registered in step 5

- Update port settings to only accept https connections

- 301 permanent redirect when www.goodmaneat.com/* is detected, the first sever block configuration accomplish such effect by recognizing domain name "www.goodmaneat.com" and all its subdomains, and return 301 header redirect to its https and www removed URL.

- The second server block serves only https URLs

- The third server block is the most tricky part, it detects server name "api.goodmaneat.com", we still need to provide certificate file here because we accepts only https connection even for internal docker container access.

- The "proxy_pass" in location block is important, it works with upstream block to guide nginx server when api.goodmaneat.com (http/https) is being accessed, it reverses proxy to send the request to the requested server (this time it is our "loopback container", which can be referred using docker service name), nginx then fetches the response and send it back to our client (frontend container), the upstream block provides group of servers for proxy_pass directive to refer to, e.g.: the value of "proxy_pass http://api.goodmaneat.com" will be parsed as "proxy_pass http://(internal docker IP of loopback container)"

6. After all the nginx configurations, reload the nginx server by "nginx -s reload -c /etc/nginx/conf.d/nginx.conf"

Finally, the file structure of the host server (not the docker container) should be as follows

- Letsencrypt folder volume amount is for storing the certificate and key files in frontend nginx web server container

- nginx-conf folder to map and permanently store the server configuration files in step 5 in frontend nginx web server container

Note

When cert is being updated, some cert file's ownership and user rights will change to root, which makes reading of certificate failed, kindly change to appropriate user and user right before deployment when cert is being renewed

References

- Nginx multiple server blocks listening to same port

- Update: Using Free Let’s Encrypt SSL/TLS Certificates with NGINX

- Multiple docker containers accessible by nginx reverse proxy

- How to Host Multiple Docker Containers on a Single Droplet with Nginx Reverse Proxy?

- How to proxy_pass to a node docker container on port 80 with nginx container

- How To Redirect HTTP To HTTPS In Nginx

- nginx with Let’s Encrypt in Docker container

- Multiple SSL certificates for a single domain on different servers

- How nginx processes a request

- NGINX multiple server blocks with reverse proxy

- Module ngx_http_upstream_module