2020年3月14日 星期六

[AWS] Setting Up Multiple Load Balancers & A Very Dumb Mistake

Background
Continue from the previous AWS dynamic port mapping story, although familiarizing the concepts of how a load balancer works in AWS, and recognizing the theory of how different containers with the same container port can co-exists in the same service, but I still got a VERY VERY VERY huge issue not yet solved.



How to Configure Multiple Target Group Within a ECS Service?

The reason why this is a blocking issue for my project is that without multiple target groups, the following requirements cannot be achieved.





  • HTTPS is a must for all URLs
  • Extra port in URL (e.g.: www.gme.com:3002) is not okay because HTTPS cannot be made use (already occupied 443 port)
  • Same container port (80) should be assigned to frontend and loopback containers for ALB
  • ALB's rule-set configured to
    • Redirect all incoming HTTP requests to HTTPS requests for security
    • When gme.com or www.gme.com is requested, direct to gme.com and forward requests to FRONTEND container
    • When api.gme.com or www.api.gme.com is requested, direct to api.gme.com and forward requests to LOOPBACK container

We can see the last requirement requires us to have 1 ALB and 2 target groups to accomplish, basically the idea is to identify which container to handle the requests by different URLs which in turn being allocated to different containers for responses. When api.gme.com is requested, the DNS (which is created by the LB) which execute the rule-set and know it needs to add a 301/302 redirection HTTPS and forward the request to gme-loopback-target-gp for responses, same for front-end requests


As we have 2 web servers (1 for serving front-end, 1 for middleware (loopback)), we cannot serve 2 servers using the same host port, we instead make use of different ephemeral ports or high number ports (32768 and 32769) for identification. 


So the most important part is to create an AWS service with 2 target groups associated with frontend and loopback container respectively. 


Problems and Solving Procedures

As of this tutorial, I get used to use ecs cli compose service up command to manage containers composing of the service and task definitions, however ecs cli compose DOES NOT SUPPORT MULTIPLE TARGET GROUPS! At least at the time when I write this article, I am frustrated for making only 1 frontend target group only.


















But hey! I want 1 more... for my loopback container. At the end I got stuck 7-10 days wondering how can it be achieved. Furthermore requesting help in stackoverflow also but no luck. I did find the article saying multiple target groups did not currently support but this official article clearly mentioned the support of multiple target groups and even multiple LB, but they aren't using esc cli compose up command, turn out I follow the very basic service creation & task definition combination method to solve my problem. Here is the procedures




1. Under the cluster, create a new version of task, press "Run New Task"


2. Because I already created tasks before (through ecs cli compose syntactic sugar command), just creating a new version of task suit my needs. My option is to configure the task through AWS console, but we can configure through CLI too.



























The important configuration here is the "task group", this affects how AWS handles task placement, in my case, the mw-gme is the same as my service name, this "connects" the task and the service.






















The funny thing is it has 2 "Tasks" columns here, if you are not linking them while configuring task definition, they are detached, we will not see the task definition "mw-gme:42" in Tasks of cluster. So better specify the service name in "task group"




3. Click "Run Task" button and your tasks will start running the containers



4. Next we deal with creating service, here we use --cli-input-json parameter, feed in the json file for parameters, here are the context of the json file


{
  "cluster": "mw-ecs-gme",
  "serviceName": "mw-gme",
  "taskDefinition": "mw-gme:42",
  "schedulingStrategy": "REPLICA",
  "launchType": "EC2",
  "loadBalancers": [
      {
          "targetGroupArn": "arn:aws:elasticloadbalancing:us-east-2:916381200858:targetgroup/gme-frontend-gp/e2cbd06d8e11d633",
          "containerName": "frontend",
          "containerPort": 80
      },
      {
        "targetGroupArn": "arn:aws:elasticloadbalancing:us-east-2:916381200858:targetgroup/gme-loopback-gp/30e3eccadbb1f904",
        "containerName": "loopback",
        "containerPort": 80
    }
  ],
  "desiredCount": 1
}

The most important parameters here are cluster, serviceName, taskDefinition, and loadBalancers! Yes! And you see it allows me to put in an array! I can put multiple target groups here! Amazing, I can now setup gme-target-group-frontend to forward the matching rules to frontend container, while gme-target-group-loopback to forward the matching rules to loopback container, and both have host port of 80, that's why we can apply HTTPS on both of the containers to prevent the following errors





This error encounters when frontend (https://gme.com) is made HTTPS but not http://api.gme.com, we can't have mixture of HTTPS and HTTP put together, we must have all HTTPS equipped. That is why we need both 80 as container port.



And we can see the service name "mw-gme" here is the same as the name of the task group specified in step 2, this is very important they are with the same name, when doing so, tasks in cluster will show up the task definition entry (mw-gme:42).



5. Run aws ecs create-service --cli-input-json file://ecs-service.json, where ecs-service.json file is the one we specified in step 4.



And that's it! You will see service is started with 2 target groups in load balancing column and task definition is with status "RUNNING", for configuration of ELB and target group, refer to the previous thread.







































You may wonder what takes me 7-10 days for such setup, 2 main points stuck me during the deployment.



1. Insisting there is such a way to allow me to deploy multiple target groups using ecs cli compose service up command, but indeed it is NOT SUPPORTED! Look at the command provided in official doc

ecs-cli compose --file ./docker-compose-aws-prod.yml --cluster mw-ecs-gme --ecs-profile mw-ecs-profile --cluster-config mw-ecs-gme-config --ecs-params ./ecs-params.yml --project-name mw-gme service up --target-group-arn arn:aws:elasticloadbalancing:us-east-2:916381200858:targetgroup/gme-gp/ed4eb5d138cf37df --container-name frontend --container-port 80  

We can see here how convenient this syntactic sugar command provided for us to easily bring up both the services and the associated task definition together, frankly speaking it is with no differences from the method I mentioned above (creating service and task definition), but it has its limitation. We can see parameter "--target-group-arn", "--container-name" and "--container-port" only allows creating 1 set of target group (with 1 set of container match). Stuck here for several days, turn out multiple target group is not supported in ecs-cli compose service command! Hope Amazon will improve this command in the future.



2. And the dumbest thing and spent me further 4-6 days is when I used service creation and task definition to finally created a multiple target groups load balancer service, the loopback server is still failing the health test of the LB, it is weird as I have confirmed these things already


- Host port 32769 incoming port has opened in security rule

- 2 target groups are created
- Containers and port mapping configured successfully through info provided by AWS console UI
- A record entry has been added in route 53
- Frontend (https://gme.com) is doing fine (passing the healthy test of LB, target port 32768)


I started a thread and seek help in stackoverflow, confirming my approach and steps are correct, but still no luck... And after 4-5 days struggling, I found a huge huge mistake, making all these not working, it is NOT related to all the settings of AWS, all are good and fine, then I started to think it may be come from the exposed port settings defined in loopback server itself...

























As we can see, I can't image the problem comes from the loopback server configuration, I totally forgot to change the port settings to 80, while keeping it as the development port, after changing it back to port 80, healthy check passed and both https://gme.com and https://api.gme.com is up and running fine, what a silly mistake I have made!



Further Notes

You may not want to stop the service and renew the task definition again if you are just updating the docker registries, in this case, "aws ecs update-service" command is the way to go.

Executing "aws ecs update-service --service mw-gme --force-new-deployment --cli-input-json file://ecs-service-update.json", will update the service with the parameters provided, and most important is the "--force-new-deployment parameters, forcing ecs to retrieve the docker images again even they have the same tag.


And 1 more, 301 (permanent redirection) is very dangerous, if you tried HTTP 301 settings in server, redirecting all HTTP requests to HTTPS requests, in client devices, these permanent redirection is being cached with no expiry date, which means when you try to remove HTTPS and turn back to HTTP (I needed to do so at that time because I am stuck in setting up LB and need to provide services to users first), client device will still being cached to navigate to HTTPS version of the site which will cause 404 error, I would definitely recommend 302 (temporary redirect) if it is the halfway of the project and you intend to switch back temporarily.



References

2020年3月3日 星期二

[AWS] Dynamic Port Mapping & SSL Equipment through ELB

Background
Deployment of the container in Amazon cloud has completed, and there are other challenges which include

  • Adding SSL to current production site for increased security
  • Dynamic port mapping for serving multiple services within 1 cluster
The reason why I put these 2 features together is because of their similarity on how they configure. Dynamic port mapping is used to deal with running multiple services within a cluster, imagine when more than 1 web server is built in a ec2 instance, how is the port 80 being "shared" by these 2 services (web server)? Dynamic port mapping is the solution.


Concepts
The solution is to make use of ELB (Elastic Load Balancers), which is divided into ALB (Application Load Balancers) and NLB (Network Load Balancers), ALB is to load balance the traffic in application level, L7, which is our case (HTTP & HTTPS). NLB is for binary protocols (non HTTPx protocols).


Here is how application load balancers (ALB) deals with dynamic port mapping


























1. User fire requests either HTTP (port 80) or HTTPS (port 443) directly to the load balancer

2. Each load balancer has its corresponding listeners which checks the connections of the port / protocol. Note that we need to pass the healthy test before the load balancer operates

3. When such connection happens, listeners will use the listener rules (user defined, can be more than 1, depending on your needs) to direct the traffic to designated target groups

4. Here, the target group is "TG1", this group contains the frontend container which is composed by the docker compose file. We need to wire up the group arn to the container when we create the service using "ecs cli create service" command, note we cannot modify it after the service has started. And the limitation using this command is that we can only bind 1 target group to 1 container.

5.A "registered target" should be configured in the target group, and the load balancer will use the registered target to check the healthy status before operate. When the request is forwarded to the target group, the listener will choose the port specified in the registered target to transfer / map the request to the desired containers.

e.g.: If we configured a target group as ec2-instance (target instance) | 32768 (port) | ... The listener will know for the port 80, which container port should be used to map the host port 80 request to.

But remember, to ensure this happens, we need to configure the security groups to allow inbound port 80 or any desired host ports to allow load balancers correctly listens to outside requests

6. Sometimes there may be more than 1 container served in the same ec2 instance, we therefore need dynamic port mapping to accomplish 2 web servers situated in the same ec2 instance, by dividing the 2 containers into 2 different target groups, serving 2 websites using 1 ec2 instance become possible.

Procedures
1. Follow AWS tutorial to create load balancers, choose HTTP / HTTPS application load balancer

2. Choose "Internet facing" scheme and add both HTTP and HTTPS load balancer protocol if you want SSL over HTTP.





















3. Choose the desired VPC to apply the load balancer to

4. Choose the certificate type (we normally use ACM as it is super easy to deploy the site with SSL enabled), choose the certificate registered in ACM (I am using route 53 for the domain and use Amazon ACM here)














5. Choose security group, if the outgoing port is 80 and 443, you need to allow inbound connection of port 80 and 443 to allow load balancer to do its job.













6. Configure the target group and health check parameters. Here, remember we use HTTP protocol (port 80) as the protocol for the load balancer to forward traffic to designated target group, we left SSL handling to load balancer and prevent configuring the certificates server configuration, deployment and renewal in application level, we let ACM to handle them all for us. So, we only need load balancer to do health check on port 80. (As our docker compose file exposed port 80 for the LB to do the health check)



















7. After creating the load balancer, navigate to "target groups", you will see the group we created when we create the load balancer, head to "Description", copy the arn of the target group and use it to associate with the service using the following command


ecs-cli compose --file ./docker-compose-aws-prod.yml --cluster mw-ecs --ecs-profile mw-ecs-profile --cluster-config mw-ec2 --project-name supremeav-2-dot-0 --ecs-params ./ecs-params.yml service up --target-group-arn arn:aws:elasticloadbalancing:us-east-2:916381200858:targetgroup/http-port-80-access/222d18e3d1cddf3a --container-name frontend --container-port 80

Note that "target-group-arn" is the arn you copied in description and "container-name", "container-port" must be specified to let balancer forward the request to the right container.














8. After the container is initialized, you should see a new entry appeared in the registered target (in the target groups) automatically when the container is up and run. Health check should be carried out accordingly.

9. Finally, navigate to load balancer and copy the DNS name, navigate to route 53, find the domain name registered, add or edit the existing A type entry, with alias selected, choose from the input the DNS A type entry (or you can paste the DNS you just copied) and you are all done!



















Now when user access the website through Internet, they are just forming connection with the load balancer and the load balancer will forward / direct the request to the container port through evaluating the rules.

For SSL connection, there is no extra procedure needed, just ensure there is an entry in listeners of the load balancers, the rest would be handled by them, no extra nginx configuration play around!


























References

1. https://stackoverflow.com/questions/58587976/nginx-docker-container-on-aws-ecs-the-plain-http-request-was-sent-to-https-po
2. SSL redirection in docker container on aws ecs
3. Understanding dynamic port mapping in amazon ecs with load balancer

2020年2月22日 星期六

[Android] JS debugging

Problems
I know for many of them it may be an old news, but I am quite surprise chrome debugging can be so easy even in Android device. Facing some weird bugs, when the front-end tries to fire an API to perform some server actions, it only succeed in desktop chrome but not mobile chrome.

I know it has a high chance of not a coding issue, but some specific settings that only occurs in mobile chrome, and even more, the API call works in incognito mode, which makes me even more curious on why it happens.
Solutions
To find out what actually malfunctioned behind, I planned to find a way to let me get a sort of like "remote logging", which connects my android device to the development machine and view the console log of the Android chrome for further bug tracing.

Turns out, it is quite easy, which use me only 5 minutes to setup

1. Enable USB debugging in Android device

2. Connect the Android device and accept the permission dialog raised on phone

3. Open the desktop chrome > console log > more tools > remote devices

4. Check "Discover USB devices", and wait until your device comes up to list (mine is Oneplus 7 pro, so the model is shown up as "GM1913"


















5. Interesting part comes up, when you type in the address and press open, the remote device will pop up chrome and navigate to the designated site! YES! We've now got a console log for our mobile version of chrome! 

6. Press "inspect fallback" or "inspect", a remote device devTools come up and you will see a REALTIME mapping of your device's chrome screen in the devTool, they are in sync now!















7. You can of course do anything just like you inspect your desktop site, like inspecting network tab, console logging, capturing back-end data etc. And more surprisingly, as both of them are insync, you can control the phone's surfing behavior through desktop browser's debugging window too!

















8. So finally I found the data save mode add a data save header causing my API call being cache and causing the failure. Disabling that mode brings the API call succeed again! 

References

2020年2月13日 星期四

[JS] RequireJS - Prevent JS File Caching

Problems
Very often when we updated the JS files, directly refreshing is not reflecting the changes, instead we need to manually clear cache, which makes end users using the web application quite annoying.


Solutions
RequireJS provides "cache busting" which can force browser to download the latest version of JS files every time new pages loaded

Basically, we write a general function accepting target path not to cache, the "require.config" will then execute the function, the bust function actually use current time and attach as the parameter of the JS URL, since time is always changing, it can force the browser to retrieve the JS files for every load


function bust(path) {
    return path + '?bust=' + (new Date()).getTime();
}

require.config({
    baseUrl: '/base/path',
    paths: {
        'fileAlias': bust('fileLikelyToChange'),
        'anotherFileAlias': bust('anotherFileLikelyToChange'),
        'jQuery': 'jQuery'
    },
});



References
- https://stackoverflow.com/questions/16523755/cache-busting-specific-modules-with-requirejs

2020年1月16日 星期四

[JS] Useful ES6 / ES2015 Syntax Enhancements (Keep Updating...)

Below are the es6 enhancements that are easily forgot but use a lot. Keep them noted here

Details
1. Function definitions in object (shorthand)
const obj = {
  foo() {
    return 'bar';
  }
}

2. Shallow copy using spread operator
let obj1 = { foo: 'bar', x: 42 }

let clonedObj = { ...obj1 }

3. Nested object properties destructuring
const nestedObj = {
	open: true,
	btnAct: {
		proceedLbl: 'aaa',
		cancelLbl: 'bbb',
	}
}
const {
	open,
	btnAct: {proceedLbl, cancelLbl}
} = nestedObj

proceedLbl and cancelLbl variable is created with value nestedObj.btnAct.proceedLbl and nestedObj.btnAct.cancelLbl