[Node] Custom Metrics E2E Test

[Miqueasher]

This PR adds custom metrics to the existing sample app to verify public documentation is correct and functionality continues to work as changes happen in the future (Regression testing). Specifically for OpenTelemetry metrics using the CloudWatch Agent.

The changes made in this PR are:

New imports to index.js and dependencies to package.json (sample app's frontend service for the 'main' instance and the dependency installer)
Global meter, and gauge/histogram/counter added to FrontendServiceController.java from OTEL SDK Metrics page
Update to environment variables in main.tf
Update to jobs & env sections in node-ec2-default-test
Update to Cloudwatch agent config to include otlp ports
New Validator file to call updated predefined template file
Update to PredefinedExpectedTemplate to include new mustache file
New mustache file for validation of custom metrics

Git revert back to last passing CSHA: af399c7

Rollback procedure:

<Can we safely revert this commit if needed? If not, detail what must be done to safely revert and why it is needed.>

Ensure you've run the following tests on your changes and include the link below:

To do so, create a test.yml file with name: Test and workflow description to test your changes, then remove the file for your PR. Link your test run in your PR description. This process is a short term solution while we work on creating a staging environment for testing.

ADOT-SIGV4
EC2-DEFAULT
ASG

ECS
EKS
K8s

NOTE: TESTS RUNNING ON A SINGLE EKS CLUSTER CANNOT BE RUN IN PARALLEL. See the needs keyword to run tests in succession.

• Run Java EKS on e2e-playground in us-east-1 and eu-central-2
• Run Python EKS on e2e-playground in us-east-1 and eu-central-2
• Run metric limiter on EKS cluster e2e-playground in us-east-1 and eu-central-2
• Run EC2 tests in all regions
• Run K8s on a separate K8s cluster (check IAD test account for master node endpoints; these will change as we create and destroy clusters for OS patching)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.