[Oracle Cloud] Helm

This deployment option utilizes helm for configuration and installation onto a Kubernetes cluster. It is recommended to use an Oracle Container Engine for Kubernetes cluster, however other standard Kubernetes clusters will also work.

cd deploy/complete/helm-chart

dir deploy/complete/helm-chart

Path for Cloud Native deployment configurations using helm

Deploying the complete MuShop application with backing services from Oracle Cloud Infrastructure involves the use of the following helm charts:

setup: Installs umbrella chart dependencies on the cluster (optional)
Not supported yet provision: Provisions OCI resources integrated with Service Broker (optional)
mushop: Deploys the MuShop application runtime

To get started, create a namespace for the application and its associative deployments:

kubectl create ns mushop

Setup

MuShop provides an umbrella helm chart called setup, which includes several recommended installations on the cluster. These represent common 3rd party services, which integrate with Oracle Cloud Infrastructure or enable certain application features.

Chart	Purpose	Option	Default
Prometheus	Service metrics aggregation	`prometheus.enabled`	true
Grafana	Infrastructure/service visualization dashboards	`grafana.enabled`	true
Metrics Server	Support for Horizontal Pod Autoscaling	`metrics-server.enabled`	true
Ingress Nginx	Ingress controller and public Load Balancer	`ingress-nginx.enabled`	true
Cert Manager	x509 certificate management for Kubernetes	`cert-manager.enabled`	true
Jenkins	Jenkins automation server on Kubernetes	`jenkins.enabled`	false

Dependencies installed with setup chart. NOTE as these are very common installations, each may be disabled as needed to resolve conflicts.

From deploy/complete/helm-chart directory:

Install chart dependencies:
```
helm dependency update setup
```

Install setup chart:

kubectl create ns mushop-utilities

helm install mushop-utils setup \
  --namespace mushop-utilities

OPTIONAL The Jenkins automation server can be enabled by setting jenkins.enabled to true in values.yaml or by adding the command line flag --set jenkins.enabled=true in the helm install command above.

helm install mushop-utils setup \
  --namespace mushop-utilities \
  --set jenkins.enabled=true

NOTE the public EXTERNAL-IP assigned to the ingress controller load balancer:

kubectl get svc mushop-utils-ingress-nginx-controller \
  --namespace mushop-utilities

Deploy MuShop

Deploying the full application requires cloud backing services from Oracle Cloud Infrastructure. These services must be provisioned manually and are configured using kubernetes secrets.

Configure

Provision an Autonomous Transaction Processing (ATP) database. Once RUNNING:
- Configure Oracle Database API for MongoDB
  - Update network access type to Secure access from allowed IPs and VCNs only and set your public IP address.
  - Go to Database Actions, select Oracle Database API for MongoDB related service, extract the database host from the url and copy it for later usage (oadb_host value in the oadb-connection secret)
    
    Extracted host format: <adb-id>-<db-name>.adb.<oci-region>.oraclecloudapps.com
  - Update network access type by replacing your public IP address with the OKE virtual cloud network.
- Create oadb-admin secret containing the database administrator password. Used once for schema initializations.
```
kubectl create secret generic oadb-admin \
  --namespace mushop \
  --from-literal=oadb_admin_pw='<DB_ADMIN_PASSWORD>'
```
- Download the DB Connection Wallet and create oadb-wallet secret with the Wallet contents using the downloaded Wallet_*.zip. The extracted Wallet_* directory is specified as the secret contents.
```
kubectl create secret generic oadb-wallet \
  --namespace mushop \
  --from-file=<PATH_TO_EXTRACTED_WALLET_FOLDER>
```
- Create oadb-connection secret with the Wallet password and the service TNS name to use for connections.
```
kubectl create secret generic oadb-connection \
  --namespace mushop \
  --from-literal=oadb_wallet_pw='<DB_WALLET_PASSWORD>' \
  --from-literal=oadb_service='<DB_TNS_NAME>' \
  --from-literal=oadb_ocid='<DB_OCID>' \
  --from-literal=oadb_host='<DB_HOST>'
```
  Each database has 5 unique TNS Names displayed when the Wallet is downloaded an example would be mushopdb_TP.
Optional: Instead of creating a shared database for the entire application, you may establish full separation of services by provisioning individual ATP instances for each service that requires a database. To do so, repeat the previous steps for each database (Oracle Database API for MongoDB configuration step and oadb_host in the oadb-connection secret are required only for the carts database) and give each secret a unique name, for example: carts-oadb-admin, carts-oadb-connection, carts-oadb-wallet.
- carts
- catalogue
- orders
- user

Authorize instances to manage ATP databases

Create a Dynamic Group

Navigate to Identity -> Dynamic Groups -> Create Dynamic Group

  Name: <DynamicGroupName>
  Description: <DynamicGroupDescription>
  Matching Rules: ANY {ALL {instance.compartment.id = '<COMPARTMENT ID>'},ALL {resource.type = 'cluster', resource.compartment.id = '<COMPARTMENT ID>'}}

Create an IAM Policy

Navigate to Identity -> Policies -> Create Policy

  Name: <PolicyName>
  Description: <PolicyDescription>
  Statement: Allow dynamic-group <DynamicGroupName> to manage autonomous-database-family in compartment id <COMPARTMENT ID>

Create a public bucket in Object Storage service and authorize instances to manage objects
- Create a public bucket
  
  Navigate to Object Storage -> Buckets and create a public bucket
- Authorize instances to manage objects in compartment
  
  Navigate to Identity -> Policies, open the policy created in the previous step and edit policy statements by adding another statement
```
  Statement: Allow dynamic-group <DynamicGroupName> to manage objects in compartment id <COMPARTMENT ID>
```
- Create oos_bucket secret containing the bucket name and object storage namespace
```
kubectl create secret generic oos-bucket \
  --namespace mushop \
  --from-literal=name='<BUCKET_NAME>' \
  --from-literal=namespace='<OBJECT_STORAGE_NAMESPACE>'
```
Optional: Provision a Streaming instance from the Oracle Cloud Infrastructure Console, and make note of the created Stream Pool configuration values bootstrapServers and stream pool ID.
- Create oss-connection secret containing the Stream connection details.
```
kubectl create secret generic oss-connection \
  --namespace mushop \
  --from-literal=bootstrapServers='<OSS STREAM BOOTSTRAP SERVERS>' \
  --from-literal=jaasConfig='<JAAS CONFIG>'
```
  Note that <OSS STREAM BOOTSTRAP SERVERS> and <JAAS CONFIG> values can can be found in the Stream Pool -> Kafka Connection Setting. In case you want to connect under different user then the <JAAS CONFIG> format is:
```
jaasConfig="org.apache.kafka.common.security.plain.PlainLoginModule required username=\"<USER_COMPARTMENT_NAME>/<USER_NAME>/<OSS_POOL_ID>\" password=\"<USER_TOKEN>\";"
```
  Make sure the user has permission to write to the given stream.

Configure a config map with deployment details:

kubectl create cm oci-deployment \
  --namespace mushop \
  --from-literal=compartment_id='<COMPARTMENT ID>' \
  --from-literal=region='<OCI REGION>'

Optional: If you want to configure and use Oracle Application Monitoring service, make sure you create the APM domain and create K8s secret with connection details.
1. Navigate to Observability & Management -> Application Performance Monitoring -> Administration.
2. Hit the button Create APM domain and enter the domain name.
3. Once created make a note of Data Upload Endpoint and auto_generated_public_key.
4. Edit the mushop.tfvars:
```
  kubectl create secret generic oapm-connection \
    --namespace mushop \
    --from-literal=zipkin_enabled=true \
    --from-literal=zipkin_path='<APM DOMAIN DATA UPLOAD ENDPOINT>' \
    --from-literal=zipkin_url='/20200101/observations/public-span?dataFormat=zipkin&dataFormatVersion=2&dataKey=<AUTO_GENERATED_PUBLIC_KEY>'
```
Optional: If you want to configure and use the functions/API Gateway functionality, make sure you create and deploy the function and the API Gateway by following the instructions in the src/functions/newsletter-subscription folder.

To configure the api chart to use the newsletter subscribe function, create K8s external service:
```
cat <<EOF | kubectl apply -f -
piVersion: v1
kind: Service
metadata:
  name: mushop-newsletter
  namespace: mushop
spec:
  externalName: <API_GATEWAY_URL>
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  type: ExternalName
status:
  loadBalancer: {}
EOF
```
Replace the API_GATEWAY_URL with an actual hostname URL you got when you deployed your API gateway instance. For example: jbwyanwkmxqweq.apigateway.us-ashburn-1.oci.customer-oci.com

Make a copy of the values-dev.yaml file in this directory. Then complete the missing values (e.g. secrets) like the following:

global:
  cloud: oci
  ossConnectionSecret: oss-connection     # Name of Stream connection secret
  oadbAdminSecret: oadb-admin             # Name of DB Admin secret
  oadbWalletSecret: oadb-wallet           # Name of Wallet secret
  oadbConnectionSecret: oadb-connection   # Name of DB Connection secret
  oosBucketSecret: oos-bucket             # Name of Object Storage bucket secret
  ociDeploymentConfigMap: oci-deployment  # Name of Deployment details config map
tags:
  atp: true                               # General flag to use Oracle Autonomous Database
  streaming: true                         # General flag to use Oracle Streaming Service

NOTE: If it’s desired to connect a separate databases for a given service, you can specify values specific for each service, such as carts.oadbAdminSecret, carts.oadbWalletSecret…

Deploy

From deploy/complete/helm-chart directory:

helm install mushop mushop \
  --namespace mushop \
  -f <edited values-dev.yaml>

Wait for services to be Ready:

kubectl get pod --watch --namespace mushop

Open a browser with the EXTERNAL-IP created during setup, OR port-forward directly to the edge service resource:
```
kubectl port-forward \
  --namespace mushop \
  svc/edge 8000:80
```
Using port-forward connecting localhost:8000 to the edge service
```
kubectl get svc mushop-utils-ingress-nginx-controller \
  --namespace mushop-utilities
```
Locating EXTERNAL-IP for Ingress Controller. NOTE this will be localhost on local clusters.

Cleanup

The following list represents cleanup operations, which may vary depending on the actions performed for setup and deployment of MuShop.

List any helm releases that may have been installed:

helm list --all-namespaces

NAME                    NAMESPACE               REVISION        UPDATED                                 STATUS          CHART                           APP VERSION   
mushop                  mushop                  1               2020-01-31 21:14:48.511917 -0600 CST    deployed        mushop-0.1.0                    1.0         
oci-broker              mushop-utilities        1               2020-01-31 20:46:30.565257 -0600 CST    deployed        oci-service-broker-1.3.3                   
mushop-provision        mushop                  1               2020-01-31 21:01:54.086599 -0600 CST    deployed        mushop-provision-0.1.0          0.1.0      
mushop-utils            mushop-utilities        1               2020-01-31 20:32:05.864769 -0600 CST    deployed        mushop-setup-0.0.1              1.0

Remove the application from Kubernetes where --name mushop was used during install:
```
helm delete mushop -n mushop
```
If used OCI Service broker, remove the provision dependency installation, including ATP Bindings (Wallet, password) and instances:
```
helm delete mushop-provision -n mushop
```
After delete, kubectl get serviceinstances -A will show resources that are deprovisioning
If used OCI Service broker, remove the oci-broker installation:
```
helm delete oci-broker -n mushop-utilities
```

Uninstall Istio service mesh (if applicable):

istioctl manifest generate --set profile=demo | kubectl delete -f -

Remove the setup cluster dependency installation:
```
helm delete mushop-utils -n mushop-utilities
```