Add StorageOS chart (#159)

proposed/storageos-operator/Chart.yaml

+apiVersion: v1
+appVersion: "1.1.0"
+description: Cloud Native storage for containers
+name: storageos-operator
+version: 0.1.0
+keywords:
+- storage
+- block-storage
+- volume
+- operator
+home: https://storageos.com
+icon: https://storageos.com/wp-content/themes/storageOS/images/logo.svg
+sources:
+- https://github.com/storageos
+maintainers:
+- name: croomes
+  email: simon.croome@storageos.com
+- name: darkowlzz
+  email: sunny.gogoi@storageos.com

proposed/storageos-operator/LICENSE

+MIT License
+
+Copyright (c) 2019 StorageOS
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

proposed/storageos-operator/README.md

+# StorageOS Operator Helm Chart
+
+> **Note**: This is the recommended chart to use for installing StorageOS. It
+installs the StorageOS Operator, and then installs a StorageOS cluster with a
+minimal configuration. Other Helm charts
+([storageoscluster-operator](https://github.com/storageos/charts/tree/master/stable/storageoscluster-operator)
+and
+[storageos](https://github.com/storageos/charts/tree/master/stable/storageos))
+will be deprecated.
+
+[StorageOS](https://storageos.com) is a software-based storage platform
+designed for cloud-native applications. By deploying StorageOS on your
+Kubernetes cluster, local storage from cluster node is aggregated into a
+distributed pool, and persistent volumes created from it using the native
+Kubernetes volume driver are available instantly to pods wherever they move in
+the cluster.
+
+Features such as replication, encryption and caching help protect data and
+maximise performance.
+
+This chart installs a StorageOS Cluster Operator which helps deploy and
+configure a StorageOS cluster on kubernetes.
+
+## Prerequisites
+
+- Kubernetes 1.9+.
+- Privileged mode containers (enabled by default)
+- Kubernetes 1.9 only:
+  - Feature gate: MountPropagation=true.  This can be done by appending
+    `--feature-gates MountPropagation=true` to the kube-apiserver and kubelet
+    services.
+
+Refer to the [StorageOS prerequisites
+docs](https://docs.storageos.com/docs/prerequisites/overview) for more
+information.
+
+## Installing the chart
+
+```console
+# Add storageos charts repo.
+$ helm repo add storageos https://charts.storageos.com
+# Install the chart in a namespace.
+$ helm install storageos/storageos-operator --namespace storageos-operator
+```
+
+This will install the StorageOSCluster operator in `storageos-operator`
+namespace and deploys StorageOS with a minimal configuration.
+
+> **Tip**: List all releases using `helm list`
+
+## Creating a StorageOS cluster manually
+
+The Helm chart supports a subset of StorageOSCluster custom resource parameters.
+For advanced configurations, you may wish to create the cluster resource
+manually and only use the Helm chart to install the Operator.
+
+To disable auto-provisioning the cluster with the Helm chart, set
+`cluster.create` to false:
+
+```yaml
+cluster:
+  ...
+  create: false
+```
+
+Create a secret to store storageos cluster secrets:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: "storageos-api"
+  namespace: "default"
+  labels:
+    app: "storageos"
+type: "kubernetes.io/storageos"
+data:
+  # echo -n '<secret>' | base64
+  apiAddress: c3RvcmFnZW9zOjU3MDU=
+  apiUsername: c3RvcmFnZW9z
+  apiPassword: c3RvcmFnZW9z
+```
+
+Create a `StorageOSCluster` custom resource and refer the above secret in
+`secretRefName` and `secretRefNamespace` fields.
+
+```yaml
+apiVersion: "storageos.com/v1"
+kind: "StorageOSCluster"
+metadata:
+  name: "example-storageos"
+  namespace: "default"
+spec:
+  secretRefName: "storageos-api"
+  secretRefNamespace: "default"
+```
+
+Once the `StorageOSCluster` configuration is applied, the StorageOSCluster
+operator will create a StorageOS cluster in the `storageos` namespace by
+default.
+
+Most installations will want to use the default [CSI](https://kubernetes-csi.github.io/docs/)
+driver.  To use the [Native Driver](https://kubernetes.io/docs/concepts/storage/volumes/#storageos)
+instead, disable CSI:
+
+```yaml
+spec:
+  ...
+  csi:
+    enable: false
+  ...
+```
+
+in the above `StorageOSCluster` resource config.
+
+Learn more about advanced configuration options
+[here](https://github.com/storageos/cluster-operator/blob/master/README.md#storageoscluster-resource-configuration).
+
+To check cluster status, run:
+
+```bash
+$ kubectl get storageoscluster
+NAME                READY     STATUS    AGE
+example-storageos   3/3       Running   4m
+```
+
+All the events related to this cluster are logged as part of the cluster object
+and can be viewed by describing the object.
+
+```bash
+$ kubectl describe storageoscluster example-storageos
+Name:         example-storageos
+Namespace:    default
+Labels:       <none>
+...
+...
+Events:
+  Type     Reason         Age              From                       Message
+  ----     ------         ----             ----                       -------
+  Warning  ChangedStatus  1m (x2 over 1m)  storageos-operator  0/3 StorageOS nodes are functional
+  Normal   ChangedStatus  35s              storageos-operator  3/3 StorageOS nodes are functional. Cluster healthy
+```
+
+## Configuration
+
+The following tables lists the configurable parameters of the StorageOSCluster
+Operator chart and their default values.
+
+Parameter | Description | Default
+--------- | ----------- | -------
+`operator.image.repository` | StorageOSCluster container image repository | `storageos/cluster-operator`
+`operator.image.tag` | StorageOSCluster container image tag | `1.1.0`
+`operator.image.pullPolicy` | StorageOSCluster container image pull policy | `IfNotPresent`
+`podSecurityPolicy.enabled` | If true, create & use PodSecurityPolicy resources | `false`
+`podSecurityPolicy.annotations` | Specify pod annotations in the pod security policy | `{}`
+`cluster.create` | If true, auto-create the StorageOS cluster | `true`
+`cluster.name` | Name of the storageos deployment | `storageos`
+`cluster.namespace` | Namespace to install the StorageOS cluster into | `kube-system`
+`cluster.secretRefName` | Name of the secret containing StorageOS API credentials | `storageos-api`
+`cluster.admin.username` | Username to authenticate to the StorageOS API with | `storageos`
+`cluster.admin.password` | Password to authenticate to the StorageOS API with |
+`cluster.sharedDir` | The path shared into to kubelet container when running kubelet in a container |
+`cluster.kvBackend.embedded` | Use StorageOS embedded etcd | `true`
+`cluster.kvBackend.address` | External etcd address |
+`cluster.kvBackend.backend` | Key-Value store backend name | `etcd`
+`cluster.kvBackend.tlsSecretName` | Name of the secret containing kv backend tls cert |
+`cluster.kvBackend.tlsSecretNamespace` | Namespace of the secret containing kv backend tls cert |
+`cluster.nodeSelectorTerm.key` | Key of the node selector term used for pod placement |
+`cluster.nodeSelectorTerm.value` | Value of the node selector term used for pod placement |
+`cluster.toleration.key` | Key of the pod toleration parameter |
+`cluster.toleration.value` | Value of the pod toleration parameter |
+`cluster.disableTelemetry` | If true, no telemetry data will be collected from the cluster | `false`
+`cluster.images.node.repository` | StorageOS Node container image repository | `storageos/node`
+`cluster.images.node.tag` | StorageOS Node container image tag | `1.2.0`
+`cluster.csi.enable` | If true, CSI driver is enabled | `true`
+
+## Deleting a StorageOS Cluster
+
+Deleting the `StorageOSCluster` custom resource object would delete the
+storageos cluster and all the associated resources.
+
+In the above example,
+
+```bash
+kubectl delete storageoscluster example-storageos
+```
+
+would delete the custom resource and the cluster.
+
+## Uninstalling the Chart
+
+To uninstall/delete the storageos cluster operator deployment:
+
+```bash
+helm delete --purge <release-name>
+```
+
+Learn more about configuring the StorageOS Operator on
+[GitHub](https://github.com/storageos/cluster-operator).

proposed/storageos-operator/app-readme.md

+# StorageOS Operator
+
+[StorageOS](https://storageos.com) is a cloud native, software-defined storage
+platform that transforms commodity server or cloud based disk capacity into
+enterprise-class persistent storage for containers. StorageOS is ideal for
+deploying databases, message busses, and other mission-critical stateful
+solutions, where rapid recovery and fault tolerance are essential.
+
+The StorageOS Operator installs and manages StorageOS within a cluster.
+Cluster nodes may contribute local or attached disk-based storage into a
+distributed pool, which is then available to all cluster members via a
+global namespace.
+
+By default, a minimal configuration of StorageOS is installed. To set advanced
+configurations, disable the default installation of StorageOS and create a
+custom StorageOSCluster resource
+([documentation](https://docs.storageos.com/docs/reference/cluster-operator/examples)).
+
+`Notes: The StorageOS Operator must be installed in the System Project with
+Cluster Role`

proposed/storageos-operator/ci/std-values.yaml

+podSecurityPolicy:
+  enabled: true
+cluster:
+  # Disable cluster creation in CI, should install the operator only.
+  create: false

proposed/storageos-operator/questions.yml

+categories:
+- storage
+labels:
+  io.rancher.certified: partner
+questions:
+- variable: k8sDistro
+  default: rancher
+  description: "Kubernetes Distribution"
+  show_if: false
+
+# Operator image configuration.
+- variable: defaultImage
+  default: true
+  description: "Use default Docker images"
+  label: Use Default Images
+  type: boolean
+  show_subquestion_if: false
+  group: "Container Images"
+  subquestions:
+  - variable: operator.image.pullPolicy
+    default: IfNotPresent
+    description: "Operator Image pull policy"
+    type: enum
+    label: Operator Image pull policy
+    options:
+      - IfNotPresent
+      - Always
+      - Never
+  - variable: operator.image.repository
+    default: "storageos/cluster-operator"
+    description: "StorageOS operator image name"
+    type: string
+    label: StorageOS Operator Image Name
+  - variable: operator.image.tag
+    default: "1.1.0"
+    description: "StorageOS Operator image tag"
+    type: string
+    label: StorageOS Operator Image Tag
+
+# Default minimal cluster configuration.
+- variable: cluster.create
+  default: true
+  type: boolean
+  description: "Install StorageOS cluster with minimal configurations"
+  label: "Install StorageOS cluster"
+  show_subquestion_if: true
+  group: "StorageOS Cluster"
+  subquestions:
+
+  # CSI configuration.
+  - variable: cluster.csi.enable
+    default: true
+    description: "Use Container Storage Interface (CSI) driver"
+    label: Use CSI Driver
+    type: boolean
+
+  # Cluster metadata.
+  - variable: cluster.name
+    default: "storageos"
+    description: "Name of the StorageOS cluster deployment"
+    type: string
+    label: Name
+  - variable: cluster.namespace
+    default: "kube-system"
+    description: "Namespace of the StorageOS cluster deployment. `kube-system` recommended to avoid pre-emption when node is under load."
+    type: string
+    label: Namespace
+
+  # Node container image.
+  - variable: cluster.images.node.repository
+    default: "storageos/node"
+    description: "StorageOS node container image name"
+    type: string
+    label: StorageOS Node Container Image Name
+  - variable: cluster.images.node.tag
+    default: "1.2.0"
+    description: "StorageOS Node container image tag"
+    type: string
+    label: StorageOS Node Container Image Tag
+
+  # Credentials.
+  - variable: cluster.admin.username
+    default: "admin"
+    description: "Username of the StorageOS administrator account"
+    type: string
+    label: Username
+  - variable: cluster.admin.password
+    default: ""
+    description: "Password of the StorageOS administrator account.  If empty, a random password will be generated."
+    type: password
+    label: Password
+
+  # Telemetry.
+  - variable: cluster.disableTelemetry
+    default: false
+    type: boolean
+    description: "Disable telemetry data collection.  See https://docs.storageos.com/docs/reference/telemetry for more information."
+    label: Disable Telemetry
+
+  # KV store backend.
+  - variable: cluster.kvBackend.embedded
+    default: true
+    type: boolean
+    description: "Use embedded KV store for testing.  Select false to use external etcd for production deployments."
+    label: "Use embedded KV store"
+  - variable: cluster.kvBackend.address
+    default: "10.0.0.1:2379"
+    description: "List of etcd targets, in the form ip[:port], separated by semi-colons.  Prefer multiple direct endpoints over a single load-balanced endpoint.  Only used if not using embedded KV store."
+    type: string
+    label: External etcd address(es)
+    show_if: "cluster.kvBackend.embedded=false"
+  - variable: cluster.kvBackend.tlsSecretName
+    required: false
+    default: ""
+    description: "Name of the secret that contains the etcd TLS certs. This secret is typically shared with etcd."
+    type: string
+    label: External etcd TLS secret name
+    show_if: "cluster.kvBackend.embedded=false"
+  - variable: cluster.kvBackend.tlsSecretNamespace
+    required: false
+    default: ""
+    description: "Namespace of the secret that contains the etcd TLS certs. This secret is typically shared with etcd."
+    type: string
+    label: External etcd TLS secret namespace
+    show_if: "cluster.kvBackend.embedded=false"
+
+  # Node Selector Term.
+  - variable: cluster.nodeSelectorTerm.key
+    required: false
+    default: ""
+    description: "Key of the node selector term match expression used to select the nodes to install StorageOS on, e.g. `node-role.kubernetes.io/worker`"
+    type: string
+    label: Node selector term key
+  - variable: cluster.nodeSelectorTerm.value
+    required: false
+    default: "true"
+    description: "Value of the node selector term match expression used to select the nodes to install StorageOS on."
+    type: string
+    label: Node selector term value
+
+  # Pod tolerations.
+  - variable: cluster.toleration.key
+    required: false
+    default: ""
+    description: "Key of pod toleration with operator 'Equal' and effect 'NoSchedule'"
+    type: string
+    label: Pod toleration key
+  - variable: cluster.toleration.value
+    required: false
+    default: ""
+    description: "Value of pod toleration with operator 'Equal' and effect 'NoSchedule'"
+    type: string
+    label: Pod toleration value
+
+  # Shared Directory
+  - variable: cluster.sharedDir
+    required: false
+    default: "/var/lib/kubelet/plugins/kubernetes.io~storageos"
+    description: "Shared Directory should be set if running kubelet in a container. This should be the path shared into to kubelet container, typically: '/var/lib/kubelet/plugins/kubernetes.io~storageos'.  If not set, defaults will be used."
+    type: string
+    label: Shared Directory

proposed/storageos-operator/templates/NOTES.txt

+StorageOS Operator deployed.
+
+If you disabled automatic cluster creation, you can deploy a StorageOS cluster
+by creating a custom StorageOSCluster resource:
+
+1. Create a secret containing StorageOS cluster credentials. This secret
+contains the API username and password that will be used to authenticate to the
+StorageOS cluster. Base64 encode the username and password that you want to use
+for your StorageOS cluster.
+
+apiVersion: v1
+kind: Secret
+metadata:
+  name: storageos-api
+  namespace: default
+  labels:
+    app: storageos
+type: kubernetes.io/storageos
+data:
+  # echo -n '<secret>' | base64
+  apiUsername: c3RvcmFnZW9z
+  apiPassword: c3RvcmFnZW9z
+
+2. Create a StorageOS custom resource that references the secret created
+above (storageos-api in the above example). When the resource is created, the
+cluster will be deployed.
+
+apiVersion: storageos.com/v1
+kind: StorageOSCluster
+metadata:
+  name: example-storageos
+  namespace: default
+spec:
+  secretRefName: storageos-api
+  secretRefNamespace: default
+  csi:
+    enable: true

proposed/storageos-operator/templates/_helpers.tpl

+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "storageos.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "storageos.fullname" -}}
+{{- if .Values.fullnameOverride -}}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- if contains $name .Release.Name -}}
+{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "storageos.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "storageos.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create -}}
+    {{ default (include "storageos.fullname" .) .Values.serviceAccount.name }}
+{{- else -}}
+    {{ default "default" .Values.serviceAccount.name }}
+{{- end -}}
+{{- end -}}

proposed/storageos-operator/templates/cleanup.yaml

+{{- if .Values.cluster.create }}
+
+# ClusterRole, ClusterRoleBinding and ServiceAccounts have hook-failed in
+# hook-delete-policy to make it easy to rerun the whole setup even after a
+# failure, else the rerun fails with existing resource error.
+# Hook delete policy before-hook-creation ensures any other leftover resources
+# from previous run gets deleted when run again.
+# The Job resources will not be deleted to help investigage the failure.
+# Since the resources created by the operator are not managed by the chart, each
+# of them must be individually deleted in separate jobs.
+
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: storageos-cleanup-sa
+  namespace: {{ .Release.Namespace }}
+  annotations:
+    "helm.sh/hook": post-delete
+    "helm.sh/hook-delete-policy": "hook-succeeded, hook-failed, before-hook-creation"
+    "helm.sh/hook-weight": "1"
+
+---
+
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: storageos-cleanup-cr
+  annotations:
+    "helm.sh/hook": post-delete
+    "helm.sh/hook-delete-policy": "hook-succeeded, hook-failed, before-hook-creation"
+    "helm.sh/hook-weight": "1"
+rules:
+# Using apiGroup "apps" for daemonsets fails and the permission error indicates
+# that it's in group "extensions". Not sure if it's a Job specific behavior,
+# because the daemonsets deployed by the operator use "apps" apiGroup.
+- apiGroups:
+  - "extensions"
+  resources:
+  - "daemonsets"
+  verbs:
+  - "delete"
+- apiGroups:
+  - "apps"
+  resources:
+  - "statefulsets"
+  verbs:
+  - "delete"
+- apiGroups:
+  - rbac.authorization.k8s.io
+  resources:
+  - roles
+  - rolebindings
+  - clusterroles
+  - clusterrolebindings
+  verbs:
+  - delete
+- apiGroups:
+  - storage.k8s.io
+  resources:
+  - storageclasses
+  verbs:
+  - delete
+- apiGroups:
+  - apiextensions.k8s.io
+  resources:
+  - customresourcedefinitions
+  verbs:
+  - delete
+- apiGroups:
+  - ""
+  resources:
+  - serviceaccounts
+  - secrets
+  - services
+  verbs:
+  - delete
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: storageos-cleanup-crb
+  annotations:
+    "helm.sh/hook": post-delete
+    "helm.sh/hook-delete-policy": "hook-succeeded, hook-failed, before-hook-creation"
+    "helm.sh/hook-weight": "2"
+subjects:
+- name: storageos-cleanup-sa
+  kind: ServiceAccount
+  namespace: {{ .Release.Namespace }}
+roleRef:
+  name: storageos-cleanup-cr
+  kind: ClusterRole
+  apiGroup: rbac.authorization.k8s.io
+
+---
+
+# Iterate through the Values.cleanup list and create jobs to delete all the
+# unmanaged resources of the cluster.
+
+{{- range .Values.cleanup }}
+
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: "storageos-{{ .name }}-cleanup"
+  namespace: {{ .namespace }}
+  annotations:
+    "helm.sh/hook": post-delete
+    "helm.sh/hook-delete-policy": "hook-succeeded, before-hook-creation"
+    "helm.sh/hook-weight": "3"
+spec:
+  template:
+    spec:
+      serviceAccountName: storageos-cleanup-sa
+      containers:
+      - name: "statefulset-{{ .name }}-cleanup"
+        image: bitnami/kubectl:1.14.1
+        command:
+          - kubectl
+          - -n
+          - {{ $.Values.cluster.namespace }}
+          - delete
+          {{- range .command }}
+          - {{ . | quote }}
+          {{- end }}
+      restartPolicy: Never
+  backoffLimit: 4
+
+---
+
+{{- end }}
+
+
+{{- end }}

proposed/storageos-operator/templates/job_crd.yaml

+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: jobs.storageos.com
+  annotations:
+    "helm.sh/hook": crd-install
+spec:
+  group: storageos.com
+  names:
+    kind: Job
+    listKind: JobList
+    plural: jobs
+    singular: job
+  scope: Namespaced
+  version: v1
+  validation:
+    openAPIV3Schema:
+      properties:
+        apiVersion:
+          type: string
+        kind:
+          type: string
+        metadata: {}
+        spec:
+          properties:
+            image:
+              type: string
+            args: {}
+            mountPath:
+              type: string
+            hostPath:
+              type: string
+            completionWord:
+              type: string
+            labelSelector:
+              type: string
+            nodeSelectorTerms: {}
+            tolerations: {}
+        status:
+          properties:
+            completed:
+              type: boolean

proposed/storageos-operator/templates/operator.yaml

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ template "storageos.fullname" . }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: {{ template "storageos.name" . }}
+      release: {{ .Release.Name }}
+  template:
+    metadata:
+      labels:
+        app: {{ template "storageos.name" . }}
+        release: {{ .Release.Name }}
+    spec:
+      serviceAccountName: {{ template "storageos.serviceAccountName" . }}
+      containers:
+        - name: storageos-operator
+          image: "{{ .Values.operator.image.repository }}:{{ .Values.operator.image.tag }}"
+          imagePullPolicy: {{ .Values.operator.image.pullPolicy }}
+          ports:
+          - containerPort: 60000
+            name: metrics
+          command:
+          - cluster-operator
+          env:
+            - name: WATCH_NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+            - name: OPERATOR_NAME
+              value: "cluster-operator"

proposed/storageos-operator/templates/psp.yaml

+{{- if .Values.podSecurityPolicy.enabled }}
+
+apiVersion: extensions/v1beta1
+kind: PodSecurityPolicy
+metadata:
+  name: {{ template "storageos.fullname" . }}-psp
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+  annotations:
+{{- if .Values.podSecurityPolicy.annotations }}
+{{ toYaml .Values.podSecurityPolicy.annotations | indent 4 }}
+{{- end }}
+spec:
+  volumes:
+  - '*'
+  runAsUser:
+    rule: 'RunAsAny'
+  seLinux:
+    rule: 'RunAsAny'
+  supplementalGroups:
+    rule: 'RunAsAny'
+  fsGroup:
+    rule: 'RunAsAny'
+
+{{- end }}

proposed/storageos-operator/templates/rbac.yaml

+# Role for storageos operator
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: storageos:operator
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+rules:
+- apiGroups:
+  - storageos.com
+  resources:
+  - storageosclusters
+  - storageosupgrades
+  - jobs
+  verbs:
+  - "*"
+- apiGroups:
+  - apps
+  resources:
+  - statefulsets
+  - daemonsets
+  verbs:
+  - "*"
+- apiGroups:
+  - ""
+  resources:
+  - nodes
+  verbs:
+  - list
+  - watch
+  - get
+  - update
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - list
+  - watch
+  - get
+  - update
+  - patch
+  - delete
+- apiGroups:
+  - ""
+  resources:
+  - events
+  - namespaces
+  - serviceaccounts
+  - secrets
+  - services
+  - persistentvolumeclaims
+  - persistentvolumes
+  verbs:
+  - create
+  - patch
+  - get
+  - list
+  - delete
+  - watch
+  - update
+- apiGroups:
+  - rbac.authorization.k8s.io
+  resources:
+  - roles
+  - rolebindings
+  - clusterroles
+  - clusterrolebindings
+  verbs:
+  - create
+  - delete
+- apiGroups:
+  - storage.k8s.io
+  resources:
+  - storageclasses
+  - volumeattachments
+  - csinodeinfos
+  verbs:
+  - create
+  - delete
+  - watch
+  - list
+  - get
+  - update
+- apiGroups:
+  - apiextensions.k8s.io
+  resources:
+  - customresourcedefinitions
+  verbs:
+  - create
+  - delete
+- apiGroups:
+  - csi.storage.k8s.io
+  resources:
+  - csidrivers
+  verbs:
+  - create
+  - delete
+
+---
+
+# Bind operator service account to storageos-operator role
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: storageos:operator
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+subjects:
+- kind: ServiceAccount
+  name: {{ template "storageos.serviceAccountName" . }}
+  namespace: {{ .Release.Namespace }}
+roleRef:
+  kind: ClusterRole
+  name: storageos:operator
+  apiGroup: rbac.authorization.k8s.io
+
+{{- if .Values.podSecurityPolicy.enabled }}
+---
+
+# ClusterRole for using pod security policy.
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: storageos:psp-user
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+rules:
+- apiGroups: ["extensions"]
+  resources: ["podsecuritypolicies"]
+  verbs: ["use"]
+  resourceNames:
+  - {{ template "storageos.fullname" . }}-psp
+
+---
+
+# Bind pod security policy cluster role to the operator service account.
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: storageos:psp-user
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+roleRef:
+   apiGroup: rbac.authorization.k8s.io
+   kind: ClusterRole
+   name: storageos:psp-user
+subjects:
+  - kind: ServiceAccount
+    name: {{ template "storageos.serviceAccountName" . }}
+    namespace: {{ .Release.Namespace }}
+
+{{- end }}

proposed/storageos-operator/templates/secrets.yaml

+{{- if .Values.cluster.create }}
+
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ .Values.cluster.secretRefName }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+type: "kubernetes.io/storageos"
+data:
+  apiUsername: {{ default "" .Values.cluster.admin.username | b64enc | quote }}
+  {{ if .Values.cluster.admin.password }}
+  apiPassword: {{ default "" .Values.cluster.admin.password | b64enc | quote }}
+  {{ else }}
+  apiPassword: {{ randAlphaNum 10 | b64enc | quote }}
+  {{ end }}
+  # Add base64 encoded TLS cert and key below if ingress.tls is set to true.
+  # tls.crt:
+  # tls.key:
+  # Add base64 encoded creds below for CSI credentials.
+  # csiProvisionUsername:
+  # csiProvisionPassword:
+  # csiControllerPublishUsername:
+  # csiControllerPublishPassword:
+  # csiNodePublishUsername:
+  # csiNodePublishPassword:
+
+{{- end }}

proposed/storageos-operator/templates/service-account.yaml

+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: {{ template "storageos.serviceAccountName" . }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app: {{ template "storageos.name" . }}
+    chart: {{ template "storageos.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}

proposed/storageos-operator/templates/storageoscluster_cr.yaml

+{{- if .Values.cluster.create }}
+
+apiVersion: storageos.com/v1
+kind: StorageOSCluster
+metadata:
+  name: {{ .Values.cluster.name }}
+  namespace: {{ .Release.Namespace }}
+spec:
+  namespace: {{ .Values.cluster.namespace }}
+  secretRefName: {{ .Values.cluster.secretRefName }}
+  secretRefNamespace: {{ .Release.Namespace }}
+  disableTelemetry: {{ .Values.cluster.disableTelemetry }}
+
+  {{- if .Values.k8sDistro }}
+  k8sDistro: {{ .Values.k8sDistro }}
+  {{- end }}
+
+  {{- if .Values.cluster.images.node.repository }}
+  images:
+    nodeContainer: "{{ .Values.cluster.images.node.repository }}:{{ .Values.cluster.images.node.tag }}"
+  {{- end }}
+
+  csi:
+    enable: {{ .Values.cluster.csi.enable }}
+
+  {{- if .Values.cluster.sharedDir }}
+  sharedDir: {{ .Values.cluster.sharedDir }}
+  {{- end }}
+
+  {{- if eq .Values.cluster.kvBackend.embedded false }}
+  kvBackend:
+    address: {{ .Values.cluster.kvBackend.address }}
+    backend: {{ .Values.cluster.kvBackend.backend }}
+  {{- end }}
+
+  {{- if .Values.cluster.nodeSelectorTerm.key }}
+  nodeSelectorTerms:
+    - matchExpressions:
+        - key: {{ .Values.cluster.nodeSelectorTerm.key }}
+          operator: In
+          values:
+          - "{{ .Values.cluster.nodeSelectorTerm.value }}"
+  {{- end }}
+
+  {{- if .Values.cluster.toleration.key }}
+  tolerations:
+    - key: {{ .Values.cluster.toleration.key }}
+      operator: "Equal"
+      value: {{ .Values.cluster.toleration.value }}
+      effect: "NoSchedule"
+  {{- end }}
+
+{{- end }}

proposed/storageos-operator/templates/storageoscluster_crd.yaml

+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: storageosclusters.storageos.com
+  annotations:
+    "helm.sh/hook": crd-install
+spec:
+  group: storageos.com
+  names:
+    kind: StorageOSCluster
+    listKind: StorageOSClusterList
+    plural: storageosclusters
+    singular: storageoscluster
+    shortNames:
+    - stos
+  scope: Namespaced
+  version: v1
+  additionalPrinterColumns:
+  - name: Ready
+    type: string
+    description: Ready status of the storageos nodes.
+    JSONPath: .status.ready
+  - name: Status
+    type: string
+    description: Status of the whole cluster.
+    JSONPath: .status.phase
+  - name: Age
+    type: date
+    JSONPath: .metadata.creationTimestamp
+  validation:
+    openAPIV3Schema:
+      properties:
+        apiVersion:
+          type: string
+        kind:
+          type: string
+        metadata: {}
+        spec:
+          properties:
+            join:
+              type: string
+            namespace:
+              type: string
+            disableFencing:
+              type: boolean
+            disableTelemetry:
+              type: boolean
+            images:
+              properties:
+                nodeContainer:
+                  type: string
+                initContainer:
+                  type: string
+                csiDriverRegistrarContainer:
+                  type: string
+                csiExternalProvisionerContainer:
+                  type: string
+                csiExternalAttacherContainer:
+                  type: string
+            csi:
+              properties:
+                enable:
+                  type: boolean
+                enableProvisionCreds:
+                  type: boolean
+                enableControllerPublishCreds:
+                  type: boolean
+                enableNodePublishCreds:
+                  type: boolean
+            service:
+              properties:
+                name:
+                  type: string
+                type:
+                  type: string
+                externalPort:
+                  type: integer
+                  format: int32
+                internalPort:
+                  type: integer
+                  format: int32
+            secretRefName:
+              type: string
+            secretRefNamespace:
+              type: string
+            tlsEtcdSecretRefName:
+              type: string
+            tlsEtcdSecretRefNamespace:
+              type: string
+            sharedDir:
+              type: string
+            ingress:
+              properties:
+                enable:
+                  type: boolean
+                hostname:
+                  type: string
+                tls:
+                  type: boolean
+                annotations: {}
+            kvBackend:
+              properties:
+                address:
+                  type: string
+                backend:
+                  type: string
+            pause:
+              type: boolean
+            debug:
+              type: boolean
+            nodeSelectorTerms: {}
+            tolerations: {}
+            resources:
+              properties:
+                limits: {}
+                requests: {}
+        status:
+          properties:
+            phase:
+              type: string
+            nodeHealthStatus: {}
+            nodes:
+              type: array
+              items:
+                type: string
+            ready:
+              type: string
+            members:
+              properties:
+                ready: {}
+                unready: {}

proposed/storageos-operator/templates/storageosupgrade_crd.yaml

+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: storageosupgrades.storageos.com
+  annotations:
+    "helm.sh/hook": crd-install
+spec:
+  group: storageos.com
+  names:
+    kind: StorageOSUpgrade
+    listKind: StorageOSUpgradeList
+    plural: storageosupgrades
+    singular: storageosupgrade
+  scope: Namespaced
+  version: v1
+  validation:
+    openAPIV3Schema:
+      properties:
+        apiVersion:
+          type: string
+        kind:
+          type: string
+        metadata: {}
+        spec:
+          properties:
+            newImage:
+              type: string
+        status:
+          properties:
+            completed:
+              type: boolean

proposed/storageos-operator/values.yaml

+# Default values for storageos.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+name: storageos-operator
+
+k8sDistro: default
+
+serviceAccount:
+  create: true
+  name: storageos-operator-sa
+
+podSecurityPolicy:
+  enabled: false
+  annotations: {}
+    ## Specify pod annotations
+    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#apparmor
+    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#seccomp
+    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#sysctl
+    ##
+    # seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
+    # seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default'
+    # apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default'
+
+# operator-specific configuation parameters.
+operator:
+
+  image:
+    repository: storageos/cluster-operator
+    tag: 1.1.0
+    pullPolicy: IfNotPresent
+
+# cluster-specific configuation parameters.
+cluster:
+  # set create to true if the operator should auto-create the StorageOS cluster.
+  create: true
+
+  # Name of the deployment.
+  name: storageos
+
+  # Namespace to install the StorageOS cluster into.
+  namespace: kube-system
+
+  # Name of the secret containing StorageOS API credentials.
+  secretRefName: storageos-api
+
+  # Default admin account.
+  admin:
+
+    # Username to authenticate to the StorageOS API with.
+    username: storageos
+
+    # Password to authenticate to the StorageOS API with. If empty, a random
+    # password will be generated and set in the secretRefName secret.
+    password:
+
+  # sharedDir should be set if running kubelet in a container.  This should
+  # be the path shared into to kubelet container, typically:
+  # "/var/lib/kubelet/plugins/kubernetes.io~storageos".  If not set, defaults
+  # will be used.
+  sharedDir:
+
+  # Key-Value store backend.
+  kvBackend:
+    embedded: true
+    address:
+    backend: etcd
+    tlsSecretName:
+    tlsSecretNamespace:
+
+  # Node selector terms to install StorageOS on.
+  nodeSelectorTerm:
+    key:
+    value:
+
+  # Pod toleration for the StorageOS pods.
+  toleration:
+    key:
+    value:
+
+  # To disable anonymous usage reporting across the cluster, set to true.
+  # Defaults to false. To help improve the product, data such as API usage and
+  # StorageOS configuration information is collected.
+  disableTelemetry: false
+
+  images:
+    # nodeContainer is the StorageOS node image to use, available from the
+    # [Docker Hub](https://hub.docker.com/r/storageos/node/).
+    node:
+      repository: storageos/node
+      tag: 1.2.0
+
+  csi:
+    enable: true
+
+# The following is used for cleaning up unmanaged cluster resources when
+# auto-install is enabled.
+cleanup:
+  - name: daemonset
+    command:
+      - "daemonset"
+      - "storageos-daemonset"
+  - name: statefulset
+    command:
+      - "statefulset"
+      - "storageos-statefulset"
+  - name: serviceaccount
+    command:
+      - "serviceaccount"
+      - "storageos-daemonset-sa"
+      - "storageos-statefulset-sa"
+  - name: role
+    command:
+      - "role"
+      - "storageos:key-management"
+  - name: rolebinding
+    command:
+      - "rolebinding"
+      - "storageos:key-management"
+  - name: secret
+    command:
+      - "secret"
+      - "init-secret"
+  - name: service
+    command:
+      - "service"
+      - "storageos"
+  - name: clusterrole
+    command:
+      - "clusterrole"
+      - "storageos:driver-registrar"
+      - "storageos:csi-attacher"
+      - "storageos:csi-provisioner"
+  - name: clusterrolebinding
+    command:
+      - "clusterrolebinding"
+      - "storageos:csi-provisioner"
+      - "storageos:csi-attacher"
+      - "storageos:driver-registrar"
+      - "storageos:k8s-driver-registrar"
+  - name: storageclass
+    command:
+      - "storageclass"
+      - "fast"