Bump hadoop to v1.1.1

charts/hadoop/v1.1.1/Chart.yaml

 apiVersion: v1
 description: The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.
 name: hadoop
-version: 1.0.7
-appVersion: 2.7.3
+version: 1.1.1
+appVersion: 2.9.0
 home: https://hadoop.apache.org/
 sources:
 - https://github.com/apache/hadoop
-icon: http://hadoop.apache.org/images/hadoop-logo.jpg
+icon: file://../hadoop-logo.png
 maintainers:
 - name: danisla
   email: disla@google.com

charts/hadoop/v1.1.1/README.md

@@ -25,14 +25,16 @@ The following table lists the configurable parameters of the Hadoop chart and th
 | Parameter                                         | Description                                                                        | Default                                                          |
 | ------------------------------------------------- | -------------------------------                                                    | ---------------------------------------------------------------- |
 | `image.repository`                                | Hadoop image ([source](https://github.com/Comcast/kube-yarn/tree/master/image))    | `danisla/hadoop`                                                 |
-| `image.tag`                                   | Version of hadoop libraries being used                                              | `{VERSION}`                                                      |
-| `image.pullPolicy`                                 | Pull policy for the images                                                         | `IfNotPresent`                                                   |
+| `image.tag`                                       | Hadoop image tag                                                                   | `2.9.0`                                                          |
+| `imagee.pullPolicy`                               | Pull policy for the images                                                         | `IfNotPresent`                                                   |
+| `hadoopVersion`                                   | Version of hadoop libraries being used                                             | `2.9.0`                                                          |
 | `antiAffinity`                                    | Pod antiaffinity, `hard` or `soft`                                                 | `hard`                                                           |
 | `hdfs.nameNode.pdbMinAvailable`                   | PDB for HDFS NameNode                                                              | `1`                                                              |
 | `hdfs.nameNode.resources`                         | resources for the HDFS NameNode                                                    | `requests:memory=256Mi,cpu=10m,limits:memory=2048Mi,cpu=1000m`   |
 | `hdfs.dataNode.replicas`                          | Number of HDFS DataNode replicas                                                   | `1`                                                              |
 | `hdfs.dataNode.pdbMinAvailable`                   | PDB for HDFS DataNode                                                              | `1`                                                              |
 | `hdfs.dataNode.resources`                         | resources for the HDFS DataNode                                                    | `requests:memory=256Mi,cpu=10m,limits:memory=2048Mi,cpu=1000m`   |
+| `hdfs.webhdfs.enabled`                            | Enable WebHDFS REST API                                                            | `false`
 | `yarn.resourceManager.pdbMinAvailable`            | PDB for the YARN ResourceManager                                                   | `1`                                                              |
 | `yarn.resourceManager.resources`                  | resources for the YARN ResourceManager                                             | `requests:memory=256Mi,cpu=10m,limits:memory=2048Mi,cpu=1000m`   |
 | `yarn.nodeManager.pdbMinAvailable`                | PDB for the YARN NodeManager                                                       | `1`                                                              |

charts/hadoop/v1.1.1/charts/zeppelin/Chart.yaml

 apiVersion: v1
 description: Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.
 name: zeppelin
-version: 1.0.1
+version: 1.1.0
 appVersion: 0.7.2
 home: https://zeppelin.apache.org/
 sources:

charts/hadoop/v1.1.1/charts/zeppelin/README.md

@@ -18,6 +18,12 @@ The following table lists the configurable parameters of the Zeppelin chart and
 | `hadoop.useConfigMap`                | Use external Hadoop configuration for Spark executors             | `false`                                                    |
 | `hadoop.configMapName`               | Name of the hadoop config map to use (must be in same namespace)  | `hadoop-config`                                            |
 | `hadoop.configPath`                  | Path in the Zeppelin image where the Hadoop config is mounted     | `/usr/hadoop-2.7.3/etc/hadoop`                             |
+| `ingress.enabled`                    | Enable ingress                                                    | `false`                                                    |
+| `ingress.annotations`                | Ingress annotations                                               | `{}`                                                       |
+| `ingress.hosts`                      | Ingress Hostnames                                                 | `["zeppelin.local"]`                                       |
+| `ingress.path`                       | Path within the URL structure                                     | `/`                                                        |
+| `ingress.tls`                        | Ingress TLS configuration                                         | `[]`                                                       |
+| `nodeSelecor`                        | Node selector for the Zeppelin deployment                         | `{}`                                                       |
 
 ## Related charts
 

charts/hadoop/v1.1.1/charts/zeppelin/questions.yml

-questions:
-- variable: defaultImage
-  default: "true"
-  description: "Use default Docker image"
-  label: Use Default Image
-  type: boolean
-  show_subquestion_if: false
-  group: "Container Images"
-  subquestions:
-  - variable: zeppelin.image.repository
-    default: "dylanmei/zeppelin"
-    description: "Zeppelin image name"
-    type: string
-    label: Zeppelin Image Name
-  - variable: zeppelin.image.tag
-    default: "0.7.2"
-    description: "Zeppelin image tag"
-    type: string
-    label: Zeppelin Image Tag
-- variable: hadoop.useConfigMap
-  default: true
-  description: "Expose Zeppelin using Layer 7 Load Balancer - ingress"
-  type: boolean
-  label: Expose Zeppelin using Layer 7 Load Balancer
-  show_subquestion_if: true
-  group: "Hadoop Configuration"
-  required: true
-- variable: ingress.enabled
-  default: "true"
-  description: "Expose Zeppelin using Layer 7 Load Balancer - ingress"
-  type: boolean
-  label: Expose Zeppelin using Layer 7 Load Balancer
-  show_subquestion_if: true
-  group: "Zeppelin"
-  required: true
-  subquestions:
-  - variable: ingress.hosts[0]
-    default: "xip.io"
-    description: "Hostname to your Zeppelin installation"
-    type: hostname
-    required: true
-    label: Hostname
-- variable: server.type
-  default: "ClusterIP"
-  description: "yarn ui service type"
-  type: enum
-  group: "Zeppelin"
-  options:
-    - "ClusterIP"
-    - "NodePort"
-  required: true
-  label: Zeppelin Service Type
-  show_subquestion_if: "NodePort"
-  show_if: "ingress.enabled=false"
-  subquestions:
-  - variable: service.nodePort
-    default: ""
-    description: "NodePort http port(to set explicitly, choose port between 30000-32767)"
-    type: int
-    min: 30000
-    max: 32767
-    show_if: "ingress.enabled=false"
-    label: Zeppelin NodePort Number

charts/hadoop/v1.1.1/charts/zeppelin/templates/deployment.yaml

-apiVersion: extensions/v1beta1
+apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: {{ template "zeppelin.fullname" . }}
@@ -8,6 +8,10 @@ metadata:
     release: {{ .Release.Name }}
     heritage: {{ .Release.Service }}
 spec:
+  selector:
+    matchLabels:
+      app: {{ template "zeppelin.name" . }}
+      release: {{ .Release.Name }}
   strategy:
     rollingUpdate:
       maxUnavailable: 0

charts/hadoop/v1.1.1/charts/zeppelin/values.yaml

 zeppelin:
-  zeppelin:
   replicas: 1
   image:
-      repository: dylanmei/zeppelin
+    repository: ranchercharts/dylanmei-zeppelin
     tag: 0.7.2
   resources:
     limits:
@@ -12,18 +11,18 @@ zeppelin:
       memory: "512Mi"
       cpu: "200m"
 
-  hadoop:
+hadoop:
   useConfigMap: true
   # configMapName: hadoop-hadoop
   configPath: /usr/hadoop-2.7.3/etc/hadoop
 
-  spark:
+spark:
   driverMemory: 1g
   executorMemory: 1g
   numExecutors: 2
 
 
-  ingress:
+ingress:
   ## If true, Grafana Ingress will be created
   ##
   enabled: false
@@ -53,7 +52,7 @@ zeppelin:
     #   hosts:
     #     - alertmanager.example.com
 
-  service:
+service:
 
   ## Annotations to be added to the Service
   ##

charts/hadoop/v1.1.1/questions.yml

+labels:
+  io.cattle.role: project # options are cluster/project
 categories:
 - hadoop
 questions:
@@ -10,17 +12,17 @@ questions:
   group: "Container Images"
   subquestions:
   - variable: image.repository
-    default: "danisla/hadoop"
+    default: "ranchercharts/danisla-hadoop"
     description: "Hadoop image name"
     type: string
     label: Hadoop Image Name
   - variable: image.tag
-    default: "2.7.3"
+    default: "2.9.0"
     description: "Hadoop image tag"
     type: string
     label: Hadoop Image Tag
   - variable: zeppelin.zeppelin.image.repository
-    default: "dylanmei/zeppelin"
+    default: "ranchercharts/dylanmei-zeppelin"
     description: "Zeppelin image name"
     type: string
     label: Zeppelin Image Name

charts/hadoop/v1.1.1/templates/NOTES.txt

 1. You can check the status of HDFS by running this command:
-   kubectl exec -n {{ .Release.Namespace }} -it {{ template "hadoop.fullname" . }}-hdfs-nn-0 -- /usr/local/hadoop/bin/hdfs dfsadmin -report
+   kubectl exec -n {{ .Release.Namespace }} -it {{ include "hadoop.fullname" . }}-hdfs-nn-0 -- /usr/local/hadoop/bin/hdfs dfsadmin -report
 
 2. You can list the yarn nodes by running this command:
-   kubectl exec -n {{ .Release.Namespace }} -it {{ template "hadoop.fullname" . }}-yarn-rm-0 -- /usr/local/hadoop/bin/yarn node -list
+   kubectl exec -n {{ .Release.Namespace }} -it {{ include "hadoop.fullname" . }}-yarn-rm-0 -- /usr/local/hadoop/bin/yarn node -list
 
 3. Create a port-forward to the yarn resource manager UI:
-   kubectl port-forward -n {{ .Release.Namespace }} {{ template "hadoop.fullname" . }}-yarn-rm-0 8088:8088 
+   kubectl port-forward -n {{ .Release.Namespace }} {{ include "hadoop.fullname" . }}-yarn-rm-0 8088:8088
 
    Then open the ui in your browser:
 
    open http://localhost:8088
 
 4. You can run included hadoop tests like this:
-   kubectl exec -n {{ .Release.Namespace }} -it {{ template "hadoop.fullname" . }}-yarn-nm-0 -- /usr/local/hadoop/bin/hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-{{ .Values.image.tag }}-tests.jar TestDFSIO -write -nrFiles 5 -fileSize 128MB -resFile /tmp/TestDFSIOwrite.txt
+   kubectl exec -n {{ .Release.Namespace }} -it {{ include "hadoop.fullname" . }}-yarn-nm-0 -- /usr/local/hadoop/bin/hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-{{ .Values.hadoopVersion }}-tests.jar TestDFSIO -write -nrFiles 5 -fileSize 128MB -resFile /tmp/TestDFSIOwrite.txt
 
 5. You can list the mapreduce jobs like this:
-   kubectl exec -n {{ .Release.Namespace }} -it {{ template "hadoop.fullname" . }}-yarn-rm-0 -- /usr/local/hadoop/bin/mapred job -list
+   kubectl exec -n {{ .Release.Namespace }} -it {{ include "hadoop.fullname" . }}-yarn-rm-0 -- /usr/local/hadoop/bin/mapred job -list
 
 6. This chart can also be used with the zeppelin chart
-    helm install --namespace {{ .Release.Namespace }} --set hadoop.useConfigMap=true,hadoop.configMapName={{ template "hadoop.fullname" . }} stable/zeppelin
+    helm install --namespace {{ .Release.Namespace }} --set hadoop.useConfigMap=true,hadoop.configMapName={{ include "hadoop.fullname" . }} stable/zeppelin
 
 7. You can scale the number of yarn nodes like this:
    helm upgrade {{ .Release.Name }} --set yarn.nodeManager.replicas=4 stable/hadoop

charts/hadoop/v1.1.1/templates/_helpers.yaml → charts/hadoop/v1.1.1/templates/_helpers.tpl

@@ -14,3 +14,10 @@ We truncate at 63 chars because some Kubernetes name fields are limited to this
 {{- $name := default .Chart.Name .Values.nameOverride -}}
 {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
 {{- end -}}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "hadoop.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end -}}

charts/hadoop/v1.1.1/templates/hadoop-configmap.yaml

 apiVersion: v1
 kind: ConfigMap
 metadata:
-  name: {{ template "hadoop.fullname" . }}
+  name: {{ include "hadoop.fullname" . }}
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
 data:
   bootstrap.sh: |
     #!/bin/bash
@@ -42,7 +41,7 @@ data:
       mkdir -p /root/hdfs/datanode
 
       #  wait up to 30 seconds for namenode
-      (while [[ $count -lt 15 && -z `curl -sf http://{{ template "hadoop.fullname" . }}-hdfs-nn:50070` ]]; do ((count=count+1)) ; echo "Waiting for {{ template "hadoop.fullname" . }}-hdfs-nn" ; sleep 2; done && [[ $count -lt 15 ]])
+      (while [[ $count -lt 15 && -z `curl -sf http://{{ include "hadoop.fullname" . }}-hdfs-nn:50070` ]]; do ((count=count+1)) ; echo "Waiting for {{ include "hadoop.fullname" . }}-hdfs-nn" ; sleep 2; done && [[ $count -lt 15 ]])
       [[ $? -ne 0 ]] && echo "Timeout waiting for hdfs-nn, exiting." && exit 1
 
       $HADOOP_PREFIX/sbin/hadoop-daemon.sh start datanode
@@ -74,7 +73,7 @@ data:
       chmod +x start-yarn-nm.sh
 
       #  wait up to 30 seconds for resourcemanager
-      (while [[ $count -lt 15 && -z `curl -sf http://{{ template "hadoop.fullname" . }}-yarn-rm:8088/ws/v1/cluster/info` ]]; do ((count=count+1)) ; echo "Waiting for {{ template "hadoop.fullname" . }}-yarn-rm" ; sleep 2; done && [[ $count -lt 15 ]])
+      (while [[ $count -lt 15 && -z `curl -sf http://{{ include "hadoop.fullname" . }}-yarn-rm:8088/ws/v1/cluster/info` ]]; do ((count=count+1)) ; echo "Waiting for {{ include "hadoop.fullname" . }}-yarn-rm" ; sleep 2; done && [[ $count -lt 15 ]])
       [[ $? -ne 0 ]] && echo "Timeout waiting for yarn-rm, exiting." && exit 1
 
       ./start-yarn-nm.sh
@@ -96,7 +95,7 @@ data:
     <configuration>
       <property>
             <name>fs.defaultFS</name>
-            <value>hdfs://{{ template "hadoop.fullname" . }}-hdfs-nn:9000/</value>
+            <value>hdfs://{{ include "hadoop.fullname" . }}-hdfs-nn:9000/</value>
             <description>NameNode URI</description>
         </property>
     </configuration>
@@ -105,6 +104,14 @@ data:
     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
+    
+{{- if .Values.hdfs.webhdfs.enabled -}}
+      <property>
+          <name>dfs.webhdfs.enabled</name>
+          <value>true</value>
+      </property> 
+{{- end -}}
+
       <property>
         <name>dfs.datanode.use.datanode.hostname</name>
         <value>false</value>
@@ -161,11 +168,11 @@ data:
       </property>
       <property>
         <name>mapreduce.jobhistory.address</name>
-        <value>{{ template "hadoop.fullname" . }}-yarn-rm-0.{{ template "hadoop.fullname" . }}-yarn-rm.{{ .Release.Namespace }}.svc.cluster.local:10020</value>
+        <value>{{ include "hadoop.fullname" . }}-yarn-rm-0.{{ include "hadoop.fullname" . }}-yarn-rm.{{ .Release.Namespace }}.svc.cluster.local:10020</value>
       </property>
       <property>
         <name>mapreduce.jobhistory.webapp.address</name>
-        <value>{{ template "hadoop.fullname" . }}-yarn-rm-0.{{ template "hadoop.fullname" . }}-yarn-rm.{{ .Release.Namespace }}.svc.cluster.local:19888</value>
+        <value>{{ include "hadoop.fullname" . }}-yarn-rm-0.{{ include "hadoop.fullname" . }}-yarn-rm.{{ .Release.Namespace }}.svc.cluster.local:19888</value>
       </property>
     </configuration>
 
@@ -253,7 +260,7 @@ data:
     <configuration>
       <property>
         <name>yarn.resourcemanager.hostname</name>
-        <value>{{ template "hadoop.fullname" . }}-yarn-rm</value>
+        <value>{{ include "hadoop.fullname" . }}-yarn-rm</value>
       </property>
 
       <!-- Bind to all interfaces -->

charts/hadoop/v1.1.1/templates/hdfs-dn-pdb.yaml

 apiVersion: policy/v1beta1
 kind: PodDisruptionBudget
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-dn
+  name: {{ include "hadoop.fullname" . }}-hdfs-dn
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-dn
 spec:
   selector:
     matchLabels:
-      app: {{ template "hadoop.name" . }}
+      app: {{ include "hadoop.name" . }}
       release: {{ .Release.Name }}
       component: hdfs-dn
   minAvailable: {{ .Values.hdfs.dataNode.pdbMinAvailable }}
-  
\ No newline at end of file

charts/hadoop/v1.1.1/templates/hdfs-dn-pvc.yaml

@@ -2,12 +2,11 @@
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-dn
+  name: {{ include "hadoop.fullname" . }}-hdfs-dn
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-dn
 spec:
   accessModes:

charts/hadoop/v1.1.1/templates/hdfs-dn-statefulset.yaml

-apiVersion: apps/v1beta1
+apiVersion: apps/v1
 kind: StatefulSet
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-dn
+  name: {{ include "hadoop.fullname" . }}-hdfs-dn
   annotations:
     checksum/config: {{ include (print $.Template.BasePath "/hadoop-configmap.yaml") . | sha256sum }}
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-dn
 spec:
-  serviceName: {{ template "hadoop.fullname" . }}-hdfs-dn
+  selector:
+    matchLabels:
+      app: {{ include "hadoop.name" . }}
+      release: {{ .Release.Name }}
+      component: hdfs-dn
+  serviceName: {{ include "hadoop.fullname" . }}-hdfs-dn
   replicas: {{ .Values.hdfs.dataNode.replicas }}
   template:
     metadata:
       labels:
-        app: {{ template "hadoop.name" . }}
+        app: {{ include "hadoop.name" . }}
         release: {{ .Release.Name }}
         component: hdfs-dn
     spec:
@@ -27,7 +31,7 @@ spec:
           - topologyKey: "kubernetes.io/hostname"
             labelSelector:
               matchLabels:
-                app:  {{ template "hadoop.name" . }}
+                app:  {{ include "hadoop.name" . }}
                 release: {{ .Release.Name | quote }}
                 component: hdfs-dn
         {{- else if eq .Values.antiAffinity "soft" }}
@@ -37,15 +41,15 @@ spec:
               topologyKey: "kubernetes.io/hostname"
               labelSelector:
                 matchLabels:
-                  app:  {{ template "hadoop.name" . }}
+                  app:  {{ include "hadoop.name" . }}
                   release: {{ .Release.Name | quote }}
                   component: hdfs-dn
         {{- end }}
       terminationGracePeriodSeconds: 0
       containers:
       - name: hdfs-dn
-        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
-        imagePullPolicy: {{ .Values.image.pullPolicy }}
+        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+        imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
         command:
            - "/bin/bash"
            - "/tmp/hadoop-config/bootstrap.sh"
@@ -72,11 +76,11 @@ spec:
       volumes:
       - name: hadoop-config
         configMap:
-          name: {{ template "hadoop.fullname" . }}
+          name: {{ include "hadoop.fullname" . }}
       - name: dfs
       {{- if .Values.persistence.dataNode.enabled }}
         persistentVolumeClaim:
-          claimName: {{ template "hadoop.fullname" . }}-hdfs-dn
+          claimName: {{ include "hadoop.fullname" . }}-hdfs-dn
       {{- else }}
         emptyDir: {}
       {{- end }}

charts/hadoop/v1.1.1/templates/hdfs-dn-svc.yaml

@@ -2,12 +2,11 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-dn
+  name: {{ include "hadoop.fullname" . }}-hdfs-dn
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-dn
 spec:
   ports:
@@ -18,6 +17,6 @@ spec:
     port: 50075
   clusterIP: None
   selector:
-    app: {{ template "hadoop.name" . }}
+    app: {{ include "hadoop.name" . }}
     release: {{ .Release.Name }}
     component: hdfs-dn
\ No newline at end of file

charts/hadoop/v1.1.1/templates/hdfs-nn-pdb.yaml

 apiVersion: policy/v1beta1
 kind: PodDisruptionBudget
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-nn
+  name: {{ include "hadoop.fullname" . }}-hdfs-nn
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-nn
 spec:
   selector:
     matchLabels:
-      app: {{ template "hadoop.name" . }}
+      app: {{ include "hadoop.name" . }}
       release: {{ .Release.Name }}
       component: hdfs-nn
   minAvailable: {{ .Values.hdfs.nameNode.pdbMinAvailable }}
-  
\ No newline at end of file

charts/hadoop/v1.1.1/templates/hdfs-nn-pvc.yaml

@@ -2,12 +2,11 @@
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-nn
+  name: {{ include "hadoop.fullname" . }}-hdfs-nn
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-nn
 spec:
   accessModes:

charts/hadoop/v1.1.1/templates/hdfs-nn-statefulset.yaml

-apiVersion: apps/v1beta1
+apiVersion: apps/v1
 kind: StatefulSet
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-nn
+  name: {{ include "hadoop.fullname" . }}-hdfs-nn
   annotations:
     checksum/config: {{ include (print $.Template.BasePath "/hadoop-configmap.yaml") . | sha256sum }}
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-nn
 spec:
-  serviceName: {{ template "hadoop.fullname" . }}-hdfs-nn
+  selector:
+    matchLabels:
+      app: {{ include "hadoop.name" . }}
+      release: {{ .Release.Name }}
+      component: hdfs-nn
+  serviceName: {{ include "hadoop.fullname" . }}-hdfs-nn
   replicas: 1
   template:
     metadata:
       labels:
-        app: {{ template "hadoop.name" . }}
+        app: {{ include "hadoop.name" . }}
         release: {{ .Release.Name }}
         component: hdfs-nn
     spec:
@@ -27,7 +31,7 @@ spec:
           - topologyKey: "kubernetes.io/hostname"
             labelSelector:
               matchLabels:
-                app:  {{ template "hadoop.name" . }}
+                app:  {{ include "hadoop.name" . }}
                 release: {{ .Release.Name | quote }}
                 component: hdfs-nn
         {{- else if eq .Values.antiAffinity "soft" }}
@@ -37,15 +41,15 @@ spec:
               topologyKey: "kubernetes.io/hostname"
               labelSelector:
                 matchLabels:
-                  app:  {{ template "hadoop.name" . }}
+                  app:  {{ include "hadoop.name" . }}
                   release: {{ .Release.Name | quote }}
                   component: hdfs-nn
         {{- end }}
       terminationGracePeriodSeconds: 0
       containers:
       - name: hdfs-nn
-        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
-        imagePullPolicy: {{ .Values.image.pullPolicy }}
+        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+        imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
         command:
         - "/bin/bash"
         - "/tmp/hadoop-config/bootstrap.sh"
@@ -72,11 +76,11 @@ spec:
       volumes:
       - name: hadoop-config
         configMap:
-          name: {{ template "hadoop.fullname" . }}
+          name: {{ include "hadoop.fullname" . }}
       - name: dfs
       {{- if .Values.persistence.nameNode.enabled }}
         persistentVolumeClaim:
-          claimName: {{ template "hadoop.fullname" . }}-hdfs-nn
+          claimName: {{ include "hadoop.fullname" . }}-hdfs-nn
       {{- else }}
         emptyDir: {}
       {{- end }}

charts/hadoop/v1.1.1/templates/hdfs-nn-svc.yaml

@@ -2,12 +2,11 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: {{ template "hadoop.fullname" . }}-hdfs-nn
+  name: {{ include "hadoop.fullname" . }}-hdfs-nn
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: hdfs-nn
 spec:
   ports:
@@ -18,6 +17,6 @@ spec:
     port: 50070
   clusterIP: None
   selector:
-    app: {{ template "hadoop.name" . }}
+    app: {{ include "hadoop.name" . }}
     release: {{ .Release.Name }}
     component: hdfs-nn
\ No newline at end of file

charts/hadoop/v1.1.1/templates/yarn-nm-pdb.yaml

 apiVersion: policy/v1beta1
 kind: PodDisruptionBudget
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-nm
+  name: {{ include "hadoop.fullname" . }}-yarn-nm
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-nm
 spec:
   selector:
     matchLabels:
-      app: {{ template "hadoop.name" . }}
+      app: {{ include "hadoop.name" . }}
       release: {{ .Release.Name }}
       component: yarn-nm
   minAvailable: {{ .Values.yarn.nodeManager.pdbMinAvailable }}
-  
\ No newline at end of file

charts/hadoop/v1.1.1/templates/yarn-nm-statefulset.yaml

-apiVersion: apps/v1beta1
+apiVersion: apps/v1
 kind: StatefulSet
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-nm
+  name: {{ include "hadoop.fullname" . }}-yarn-nm
   annotations:
     checksum/config: {{ include (print $.Template.BasePath "/hadoop-configmap.yaml") . | sha256sum }}
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-nm
 spec:
-  podManagementPolicy: Parallel
-  serviceName: {{ template "hadoop.fullname" . }}-yarn-nm
+  selector:
+    matchLabels:
+      app: {{ include "hadoop.name" . }}
+      release: {{ .Release.Name }}
+      component: yarn-nm
+  serviceName: {{ include "hadoop.fullname" . }}-yarn-nm
   replicas: {{ .Values.yarn.nodeManager.replicas }}
 {{- if .Values.yarn.nodeManager.parallelCreate }}
   podManagementPolicy: Parallel
@@ -20,7 +23,7 @@ spec:
   template:
     metadata:
       labels:
-        app: {{ template "hadoop.name" . }}
+        app: {{ include "hadoop.name" . }}
         release: {{ .Release.Name }}
         component: yarn-nm
     spec:
@@ -31,7 +34,7 @@ spec:
           - topologyKey: "kubernetes.io/hostname"
             labelSelector:
               matchLabels:
-                app:  {{ template "hadoop.name" . }}
+                app:  {{ include "hadoop.name" . }}
                 release: {{ .Release.Name | quote }}
                 component: yarn-nm
         {{- else if eq .Values.antiAffinity "soft" }}
@@ -41,15 +44,15 @@ spec:
               topologyKey: "kubernetes.io/hostname"
               labelSelector:
                 matchLabels:
-                  app:  {{ template "hadoop.name" . }}
+                  app:  {{ include "hadoop.name" . }}
                   release: {{ .Release.Name | quote }}
                   component: yarn-nm
         {{- end }}
-      terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
+      terminationGracePeriodSeconds: 0
       containers:
       - name: yarn-nm
-        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
-        imagePullPolicy: {{ .Values.image.pullPolicy }}
+        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+        imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
         ports:
         - containerPort: 8088
           name: web
@@ -90,4 +93,4 @@ spec:
       volumes:
       - name: hadoop-config
         configMap:
-          name: {{ template "hadoop.fullname" . }}
+          name: {{ include "hadoop.fullname" . }}

charts/hadoop/v1.1.1/templates/yarn-nm-svc.yaml

@@ -2,12 +2,11 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-nm
+  name: {{ include "hadoop.fullname" . }}-yarn-nm
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-nm
 spec:
   ports:
@@ -19,6 +18,6 @@ spec:
     name: api
   clusterIP: None
   selector:
-    app: {{ template "hadoop.name" . }}
+    app: {{ include "hadoop.name" . }}
     release: {{ .Release.Name }}
     component: yarn-nm
\ No newline at end of file

charts/hadoop/v1.1.1/templates/yarn-rm-pdb.yaml

 apiVersion: policy/v1beta1
 kind: PodDisruptionBudget
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-rm
+  name: {{ include "hadoop.fullname" . }}-yarn-rm
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-rm
 spec:
   selector:
     matchLabels:
-      app: {{ template "hadoop.name" . }}
+      app: {{ include "hadoop.name" . }}
       release: {{ .Release.Name }}
       component: yarn-rm
   minAvailable: {{ .Values.yarn.resourceManager.pdbMinAvailable }}
-  

charts/hadoop/v1.1.1/templates/yarn-rm-statefulset.yaml

-apiVersion: apps/v1beta1
+apiVersion: apps/v1
 kind: StatefulSet
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-rm
+  name: {{ include "hadoop.fullname" . }}-yarn-rm
   annotations:
     checksum/config: {{ include (print $.Template.BasePath "/hadoop-configmap.yaml") . | sha256sum }}
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-rm
 spec:
-  serviceName: {{ template "hadoop.fullname" . }}-yarn-rm
+  selector:
+    matchLabels:
+      app: {{ include "hadoop.name" . }}
+      release: {{ .Release.Name }}
+      component: yarn-rm
+  serviceName: {{ include "hadoop.fullname" . }}-yarn-rm
   replicas: 1
   template:
     metadata:
       labels:
-        app: {{ template "hadoop.name" . }}
+        app: {{ include "hadoop.name" . }}
         release: {{ .Release.Name }}
         component: yarn-rm
     spec:
@@ -27,7 +31,7 @@ spec:
           - topologyKey: "kubernetes.io/hostname"
             labelSelector:
               matchLabels:
-                app:  {{ template "hadoop.name" . }}
+                app:  {{ include "hadoop.name" . }}
                 release: {{ .Release.Name | quote }}
                 component: yarn-rm
         {{- else if eq .Values.antiAffinity "soft" }}
@@ -37,15 +41,15 @@ spec:
               topologyKey: "kubernetes.io/hostname"
               labelSelector:
                 matchLabels:
-                  app:  {{ template "hadoop.name" . }}
+                  app:  {{ include "hadoop.name" . }}
                   release: {{ .Release.Name | quote }}
                   component: yarn-rm
         {{- end }}
       terminationGracePeriodSeconds: 0
       containers:
       - name: yarn-rm
-        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
-        imagePullPolicy: {{ .Values.image.pullPolicy }}
+        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+        imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
         ports:
         - containerPort: 8088
           name: web
@@ -73,4 +77,4 @@ spec:
       volumes:
       - name: hadoop-config
         configMap:
-          name: {{ template "hadoop.fullname" . }}
+          name: {{ include "hadoop.fullname" . }}

charts/hadoop/v1.1.1/templates/yarn-rm-svc.yaml

@@ -2,12 +2,11 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-rm
+  name: {{ include "hadoop.fullname" . }}-yarn-rm
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-rm
 spec:
   ports:
@@ -15,6 +14,6 @@ spec:
     name: web
   clusterIP: None
   selector:
-    app: {{ template "hadoop.name" . }}
+    app: {{ include "hadoop.name" . }}
     release: {{ .Release.Name }}
     component: yarn-rm
\ No newline at end of file

charts/hadoop/v1.1.1/templates/yarn-ui-ingress.yaml

-{{- if .Values.yarn.ingress.enabled -}}
-{{- $serviceName := include "hadoop.fullname" . }}
-{{- $routePrefix := .Values.yarn.routePrefix }}
-{{- $releaseName := .Release.Name }}
-apiVersion: extensions/v1beta1
-kind: Ingress
-metadata:
-{{- if .Values.yarn.ingress.annotations }}
-  annotations:
-{{ toYaml .Values.yarn.ingress.annotations | indent 4 }}
-{{- end }}
-  labels:
-    app: {{ template "hadoop.fullname" . }}
-    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
-    heritage: "{{ .Release.Service }}"
-    release: "{{ .Release.Name }}"
-{{- if .Values.yarn.ingress.labels }}
-{{ toYaml .Values.yarn.ingress.labels | indent 4 }}
-{{- end }}
-  name: {{ template "hadoop.fullname" . }}
-spec:
-  rules:
-    {{- range $host := .Values.yarn.ingress.hosts }}
-    - host: {{ $host }}
-      http:
-        paths:
-          - path: "{{ $routePrefix }}"
-            backend:
-              serviceName: {{ printf "%s-%s" $serviceName "yarn-ui" }}
-              servicePort: 8088
-    {{- end -}}              
-{{- if .Values.yarn.ingress.tls }}
-  tls:
-{{ toYaml .Values.yarn.ingress.tls | indent 4 }}
-{{- end }}
-{{- end }}

charts/hadoop/v1.1.1/templates/yarn-ui-svc.yaml

@@ -2,23 +2,16 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: {{ template "hadoop.fullname" . }}-yarn-ui
+  name: {{ include "hadoop.fullname" . }}-yarn-ui
   labels:
-    app: {{ template "hadoop.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app: {{ include "hadoop.name" . }}
+    chart: {{ include "hadoop.chart" . }}
     release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
     component: yarn-ui
 spec:
-  type: {{ .Values.yarn.service.type }}
   ports:
-  - name: web
-    port: 8088
-    targetPort: web
-    {{- if (and (eq .Values.yarn.service.type "NodePort") (not (empty .Values.yarn.service.nodePort)))}}
-    nodePort: {{ .Values.yarn.service.nodePort }}
-    {{- end }}
+  - port: 8088
+    name: web
   selector:
-    app: {{ template "hadoop.name" . }}
-    release: {{ .Release.Name }}
+    app: {{ include "hadoop.name" . }}
     component: yarn-rm

charts/hadoop/v1.1.1/values.yaml

 # The base hadoop image to use for all components.
 # See this repo for image build details: https://github.com/Comcast/kube-yarn/tree/master/image
 image:
-  repository: danisla/hadoop
-  tag: 2.7.3
+  repository: ranchercharts/danisla-hadoop
+  tag: 2.9.0
   pullPolicy: IfNotPresent
 
-# Select antiAffinity as either hard or soft, default is hard
-antiAffinity: "soft"
+# The version of the hadoop libraries being used in the image.
+hadoopVersion: 2.9.0
 
-terminationGracePeriodSeconds: 30  # Duration in seconds a pod needs to terminate gracefully.
+# Select antiAffinity as either hard or soft, default is soft
+antiAffinity: "soft"
 
 hdfs:
   nameNode:
@@ -35,6 +36,9 @@ hdfs:
         memory: "2048Mi"
         cpu: "1000m"
 
+  webhdfs:
+    enabled: false
+
 yarn:
   resourceManager:
     pdbMinAvailable: 1
@@ -66,40 +70,6 @@ yarn:
         memory: "2048Mi"
         cpu: "1000m"
 
-  service:
-    type: ClusterIP
-    # nodePort: 32000 #range form 32000-32767
-
-  ingress:
-    ## If true, Grafana Ingress will be created
-    ##
-    enabled: false
-
-    ## Annotations for Alertmanager Ingress
-    ##
-    annotations: {}
-      # kubernetes.io/ingress.class: nginx
-      # kubernetes.io/tls-acme: "true"
-
-    ## Labels to be added to the Ingress
-    ##
-    labels: {}
-    
-    ## Hostnames.
-    ## Must be provided if Ingress is enabled.
-    ##
-    # hosts:
-    #   - grafana.domain.com
-    hosts: []
-
-    ## TLS configuration for Alertmanager Ingress
-    ## Secret must be manually created in the namespace
-    ##
-    tls: []
-      # - secretName: alertmanager-general-tls
-      #   hosts:
-      #     - alertmanager.example.com
-
 persistence:
   nameNode:
     enabled: false
@@ -112,94 +82,3 @@ persistence:
     storageClass: "-"
     accessMode: ReadWriteOnce
     size: 200Gi
-
-zeppelin:
-  enabled: true
-
-  #zeppelin configurations
-  zeppelin:
-    replicas: 1
-    image: 
-      repository: dylanmei/zeppelin
-      tag: 0.7.2
-    resources:
-      limits:
-        memory: "4096Mi"
-        cpu: "2000m"
-      requests:
-        memory: "512Mi"
-        cpu: "200m"
-
-  hadoop:
-    useConfigMap: true
-    # configMapName: hadoop-hadoop
-    configPath: /usr/hadoop-2.7.3/etc/hadoop
-
-  spark:
-    driverMemory: 1g
-    executorMemory: 1g
-    numExecutors: 2
-
-
-  ingress:
-    ## If true, Grafana Ingress will be created
-    ##
-    enabled: false
-
-    ## Annotations for Alertmanager Ingress
-    ##
-    annotations: {}
-      # kubernetes.io/ingress.class: nginx
-      # kubernetes.io/tls-acme: "true"
-
-    ## Labels to be added to the Ingress
-    ##
-    labels: {}
-    
-    ## Hostnames.
-    ## Must be provided if Ingress is enabled.
-    ##
-    # hosts:
-    #   - grafana.domain.com
-    hosts: []
-
-    ## TLS configuration for Alertmanager Ingress
-    ## Secret must be manually created in the namespace
-    ##
-    tls: []
-      # - secretName: alertmanager-general-tls
-      #   hosts:
-      #     - alertmanager.example.com
-
-  service:
-
-    ## Annotations to be added to the Service
-    ##
-    annotations: {}
-
-    ## Cluster-internal IP address for Alertmanager Service
-    ##
-    clusterIP: ""
-
-    ## List of external IP addresses at which the Alertmanager Service will be available
-    ##
-    externalIPs: []
-
-    ## External IP address to assign to Alertmanager Service
-    ## Only used if service.type is 'LoadBalancer' and supported by cloud provider
-    ##
-    loadBalancerIP: ""
-
-    ## List of client IPs allowed to access Alertmanager Service
-    ## Only used if service.type is 'LoadBalancer' and supported by cloud provider
-    ##
-    loadBalancerSourceRanges: []
-
-    ## Port to expose on each node
-    ## Only used if service.type is 'NodePort'
-    ##
-    # nodePort: 30902
-
-    ## Service type
-    ##
-    type: ClusterIP