Whether a component supports sanity testing depends on the
component. For example, csi-driver-host-path enables it because it
makes sense there (and only there). Letting the prow.sh script decide
whether it actually runs simplifies the job definitions in test-infra.

This pulls in
https://github.com/kubernetes-csi/csi-driver-host-path/pull/37, which
turned out to be necessary for pre-submit testing of the
livenessprobe.

This was already meant to be done earlier in cda2fc58.
While at it, extend the permanent TODO with guidance on future feature
gates.

prow enhancements

It turned out to not work. Instead of reverting the commit which
introduced this, let's better document this explicitly.

When KinD fails in a Prow job, we need additional information to
understand why it failed.

Not all environments have Docker. The simplifying assumption here is
that if the Docker command is available, it's also usable.

When running only some tests, sometimes extra, unnecessarily work was
done, like bringing up the cluster without alpha gates.

The previous logic failed for canary jobs, those also deploy a recent
driver. Instead of guessing what driver gets installed based on job
parameters, check what really runs in the cluster and base the
decision on that.

We only need to maintain this blacklist for 1.0.x until we replace it
with 1.1.0, then this entire hostpath_supports_block can be removed.

"grep -w" treated "serial-alpha" as two words and therefore
CSI_PROW_TESTS sometimes ran too many tests.

This ensures that also new, currently unknown alpha gates are enabled
when testing against a future Kubernetes versions. For all currently
known Kubernetes versions we just use the minimal set of alpha gates,
which ensures that we don't miss any of them in our documentation.

Prow testing

The temporary fork was merged, we can use the upstream repo again.

Instead of always using the latest E2E tests for all Kubernetes
versions, the plan now is to use the tests that match the Kubernetes
version. However, for 1.13 we have to make an exception because the
suite for that version did not support the necessary
--storage.testdriver parameter.

While switching back and forth between release-1.13 and release-1.14
locally, there was the problem that the local kind image kept using
the wrong kubelet binary despite rebuilding from source. The problem
went away after cleaning the Bazel cache. Exact root cause unknown,
but perhaps using unique tags and properly cleaning the repo helps.

If not, then the unique tags at least will show what the problem is.

Using the same (recent) Go version for all Kubernetes versions can
break for older versions when there are incompatible changes in Go. To
avoid that, we use exactly the minimum version of Go required for each
Kubernetes version. This is based on the assumption that this
combination was tested successfully.

When building the E2E suite from Kubernetes (the default) we do the
same, but still allow building it from elsewhere.

We allow the Go version to be empty when it doesn't matter.

The travis.yml is now the only place where the Go version for the
component itself needs to be configured.

verify shellcheck

This enables testing of other repos and of this repo itself inside
Prow. Currently supported is unit testing ("make test") and E2E
testing (either via a local test suite or the Kubernetes E2E test
suite applied to the hostpath driver example deployment).

The script passes shellcheck and uses Prow to verify that for future
PRs.

By default this only tests the scripts inside the "release-tools"
directory, which is useful when making experimental changes to them in
a component that uses csi-release-tools. But a component can also
enable checking for other directories.

These are the modifications that were necessary to call this outside
of Kubernetes. The support for excluding files from checking gets
removed to simplify the script. It shouldn't be needed, because
linting can be enabled after fixing whatever scripts might fail the
check.

build.make: avoid unit-testing E2E test suite

In repos that have a test/e2e, that test suite should be run
separately because it depends on a running cluster.

check vendor directory

This runs "dep check" to verify that the vendor directory is
up-to-date and meets expectations (= done with dep >= 0.5.0).

This is an unmodified copy of kubernetes/hack/verify-shellcheck.sh
revision d5a3db003916b1d33b503ccd2e4897e094d8af77.

In repos that have a test/e2e, that test suite should be run
separately because it depends on a running cluster.

verify-subtree.sh: relax check and ignore old content

If for whatever reasons a repo already had a `release-tools` directory
before doing a clean import of it with `git subtree`, the check used
to fail because it found those old commits.

This can be fixed by telling `git log` to stop when the directory
disappears from the repo. There has to be a commit with removes the
old content, because otherwise `git subtree add` doesn't work.

Fixes: https://github.com/kubernetes-csi/external-resizer/issues/21

test enhancements

build.make: fix pushing of "canary" image from master branch

The introduction for each individual test looked like an actual
command:

  test-subtree
  ./release-tools/verify-subtree.sh release-tools
  Directory 'release-tools' contains non-upstream changes:
  ...

It's better to make it look like a shell comment and increase its
visibility with a longer prefix:

  ### test-subtree:
  ./release-tools/verify-subtree.sh release-tools
  ...

After merging into external-attacher, the next Travis CI run did not
push the "canary" image because the check for "canary" only covered
the case where "-canary" is used as
suffix (https://travis-ci.org/kubernetes-csi/external-attacher/builds/484095261).

Individual repos may have to filter out certain packages from
testing. For example, in csi-test the cmd/csi-sanity directory
contains a special test that depends on additional parameters that set
the CSI driver to test against.

This may or may not work, depending on which packages have tests and
whether they contain glog.

check subtree for changes

We don't want to allow local modifications in the subtree. Everything
should go to the csi-release-tools repo first.

"make test" used to abort after the first test failure. That was
partly intentional: if the simple tests already fail (for example,
because of a syntax error), then there is no point in continuing to
test.

However, it also makes it harder to find all errors in a CI system
when the errors are unrelated (first error shows up, gets fixed, next
error shows up, etc.).

Now "make test" still aborts early, but "make -k test" is used in the
CI and will run all individual tests because they are split up into
different targets.

The goal is to enforce that changes get merged upstream first and only
get into the local repo via a normal "git subtree merge".