How to set up Kubeflow for ML on K8s

In this blog we want to present a quick tutorial on how to set up the latest version of Kubeflow, the open source package for managing Machine Learning on Kubernetes. Please follow these instructions on how to get Kubeflow 1.5 started:

Launch a cluster on EKS (we recommend 4 worker nodes with at least t3.2xlarge to give training enough juice).
We recommend using Spectro Cloud Palette for ease of management :) Here’s a quick link on how to launch an EKS cluster on our platform. But you may do this any way you like.
Install “kustomize” if not done already (version 3.2.0) (download link).
have “kubectl” handy (link to install instructions).

Setting it up using command lines:

Let’s download from the source.

Clone the manifest from the official repo and switch to v1.5 branch.

 
git clone https://github.com/kubeflow/manifests.git
cd manifests
git checkout v1.5-branch

2.Download kubeconfig from Palette and ensure the kubectl command knows where to find it e.g.

 
export KUBECONFIG=

If you need a quick way to context switch KUBECONFIG we recommend a good trick “kctx” mentioned on our blog post.

3.Now it is time to apply the required resource files — there are quite a few! Please be aware that this may take up to 20 minutes because of the number of resources required.

 
while ! kustomize build example | kubectl apply -f -; do echo “Retrying to apply resources”; sleep 10; done

When this is done let’s setup port forwarding:

 
export NAMESPACE=istio-system && kubectl port-forward -n ${NAMESPACE} svc/istio-ingressgateway 8080:80

And access Kubeflow dashboard as follows with default credentials:

URL: http://localhost:8080/
> username : user@example.com
> password: 12341234

Voila! You now have a kubeflow cluster at your disposal for your ML projects

Let’s try an example:

Notebook example

Text Classification with Movie Reviews

Using the same specifications above for the notebook server, we can run an example of NLP notebook as illustrated below

This notebook classifies movie reviews as positive or negative using the text of the review. This is an example of binary classification, an important and widely applicable kind of machine learning problem.

We’ll use the IMDB dataset that contains the text of 50,000 movie reviews from the Internet Movie Database. These are split into 25,000 reviews for training and 25,000 reviews for testing.