Running Bevel DLT network on Minikube

Pre-requisites

Before proceeding, first make sure that you’ve completed Developer Pre-requisites.

Clone forked repo

If you have followed the pre-requisites, you should already have the bevel repo cloned; if not, follow the pre-requisite steps above.

Add a “local” branch to your machine

cd ./project/bevel
git checkout develop # to get latest code
git pull
git checkout -b local
git push --set-upstream origin local

Update kubeconfig file

Create a build folder inside your Bevel repository:
```
mkdir build
```

Copy ca.crt, client.key, client.crt from ~/.minikube to build:

cp ~/.minikube/ca.crt build/
cp ~/.minikube/profiles/minikube/client.key build/
cp ~/.minikube/profiles/minikube/client.crt build/

Copy ~/.kube/config file to build:
```
cp ~/.kube/config build/
```
Open the config file and remove the paths for certificate-authority, client-certificate and client-key as in the figure below.

NOTE: If you ever delete and recreate minikube, the above steps have to be repeated.

Get your local IP Address for Vault. Follow this guide to get your machine’s local IP Address.

In a new terminal, execute the following (assuming vault is in your PATH):

export VAULT_ADDR='http://<Your Vault local IP address>:8200' #e.g. http://192.168.0.1:8200
export VAULT_TOKEN="<Your Vault root token>"

# enable Secrets v2
vault secrets enable -version=2 -path=secretsv2 kv   

Additional configurations

For Windows, ensure that you have set the following git config before cloning the repo.
```
git config --global core.autocrlf false
```
- If not, update the EOL to LF for platforms/hyperledger-fabric/scripts/*.sh files.
For Windows, and if using Execute following to correctly set docker environment.
```
eval $('docker-machine.exe' env)
```
For Windows, mount local folder (bevel folder) to VirtualBox docker VM ( the machine named “default” by default) from right-click menu, Settings -> Shared Folders. All paths in network.yaml should be the mounted path. Shut down and restart the “default” machine after this.

For Mac, mount the local folder using minikube mount as we are using minikube as the docker runtime.
```
eval $(minikube -p minikube docker-env)
cd ./project/bevel
minikube mount $(pwd):/mnt/bevel/ --gid=root --uid=root
```
This terminal window should be kept open. If you close this terminal you have to minikube stop and start to overcome the error when re-mounting.

Edit the configuration file

Choose the DLT/Blockchain platform you want to run and copy the relevant sample network.yaml to build folder; rename it to network.yaml.
```
cd ./project/bevel
cp platforms/hyperledger-fabric/configuration/samples/network-proxy-none.yaml build/network.yaml
```
Open the above build/network.yaml in your favourite editor and update the following.

Update Docker configurations:

docker:
  url: "ghcr.io/hyperledger"
  # Comment username and password as it is public repo
  #username: "<your docker username>"
  #password: "<your docker password/token>"

For each organization, update ONLY the following and leave everything else as is:

cloud_provider: minikube
k8s:
    context: "minikube"
    config_file: "/home/bevel/build/config"
vault:
  url: "http://<Your Vault local IP address>:8200" # Use the local IP address rather than localhost e.g. http://192.168.0.1:8200
  root_token: "<your vault_root_token>"
gitops:
  git_protocol: "https" # Option for git over https or ssh
  git_url: "https://github.com/<username>/bevel.git"  #e.g. "https://github.com/hyperledger/bevel.git"
  git_repo: "github.com/<username>/bevel.git"       #e.g. "github.com/hyperledger/bevel.git"
  username: "<github_username>"
  password: "<git_access_token>"
  email: "<github_email>"

If you need help, you can use each platform’s sample network-minikube.yaml:

For Fabric, use platforms/hyperledger-fabric/configuration/samples/network-proxy-none.yaml
For Besu, use platforms/hyperledger-besu/configuration/samples/network-proxy-none.yaml
For Indy, use platforms/hyperledger-indy/configuration/samples/network-minikube.yaml
For Quorum, use platforms/quorum/configuration/samples/network-minikube.yaml
For Corda, use platforms/r3-corda/configuration/samples/network-minikube.yaml

And simply replace the placeholder values as explained above.

NOTE: If you have 2-Factor Authentication enabled on your GitHub account, you have to use GitHub token. Otherwise, password is fine.

How To Generate GitHub Token

On GitHub page, click your profile icon and then click Settings.
On the sidebar, click Developer settings.
On the sidebar, click Personal access tokens.
Click Generate new token.
Add a token description, enable suitable access and click Generate token.
Copy the token to a secure location or password management app.

For security reasons, after you leave the page, you can no longer see the token again.

Deploying the sample “supplychain” chaincode for Fabric is optional, so you can delete the “chaincode” section. If deploying chaincode, update the following for the peers.
```
chaincodes:
- name: "supplychain"
    repository:
      username: "<github_username>"
      password: "<github_token>"
```

Execute

Make sure that Minikube and Vault server are running. Double-check by running:

minikube status
vault status

Now run the following to deploy your chosen DLT on minikube:

Mac users should use following (make sure that the local volume was mounted as per this step):

docker run -it -v /mnt/bevel:/home/bevel/ --network="host" ghcr.io/hyperledger/bevel-build:latest

Windows users should use following (make sure that the local volume was mounted as per this step):

docker run -it -v /bevel:/home/bevel/ --network="host" ghcr.io/hyperledger/bevel-build:latest

Meanwhile you can also check if pods are being deployed:

kubectl get pods --all-namespaces -w

NOTE: If you need public address for nodes in your network.yaml file, you can use the output of minikube ip.

NOTE. baf-build image is using jdk14 but Corda and Corda Enterprise requires jdk8. In this case, you can use the prebuild image tag jdk8 ghcr.io/hyperledger/bevel-build:jdk8-latest

Troubleshooting

Failed to establish a new connection: [Errno 111] Connection refused

This is because you have re-created minikube but have not updated K8s config file. Repeat “Update kubeconfig file” steps 3 - 4 and try again.

kubernetes.config.config_exception.ConfigException: File does not exists: /Users/.minikube/ca.crt

This is because you have not removed the absolute paths to the certificates in config file. Repeat “Update kubeconfig file” step 4 and try again.

error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/version: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running

This is because docker isn’t running. To start it, just close all the instances of Docker Quickstart Terminal and open again.

ERROR! the playbook: /home/bevel/platforms/shared/configuration/site.yaml could not be found

This is because the bevel repository isn’t mounted to the default VM. Check this step.