<div id="readability-page-1" class="page"><p><cite>seungkyua@gmail.com</cite>
                                    <span> 2023. 6. 12. 16:38</span>
                                </p><div>
                                <p data-ke-size="size16">Kubebuilder 의 아키텍처에 대해서 살펴보고 Kubebuilder 로 프로젝트를 생성하는 방법을 알아본다.</p>
<h2>Kubebuilder Architeture</h2>
<figure data-ke-mobilestyle="widthOrigin" data-origin-width="2644" data-origin-height="2374">
    <span data-url="https://blog.kakaocdn.net/dn/nrTgI/btsjFSQdaw2/IRH2hPSDbJQJHiw1qYOObK/img.png" data-lightbox="lightbox">
        <img src="https://blog.kakaocdn.net/dn/nrTgI/btsjFSQdaw2/IRH2hPSDbJQJHiw1qYOObK/img.png" onerror="this.onerror=null; this.src='//t1.daumcdn.net/tistory_admin/static/images/no-image-v1.png'; this.srcset='//t1.daumcdn.net/tistory_admin/static/images/no-image-v1.png';" data-origin-width="2644" data-origin-height="2374">
    </span>
    
</figure>
<p data-ke-size="size16">[출처: <a href="https://book.kubebuilder.io/architecture.html">https://book.kubebuilder.io/architecture.html</a>]</p>
<p data-ke-size="size16"><br>위의 다이어그램에서 Kubebuilder 는 controller-runtime 모듈을 사용하는 것을 알 수 있다. 또한 사용자의 비즈니스 로직은 Reconciler 에 위치 시킨다는 것을 알 수 있다.</p>

<h2>Kubebuilder 로 프로젝트 생성</h2>
<p data-ke-size="size16">Kubebuilder 를 사용하기 위해서 사전 준비 작업이 필요하다.</p>
<h4 data-ke-size="size20"><br>사전 준비 작업</h4>
<ul data-ke-list-type="disc">
<li><a href="https://golang.org/dl/">go</a> version v1.19.0+</li>
<li><a href="https://docs.docker.com/install/">docker</a> version 17.03+.</li>
<li><a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">kubectl</a> version v1.11.3+.</li>
<li>Access to a Kubernetes v1.11.3+ cluster.</li>
</ul>

<h3 data-ke-size="size23">Kubebuilder 설치</h3>
<p data-ke-size="size16">kubebuilder 는 간단히 다운 받아서 설치할 수 있다. <code>~/bin/</code> 디렉토리가 <code>path</code> 로 잡혀있기 때문에 다운 받은 바이너리 파일을 이 곳으로 이동시켰다.</p>
<pre><code>$ cd ~/Documents/tmp

$ curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)
$ chmod +x kubebuilder
$ mv ~/Documents/tmp/kubebuilder ~/bin/kubebuilder

$ kubebuilder version
--- output ---
Version: main.version{KubeBuilderVersion:"3.10.0", KubernetesVendor:"1.26.1", GitCommit:"0fa57405d4a892efceec3c5a902f634277e30732", BuildDate:"2023-04-15T08:10:35Z", GoOs:"darwin", GoArch:"amd64"}</code></pre>

<h3 data-ke-size="size23">kustomize 설치</h3>
<pre><code>$ cd ~/Documents/tmp

$ curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash
$ mv ~/Documents/tmp/kustomize ~/bin/kustomize

$ kustomize version
--- output ---
v5.0.3</code></pre>

<h3 data-ke-size="size23">kind 설치 및 cluster 생성</h3>
<pre><code>$ cd ~/Documents/tmp

$ [ $(uname -m) = x86_64 ]&amp;&amp; curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.19.0/kind-darwin-amd64
$ chmod +x kind
$ mv ~/Documents/tmp/kind ~/bin/kind

$ kind version
--- output ---
kind v0.19.0 go1.20.4 darwin/amd64

$ kind create cluster</code></pre>

<h3 data-ke-size="size23">kubectl 설치</h3>
<pre><code>$ cd ~/Documents/tmp

$ curl -LO "https://dl.k8s.io/release/v1.27.1/bin/darwin/amd64/kubectl"
$ chmod +x kubectl
$ mv kubectl ~/bin/kubectl

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-04-14T13:21:19Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-05-12T19:03:40Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/amd64"}</code></pre>

<h3 data-ke-size="size23"><br>프로젝트 생성</h3>
<p data-ke-size="size16">kubebuilder 명령어로 간단히 프로젝트와 API 를 생성할 수 있다. 즉, 필요한 코드들이 자동으로 생성된다.</p>
<p data-ke-size="size16">먼저 프로젝트를 생성한다.</p>
<pre><code>$ mkdir -p guestbook-kubebuilder
$ cd guestbook-kubebuilder

$ kubebuilder init --domain my.domain --repo my.domain/guestbook
--- output ---
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.14.4
go: downloading sigs.k8s.io/controller-runtime v0.14.4
go: downloading k8s.io/apimachinery v0.26.1
go: downloading github.com/prometheus/client_golang v1.14.0
go: downloading k8s.io/client-go v0.26.1
go: downloading k8s.io/utils v0.0.0-20221128185143-99ec85e7a448
go: downloading github.com/prometheus/client_model v0.3.0
go: downloading k8s.io/api v0.26.1
go: downloading k8s.io/component-base v0.26.1
go: downloading golang.org/x/time v0.3.0
go: downloading k8s.io/apiextensions-apiserver v0.26.1
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2
go: downloading golang.org/x/net v0.3.1-0.20221206200815-1e63c2f08a10
go: downloading github.com/imdario/mergo v0.3.6
go: downloading k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280
go: downloading golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b
Update dependencies:
$ go mod tidy
go: downloading go.uber.org/goleak v1.2.0
Next: define a resource with:
$ kubebuilder create api</code></pre>

<p data-ke-size="size16">다음으로 api 를 생성한다.</p>
<pre><code>$ kubebuilder create api --group webapp --version v1 --kind Guestbook
--- output ---
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1/guestbook_types.go
api/v1/groupversion_info.go
internal/controller/suite_test.go
internal/controller/guestbook_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen &amp;&amp; /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
    GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
go: downloading sigs.k8s.io/controller-tools v0.11.3
go: downloading golang.org/x/tools v0.4.0
go: downloading k8s.io/utils v0.0.0-20221107191617-1a15be271d1d
go: downloading github.com/mattn/go-colorable v0.1.9
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests</code></pre>
<p data-ke-size="size16">CR 이나 CRD 를 수정하면 마지막의 <code>make manifests</code> 를 수행하여 다신 generation 해야 한다고 친절히 알려주고 있다.</p>
<p data-ke-size="size16">CR 과 CRD 는 아래 <code>guestbook_types.go</code> 파일에 struct 로 생성되어 있다. 이곳을 원하는 대로 변경하면 된다.</p>

<p data-ke-size="size16"><br>테스트로 아래과 같이 변경하자.</p>
<pre><code>// GuestbookSpec defines the desired state of Guestbook
type GuestbookSpec struct {
    // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
    // Important: Run "make" to regenerate code after modifying this file

    // Quantity of instances
    // +kubebuilder:validation:Minimum=1
    // +kubebuilder:validation:Maximum=10
    Size int32 `json:"size"`

    // Name of the ConfigMap for GuestbookSpec's configuration
    // +kubebuilder:validation:MaxLength=15
    // +kubebuilder:validation:MinLength=1
    ConfigMapName string `json:"configMapName"`

    // +kubebuilder:validation:Enum=Phone;Address;Name
    Type string `json:"alias,omitempty"`
}

// GuestbookStatus defines the observed state of Guestbook
type GuestbookStatus struct {
    // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
    // Important: Run "make" to regenerate code after modifying this file

    // PodName of the active Guestbook node.
    Active string `json:"active"`

    // PodNames of the standby Guestbook nodes.
    Standby []string `json:"standby"`
}

//+kubebuilder:object:root=true
//+kubebuilder:subresource:status

// Guestbook is the Schema for the guestbooks API
type Guestbook struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   GuestbookSpec   `json:"spec,omitempty"`
    Status GuestbookStatus `json:"status,omitempty"`
}

//+kubebuilder:object:root=true

// GuestbookList contains a list of Guestbook
type GuestbookList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []Guestbook `json:"items"`
}</code></pre>
<p data-ke-size="size16"><code>Guestbook</code> struct 에 있는 <code>metav1.TypeMeta</code> 와 <code>metav1.ObjectMeta</code> 를 설명하면, 전자는 우리가 흔히 보는 yaml 파일에서 <code>apiVersion</code> 과 <code>Kind</code> 이고 후자는 <code>metadata</code> 의 <code>name</code>, <code>namespace</code> 등을 나타낸다. 다음에 우리가 정의한 <code>Spec</code> 과 <code>Status</code> 가 있음을 알 수 있다.</p>
<h3 data-ke-size="size23"><p>테스트 방법 1 - Cluster 밖에서 테스트 하기</p></h3>
<p data-ke-size="size16">CRD 를 cluster 에 설치한다.</p>
<pre><code>$ make install
--- output ---
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" --output install_kustomize.sh &amp;&amp; bash install_kustomize.sh 5.0.0 /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin; rm install_kustomize.sh; }
v5.0.0
kustomize installed to /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.my.domain created</code></pre>

<p data-ke-size="size16">controller 를 실행시킨다. (터미널에서 포그라운드로 실행한다)</p>
<pre><code>$ make run
--- output ---
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen &amp;&amp; /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
        GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...

go run ./cmd/main.go
2023-05-24T17:18:18+09:00       INFO    controller-runtime.metrics      Metrics server is starting to listen    {"addr": ":8080"}
2023-05-24T17:18:18+09:00       INFO    setup   starting manager
2023-05-24T17:18:18+09:00       INFO    Starting server {"kind": "health probe", "addr": "[::]:8081"}
2023-05-24T17:18:18+09:00       INFO    Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
2023-05-24T17:18:18+09:00       INFO    Starting EventSource    {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "source": "kind source: *v1.Guestbook"}
2023-05-24T17:18:18+09:00       INFO    Starting Controller     {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook"}
2023-05-24T17:18:18+09:00       INFO    Starting workers        {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "worker count": 1}</code></pre>
<p data-ke-size="size16">참고로 앞서 api 를 생성할 때 <code>Create Resource [y/n] y</code> 로 했다면 CR 이 <code>config/samples</code> 디렉토리 아래에 생성되어 있다.</p>
<p data-ke-size="size16"><br>여기에 Spec 부분만 추가한다.</p>
<pre><code>$ tree config/samples
config/samples
├── kustomization.yaml
└── webapp_v1_guestbook.yaml

$ vi config/samples/webapp_v1_guestbook.yaml
--- output ---
apiVersion: webapp.my.domain/v1
kind: Guestbook
metadata:
  labels:
    app.kubernetes.io/name: guestbook
    app.kubernetes.io/instance: guestbook-sample
    app.kubernetes.io/part-of: guestbook-kubebuilder
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: guestbook-kubebuilder
  name: guestbook-sample
spec:
  # TODO(user): Add fields here
  size: 1
  configMapName: "myconfig"
  alias: "Address"</code></pre>

<p data-ke-size="size16">터미널을 새로 열어서 이를 설치한다.</p>
<pre><code>$ kubectl apply -k config/samples/
--- output ---
guestbook.webapp.my.domain/guestbook-sample created

$ kubectl get guestbook
--- output ---           
NAME               AGE
guestbook-sample   29s</code></pre>
<h3 data-ke-size="size23"><br>테스트 방법 2 - Cluster 안에서 돌리기</h3>
<p data-ke-size="size16">controller 를 cluster 안에서 돌리기 위해서는 먼저 이미지를 만들어야 한다.</p>
<pre><code>$ docker login -u seungkyua
--- output ---         
Password: 
Login Succeeded

$ make docker-build docker-push IMG=docker.io/seungkyua/guestbook-kubebuilder:1.0</code></pre>

<p data-ke-size="size16">다음은 image 를 가지고 deploy 한다.</p>
<pre><code>$ make deploy IMG=docker.io/seungkyua/guestbook-kubebuilder:1.0
--- output ---
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen &amp;&amp; /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
        GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" --output install_kustomize.sh &amp;&amp; bash install_kustomize.sh 5.0.0 /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin; rm install_kustomize.sh; }
cd config/manager &amp;&amp; /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize edit set image controller=docker.io/seungkyua/guestbook-kubebuilder:1.0
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize build config/default | kubectl apply -f -
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
namespace/guestbook-kubebuilder-system created
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.my.domain configured
serviceaccount/guestbook-kubebuilder-controller-manager created
role.rbac.authorization.k8s.io/guestbook-kubebuilder-leader-election-role created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-manager-role created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-metrics-reader created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-proxy-role created
rolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-proxy-rolebinding created
service/guestbook-kubebuilder-controller-manager-metrics-service created
deployment.apps/guestbook-kubebuilder-controller-manager created</code></pre>

<p data-ke-size="size16">확인하면 다음과 같이 pod 가 설치된 것을 알 수 있다.</p>
<pre><code>$ kubectl get pods -n guestbook-kubebuilder-system 
--- output ---
NAME                                                        READY   STATUS    RESTARTS   AGE
guestbook-kubebuilder-controller-manager-5f74f9d765-r68gn   2/2     Running   0          2m55s</code></pre>

<h3 data-ke-size="size23"><br>삭제하기</h3>
<h3 data-ke-size="size23">crd 삭제</h3>
<pre><code>$ make uninstall</code></pre>

<h3 data-ke-size="size23">Cluster 에 설치된 controller 삭제</h3>
<pre><code>$ make undeploy</code></pre>
            
            
                </div></div>

안승규(Ahn Seungkyu)의 블로그 (Stay hungry, stay foolish)

Kubebuilder 의 아키텍처에 대해서 살펴보고 Kubebuilder 로 프로젝트를 생성하는 방법을 알아본다.
Kubebuilder Architeture
[출처: https://book.kubebuilder.io/architecture.html]
위의 다이어그램에서 Kubebuilder 는 controller-runtime 모듈을 사용하는 것을 알 수 있다. 또한 사용자의 비즈니스 로직은 Reconciler 에 위치 시킨다는 것을 알 수 있다.
 
Kubebuilder 로 프로젝트 생성
Kubebuilder 를 사용하기 위해서 사전 준비 작업이 필요하다.
사전 준비 작업
go version v1.19.0+
docker version 17.03+.
kubectl version v1.11.3+.
Access to a Kubernetes v1.11.3+ cluster.
 
Kubebuilder 설치
kubebuilder 는 간단히 다운 받아서 설치할 수 있다. ~/bin/ 디렉토리가 path 로 잡혀있기 때문에 다운 받은 바이너리 파일을 이 곳으로 이동시켰다.
$ cd ~/Documents/tmp
$ curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)
$ chmod +x kubebuilder
$ mv ~/Documents/tmp/kubebuilder ~/bin/kubebuilder
$ kubebuilder version
--- output ---
Version: main.version{KubeBuilderVersion:"3.10.0", KubernetesVendor:"1.26.1", GitCommit:"0fa57405d4a892efceec3c5a902f634277e30732", BuildDate:"2023-04-15T08:10:35Z", GoOs:"darwin", GoArch:"amd64"}
 
kustomize 설치
$ cd ~/Documents/tmp
$ curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash
$ mv ~/Documents/tmp/kustomize ~/bin/kustomize
$ kustomize version
--- output ---
v5.0.3
 
kind 설치 및 cluster 생성
$ cd ~/Documents/tmp
$ [ $(uname -m) = x86_64 ]&& curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.19.0/kind-darwin-amd64
$ chmod +x kind
$ mv ~/Documents/tmp/kind ~/bin/kind
$ kind version
--- output ---
kind v0.19.0 go1.20.4 darwin/amd64
$ kind create cluster
 
kubectl 설치
$ cd ~/Documents/tmp
$ curl -LO "https://dl.k8s.io/release/v1.27.1/bin/darwin/amd64/kubectl"
$ chmod +x kubectl
$ mv kubectl ~/bin/kubectl
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-04-14T13:21:19Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-05-12T19:03:40Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/amd64"}
 
프로젝트 생성
kubebuilder 명령어로 간단히 프로젝트와 API 를 생성할 수 있다. 즉, 필요한 코드들이 자동으로 생성된다.
먼저 프로젝트를 생성한다.
$ mkdir -p guestbook-kubebuilder
$ cd guestbook-kubebuilder
$ kubebuilder init --domain my.domain --repo my.domain/guestbook
--- output ---
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.14.4
go: downloading sigs.k8s.io/controller-runtime v0.14.4
go: downloading k8s.io/apimachinery v0.26.1
go: downloading github.com/prometheus/client_golang v1.14.0
go: downloading k8s.io/client-go v0.26.1
go: downloading k8s.io/utils v0.0.0-20221128185143-99ec85e7a448
go: downloading github.com/prometheus/client_model v0.3.0
go: downloading k8s.io/api v0.26.1
go: downloading k8s.io/component-base v0.26.1
go: downloading golang.org/x/time v0.3.0
go: downloading k8s.io/apiextensions-apiserver v0.26.1
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2
go: downloading golang.org/x/net v0.3.1-0.20221206200815-1e63c2f08a10
go: downloading github.com/imdario/mergo v0.3.6
go: downloading k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280
go: downloading golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b
Update dependencies:
$ go mod tidy
go: downloading go.uber.org/goleak v1.2.0
Next: define a resource with:
$ kubebuilder create api
 
다음으로 api 를 생성한다.
$ kubebuilder create api --group webapp --version v1 --kind Guestbook
--- output ---
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1/guestbook_types.go
api/v1/groupversion_info.go
internal/controller/suite_test.go
internal/controller/guestbook_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
    GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
go: downloading sigs.k8s.io/controller-tools v0.11.3
go: downloading golang.org/x/tools v0.4.0
go: downloading k8s.io/utils v0.0.0-20221107191617-1a15be271d1d
go: downloading github.com/mattn/go-colorable v0.1.9
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests
CR 이나 CRD 를 수정하면 마지막의 make manifests 를 수행하여 다신 generation 해야 한다고 친절히 알려주고 있다.
CR 과 CRD 는 아래 guestbook_types.go 파일에 struct 로 생성되어 있다. 이곳을 원하는 대로 변경하면 된다.
 
테스트로 아래과 같이 변경하자.
// GuestbookSpec defines the desired state of Guestbook
type GuestbookSpec struct {
    // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
    // Important: Run "make" to regenerate code after modifying this file
    // Quantity of instances
    // +kubebuilder:validation:Minimum=1
    // +kubebuilder:validation:Maximum=10
    Size int32 `json:"size"`
    // Name of the ConfigMap for GuestbookSpec's configuration
    // +kubebuilder:validation:MaxLength=15
    // +kubebuilder:validation:MinLength=1
    ConfigMapName string `json:"configMapName"`
    // +kubebuilder:validation:Enum=Phone;Address;Name
    Type string `json:"alias,omitempty"`
}
// GuestbookStatus defines the observed state of Guestbook
type GuestbookStatus struct {
    // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
    // Important: Run "make" to regenerate code after modifying this file
    // PodName of the active Guestbook node.
    Active string `json:"active"`
    // PodNames of the standby Guestbook nodes.
    Standby []string `json:"standby"`
}
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
// Guestbook is the Schema for the guestbooks API
type Guestbook struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec   GuestbookSpec   `json:"spec,omitempty"`
    Status GuestbookStatus `json:"status,omitempty"`
}
//+kubebuilder:object:root=true
// GuestbookList contains a list of Guestbook
type GuestbookList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []Guestbook `json:"items"`
}
Guestbook struct 에 있는 metav1.TypeMeta 와 metav1.ObjectMeta 를 설명하면, 전자는 우리가 흔히 보는 yaml 파일에서 apiVersion 과 Kind 이고 후자는 metadata 의 name, namespace 등을 나타낸다. 다음에 우리가 정의한 Spec 과 Status 가 있음을 알 수 있다.
테스트 방법 1 - Cluster 밖에서 테스트 하기
CRD 를 cluster 에 설치한다.
$ make install
--- output ---
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" --output install_kustomize.sh && bash install_kustomize.sh 5.0.0 /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin; rm install_kustomize.sh; }
v5.0.0
kustomize installed to /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.my.domain created
 
controller 를 실행시킨다. (터미널에서 포그라운드로 실행한다)
$ make run
--- output ---
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
        GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./cmd/main.go
2023-05-24T17:18:18+09:00       INFO    controller-runtime.metrics      Metrics server is starting to listen    {"addr": ":8080"}
2023-05-24T17:18:18+09:00       INFO    setup   starting manager
2023-05-24T17:18:18+09:00       INFO    Starting server {"kind": "health probe", "addr": "[::]:8081"}
2023-05-24T17:18:18+09:00       INFO    Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
2023-05-24T17:18:18+09:00       INFO    Starting EventSource    {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "source": "kind source: *v1.Guestbook"}
2023-05-24T17:18:18+09:00       INFO    Starting Controller     {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook"}
2023-05-24T17:18:18+09:00       INFO    Starting workers        {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "worker count": 1}
참고로 앞서 api 를 생성할 때 Create Resource [y/n] y 로 했다면 CR 이 config/samples 디렉토리 아래에 생성되어 있다.
여기에 Spec 부분만 추가한다.
$ tree config/samples
config/samples
├── kustomization.yaml
└── webapp_v1_guestbook.yaml
$ vi config/samples/webapp_v1_guestbook.yaml
--- output ---
apiVersion: webapp.my.domain/v1
kind: Guestbook
metadata:
  labels:
    app.kubernetes.io/name: guestbook
    app.kubernetes.io/instance: guestbook-sample
    app.kubernetes.io/part-of: guestbook-kubebuilder
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: guestbook-kubebuilder
  name: guestbook-sample
spec:
  # TODO(user): Add fields here
  size: 1
  configMapName: "myconfig"
  alias: "Address"
 
터미널을 새로 열어서 이를 설치한다.
$ kubectl apply -k config/samples/
--- output ---
guestbook.webapp.my.domain/guestbook-sample created
$ kubectl get guestbook
--- output ---           
NAME               AGE
guestbook-sample   29s
테스트 방법 2 - Cluster 안에서 돌리기
controller 를 cluster 안에서 돌리기 위해서는 먼저 이미지를 만들어야 한다.
$ docker login -u seungkyua
--- output ---         
Password: 
Login Succeeded
$ make docker-build docker-push IMG=docker.io/seungkyua/guestbook-kubebuilder:1.0
 
다음은 image 를 가지고 deploy 한다.
$ make deploy IMG=docker.io/seungkyua/guestbook-kubebuilder:1.0
--- output ---
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
        GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" --output install_kustomize.sh && bash install_kustomize.sh 5.0.0 /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin; rm install_kustomize.sh; }
cd config/manager && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize edit set image controller=docker.io/seungkyua/guestbook-kubebuilder:1.0
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize build config/default | kubectl apply -f -
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
namespace/guestbook-kubebuilder-system created
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.my.domain configured
serviceaccount/guestbook-kubebuilder-controller-manager created
role.rbac.authorization.k8s.io/guestbook-kubebuilder-leader-election-role created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-manager-role created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-metrics-reader created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-proxy-role created
rolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-proxy-rolebinding created
service/guestbook-kubebuilder-controller-manager-metrics-service created
deployment.apps/guestbook-kubebuilder-controller-manager created
 
확인하면 다음과 같이 pod 가 설치된 것을 알 수 있다.
$ kubectl get pods -n guestbook-kubebuilder-system 
--- output ---
NAME                                                        READY   STATUS    RESTARTS   AGE
guestbook-kubebuilder-controller-manager-5f74f9d765-r68gn   2/2     Running   0          2m55s
 
삭제하기
crd 삭제
$ make uninstall
 
Cluster 에 설치된 controller 삭제
$ make undeploy

Custom Controller 2 - Kubebuilder Architecture

댓글