<div id="readability-page-1" class="page"><article><header><small>Created: 2020, 08 25 &gt;<a href="https://github.com/shdkej/shdkej.github.io/blame/master/content/Tool.md">Updated: 2021, 06 11</a></small></header><nav><strong>Table of Contents</strong><div><ul>
<li>
<p><a href="/tool/#prometheus">prometheus</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#elk-vs-tick">ELK vs TICK</a></li>
<li><a href="/tool/#grafana-alert">grafana alert</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="/tool/#tick">TICK</a></li>
<li>
<p><a href="/tool/#benchmark">Benchmark</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#nginx-alternative">nginx alternative</a></li>
</ul>
</li>
</ul>
</li>
<li>
<p><a href="/tool/#ansible">ansible</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#ansible-command-vs-shell">ansible command vs shell</a></li>
<li><a href="/tool/#ansible-1">ansible</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="/tool/#nginx-letsencrypt">Nginx letsencrypt</a></li>
<li>
<p><a href="/tool/#terraform-vs-serverless">terraform vs serverless</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#%ED%85%8C%EB%9D%BC%ED%8F%BC%EB%A7%8C-%EC%93%B8%EC%A7%80-%EC%84%9C%EB%B2%84%EB%A6%AC%EC%8A%A4%EB%A5%BC-%EA%B0%99%EC%9D%B4-%EC%93%B8%EC%A7%80-%EA%B3%A0%EB%AF%BC%EC%A4%91">테라폼만 쓸지 서버리스를 같이 쓸지 고민중</a></li>
</ul>
</li>
</ul>
</li>
<li>
<p><a href="/tool/#terraform">terraform</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#terraform-ansible">terraform ansible</a></li>
<li><a href="/tool/#use-module-for-different-folders-can-use-once">use module. For different folders can use once.</a></li>
<li><a href="/tool/#github-action-terraform">github action terraform</a></li>
<li><a href="/tool/#-change-backend-bucket">! change backend bucket</a></li>
<li><a href="/tool/#aws-dynamodb-terraform">AWS Dynamodb terraform</a></li>
</ul>
</li>
</ul>
</li>
<li>
<p><a href="/tool/#serverless">serverless</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#serverless-1">serverless</a></li>
</ul>
</li>
</ul>
</li>
<li>
<p><a href="/tool/#vault">vault</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#vault-in-gcp">vault in gcp</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="/tool/#google-calendar-api">google calendar api</a></li>
<li>
<p><a href="/tool/#nagios">nagios</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#for-docker-monitoring">for docker monitoring</a></li>
<li><a href="/tool/#nagios-%EB%B9%84%EB%B0%80%EB%B2%88%ED%98%B8-%EB%B3%80%EA%B2%BD">nagios 비밀번호 변경</a></li>
</ul>
</li>
</ul>
</li>
<li>
<p><a href="/tool/#aws-lambda-cronjob">aws lambda cronjob</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#serverless-aws-sqs-lambda">serverless aws sqs lambda</a></li>
</ul>
</li>
</ul>
</li>
<li>
<p><a href="/tool/#devdash">devdash</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#cloud">cloud</a></li>
<li><a href="/tool/#current-used-infra">Current Used infra</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="/tool/#%EC%B9%B4%ED%94%84%EC%B9%B4%EC%99%80-%EB%8B%A4%EB%A5%B8-%EB%A9%94%EC%8B%9C%EC%A7%80%ED%81%90">카프카와 다른 메시지큐</a></li>
<li>
<p><a href="/tool/#grpc">gRPC</a></p>
<ul>
<li>
<ul>
<li><a href="/tool/#grpc-1">grpc</a></li>
<li><a href="/tool/#grpc-2">grpc</a></li>
<li><a href="/tool/#grpc-3">grpc</a></li>
<li><a href="/tool/#reference">reference</a></li>
</ul>
</li>
</ul>
</li>
</ul></div></nav><p>---</p><section><h2 id="prometheus"><a href="#prometheus" aria-label="prometheus permalink"></a>prometheus</h2>
<ul>
<li>
<p>node-exporter</p>
<ul>
<li>collect system metrics</li>
</ul>
</li>
<li>alert-manager</li>
</ul>
<h4 id="elk-vs-tick"><a href="#elk-vs-tick" aria-label="elk vs tick permalink"></a>ELK vs TICK</h4>
<ul>
<li>ELK -- log metrics</li>
<li>TICK -- system metrics</li>
<li>what is prometheus, loki,</li>
</ul>
<h4 id="grafana-alert"><a href="#grafana-alert" aria-label="grafana alert permalink"></a>grafana alert</h4>
<p>monitoring 서버도 테스트 서버용과 프로덕션 서버용을 따로 두나?</p>
<p>alert를 만드려면 graph여야하고, $variable 로 되있는 템플릿을 쓸 수 없다.
alert를 만들고 싶은 graph를 복사해서 variable을 고치고 사용하면 된다.</p>
<p>sensu는 어떤 기능들을 제공해주고 있지?</p>
<p>grafana daily report</p>
<ul>
<li>enterprise 기능이었다. 오픈소스로 구현된 것도 있을 것이다</li>
</ul>
<h2 id="tick"><a href="#tick" aria-label="tick permalink"></a>TICK</h2>
<ul>
<li>too heavy</li>
<li>I want to find lightweight visualistic monitoring service</li>
<li>go to prometheus</li>
</ul>
<h2 id="benchmark"><a href="#benchmark" aria-label="benchmark permalink"></a>Benchmark</h2>
<ol>
<li>구글개발자도구 audits</li>
</ol>
</section><section>
<h4 id="nginx-alternative"><a href="#nginx-alternative" aria-label="nginx alternative permalink"></a>nginx alternative</h4>
<ul>
<li><a href="https://openlitespeed.org/">openlitespeed</a></li>
</ul>
<h2 id="ansible"><a href="#ansible" aria-label="ansible permalink"></a>ansible</h2>
<ul>
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-ubuntu-18-04">https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-ubuntu-18-04</a></li>
<li>setting /etc/ansible/hosts</li>
<li>terraform provision execute only during create</li>
</ul>
<h4 id="ansible-command-vs-shell"><a href="#ansible-command-vs-shell" aria-label="ansible command vs shell permalink"></a>ansible command vs shell</h4>
<p>command isn't running in $HOME, shell is
command can't use operations like &lt;,&gt;,|,&amp;
command is more secure.
<a href="https://blog.confirm.ch/ansible-modules-shell-vs-command/">https://blog.confirm.ch/ansible-modules-shell-vs-command/</a></p>
<h4 id="ansible-1"><a href="#ansible-1" aria-label="ansible 1 permalink"></a>ansible</h4>
<ul>
<li>ansible-galaxy</li>
<li>
<p>setting <code>/etc/ansible/hosts</code> file or make hosts file with <code>-i</code></p>
<ul>
<li>
<p>it need <code>,</code></p>
<pre><code>[webserver]
server_ip
# &lt;work_directory&gt;/hosts
</code></pre>
</li>
</ul>
</li>
<li>hosts test <code>ansible webserver -m ping -i hosts --private-key &lt;private-key&gt; -u &lt;user&gt;</code></li>
<li><a href="https://alex.dzyoba.com/blog/terraform-ansible/">https://alex.dzyoba.com/blog/terraform-ansible/</a></li>
<li>how to connect with pub file</li>
</ul>
<h2 id="nginx-letsencrypt"><a href="#nginx-letsencrypt" aria-label="nginx letsencrypt permalink"></a>Nginx letsencrypt</h2>
<ul>
<li>DNS setting</li>
<li>install letsencrypt</li>
<li><code>letsencrypt certonly --standalone</code></li>
<li>setting Nginx</li>
<li>run Nginx</li>
</ul>
<h2 id="terraform-vs-serverless"><a href="#terraform-vs-serverless" aria-label="terraform vs serverless permalink"></a>terraform vs serverless</h2>
<ul>
<li>note-reminder has terraform trigger option.</li>
<li>it's not good..</li>
</ul>
<h4 id="테라폼만-쓸지-서버리스를-같이-쓸지-고민중"><a href="#%ED%85%8C%EB%9D%BC%ED%8F%BC%EB%A7%8C-%EC%93%B8%EC%A7%80-%EC%84%9C%EB%B2%84%EB%A6%AC%EC%8A%A4%EB%A5%BC-%EA%B0%99%EC%9D%B4-%EC%93%B8%EC%A7%80-%EA%B3%A0%EB%AF%BC%EC%A4%91" aria-label="테라폼만 쓸지 서버리스를 같이 쓸지 고민중 permalink"></a>테라폼만 쓸지 서버리스를 같이 쓸지 고민중</h4>
<p>서버리스는 개별적인 앱을 빠르게 빌드하고 다시 만들 때 가볍게 사용하기 좋고
테라폼은 좀 더 넓게 공유되는 자원을 관리할 때 쓰기 좋다</p>
<p>서버리스는 앱을 빠르게 띄우기 좋고 테라폼은 인프라 셋팅하기에 좋다
<a href="https://www.serverless.com/blog/definitive-guide-terraform-serverless/">https://www.serverless.com/blog/definitive-guide-terraform-serverless/</a></p>
<h2 id="terraform"><a href="#terraform" aria-label="terraform permalink"></a>terraform</h2>
<ul>
<li>Need update when changing a provisioner</li>
<li>
<p>! resource "null_resource" -&gt; null</p>
<ul>
<li>Do <code>terraform init</code></li>
</ul>
</li>
<li>
<p>terraform taint aws_instance.example-server</p>
<ul>
<li>aws_instance recreate when terraform apply</li>
<li>terraform null_resource is better then an instance make to a taint</li>
<li><a href="https://github.com/gruntwork-io/terratest/blob/master/examples/terraform-remote-exec-example/main.tf">https://github.com/gruntwork-io/terratest/blob/master/examples/terraform-remote-exec-example/main.tf</a></li>
<li>null resource need <code>terraform init</code></li>
</ul>
</li>
<li>recreate instance with same eip</li>
<li>
<p>!WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!</p>
<ul>
<li>clear <code>~/.ssh/known_hosts</code> or recreate ssh key <code>ssh-keygen -R &lt;host IP&gt;</code></li>
<li><a href="https://www.ssh.com/ssh/keygen/">https://www.ssh.com/ssh/keygen/</a></li>
</ul>
</li>
<li>
<p>! aws_instance remote-exec ssh connection not working</p>
<ul>
<li>user = "root" -&gt; user = "ubuntu"</li>
</ul>
</li>
<li>
<p><code>letsencrypt -d &lt;domain&gt; -m &lt;email&gt; -n(all agree) --agree-tos</code></p>
<ul>
<li><a href="https://github.com/ployst/docker-letsencrypt/issues/18">https://github.com/ployst/docker-letsencrypt/issues/18</a></li>
</ul>
</li>
<li>gcp metadata need ${}</li>
<li><code>ssh-keys = "username:${file("&lt;PATH&gt;")}"</code></li>
<li>Event Handling with sns, sqs</li>
<li><a href="https://dev.to/frosnerd/event-handling-in-aws-using-sns-sqs-and-lambda-2ng">https://dev.to/frosnerd/event-handling-in-aws-using-sns-sqs-and-lambda-2ng</a></li>
<li>lambda function to python</li>
<li>save s3</li>
<li>
<p>! s3 access-denied problem</p>
<ul>
<li>bucket name not allow var. I have just input text</li>
<li>var allow. but name is global. it is really name exist problem.</li>
</ul>
</li>
<li>
<p>! s3<em>bucket</em>notification invalid argument</p>
<ul>
<li>
<ol>
<li>create SNS 2.create S3 bucket 3. Policy 4. notification</li>
</ol>
</li>
<li>sns - aws<em>iam</em>policy - condition - values - (bucket arn -&gt; bucket name...)</li>
</ul>
</li>
<li>every apply update s3, using <code>etag</code></li>
</ul>
<h4 id="terraform-ansible"><a href="#terraform-ansible" aria-label="terraform ansible permalink"></a>terraform ansible</h4>
<ul>
<li>for provision <code>sleep 120;</code> is good to waiting ec2 instance creation</li>
<li>ansible-playbook to make with ip, ip + <code>,</code></li>
<li>terraform has <code>depends_on</code></li>
</ul>
<h4 id="use-module-for-different-folders-can-use-once"><a href="#use-module-for-different-folders-can-use-once" aria-label="use module for different folders can use once permalink"></a>use module. For different folders can use once.</h4>
<h4 id="github-action-terraform"><a href="#github-action-terraform" aria-label="github action terraform permalink"></a>github action terraform</h4>
<ul>
<li>
<p>how to hide secret file</p>
<ul>
<li>gcp credential file</li>
</ul>
</li>
</ul>
<h4 id="-change-backend-bucket"><a href="#-change-backend-bucket" aria-label=" change backend bucket permalink"></a>! change backend bucket</h4>
<p>need delete <code>.terraform</code> dir, and <code>terraform init</code></p>
<h4 id="aws-dynamodb-terraform"><a href="#aws-dynamodb-terraform" aria-label="aws dynamodb terraform permalink"></a>AWS Dynamodb terraform</h4>
<ul>
<li>attribute need index</li>
<li>any key can write, if exist with attribute</li>
</ul>
<h2 id="serverless"><a href="#serverless" aria-label="serverless permalink"></a>serverless</h2>
<ul>
<li>install <code>curl -o- -L https://slss.io/install | bash</code></li>
<li>
<p>!Error: spawn /home/sh/.serverless/bin/xdg-open ENOENT</p>
<ul>
<li>no install xdg-open. manual install and copy to serverless/bin directory</li>
</ul>
</li>
<li>
<p>!"service" property is missing in serverless.yml</p>
<ul>
<li>get started is sucks</li>
<li>run <code>serverless</code> for first setting</li>
</ul>
</li>
</ul>
<h4 id="serverless-1"><a href="#serverless-1" aria-label="serverless 1 permalink"></a>serverless</h4>
<ul>
<li>
<p>python requirements</p>
<ul>
<li>need install plugin serverless-python-requirements</li>
</ul>
</li>
</ul>
<h2 id="vault"><a href="#vault" aria-label="vault permalink"></a>vault</h2>
<ul>
<li>install file</li>
<li>move bin directory</li>
</ul>
<h4 id="vault-in-gcp"><a href="#vault-in-gcp" aria-label="vault in gcp permalink"></a>vault in gcp</h4>
<ol>
<li>
<p>run docker <code></code></p>
<ul>
<li>what is different with server-mode and another</li>
</ul>
</li>
<li>add ssh</li>
<li>save file</li>
<li>read file</li>
</ol>
<h2 id="google-calendar-api"><a href="#google-calendar-api" aria-label="google calendar api permalink"></a>google calendar api</h2>
<ol>
<li>credential.json 생성</li>
<li><code>pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib</code></li>
<li>코드 실행</li>
<li>인증</li>
<li>token.pickle 생성되고 코드 실행됨</li>
<li><a href="https://developers.google.com/calendar/quickstart/python">https://developers.google.com/calendar/quickstart/python</a></li>
</ol>
<h2 id="nagios"><a href="#nagios" aria-label="nagios permalink"></a>nagios</h2>
<p>Docker로 설치 후
<code>/opt/nagios/etc/</code> 설정파일 설정</p>
<ul>
<li><code>/etc/resource</code> : 환경변수 설정파일</li>
<li>object/Commend</li>
<li>object/contacts</li>
<li>object/template</li>
<li>object/windows</li>
<li>nagios</li>
</ul>
<p>window 는 nsclient 설치</p>
<ul>
<li>서버에서 command.cfg 파일에서 check_nt 부분 비밀번호 명시</li>
<li>Telegram 연동 소스 받아서 설정</li>
</ul>
<h4 id="for-docker-monitoring"><a href="#for-docker-monitoring" aria-label="for docker monitoring permalink"></a>for docker monitoring</h4>
<p><code>chown nagios /var/run/docker.sock</code></p>
<h4 id="nagios-비밀번호-변경"><a href="#nagios-%EB%B9%84%EB%B0%80%EB%B2%88%ED%98%B8-%EB%B3%80%EA%B2%BD" aria-label="nagios 비밀번호 변경 permalink"></a>nagios 비밀번호 변경</h4>
<ul>
<li><code>htpasswd -c /opt/nagios/etc/htpasswd.users nagiosadmin</code></li>
<li>콘솔로 비밀번호 입력</li>
</ul>
<h2 id="aws-lambda-cronjob"><a href="#aws-lambda-cronjob" aria-label="aws lambda cronjob permalink"></a>aws lambda cronjob</h2>
<ul>
<li>using cloudwatch event rule, event target, lambda permission</li>
<li>more option cloudwatch log group, cloudwatch log subscription filter</li>
<li><a href="https://www.thedevcoach.co.uk/terraform-lambda-scheduled-event/">https://www.thedevcoach.co.uk/terraform-lambda-scheduled-event/</a></li>
</ul>
<h4 id="serverless-aws-sqs-lambda"><a href="#serverless-aws-sqs-lambda" aria-label="serverless aws sqs lambda permalink"></a>serverless aws sqs lambda</h4>
<ul>
<li>sqs to lambda message parsing</li>
<li>event['Records'][0]['body']</li>
<li>lambda python requests</li>
<li>cannot version 3.8, can 3.6</li>
</ul>
<h2 id="devdash"><a href="#devdash" aria-label="devdash permalink"></a>devdash</h2>
<ul>
<li>google analytic settings</li>
<li>enable google report api</li>
<li>export project json</li>
</ul>
<h4 id="cloud"><a href="#cloud" aria-label="cloud permalink"></a>cloud</h4>
<ul>
<li>
<p>L/B free cloud</p>
<ul>
<li>nothing</li>
</ul>
</li>
<li>
<p>GCP app engine 28/d free</p>
<ul>
<li>it can be scaling</li>
</ul>
</li>
</ul>
<h4 id="current-used-infra"><a href="#current-used-infra" aria-label="current used infra permalink"></a>Current Used infra</h4>
<ul>
<li>telegrambot(serverless)</li>
<li>monitoring</li>
<li>content based recommend(need s3 csv file)(terraform)</li>
<li>s3 hosting(terraform)</li>
<li>netlify(wiki homepage)</li>
<li>github pages</li>
<li>cloudflare</li>
<li>empty</li>
<li>ec2</li>
<li>gce</li>
<li>gcp app engine</li>
<li>heroku</li>
<li>oracle 2대</li>
</ul>
<h2 id="카프카와-다른-메시지큐"><a href="#%EC%B9%B4%ED%94%84%EC%B9%B4%EC%99%80-%EB%8B%A4%EB%A5%B8-%EB%A9%94%EC%8B%9C%EC%A7%80%ED%81%90" aria-label="카프카와 다른 메시지큐 permalink"></a>카프카와 다른 메시지큐</h2>
<p>카프카는 분산, 고가용성, 고속에 특화
근데 무겁다</p>
<p>가벼운데 고가용성만 지원되면 좋겠다</p>
<h2 id="grpc"><a href="#grpc" aria-label="grpc permalink"></a>gRPC</h2>
<p>(Remote Procedure Call)</p>
<p>gRPC 4가지 스트리밍 방식</p>
<ul>
<li>단일</li>
<li>서버 스트리밍</li>
<li>클라이언트 스트리밍</li>
<li>양방향 스트리밍</li>
</ul>
<p>3가지 stub</p>
<ul>
<li>Blocking stub</li>
<li>(Async) stub</li>
<li>Future stub</li>
</ul>
<p>gRPC가 필요한 이유</p>
<ul>
<li>CORBA 등 과거의 RPC는 높은 복잡도, 높은 학습곡선, 낮은 개발생산성이 문제였다</li>
<li>RESTful이 낮은 복잡도, 낮은 학습곡선 등으로 잘 사용하게 됨</li>
<li>그러나 게임 등에서는 성능상의 이유로 위의 CORBA 등이 사용되고 있었음.</li>
<li>구글에서 Stubby를 만들고 gRPC는 오픈소스 버전</li>
</ul>
<p>Binary Protocol - Text Protocol(REST)
HTTP/2 base</p>
<ul>
<li>Connection Multiplexing</li>
<li>Header Compression</li>
<li>양방향 Streamming</li>
</ul>
<p>Browser에서 지원해야 함
데이터가 Binary라 바로 읽기 힘듬</p>
<h4 id="grpc-1"><a href="#grpc-1" aria-label="grpc 1 permalink"></a>grpc</h4>
<ul>
<li>protoc 설치: <code>apt install -y protobuf-comiler</code></li>
<li>buf 설치</li>
<li>
<p><code>buf.yaml</code>로 디펜던시 설치</p>
<ul>
<li><code>buf beta mod update</code></li>
</ul>
</li>
<li><code>buf.gne.yaml</code>로 proto 파일 변환</li>
<li>생성된 swagger.json 을 브라우저에서 보려면 swagger-ui를 설치해야 한다</li>
</ul>
<h4 id="grpc-2"><a href="#grpc-2" aria-label="grpc 2 permalink"></a>grpc</h4>
<p>gateway에 grpc를 어떻게 등록시키나를 놓쳤는데
알고보니 grpc서버를 따로 실행하고 gateway에 포트를 알려줘서 접근하게 하는
방식이었다.</p>
<p>grpc gateway에서 루트 url은 지원을 안한다. 따로 http server에서 작업을 해줘야
한다.</p>
<h4 id="grpc-3"><a href="#grpc-3" aria-label="grpc 3 permalink"></a>grpc</h4>
<p>데이터 아웃풋을 배열로 출력할 때 배열만으로 출력이 안되고 메시지 형태로 된다
name: [arr1,arr2]
그래서 받는 쪽에서 name을 골라서 받아야 되는데 이러면 안된다</p>
<p>람다는 reponse body 안에 다 넣어놓는 방식을 쓴다</p>
<p>array 안에서 이름 말고 다른 방식으로 값을 가져오는 방법은?
각 값마다 일일이 찾는 방법 밖에 없나...</p>
<ul>
<li>marshal, unmarshal</li>
</ul>
<p>grpc message에 담는 방법 외에 google http body를 이용해서 담을 수 있다.
<code>import "google/api/httpbody.proto";</code>
<code>returns (google.api.HttpBody)</code></p>
<h4 id="reference"><a href="#reference" aria-label="reference permalink"></a>reference</h4>
<ul>
<li><a href="https://devjin-blog.com/golang-grpc-server-4/">https://devjin-blog.com/golang-grpc-server-4/</a></li>
<li><a href="https://deepbaksu.github.io/2021/05/01/how-to-REST-from-gRPC/">https://deepbaksu.github.io/2021/05/01/how-to-REST-from-gRPC/</a></li>
<li><a href="https://tech.buzzvil.com/handbook/grpc/">https://tech.buzzvil.com/handbook/grpc/</a></li>
</ul></section><hr></article></div>

shdkej

prometheus
node-exporter
collect system metrics
alert-manager
ELK vs TICK
ELK -- log metrics
TICK -- system metrics
what is prometheus, loki,
grafana alert
monitoring 서버도 테스트 서버용과 프로덕션 서버용을 따로 두나?
alert를 만드려면 graph여야하고, $variable 로 되있는 템플릿을 쓸 수 없다.
alert를 만들고 싶은 graph를 복사해서 variable을 고치고 사용하면 된다.
sensu는 어떤 기능들을 제공해주고 있지?
grafana daily report
enterprise 기능이었다. 오픈소스로 구현된 것도 있을 것이다
TICK
too heavy
I want to find lightweight visualistic monitoring service
go to prometheus
Benchmark
구글개발자도구 audits
[[Architecture]]
nginx alternative
openlitespeed
ansible
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-ubuntu-18-04
setting /etc/ansible/hosts
terraform provision execute only during create
ansible command vs shell
command isn't running in $HOME, shell is
command can't use operations like <,>,|,&
command is more secure.
https://blog.confirm.ch/ansible-modules-shell-vs-command/
ansible
ansible-galaxy
setting /etc/ansible/hosts file or make hosts file with -i
it need ,
[webserver]
server_ip
# <work_directory>/hosts
hosts test ansible webserver -m ping -i hosts --private-key <private-key> -u <user>
https://alex.dzyoba.com/blog/terraform-ansible/
how to connect with pub file
Nginx letsencrypt
DNS setting
install letsencrypt
letsencrypt certonly --standalone
setting Nginx
run Nginx
terraform vs serverless
note-reminder has terraform trigger option.
it's not good..
테라폼만 쓸지 서버리스를 같이 쓸지 고민중
서버리스는 개별적인 앱을 빠르게 빌드하고 다시 만들 때 가볍게 사용하기 좋고
테라폼은 좀 더 넓게 공유되는 자원을 관리할 때 쓰기 좋다
서버리스는 앱을 빠르게 띄우기 좋고 테라폼은 인프라 셋팅하기에 좋다
https://www.serverless.com/blog/definitive-guide-terraform-serverless/
terraform
Need update when changing a provisioner
! resource "null_resource" -> null
Do terraform init
terraform taint aws_instance.example-server
aws_instance recreate when terraform apply
terraform null_resource is better then an instance make to a taint
https://github.com/gruntwork-io/terratest/blob/master/examples/terraform-remote-exec-example/main.tf
null resource need terraform init
recreate instance with same eip
!WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
clear ~/.ssh/known_hosts or recreate ssh key ssh-keygen -R <host IP>
https://www.ssh.com/ssh/keygen/
! aws_instance remote-exec ssh connection not working
user = "root" -> user = "ubuntu"
letsencrypt -d <domain> -m <email> -n(all agree) --agree-tos
https://github.com/ployst/docker-letsencrypt/issues/18
gcp metadata need ${}
ssh-keys = "username:${file("<PATH>")}"
Event Handling with sns, sqs
https://dev.to/frosnerd/event-handling-in-aws-using-sns-sqs-and-lambda-2ng
lambda function to python
save s3
! s3 access-denied problem
bucket name not allow var. I have just input text
var allow. but name is global. it is really name exist problem.
! s3bucketnotification invalid argument
create SNS 2.create S3 bucket 3. Policy 4. notification
sns - awsiampolicy - condition - values - (bucket arn -> bucket name...)
every apply update s3, using etag
terraform ansible
for provision sleep 120; is good to waiting ec2 instance creation
ansible-playbook to make with ip, ip + ,
terraform has depends_on
use module. For different folders can use once.
github action terraform
how to hide secret file
gcp credential file
! change backend bucket
need delete .terraform dir, and terraform init
AWS Dynamodb terraform
attribute need index
any key can write, if exist with attribute
serverless
install curl -o- -L https://slss.io/install | bash
!Error: spawn /home/sh/.serverless/bin/xdg-open ENOENT
no install xdg-open. manual install and copy to serverless/bin directory
!"service" property is missing in serverless.yml
get started is sucks
run serverless for first setting
serverless
python requirements
need install plugin serverless-python-requirements
vault
install file
move bin directory
vault in gcp
run docker 
what is different with server-mode and another
add ssh
save file
read file
google calendar api
credential.json 생성
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
코드 실행
인증
token.pickle 생성되고 코드 실행됨
https://developers.google.com/calendar/quickstart/python
nagios
Docker로 설치 후
/opt/nagios/etc/ 설정파일 설정
/etc/resource : 환경변수 설정파일
object/Commend
object/contacts
object/template
object/windows
nagios
window 는 nsclient 설치
서버에서 command.cfg 파일에서 check_nt 부분 비밀번호 명시
Telegram 연동 소스 받아서 설정
for docker monitoring
chown nagios /var/run/docker.sock
nagios 비밀번호 변경
htpasswd -c /opt/nagios/etc/htpasswd.users nagiosadmin
콘솔로 비밀번호 입력
aws lambda cronjob
using cloudwatch event rule, event target, lambda permission
more option cloudwatch log group, cloudwatch log subscription filter
https://www.thedevcoach.co.uk/terraform-lambda-scheduled-event/
serverless aws sqs lambda
sqs to lambda message parsing
event['Records'][0]['body']
lambda python requests
cannot version 3.8, can 3.6
devdash
google analytic settings
enable google report api
export project json
cloud
L/B free cloud
nothing
GCP app engine 28/d free
it can be scaling
Current Used infra
telegrambot(serverless)
monitoring
content based recommend(need s3 csv file)(terraform)
s3 hosting(terraform)
netlify(wiki homepage)
github pages
cloudflare
empty
ec2
gce
gcp app engine
heroku
oracle 2대
카프카와 다른 메시지큐
카프카는 분산, 고가용성, 고속에 특화
근데 무겁다
가벼운데 고가용성만 지원되면 좋겠다
gRPC
(Remote Procedure Call)
gRPC 4가지 스트리밍 방식
단일
서버 스트리밍
클라이언트 스트리밍
양방향 스트리밍
3가지 stub
Blocking stub
(Async) stub
Future stub
gRPC가 필요한 이유
CORBA 등 과거의 RPC는 높은 복잡도, 높은 학습곡선, 낮은 개발생산성이 문제였다
RESTful이 낮은 복잡도, 낮은 학습곡선 등으로 잘 사용하게 됨
그러나 게임 등에서는 성능상의 이유로 위의 CORBA 등이 사용되고 있었음.
구글에서 Stubby를 만들고 gRPC는 오픈소스 버전
Binary Protocol - Text Protocol(REST)
HTTP/2 base
Connection Multiplexing
Header Compression
양방향 Streamming
Browser에서 지원해야 함
데이터가 Binary라 바로 읽기 힘듬
grpc
protoc 설치: apt install -y protobuf-comiler
buf 설치
buf.yaml로 디펜던시 설치
buf beta mod update
buf.gne.yaml로 proto 파일 변환
생성된 swagger.json 을 브라우저에서 보려면 swagger-ui를 설치해야 한다
grpc
gateway에 grpc를 어떻게 등록시키나를 놓쳤는데
알고보니 grpc서버를 따로 실행하고 gateway에 포트를 알려줘서 접근하게 하는
방식이었다.
grpc gateway에서 루트 url은 지원을 안한다. 따로 http server에서 작업을 해줘야
한다.
grpc
데이터 아웃풋을 배열로 출력할 때 배열만으로 출력이 안되고 메시지 형태로 된다
name: [arr1,arr2]
그래서 받는 쪽에서 name을 골라서 받아야 되는데 이러면 안된다
람다는 reponse body 안에 다 넣어놓는 방식을 쓴다
array 안에서 이름 말고 다른 방식으로 값을 가져오는 방법은?
각 값마다 일일이 찾는 방법 밖에 없나...
marshal, unmarshal
grpc message에 담는 방법 외에 google http body를 이용해서 담을 수 있다.
import "google/api/httpbody.proto";
returns (google.api.HttpBody)
reference
https://devjin-blog.com/golang-grpc-server-4/
https://deepbaksu.github.io/2021/05/01/how-to-REST-from-gRPC/
https://tech.buzzvil.com/handbook/grpc/

Tool

댓글