<div id="readability-page-1" class="page"><article>
    <section id="post-body">
        <p>Project: Software Studio @ cs.hanynag</p>

<h2 id="project-requirements">Project Requirements</h2>

<ul>
  <li>Use <a href="https://www.docker.com/">Docker</a> (10pts)</li>
  <li>Performance Comparison (using <a href="http://naver.github.io/arcus/">arcus</a>) (20pts)</li>
  <li>Use <a href="https://github.com/naver/nbase-arc">nBase-ARC</a> (20pts)</li>
  <li>Use multi-node (10pts)</li>
  <li>Use <a href="https://github.com/naver/hubblemon">Hubblemon</a> (10pts)</li>
  <li>Use <a href="http://naver.github.io/ngrinder/">nGrinder</a> (10pts)</li>
  <li><a href="https://github.com/naver">Naver Open Source</a> Contribution
    <ul>
      <li>Oppend Issues
        <ol>
          <li><a href="https://github.com/naver/arcus-python-client/issues/11">arcus-python-client: <code>Pypi's arcus package seems to have expired</code></a></li>
          <li><a href="https://github.com/naver/arcus-python-client/issues/12">arcus-python-client: <code>Not working in macOS</code></a></li>
          <li><a href="https://github.com/naver/arcus/issues/35">arcus: <code>Support docker</code></a></li>
        </ol>
      </li>
      <li>Pull Requests
        <ol>
          <li><a href="https://github.com/naver/hubblemon/pull/22">hubblemon: <code>update dependency</code></a></li>
          <li><a href="https://github.com/naver/arcus-python-client/pull/13">arcus-python-client: <code>Update Poller for darwin(macOS)</code></a></li>
          <li><a href="https://github.com/naver/arcus/pull/36">arcus: <code>Update dockerfile</code></a></li>
        </ol>
      </li>
    </ul>
  </li>
</ul>

<h2 id="project">Project</h2>

<p>Using <a href="https://www.docker.com/">Docker</a> with <a href="https://hub.docker.com/r/ruo91/arcus/">arcus</a>, <a href="https://hub.docker.com/_/mysql/">MySQL</a>, <a href="https://hub.docker.com/r/hyeongseok05/nbase-arc/">nBase-ARC</a>.</p>

<p>API app is simple flask app for performance comparison.</p>

<p>All settings are stored in <code>settings.json</code>. you can change enviroments for you own service.</p>

<h3 id="docker">Docker</h3>

<p>Using dockerized images and dockerfiles.</p>

<p>Check <code>app.py</code> and <code>settings.json</code> for docker run.</p>

<p><img src="https://github.com/MaybeS/ITE3068/blob/master/results/docker%20process.png?raw=true" alt="Docker ps -a"></p>

<h3 id="api-app-for-performance-comparison">API App (for performance comparison)</h3>
<p>Powered by flask, support simple RestfulAPI for performance comparison.</p>

<p>API Lists</p>

<ul>
  <li><code>GET</code>: <code>/init</code>
  Initialize database, create testset table, and insert some records.</li>
  <li><code>GET</code>: <code>/mysql</code> 
  Select some integer from mysql (range 0 - testsize)</li>
  <li><code>GET</code>: <code>/arcus</code>
  Select some integer from arcus if missed, select from mysql and add to arcus</li>
  <li><code>GET</code>: <code>/nbase</code>
  Select some integer from nbase if missed, select from mysql and add to nbase</li>
</ul>

<h3 id="mysql">MySQL</h3>

<p>Pulling from public mysql dockerfile(version <code>5.7</code>, but it can also <code>latest</code>).
There are some enviroments for mysql db.</p>

<ul>
  <li><em>MYSQL_ROOT_PASSWORD</em>: password</li>
  <li><em>MYSQL_USER</em>: maybe</li>
  <li><em>MYSQL_PASSWORD</em>: password</li>
  <li><em>MYSQL_DATABASE</em>: ite3068</li>
</ul>

<h3 id="arcus">Arcus</h3>

<p>Pulling from <a href="https://hub.docker.com/r/ruo91/arcus/">ruo91/arcus</a> and some appendix scripts for memcached server.
See <code>arcus/install.sh</code>. It provide arcus to memcached server list and set up zookeeper and memcached.
It automatically run after docker container started.</p>

<p>Arcus is memory cache cloud based on <a href="https://memcached.org/">memcached</a> and <a href="https://zookeeper.apache.org/">zookeeper</a>. So Arcus is distributed cache cloud. Therefore Arcus can configure multiple nodes to improve performance.</p>

<p>In this project, use 3 memcached-server on arcus. You can see how many nodes are used in settings.json.</p>

<h3 id="nbase-arc">nBase-ARC</h3>

<p>Pulling from <a href="https://hub.docker.com/r/hyeongseok05/nbase-arc/">hyeongseok05/nbase-arc</a>.
Dockerfile prepare all for start nbase-arc, so just start docker container is enough.</p>

<p>nBase-ARC is distributed storage platform using <a href="https://redis.io/">redis</a> and clustering.
A cluster consists of several gateways and replication groups, and each replication group has its own storage unit called redis.
There is an advantage that the redis API can be used as it is.</p>

<p>In this project use 4-cluster node server.</p>

<h3 id="hubblemon">Hubblemon</h3>

<p>It’s difficult to compose all of docker container in a single <code>docker-compose</code>. 
<em>Because, arcus and memcached require settins after container started.</em></p>

<p>So, hubblemon run each <code>mysql</code>, <code>arcus</code>, <code>nbase</code>.
To do this, after each container started, process <code>hubblemon/install.sh</code>.</p>

<p>Script contains below</p>

<ul>
  <li>Install depedency (It takes time depending on the internet(or repo server) speed)</li>
  <li>Clone hubblemon repository</li>
  <li>Copy each setting</li>
  <li>Install python dependency</li>
  <li>Run server</li>
</ul>

<p>Hubblemon initially started to monitor Arcus. So it is not very difficult and kind to use with other clients while currently support other clients,</p>

<p>Monitoring is based on a django, web and information about the listener is delivered when the client is connected, and the client executes the information collected by the server to be monitored.</p>

<p>Since hubblemon runs in the container, it can take a very long time to start(In my case, about 5-6m).</p>

<p>In settings.json, you can see <code>HUBBLEMON</code> environments. This port is the port where HubbleMon runs in each container. And <code>install.sh</code> script on hubblemon dir is for run hubblemon in each container.</p>

<ul>
  <li>MySQL: 4584</li>
  <li>Arcus: 4585</li>
  <li>nBase-ARC:  4586</li>
</ul>

<p><img src="https://github.com/MaybeS/ITE3068/blob/master/results/hubblemon.png?raw=true" alt="Running hubblemon on arcus"></p>

<h3 id="ngrinder-test">nGrinder Test</h3>

<p>nGrinder supports writing a script that sends an HTTP request and sends it to the agent for testing. 
The web server receives the HTTP request and can communicate with the mysql server or the arcus server. 
Now you can actually write those scripts through nGrinder to compare performance differences between using the mysql server directly and the arcus server.</p>

<p>You can test the performance in nGrinder by calling the api created in the flask app above.</p>

<p>Check my results in <code>results</code> directory.</p>

<h2 id="review">Review</h2>

<p>The performance test of <code>mysql</code>, <code>arcus</code> and <code>nbase</code>.</p>

<p><img src="https://github.com/MaybeS/ITE3068/blob/master/results/mysql.png?raw=true" alt="mysql"></p>

<p><em>mysql ngrinder results</em></p>

<p><img src="https://github.com/MaybeS/ITE3068/blob/master/results/arcus.png?raw=true" alt="arcus"></p>

<p><em>arcus ngrinder result</em></p>

<p><img src="https://github.com/MaybeS/ITE3068/blob/master/results/nbase.png?raw=true" alt="nbase"></p>

<p><em>nbase ngrinder result</em></p>

<p>nBase-ARC is fastest because memory cache is faster than access raw db. Over 10 times faster than mysql. 
Now that mysql is running too slow, arcus is slowing down.
If you increase the experiment time or if mysql is fast enough, the acus will be even faster.</p>

<h2 id="usage">Usage</h2>

<p>Just run application, it uses docker service.</p>

<div><pre><code>pip3 install -r api/requirements.txt

python3 app.py start

// To setup arcus memcached server
docker exec -it arcus-admin /bin/bash -c /opt/install.sh

// To setup hubblemon, each container
docker exec -it arcus-memcached-1 /bin/bash -c /opt/install.sh
docker exec -it mysql /bin/bash -c /opt/install.sh
docker exec -it nbase-arc /bin/bash -c /opt/install.sh

// You can run this command in other shell after above execution print `Docker started!`
// Waiting hubblemon takes too long, You do not have to wait for hubblemon to start the api server.
python3 api/app.py
</code></pre></div>

<p>for stop service,</p>



    </section>
</article></div>

ITE3068

댓글