Difference between revisions of "Team:USTC-Software/Model"

Line 1: Line 1:
{{USTC-Software}}
 
<html>
 
  
 +
<html lang="en">
 +
<head>
 +
    <meta charset="UTF-8">
 +
    <title>Team</title>
  
<div class="column full_size judges-will-not-evaluate">
+
<script src="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/js/jquery&action=raw&ctype=text/javascript" type="text/javascript"></script>
<h3>★  ALERT! </h3>
+
<p>This page is used by the judges to evaluate your team for the <a href="https://2017.igem.org/Judging/Medals">medal criterion</a> or <a href="https://2017.igem.org/Judging/Awards"> award listed above</a>. </p>
+
<p> Delete this box in order to be evaluated for this medal criterion and/or award. See more information at <a href="https://2017.igem.org/Judging/Pages_for_Awards"> Instructions for Pages for awards</a>.</p>
+
</div>
+
<div class="clear"></div>
+
  
<div class="column full_size">
+
<script src="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/js/scroll&action=raw&ctype=text/javascript" type="text/javascript"></script>
<h1> Modeling</h1>
+
  
<p>Mathematical models and computer simulations provide a great way to describe the function and operation of BioBrick Parts and Devices. Synthetic Biology is an engineering discipline, and part of engineering is simulation and modeling to determine the behavior of your design before you build it. Designing and simulating can be iterated many times in a computer before moving to the lab. This award is for teams who build a model of their system and use it to inform system design or simulate expected behavior in conjunction with experiments in the wetlab.</p>
+
<script src="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/js/smooth-scroll&action=raw&ctype=text/javascript" type="text/javascript"></script>
  
</div>
+
<script src="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/js/bootstrap&action=raw&ctype=text/javascript" type="text/javascript"></script>
<div class="clear"></div>
+
  
<div class="column half_size">
+
<link rel="stylesheet" href="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/css/bootstrap&action=raw&ctype=text/css">
<h3> Gold Medal Criterion #3</h3>
+
<p>
+
To complete for the gold medal criterion #3, please describe your work on this page and fill out the description on your <a href="https://2017.igem.org/Judging/Judging_Form">judging form</a>. To achieve this medal criterion, you must convince the judges that your team has gained insight into your project from modeling. You may not convince the judges if your model does not have an effect on your project design or implementation.
+
</p>
+
  
<p>
+
<link rel="stylesheet" href="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/css/navbar&action=raw&ctype=text/css">
Please see the <a href="https://2017.igem.org/Judging/Medals"> 2017 Medals Page</a> for more information.  
+
<link rel="stylesheet" href="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/css/style&action=raw&ctype=text/css">
</p>
+
<link rel="stylesheet" href="https://2017.igem.org/wiki/index.php?title=Team:USTC-Software/css/footer&action=raw&ctype=text/css">
 +
   
 +
 
 +
</head>
 +
<body>
 +
<script>
 +
    var scroll = new SmoothScroll('a[href*="#"]');
 +
</script>
 +
<section id="dowebok" class="fullscreenslider">
 +
    <nav class="navbar navcolor" id="nav">
 +
            <div>
 +
                <div class="navbar-header">
 +
                    <a class="navbar-brand" href="https://2017.igem.org/Team:USTC-Software">Brand</a>
 +
                </div>
 +
                <ul class="nav navbar-nav navbar-right page-1">
 +
                    <li><a href="https://2017.igem.org/Team:USTC-Software">Home</a></li>
 +
                    <li>
 +
                        <a href="#">
 +
                            Team
 +
                            <b class="caret"></b>
 +
                        </a>
 +
                        <ul class="menu">
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/team">Members</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/Collaborations">Collaborations</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/Attributions">Attributions</a></li>
 +
                        </ul>
 +
                    </li>
 +
                    <li>
 +
                        <a href="#">
 +
                            Project
 +
                            <b class="caret"></b>
 +
                        </a>
 +
                        <ul class="menu">
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/project">Description</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/implementation">Implementation</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/Validation">Validation</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/Improve">Improve</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/Demonstrate">Demonstrate</a></li>
 +
                        </ul>
 +
                    </li>
 +
                    <li>
 +
                        <a href="#">
 +
                            Human Practice
 +
                            <b class="caret"></b>
 +
                        </a>
 +
                        <ul class="menu">
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/HP/Silver">Silver</a></li>
 +
                            <li><a href="https://2017.igem.org/Team:USTC-Software/HP/Gold_Integrated">Gold_Integrated</a></li>
 +
                        </ul>
 +
                    </li>
 +
                </ul>
 +
            </div>
 +
        </nav>
 +
</section>
 +
<div id="top"></div>
 +
<div id="menu">
 +
    <ul>
 +
        <li><a data-scroll href="#item1" class="focused">Server Part</a></li>
 +
        <li><a data-scroll href="#item2">Frontend Part</a></li>
 +
    </ul>
 
</div>
 
</div>
 +
<!--<div class="scroll-img">
 +
    <img src="img/up.png" width="50px" href="#top" class="scroll_top">
 +
    <img src="img/down.png" width="50px" href="#bottom" class="scroll_bottom">
 +
</div>-->
  
<div class="column half_size">
+
<div id="team-content" style="border: solid;border-width: 1px;border-color: #e1e1e1; background-color: white;border-radius: 2px;">
<h3>Best Model Special Prize</h3>
+
    <h1 style="text-align: center;">Implementation</h1>
 
+
    <div id="item1" class="item one">
<p>
+
        <h2 style="text-align: center">Server Part</h2>
To compete for the <a href="https://2017.igem.org/Judging/Awards">Best Model prize</a>, please describe your work on this page  and also fill out the description on the <a href="https://2017.igem.org/Judging/Judging_Form">judging form</a>. Please note you can compete for both the gold medal criterion #3 and the best model prize with this page.  
+
        <div style="padding: 10px 5px 10px 5px;margin-top: 20px">
<br><br>
+
            <p>The server of Biohub 2.0 are written in Python 3.6, which is an easy-to-write and cross-platform language and can lower the learning curve of developing plugins. We choose Django to be the basic framework, partly for it allows fast prototyping and large-scale deployment, partly for it already contains a matured plugin system (Django App Framework). However such plugin system is static, which means plugins cannot be loaded/unloaded during runtime. It will be a great deficiency for a website that may frequently alternate its components. To solve the problem, we build our own plugin system on the basis of the one in Django. We carefully monkey-patched Django's underlying implementation to make its core components (URL resolving, data model registration, and etc.) dynamically changeable,  whilst avoiding possible memory leaks. We also supplemented Django by adding many new features:</p>
You must also delete the message box on the top of this page to be eligible for the Best Model Prize.
+
            <ul>
</p>
+
                <li><strong>Hot Reload:</strong> Other processes can send <code>SIGUSR1</code> signal to Biohub 2.0's worker processes to inform them that the installed plugin list has changed and the server needs to be reloaded. By this way the server can be renewed without stopping.</li>
 
+
                <li><strong>WebSocket Routing:</strong> Websocket is a protocol providing full-duplex communication channels over a single TCP connection, which is supported by most of modern browsers. Biohub 2.0 develops a customized protocol over Websocket, making it easier to route Websocket packets. We have used this protocol to achieve real-time notification sending and notifying accomplishment events in ABACUS plugin.</li>
 +
                <li><strong>Background Tasks:</strong> As a website designed for synthetic biology, their must be many computation intensive tasks, which will blocks requests if handled inappropriately. Thus we split out and encapsulate this logic and make it an independent module. Background tasks can use websocket to notify certain events. Currently we have used in ABACUS computing.</li>
 +
            </ul>
 +
            <p>The modules are all available for plugins, so developers can take advantages of them to build amazing artworks.</p>
 +
            <div class="subtitle">About the Bricks Data</div>
 +
            <p>Biohub 2.0 is constructed on the base of Biobricks data, which comes from two channels. We use the data downloaded from <a href="http://parts.igem.org/partsdb/download.cgi?type=parts_sql">offical interface</a> as our initial data. The initial data is actually a snapshot of iGEM, containing most information of the bricks. However certain fields, such as group name or parameters, are missing in it, so we complement it by crawling iGEM's web pages. Such behaviour is "lazy", which means it will not be invoked before deploying, but before a specific brick's data is accessed for the first time. Fetched data will be cached in the database, and be updated per 10 days to keep updated.</p>
 +
            <div class="subtitle">About Data Organizing</div>
 +
            <p>The initial data will be imported into a separate database (by default it is named <code>igem</code>). Biohub will link to it by creating virtual tables using MySQL's database view technology. Using database views may lower the speed of querying, but it provides more flexibility for data upgrading. If newer initial is available, we can upgrade the database simply by reloading <code>igem</code> database, rather than dropping and recreating the tables in production environment. Also it can prevent data redundancy. If multiple instances with different database configuration are deployed on the same machine, only one copy of initial data needs to exist in the database, saving the space of disks.</p>
 +
            <div class="subtitle">About About Bricks Ranking</div>
 +
            <p>Before ranking, we have to filter the initial data, since there exists many apparently useless bricks on iGEM. Such process will occur before deploying. We simply drop those bricks without DNA sequence, and dumped the filtered data into a new table (by default it is named <code>igem.parts_filtered</code>). At the same time we will pre-process some fields, such as extracting subparts information from edit cache, and pre-calculating some components of the ranking weight. You may refer to <a href="https://github.com/igemsoftware2017/USTC-Software-2017/blob/master/biohub/biobrick/bin/updateparts.py">updateparts.py</a> to see this process.</p>
 +
            <p>Then we will rank the bricks for the first time, using multiple statistical methods. You may refer to <a href="https://github.com/igemsoftware2017/USTC-Software-2017/blob/master/biohub/biobrick/management/commands/refreshweight.py">refreshweight.py</a> to see such process. At this stage, the bricks are completely ranked by initial data, without any factors from Forum.</p>
 +
            <p>After the server starts up, we will recalculate the ranking weights every 30 minutes. The reason for not evaluating them in real time is that the task may update the whole table and become time-consuming. From now on, Biohub will gradually correct the deviation in bricks ranking.</p>
 +
            <div class="subtitle">About Optimizing ABACUS</div>
 +
            <p>ABACUS is a plugin inherited from last year's project. It consumes great amount of memory during executing, and causes the prossibility to break down the main server. Such accident did happen in testing phase of USTC-Software 2016. To avoid it, we improved ABACUS and enable it to run in two ways: locally or distributedly. If no available executable file is detected, ABACUS will connect to remote slave servers and distribute the computation tasks. This design can largely save the expense of master server, and makes ABACUS infinitely scalable.</p>
 +
        </div>
 +
    </div>
 +
    <div id="item2" class="item two">
 +
        <h2 style="text-align: center">Frontend Part</h2>
 +
        <div style="padding: 10px 5px 10px 5px;margin-top: 20px">
 +
            <p>To improve the user experience, we build Biohub 2.0 as a single page application (or short for SPA), using the advanced MVVM framework <a href="http://vue.js/">Vue.js</a>. <a href="http://vue.js/">Vue.js</a> relies on webpack (not forcibly, but recommended) for pre-compiling, which encapsulating everything into a single file. This is inconvinient for a website with a plugin system, so we analyzed the code generated by webpack and developed an approach to load components dynamically. It is the theoretical basis of Biohub 2.0's plugin system. You can refer to <a href="https://github.com/USTC-Software2017-frontend/Biohub-frontend/blob/master/src/components/plugins/Plugins.vue">Main.vue</a> to see such mechanism.</p>
 +
            <div class="subtitle">About UI</div>
 +
            <p>We choose Bootstrap 3 as our basic framework. Bootstrap is a UI framework developed by Twitter team, with multiple elegant components and the ability to prototype quickly. Based on it, we added many customized styles and components to provides better experience.</p>
 +
            <p>To visualize the data, we use serveral data representing frameworks. For example, <a href="http://d3.js/">d3.js</a> for DNA sequance displaying in Forum and <a href="http://ngl.js/">ngl.js</a> for protein structure illustration. Such frameworks transform certain data into graphics, and make it easier for users to learn the content.</p>
 +
            <div class="subtitle">About Websocket</div>
 +
            <p>Based on the customized Websocket protocol, we encapsulate a handy library to handle Websocket packets tranferring. You may refer to it via <a href="https://github.com/USTC-Software2017-frontend/Biohub-frontend/blob/master/src/utils/websocket.js">websocket.js</a>.</p>
 +
        </div>
 +
    </div>
 
</div>
 
</div>
<div class="clear"></div>
+
<style>
 +
    html {
 +
        overflow-x: hidden;
 +
        font-size: 625%;
 +
    }
 +
    body {
 +
        min-width: 1020px;
 +
    }
  
<div class="column full_size">
+
    #content {
<h5> Inspiration </h5>
+
        top: -25px
<p>
+
    }
Here are a few examples from previous teams:
+
</p>
+
<ul>
+
<li><a href="https://2016.igem.org/Team:Manchester/Model">Manchester 2016</a></li>
+
<li><a href="https://2016.igem.org/Team:TU_Delft/Model">TU Delft 2016  </li>
+
<li><a href="https://2014.igem.org/Team:ETH_Zurich/modeling/overview">ETH Zurich 2014</a></li>
+
<li><a href="https://2014.igem.org/Team:Waterloo/Math_Book">Waterloo 2014</a></li>
+
</ul>
+
  
 +
    .subtitle {
 +
        font-size: 24px;
 +
        text-align: left;
 +
        margin: 20px 0 20px 0;
 +
    }
  
</div>
+
</style>
  
 +
</body>
 
</html>
 
</html>

Revision as of 11:19, 1 November 2017

Team

Implementation

Server Part

The server of Biohub 2.0 are written in Python 3.6, which is an easy-to-write and cross-platform language and can lower the learning curve of developing plugins. We choose Django to be the basic framework, partly for it allows fast prototyping and large-scale deployment, partly for it already contains a matured plugin system (Django App Framework). However such plugin system is static, which means plugins cannot be loaded/unloaded during runtime. It will be a great deficiency for a website that may frequently alternate its components. To solve the problem, we build our own plugin system on the basis of the one in Django. We carefully monkey-patched Django's underlying implementation to make its core components (URL resolving, data model registration, and etc.) dynamically changeable, whilst avoiding possible memory leaks. We also supplemented Django by adding many new features:

  • Hot Reload: Other processes can send SIGUSR1 signal to Biohub 2.0's worker processes to inform them that the installed plugin list has changed and the server needs to be reloaded. By this way the server can be renewed without stopping.
  • WebSocket Routing: Websocket is a protocol providing full-duplex communication channels over a single TCP connection, which is supported by most of modern browsers. Biohub 2.0 develops a customized protocol over Websocket, making it easier to route Websocket packets. We have used this protocol to achieve real-time notification sending and notifying accomplishment events in ABACUS plugin.
  • Background Tasks: As a website designed for synthetic biology, their must be many computation intensive tasks, which will blocks requests if handled inappropriately. Thus we split out and encapsulate this logic and make it an independent module. Background tasks can use websocket to notify certain events. Currently we have used in ABACUS computing.

The modules are all available for plugins, so developers can take advantages of them to build amazing artworks.

About the Bricks Data

Biohub 2.0 is constructed on the base of Biobricks data, which comes from two channels. We use the data downloaded from offical interface as our initial data. The initial data is actually a snapshot of iGEM, containing most information of the bricks. However certain fields, such as group name or parameters, are missing in it, so we complement it by crawling iGEM's web pages. Such behaviour is "lazy", which means it will not be invoked before deploying, but before a specific brick's data is accessed for the first time. Fetched data will be cached in the database, and be updated per 10 days to keep updated.

About Data Organizing

The initial data will be imported into a separate database (by default it is named igem). Biohub will link to it by creating virtual tables using MySQL's database view technology. Using database views may lower the speed of querying, but it provides more flexibility for data upgrading. If newer initial is available, we can upgrade the database simply by reloading igem database, rather than dropping and recreating the tables in production environment. Also it can prevent data redundancy. If multiple instances with different database configuration are deployed on the same machine, only one copy of initial data needs to exist in the database, saving the space of disks.

About About Bricks Ranking

Before ranking, we have to filter the initial data, since there exists many apparently useless bricks on iGEM. Such process will occur before deploying. We simply drop those bricks without DNA sequence, and dumped the filtered data into a new table (by default it is named igem.parts_filtered). At the same time we will pre-process some fields, such as extracting subparts information from edit cache, and pre-calculating some components of the ranking weight. You may refer to updateparts.py to see this process.

Then we will rank the bricks for the first time, using multiple statistical methods. You may refer to refreshweight.py to see such process. At this stage, the bricks are completely ranked by initial data, without any factors from Forum.

After the server starts up, we will recalculate the ranking weights every 30 minutes. The reason for not evaluating them in real time is that the task may update the whole table and become time-consuming. From now on, Biohub will gradually correct the deviation in bricks ranking.

About Optimizing ABACUS

ABACUS is a plugin inherited from last year's project. It consumes great amount of memory during executing, and causes the prossibility to break down the main server. Such accident did happen in testing phase of USTC-Software 2016. To avoid it, we improved ABACUS and enable it to run in two ways: locally or distributedly. If no available executable file is detected, ABACUS will connect to remote slave servers and distribute the computation tasks. This design can largely save the expense of master server, and makes ABACUS infinitely scalable.

Frontend Part

To improve the user experience, we build Biohub 2.0 as a single page application (or short for SPA), using the advanced MVVM framework Vue.js. Vue.js relies on webpack (not forcibly, but recommended) for pre-compiling, which encapsulating everything into a single file. This is inconvinient for a website with a plugin system, so we analyzed the code generated by webpack and developed an approach to load components dynamically. It is the theoretical basis of Biohub 2.0's plugin system. You can refer to Main.vue to see such mechanism.

About UI

We choose Bootstrap 3 as our basic framework. Bootstrap is a UI framework developed by Twitter team, with multiple elegant components and the ability to prototype quickly. Based on it, we added many customized styles and components to provides better experience.

To visualize the data, we use serveral data representing frameworks. For example, d3.js for DNA sequance displaying in Forum and ngl.js for protein structure illustration. Such frameworks transform certain data into graphics, and make it easier for users to learn the content.

About Websocket

Based on the customized Websocket protocol, we encapsulate a handy library to handle Websocket packets tranferring. You may refer to it via websocket.js.