Difference between revisions of "Team:Heidelberg/Sandbox1025"

Revision as of 20:52, 20 October 2017

Software

KW35

==

Performance of the Squeezenet Architecture - singlelabel 599

The model was run successfully on the old 599 classes dataset. Parameters: lr = E-2, batchsize=64, epsilon=0.1 [ROC](DeeProtein_TFRECORDS_PURECONV_1x1tuned_750k_restored750kfull_sce_adam_1dconv637_1000_one_hot_padded_64_0.001_0.1.roc_16.svg) [Precision](DeeProtein_TFRECORDS_PURECONV_1x1tuned_750k_restored750kfull_sce_adam_1dconv637_1000_one_hot_padded_64_0.001_0.1.precision_16.svg)

Performance of the Squeezenet Architecture - singlelabel 679

The model was run successfully on the 679 classes dataset. Parameters: lr = E-5, batchsize=64, epsilon=0.1 [ROC](DeeProtein_TFRECORDS_PURECONV_1x1tuned_restored679_sce_adam_1dconv_EC_679_1000_one_hot_padded_64_0.001_0.1.roc_9.svg) [Precision](DeeProtein_TFRECORDS_PURECONV_1x1tuned_restored679_sce_adam_1dconv_EC_679_1000_one_hot_padded_64_0.001_0.1.precision_9.svg)

Performance of the Squeezenet Architecture - Multilabel 1084

The model was run successfully on the 1084 GO-classes dataset. Parameters: lr = E-3, batchsize=64, epsilon=0.1 [ROC](DeeProtein_TFRECORDS_PURECONV_1x1LARGE_MULTI_restored1084_sce_adam_1dconv_EC_1084_1000_one_hot_padded_64_0.0001_0.1.roc_39.svg) [Precision](DeeProtein_TFRECORDS_PURECONV_1x1LARGE_MULTI_restored1084_sce_adam_1dconv_EC_1084_1000_one_hot_padded_64_0.0001_0.1.precision_39.svg)

Corrected datasets for missing classes, reworte ```eval()``` to enclude the whole validation set

- Dataset 637, was missing 138 classes due to the min. length requirement in the ```DatasetGenerator``` class. The requirement was lowered to 175AA. Further the ```DatasetGenerator``` class was rewritten, to ensure to contain 5 samples from every class in the validation set. - the ```eval()``` function of ```DeeProtein``` was rewritten to perform the validaion on the _whole_ validation set at given steps.

Performance on 679 classes with minlength 175: lr=0.01, e=0.1, batchsize=64 [ROC](DeeProtein_TFRECORDS_PURECONV_1x1tuned_restored637750kfull_sce_adam_1dconv679_1000_one_hot_padded_64_0.01_0.1.roc_32.svg) [Precision](DeeProtein_TFRECORDS_PURECONV_1x1tuned_restored637750kfull_sce_adam_1dconv679_1000_one_hot_padded_64_0.01_0.1.precision_32.svg)

lr=0.001, e=0.1, batchsize=64 [ROC](DeeProtein_TFRECORDS_PURECONV_1x1tuned_restored637750kfull_sce_adam_1dconv679_1000_one_hot_padded_64_0.01_0.1.roc_32.svg) [Precision](DeeProtein_TFRECORDS_PURECONV_1x1tuned_restored637750kfull_sce_adam_1dconv679_1000_one_hot_padded_64_0.01_0.1.precision_32.svg)

Reinitialization with pretrained parameters and lower learning rate allowed finetuning of the classifier. Especially as the validation set is uniformally distributed (in contrast to the training set) the classifier can be considered as trained.

ROC/ACC/AUC-metrics

ROC and AUC was added to be calculated on the fly (after validation on the whole validation set.).

Training models on the embedded sequences

We generated batches from the word embeddings (dim=100, kmer-length=3) for the 679(EC) and the 1084 mulilabel network. However training proceeds much more slowly as the parametersize is 5 times the size of the one-hot network.

Multilabel-classification

In order to be able to perform multilabel classification, we rewrite the input pipeline (```DatasetGenerator, BatchGenerator, TFrecordsgenerator```) and generated two datasets with 339 and 1084 classes respectively. The considered labels were chosen solely based on their polulation. As the GO-term hierarchy follows a directed acyclic graph (DAG) we looked up all parent nodes for each leaf nodes and included the total set of annotations for each sequence.

First models were run after extending the network for 2 convolutional and 2 1x1 layers on the 1084 classes dataset. Results were disenchanting.

Comparison of datasets

Total seqs after filtering (EC): 220488 Total seqs after filtering (GO): 235767

| Datatype | Dataset | Samples | % of filtered sequences considered | % of total sequences considered | |----------|---------|---------|------------------------------------|---------------------------------| | EC | 679 | 165658 | 75.13 | 63.18 | | GO | 1084 | 233386 | 98.99 | 89.00 |

Word2Vec

The new word2vec adaption we implemented last week was optimized by a few minor changes. Metadata of the embeddings can now be used to analyse the embeddings with tensorboard. Search for a single k-mer, finding the nearest neighbours, annotating frequencies or other properties to the points and search for groups of k-mers defined by regular expressions work. Embeddings of 3-mers and 4-mers in 50, 100 and 200 dimensions were calculated on whole swissprot and whole uniprot. Principle Component Analysis (PCA) was performed on a 100-dimensional embedding of 3-mers from swissprot and showed interesting properties. However, reduction of 100 dimensions to two or three may account for at least some of these.  Here the 3-mers marked darker are more frequent, those seem to cluster together.  For each amino acid all 3-mers containing it are marked in read. Some of those selection cluster together, others do not. This may also be due to the dimension reduction.  All 3-mers containing cysteine are marked in red.  All 3-mers containing lysine are marked in red.  All 3-mers containing proline are marked in red.  All 3-mers containing valine are marked in red. Especially cysteine, proline and lysine containing 3-mers cluster together, while the ones containing valine appear to be more equally distributed. Clustering of k-mers based on amino acid content is a first hint, embeddings can be useful for input of neural networks. However their performance can probably only be measured indirectly by comparing the performance of a single architecture on different inputs.

Modeling

Calculations of medium consumption for iGEM goes green were performed and made interactive. A heatmap visualizes medium consumption, calculation of ideal turbidostat and lagoon sizes are possible. The user can annotate own experiments and compare them to others. [Heatmap with default values](170827mod-heatmap.png) Experimentation with elements of a Predcel model based on distributions instead of scalars were started. The idea is that a population of phages does not have one fitness between 0 and 1 but rather has individuals that have different fitness values. In this more complex model the concentrations have to be calculated for each phage fitness value, depending on the amount of phages that have that fitness value. The fitness distribution is changed by mutation and by selection. The first naive approach for mutation was programmed. It simply substracts a given percentage of the difference between the amount of a fitness value and the mean amount from the amount of a fitness value. Obviously this is oversimplified and will therefore be replaced by a model based on the idea that every sequence that mutates gets better or worse with normally distributed changes.

}} }}