FlowPicRefresh/_reference/Intrusion-Detection-System-Using-CNN-and-Transfer-Learning/3-Ensemble_Models-CAN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehicles \n",
    "This is the code for the paper entitled \"**A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehicles**\" published in **IEEE International Conference on Communications (IEEE ICC)**, doi=[10.1109/ICC45855.2022.9838780](https://ieeexplore.ieee.org/document/9838780).    \n",
    "Authors: Li Yang (lyang339@uwo.ca) and Abdallah Shami (Abdallah.Shami@uwo.ca)  \n",
    "Organization: The Optimized Computing and Communications (OC2) Lab, ECE Department, Western University\n",
    "\n",
    "**Notebook 3: Ensemble Models**  \n",
    "Aims:  construct three ensemble techniques: Bagging, Probability averaging, and Concatenation, to further improve prediction accuracy  \n",
    "* Bagging: use majority voting of top single models  \n",
    "* Probability averaging: calculate the average probability of the single model prediction results (the last layer of CNN models), and select the largest probability class to be the final class  \n",
    "* Concatenation: extract the features in the last several layers of single models, and concatenate together to generate the new layers, and add a dense layer to do prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "import keras\n",
    "from keras.models import Model,load_model\n",
    "from keras import Input\n",
    "from keras.layers import concatenate,Dense,Flatten,Dropout\n",
    "from keras.preprocessing.image import  ImageDataGenerator\n",
    "import keras.callbacks as kcallbacks\n",
    "import os\n",
    "import math\n",
    "from keras.utils import plot_model\n",
    "from keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint, LearningRateScheduler\n",
    "from keras.optimizers import SGD\n",
    "import operator\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "from collections import defaultdict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 5845 images belonging to 5 classes.\n"
     ]
    }
   ],
   "source": [
    "#generate images from train set and validation set\n",
    "TARGET_SIZE=(224,224)\n",
    "INPUT_SIZE=(224,224,3)\n",
    "BATCHSIZE=128\n",
    "\n",
    "test_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "\n",
    "validation_generator = test_datagen.flow_from_directory(\n",
    "        './test_224/',\n",
    "        target_size=TARGET_SIZE,\n",
    "        batch_size=BATCHSIZE,\n",
    "        class_mode='categorical')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#generate labels indicating disease (1) or normal (0)\n",
    "label=validation_generator.class_indices\n",
    "label={v: k for k, v in label.items()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: '0', 1: '1', 2: '2', 3: '3', 4: '4'}\n"
     ]
    }
   ],
   "source": [
    "print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 ./test_224/0\\100015.png\n"
     ]
    }
   ],
   "source": [
    "#read images from validation folder\n",
    "rootdir = './test_224/'\n",
    "test_laels = []\n",
    "test_images=[]\n",
    "for subdir, dirs, files in os.walk(rootdir):\n",
    "    for file in files:\n",
    "        if not (file.endswith(\".jpeg\"))|(file.endswith(\".jpg\"))|(file.endswith(\".png\")):\n",
    "            continue\n",
    "        test_laels.append(subdir.split('/')[-1])\n",
    "        test_images.append(os.path.join(subdir, file))\n",
    "        \n",
    "print(test_laels[0],test_images[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load 5 trained CNN models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    " #load model 1: xception\n",
    "xception_model=load_model('./xception.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    " #load model 2: VGG16\n",
    "vgg_model=load_model('./VGG16.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    " #load model 3: VGG19\n",
    "vgg19_model=load_model('./VGG19.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    " #load model 4: inception\n",
    "incep_model=load_model('./inception.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    " #load model 5: inceptionresnet\n",
    "inres_model=load_model('./inceptionresnet.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use the original CNN base models to make predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Xception"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted result for the first image: 0\n",
      "Confidence level: 1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQUAAAD8CAYAAAB+fLH0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzsvV/Idd923/UdY661n/d3/kh7mngIbTBWQm8EI4Tmor2olIqKGHoTm4u2Wml7kwulFz3mQsTcHKS1FIRiisUG1CpoaChBLYUiXiipQbC1RtqS0ByapJViSnPeZ685x/BijDHnmGuvvZ9nv+/znD7nl3e9zHfNNdfaa+9n7zU/8zvGHHNOUlV82j5tn7ZPW2z8T/oDfNo+bZ+2t7V9gsKn7dP2aZu2T1D4tH3aPm3T9gkKn7ZP26dt2j5B4dP2afu0TdsnKHzaPm2ftml7NSgQ0b9CRD9HRH+LiL72Wu/zafu0fdpedqPXiFMgogLg/wHwewD8IoCfAfDDqvp/vfibfdo+bZ+2F91eSyn8dgB/S1X/jqqeAfwFAD/4Su/1afu0fdpecFte6b6/GcDfTce/COAHrl38xc++pF/5p37T3W9Cu/z+GESXZbfylPIKqBJUAfQ9Do6pl+c8lKByUBbCjBRE/kaRJwWR+h/jeczltp/zcZ/5tfkPIngBQJzydHANPeO8fWzxv0tE7e8XQFS9DNBd3vaAivbXtqbTPeIa+3sA9rftCZHXXsYX53B5Lr8mXXftWeh7v+ipZ+fWM3T3c7O7RsTP7+557/ZL+Ll/oKrf+dR1rwWFJzci+iMA/ggA/MYvfwX/3r/5o3e9nuE/uO8ZhCUeBNgxe76X0a68l41zARfZGG0jtMo938s2htSU972Vj/z2awWtxuvGdSoALQ28VvDSLK0NvNSx72UNtFaUpYH8eFx3PU+LArwApfh+AXEB1gegLAAXUFl219ie0mvAZdqT72sjnM+Cx0fF42Pee/4sF+XnfM3Z8r/6q206/973gOK0KtYFOK2K0wKsK3BadmWeXxc/twKna2U9b/tC6fcHwEQoBJT+zNDu/HieCmj3XO2eIwK0Pwv+DNT8rIznYjwj8TzlZ49w/rVlvC7d497t6/idv/DcuvUa2zcAfHc6/i1e1jdV/XFV/X5V/f4vfvall3vnWwTtqL1x+rBJeCaW6Ynja3e7uI4ujuKj91ff2VJc+SiH27Vb376Hznm9/0b98lA8T7zn5fd4fHWU6jO+hP5d79+Brl1z/T5Hm6YrLu9x9Kr0RYSyfOXttaDwMwC+l4j+WSI6Afh9AH7qld4Lx1/mwZdH/b/Dr5b2wND5ztd+juPy6/AZn+XGXejy8FLIPnnTG9cfPZRx5vg1+sT9jvN6YdrdBAbQ5TPSZce/1+UHvP4k6G2oXQHKNQg/xZfDP+/JP0KvXvat3F7FfFDVSkQ/AuB/hKmxP6eqf+PF3we5il9vpqcGn46uvfbK5/1AV+928YPfUaEP3ni6w+Gtjj7tfSizM8cViK4eXLmfQ1ihz/geCRQAOVBOF8A+vMUTCLnZ06YIUI6WcicPLq6+8VGmT3FD9hwwh25ecI/m+7Dt1XwKqvrTAH76te4PHFb748/y5BVxnduC3Rk4t863W5orb3z98Pb2+r/9s9r8q9uhDbSrlLf+/gOohEBQpa5j7jIfOjeeI/D329FvnJ6c3UN0+Lno6plxj5vv/pztuU/zh2+fk4jG2y3k7dZ1bEeN32Hj8ixrhXbXPYWwjyFI3PhW+/78G97WFjrf9vBVt237421UyujkuHjPu+535wt0vNPhuz0t5G6cOf7OMkOu3++ZjqoX3D4nULgtdvUi88yvluZ6dl89vZS7U/W801N16Py6eMtbcLx8uK5XuOebOTf/jFst6xVr4/bXks4eeQ2vfOznYTFB6akbPuNuz37Jbfn0Ye/zkdu3NRRuV5Tbjsbj+106Gu/6SW5e/MEU2J19qZbiGixuuSAvX371c9F84RO+RYT5QDRfPN/1CW/PlSb++l+aLzt6mjKE5ntec8Y+8VGull2/27fAjtxt39ZQOHwgnwP3qy3KbQ/0kz/PE0rvyapGx/m5Qh3b64fvf8fb5e3a57ywhqYLr1XSK9g4cCYCLnZSl+TV7+xJT+jB9lxZ84zTz5f71z/M3T6mb5Fo+LaGwtieWUOeoyOvVMz7N7pfZuxUysEdx4c6VJr3fOD726bJ3Ljwdx204nd9fwl7F7LiWV6YK9/3swyvK6bXtXvf6t48akDucz7qtaNvkWj4nEDhOS4sPGnMHz1kSVU+Xcf3LeftpnG3XZPf4/Bpxty6Yn/uekt17ZE/8s8f3/+ySjxHWPTrk2FPNz7rFbFx+77Xzz7x4vnwuX/LdMGNPyID9xAo30LXwucECmk78Cr2XPIpzA3R7F7q11I0vnTzsriWEkH6eIT0Kfr4Bti1lK6bIZJbh3EPi9vX9HDvwEPHe8rH+2v6e80yRQ/OqZ/JpXMMwvV7XTo1j67F9H1dlNNRzTii9vF1t60HPb4mFETaXfWS6PydXP61l9/CkULR9LvGnZSms5fv/YLbP7GxD3kjBcp2nzaywS4em07jeB+PTqBxTADFNRSVzMoQx+RuJDJnEhNBd69TUoAESgxlATO5HUwAi78ZY2mCxgRiAhWLbye2gVLsYx8oxj5MYyDG+Icx3qH11+Tj+Xwa+1AUKAplAViBYnst/hlLgbJAuaTjBpRm4xy4Qcn2oALlCtACUAFoQSXCGYIzCTZSnElwJsXmxxsEFYIKRYOgxZ4UjQSNBAKFokEhUAjsgbf9qF626a5a9cpH2OVtL7DfyccSQUj7Xvw8UXonBYQAqDkmRdO4GU3PUn+uLsfM7MtjfIxUH9/QfF8Zrfmx+L4xRAhNvCz2yti0oKnlmzIEhPaK7fmbgAKUwPU+KEQFZ075Ej8KjR80wcIGCB7kIzGlVpUcFhkGALH2PLNgIg6TxW86CFRs3wIKTKACaEMHwAUUrlb4VH4DJHGOCqBFARYoOxyKdCgoF2gRgA0GykvKOxTYYcAFSguUqgECCxoBj6QGBth+g8FgI0UlQU17A4GBwRAQ+4CCQsnBoDKprgsgROXHDIQAQZQLEggAh4FDIRoEDDDEW06VXEdlp/055HN0UabVBzXt920HhuYQ8L3BwSEghE0LRBmiBoNfJ1DAXVCI1p3ZfzSiXsmZg/DjR+rXEsCcoBAQ2OcTLIZSsDoPtgePGdaEEAwCqYmwe9uQV2K4SgBaAWgjqOiuksvTFX8RKytp9GTk+96vWU0paAdC5AUo5JXdlUIpUCrQ0lL5gIOSJXEgCBkgGgFnUk+uDkItJEBUaFIMQzUEFEIp9OpNMkt5ulQI/SgpA8UAglX6lE/QGICw345BIJf9YckJXC2mn3hAASCluWy6hvqxVf4YIWkgkJbA0MgUQlYNCQYiBgKDwgBC5F9rexNQ+BDzgb3yxZ7U8izWGk9yL+R/VH7GLm/7DIc+3r6riNirw0HHU5ISpaSiEHYYsClv8XJ2ENCu4nPJZTIqfz6+cX1WCpKAICwAC6iQmxTSK744COZ9HcdUOiDEIVGJJiDkfD1UCkkteN6kfLMKTN6ua0yosLPCaWc60IyIC+WAW0Cw8nhtAIDU3jryuXza5/P7c31PUIdCB0J1SIQ6cBh0ZeAqIWDQ1PI1zAY1IIjD4bW8Cm8CCqYU7iMfeyVjDTlHKNEya9A6Ed39Dx0EJcGhjLJud0RLEr8y66wlw9Bk7XliNZObYS21aC+TopANkAJABFTkUgHs8l1FlJ2iKFldZDCMvH0Og4FwA1ExX8ECNwcKlBniEBCaoSDMVkbjfD9HjErUfQkbKc7QYT54PiuE2v0K2XQIpRD/XCVQdtPtnJbJhwBcqoWo+Hv/Qi7PSqHPY+PPivpbTpXcvXy0K0cGQ/gGUz4A0GEwgWEPB7+20QCD0FAKIE/c86+1vQkokBDWbz4fCgQDAhUCLQAXs9mZHRAUjsJkWnC+1l9bjsqsUnNVoCqkiuU3gKr2xJudU79GNu15rQrZFGVtkKqQLa5TaBWDRVT8InN+MTOAl1E+9gGFXb4rCulAQQEaMZpX4sYFjRgoG5TZWn1mtLTfX5uPe3mcU+omw0aKjT1FvpsUOY2ySurAaAkQc3tv26UC2ANg7ysQAhppcvVo+H5RXe3lmZziDcx/TNMxJJ7RuI4GENJHJcn3sXtoVPa633M/nq/hy2sboUnpgDCl8OsBCgqUOx2N3ebzyk/kpgOTKYV0DTkkLpRCQGBJ+8XOKwuIBUQMkIDJTIHwJQwJbs48YoEU37OCWACNiirQzc5rUajMAKAOgVGp43X7azJE4poBBOnKwqYPYpPlzCASUDhcHArWe2IVvgOCGDVBoRHv8rYPpVB7b4NV8p6nkdqkEiw1qFfq4V04hoJtxz0Nx8dXYbE71584lwgGiHyjBIb8sVxyXJzzfJRHpdbGU0XXSR14mXg+KQT1fXXTQQMIGlB4HTC8GSjwHT6FsNmG3ef+BeVezkrWq4G9PyGpgsXAgMWAgMWUA2IaQwKIoosx/A4KZoX4Xrl1mU7uzKMi0NIArQaEIpDSwNXgoCqjoud9mApeRst8nqeyI6CMvRaFUkFxKCgJhBgoClCBUoM6DMQregZCdRjUBIbqUKgOhRqtPmTkd0qgJkBY78OAQ8PwJehFrYst99ED2deQnYuhIKLHITsfxe9oae6S3IkSa+HDIZErvezyV67RBAltBxCo87HuweBJkzJoKD2vb1kpENF3A/gJAF+FfQ0/rqp/moj+QwB/GMDf90t/1OdWuHW3+5VCmAjR+jdXCuIKAOEw3OWJZrXg6qCDYTVogAF1P0E4EdnLBhDEPfhWScVhQKVZmW7Q0gwIfk6LQKUdQ6HoqOB8CYWb+x0YlIGIpWBmCJorBYVScxVhyZTCvvKzV35PXFCRy2hUfNqlpBqGmeCJQiFEuqYSRuWH53JlPyoLp+JAS/Iv0Byj0Nx9NHssRqaDwSu5JrJogoSG7MhQiNccKIEZCBhlcgAF30dsgjkXx/61to9RChXAH1PVnyWiLwP434noL/u5P6Wqf+K5NyK9r0sSyJV9VHKWUArhYBzXIOBRRjDRMB0cDiuBVrj0jtmRwzBFdyxOQOhe/Abi5hXfEukZWswxmMsNCnqlkh+XU9HLa9nKj65XhgHBE0VivQSCp5ZSJca2A8N8DAcCJhg0XMJhVgkYcIA5F621n/+Np2EPh0vfwoU5kXsdaCiE4WT0eqzpHRwE86zdAw46QeCpvc/k3TDDQAYURjkuyx0GGkpBR/5NQ0FV/x6Av+f5f0REfxM2tfsH3OwDIhoTECJYiAMInqBJTXRlQUD2JxSHgUMBa5gQOlIVC0ZypTD1/ZcICvIKX6tDorpSqGZKlApd7Hz0PhAnhRCVn/UYELwHg6Z7ZDBYXkl7pacEBosDGECIfQN1MEwgQPgPBhQ2YlTAK7vvU+UPGEznEF2RBgaT8UkpRDdAUgq5JHJqP/5V8yErhhzNKDsgCMzZqN5g9HsEHFIl7/lGu+Pd+cj38lHRDQpxjx0YvDzgEKZDLBEQ3ZEKm/79zUIhb0T0PQD+RQD/G4DfAeBHiOgPAPhrMDXxD2++Xu/vkiQmUB1+AhTqSgEaGgFJKdDwKXACQzgZV/cprG5KmJPCnYgErehKgToUFNrEo/8atFSgNAdBBePc8ygN2jwvbQcAvaz46dxeFUznMljSNcqAgD0U3FUCCMQLABpKAQyhAYQGGmAAoRLjTIwKxjngAFMKLaCQKnzAoE0wwMh3GKArhe4H2JkO47nXyWwI7bCHQVcMQ/X3e89AMFCQZgSN8OYcKiG7Cp8VQIdC258DEApAdpW+v+7GsThAuvlAvgaJKwXF24YCEX0JwH8H4N9V1V8loj8D4Mdg3/WPAfiTAP7Qwev6ug/fuX7HfUrBTQG4L4HYfmn25qGrBUpGxBNgQDcfCFgBsAB1OBnBADV0tUChEupQCagBBlcI2KBlA5ZqCqFW6LIZFKKFZx0tP+eKns7xEQwGPPavZxaIQ6EFFEC9J0XJ+maU2B4y8tBZYjSEv4CxgVwVEM7EOGNAYesVH0kBzECY8wMOw3yAIyFMgjAMkPZ524GARune5zfFKSRQ9B4I732YOhYU7sxLcNhVdvMDpLJuIsBBkMtHxd5X9HxO2nxdVxn9s7B/vuFonL+Xl90+CgpEtMKA8F+q6n8PAKr6y+n8nwXwl45eq6o/DuDHAeB7P/vn9G6fQlTs8A80AgknlZDNh2Fi9OuLQ6H3OiTzYQWUinUr1vFaUwtWIVEs5gBFbM8BhmqmQqlQnIGyAa1Cy2ZAqBWQuqvU+0o+g6ErAD/HfHzdAIdJ40bk40DYu26dbA4DpXBgGTwipr4RuVORsO2AcAbhDEIlmiq+7CAg+3zAAOEADCfggII/Gdi3g3OgUr5q55qcVMLoisxKIfdG5HsGHJrC7fdhSkgCg+zhkPbaoUA7KOxUQ/cZ4LLsIC9dJYSioal/5qW3j+l9IAD/OYC/qar/SSr/Lvc3AMDvBfDXn7yZ3h+n0Ft5exKt18GbBItAG+s97VVCgCHu0c2GlbtSyEBANx2sToEVaGY+oLULIKAaBBBKoRkctG0GCaleuXWu3AkQ5A7NqOi9svMOABevcYAQUIhQYC19OGaBApDbp2AoDefVAIJDAYTNwbC5+fCYoUCjwsuURx/RkIGR96NrMGkEygphrxHiARswAAYIRsWeHYy9E6G/7zgG0jUYUJAOAupAyPsOgzqgsAfEaP29hQ9npdIEi3xOj87pUAzq0ijA8FrbxyiF3wHg9wP4P4no//CyHwXww0T0fbDv+OcB/NGnbnRvnAIIqVKzQaERSHlAQQMIFnSQwTCgwF0phFrASqATHAg8wGANLOBQ0GYVE64WUH3YcfgN2gZ0pXAGlg1abQ/ZdkBIEHhm+VTWFcRc1lXC6Idxqtl3oylkVojMn+CpgyHUQkqPDoXuwKNU2dM+gCC7cy1V0Lkz8tK1OLajWMesEnTyJfReCDqABQ2lkHoaoWr8lzAfEiAkZH6CwHRcd2V1V8mjF0Mxq4RnlGelgKQYXmv7mN6H/wXHIVUftNbD/UqBeysfSgE7R2M8/IQEhvAlMNtrdz4FnNh9CgVo3rfvagQMa0qKgqoA7RIIphbcTMAGLGegnh0MlidXCkRPVHRWD7q6dT7OYb4n23qHnJJ9J246IEvRUAoZCKEQRspg2JAqXI8STPsJBJjnM8h7Vxm5hyH20dNwsYU6oJ35gBkER8ph+BXsVv1YbR9KoXUYjBRKoWUwVJqhkPbzQsM5v4OBhorI+QyIuM5h8NYdjS+xkd4/nwLVK0AIr5Km5WLJFUPvvmRbXTRMh8VMB4RP4QSgDZVgJgocCuhggIOBigMiw6FsIHIYlDPQHh0MjwaFXKkJlxX8Bhys4mMHCVzccwYCulkF0AQGgY/IcxiYzyyBQQMGSEphtLqaIDC6AvUQGlnmD6Uw+wewQ8RIxxC4KAtTZA8DmlNMrBLnG4Ca1MEeDE1mldB2KqFVz1cavoeAgX+4i0q/Px8qIZVJmA1R9pYdjS+2KVDuXUV3IXjkDKh5a56UArxVHKbDFSgUAi3cTQiDg9o1oRJq9HYAaApqDgRWUPOmo1VQqZNaIAp/wiMooNAeQbLNFZn2ldpHZ04VH5MKOAYKxr1Iu1KIYC4g/C1uPiiG+QADQdOAAUwxKGEjdKXwqBhQQFRKHSymeZTikPMpn66JLsPwFUwPRWxX2ovZwbgfD4EOoD0MJH32gEGohA4FdQgkGHQIJLXQ9w6EvN/DAB0C++MdNCIfqqDDIL6VXwdQ+JCIRoMBWYvuQIgnk2wmFGQwxKQI1hXJ3YyAA2FSC64UqPLotWhujse+2FNDzfwK1OwJoebOxmZKgdxsoHYG2vsBhQyCXrHxcuXwSaBAPrKbfJgwdxk6oACPsQeahhkBbAEEHSrhjAGF7uSj0TKHiTAqa2q5kY4xADH7Cex4/zTk+0+Oxn3aVX6OzzQl7UqhAyGS0mRCZDD0/A4CPb/N5RMAEHn7kJo+fHcgpj9C99dnUKTv6zW2NwEFUwp3QIHgKiHZ+81MhtGBnSbOsimXUsxBVgqez+bDatcYDLibD+SmAzUxtRBQKA3UmoOhGhhaBbtSMJVgaoHaIyDnXaueK/Xc2hPvy3YAuHgNTMGQxW3ElA9dLUQL5VBQULenAwxVySIW1YEwgQHePYkRJ7CDwHibXQTiQZnsKvj0UByUDxBlSMzKgHfHBge9NB9oD4RZLTQJMFCHQsvqoA1A7JNsGK27jg+v6Y/V+IOm43gNpfy47qmZ6F9iexNQuNun0KFAs0qQBIaYDWXMsZZMCB5AWIZagHdL4gRQ5blno9GkFDIUDAgCiAGB2mbJlQKF2dAeQe09oOcsXqxC9/y+stv5/CdcXn9Z1sjEDCu6P4GAoZuByXkVFSJay6r+FSthU+BMwNlNh0cAmwsx7Xt7Wvsx0nmk80jnMVr+fm63XTEohrrIwOn50RORlQJjVgwKb08w/vYaeQdCTSqhtRkMAYSalENNimEPgf7Zez5V/IM/UPM1mk6/MhjeBBSgH9D7sFqF1Ww+dKWwB4P1x3e1UHhWCcuAg64MWrX7E8aIygGDAQQ1IIg4DEIx2BPCtHUoUHsEiYNBzkBu+SdAjBQt/uH5zrw9QPwc5uEbDDPTLEYBo7vNYWA2NfWWMtJGoRjCrwA8qlWebqEB4aawCp7Kp2viGOM1UbHn85dP/d5cyPlutsB+Xtnn/XsJc2LvW+hqwf/mrhhkl3cg1DaDoSaFUB0IbcPcqu//UByA4OgPxhEM7qwrd25vAwqASfRnX4z+a5NHf5GE5Iqn0J4EjfiEqXaRTTxSCJoB0YdRj0jIHhbddvY7K3pYcvNxC2xDqAMMjKwcqvkS2hmUlUIA4BoYdteFirgsn69XIJkOI4HQbdWcRK2SZHu6gwGWzhj5bu0dgGH6nfa/20dsFzB4ImV1kFUCwSCgSKYDcGFCZBgc7hMg6gEkniX16SJzfPr6Jc/fnqkw3gYUCKgPd2qiE4CTWExByP4iXsFhe1JXCQqKPdhaMxU3O9I+/BQVXomtgnN3QeuoNd19HbrVQx61wD7AAmCB0gqiBpC41PDHMsvupAwOyym3vul7ulYerfUTX+n0PKYm+JXV6e7Nr28dAorktMM8JiHiB5qJO3EfjCQzKvb7t1Ux279ZPJl3Jw5T4ChIaf++ECQH4QEIiNL3PD4EzRkA1E+PDKwxS2U0/3fXtn3z/bOuexNQUFLUkzz/BYQRZHQSA8IiPRjJjGlyZ6G6no5mUpKZkYDgPRlUxcPahkdJxEwE7s0oZsO0my7sUZULgALFCoqIfxKALDRZxR4A7XM2IDXnUT6X5X0/P4np/m1Ou1tf4fQSGvkPbpBemia5kiUg5LwIwM2+fvGoU+ZhMoTZQF7/MhxEHAip9+AodLmPaZDxnhGDMMFq/z1Enfc3Jy+gLunmSm/F1K/rQOEoC3DQB/1I315QYGC7RykQgJNakNEKYBWfB8Gk/4hHcCDEZCnerFKAQagDIbo3NeISpIJ7U2EqQcT8CDxFwww/BilD4UpBlw6FAIKNUFQQ0xUghMZPaQcGm4DkSCwD89Poaubad36j8Nov8bHq9ekPcHxdNnNC5OXRi8Imwpj9eGda7S1TCq7LsP8ndbDf+/tM76u4AMMlo1NFzqH20x7TcV6AqJcxp/PpXndu33zmdW8DCh9iPjwI6ESjGzFClnNQEpNLchoEdpc4qQzZLwEFsUjJol0h8DQSxpSC9j48//Bq4y5Ui5kPWkA2vxuU1qESsmG/A0FfBCVgAcU0RTBS2R4itK/9Olr/OypfXH/rcXsxMXDPMx1/fqqE10yIva8mFEJOLW4rQyXk1LJCOFAK2WyYlILOv1Y2G8bYG8Y0cjf5rg4h4MF2xCMIr+dfaXs7UDjd+bidYErhBJtCbQVokQGGmANhB4Qe8agRHi3dWdkVQ1Ww2hMi0sA61AJUTbJ2nwJAwtCYaAEFBFtazZTCqMRKIVjcnHHlMICRKnx/nUzQsIfBz2d3Ggkm38I9QLjj+puP4ks8p3tzYZ/ugcEOCNN7+L0u4gsqoEk17Ec+zuHL43Ne/Ur2QOBR8S2fKvm+zPO82CQ5EY3bIfJK29uAAn+Yo7HDYNU+EzO5kzFP0zZJNf8Xk7FAMIZfN4JWMztEq8OgQVTMp6ADDH3yTqXR6xEmRHI2qnWLDFOB/Q/2im7QyADwZijBgEicJqEydgBwn7oCCRq3t4tHSg+z3/rt2rO+b5GTT4GyKXEAhen2yUJT70HYq4TsYMyDobLpILvjQxOiE4kGDHqF5w4GTvl57+eX5aKMmF/j2wfwVqBAep9PAaYS6KQjAnFVW2B2wZhrgTFLNxgUTO572G/Mw9AAqm6GVAW0QbSBdMBAk0aMQS0GheFkDCBQVwpJj1OMg2dXB3sIpJTKFBkeqe1LcIh+lWd/57e+23tepzdO5uMPbdhuqAXxr2CCQeR3b9kZmuHiKqNVXIQuXzgadz6FyXxIzs95609cMgUIVDiBgOfjYnsOCBRGWZbdOQ+se6XtTUABH2A+0EnNp3DSPk27LGJBRn0ylcjHTzOUwgACfIIWQBsZGFhBSSkorLkQKFjsqeoPA9z0QABBQLpAVWBKQW0uwHAogdx8iEoe3ZQjrxAQtaESkPSxf1+2ab8fORh6B9hHmA53AeO5lf0WMK5drvlgVEBlr5w076X1hrnHIwDoS8Pt3TOqAwjTkOe9WmhDldx0NMZnzF9PfCAaLX0GAJdiZVHhS1T+0svKaZmOY/+ynt+xvQkomPlww1V+sZErBZj5cFLQStPybwg4hMnQWw8HgiuFAEIPY65kzklXCqy2vhH70zAGpsQ9zHywngdTDNpNh3UAQQnK4dNwH0DvmYj4aVukhWALtRB532c0gaOPC73Xwr7BpCDivOLZYAivWHJHvPr2zDe5cORFZaQBB/GvJzi9Z+e+Qyc7LQMI8xwJO39CAOGgByJ/tuPvOzkQs+PQgWAQKN76lw6FfK6cVnAApDCYbf9moUBEPw/lZDKwAAAgAElEQVTgH8FDx1X1+4noKwD+GwDfA5t96Yduzeh8t6ORTCXQqn1Kdl4Usgwo9IFEoRZ2UJiBMKBADaAKczSqxxioLW3G6sN5OhTgI9zc0QgGaUndhku389UHaCkYRAswRlhBaeSJPAiKXB0EFBBgwKyFe5jizqX2EY6BF/MpvNBD21XDkU/Bf9doUno++Q72MIjUoeDpCBDTHIxHPRAyf7Yjn8I8T+hQCx0IDgBeSgcDezktBctp7fl87s1Cwbd/SVX/QTr+GoC/oqpfJ6Kv+fEfv/biD/EpsKsEPiloBWQFuENhBwNCWkya5tCACQwAVbIRdahgrQ4DewIUClGNmEiHg/sVQGbVa3Epr+ZTIFcSFF7GYqChigADdUBUNy047av9wUkExBOvlMGgbmqka+/5StO1dz1rL0aQg1vvfQDhywm/Ls3qIAZBxTZZWbnSZodhwECS+SCYTYcEhH3gUu6KnL4SHQrV3EhZKdDwDxTuQODFAbGUBAFXCgfnX2t7LfPhBwH8Ls//eQB/Fbeg8AG9D7Qq2GFAoRaKOxt5gMG7fKHd0QhfaxIXvj1uANjmHmRfFpXR3D+wn1UQHQZwBWBmg8KmNwGAZrDwCq7wJeHRYL0TGQ4+O2zfRzODtAfsaRtqZMzt+5FKIZkPd730pVsrvTycHHq0MyFwCYP8YlWMsdTRupcZCh0IslMI+/K9P2GnEvZwsC2bDsm5yKW3+Fbhl13FH8fLw2kGgp97re0loKAA/ieyiJv/zKdu/2qa0fmXYOtNTlte9+E3fuE77nc0rpZ4GYDQBZAFAwwetMhZLXjSAIOiR8NJn8YMAwgdDuqwyJOCJIXQUVGSWl0TEBgR8myVuCQANNujJhikCh71nIAezUiuSHwsBWUgxGue+Er16sErbHt4XHu/vcpJ+8mngASFXRL/XbV4nv11nF7v+fAVxAQqUgcEctTktD9QCxefN/4WdzTasn0ed1CSQ3ECg1X4kip+WZeuFMqygNelX/davp+XgMLvVNVvENE/DeAvE9H/nU+qqjowsCvv6z5893f8Vr3HfCDAlMFqEKBFwYsBggsgBX2eQ+aoTOj+NFYd8UHRA+gmPPtUYQzz/gsEDPHKF0uc9ZAhfzjdr+C6IaZNt7BkUxBE1otBMXCCIpYh1AG5mnHQEPXPG5+/mw2uEJRsBsYevHSnUqD9gWKPlpfb9p/nOW8SyuUaEGSnEPx6VoMBdFhhHIrBfzj1H28/OWsL8yGng4jGyZzZ+xJ2f+jlQkRhOgxfQqgDq/gDAGVZzKewLnZ+HbB4LafCR0NBVb/h+18hop8E8NsB/HKs/0BE3wXgV27eg+43H7ioOxgtyaIoHmVMRW3P6IOH2G1O9sAj9h+VRTsUbKy92kPlEGDvIlQPOY7FUKlL1/AZcD8mD11UwO18cTBE16NAvTLb63I+gwEJDNp9FRaz4E/7XlUA6ObGEx06/RvfXXvrl7ik++33+NhNh8vkUC1gnBp/QrqWg96hDjxxUgoiKZ96G/aBS0eBTNNnOoDDNIaBOAEhdUm6UihLMSAkAJR1QXk4DUisY/9a28euEPVFAOwLzH4RwL8M4D8C8FMA/iCAr/v+L96+kd5tPvCioGLKIPJY1FRCUev1o9hbpbLGwfZQgHwMAzVvIpu196IwGHhEodjFYBqLlphZ4tNvE7sKGGAYc+lF5KKiRyn6taAwK9wJ2ePZw2zYPfYhb1DQm7yYbWXfxnea3Pja99fu/JQ3X3O14CM3nbPhV41KR1ktxHWuDLpCyF+NKwIVB0GkcFLuKvwEBR35vS9hD4Suai6+HHd284FSYDZn4mQ6BBBWlNPSzQcDxNpBwaf1hb/4sX0sbr4K4Cd9cMYC4L9S1f+BiH4GwH9LRP8OgF8A8EO3bvIhjkZbaXk4F7kosNgqSmY+hOmgPiWXWfwM9R/TFAHEzksbFZDVXideidmhog4K6WUBAJiZQNbOq6qDgpMPwFUGRS8Bp0QJCMBFTetN5VAaw0OS7pHvk4Bwrd5efOP+mldu/G9vB74Hi/NIcj3iE+AiIPxAAYYwhZKpwDTvuz9iV/mlDdWQw5nzwrP7fcDhwoTonPbfJY95KDx3PwYY1qQQHATLaR1gOK0dGG/SfFDVvwPgXzgo/38B/O7n3kdY8c0vt6cvTJvNwG4LpLCvjrQsOsqm8553RyG72WB5i1LkBmhRX03aTQ0a+TwnLPfJUMKLiVQvtUvT/JpQLmNetJjH6Nre5mTofofJTdq/ab9Xu3K+X3W43ftIHd7npQmic14Bk/5e0YVHxSeJ3wldEeYp13p3NI98/8kCCgk2kmGQzZR8rDtA7M515YB4RCjFJZThPFwXlNPiFf5klf2UAPCw9nMPX/rC5fnT6bWY8JYiGu9UCpQqPykKqy/BriP1Sp0WJ9t3LodHqhuhOjXCrGSVWqNy06jwTMmT7eYCeZ5g7sCk7NUjqHQKOELf23OfJ/F2ZYAFw2xI3ZmTWvhIF+GHVu6XfjAPlELaDUCkJK4kOB+n1INCAwY03uaiwu8hgMsKfwGAI5UQfwx5rEIa9Zgdjbn3oUxKYR3mwmnF8nAaUPD8a42efhtQoPvHPkTLX5IsR5EOCKXLFL+cwgc3Rd9VAIEUpDZfglV4C02OSj8BIaqu2xEGh+RkVCAqqxL1sQ9KNrbC6YBcmTXl+iConmLylgyEvYI48C08d3sN02EvpT/ktbQ7jmzmql8zwQCXEDgaPbmv+BetPnZluDy/Vwj5bzbrYRe4NI11KOCF5+7IbD5kIDycUB5OWDz/+VYKdN/MSwZe9e5DQfE8sXRVUCYgSK/8XS1M7mw7T6SQAIXbfrHyr/poyACCtePkXgNK17I7HwmKAs1Pbtey/kRPD3yGlsw9DRojJW18xGxS8AVcxj3v+EKf6Kn46O1O2ug+Qymv6djNgJjK/QIKV/YZgrlyh3Dcl1/sNf1sR/vY8rgH2gcv7UKdAwzJmdjNhofTgMLDCcvDijfpU3ix7QPMhwJbM7kkCFA6VhKUgMB+JqMIj1Of50Ctmouft14LhpThULBh0daVGFAA4KogYBDnopvR+kVNKcRADDr4Lfca1NWBCtRDoLn7Fwr6JLH7LkkF+s1zJXpqi0r2jN6Hw9d+yHbtdUfl2YTIQMBQAQJMcrqrg1ROu7IcgThZk9hBwAt0V74vO/wbskTJPRAOA9o5GSMeYTIjup/BgfDO9q8kFN4GFPQDuiQV0uMI2YOKyPcFse5QVLB9XkCuFsj7lkbbb8unU7FxDFq4tyA5cjH8BRZuywMKfQYmhtKK3jdGbNeRqQ3AWZWhlYPze/JBU2ggjWCnG3A4aLGe/fC8tAlBV/JHx0flqeJDEwtSxZ1AgFkvZf5O5RhQiNfF3yxekMumvB6X78+NPyfG34xQZ04DoubAJQtamnwK7owMdbC8O2F9OGF593DlC/z47Y1AAZDy/N4HaxlM9k+VWRsQkjtV+F7RegB92qcyjWtIga4zosVX9CEG7nncj8KLoCMBO0AYSmXsOQZlwP+CUemhPihKY3p4W/d5+BEKtOcHeHKecrnZOfb95s+YUj/GKIvFY6ISZezEGruRP3Rl5BfGjRmXN5zy2vM5tgDpZdjtj8rCInvO9RNk/CNYh3IqS/lJhR1tQzr2IqKh7izM2fdchgnBNhS6p8IopaAURlkKlqWgrAVLpKVgOVn+c+1TYFGczuf7XqMKJomqB/ZwZHZAUFIIGiMcAwY0yizsaNyneZASk6BQARO7z6KgkEKoWOAyFQjBz8GdmzZnbJj4HDMlMcFmWiKPfQDITRerjOag5FjVyvsxRW2+R9IC0gWkzVKM3NzN8iGpwgtKarmGgggIsA7PhMd92UoVCqwgmz1fbbb7E43FUoRMNZX+vOvYL7BAsAW2PsYCYFV74YqxFFVfN0OBB/UvDL3pJtULU2bfyr/EOQDzMu8AmsNUNFbSet5+/A35eAVwim8T1JN9w4zFU0EBg+3X7r9JcdW7aEPRikUZK8h/I7ygpJu3NwEFUsH6+HwoWJ2zihsWPJFDwX0L1GX5cD5KAEH9iPJr3RtAIxUHQ/O8RJ4HEISKg2AENBUGQDSg4Dp2zOALDJWC3opLnybOJm0hMV8GaQGrTRsfw/diqjhSHUkcNgozfcL3Qfl765oiPXy+eh6ABWR1GQ4EOBAIfSiXsDHAgKAgDxzzGTU8rwkIbn+J9ijS7tI/Rbyy9j0Bfcrli0q++1tCHeADjgGyyq++ArcSmnB3LI9zKY+jcvQ84HN2gvzbtDH+AwRjz1hBalDg/j91ICyqKKpY0LCiTr/NojGJ78tvbwMKojjdAQUA1srCZkGOxNRMJdhECYjBQ/DoRBtEpB6u7CaHd0MSpXthAKF4t2dzxVA8olEmIDgMehwCPMipdCiYTRlqwQ2NkOywh4qUwH0tCfa9jfASLSBxxSBiQPBKRh6A1WP9lSC6WGMbIiHABDcDyFskylAgW2cXsXYvDXUA/7tgvpSagdCTDCDEmpsdCNLBAIcYRI06k0fPgD8pBTqo4HRwPpc/43wf7xAVXAhNCkQIogyRqPiRZwN3ymNXDg9bNzBYFSavzhR5XcG6TECIfwU0vk4IFggWNBTXG4saENaA5ytsbwMKer/5gAgXpoCAgrh1hdDXWOizImvf9+nS/UkMwCDaVlIsJA4GcRiUw6XMj1JxEAgkdUdF8tWdUsC8uAkBsQeThFNy80EKSBZbhyJXMPdH2qzG8XqCaOth2NGs9hh8t2gKUSyRgYWoL7xdCbGMja3dC0A81kL8ntUJRIsirdnuMPDj/jnFVYLsACGgh/geok/ULXvz7B5D4QASz7427Q0K5Mkqdq3F8h0MjJaO43eRWOGc7BoScyJqAB2c/EO2NyAsYLKJfXlKjKJkCWE6mPlQtGJVdLNhhWDV1+tDfjNQuMd88BdZszh5yNL6CWllVeUBhH4ue94mL5yACGikWFwZFM+HQsgBUXY8gNB1LREaD/MhpvhmYlMnNEUwWAq/ghBUGCwMNAakALIATSCymDporhIaXDEYFKhFS9asK7Q79aw2cKFYUc/UAseC2waFhYCVacAANCDII0KzIlSCjLSIAyFV/iYOgzGyiGJyRRDwGA5h8t/BwNmVQnw5kd/vUyU/Kuv7LhUSFITQOhTMdOCtOAS471kY0ubygEOOGVHh7lCMMlMIZU5aXCGUYToou1kXV8FMBzWl4CuT2rEuWF8xsORtQOEDzAdlq9waQCCFepgz2KISe7m3anFtnFOPUehmBmLAkpkPAQYDAjoc1I9jTAO8FYXnLTU0GkpBiC04yssmICm81sFbHW+NWrEVrSPJYjDIQOjJTQ9fw0JUUkcEuWffZqgihwEzobD5QIoSFjY4rNGYF+pgCBjEvSpZWPkAQjMgLAEFP3YY0G7mEgrH3AOnSh0qD72nZw+AqxX9A64TjZWiTSm0Zt2ErRU0KSiN0aSgNUbjAhI7BymgZj1J0TulKOb/oey+Hfte3Wm4E2cghF8BAwjhaERzQKipBDSsKL8OHI13mQ/eUpehBJTVK4IOMOiY/0DVy+HX6+58+gdCgoGlhTCpgg4FoIcxh0KIlllI0DzeXbo/gZ0bB0rBpb+1+JxSAVUBd5muozugwj5MJUh6jToUtNDoQyz2HjZAzIFQzIZdlLyTwEFQyMz//d/m91qgoNIcCs3yi1f8JqC0CCN1GJCpAm2WB8zRiHCauao6gELPH1X4vWLI5UjqYXetiAFhJAa4oLSC1gpqK+BmXYXExSBNBWijq1HVEmuBxAw/sIlCqasFGvmuCPx/peRPMBMieoYKxHuGGtbuX2CsSliUpxiNl9zeCBQUp8fHu14jMeipjDxgUJCAApvHRgMEvSPSZb+DQXQMoBLvs5OuFOBggKsEuOmAUUkoP6XNH2zpSkFIzLnIAQdTDX3xlkEXJw73VbBRi0GgCqQpqMKUQkWfaJZivYo6oGCAJA9xIG+wvGdD3IzwAV+l+GJ3ZJZvI3O+dV8C3DfRAxXI/rbSAIcBxQyopYGWMespiTlMyUEAbV5X/f9TxJYoorZeQOGgUufjq+B44rXSgBpAqIzqUKitgOsCbgW1Lg7lgupTe+lYigzs7bnqYmCgAputuwAOg5gqb//PgHCZUlSKmw+1uyoXNW/ECvq8KwW9UykYCKQA2tTz3s1YFJSAEPI8gCBkEJAAAnbJK30AYfgP9g5FmvwISkhLgzUQUYcCk6AR+8Qt7IOz/LlXdH8CaVIKkpRCLaDqIPDElfriNVwJqAytDK5seRVoIevFjAV3i5sOYqnDAAaD7kOIgV/e3TZ6UGBgKUATc+yi1ASHBpKagFARIeIxjjzW8iStABT0wIgYjul9nBHPrejXgAG+fp00Qq1kYKiEUk3ucyuopYDqAuLF9hSTfy7QsPB1QXEgiE3/5UBY/FsdtOq/cf9Yo1u4g6A7GYcJUWDdkkWBVckdjWbi9du/8PbBUCCi3wZb2yG23wrgPwDwGwD8YQB/38t/VFV/+ta9+AN8Cq1oB0MrZmMLuYOrAFp8PAN2ld9b/iYGiJbOmXNNu3kgGP6DeF0GxDAXTCHMS4oLhBrEHY52HE5Gn8XJ1UKM2B5v5N2RjU2qNjXFUNXMhQ2QakDARpDKphK2AqnWqomoASG8hwW23iYReCGPXExOLRAKEdYYEapJIUQLHj0XTGikoGLrrBkUKmip7hStrgzYKr+ywwCICmImA0CnCAN3MERU4y0o8PPOXZzfnZMG1ArUyqiVUTYLYeO6gNlS5QXEq1V2CnffioCCqF1HvIJpgVCAYcyMNOb8tnwEcUae4cF42AMhYkck3tG7JIG1x3W8/PbBUFDVnwPwfQBA5l35BoCfBPBvA/hTqvonnnuve4OXAICLoi3qXXFWyRuZOpDoB/dvTcMH4eaBKXP1EQWWLODY7hEACB9C7n40p+Jo1cJ0oAAD804p2DyPEokFDButQf6+pClwafIphEIophA2mJmwAbS5ybAxaKu2rw20FdBWDIQLAQGG1cwGYvZeDoeCd0suZB54m4HIhon3poii58TAwExokASFCizWbYqlmsmg1frtuzrI7ebo7aGHmNiUQSSI9RHMAYyLlv5Djq8ph9aArRLKZokXtniQUkB9JuDVYbBCaUWsJq66gmVB0RUiDgT2KcYxoND/Vh1D5Qhz3oCgPshPExAieEl6MJO/uwcv3VVlnr29lPnwuwH8bVX9hQ+JsvqQOIVaFCymEtxTZqHGal+yddi4Y7G3/mowUJO/lU0pmL9Or0Jh3+WYux6HXetTeKMlk2EHhABTj40YYf4jrILAwtDoZWjFzQUAFdAAwsbgs5kKODN0a6CtgbcGbMXMo4WgK4345GZL64UtW0BoDgUhwhJzSHiLHn8ogUbAE9s9RAMK29gvNjYjIjGhbk8rdX3glhN6PMlm3XjWnYxjpbAfO3GtbK8U+EYZmZOxbIRtIZSFwRtDxKBq4xPMfIjlzQMG6iAQOYFlBfPqasJSXJ/H5VDea4TkK1jFgWBlAYQe4gyPaFRxxSC9F+LNKYXd9vsA/Nfp+EeI6A8A+GsA/titJeMAa+nvVgqLojZ1RWckZh5ACEJbz1cyA1xRVAaaKirUVbmihooIdcAJCMAkPSelEDPqeDwCezxCQKHP/eBwYBCiOzKUwpAkbA5F4dHL4KshGwwIOIuZDWe2SnUuoK0Bm0DODbQJVAS6mt8BK3XThBaPnIR9zsLhGyHzPSQCdiQQDSCwOSgFCpLN1cFmIzhls8+9BBRG2Db6bwJELAkFFHy+tLG0GuwL3wNgt6ejsmvXHpS3CpQNKGcDQ3RHEpuzcJgBNnZBOxhOEDUoiKzgdjIw0MnA4KHN3teMCBC31MC+pz5ep5npoEhqIUwIwaLVgdCwqPRuydfaXmItyROAfwPAv+9FfwbAj8G+kR8D8CcB/KGD1/XFYL7z3cN9PgUCavPwLg+bVQW4SLfbCOjNrxKgot5lbr6IpkBVRVXFBoeC7xuNHoZeYXjkh0PMExIU3KHI1CA8mw2hFiRaDUqMUSTzgT32IMUiVHLzgECbgM4MOou1audmYDhLT6oKbT4jVJLxtLJDyyp48b9NxAFSaL6+KwXygV8GEiHxKEvvt5etw4A8YmpgJUyHNETdTSuq3E2GvlJ4wVAKuVIfVfijSn8LEOlcrQCfXT35lOu1+RThNJyKAQPRkyVZ0eQELidwO4F4JFv52CBif287SNU7IVsHgs0CqLveB0XpwUvNuyYbVrX0WttLKIV/FcDPquovA0DsAYCI/iyAv3T0orwYzPd++Ut6b+8DLR71tka3ovkZWMcch+S1TUnNd+dgaJqgAAPBBsVGwEYBhUuzIXwHtAcDBwxMIVSHQ4P4aMsAg4VYa1IKU1dk93BGKhZzHGkjYJOUFHgU6LmAHwU4C9ShAIFV8l1MNp24/w02h0xMOWdAwBUgFFcVhQmleAg3m/0NKQMG6XUIk8hVG2GEpZOvwsOVEbNcUcRSbBhK4RoUjo6fcR1x9BQBtbrTtcSK5Yy6MTA5FYcPIUwGkRNKO6GVE7g8gPnkKuHB9jjBoCAwm89AYKnBIhMCDNELkcOaY+Sq+xO0uVqoWLR5F2X15/zlt5eAwg8jmQ6xCIwf/l4Af/2pG1icwp1Q8AFA6ipBoODFPbuUgQCLQWjwACfzJ5gyV2wKAwJsf3bzIauEC3MhORd5UgjmZGTipBQ0+RNCKeQwZx1KQcLZuFMKNcUhbALa1BTB49jLo/Q8HsVnmfOBOYlsdB5KQb2Sm6LYASGF3bYMBvc9CLU+LsPiEPxz9y5HDP8B7caZkABUQCzmUPV5C627080H4LhyZz/BNTjsyunKtbVGvAb1ORO3zVVCd/W5H0FcKcgJ0k4oywNKe0DjAEMA4QGgBwAPMNtvA5E1PYQNpNVREGacNWR9JjHvEVoSJEwpVAfDhhUVq1YcLLz2IttLLAbzewD80VT8HxPR98FY//O7c4cb3xm8pATE1IoqVtEXNSgQYMOno3X32ZmVXSX49U3dlxBQIOAMxRnWKJsZPEfz0U4dGAS4w6E6GBq35FNoPsJShvOye4h07MJxYZLGxj40MwGoKXgj6KYGhrOO9Ohw2CUzl8ackdGC8wND3TegxVLxQKno+WAfg1Hd99AcBs2BUJkBbjuFwFM/PPrXtgMC2zyTRMUGsLWYs5B6DEQ3H44qeYDB93oNEs9IdXMroRiMQIzTOYS7qQTRSAaE1k4ozYDAxRJxSvQOBgSDAuFsPTHYMMKTqE8anucDGUOntfsXLIoxAUE9YXubjkZV/ccAftOu7Pffex+S+4OX+lJe7qwTKLheB4IWdJ9CkwQFoPsVzgScSVGTuRCKIKuEPgln76ojVweM1uFgvQ9l6n2w5ecElAVHUgpIPgX1CEZ1RyNbfkvprLZMxKP2pO8V9B7eC3bxh7hn3aDQh0gWstY+hVlnldDAHm9haSlkH7aEDyE7E7NKGN1vYS5Yd60YEHw8wWQ6RAUFPqiiH6ZyXF43jDkTfcq8c4aCLu5QtNTaisXBwPUBpTyA+R2YH8D0DkQPQOzxDvCZswYQKO3RVUJ3NmoOYgpTQpPJsHk6Y9XzoO8Lb28kolHuMh8UGBNtYkxqxpv6Qh9qLUkDEEDwxV5EImrY/QpwswG+FAsptgtTIcNgTgGDfWoe0iyc518wJ6fqUAqjO5ISGBwO3XxI0YybuvkAVwppH0B4r910iB5xwD/z2eYH1OJ+CgtSsNDopBRECazcuy2FbNyGOBhs9RyaoBBuku5EpTG3RYcBNRA1wKckY/HeB4dBVwrA1cp8q6Lfc27baCzj5lB4fM99PIPKApEF0gIKrhSWB5TlAVzfgYtBgfgdiN65UrBkPoQFwCMAHr0+2CuEAtYYATGiGrtPAQ3FTYYVG1Y9Y8UZ82yxL7e9ESh8gFJAjA3SvjwKVx0TJrPllWH+hASEeN1kPiCUgvnz0M3wIyj4D3gBg+KjKyM1NCpzLwTZ5G9DJYSzEcOEsGmOLgY+WYyCp4DCGQ6EOVk39gCCdZ16F6Z722OsNPlgIPaRluKThhR4RxqZUhAmCBsYwC0phUmPIEJVKPcykJgPgRvA1QKyuBgUehi2p3gqjyr5vsLvZ7t/4nw+Llv6wGQAPb2zCW262SALmqxosqLWE8riyU0H5gcwf5ag8BnQwRBKobtekx/JuiijB8IGUaf5FHT4FMKHsGDDomcsOGPRx/49v/T2NqBwZ5izwoe9wuY9aN7o8Qaww4CiMnnlmlcSjt6HHDmsY9G2K92O3M2GOpkMTNb6FmqoxCjEaFTQPNS5UZmCl2Ik5uh1oz5f7KwUMI13sLBmzCvMPRLwCOijwYHfA/pNcijw6EJlN3e2Ag51UBkUYyh84hDOMw6Bx+PrU9ULk01AW6opBZ/TtptA3WSQDoWuFLiBqHpQUAW1YvNHVPYpzw0IVP2H3o9C3uX1ifOR7yuQH0KBulNCwTh9s5g6UNu3tmJpbjrUFa2eUMsDuIPhHYjfdTCAPgPhMwCf+Q/EOyhYtC1DwN4rwT50urgyi48ZXZJFGwqGCbHClELEfbz09jagcO8kK+SdPWRxZhVuGlfEbOro409j1G4bUBi9D/Beh2E6mFoYPQ6hDEaAUk0wGHmbuq2hUEElQaHWR1raEGpNSkE6aKAYC1XFvGcSSoFskufofejRjACdydUCmVJ4JOA9IO8J9N56EmzQFkH984M9am9hG1BVGdQMCuzmg6Y5ClVjGlwPyHUg2ACrlvwIyT+Suhy774AaiFb3IyygWkFcgMZgdSBUVwltpxR2Ff8CBAd7feb5cr4clXB6VyDiYGgLWjMwtHbCsp1QL5TCO3BSCaYUvgDQZ7CBUdy/o6HdIoipIiZuHd2T1J2MEeLc52h0J+OiZ6x4/PybD/eEOSvGVAINNqS5MMzRyOixJ1y8JU8AACAASURBVH2G9BjaL7MJMRyNo+ENE+LIqcgpFsG66NhBwKi+j+OhFNzBiBTq7L4EW9XIW5Bkl4/xD5YuYxUAPXOCQkrvCfRNg0IoBI86Mol+tmXKbDwEQ10paOM+P2Rf3CaAYPaYxzRY7wVxG7PVI6pVMhV6vg2VwIuHDxcL325mS1NLQPBgLQBjdNC+sh+B4ggCR4BI+bKgQ0HBEGWcHnxiFRlAqHXBUlcs64qyuU+hPKCUdx0MxJ85FL7gSuELsOpFXR3MUY0RxLRhzObsJgSSozG6JMN8cJWw6uPn3NEo9zsaG7yeMPrsQbzZgrPdWeVAUFcLMTXgZDrorBYe4UoBGQgZCq4Q2GIROMUlFGIsbi60ybcgaLwf+xBTBlDyKcC7JF3SN/KBUaEUyAdAkYc7k5sPDDwS9D2B3rOBQVzmx9xrHpyDrQAbgVYGqsHApnwjn1/QgBCGjQVaGQjCYaPMoFJnP0I3GXYqoacwGxZQ8TkKmreP8TdGipnG9lA4qNwdCLeuu3KOl/ja/dOrKQWbbamgtQW1ORDqirKesKwnlO0EXqJLMpyNAwygL8CgEB7TEbw1RzVu4N1szsOnkKdfM9MhOxoXnH3Q38tvbwMKgM/Td8dr3P5GmqYR+5Q3RV4iIYrm+piPk/2f141IE7ddzNoUzsmpLH+Q5E/AdGbybLp68IqpUVHd04/s8efRA5DmYYAf94qehmSrlDQ8e38Nj4EfzfLUJ3wZr+kTp/j8hRZ0xdPnivkmc2Rktq17Bx3R8OUy+tSafpF9JbsehJgSLld6vQWBXkZTmTTGUnIqV9JymEpZsPCKwitKWUeePV8UhSsKLyORTdza52f0PGlB7whO805YlCuSGaxQnyT3lcIU3gYUhAjn0+mu12wnS/WkqCvQVkAW+BwCluJhCD8De3IlDR9Z3Ae62hoHCijhpCtOunjyPBasWjzlfMGiBYsyirh/QZ3/Gon63ir2qBom83PLTj1+QBcCudSn1Z1yK4FOZk6QV2LywVQUkYtC0M/InOAPgJ5gIeGL2L1Zx1wJKhbTEJDIpsrZPoO9hsZrSgPLN0HtPVgeQe09qD2C5QxqjyA5g9sGahtYNlCroFbBUkHNpmjj1sCbTTNHoh6s5Y5WwCsw7Sr/fByzSumu4o8ymqCQISHbAt0WoC7QutjcFWKxE/G7lSTpbcZrm8tyZZ/qYlG0VSGbQk8KraPSrovg4SR4WBWnFVgXYCnAUixUnGOYeIrE0nBwuvlS64pzXcF8AnMFk4VK99bwru3XnnXVm4CCMmF7WJ++MK4HsD0A9WSpBRQ6GLRDgZI5EUCIOrfApzSH5o4KkNIEg7WnAYPlIJUMhw6IBATxQKcsvL2l7M2lf7iYLQkx/HkjhwL5kk0BhNHi99YFXsG9Z0wfAJwUegK0GBRiwlnrioOpEYE7N8m7ZMZ0bhHurW7uUGkgeQ9u733/6HAIMJzBcga3M0iqAUIqOANBBGUTGyYuVpnYJ6QFIVVoSi0/TTCw7wpz5ffr+7XL7tivkfOawFCAaiYNSQFLSWDIId4+23UJKPjztiq0CvQkUJ9odymCd6vidFKsqxoUHAwxjoSJu0JQHd2hzXs+aqvYtpOtacLmRVON6Jx7t28nKNypFJRCKZhKCDDIMsDQDbOkFvZKoZB2MAQQYg2Gh0kpLDs4DEAsCQQGBgPCoumh6kphmADzSCtOaoG6WugtdKGxIMNOyk+DnZC86Q4F9YjbUAqyqL+PvW0Po/KeD22ATkBAnyY+gKAC8ynII9iVQkBhgGADu0pg2cChEqR63sBgc0+KLWbjXcc8+RRy5b8EwiUU9mX+4+7LCqDbAt1WaFcKSwcChcJzoR/rYjR2MBSfZMobIINCBMHY7NVLETysgtOqOC3AuihWVwmFCczcfVbe3+BKYfgzSltwrqt15dLJVB0Eqh+iFJ63vRko3K0UTnuloJD1yHzQDgTirMxtDsZGvswhhk+BFEkpXKqFZTIZklIQxkKMkuBQHAYsHvTUJxyZ1UKfGNV7DKwC8FAK0QsRLXkaTWljG4ZzEOQ9CgGEB3UoBDClDwUHJV+LqKsFA4NumoAA98vYNcTNYeBQ2INBzihtA8mGkqBgaSgFVDGzQTzFepMUvx9SRd9X/FHBkQF6AYDL16IQZFsdDGZCmFIo3VfCyn2AUpgPjUwpCBN81LgrU1MLaDJB4bRqT5NSiIFYKZoSKNDoDg2lUE0pxAyaqpZEPufrPigzzg/3+hQUm6uE4VPQ4Vcotqeik/kQaUkTqsT6pzHikpJPYc1qQRastLhCWLBIwUqXJsQAQvYpJECA+rgEDZ9CSPmd+aCFgNyF2B2GCQgYIUMdMI3MxvVRvHpS6AqIz1RlUIihWdoru3rXjI1x0u44HUBwu59rB0HJIJBhNlh+QwswuPlQpBkYJE0II+IxGpaI9pV8Pg5/S4dCO7h2uf5aFILUxcyHukCr2QMkY+aoaep19yc0Nx1ifZ4AQiyPF0v6sQoKh9mQoBA+BQ4naziVY7p4C5zqPgU2n4KiQbVBVCzOpoXj5eW3NwEFYcY//vIX73rNeVWcV6SkqJ+Fb8EqQEyaQ6uCndJS4DM9IxppMLQ7G1drvHCiFSutOGHBiVY/XnCSdRyncydacIK/xl+3lgUrOThQPI7BHrIeB+DBRRqxBIstTKIxwhF+PmIECkEdEnpi6CNBHxj6jqCPfnxmSAOwKGQVV07WkslDczNCoMV7UFSssldfjFcV2sScZmcZNvMifU/UUOTcgVD07McBg8fp+KiMZYPU6pAQFBEUsaAdIsyt/ULd7gtQjgrO/fwAAid47K71e9T2AHl8Bzk/QM/vgMcH0GYzKS1t7ZGN0apE1GVZCMtqYxNOqjiR4lwE70rDeWk4rw3byZyCy9KwFEEp6kAAlsUGlS2lYOExw5NqQxOxgW8KqNoKVqpnLMuCpayeTljK6fMNBWXC+Q7zAQC2BaYUVu99WNyE6OaDAkv8nmoBTd2noP05i/kbo8sSULA7GtedL+F0YUJEnrFIQaHkZERSCmItTTgbbWJSIIJm4PEQ4sOZkYY160IQn+9AZV7xWGnM/DdCkAlSLBJSF+2KydST+RSkCJRtxKZS2KgKFbPvexm0t+LaHA5VzFlJ1Sv3dgGEUW4KIdIiFSINog0iDYs01GpAWtScjbE+JlM2AWmnBLybtHcj2bEWHjBoPIDQZnBEuSalgFqgtbhPgbtPoXiYUSgFdX9M+FtoAWI1bY7FftWCjpgFhcW7JmH7MhyN0ftA7myED8QKpYDWoFjB9WSh+UVR+71ejQlvBApE2O50NNZFzQxcvVtocfPB/Qq2rqcpAioAsQ7zgQwK5p8bsxsACp8QGquuDoL1Agi5O3L2K7D3QDCKFLNHxaMgozvSJy8Rj0OIYdma9jETkhaCFFMFoR7ERzDmITUGBFMbARYbFRr38JWziqItAi2tQ0FignttvfJLt12bHTcDgVaBcoMUUwrLBIXNjnXrUFjkDNENIhtEKlQqitpexaLKuHpUmQ4gkLiLpIZCGPmuBBqPit54AKEDwFWDpOg2cTWxWJyFtKX7FKJLcu9TCDDI7rfp/o4FBgRNox59yDyRTSZc2OcPZYdBMcgw8/ApRAyjLrawjnu5VAWEk92jeeQue9Tuy1XBaXsWFIjozwH41wH8iqr+8172Fdi6D98Dm0zlh1T1H5JN5/ynAfxrsD6Qf0tVf/bW/YXuVwp1AbYAw2JgMCC4Wki9D+bHUV8uzZRCBkLY1REEZSuoOxCwh8ExEErvktz7E2jqfegBPRFPEL6AHp8wnIy60AQC0Zjq00wKiSHN3fHlKqGQ1TN79u1vLbBh3IvNF6ksEPYJ7lUgYhPdKzWHQoO25teOfeSJKpZQAK4IWoeDqQTxvSmEDarV4KDNoCCC4qtTk9vKULU4BfKns2L4Bro64KEcJiBwBwKag6El9eCAiDJp6wBC9f5Fn3eSvPcoRooOfw16zw0tcGUQms9HNpKtWg6Iw8Am7u1DUOhAJWAM2RZx2qhAVACsqKTIAarmi3id7blK4b8A8J8C+IlU9jUAf0VVv05EX/PjPw6bs/F7Pf0AbCLXH7h1c+U7lQIMAhkIrWRHo3a53Lski5N6mpB1RBeOzjyFKA0AZOeilNnRuFMJoRSKuP9g6o5MQUxkH0DC/Z9jFKJrsvhqyEva64ieF0rzHExQYLu2AdPy8T6vQwuzgRuEmkPAnVhikBDx8jjP6dhfx6ioYoN0moOh+bEkKCxa+14TEFRtFaniK1STKwUyRpgzvsLB4E7FrhzostI3Dl1ujljfY7fvAAkvf3c0LnOcwtQleakSSAwItoh5TKkmsFXKBZV9va2Ytdp/XiB+5jQCl0Y4pqpP7N7EJ8cViJyyLzqN3H2d7VlQUNX/mYi+Z1f8gwB+l+f/PIC/CoPCDwL4CbWZRP5XIvoNu3kbL+//AUqhLbb2Q97Lqr2LqEs7731gtslbu4wj9H7eAAI79cWVQqQl8pi7JK+bEEe9D9T9CXkgLWH6tYfZIMmPsKahNLQDAhFaGSqhFQNIaxZX4ItZ94FYtQ/OcgCgQWCVVdCgrdqeYm/n7fpRRupA0IomFc0VQfOyRTaIK4NFTSUYFGoHAlSwNFMJ1IFg0+VZPfFelIIea4BG3WToKqG5kd1BUSa1gDgWhpbSISGyQOsKbaEURu/D6JJkG2UaU0P7FPRU4AvqXCqEyoJWRiyBVV6fcMa3vqAuGLaWUoPqAAOw+PUKkvXidTH68jW2j/EpfDVV9F8C8FXP/2YAfzdd94tedh0KTNju7JKsxQBQi6mDVsJsCOdaqAQd5oN3s5X4PXwcQkyiWmBQUMUAgZaeX1w1LE9ENXYg9IjG4WDMXZKEAYnelKSZlXXnP+jTogUc2CDQNjYgLIy2EKQGFLyHz7v+VRXVg18EAtFR0UWtwopWg0AHhU1+b+V+HapDwZPYYJ3mefH7GAgcCAkKFgRhYKgi3uL6SEIf08IMU9BhPnRzgWytzABDSZW+GRh02ZW5MtBQE2IQUfXApVYcDDZQi6VAHObqMQoGBFcIPsFszLNoPaLmO2isWFpAwR4m7REw4cwOTcqAmifC7iLDdMijbPTk16MHl+gbhULfVFXpzqll87oPX/niZzif7lQKxfrcYy/FlYJPvzZMBx09EA4FuEqI7shw8zTfC6jDYMGyy4fZkPMBBL6IbDSfwmw+zEODkkroYPCuR2Wf7CQS0IjQmL2/nA0KxRZIlUpoi62g3JoOp6SORXVrU+/vHr0AHQa69QotEpBIZV6xRTeQTzu+ulIQbWj9upaudZNBAwihEiw1yUrBF/NRj7rMQAjToXlAlw/Y6oogq4Re+cOcsKCgyEe5qM2ZoLJ499WsFKLS9ohT8vkk1VVC8efGVx5b2Myz1ixFoJGqV+8+KI/67xqBZ9rTYjjQSIDoOhqIfl25q77cs30MFH45zAIi+i4Av+Ll3wDw3em63+Jl05bXffhnvvMrek/wEsGUgfLYS+9u88CbYhinHtHoWKeYsm0sD8cABDaDrvgP0X0GsmCh0n0KC5ULf8IqBQvxhVLYOxzDyciwH7j/cxsxVIKGSijJuUhpjsTqQTTVgFCLw8BXT27NlIIIpdmm4KtjCZp3M4qYf6ADwR2CY79Zxd6ViZhSWB0EqwOmaXQ5GhgGDNyHEFDQBvLx7M2VAqvvxSqcMhwCmEwHbaO7McMArQyzQWJggvX/2chQNzECCkuxyuZA0HiNOxpZiw8hd7MBNkZFOH5D910SbIZwVoibDdIUIhZ1KL4mScw6LkK2VxttKj6PQzdCVCEq3YxSBWo7+esMCiK25uVrbR8DhZ8C8AcBfN33fzGV/wgR/QWYg/H/u+VPAMJ8uE8piLf6EYwkbKPVzOOuPgDGQcDa52xkVwrEQymYuIsVqgEVjMqP0pWB5QMWAQgPTLqAQZn9CdmvwJaXaIF6S+RgUAeDezsk+RGaOxVbcxg0A0JtXtYSFCIJIE3RmhkL1u3Y/n/23jfktm+77/qOMdf+3aRNJWhDubYJttoI5k0koC/EKtYXKkhQJNoXamIQAy0iFLSpBaWlUNFW+qqgBKJQ0wYitUhAoyD0hbGYVvBPLCa1xZQYtWmaYvI7e805hi/G37n2fs45z7n33Ht6r/uwzpprrrWfZz977/lZ4//AcvVAQj2QE7LO2vftMkc6sbxTkbywadvwwjY1IgBLSljq6kMUzfBakrq61FASQhobhTcYBABywbc5yMgajKqHJ84cbmisOAVEOnPacExC6J3ID9HsPCbagCAGPdtHiwEDwxKXFMRdkjpM1VCtpkUO8/tcDnjGEoPIkiM0ki/7431dkj8CMyr+eiL6OQD/FgwGP0pE3w/gLwP4Hr/8x2HuyJ+BuSS/710//9VxCvAQXI69w+DQ7TiUPjKLUKoP1iXI3ES7a1KzwlVmPcrAQYfnNhQIRpcWPF06xkekToc7MmMVDA4klT5t9gR4tA4/hC2Hfzw8DGuwL3r7Ys1lx1MY0/cBhbUI4n0oZQEyFVPMtBiqQ0kKrgbEwp93359Y6w6ZJ6TtaS0sBACkwUAyJFejDl7sW2080mVuyAaEMPSywmAe8SarSwm7oXEDwxUGkcb4AARzPQpMdVA9TBzv7sioS+E2nSy0w2QiPZCdx1RCwvQAMHclyjIYzOVtBZaClzEt7Qpq7mVyScGg4ikU/hmfU7CWVYSaK0Kg53VZfNke7+t9+B0vnPrtT65VAL/zNS/iQ+IU1FWBsBFEcA46EKhBgQoGCHck9th/K6Vg+4QCjkyJHrobFF/a76rDblMgtZ6Mpj64/yFKULO5KrNbArn3gdwtybZfg/xL4mDo27L9WsCagJyWuLOmuSTP5eKpxynImr64TVJY0hb/vGOddwPEvEPOE8vHvGYaK3MRRHyD69IBgT6GSwaEBVbBFG+y6kAYMLEZISl4ckoCIdSHgzwHJMAwGiRCMnC1oIEg9ioDFlpsTV8yYcYzJAH7HKN4LaGAYAu52QhGtwFo/v1TTDob03qfThcGsXYgWLmUgcgvEc+ris/zPAVzMeZamGvk/mM9Po2IRqbXJUQpatFTG48CQUQiUdgRfF5jfotTqBAmaEQ1lqRgRVNCTeAGgCPTpCtluvaVNl2qA1OvRuT66tUlSbVlHIIUBJY+gYESzhwz5pQCwunqFSnmKVhSUsJyl+PSUhFWB8J5h9zf1Pi0eV4nlrofI4HgW6T2xl7dTOp7CpUBJl5PVETg8o8xkpx0IopeeI47Fxi8tmSqDrKrBzsQCgoac+SqQ6QtwztewewJ6slQEYpeSWf+bdHwEZSXoY/nEsxp4c48HQjIr5mVy49IzvA1hN0h1MHJuE/FnAunb3MJzvnxlu6nAQUinK/0PoQrMRc8ATTkMo+Cg4/5eg7Nj+y6A6kWFDZ7wdi2453z18pLraZCc0lax1OXFPIO4qoDWw+GMDSJNjAoJwRONdXhVAPEXAS5u+3M15FAcT9c313SgGDehiUTa7mkcN6xzjeQ+x3rXvsY85pY6kHSCQLd9paHrS4ZqENBEwis1vWbYWqESQnmBQLDw9W7yuD7WwPCC1KChrRwAUJsKgOgG6KRbGwWMxDifDM0enyA9pXdF3ncWKJGIATnFJys4Onfu7y2S35c8QvOh/iMQzo4T8V9Dpyn4JwL92n7j/X4ZKDw2tTp6FFIVGNmTSDsMQh7x6Knx/ZK7OcowA8L3vMZ6BkInkQySq++RAmIrNqM3fuQIVSpQngug+5bwMD2sBKgDoKz75d7KMJC7gvuPNxlxldJYZrqICYprHkvILx5g/XmzTbmNU2PhovL8U/Dv+75DH7nJDfnZo6AiksGZuOwpj7uFlZFFkfJrdQHzRqS/AgDeS417EDwMR8gusGi3Cx7jiLP3hv+hCsywEBhD0hvcnzX/GYTyRu0MIaCWfw58d4gvRArg9ks5iBUkzi3lrikQLif8rB9rMcnA4XXSgpE1WE6ipGH1MC+6IFa9ExlvmMPVuIEhD9HKyykexBi4e+guJ5/vOahNuMl92GLUwgR1QNmquVbBTGZ+ypqAbuaEFDw/YknUACyK9Y5zG223KZg/xwKOlN9WNOlhZAQ3ryBfP65weHzCxTCJoO2aTOXan1G3I4ZcCg4EBwMAhgcD3KV4QIEabaFrEbNqMK0jyDAajDoUPDUZcr0aANCFD6x3IRKXLPmwu1ji5tLuyFFNevBcVOyBRw2CBFTDawiOBI0+VkLY4nBey7FeRLup+LNXfDmVNzvgjdf81D4gCIr1YvPFnz1MtI8n6BwSSCup8vz+kYakY0j7+598ZedoKQHpn0urtlyH1q2JBwOiBgF3b5hBYMMWtqLg/dxAsK3E2xgmOQSAlV7PVGch2JNkxSsrd3MoKMlJ9Yyb8MKm8L9DWYA4fM3WJ9/jvWrn4PnDG0699Ay3AIZg+c9DxqU47NQq49pdne4KmKCASK0eQG40W5ovAUMaAfCakAIr0MHhAYYLJWW2MAAKkAA3s6OvbqyBhi8wxa3zZOUkAlP3hrPW+TxKElCod5igDBFrSTbhHs1gChhrcKu3jGmey7uJ+N+V7y5Gxg+9/HHenwaUKDXxylYAgouC1ow0PS7vGvVl7KFibjHsh33+UuswRUQ/ACMmGtgEO8V0YOXpFSGjGq8ACH24jpsJEAlDDzUeVIDAhkMJrm0sPw69eAat4Tfj5AUHArxL6AQYDhPrLOkBJMQDAjrV38VfM58h5H7/YtKsWlAucKCAwSbhuDqg8Al9hsQXbJKYmBkCfqUDriMJ1e1QS8qQ1X3BfEBHg6GaFJD0RCewcPbxvvnxZ73kKX9PDiO3cPFrH5sgGCGxx6g3IxilZcmVxWwUh8i8pS9upJiTsU5B96cijen4vO7b2++1qEAa8Tymsfw55moXw6J+HkAMuYAKEeF+DjKAG79NPp4tLkR1maqhaBA3QsDP3VeAUtocjui+BcrXFnibi3RiEWIcc0FBLZ8h4BCHz8BxdIqHbZGtZGfV1Wc4f04W3m6PPa5EO8RKVR1d+/bpX2nSzLIEO2+xXsk/hkUYMJ6x1DtRTetVJnHsKf7MEKEtzzxVq6uNvtlGi9g+W+iUhmFFCwCIQZ7HUQGQcliKywk2fR+91xXv5HLNwEo8T5vUopL+LJm5KJID1pShwIqzmHZfi5gTuD8eGEKnwgUVLHevBYL9uayv+UhKWS4st/1xa+x74GV+sovtaJJByEtuPowBngIBg/wYPAYZjgSUxkWW0juEMu/YHelMZshiZVMrPTfycNew7CXbt2t8LhfRL7vbvpmG/B5AwUcHuFdiKxIc2nZxrZX9uQql41oQNm3cUCPAT0O6M3z0T/zfIC4u17u6bhx/yjQTfHp9CmrSUI55/qxNqkijgdXldPjgB4HyI/1OOxcFT30OpYe/uyU0gEPYPNqUUMAXoiUBmJbsExWhs7cpmzeEfFKCjSsL4UOkC4wH17efliNSR5Wr3Id2QWL2Xpl3k/g8zvw5k2I/sD9VN+Ac6pvwJy+4B0AAYQllbeiAZV60z/K45OAAkSwXtNgFv5hwhZwwIESAAWGUA/EARBgsNgYA8Fqhq+h6rVOBHwMMCvGYPChGIvBx8BYAA/FcKoPL0u+BBjD7tLDa5WzWqxAwEf8jhENpB/hoG4n8OLNeASEXMGQ6gVSwghft5Vyq5qP8gCEYUAYB3BMW3S3A7oOqNwqRDlgQMsW1tlcYkrxoSDhEOMX52xstrkODx8zNyAM0GGvFTFu5wwUHq8QRW4zKU4svdqLxETYu4YRkM2davETBxRkUY18GBzWsLZ3GCA9PBLT1AxrnjsSDDa2fplMA2+mAeHzOwwKJ8xQ6FC4nwaEc7oUMBsUJNQ+WEUsB0K9kV/jUFBVzPvrJIWAQUgJrM2AqOpAQEoA0q5bcS4BUdeGWMzHwBA1aUEGWBRjDJMOhoJlYIhJBwYBeC0Gsx2IEjAapIDcAw4A7FCwUH/KxrnLF/3MMRUYUnoIVaIBAbBExOkRkWKSgnk2hreV5w0MaJJCGuNkmXEOTVIgJ+D0VvS+0FMyeLaPhf5sr/u1HM9JKLAlMh1+PAIAY58/RhZkxUiNAxhqUkLvdUFWsDYa4YLKnGth5pG56YsfFvoMWg6McF16f0waCQfQzPH9REoIn18lhYmUElJaSDUBluUaYGgqR6Rjf8zHJwOF10sKtchjUUOlLNtNPZCLUZHb8boAIWGxBHwIRqgNQzEONTg4ECL6rsZUUAABs6SDBIO7uE9oAiDBgAADHsCwqRIodSEkhw0OcBhcJIXKxnNpgazoiKkPB/Q20yBn1ZEORFcMxa2AwGLGCDeYmPfXJQD3qlzH5YZFgaC5Z7kBhBMK3OotMuioykkUcIiai36uSuOHpKBWj9Ksm5kvA8+R6RYPgnj4uYc7h5tSxxbDQMLteNZ8v4YcCqfBwAyFITG4+nA21SFCoZvaYJmuZXfIFOyvF0lhvUZS0HDyuJ7ukgESEo8woBcWf4cFxXnyyrvLQTAU43CV1PcmJZCpDJ7fsHw/IgNyBAwaGNy6mUDADoLNnoAuJYTK0MAAJBCkwaGrD7qukoIBQbhUCIxQIw5AVlMbbvYOk0kIylJ/fINCAgH7cV/0oWL0OSuowjk2MPhG5BCwRR6Lfi/3zjnuZd/1oNbdynJilL0BDov17CQyyHltNYpIRAXg7V/CMJHl0mLBZ+HPGDtxAgo+d06DQIdDSArnCdxnkxhWQWE2m4L0ZKuwyH5EIACfCBQgivkKSSG8CLHAFe4H0La4L4CgDQBucX7pfEAhJIMAwaEY4pWeQkrQw5OCSm0QZQgkoVDqg1m4CU19QLMd5BztagSa+hA/J70SaIBo6oM3ipXFm6SQqsNFfSggeKozCghKN6ADYYg1id2g4FJDkwKwAYC268l7X+b+sjER5CDQdZeUqAAAIABJREFU1sAFDgJUB62BPK8NCKE6yADAAmFy6YAcDAGFKAgZbgkgszpjgfuCL4eqQ+Ed43MipYKwI+TxNDDs0kKpDmFTWIJ0VZb6AHx9SAqv9D4EDKQtbjRjYy58oNkbwktxkQ4cCJTPA44AghsUDQgwQLgNYQlh6PLj5RICwdKDGHQEFAwGA7YmHqGgHm+A56oCnsOh3H3N0BhzAlMfwqYg7FGSDKFRxsZhiUNpcNQDGTHwFAgKHApdaweCXBa9L/g65g0Cec3iRyiIBQfxgC3iA74eqUEAl/HjnLgKYZvDIvo2cIS0if9w90+p+aAVjNQ5ml/q6bE+P3+667AMi0h7woNdYZm0kJ4HtymE+qAS0nGoDx/v8elA4f46m4KkWlALXLWOQxWo4+vC97324wqyMa+CFgwSCAWFsCGEpBAwiMxBmmJZdm7YUlIz5CObOu9wuGyLzIjZ1YmQFMLYWGpDAwTIcvxDfVgePq1pRrVuUxdJIXpvKw4HghsWR7j01GpfngJa4wIE3uEgDQYdDlKAgFCWPuvAYFcf4FW08oada1C343A/2qIPEPi+2RGiDV6+nITB8PwNdhue+aWgKxd7VGFSB4Vqg0NCwc7H+JxINcH25XHYPQ+7S9LsCiUxPBgaP/Ljk4CCqQ/vLymY+crVBi3PQkgNcb7UhP24A2Ob7+pDQKHDQLQBoWwJpllKbodLC7SshoFk6Tc3NJK7I59sffHX3K46LDIoricwiB4FIurqQ4TOlqHRVIgGBD18AQQYwn0nu4//UGCq902UWvjSgNDHyts11OZTbVijQCER9emhv8NyCtxinPUyaNQ47Abh/ik4aNurV+cqOAirQ4DtbydPl1Y1K5CuBoJa7BEoFYDVZqtBXOfHEWR0zh6XgAcoJBwm0tAY+5QWOhC+2t6HFxrB/LsA/kkAdwA/C+D7VPWXvAz8TwP4C/70n1TVH3jX71B9fZyCMb4t7ItBsS/yuMYs3zscOjQKEpTSwdEl52OXEo6UFCI4SFJaGFDQLJuCNGnhQX1AMzjiEQ617TkPkmpEsyXEFt4HVx/iy5txCjRKWhhdbTgsgm81IEy142i1PtWbucSi5wsQQjrY50l4AwAJm8SR9oU2T4AOyapZMQ4AEEuBgT0egR9BYRAQr90ZNTzFS96HfFiSgpXkW7XYr5v4+9gB8MI2Fx4WvtkPnrkjL4ZG/whEWtBS80B8zMf7SAo/jMdGMD8B4AdVdRLRvwPgB2E9HwDgZ1X1O1/zIl7tfYADoUkAEZewJ980KFzmXwJEqA8hKawOhwcYMIaGdLBclHf1AdZiPWFQN1AQ1Be/+sZlYETzOlw3vaoQ1xDiJjFoqA/skkIrEEquPqgDIYqM0JFeBg0grLAhaIthUpAXQ6ULGPrcDoEGiD4/vXejlDRBwva5DiuGCk80UnYwRLRY26wsfrz2UBtMSpAAQ3bGsmsqwlNyvFRy8UsDgTQg1DkPCLvOyw6FuXZp4HR1IWDQpYR5sSmU+tCB8FWWFJ41glHV/7Id/iSAf+ZLehXy+jiFvojlYkugBMbjgn/ruQAHlbQ8xCQDae7GpYQDViPBVAWTEgYER/ZnVPCUhIJ4hyblUB/UU54NDhbFqLDeS/Y7Jvkej8bFBco06jx+AoWUFCTSsV3JIduUCwja3Y5LvD28AQHR/DXHgkhI2hd7Lfp+XJDgXSIYR419sx4ZajDwblY02pitH2a5SWMMlyo8xyTVhYCFtbwT8vZ4RLmQxdPWl2oCvy9yew8bXNN463M9FsSP8+7fFnxKBbPlMbSchtVtCg/qA4BPRFJ41+NfgvWUjMdvJqI/D+CXAfw+Vf0zz57U+z589o1feJVNAbjc8S92AjyAwBb8S88j3Z/DRBmQFBKCOBwGCIdaP8fYDzAOsH3ZXEpQMpuCenFYif6VbNJJqAmnNjdkgsEldVwNjJ7ohC4p+Dr1scZekR2mukgrXicgbQrUgECmRqi4lKAKlQYEjTEuUPBtvQQIzirJV2DQeTQY1LVMCqFpocQZorxA3rrOUgxX5jL4F6B5F+ASmksFLiFk6zte9qeQ21g8tmRCMk9EpIAQCz89Odv8fl6XgSMW/1pX9WCHxdrmUclQPu5SguZ/H+/xJUGBiP5N2Hf6j/vUzwP4NlX9q0T0XQD+FBF9h6r+8vW5ve/Dr/lbvklf632IBYwX7vpAVxn82ofn1XlssAhJgVJCCOng8C/McAnBgGAl0w/3PBykBoUZqoNW1Wh+4pIEYaqmfWCiVIVUI66Q0KtkcJEYXLrZS7mFoXFA2TM/dLiE4EBQNQkhwmm1waCnM4omDGg1CKwdFNdztEHCcwUu4DD1wTpbW5LRAjkglCay06rV6Le4qPQuWEhzzMdcNMdVmgYEWsjQcGJ/HxlTJYGQe/HEMiErsb58fj2eFy+oG4VUKmwZXh8BCYntnNsRZoNB9utQbW//J+x9IKLvhRkgf7tXcIaqvgHwxsc/RUQ/C+DbAfz3b/1hH5gl+bCgtQEBOwjQ1AZ0cDyDCLl0oKlSb0bFAXIA8LZXhGhqagOvMHSV6qD+fZ7aoaBeRQmPQEgY7CpDB4UACQW9QgEtO/Ia5pxgEFgwgFrgDo6MoIuqSi5MxVtv39DVJQQGvI17Lv4Agtc6SGkhgLD4AgW3L6yAwvREo+n5BtOCg6IKqpXTynwGsywzlFaTFNQ/DykY8ITQrHqI4ATq0mGLWmtx50L3jNPlMFgtE3XJ43ilaxGY4WLc5rpE4JDwc1cwhKERXwEwfBAUiOgfA/CvA/iHVPVX2vy3APhFVV1E9Ftgnaf/4rt+niVEvVJS8PflQQLADoHrNQ/HDSRxjsiMiasZFZcWAMYDEEZ+8Y6EgmDM3eugw/RziviErjp0SOgTIHQjY6oLJSFIn1dyVaJqHXmpVLcfePI4HWj+Ed+8bVmKBPUF1PYuJRQWwyodNTA8GdNidz82KWExaNz8XJtLKJwgngCdiLqJlC3c40M324O3/UJ2h87QiACy2RE0oXCaIY8Ii7xHpxKmiklhUSdRCgABgRWL3ztyxdxq0DBwtAXf4w6kFn2dKwhszwl7QoBh/yA+yuN9XJLPGsH8IIAvAPgJKzqarsffBuD3E9EJ+37+gKr+4rt+h8rrJQXav6LoEOiSwMO1+o7zvq1NOmAMnXnnPdoiOzDs7kPWBEY9N1/IcieqOY1Z79VV4R0AWjaEriaEVLCdi7kW7RjzKS3AdOQoEQ9fLKCKUXD7h5KVq7GGOH639X00ydnEr9irmgTgUKCtqnJIDHZMazQpYSQgsBjEtzrvz+NlNQ2sduLpiUYnKAsaUn1QLGZkJIF2MBAVEBqohVyFoOlxI6V2meq27H3UgsKSWujWlo9be74GhutcuhULANe5/RheIWu/TpuhUb8CVHgf78OzRjA/9MK1Pwbgx177IlQE97/x/772WdvuekBP5van6+W6upaIcNwWxmcLx23h+Exqf6ofK44bcHxGGAs4JuGYjGMRjsU4hCHHsg5TqrZBcShAh/U3CCiEYXE/pifH5KECVajVPBVVwNWqOzMWqQGA2ZKfSMwDchzlQWSy88QQ9muz1sIB5RtAp+35BGhC+TTxXRXZdCVAENvkGvcFvwZ4DVv008f3m8/ZPK8BngOMBeAOpTsYb6B0B9EdRG9AuAN0N+nBgQfAF/+ClbDzoDGIV60+sXCH0B2L7hB6Y+K6ekWqZZB9I1zNdBZhTlvgcwLr9OvONjcv17TjbhOwLlJxXPvqMXmdd1uCmOqxmXm+2lD4SjxU5FUJUS/8lBemXnoDH+djhghpWBQtqcCKkc8ak20HeR5k+Ph9P2ZICG5PWIAOTUnB44A2KBgsotiqGx5DKtDwQNQ4VIWyIbg9wcVoMy6SLfoWp2BZkuo++5FRfxkmbMF80AgjZpgOP1yXVyQIsAYwCw427pAoOMR5OnyOPktYJEyGVTnKYqmAl8FHlVSHSQiIHA24VwXkCZmuQkDTK6QUxeRCUoiKXFWnYmqoEYQpSEDM6NM5fTx9fNo4oeFzCQUFouuT5jGaSlBxCE+vcXB0m+9XXX34ijw+yNAIxLtzfY+eacHPZy7PDxsEXWHQNtqBIK6nqru67A6r0OnqQ6x8zlXuhsaSFh7g0MapHqSKsMNBfL7gYGMlWNdqVAk2e+2jbBwZ9YfMF6iKRQ0ILRMxN6AW/7ws/hkSRAFiu/Zo1+OzHPMYKSkkFGAqQ9mMFNVwZaGyFwN47UoK64irDhADAsymsESbCoGMB5kK77zV9ovcZegLfxLmGVDw/bnPhyuxi/4vOnW0jIk5jyfX4OtFUviAhKiXQHDFRHfp6pPr42g7pnDtXWHgiwoBAil9NQNkahsBg5AWPCqQCRsQrlAoGwJt44wdSiCERwLlogyJgbB5I5RpkxS0qxaR+TgAzT1lOjJi3IqeALCFvi36Y1/8GyyOx7ljgPCFkg5ing+QTvQaj9iAEObXA/YuBhjKKR031A6GKDubnbZZ9iI2AQS1/dnhsIAzgWCLP0KW50mYd2CewBlwOC8LWeu7+H7zNlC46hDf5Y8sJQCfChRemRCVz7v8/2zRX/fvPmfeh4DBQsHAJIQJpfFEUughtBYwcyyLgLqGCDNdALABgi5geIRDAaGpDdrUCUUVcAUZAITttXWXpCWHZHoxDkC9YVKUNIuGLFbdaHpb+CsUjho/HB8JCcxjh8I8TFKYh0FhDotwHAMkM4EQyzyjMtTMghYkfiDDtONvy+9DuFPDs3KBglhUY1axRgOC6/KnSwlngsGjEF0iOE+0PeEMONz3xZ7fs8t42+P5c2pOt3Mf6/FJQAEfICkAOwiu4+v+4U1/yzWgLiWM2qflfmVknG27pADfH9MXW8n/0GU3W5MOdFv8HRJLaZMkYi7UhQCDNEh09cGSE8tF1ysvmTERlUk41GoaOhD0ZsVKcAsg+Dj6ON6mvU/zqIW/AaBJBT5G26MvfnzB9nE8rfDpBgWNe7+/genIzUKMKANIkxQUpT5o2RMUE6onhKS11ds/hwTDsv3ZJIVzIbMdzxOYd9ufd9um7zsErt/blyZfXO/6lnNf5senAQUiML/+pewLW1P0up7bJAp6ck1+IFqvB9YSxMt2gtFbvsFTfOFx+6jxQt524vu75/TD7ltbkovCOjaT73fRsuubEnc/ZagKJFN1JedU2X4fvFsDzRrPWTYQd89Fheaw3iPSpldBDktNFRIfEyowYnvB/TXXPV7DLUomuVg3bVfLvDxclIiTMcCkiIhLwbC/MyQCrT0i5TnUorSQmi1C4a4WCmhECCSlegItDyelt5MyaNLjo3oQZdpcLWKbLscGflX7obmYX1j4cR389z69npCv9cPhsN7rqk8CCkSMz77h17z6eVvOmLZF7YDo1wCx+LeZJor1ecJxu+E4fBs3jHHgYN/owEHekRq2P9Tazx9SbskxySNyKSJvbXmI3/00lEX/rZ0EEMvXqMweGGmSML4AaJ+LvH5CllZTXfDyztBxt7F42NxtQdeCrgkcLQZ3LjsOmfnmt8ebzxMBU6BLrBz8MuOqVYR3lWnWr5IZEX8CWavG6j0kVSwZjBQH/O6tzldXhypQa99SddhcJt4Fqu1JD7BXp1ZMsIi3AoAD16wUQQFyAhADPAAWqwY1jsjWjnwYLzUB4CDCjYBJDY4AoGXniIW92RLaNXnex5LXmQE5GtG8/vF+0vgnAQVmxmff+DoobDXwAwhNXuvnt4APbSh54XqAcBw3jOOG4zgwDoOBgWFg8MBB1oH6gLWMO3RgiEFhCGMswlgEnvsdhwDQ8uAp1YTD9di+3FIVjbLAB/niDxjUcRUAcQJ5zwaV2sB3YDkIHAgJg4DAbUHngh4dCH6+QUHXsek3KmVQNRgQ1JvCVnOaiAAULBFMERwJBmTNSu5AaMFFkaMgYT/I98LcJFpdYACt8uyklqWiDgjWwzpfK2WHMQXhaOICRZu4AILACvf6cyKWxMydhOla1iTrxGUGQio4bBJgU3HyHG3XVWYkYfu6E/y/1z5+6b2u+iSgQMy4vVZSqEBwpMzqovXT85dr9ek1MQeM2w3H7YYxYjsw2LaDDmtJTwPx79BoKEt2BxHCmGRufbiU4BI4DftkLY1b4lN2AMQ5KSkBhBa3i8r2Ib+7UMJA/dqUFOJWbfXZAL4bCGI7ZIfCYUCwsZ87GwxOnyd4yF2DgTiHxDcP9bVfH3kEkVRkUBBxKUFHmg0XSlLIDlgPCV8XOCCkpAYGtzdYE5dHacGgYJmvBQV6gELUi2ShrOKdtTWAqoNBBobpxssw+pabkTYX40vHMRd9ROACpYSO8SWpEO9+fBpQ+BD1IRd026s+n4/r2xxdIHG9ftxMUhiHSQgcUCCTFAYdCYSty3STFHgCHPYIOBDUyoml8cztFJlPkAvfldEc9+tCUbUaCftz/K7CSNldRQwQa0HpLCBk0L2N9VhWp6ADIcYuKeixoIfYwmlp1dGjMYHQ6jj0FOMlDJFlexWXEMqPEIVsSUNlcEmB9lwP9SSmlBY8qCILrrp0EMbIAINJCzewTqhabxD1JDcAWN4FmtgayjID7F2/zLRCCYUD5Ilr5DkUDgTv2Wlt5ZuHqI1VI3qR6m2UHRQiZr8KqQJwN/PHJAI+FSjwh9gUdhAQPBb0ssjpuuBdRH8Gg5wHMMYNfBwYx82A0MDALiUwxg4GrwkwlnWbHiuMlPAeBzAFdoSaEIYu9QWPNGTuUMAGggJCnc/mp35eCbaA41YtHlBPp9sQJFUEPQwIli/uIMixXI5tTh0Klmrtb6EDQeKLLPBScJGKbECwvQNBxPICVDG1wMBxp7UbZbMncNkSdJcSTH3apYQOhASDN7mxdmy2yAOqByLhysrMW2doW+jREnAlEHxLtyZlI9/FFIKUp67Dsy/dVtvGlmdhBssAx5JonBNAqWOrVffK5fKKx6cBBXq9+mDidV/o9o1MCMS4L/iIK8V+vF8nABwKsR2H7V19YAcDe9tyVsucZDUoMFn1oDEpSwiGt4IFVnzUJYOsBKVhXa4PnPo+4VDCTwEBJVJqY4n4gg9pYQkULilMX+SzLf4R+x0Eetkj2rCl7LsLahpRlWoLVy7bEsFSq3gdQFiqmS5+BAjgP2szKnYwXGslukcCLRRTj7QpkAOCvZO1hmRW1h4cxA6DkhQKBnvy1PKEs4CBBAyGbbn4fb8aEJYDc4X0EfPk7HbQkCdk1b2g3TQ+0uPTgAIzPvvGb3zdk9pCz72Iw6HPx0LXPLcDQR5/FtQhYFICj5t3Ez7A3jy0oGDGLNbhQPA7TBgZNTbz9JFXJi4wlasJaB6INlfmDn0yZ/9tQFD7QTpcOhhS9gNMG3vJNYwAg49HjdXHNBZkXK7x/IdH4xnlPmMm0KAAhugwIMDsCgsuNUAzksCcrA6GbV9g6Knh1TPc4FBAmGZcbFKC6A2sC1d1jVDu0gDDYLZFO/aSd9aur6BgQGC726+Cwly+8OVte8uvCC/VIndzu3rC3qOCls35N+V16+UVj08HCt/wa1/xDL0s6GgIugCJ41jk18X/fG4/B285bmAIINDwPR07EMAgNUmBXFIgMAZ7X0SJlH+1OAauorOZyq3dI4G0eXTbBz2sfDQgdOMpoNBUH9T7NoSkoMuPZ5wzAOjRrvXFH/NV0tqhQPz4qaTb7VLHQSOJzILAFhZEByKcaKHsCpFmwW78k/bzpMU99JyOrRFgUx/IJQdyyYDEIrRYF0QPsCrCkEtqi9ta6tnvYrVFPpotw0LeHQgcsRZkeSaTIMPqLGgkUDUwzEjBznHbE1ltyuWuUAIih2OQu5iJvEXAxwMC8KlAgRi3b3idpEB9EUtBgVxi2GARkBBpC1/K/Sd1TQCC+bBcf/K9SwnRatxLt5oZUc2mQLEt+5INorJg+82V3BUREkkZHHVb6KUnXPdwAOzzmnuXKEjNjjBk3zAf56I68rP5hy0kBYNCRg9mIpIvVoq7uUdRYlmQEpaDwZKUMs6QNLJK3KZQwWgJGuqSSPvZWYPysTuMAWEC4l6HBMLN6k+G5wYU5uAKriKCDldVhhd33cDgfyeTVWXicL96sRUxMPSEqjVsbixgskmU09UEXntGaHiWQvo0LSJUnY/3+NC+D/82gH8ZwP/tl/1eVf1xP/eDAL4fFj71r6rqf/Gu38EfYGgs6SAWuIDX2uZz8TcY9OvpybmUGqhDwaQDkAOBWvvxMCVq1BZkEFl2H02qSkHsni7PUEz1IWwjCYh2nAtfngBCHo47KJTE3ISj3f1ZoDpzvLWCewaF6xy367fAi7ZoKUKpXRSPZDJPW5bIG8HCooHFXW2oJMyU7Js6ku7WrjroC4bGDgaxZjfwpjfsDXSNnVZqPqFAo6Iu2aWd+J1ey9FySTwdnc2Iqmzg0Cjc6lLBGWBYnkOxyHvjuopJSHXz3FSDkrqYW9NdCgXiq6s+/DAe+z4AwL+vqv9enyCivwfAPwfgOwD87QD+KyL6dtVolvb8QR8QvJSLuUkJLHObhwrYpYeAA3cwyAJfwMEhZdAwCAQM2r63G4cHxyDgIGRdit2LQALzeVO8RwAoDIzS9lIACPvHtvhj0bumrZHo83iNqtgvtoaXbUGr3TX9OLoxZ1s4b6qSiz+es81pSgrqPn19tjFZjkVLGrNiJwaGRWIbFIt0209Y0lgKRuR/XoAhpBGtlOkAQQRwPaoQnuElC6QLqjdEuDg5ZBgMpuFACPBUSnZml3q5d3UYqLBJFH3eYXBzEJzT3JSDCSeTQSGA4IsdbcEnDBVmuPYSdESNxB/p8UF9H97y+G4Af8ILuP7vRPQzAP4+AP/t2570IcFLLJ4t16GwJtgDdfiJShHXc8y/eI1JCoCBATgMAjj82KAA7zIckkKG2BIlIKJ82KYGUqkP6dyHJBxSgmjncv9krmcC5lxIChxSggMgVAquBW6dleq66/mH8VBTHwgtIYAyHTs7WtO0xUXDiqbyMpWBo5CqGhi4oBDuSPb3CTBbBYhynwFbvmjjvY9Yha46qBoEIrqTPMKTVLx4SYWMGxiOllpOqDoNXsSFyP5GX/zQgoBaOavWDMbSq49FONilhGlAuPsCj60vdG1QEHUviLTr6KsMhbc8fhcR/QuwSs2/W1X/GoDfCGsOE4+f87mHR+/78Gt/3Te/Xn3wD5ZzcS+MeTapoUHDJYIAAbf5688gGSZJoEFBB+Apuh7pDm9oaBuN/GKB7IsRX950Jbn6mhqy6iYlUC7ohX3h2zE5NCz/oQNiocNCsaAa1Z+0LWJb2CrLQVFz+6LXTP2+Pnffs4FgUC2UKOk2IhPT4CA8IWNURWU2MCyIg0DAbH1AB5lrUmHvl/p7eHUdloRg4wJDUx1CilMrX08SYPD3SoHMF0l7gnsxfOHDIWefqf19AQIEBPQy59+FOQn3QTimqwzTF/isOIinUHB3bho8ySpQMT4d9eHZ448B+AOwz+4PAPjDsKYw7/3ofR9+/Re/TT8MCr74ZYFlWTxBLHxZl0W/UpXIcVyn+3NYoqpxfbk0wNCPyffKfm3lJwTpM8kFFYmm4XloKkPUCQgoRM2AbeHrQtQFKBgYBDL518Fglk1f4FFAtvWI7ItftwWvD+fx7PwYfks3YxzYRGjNvWc+jml7cUlheBpzuCS5uyORRkb32iYQzBhPBQHEvm0t56G2IyWEUB+QxmVFtqqOn0FHjgMABv5a8FD7ex+S01JtsfE5TDI4XF0YHLkwDQbUS8mEl8UCloa7K0t92K/9WI8PgoKq/kKMieg/BPCf++FfAfCt7dLf5HNvfRAzbq+0KXAu7NqPeTzMmVRwgcf1mgsgWCQDYfqGfozacGk0Gl+SiObLDDm3A27BUikl2KKO0Jjt2Be9YiU4TCKIpVTjAEZJCpqASEkhkjA6AOgJHOgKgz43vDITA4O3fTatHZYGLWMZFMaC6MIaFrw0IFjDbQm6qxAjvveKy53RF4YWGCizSDklg25TqJgFSSCYsRm1kCMYnW6Ap0ZSX+RhTEYt+vp9z8+frja8iUCoLh3kP/s7Ujrw6EVhc3MuITDzpmrQp6g+ENEXVfXn/fCfAvA/+fhPA/hPiOiPwAyNvxXAn33Xz+MPyH3oC9oMjAvHObaFv1/ztvH0cXQoEqgEALiNh/cPHDXX2pFHj0EL8SWQUBXnFCQVKkahQyFUgrjjx33TF7nfR/XJuRgXKCY8P9slBC1ArJXztcjbnuXJ3PU6g0KUZ9PB0IOhYwBjQI8Cgo4DejgUZGHpwlCBDDM0iqjZFDzMmcPIiHK/5R21BRkB3uo+JYUwKvpibqoD6ZFu54xdkYj7qIVsAvrNDcjsBuMIVHcAxPUOoxo3sPhzzmnBT5yGxe5BKNtARH52KCyXFOx59jPCq/VVlxRe6PvwDxPRd8I+u78E4F8BAFX9n4noRwH8L7Bv6u98l+cBwAflPuRCXgWGMYbPzQsY2vEqiNQ1A7zmJj2oDO8X6NZoB4HAdEehKABiqkNvLKrecoymx/6Xw8BCWF1aoB6npxVAa5VZ2sLHBOmlWf3lvF7OW/ViX/y+oK0hjWzHudD7NQ9zeJgDi5Vl883AMCDHgC4HwnFAjgnRw3MeFqx86rJoBTKJgR0MXH9NSgpEQDX1qX/wuyyhg8EWqaK8DR0EZkcSVFwKEGV04rlMN/+5HoOSgBj7/Lav5/dz98G7hBA2AQopx6XIgIFa9GRGOQZMeOxeilSdPs7jy9r3wa//gwD+4GtexOtzH7Qt8plQOE5+Os/rsH0s+jUwAhYBlcH5PAogLIOC+FjRyrNpgGGP66/Go1b624QA28uCeTbE/ga4XSGAUOVDl0Pg0nHyOqdRUelEh4PCqivFIo4mt7GYa+EDWdl5AwR2AJBWfEVkd/EAbi4hHAwcA3qcwM2goMeAyITKAZEJuR2PQBDFEgWzpJQwgcz3qXvi/g+nn2guAAAgAElEQVTmPLSzLqWlCrGpDJLH5IComJTwXwy/A1uEqkHBQthzgVMBghssMpq1xavkNcQ4Rgt7dxh0o+ImHTgUemequVzSIC6YbHaFj/P4NCIaXxunoNoWvEsJa9qHsGJ+Pr3GJIHh41HXLbZr/WfIYotlX76hbcreamxsc5XwY63PdBFoEaRsgBbmLBUITM2T0MEAjSz97E9tS0ZtDK81CJw5p9v1qy14JBy6ZKB98TsgHseXY4eCOhRwsIHg5iBYE3qbVo1JloU4q1VJSEnBYxSYFpYIOICgERlg9kRyP37450sLt6jCKAGf4j0GvLnGBQoBhMMNz+puZ8IWqk4Dgz7LRc2x8D3PJeCQxwGPy7UBhiOlBE4VCBF4FQljHhxlmZLWh/JYhMmMw6Mf7feF2sGpdnysx6cDhddKCn2hr1mSwpoYDQrDz/EaDoGFId5fQKbDYGEsLjBwASGqOUfJ94Va+LaM/bya6rDclrAWQSdVvcYJsJsALNlFEUFHaVfoYNDI1A8IGAgMAvuxJhDsXEgKirrrdzAo9fmQzfvi93PoUkPbQ4FxGAhuA+pl3fU2ABkQOaDiUkuEFkclZVrevs3jFNRckQEGhoc8hycSJTGwLwpNHITEwG5PWLAgMrG9gwAJBHVXtdlzWKkyXTMV/mZj3mHA79ieXTNGswNQuE6vUgKXunDYeA7CIRbnwKk+cKkf6X35OI9PBgqvzX2oxT5zfGN6AMVYA7yOAsWaBYU1zA6xDARjOBh4ZePR5Qt/OdGXcDUljXNqX8+lljK9hEFeZUi9vpi6GYCmqxARp4AyMFKqD6UeFBD2TWOs+5zmOTcooha6+f6blIAGAqAWPi77NPx3KAjwGWefB11W2l3WBOSEyOFSwiMQFi8wC3iYCsESYPCX4K+VM9TDIJB5AI7RkBg4Yw2G2xBWkxQKCPm+J4RMUrDaGIfd3ekzjJ4J6+NBo1Lm2/nHay1HxorxNJUh7QiMyiDlTUJYwgaE4TU+OdSH0QyU9nMMjh/n8WlA4QO8DyMXfYDhNCisNidnOx4GCBkpFYwVRVHimMCDwJO9pFZAoRZ/fnieMhtVn1kJ7B+uSQeMEQUHp61dnYBMBa2wJ5SUQM/AoKU6ACcIJ4C7L/o7Cg73hIRqzK9tsesVAMAGBrtO93l/bj+3QWEO4DO3IawDKifgMEjbBmblPbADYVoQU9RpZO2Q9GxRM8OYHOCJSBy6eEgIWkAorwN7WHNEiVbWbPweFjUHCpEXyLFKWoyBG91sYUeFLTYYDF/0wxe+zT85dnjYczkX8u5hCLtTVaNai7CGQeEcjDEMCKNJCrtd4qtoaPxKPD4kISoWfUHhwI0JYxUIhqsIw6GRYFiM4eOx2KUEL7S6GMzTFj0I02EwJcpsuVWYqK7xpBpLriHLf/DOwwmD07wRdMLz4jO/sAEhXJLdk3C6gTFAcM+x+rFux6fvLS5Qm5QZkkK/w1TtlgsEYvbh+T4aukkIKrGF2jDRciANDB7NGEBgr76Ui9WBECHhHHkU/jrVXyDnu7enTpvbuLl4M/P1SCmhpBLFULaFrANDDy/I+5kDIRb2scOBx2X+MGkzSvSNOsfsMQioMGp16UAkbjKM6arCGZXAB1vA03Ao0JFAKM/D14P68MrgpTFPjHXDWAGGA58RMNYB9jlb9B0IZ4HAJQWruuxSQpKZveNzNBolHGL7wVWck2FdnwmUQKCsH2aZb2MS9CTwCcipoBPACiB0W0LFKFTlv11SSJUBdwBvUJDogIj9yoWuyPv+w/6tc2/73g3dgICAgh6IompK0bfRgLB4gcfCCtUhJYVIDKuMBiVz9NVr36P+elQjRdUlSAUjRbp8xoZouYK1Swq9UL/ZFKww7w6FY/jx2GFwcAdBjY9hd/cyLHJJCNolBOtyPRfjtiwK8mAuMIT6wM0D8XUBBShuPF/1HB4TjImBth/Tl2no5S3gB+G+szsYuXusFqPnDbg5cS4z+EypohhzlaQQRTpjy67FsSlhqbQxsqTXEsXCtM0XzvJWZtEA1Rq4RMxBdESaAPW/sZk/qfpZRS/lDQb6uOhVy5i36acXSsTh3m/AQ6Z909ysQKxMi2DUsSBrQua0kGff1pygcWLNE8QHEHUqxkBkYKqcKempGNxVJiATWNFUYoKW1aAkEYjHIljtSK/u7V2b4w/NArimT216ei66ENHDHpB2Ac9v4UiQqmQpL+iIyJeI5DBzo7CHhcdzqM5VXn2OtUGwsum61+FrHAqAYNCvvP/lCjCdYJq5EZ0AfY6Q0YVsAYHPFGdZJ0RPsE4sP2Y/ZjW4sJ4gXQ4EYAUIyFJfa2zHK6HQy3vb3J0oK/3GsjZwKCbdMXHHxIlJJxZOLDqxoq0ZGbyUmpuS3P5Are4QefgU5Tce8K9ULPy+58vxs2uoneuASFWiEcTSj6PDlS3K7DOx5g6EeWINX/zMIGbM+5sWvguERVSZoMs+F3WPksrCcCgkHHJbIAcF+SYOCnttvfw/yq2x/4lPt7htAPWcIuX2tdze10Q6lczXgtEjiwVL1atWq8dy7HJkWVvit/D+IXyZH58EFAiCA6+AAsG1/QlyIBBNgD63O6wDQdgDfHxOaILZdHRyEOTe78QmbSyrkpM2BKSEsB2TjxsIVgPDHc3u0CQJk1fuWDgx6e4wMJCJg0EopAUHg3dbJgooyAaEKNwSN0HCZbH7ODoYiz4/3wtA5cJ5pmugP9GzDsU2WZJhzZJgOC0PYg4sBwLRwDrf7LH8TiIlgjgErNPVxHDJICQEaB2TTN/sWsnIVHtN0AaHJwtqh8B2n46vnG36fDGqG3P7z8moEe3yKkUWS8GBxL1Z4hGygCCaAQvK/NqB8PFiFT4ZKLxWUjAY1GI20foNQBPKJhmQA4EcEAaDBcpxbRExGNfM4fX0GhASAK4+LPZFv0kMvvi7pNCsBKtDgc62TSw6XQefXpgkABdgcJsDGRCUFCP3mi7FdOuVSl3dhqRJA5dz/Tje502FeLIerKiL34W1SwkNCL6ZqjBA7GAgxrq/efozhQgjultpNLUJEKzck0Y27MzM2dgHEFQDDJ14/U/UBoTiYIyBtgQv0ns8z7pM0fZzKpRMm0KbKWzIPFdFAqEkBXYwhIQQRV8CDF/j6sOHSAqlY5vzn9igEAuJfFER+xdJa0w6AQfDFiSk9dGFpBAwWCkptPr+5OOQELjGC01SIHc0xh4odcE3STA0KYGrESyoSQmxsYATCLG5aop9kbOPSRwCUjCgBobsTCT1dl8Fhjz2X2D6uwPBwSAywXJcgMBYPNKttogxz8/Rf1OqIkQWjBRxBw4CvRzb2FPNY/Nj1uilWeCqknUPX6gHlaHOlEf2amfJPSENoxY6VeFlk7zDeH721OCA7G1hFq2Iby0JIcBVMFCXHD7O49OAAk18gf6fVz3HuiNP+9DZFr3yHdoyCdWhEZ2XsWKBTe+W5GOxRafsoiotyINx0Gv4H608t9f3Xwte4nsHyZ28HHgDRdwlQlUQmhBue54Pc8ruy2RTlZgdam7VZxIMtghBIYWwf7F84cdN0kqOFxCkA8LHJA0ePsfXc/YJIBdxUxVonlhhOHP7AAIaateZreHEmCcAYN3eYNy/gHn7DOP2BYzbZziYMXTh8BTrQwVDBQcsierwTRBqgWnkGq5Ivy5yHRiSkNFcZvudnVAw6IBgVMzEIs/JVFvwUSqWtadlmU3xDYDPAbxR4A0p3gC+Ce5+0zhjU0qJokerhCm5R3GUcvNxHp8GFCAYr5EUAA/jXW0/oXRPUVvZdMut7TrbnUPiHPY6BOmdGGJNmX2BSy52eJ1/eCOQkhQkJIh27k7eFwARhRASgyK8DCYVOAzYbR8OB2WTFsxgugBeIJbcuNVJoNzsCynki/qy4OHzJMgq0xLfMV8REpoIHAjkQgiVWrIbI0J1kJIYfJM106hI01WGJoOv801JG7G4ZZnlPvNCnFyoUnVRto48GoJUIVoCuKjmmL1cHbT6Y2zfJTSV4bnJ4NHA2J4TEIi074BEqA4mMZQ/bFMd/HeatKAtlI1crQkEX2HwNQ4FQHC8xqYApGVefDM14t4gIL74PWgGT8YZXLPvRVfe+eUChus+r6ECRoDgpALBvtdUc7SrC2770IdtAWOB2MBGHiYsLGAHhDoU2CspPUBhNZVA2kZPtna7XH2+LYjiQkgL6t4H1+mX224ojIrutrs85v2GzYMRUKEwqEn+tixmC81lE1GhAoWopqkuqlYyophttxw8PvLvoTZubwWu56nekriuSwpE4Ux2e5ICUbH6qj5Y34ueAaP1F0aoeoPDxwQC8IlAwWwKv/re1ysURGIwgAHBwHCmRCC6qiZgLnRL25UGB4ljB8KKOSbIeASBNIkhJIWIbM4OyT53gjZQVNyi5tcjXKcGhbXDYLhq5FCAAyAKqAYArCqzucKjEJBQgUAEWCEVAEBICgs7DFB3TVKXJGLc3eX5QYQKUca8vNPLgqxlxmAuP7wZ6+oH8fGmSRkrVRGrFN2XZ21EHoDk1Z9NvNd9I0049H+ANgNBKULYf8ODTQGpNpXK0Q2SrAb8yN9gbTYFwCtKdRhQ2hC6XaGkBVdvlPwvKjB8TM8D8OF9H/4kgL/bL/lmAL+kqt/pVZ9/GsBf8HM/qao/8M7f8Wrvg90RFlZ+u60u4QmQNDCIlxS3mPtY8EtzhKcjFcgAZD0CQXzB5dwFDrH4hQgnla95wc/7B46wc8Q+gWC2Egw7RoBhiMMhwKAmIVhPdK/CbHDAsC/rGgaG5arCcs/DCkcGwb6RDQhcb/EGhLjmCoYyNjYwSEgM0xqjTvsB1L/I/jweB1QEI6Fg/TvATV5Pj0rt2RcXeaFXJjfSURa+z7tswiXchrGi/bquGdgi182OIDDJgNv70qFA/VqXIBjN+7ABIWwHGX3SBLcCQt93GJSc89VVH34Yl74PqvrPxpiI/jCAv96u/1lV/c7XvIjXex+sNLgGAEjS25CqA3mcvS/26YBYCYgCwYLFFOacSgFgPAJBGhBMfSg49P1EgwT6XoGUcgIINUZKB2ZLsBZurkKExNCatAQQavMv6wJk2H6x7XNBNCBEbazQtxUoZwa5xNBXj1+9eR+aB0K8noT0oiDhEonnwp5Lw+pqqhfAkWXp7srterLFD5TEYqfVxrA7cdhSFL53CNTY/8Ar2PrYDzoQuvYUvy/g0M9zGxNa8JIbESe6MfHqfbhICCnX0LYHusTwcR5fUt8HsqiT7wHwj3xpL+MDDI1YGdKrcFXC8pIbGKSajvDCTAAUBNzchwUPNPZWZjJq8esLQMi5BgLPhzIopNTQoJBfznAzhj0kDIkFBeKCQbRr6zDAUGtY22AQ7SiUHAbLFzUbGFaHQgOCvaclLYT9MG+yoT5cJNfQ1yHegUos1FmITaeiRwkhdXxREA/veeml8g5zYSpHbcb63QkDquIraVTdtvocuo0AgEdlvryoegLYU5OLGyuDcbn5ws5wEVzqZimw2xFKQlhatoWSENCMjfH6++v+6koKb3v8gwB+QVX/tzb3m4nozwP4ZQC/T1X/zLt+CH2AoREuKbBLBUQC080lgRDbZJMUJhcEYju3YweDKqqsf0kK2qUChn/xHAgMKO0qRKkMfg5lNkufhKs/FIBgAXEzKnr/Rgop4TAwUHaELjjQoT4uKNA0GETI/uoROe2xORLUtbK2GJ8/4glNdSAzEprLl6Cr9HP7PU2qUAExgeUGHpYsxesAjwkwewmCqrxkpglKKLB7fKLTUnwG8bmoH4MM7LWw7BN4dse1t4ceYBDPjL4LD/PxvIAF9TgFtTgWlEu6GxmFyMOckUAIaSG+L/5uf3QpAfjSofA7APxIO/55AN+mqn+ViL4LwJ8iou9Q1V++PrE3g/niF7/pVeqD6X3mv5YtoGclLJSjImBUDBZMEZwOibMBwfIPfUy2Vzc0dklBnwBB+dndqdkZ4NeC8gMOKFi48srXTtz2AYQGhmgLTwGGo2Bge5MU6HAoTJcSfL8IOLoNod1Fe5izhJHxCRyofwjaFnnzHpjURi1hoCQK1h0kBLKkqXGAxwCPw+wMYZxkX2jZk7PaxAcYhM2gm6XRuadck6ki4vuYAzYNvasQTdHYa8zgLWP/Ab0OzR7mXKltV2lhadhC6sZR0RT79pV4fDAUyPqq/dMAvivmvF3cGx//FBH9LIBvh3WR2h69Gcx3fMffpq8yNAIpESy1pCBqaoOyQiQkBW0SgkUHnCK4c8HgDDCQ4C4mKejYpQPt0kLcedo4RNWAglJJCgYx2hYhkYel+GtnXhWlyLb4uTV4pcONjYeNaQj4MDDwIQWHA+DD74kBhKbwjrWLyNecB2GPfmTzWkQsA7UVEPdahWZ6sgGBIGT9M5SaO8/TlTG6QTI6ccHUhmFdpJgNDhqpwtEzwUulcyt5HttgqpbwUQSVoxmsOy6ZDWLPpCTU3/Rsri/+eHRYAE9gQdeyuzsMdptCJUjFjaNLlV9JIABfmqTwjwL4X1X152KCiL4FwC+q6iKi3wLr+/AX3/WDXmtoDEnBjEteuSd8bKRuY1D3OiiWeHuyDgJR3Fm8RIngToq7xHndYKAbHBoMAg5df6UOCLp8sN3V5QlNqQIZDJjD3WjdlLjbEQ4xKeEIINh5PjQ3gwRSqb3aAfiwKLwwKGbNApcQuEsKDoQHeTk+heaShNgChKz8UoerEOrCsNrfpyLQMbyNm0KGVcYiHuBhe/WAJ2YrYkpMoGGVjXlU7QvxepoJhqxw5L5+jqpHgDXqATLIAO8GAujyZ18E+IexT5B2GGizG3RARJYkmvrwKCX03/2VAMQH9X1Q1R+CdZf+kcvlvw3A7yeiE/b3/YCq/uK7f8crXZIwug612H9zVZnzXdn1VVIrDCoGhgnFFMFk9dpEgruqlSUhwV0Ub3xsUKBsBJS6akKiweIChC4NSAOEzZfAyhsU1BOcDAjRlYnZjXfDXI8BCFMddhiMQ8E3W/QJhYgXaot6NU/DgK1rNmneRfHab2rDC7YFdaKQ2wkMd1SWe5ckSN2FygGGBeJh59aAMBsUfA9mg0DAwetnLvYSeg4DHjaW4UVMPFxJoGbbgMIaxMJBGEFR9cfEZ1JwoIeFV2oTXSY6DHRbsVs1DKrjUh3Cjdo9EFfbQkllz1Scj/H40L4PUNXvfTL3YwB+7LUv4kMkhQELXmEPYiE1MRzQBIKIWl1AsQrBk9UlBAeDqsWhqwHhDSneiOJ09cECgaiNL4t/2x7npH25QlIIbZVJrYNEy3Ss7kwGBRoRhyDVEv5QlxZMbRg3h8Jth0NAYJMUyLycXWXgDgX/1QEEdomoxyk8lRY0pIW6QBDVjobbEsS6b7GYAdUjHVWloh49WYq9US2NAR4BBRtPh8IaBpChBpSljDGsOax4b0+LU2CXYDy9/C1r6Xrq6aX0/otxS4ja7AnXaEavqYBHScHjMZuE0L9RH+fxSUQ0Aq+XFBYpRgABJi2olyWPvUkKmq3OpyhOtkV/dyjcofg8oKAGhjuaNBA9S2PhN0kBMW5AQIdCWL6xqxBRIEXIulEqCUZryUY9n6HHIRyaBkY+FHxzaeFmIBiHYtyA0aCQr8d/Px/2owSe7htbCChcmxCqKBB2Q6PGf5GDLVKO/YAFBzCsSY7ZCAoIRAwVaWHQVPMjQGD1D+coMCwZWGKVuJcurGGF1MTNzwYEkyZNWyCXFCiNov1xVR0ehfY2pY9SRJ3eRamUEnTr5fXE+xCGxgJCSQoFBbT9x3x8ElD4IEmBLIotynZSRNuEpAB/Yz0UdnIEkoS0gJIQHAafC/C5GixCPUggNAiggQDUINGAACr7Qc4hNFKrYhgAy0ihDQYRvqxbYBId+8Y3kxLGTXHcgHE8QqF/V4/T1uvoIBCAhxsUY+MCwqM9oX0SGl9hTjCoRCiySQGQWuzkpIl6gyIrx/A9MQFuX8jtGAmDJQNLbRs6ICq2tWJ0KXpTa7wSBGzfvOt99+E+HMJFfH5R8br9GO3Xtc88qmtOlwZ2ILTqS1oF9Ep1qFzOGj19hV/2x9+0UJgABkWvAPUvr8eIs1o6MKJ5KUyFYOBUL3+qISkAb1TxuQKfp9TQJAQGopEw3M6ADol2N4ZLEykdNGNW3wP25TJYNCgQQGwSTiQ3WVPrAgIOlKQQ9oQbEgwGB91FfjxKCult0AaGJiV0KGw2hdgkfm7cIy3/3wwJ5AZfcgOwFbSthU+5pzVq3ObBBgIeh+3FiqauY2DpYVCQw6JPVfIGkEAIWEtIePGh1ep9tszir8kxPQfGPldqYUZO6sX7QFcgaJMUejm2RyA8KhD9FXz5H58EFD5EfYg89kHq3jZNY48CblNAEjiAMAU4Q0pQuKQAfK7Ar5LB4Q1q0XfpICFwHTfpYAMEsH2pYkwpziuic5NJCib9LIYnOwUQABzqEYsejxCGxdsOhuMz4Lj572hQii/qOBoQmpQQYCCPgIzIwQ4G/xNQP9VHWgMlBbkb0P7OttDR6jB2CFyuMehaEx8+vJnPcWAeC6wHhq4CAg6/w2qJ3GR/m64AgsB6Top5IJ4sKH2ybfMBEn0EfjcCxnlgb+7XqzCVsdHhoIpF2qQcvQDiCof9/f9yPz4JKLw+SxKuOTZ916FQH6pFJZqUUPrcycCptt1dKnijKDCEpOASAh7gQBXk3qWGi5itbdzdX+0wIRaLr4DgUAvj5kCFMTsQ6ChPg6kQpjYcDoYAT/zeAMG4NSAcZruUgMK6SAkdCA0M+Tlo3VM1c4mDHk33pgh0DqMExfT2g6n7/8awcv1yYBw3TF0GA10uKXjdhL6AqCz55hpuNR40nKS7Ff85BJ4EDqX0Zy9Qrs+nyx4Xl+S2YQcZRdr320DwEra+/I9PBgofKinEZt+vEgdNQvA9u7HHNwOCbygg5Oa/oEsIVxA8ZL88m8OT/T7cRPMQ3QMMBQSXGA6Y+jA6EFBA+AwYISlw+6I3dWFYa8WqszBcUli+9/FbVYd8aNOtu1j+lj8Y75y2M4MxjptBQRaGHJhyczCYyhCLTOB3WpeOhAkiDJWwJXCVe++3/csriKV2DRpKqAIe/3C9tiUtaZ0r1WE3Nj5AQkP1KQhsYNAdFV8XUFAAa8k7r+vXT3UpwBd6tiDQkg6kXRPlx3Lsx9KO4w4qgmrE1Le3hah1Cj0LR7t8htdEne4+JLhAEhIEA4PNuDrIx8O2IzbOBtC4Hf6zAigOAhXguEKh/f3cNtdmdoD1P6CFMD98P/Uy+fT842V90hZzFUgxtSAs89gDxDysWSKAicmyW4fVhJTB1rx1DNsfA+sYoMFQD5QS5haZGkFnfYkSRMVelwdoqdodPmNjhtjPkwXhA/NUzFNx3hVzKtapWFMhsS2F+hdU84uroOiv6d2sDkiFkn8JULi/53WfBhQUON+fCQYFsTv+VBtPtcCcAIS0fYdBFiyVdgf1Baz5jQOyoN79yfYSKB4T5N8aq7pJCShhI8P+Obwstg3et4MKBgGHG9tGof4EEHw/jgsMYO/NU4EoVLMAVvdGyOVvetv+Xef6MdWx+i/rRtwwGipbarYmBDyA6WArJe8AEIeAHAPrNjCPA+N2YN4GiI+CAjkUIs8lXNyqDzCQlu8hkR3KA7IcMjSgfGKeMCD4tk7F8r1OhZ7q7UIV5D1GOTaHwiGKo1ei1vK4vfbxNxcUAJzrnZft10sDQoxXSQuzSQ69gGkvYhpACEDkF32hgNDh8BrJYeGtkkLXKrpqHYDIhkINCHwBwuAGBYfBMUxaCEmhA0EPkxQCCAGDofZys2oQNSj0zYlB/OTvAx7/3vc59+y6/lk7ldRfhAHBt0EFBL/bL49uXIMdDJftNjBvA+NmTWkMBgNKBobFFXNiUZFNSgCgohAamelpz7dUcaX6OUqMeaKAEJu3DxQHA6ZLCU/AMEShorilK9UUFdaoKvVxHp8GFPTDoHCGtOBgWAEFX/ypTlzF5QCBbwmDftyh8ExSmNjBsAe1P0oJwMMXfoMBEEmBD9uD6hBgGAWGW5cUHAr9b5QmKRwOhYGCw/CXnWoLngChixHPoNfB+r7zzz73J2BIKcE3UxssyEkjzDmlBZcM2jaPgXEbWLcD63ZgOhRMVeBczJXqHsFPLinAVQVSn2d/HQxd7Fmydg8XsurLcyKBMO+mSkiHQgMDrdgAakDQJilYY1x8sKTwvo9PAwp4pfrg6saUi6RwNkAoqkZhjENa8LGuJi1EVaKwDXQIdDBcJYUrGJ7ZFy5gSN2cSlK43p0TCE/gEJJBjkdJCjcfc/joGxBTUkDFQwUcFpBVifP3Ny8E+ni0v0+f7J/Nveva/mW4WmLDTXlRH0rc34GwXFLo9gO5HVjHgdXAAPZFDC//SlZS3V6SuoRg0oIoFRjgDVscKPl8UP08IqwTqTLMUB3uSCCIqxBhhQwwsJj0MFy/WypmZ3AwhOftYz0+DSh8gKQwV5MWAgSzGRxjW7ukkADoxz6Xi3hh7/z+kj3hGRBekhSeqBBAuyOjqQ5dUuAmIVz23cC4wWG4W7OpS90l2Tycm3SQG/BcdejbFQrywvF1LmwG3S6Byzjergwfpx0I2569ZB6XGhHSwjCJQW4lLfBxgG8DfDtAkVUJQGAp1wt+w4CDQdVSsWGejAxD1gKBAH7O59R/3kTaEOQ0IMS4gICymk9TG+z7qWkIu+kOhPX/Q+HJ9bDrEwbLJYT5RI1YtThyvHYYhMqg/S4fAHhp/8ymcJUWrnfE9thsCXhchBscnoHhYk/oksJt2CLuQAhja9gUQmUY8J+Jx9+ZwUsXiSFp8syo+j5b/6wDFH3b3qsuJaBA0GwJ6mpEqRDhcdjVB751ScHepFzQGsZp90CopiHWJIS6xgAQY+Q5RTuvMCgEBJrqsNKmgFIfZqkOnC4z+4Eq4l4hk1SGRn3nj/P4NKCAD4PClJIY5uQuCQUAABQ+SURBVDIoJBDaPoHgY237AMEGhJAUYrtKCFcwPLMnvBYM9MQV2VWIq+fhokrcrt4HNzJ21+uCQWHBPvgVMLhIJt0V+gCD6If2TE1611z+wW0c700/1887CCxIrBsaXX0YdLEnlNowjnJBmj1huD1hgD87ALAvat3eI8mMRc1QcFGp9zPf03ZeAgj1s2Tawi8bgs3JBQp6VR8aENjJowkDh9TXPBS+BEnhXA6E5erDairDascXOAQQUnV4GxTetr1LfbiqEPGg2uV2sSd0l2Q3NB78XH3YvA8e+HR1PY6jSQjwtX5RH8Id+iAltMKwVqoajyBc2CWJUDWoXdMfYbjUdtzemTTFXIyMaeTj8jwIj3RHyjDbwhhddSgVYt4OEMilyEq1n7HQJeCgtdhjwUuNVTxFX32+HZvbEZv7MbwOOtEMjfbh0FTwsp/DS61Nn1jcgmgEMhkUPp7v4f2KrHwrrLz7b4B9dP+Bqv5RIvpbAfxJAH8HgL8E4HtU9a95hec/CuCfAPArAL5XVf/c237HhxgaQ0oIlSHHVxA82fQdW975rwbFZ8bFd8UqXGHg45AQNldkX5QdDPzcJXmwBzBd4xRCfSi1tFSGw6UElOrwYNR8ojJQB4LHPzyAsG/d9hDUewb+AMJVUoi3KmDgb1QZGikNjdrjFJpLcrjqwCk1mAoxbwf4ZpVoRASyfCGTFeJRH0vmIzgEPC4hF/4qCKgoZMkGhlj8mJqAyHGqDshQW6v6X5KCLvU6F5LFbDoYLm/Xl+3xPpLCBPC7VfXPEdGvA/BTRPQTAL4XwH+tqn+IiH4PgN8D4N8A8I/DyrD9VgB/P4A/5vsXH6+VFADgnC4tXCWFl8AwCwgbHOYFBl1SeLbwXxpfpYSFR9XhBfXhKi08c0tu0kKzL3QDYx9zlxK0XtpxuNrwZOtZkrl56DOuQOh/83XcDZL9j3v44P2aq13hckm8Md3A+OCODBgcjBFqhKdc8zGwDgMBHwPssQoAWcMf8gVNgrlw6e1ovgbRWPBSi38FUKTG7Vx4FSIOoRdXCFtCSgkOBLhdYbMppKUYDoZ8Zz7K430qL/08rEozVPVvENFPA/iNAL4bVqYNAP4jAP8NDArfDeA/Vqtk8ZNE9M1E9EX/Oc9/Bz7QptC3aS7JhMEsIKzZJIWAQ4NEfKn7+KkR8V3bM/XhBSDE42pkzPX0RG242hS28OaLoTGhcHlZ4yjD4qY2MMDzojIMgHyRh6RAh88FGCZKnQgJIcDwLiAMlPogT67rMEhj4xN35DVOISMaCwrzNkAuKcQGJe8NshBt9iaLGw7DriBVr0EFS6RBQCBz1TjnbK+uEoS9oNQEuMcBJSHM8D4ggUDS9g4GS+/5eEAAXmlT8KYwfy+A/w7Ab2gL/f+EqReAAeP/aE/7OZ97GQofYlNo9oRzupExpIR5AcMsGEhIBn0/nwChg+Gl8bNz72tTALZchzAyPtgVmm3hwaZwsSV0Q2NAYQG4haRAliy1CJgX4PAsMGRNhdVg0G0FfR9A6DCI8TMgaNtfJYV8Y66f+B64VBIDQ0dzS7rEsA52GFQgU4Y53wboNkC3AwiJgMLACMxw5VIV6EkoyHL1YWGtZYs/oDAXZC6sHItLAOqNv2rxh5ch5nGdD2nBgcAOA/JELvqIqgPwCigQ0TfB6i/+a6r6y9TyaFVVKfp6vf/Py74P3/ItX4L6MMuWEGAIIFxh8ACHKxCuC/0l8fhdc1fvQwdCtynEPmCAl4GwJUU98T5cXZLMwBrAzV/KQRdJoQOh/Q5mZJYkrwYG36frIiSEq9Wyw+Clb27A4JmKscGhl7mj3QNBvAUupcEx4hOGqw9NhaDj2CQFbVCICkjTcx+WdPVBPTPTmt/KcijENhfkXFgOhtgi7qAv9jqGux7hdoR2jahXwPIIRlEv31lw4HcvkQ9+vBcUiOgGA8IfV9X/1Kd/IdQCIvoigP/L5/8KgG9tT/9NPrc9et+Hv/PvIv1gQ2NXHwIKL2wyn0gIbXtrHsN1e9c1HQjPDI4NAl2FeGpsfEmFaEbGWzsO9SHtCCi74CElJVyNmNGxjpY9PzvaTf8bBqzke6gPJ1qAA0oVeJvK0N8Peftz8r54DVyKzEh+khA1OGMU2F2SCYVb7A+XFCL12jwNUZ3LuoSHpBCqw3LVYUFkuqQwTTo4J+T049OP58yFHwue3dMVc+ZlKGkhgJDX+/FQXEKc8dUNc3Zvwg8B+GlV/SPt1J8G8C8C+EO+/8/a/O8ioj8BMzD+9bfZE4APNDQuVx/mDoQNDudFYpiwAJJ5kRQcBDpRPvjrAn/X3EvXxAJ4y+MKhh5mnPsuGVwkhAdJgXcohOowCVYYmi4weGJc7C0uyWFASRaUqnCFwrM7/lVliPdlvPy8ckXW/up5SLvC2O0Ka1jp9wDCcuMixd43qNcyUIUIY4ngdEnBPk6XEjYwhIQwsZYt/jV9fz8NDueJdc5c1NYsXCtQzlWDjEdw3cUbnHnuQ0kItwQCrFgxgPFVjlP4BwD88wD+RyL6H3zu98Jg8KNE9P0A/jKs0SwA/DjMHfkzMJfk973rF7zW0AgUENLgGDA4m3TgAFgOAw0YnA0I7kEIIKTE8CwY5333L9kTntgUci0ECPCyx2GLbLzYFXpdhZAUJhwgBBzL9+Kqx3r8WVlspUsK/197VxNqyVGFv9Pd9703aEBjJAwx6ESyyUqHELII2QhqshndZWUWgpsIunAxkk22CroQRFAMRBGzUTEbwR8EcWE0ymQyMUwSNaDDmFEEFWVe960+LupU96m61X27789031gf1K3q6rrdp7uqvz7n1E/XLTl4xGfH/sQf6pgfQZOBDh3/aaZoQ82QDPwJzskYmxDlTAdTCBkURaMhkKRZ+v+NTI82dWY/CkyttqB9CqY2QgxLIYUK9bISTUHiso0z0Q5yGW+QKwKw/gLpmZA0xGxoNAT5byHaQs52eLNdunPCLklm/iVWq9nhQ5HyDOCJMUKMXU8BHHRHOgdjBV9DUHGtSMIRBKvYIwTtGwjD2PyY6QDlU6DgeYr4FGK9EM2kqLAHQsghF1KwHzZtiSFfWEJwU6/zWnU9BmSQOVKQ63MEAWdCxPwCQT2tmE/6fvX835kO7ahG1fsgax806yo4v4Ia4uzWUaDGjLBpEvMBC7uwhGHbm2DqDCYn62gkNGsmGsj3SGUpOGs6LFGbCmZpgyWGSoihQn1awZSVrIcAGVLPXpvIalazdFtiyJib9TItKTAWjhAYbqnOyTWFvaNm4L9DV4AQlNVqOL0Z5C2BqmqDiYRaBW9wUmj76gbdtS+MTSSf0TR+9xx4JoJ66BdZ++Zf5MCRhOMCOJb4pABOFsCZRRufkS9F2bnRbeAaOFmimQzmhoG7tSgWtfXVFNIgsyCQJrol4hPGdCgk1o5Id9HuXugpmh4xxLoh7YNv8kw94Ko3YbEAHxXgowX4WMLJEfjkCPWZI5iTY5gzxzY+OQbXNUwujkWyzsSqyGAMwWRsuytR23Uha0cEJcyyRF2WMKdtqE9Xt3MhhQXbt71hm3aL+jhHIrMjhNaHYP9jTYcT024XzCjYOpD3hVmQwq58CivOxgqoS9EOSp8AWAU3l4E1MYQP8i62FblrTcG9EEPToXOcAgWmg9YUtPlAbWhGQAKNWprXVqNwsX741dfrWm1BBzfGoEs7CM0FA58cdDo0QTQ8Ygj9Cr6TkYKxCp52IPaVNR1y0FEBNjWMMTBLa26YPLOORnE2WktJTAfxJzSmQ23NB6stlDBViboqYcpTG05LFG6WFANgbh98IQI3fNqVUR8HszMiGS1BsCWIhRDCW58UsAOfglGmQ5c2ULaEUJc+MXA46Sl8mENn2ZDQYTo4kNIYmjEKajtqQoS+BOVTWKjQkEJmTTP7llGkwG2cBSTgiMF9it5dB6n8pusxrMg+UtDE0EcIWluQG9V8vDezX5pu061vIXPmg1qD0WoVhSUF52AU84HJyCzKDGZJMHlmHY3iUzDNtxqsmeERw7ISUihhqgrGIwQb2JECVO+BhNztEsJwpNB8EwhoSKEQE+IILSEcxZvUTjAPUtjAp7AMfQpBD4Qjg4YYyoAYHCmUrbbQkIPTFEIiQE9eXzrYJvUAkCYDSTdrNAbOxU5HY96aGi7tSKFiYEFAlQkxQMhBEUJDDEHsSEHHqG19rUzo7yLFkBQcMQxwUrZLsblYORyzdmxCphyNDRHkrU/BFJYYsFD+hEVhF0IxS6spFHnjUzCZJgTraFzK4KWlG6dQ214Ho0wKUwkplDctKbiHHS0p5GhcCO03IoQMACEFMPTnPgrRFhYAjiQ+jj8ZO8E8SAEjNQX2CaEyQXdkqCUoQtBkUDuzQfK9KdFOMGyY1ttdmoKKvanTwOrApUBLiI1s1BOisgyoyDamhhDcceQNlAk5uDdX831bpco2X+NTWg+xukf6WkNCiI3tiGkKLkRuDgtThKMZG41BzAfKWlIgGeZMojFAfA8oCqVSFWCCEEKGZU5iPpBoCY4YaqUp1I2mYKT3YSnmQ12dwlQlltUp6vIUy/ImuJYxBU03onAie1ZF0zwIaFbRtv9ricFpCAsAx6I1TNb7cCuwjU/BEUIVG7AUEkMQnBkRXV1pz5DethVCiA1z9qZPZ+14hS5CcJrCgq2mULCUkbgxH9gnCG95d6fOKmJQn+v0NYUuDSEkBD0SUo9x6PInyA1ypkOzcGuz8rKQQEbWn+C2ZYyCWwNfk0OjJSzsR2hNJVOrc6UpOFIQTWEZmA5Gxii43oelZz7chDm1AXXtcZ7jwq5OKYqUzeGbDC4cx27XjjALUtgWrBPhnV7d9Hcw/G+O3iI09uRAUKQF9D1LsXLhfhoig1bnh5RfI0sn1h63fwWB2PU12gY530T8H/5xqcmLNCVPGvemj5di7zi7wr6IQCNU2g4Du36Ib8WdHnjenYky9EBjZaCOdBfW1ZXWnXvRX6A9Tbxc179XxVvfuLwSMWcrhr1ouEeu2KF1/j6b7GGSQscd6auH3jraNZ1vgZ2JseZAzS2MlOu9j+tacueJNtkfe4evO1b8rd318KuOjqFn8R9Kz2/EXtaQ42zaZvfZXA+TFDrgVWfwoHdWdacOPg0GvTmCQtEGsuZ6Bqvi4b51LXnMiYbs7wNFk9GCq2KTF/k26IjmQKsx6e0tsIkWsQscJCl0qWYbMeuUWkLoZcLAaxgi79AyEUJcq1WNwVaagn/ilVOzn+x+nHv2DDZfhqD1R/g5myFpCiMQc7rFC0aTcUxtPsj5N9EUNsbKW3JA8bFa1VBNYfD97zl55zGoc1efGdWHToLigQ7cAUWmUl4PkhS60HcTeytgAvNhuOMrsm9IA97C6z+aQLfRc0d7NeM2YVuFPOqaVotSJDUAQ5kgeqY4NjXxtsVbihQ2uYkb1OXuEGnzm1T2rhvI6NuxjZ67ja3C0eTAI3V3NrrfQfch7IogRLo/d49kPoTouCObaAo8lY4WYkAtjzIfxrjA12f5xw0L7E1TGFnU2TeDjmkLRU3RbXpXeLu/72LftjhIUhj0IFPvpjpY3855oct82OqtEbn+tcfb5Wtq8LG6Bwh15g3ofYo6GocrKNENXsnvPk7qktwRRjisY8necrcUO3JTb81pwfUP1rhGOwkjCLv0egqunGadx1l5ETvNhw5n67q39IrWpshl137gsfu2xUGSwiYY7M+aChMQU5ft2+ubifn+9nb/1g224Mim+k+zOzZOISiz7TU408VxzYDj7au7clsQTzHwPxSC6G8A/gPg71PLsgXuwGHLDxz+NRy6/MB+r+G9zPzudYVmQQoAQEQvMPP9U8uxKQ5dfuDwr+HQ5QfmcQ3/N+ZDQkLCMCRSSEhI8DAnUvj61AJsiUOXHzj8azh0+YEZXMNsfAoJCQnzwJw0hYSEhBlgclIgoo8S0VUiep2ILk4tz1AQ0RtE9BIRXSKiFyTvdiL6CRG9JvE7p5ZTg4ieJqIbRHRF5UVlJouvSL1cJqLz00neyBqT/ykiuib1cImIHlX7Pi/yXyWij0wjdQsiupuIfk5Evyeil4noM5I/rzpg5skC7LqUfwBwD+x6lC8CuG9KmUbI/gaAO4K8LwK4KOmLAL4wtZyBfA8DOA/gyjqZYb8H+iPYMTYPAnh+pvI/BeBzkbL3SXs6BnBO2lk+sfxnAZyX9G0AXhU5Z1UHU2sKDwB4nZn/yMwlgGcBXJhYpm1wAcAzkn4GwMcmlGUFzPwLAP8IsrtkvgDgW2zxKwDvIKKzt0bSODrk78IFAM8y8ykz/wn2g8cP7E24AWDm68z8O0n/G8ArAO7CzOpgalK4C8Cf1fZfJO8QwAB+TES/JaJPSd6dzHxd0n8FcOc0oo1Cl8yHVDefFvX6aWWyzVp+InofgA8CeB4zq4OpSeGQ8RAznwfwCIAniOhhvZOt/ndQXTuHKDOArwF4P4APALgO4EvTirMeRPR2AN8D8Flm/pfeN4c6mJoUrgG4W22/R/JmD2a+JvENAD+AVU3fdOqdxDemk3AwumQ+iLph5jeZ2TBzDeAbaE2EWcpPRAtYQvgOM39fsmdVB1OTwm8A3EtE54joCMBjAJ6bWKa1IKK3EdFtLg3gwwCuwMr+uBR7HMAPp5FwFLpkfg7AJ8QD/iCAfyoVdzYIbOyPw9YDYOV/jIiOiegcgHsB/PpWy6dBRATgmwBeYeYvq13zqoMpvbHKw/oqrHf4yanlGSjzPbCe7RcBvOzkBvAuAD8D8BqAnwK4fWpZA7m/C6tiV7D26Se7ZIb1eH9V6uUlAPfPVP5vi3yXYR+is6r8kyL/VQCPzED+h2BNg8sALkl4dG51kEY0JiQkeJjafEhISJgZEikkJCR4SKSQkJDgIZFCQkKCh0QKCQkJHhIpJCQkeEikkJCQ4CGRQkJCgof/ATLmBjNkE7qwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2167fec8438>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Single image prediction\n",
    "import cv2\n",
    "import matplotlib.pyplot as plt\n",
    "test=cv2.imread(test_images[0])\n",
    "\n",
    "img_show=test[:,:,[2,1,0]]\n",
    "test=test/255.\n",
    "test_shape=(1,)+test.shape\n",
    "test=test.reshape(test_shape)\n",
    "\n",
    "res=xception_model.predict(test)\n",
    "\n",
    "prob=res[0,np.argmax(res,axis=1)[0]]\n",
    "res=label[np.argmax(res,axis=1)[0]]\n",
    "print('Predicted result for the first image: %s'%res)\n",
    "print('Confidence level: %s'%prob)\n",
    "plt.imshow(img_show)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 1min 26s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range(length):\n",
    "    inputimg=test_images[i]\n",
    "    test_batch=[]\n",
    "    thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "    #print(thisimg)\n",
    "    test_shape=(1,)+thisimg.shape\n",
    "    thisimg=thisimg.reshape(test_shape)\n",
    "    xception_model_batch=xception_model.predict(thisimg) #use master model to process the input image\n",
    "    #generate result by model 1\n",
    "    prob=xception_model_batch[0,np.argmax(xception_model_batch,axis=1)[0]]\n",
    "    res=label[np.argmax(xception_model_batch,axis=1)[0]]\n",
    "    predict.append(res)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Xception accuracy: 1.0\n",
      "precision: 1.0\n",
      "recall: 1.0\n",
      "f1: 1.0\n",
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "pre=precision_score(test_laels,predict,average='weighted')\n",
    "re=recall_score(test_laels,predict,average='weighted')\n",
    "f1=f1_score(test_laels,predict,average='weighted')\n",
    "print('Xception accuracy: %s'%acc)\n",
    "print('precision: %s'%pre)\n",
    "print('recall: %s'%re)\n",
    "print('f1: %s'%f1)\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. VGG16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 50 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range(length):\n",
    "    inputimg=test_images[i]\n",
    "    test_batch=[]\n",
    "    thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "    #print(thisimg)\n",
    "    test_shape=(1,)+thisimg.shape\n",
    "    thisimg=thisimg.reshape(test_shape)\n",
    "    vgg_model_batch=vgg_model.predict(thisimg) #use master model to process the input image\n",
    "    #generate result by model 1\n",
    "    prob=vgg_model_batch[0,np.argmax(vgg_model_batch,axis=1)[0]]\n",
    "    res=label[np.argmax(vgg_model_batch,axis=1)[0]]\n",
    "    predict.append(res)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "VGG16 accuracy: 1.0\n",
      "precision: 1.0\n",
      "recall: 1.0\n",
      "f1: 1.0\n",
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "pre=precision_score(test_laels,predict,average='weighted')\n",
    "re=recall_score(test_laels,predict,average='weighted')\n",
    "f1=f1_score(test_laels,predict,average='weighted')\n",
    "print('VGG16 accuracy: %s'%acc)\n",
    "print('precision: %s'%pre)\n",
    "print('recall: %s'%re)\n",
    "print('f1: %s'%f1)\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. VGG19"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 56.6 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range(length):\n",
    "    inputimg=test_images[i]\n",
    "    test_batch=[]\n",
    "    thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "    #print(thisimg)\n",
    "    test_shape=(1,)+thisimg.shape\n",
    "    thisimg=thisimg.reshape(test_shape)\n",
    "    vgg19_model_batch=vgg19_model.predict(thisimg) #use master model to process the input image\n",
    "    #generate result by model 1\n",
    "    prob=vgg19_model_batch[0,np.argmax(vgg19_model_batch,axis=1)[0]]\n",
    "    res=label[np.argmax(vgg19_model_batch,axis=1)[0]]\n",
    "    predict.append(res)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "VGG19 accuracy: 1.0\n",
      "precision: 1.0\n",
      "recall: 1.0\n",
      "f1: 1.0\n",
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "pre=precision_score(test_laels,predict,average='weighted')\n",
    "re=recall_score(test_laels,predict,average='weighted')\n",
    "f1=f1_score(test_laels,predict,average='weighted')\n",
    "print('VGG19 accuracy: %s'%acc)\n",
    "print('precision: %s'%pre)\n",
    "print('recall: %s'%re)\n",
    "print('f1: %s'%f1)\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Inception"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 2min 19s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range(length):\n",
    "    inputimg=test_images[i]\n",
    "    test_batch=[]\n",
    "    thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "    #print(thisimg)\n",
    "    test_shape=(1,)+thisimg.shape\n",
    "    thisimg=thisimg.reshape(test_shape)\n",
    "    incep_model_batch=incep_model.predict(thisimg) #use master model to process the input image\n",
    "    #generate result by model 1\n",
    "    prob=incep_model_batch[0,np.argmax(incep_model_batch,axis=1)[0]]\n",
    "    res=label[np.argmax(incep_model_batch,axis=1)[0]]\n",
    "    predict.append(res)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "inception accuracy: 1.0\n",
      "precision: 1.0\n",
      "recall: 1.0\n",
      "f1: 1.0\n",
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "pre=precision_score(test_laels,predict,average='weighted')\n",
    "re=recall_score(test_laels,predict,average='weighted')\n",
    "f1=f1_score(test_laels,predict,average='weighted')\n",
    "print('inception accuracy: %s'%acc)\n",
    "print('precision: %s'%pre)\n",
    "print('recall: %s'%re)\n",
    "print('f1: %s'%f1)\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. InceptionResnet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 4min 25s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range(length):\n",
    "    inputimg=test_images[i]\n",
    "    test_batch=[]\n",
    "    thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "    #print(thisimg)\n",
    "    test_shape=(1,)+thisimg.shape\n",
    "    thisimg=thisimg.reshape(test_shape)\n",
    "    inres_model_batch=inres_model.predict(thisimg) #use master model to process the input image\n",
    "    #generate result by model 1\n",
    "    prob=inres_model_batch[0,np.argmax(inres_model_batch,axis=1)[0]]\n",
    "    res=label[np.argmax(inres_model_batch,axis=1)[0]]\n",
    "    predict.append(res)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "inceptionresnet accuracy: 0.9998289136013687\n",
      "precision: 0.999829777674089\n",
      "recall: 0.9998289136013687\n",
      "f1: 0.9998288793066084\n",
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    1  170]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       0.99      1.00      1.00       197\n",
      "           4       1.00      0.99      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "pre=precision_score(test_laels,predict,average='weighted')\n",
    "re=recall_score(test_laels,predict,average='weighted')\n",
    "f1=f1_score(test_laels,predict,average='weighted')\n",
    "print('inceptionresnet accuracy: %s'%acc)\n",
    "print('precision: %s'%pre)\n",
    "print('recall: %s'%re)\n",
    "print('f1: %s'%f1)\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 6. Resnet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    " #load model 6: resnet\n",
    "res_model=load_model('./resnet.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 1min 38s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range(length):\n",
    "    inputimg=test_images[i]\n",
    "    test_batch=[]\n",
    "    thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "    #print(thisimg)\n",
    "    test_shape=(1,)+thisimg.shape\n",
    "    thisimg=thisimg.reshape(test_shape)\n",
    "    res_model_batch=res_model.predict(thisimg) #use master model to process the input image\n",
    "    #generate result by model 1\n",
    "    prob=res_model_batch[0,np.argmax(res_model_batch,axis=1)[0]]\n",
    "    res=label[np.argmax(res_model_batch,axis=1)[0]]\n",
    "    predict.append(res)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "resnet accuracy: 0.9662959794696322\n",
      "precision: 0.9338569031275825\n",
      "recall: 0.9662959794696322\n",
      "f1: 0.9497662530625438\n",
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [ 197    0    0    0    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.96      1.00      0.98      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       0.00      0.00      0.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           0.97      5845\n",
      "   macro avg       0.79      0.80      0.80      5845\n",
      "weighted avg       0.93      0.97      0.95      5845\n",
      "\n",
      "Wall time: 35.9 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "pre=precision_score(test_laels,predict,average='weighted')\n",
    "re=recall_score(test_laels,predict,average='weighted')\n",
    "f1=f1_score(test_laels,predict,average='weighted')\n",
    "print('resnet accuracy: %s'%acc)\n",
    "print('precision: %s'%pre)\n",
    "print('recall: %s'%re)\n",
    "print('f1: %s'%f1)\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Best performing single model (vgg):  \n",
    "Accuracy: 99.96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bagging ensemble"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The testing time is :99.662228 seconds\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range((length//127)+1):\n",
    "    inputimg=test_images[127*i:127*(i+1)]\n",
    "    test_batch=[]\n",
    "    for path in inputimg:\n",
    "        thisimg=np.array(Image.open(path))/255\n",
    "        test_batch.append(thisimg)\n",
    "    #generate result by model 1\n",
    "    xception_model_batch=xception_model.predict(np.array(test_batch))\n",
    "    xception_model_batch=list(np.argmax(xception_model_batch,axis=1))\n",
    "    xception_model_batch=[label[con] for con in xception_model_batch]\n",
    "#     print(xception_model_batch)\n",
    "    #generate result by model 2\n",
    "    vgg_model_batch=vgg_model.predict(np.array(test_batch))\n",
    "    vgg_model_batch=list(np.argmax(vgg_model_batch,axis=1))\n",
    "    vgg_model_batch=[label[con] for con in vgg_model_batch]\n",
    "#     print(vgg_model_batch)\n",
    "    #generate result by model 3\n",
    "    vgg19_model_batch=vgg19_model.predict(np.array(test_batch))\n",
    "    vgg19_model_batch=list(np.argmax(vgg19_model_batch,axis=1))\n",
    "    vgg19_model_batch=[label[con] for con in vgg19_model_batch]\n",
    "#     print(vgg19_model_batch)\n",
    "    #generate result by model 4\n",
    "    incep_model_batch=incep_model.predict(np.array(test_batch))\n",
    "    incep_model_batch=list(np.argmax(incep_model_batch,axis=1))\n",
    "    incep_model_batch=[label[con] for con in incep_model_batch]\n",
    "#     print(incep_model_batch)\n",
    "    #generate result by model 5\n",
    "    inres_model_batch=inres_model.predict(np.array(test_batch))\n",
    "    inres_model_batch=list(np.argmax(inres_model_batch,axis=1))\n",
    "    inres_model_batch=[label[con] for con in inres_model_batch]\n",
    "#     print(inres_model_batch)\n",
    "    #bagging the three results generated by 3 singular models\n",
    "    predict_batch=[]\n",
    "    for i,j,k,p,q in zip(xception_model_batch,vgg_model_batch,vgg19_model_batch,incep_model_batch,inres_model_batch):\n",
    "        count=defaultdict(int)\n",
    "        count[i]+=1\n",
    "        count[j]+=1\n",
    "        count[k]+=1\n",
    "        count[p]+=1\n",
    "        count[q]+=1\n",
    "        #rank the predicted results in descending order\n",
    "        predict_one=sorted(count.items(), key=operator.itemgetter(1),reverse=True)[0][0]\n",
    "        predict_batch.append(predict_one)\n",
    "#     print('predict:',predict_batch)\n",
    "    predict.append(predict_batch)\n",
    "t2 = time.time()\n",
    "print('The testing time is :%f seconds' % (t2-t1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predict=sum(predict,[])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bagging accuracy:1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "print('bagging accuracy:%s'%acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "After bagging ensemble, the accuracy improved to 0.990"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Probability Averaging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import keras\n",
    "from keras.models import Model,load_model\n",
    "from keras import Input\n",
    "from keras.layers import concatenate,Dense,Flatten,Dropout,Average\n",
    "from keras.preprocessing.image import  ImageDataGenerator\n",
    "import keras.callbacks as kcallbacks\n",
    "import os\n",
    "import math\n",
    "from keras.utils import plot_model\n",
    "from keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint, LearningRateScheduler\n",
    "from keras.optimizers import SGD\n",
    "import operator\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "from collections import defaultdict\n",
    "import tensorflow as tf\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)\n",
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The testing time is :2.661651 seconds\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "t1 = time.time()\n",
    "img=Input(shape=(224,224,3),name='img')\n",
    "feature1=xception_model(img)\n",
    "feature2=vgg_model(img)\n",
    "feature3=incep_model(img)\n",
    "for layer in xception_model.layers:  \n",
    "    layer.trainable = False \n",
    "for layer in vgg_model.layers:  \n",
    "    layer.trainable = False  \n",
    "for layer in incep_model.layers:  \n",
    "    layer.trainable = False  \n",
    "output=Average()([feature1,feature2,feature3]) #add the confidence lists generated by 3 models\n",
    "model=Model(inputs=img,outputs=output)\n",
    "\n",
    "#the optimization function\n",
    "opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=opt,\n",
    "              metrics=['accuracy'])\n",
    "t2 = time.time()\n",
    "print('The testing time is :%f seconds' % (t2-t1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 ./test_224/0\\100015.png\n"
     ]
    }
   ],
   "source": [
    "#read images from validation folder\n",
    "rootdir = './test_224/'\n",
    "test_laels = []\n",
    "test_images=[]\n",
    "for subdir, dirs, files in os.walk(rootdir):\n",
    "    for file in files:\n",
    "        if not (file.endswith(\".jpeg\"))|(file.endswith(\".jpg\"))|(file.endswith(\".png\")):\n",
    "            continue\n",
    "        test_laels.append(subdir.split('/')[-1])\n",
    "        test_images.append(os.path.join(subdir, file))\n",
    "        \n",
    "print(test_laels[0],test_images[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The testing time is :51.721883 seconds\n"
     ]
    }
   ],
   "source": [
    "#test the averaging model on the validation set\n",
    "import time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range((length//127)+1):\n",
    "    inputimg=test_images[127*i:127*(i+1)]\n",
    "    test_batch=[]\n",
    "    for path in inputimg:\n",
    "        thisimg=np.array(Image.open(path))/255\n",
    "        test_batch.append(thisimg)\n",
    "    #print(i, np.array(test_batch).shape)\n",
    "    model_batch=model.predict(np.array(test_batch))\n",
    "    predict_batch=list(np.argmax(model_batch,axis=1))\n",
    "    predict_batch=[label[con] for con in predict_batch]\n",
    "    predict.append(predict_batch)\n",
    "\n",
    "predict=sum(predict,[])\n",
    "\n",
    "t2 = time.time()\n",
    "print('The testing time is :%f seconds' % (t2-t1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability Averaging accuracy:1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "print('Probability Averaging accuracy:%s'%acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Concatenation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import keras\n",
    "from keras.models import Model,load_model\n",
    "from keras import Input\n",
    "from keras.layers import concatenate,Dense,Flatten,Dropout\n",
    "from keras.preprocessing.image import  ImageDataGenerator\n",
    "import keras.callbacks as kcallbacks\n",
    "import os\n",
    "import math\n",
    "from keras.utils import plot_model\n",
    "from keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint, LearningRateScheduler\n",
    "from keras.optimizers import SGD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 input_1\n",
      "1 block1_conv1\n",
      "2 block1_conv1_bn\n",
      "3 block1_conv1_act\n",
      "4 block1_conv2\n",
      "5 block1_conv2_bn\n",
      "6 block1_conv2_act\n",
      "7 block2_sepconv1\n",
      "8 block2_sepconv1_bn\n",
      "9 block2_sepconv2_act\n",
      "10 block2_sepconv2\n",
      "11 block2_sepconv2_bn\n",
      "12 conv2d_15\n",
      "13 block2_pool\n",
      "14 batch_normalization_1\n",
      "15 add_1\n",
      "16 block3_sepconv1_act\n",
      "17 block3_sepconv1\n",
      "18 block3_sepconv1_bn\n",
      "19 block3_sepconv2_act\n",
      "20 block3_sepconv2\n",
      "21 block3_sepconv2_bn\n",
      "22 conv2d_16\n",
      "23 block3_pool\n",
      "24 batch_normalization_2\n",
      "25 add_2\n",
      "26 block4_sepconv1_act\n",
      "27 block4_sepconv1\n",
      "28 block4_sepconv1_bn\n",
      "29 block4_sepconv2_act\n",
      "30 block4_sepconv2\n",
      "31 block4_sepconv2_bn\n",
      "32 conv2d_17\n",
      "33 block4_pool\n",
      "34 batch_normalization_3\n",
      "35 add_3\n",
      "36 block5_sepconv1_act\n",
      "37 block5_sepconv1\n",
      "38 block5_sepconv1_bn\n",
      "39 block5_sepconv2_act\n",
      "40 block5_sepconv2\n",
      "41 block5_sepconv2_bn\n",
      "42 block5_sepconv3_act\n",
      "43 block5_sepconv3\n",
      "44 block5_sepconv3_bn\n",
      "45 add_4\n",
      "46 block6_sepconv1_act\n",
      "47 block6_sepconv1\n",
      "48 block6_sepconv1_bn\n",
      "49 block6_sepconv2_act\n",
      "50 block6_sepconv2\n",
      "51 block6_sepconv2_bn\n",
      "52 block6_sepconv3_act\n",
      "53 block6_sepconv3\n",
      "54 block6_sepconv3_bn\n",
      "55 add_5\n",
      "56 block7_sepconv1_act\n",
      "57 block7_sepconv1\n",
      "58 block7_sepconv1_bn\n",
      "59 block7_sepconv2_act\n",
      "60 block7_sepconv2\n",
      "61 block7_sepconv2_bn\n",
      "62 block7_sepconv3_act\n",
      "63 block7_sepconv3\n",
      "64 block7_sepconv3_bn\n",
      "65 add_6\n",
      "66 block8_sepconv1_act\n",
      "67 block8_sepconv1\n",
      "68 block8_sepconv1_bn\n",
      "69 block8_sepconv2_act\n",
      "70 block8_sepconv2\n",
      "71 block8_sepconv2_bn\n",
      "72 block8_sepconv3_act\n",
      "73 block8_sepconv3\n",
      "74 block8_sepconv3_bn\n",
      "75 add_7\n",
      "76 block9_sepconv1_act\n",
      "77 block9_sepconv1\n",
      "78 block9_sepconv1_bn\n",
      "79 block9_sepconv2_act\n",
      "80 block9_sepconv2\n",
      "81 block9_sepconv2_bn\n",
      "82 block9_sepconv3_act\n",
      "83 block9_sepconv3\n",
      "84 block9_sepconv3_bn\n",
      "85 add_8\n",
      "86 block10_sepconv1_act\n",
      "87 block10_sepconv1\n",
      "88 block10_sepconv1_bn\n",
      "89 block10_sepconv2_act\n",
      "90 block10_sepconv2\n",
      "91 block10_sepconv2_bn\n",
      "92 block10_sepconv3_act\n",
      "93 block10_sepconv3\n",
      "94 block10_sepconv3_bn\n",
      "95 add_9\n",
      "96 block11_sepconv1_act\n",
      "97 block11_sepconv1\n",
      "98 block11_sepconv1_bn\n",
      "99 block11_sepconv2_act\n",
      "100 block11_sepconv2\n",
      "101 block11_sepconv2_bn\n",
      "102 block11_sepconv3_act\n",
      "103 block11_sepconv3\n",
      "104 block11_sepconv3_bn\n",
      "105 add_10\n",
      "106 block12_sepconv1_act\n",
      "107 block12_sepconv1\n",
      "108 block12_sepconv1_bn\n",
      "109 block12_sepconv2_act\n",
      "110 block12_sepconv2\n",
      "111 block12_sepconv2_bn\n",
      "112 block12_sepconv3_act\n",
      "113 block12_sepconv3\n",
      "114 block12_sepconv3_bn\n",
      "115 add_11\n",
      "116 block13_sepconv1_act\n",
      "117 block13_sepconv1\n",
      "118 block13_sepconv1_bn\n",
      "119 block13_sepconv2_act\n",
      "120 block13_sepconv2\n",
      "121 block13_sepconv2_bn\n",
      "122 conv2d_18\n",
      "123 block13_pool\n",
      "124 batch_normalization_4\n",
      "125 add_12\n",
      "126 block14_sepconv1\n",
      "127 block14_sepconv1_bn\n",
      "128 block14_sepconv1_act\n",
      "129 block14_sepconv2\n",
      "130 block14_sepconv2_bn\n",
      "131 block14_sepconv2_act\n",
      "132 global_average_pooling2d_3\n",
      "133 dense_5\n",
      "134 dropout_3\n",
      "135 dense_6\n"
     ]
    }
   ],
   "source": [
    "for i,layer in enumerate(xception_model.layers):\n",
    "    print(i,layer.name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 input_2\n",
      "1 block1_conv1\n",
      "2 block1_conv2\n",
      "3 block1_pool\n",
      "4 block2_conv1\n",
      "5 block2_conv2\n",
      "6 block2_pool\n",
      "7 block3_conv1\n",
      "8 block3_conv2\n",
      "9 block3_conv3\n",
      "10 block3_pool\n",
      "11 block4_conv1\n",
      "12 block4_conv2\n",
      "13 block4_conv3\n",
      "14 block4_pool\n",
      "15 block5_conv1\n",
      "16 block5_conv2\n",
      "17 block5_conv3\n",
      "18 block5_pool\n",
      "19 global_average_pooling2d_4\n",
      "20 dense_7\n",
      "21 dropout_4\n",
      "22 dense_8\n"
     ]
    }
   ],
   "source": [
    "for i,layer in enumerate(vgg_model.layers):\n",
    "    print(i,layer.name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 input_3\n",
      "1 block1_conv1\n",
      "2 block1_conv2\n",
      "3 block1_pool\n",
      "4 block2_conv1\n",
      "5 block2_conv2\n",
      "6 block2_pool\n",
      "7 block3_conv1\n",
      "8 block3_conv2\n",
      "9 block3_conv3\n",
      "10 block3_conv4\n",
      "11 block3_pool\n",
      "12 block4_conv1\n",
      "13 block4_conv2\n",
      "14 block4_conv3\n",
      "15 block4_conv4\n",
      "16 block4_pool\n",
      "17 block5_conv1\n",
      "18 block5_conv2\n",
      "19 block5_conv3\n",
      "20 block5_conv4\n",
      "21 block5_pool\n",
      "22 global_average_pooling2d_5\n",
      "23 dense_9\n",
      "24 dropout_5\n",
      "25 dense_10\n"
     ]
    }
   ],
   "source": [
    "for i,layer in enumerate(vgg19_model.layers):\n",
    "    print(i,layer.name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 input_5\n",
      "1 conv2d_19\n",
      "2 batch_normalization_5\n",
      "3 activation_50\n",
      "4 conv2d_20\n",
      "5 batch_normalization_6\n",
      "6 activation_51\n",
      "7 conv2d_21\n",
      "8 batch_normalization_7\n",
      "9 activation_52\n",
      "10 max_pooling2d_6\n",
      "11 conv2d_22\n",
      "12 batch_normalization_8\n",
      "13 activation_53\n",
      "14 conv2d_23\n",
      "15 batch_normalization_9\n",
      "16 activation_54\n",
      "17 max_pooling2d_7\n",
      "18 conv2d_27\n",
      "19 batch_normalization_13\n",
      "20 activation_58\n",
      "21 conv2d_25\n",
      "22 conv2d_28\n",
      "23 batch_normalization_11\n",
      "24 batch_normalization_14\n",
      "25 activation_56\n",
      "26 activation_59\n",
      "27 average_pooling2d_1\n",
      "28 conv2d_24\n",
      "29 conv2d_26\n",
      "30 conv2d_29\n",
      "31 conv2d_30\n",
      "32 batch_normalization_10\n",
      "33 batch_normalization_12\n",
      "34 batch_normalization_15\n",
      "35 batch_normalization_16\n",
      "36 activation_55\n",
      "37 activation_57\n",
      "38 activation_60\n",
      "39 activation_61\n",
      "40 mixed0\n",
      "41 conv2d_34\n",
      "42 batch_normalization_20\n",
      "43 activation_65\n",
      "44 conv2d_32\n",
      "45 conv2d_35\n",
      "46 batch_normalization_18\n",
      "47 batch_normalization_21\n",
      "48 activation_63\n",
      "49 activation_66\n",
      "50 average_pooling2d_2\n",
      "51 conv2d_31\n",
      "52 conv2d_33\n",
      "53 conv2d_36\n",
      "54 conv2d_37\n",
      "55 batch_normalization_17\n",
      "56 batch_normalization_19\n",
      "57 batch_normalization_22\n",
      "58 batch_normalization_23\n",
      "59 activation_62\n",
      "60 activation_64\n",
      "61 activation_67\n",
      "62 activation_68\n",
      "63 mixed1\n",
      "64 conv2d_41\n",
      "65 batch_normalization_27\n",
      "66 activation_72\n",
      "67 conv2d_39\n",
      "68 conv2d_42\n",
      "69 batch_normalization_25\n",
      "70 batch_normalization_28\n",
      "71 activation_70\n",
      "72 activation_73\n",
      "73 average_pooling2d_3\n",
      "74 conv2d_38\n",
      "75 conv2d_40\n",
      "76 conv2d_43\n",
      "77 conv2d_44\n",
      "78 batch_normalization_24\n",
      "79 batch_normalization_26\n",
      "80 batch_normalization_29\n",
      "81 batch_normalization_30\n",
      "82 activation_69\n",
      "83 activation_71\n",
      "84 activation_74\n",
      "85 activation_75\n",
      "86 mixed2\n",
      "87 conv2d_46\n",
      "88 batch_normalization_32\n",
      "89 activation_77\n",
      "90 conv2d_47\n",
      "91 batch_normalization_33\n",
      "92 activation_78\n",
      "93 conv2d_45\n",
      "94 conv2d_48\n",
      "95 batch_normalization_31\n",
      "96 batch_normalization_34\n",
      "97 activation_76\n",
      "98 activation_79\n",
      "99 max_pooling2d_8\n",
      "100 mixed3\n",
      "101 conv2d_53\n",
      "102 batch_normalization_39\n",
      "103 activation_84\n",
      "104 conv2d_54\n",
      "105 batch_normalization_40\n",
      "106 activation_85\n",
      "107 conv2d_50\n",
      "108 conv2d_55\n",
      "109 batch_normalization_36\n",
      "110 batch_normalization_41\n",
      "111 activation_81\n",
      "112 activation_86\n",
      "113 conv2d_51\n",
      "114 conv2d_56\n",
      "115 batch_normalization_37\n",
      "116 batch_normalization_42\n",
      "117 activation_82\n",
      "118 activation_87\n",
      "119 average_pooling2d_4\n",
      "120 conv2d_49\n",
      "121 conv2d_52\n",
      "122 conv2d_57\n",
      "123 conv2d_58\n",
      "124 batch_normalization_35\n",
      "125 batch_normalization_38\n",
      "126 batch_normalization_43\n",
      "127 batch_normalization_44\n",
      "128 activation_80\n",
      "129 activation_83\n",
      "130 activation_88\n",
      "131 activation_89\n",
      "132 mixed4\n",
      "133 conv2d_63\n",
      "134 batch_normalization_49\n",
      "135 activation_94\n",
      "136 conv2d_64\n",
      "137 batch_normalization_50\n",
      "138 activation_95\n",
      "139 conv2d_60\n",
      "140 conv2d_65\n",
      "141 batch_normalization_46\n",
      "142 batch_normalization_51\n",
      "143 activation_91\n",
      "144 activation_96\n",
      "145 conv2d_61\n",
      "146 conv2d_66\n",
      "147 batch_normalization_47\n",
      "148 batch_normalization_52\n",
      "149 activation_92\n",
      "150 activation_97\n",
      "151 average_pooling2d_5\n",
      "152 conv2d_59\n",
      "153 conv2d_62\n",
      "154 conv2d_67\n",
      "155 conv2d_68\n",
      "156 batch_normalization_45\n",
      "157 batch_normalization_48\n",
      "158 batch_normalization_53\n",
      "159 batch_normalization_54\n",
      "160 activation_90\n",
      "161 activation_93\n",
      "162 activation_98\n",
      "163 activation_99\n",
      "164 mixed5\n",
      "165 conv2d_73\n",
      "166 batch_normalization_59\n",
      "167 activation_104\n",
      "168 conv2d_74\n",
      "169 batch_normalization_60\n",
      "170 activation_105\n",
      "171 conv2d_70\n",
      "172 conv2d_75\n",
      "173 batch_normalization_56\n",
      "174 batch_normalization_61\n",
      "175 activation_101\n",
      "176 activation_106\n",
      "177 conv2d_71\n",
      "178 conv2d_76\n",
      "179 batch_normalization_57\n",
      "180 batch_normalization_62\n",
      "181 activation_102\n",
      "182 activation_107\n",
      "183 average_pooling2d_6\n",
      "184 conv2d_69\n",
      "185 conv2d_72\n",
      "186 conv2d_77\n",
      "187 conv2d_78\n",
      "188 batch_normalization_55\n",
      "189 batch_normalization_58\n",
      "190 batch_normalization_63\n",
      "191 batch_normalization_64\n",
      "192 activation_100\n",
      "193 activation_103\n",
      "194 activation_108\n",
      "195 activation_109\n",
      "196 mixed6\n",
      "197 conv2d_83\n",
      "198 batch_normalization_69\n",
      "199 activation_114\n",
      "200 conv2d_84\n",
      "201 batch_normalization_70\n",
      "202 activation_115\n",
      "203 conv2d_80\n",
      "204 conv2d_85\n",
      "205 batch_normalization_66\n",
      "206 batch_normalization_71\n",
      "207 activation_111\n",
      "208 activation_116\n",
      "209 conv2d_81\n",
      "210 conv2d_86\n",
      "211 batch_normalization_67\n",
      "212 batch_normalization_72\n",
      "213 activation_112\n",
      "214 activation_117\n",
      "215 average_pooling2d_7\n",
      "216 conv2d_79\n",
      "217 conv2d_82\n",
      "218 conv2d_87\n",
      "219 conv2d_88\n",
      "220 batch_normalization_65\n",
      "221 batch_normalization_68\n",
      "222 batch_normalization_73\n",
      "223 batch_normalization_74\n",
      "224 activation_110\n",
      "225 activation_113\n",
      "226 activation_118\n",
      "227 activation_119\n",
      "228 mixed7\n",
      "229 conv2d_91\n",
      "230 batch_normalization_77\n",
      "231 activation_122\n",
      "232 conv2d_92\n",
      "233 batch_normalization_78\n",
      "234 activation_123\n",
      "235 conv2d_89\n",
      "236 conv2d_93\n",
      "237 batch_normalization_75\n",
      "238 batch_normalization_79\n",
      "239 activation_120\n",
      "240 activation_124\n",
      "241 conv2d_90\n",
      "242 conv2d_94\n",
      "243 batch_normalization_76\n",
      "244 batch_normalization_80\n",
      "245 activation_121\n",
      "246 activation_125\n",
      "247 max_pooling2d_9\n",
      "248 mixed8\n",
      "249 conv2d_99\n",
      "250 batch_normalization_85\n",
      "251 activation_130\n",
      "252 conv2d_96\n",
      "253 conv2d_100\n",
      "254 batch_normalization_82\n",
      "255 batch_normalization_86\n",
      "256 activation_127\n",
      "257 activation_131\n",
      "258 conv2d_97\n",
      "259 conv2d_98\n",
      "260 conv2d_101\n",
      "261 conv2d_102\n",
      "262 average_pooling2d_8\n",
      "263 conv2d_95\n",
      "264 batch_normalization_83\n",
      "265 batch_normalization_84\n",
      "266 batch_normalization_87\n",
      "267 batch_normalization_88\n",
      "268 conv2d_103\n",
      "269 batch_normalization_81\n",
      "270 activation_128\n",
      "271 activation_129\n",
      "272 activation_132\n",
      "273 activation_133\n",
      "274 batch_normalization_89\n",
      "275 activation_126\n",
      "276 mixed9_0\n",
      "277 concatenate_1\n",
      "278 activation_134\n",
      "279 mixed9\n",
      "280 conv2d_108\n",
      "281 batch_normalization_94\n",
      "282 activation_139\n",
      "283 conv2d_105\n",
      "284 conv2d_109\n",
      "285 batch_normalization_91\n",
      "286 batch_normalization_95\n",
      "287 activation_136\n",
      "288 activation_140\n",
      "289 conv2d_106\n",
      "290 conv2d_107\n",
      "291 conv2d_110\n",
      "292 conv2d_111\n",
      "293 average_pooling2d_9\n",
      "294 conv2d_104\n",
      "295 batch_normalization_92\n",
      "296 batch_normalization_93\n",
      "297 batch_normalization_96\n",
      "298 batch_normalization_97\n",
      "299 conv2d_112\n",
      "300 batch_normalization_90\n",
      "301 activation_137\n",
      "302 activation_138\n",
      "303 activation_141\n",
      "304 activation_142\n",
      "305 batch_normalization_98\n",
      "306 activation_135\n",
      "307 mixed9_1\n",
      "308 concatenate_2\n",
      "309 activation_143\n",
      "310 mixed10\n",
      "311 global_average_pooling2d_7\n",
      "312 dense_13\n",
      "313 dropout_7\n",
      "314 dense_14\n"
     ]
    }
   ],
   "source": [
    "for i,layer in enumerate(incep_model.layers):\n",
    "    print(i,layer.name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 input_6\n",
      "1 conv2d_113\n",
      "2 batch_normalization_99\n",
      "3 activation_144\n",
      "4 conv2d_114\n",
      "5 batch_normalization_100\n",
      "6 activation_145\n",
      "7 conv2d_115\n",
      "8 batch_normalization_101\n",
      "9 activation_146\n",
      "10 max_pooling2d_10\n",
      "11 conv2d_116\n",
      "12 batch_normalization_102\n",
      "13 activation_147\n",
      "14 conv2d_117\n",
      "15 batch_normalization_103\n",
      "16 activation_148\n",
      "17 max_pooling2d_11\n",
      "18 conv2d_121\n",
      "19 batch_normalization_107\n",
      "20 activation_152\n",
      "21 conv2d_119\n",
      "22 conv2d_122\n",
      "23 batch_normalization_105\n",
      "24 batch_normalization_108\n",
      "25 activation_150\n",
      "26 activation_153\n",
      "27 average_pooling2d_10\n",
      "28 conv2d_118\n",
      "29 conv2d_120\n",
      "30 conv2d_123\n",
      "31 conv2d_124\n",
      "32 batch_normalization_104\n",
      "33 batch_normalization_106\n",
      "34 batch_normalization_109\n",
      "35 batch_normalization_110\n",
      "36 activation_149\n",
      "37 activation_151\n",
      "38 activation_154\n",
      "39 activation_155\n",
      "40 mixed_5b\n",
      "41 conv2d_128\n",
      "42 batch_normalization_114\n",
      "43 activation_159\n",
      "44 conv2d_126\n",
      "45 conv2d_129\n",
      "46 batch_normalization_112\n",
      "47 batch_normalization_115\n",
      "48 activation_157\n",
      "49 activation_160\n",
      "50 conv2d_125\n",
      "51 conv2d_127\n",
      "52 conv2d_130\n",
      "53 batch_normalization_111\n",
      "54 batch_normalization_113\n",
      "55 batch_normalization_116\n",
      "56 activation_156\n",
      "57 activation_158\n",
      "58 activation_161\n",
      "59 block35_1_mixed\n",
      "60 block35_1_conv\n",
      "61 block35_1\n",
      "62 block35_1_ac\n",
      "63 conv2d_134\n",
      "64 batch_normalization_120\n",
      "65 activation_165\n",
      "66 conv2d_132\n",
      "67 conv2d_135\n",
      "68 batch_normalization_118\n",
      "69 batch_normalization_121\n",
      "70 activation_163\n",
      "71 activation_166\n",
      "72 conv2d_131\n",
      "73 conv2d_133\n",
      "74 conv2d_136\n",
      "75 batch_normalization_117\n",
      "76 batch_normalization_119\n",
      "77 batch_normalization_122\n",
      "78 activation_162\n",
      "79 activation_164\n",
      "80 activation_167\n",
      "81 block35_2_mixed\n",
      "82 block35_2_conv\n",
      "83 block35_2\n",
      "84 block35_2_ac\n",
      "85 conv2d_140\n",
      "86 batch_normalization_126\n",
      "87 activation_171\n",
      "88 conv2d_138\n",
      "89 conv2d_141\n",
      "90 batch_normalization_124\n",
      "91 batch_normalization_127\n",
      "92 activation_169\n",
      "93 activation_172\n",
      "94 conv2d_137\n",
      "95 conv2d_139\n",
      "96 conv2d_142\n",
      "97 batch_normalization_123\n",
      "98 batch_normalization_125\n",
      "99 batch_normalization_128\n",
      "100 activation_168\n",
      "101 activation_170\n",
      "102 activation_173\n",
      "103 block35_3_mixed\n",
      "104 block35_3_conv\n",
      "105 block35_3\n",
      "106 block35_3_ac\n",
      "107 conv2d_146\n",
      "108 batch_normalization_132\n",
      "109 activation_177\n",
      "110 conv2d_144\n",
      "111 conv2d_147\n",
      "112 batch_normalization_130\n",
      "113 batch_normalization_133\n",
      "114 activation_175\n",
      "115 activation_178\n",
      "116 conv2d_143\n",
      "117 conv2d_145\n",
      "118 conv2d_148\n",
      "119 batch_normalization_129\n",
      "120 batch_normalization_131\n",
      "121 batch_normalization_134\n",
      "122 activation_174\n",
      "123 activation_176\n",
      "124 activation_179\n",
      "125 block35_4_mixed\n",
      "126 block35_4_conv\n",
      "127 block35_4\n",
      "128 block35_4_ac\n",
      "129 conv2d_152\n",
      "130 batch_normalization_138\n",
      "131 activation_183\n",
      "132 conv2d_150\n",
      "133 conv2d_153\n",
      "134 batch_normalization_136\n",
      "135 batch_normalization_139\n",
      "136 activation_181\n",
      "137 activation_184\n",
      "138 conv2d_149\n",
      "139 conv2d_151\n",
      "140 conv2d_154\n",
      "141 batch_normalization_135\n",
      "142 batch_normalization_137\n",
      "143 batch_normalization_140\n",
      "144 activation_180\n",
      "145 activation_182\n",
      "146 activation_185\n",
      "147 block35_5_mixed\n",
      "148 block35_5_conv\n",
      "149 block35_5\n",
      "150 block35_5_ac\n",
      "151 conv2d_158\n",
      "152 batch_normalization_144\n",
      "153 activation_189\n",
      "154 conv2d_156\n",
      "155 conv2d_159\n",
      "156 batch_normalization_142\n",
      "157 batch_normalization_145\n",
      "158 activation_187\n",
      "159 activation_190\n",
      "160 conv2d_155\n",
      "161 conv2d_157\n",
      "162 conv2d_160\n",
      "163 batch_normalization_141\n",
      "164 batch_normalization_143\n",
      "165 batch_normalization_146\n",
      "166 activation_186\n",
      "167 activation_188\n",
      "168 activation_191\n",
      "169 block35_6_mixed\n",
      "170 block35_6_conv\n",
      "171 block35_6\n",
      "172 block35_6_ac\n",
      "173 conv2d_164\n",
      "174 batch_normalization_150\n",
      "175 activation_195\n",
      "176 conv2d_162\n",
      "177 conv2d_165\n",
      "178 batch_normalization_148\n",
      "179 batch_normalization_151\n",
      "180 activation_193\n",
      "181 activation_196\n",
      "182 conv2d_161\n",
      "183 conv2d_163\n",
      "184 conv2d_166\n",
      "185 batch_normalization_147\n",
      "186 batch_normalization_149\n",
      "187 batch_normalization_152\n",
      "188 activation_192\n",
      "189 activation_194\n",
      "190 activation_197\n",
      "191 block35_7_mixed\n",
      "192 block35_7_conv\n",
      "193 block35_7\n",
      "194 block35_7_ac\n",
      "195 conv2d_170\n",
      "196 batch_normalization_156\n",
      "197 activation_201\n",
      "198 conv2d_168\n",
      "199 conv2d_171\n",
      "200 batch_normalization_154\n",
      "201 batch_normalization_157\n",
      "202 activation_199\n",
      "203 activation_202\n",
      "204 conv2d_167\n",
      "205 conv2d_169\n",
      "206 conv2d_172\n",
      "207 batch_normalization_153\n",
      "208 batch_normalization_155\n",
      "209 batch_normalization_158\n",
      "210 activation_198\n",
      "211 activation_200\n",
      "212 activation_203\n",
      "213 block35_8_mixed\n",
      "214 block35_8_conv\n",
      "215 block35_8\n",
      "216 block35_8_ac\n",
      "217 conv2d_176\n",
      "218 batch_normalization_162\n",
      "219 activation_207\n",
      "220 conv2d_174\n",
      "221 conv2d_177\n",
      "222 batch_normalization_160\n",
      "223 batch_normalization_163\n",
      "224 activation_205\n",
      "225 activation_208\n",
      "226 conv2d_173\n",
      "227 conv2d_175\n",
      "228 conv2d_178\n",
      "229 batch_normalization_159\n",
      "230 batch_normalization_161\n",
      "231 batch_normalization_164\n",
      "232 activation_204\n",
      "233 activation_206\n",
      "234 activation_209\n",
      "235 block35_9_mixed\n",
      "236 block35_9_conv\n",
      "237 block35_9\n",
      "238 block35_9_ac\n",
      "239 conv2d_182\n",
      "240 batch_normalization_168\n",
      "241 activation_213\n",
      "242 conv2d_180\n",
      "243 conv2d_183\n",
      "244 batch_normalization_166\n",
      "245 batch_normalization_169\n",
      "246 activation_211\n",
      "247 activation_214\n",
      "248 conv2d_179\n",
      "249 conv2d_181\n",
      "250 conv2d_184\n",
      "251 batch_normalization_165\n",
      "252 batch_normalization_167\n",
      "253 batch_normalization_170\n",
      "254 activation_210\n",
      "255 activation_212\n",
      "256 activation_215\n",
      "257 block35_10_mixed\n",
      "258 block35_10_conv\n",
      "259 block35_10\n",
      "260 block35_10_ac\n",
      "261 conv2d_186\n",
      "262 batch_normalization_172\n",
      "263 activation_217\n",
      "264 conv2d_187\n",
      "265 batch_normalization_173\n",
      "266 activation_218\n",
      "267 conv2d_185\n",
      "268 conv2d_188\n",
      "269 batch_normalization_171\n",
      "270 batch_normalization_174\n",
      "271 activation_216\n",
      "272 activation_219\n",
      "273 max_pooling2d_12\n",
      "274 mixed_6a\n",
      "275 conv2d_190\n",
      "276 batch_normalization_176\n",
      "277 activation_221\n",
      "278 conv2d_191\n",
      "279 batch_normalization_177\n",
      "280 activation_222\n",
      "281 conv2d_189\n",
      "282 conv2d_192\n",
      "283 batch_normalization_175\n",
      "284 batch_normalization_178\n",
      "285 activation_220\n",
      "286 activation_223\n",
      "287 block17_1_mixed\n",
      "288 block17_1_conv\n",
      "289 block17_1\n",
      "290 block17_1_ac\n",
      "291 conv2d_194\n",
      "292 batch_normalization_180\n",
      "293 activation_225\n",
      "294 conv2d_195\n",
      "295 batch_normalization_181\n",
      "296 activation_226\n",
      "297 conv2d_193\n",
      "298 conv2d_196\n",
      "299 batch_normalization_179\n",
      "300 batch_normalization_182\n",
      "301 activation_224\n",
      "302 activation_227\n",
      "303 block17_2_mixed\n",
      "304 block17_2_conv\n",
      "305 block17_2\n",
      "306 block17_2_ac\n",
      "307 conv2d_198\n",
      "308 batch_normalization_184\n",
      "309 activation_229\n",
      "310 conv2d_199\n",
      "311 batch_normalization_185\n",
      "312 activation_230\n",
      "313 conv2d_197\n",
      "314 conv2d_200\n",
      "315 batch_normalization_183\n",
      "316 batch_normalization_186\n",
      "317 activation_228\n",
      "318 activation_231\n",
      "319 block17_3_mixed\n",
      "320 block17_3_conv\n",
      "321 block17_3\n",
      "322 block17_3_ac\n",
      "323 conv2d_202\n",
      "324 batch_normalization_188\n",
      "325 activation_233\n",
      "326 conv2d_203\n",
      "327 batch_normalization_189\n",
      "328 activation_234\n",
      "329 conv2d_201\n",
      "330 conv2d_204\n",
      "331 batch_normalization_187\n",
      "332 batch_normalization_190\n",
      "333 activation_232\n",
      "334 activation_235\n",
      "335 block17_4_mixed\n",
      "336 block17_4_conv\n",
      "337 block17_4\n",
      "338 block17_4_ac\n",
      "339 conv2d_206\n",
      "340 batch_normalization_192\n",
      "341 activation_237\n",
      "342 conv2d_207\n",
      "343 batch_normalization_193\n",
      "344 activation_238\n",
      "345 conv2d_205\n",
      "346 conv2d_208\n",
      "347 batch_normalization_191\n",
      "348 batch_normalization_194\n",
      "349 activation_236\n",
      "350 activation_239\n",
      "351 block17_5_mixed\n",
      "352 block17_5_conv\n",
      "353 block17_5\n",
      "354 block17_5_ac\n",
      "355 conv2d_210\n",
      "356 batch_normalization_196\n",
      "357 activation_241\n",
      "358 conv2d_211\n",
      "359 batch_normalization_197\n",
      "360 activation_242\n",
      "361 conv2d_209\n",
      "362 conv2d_212\n",
      "363 batch_normalization_195\n",
      "364 batch_normalization_198\n",
      "365 activation_240\n",
      "366 activation_243\n",
      "367 block17_6_mixed\n",
      "368 block17_6_conv\n",
      "369 block17_6\n",
      "370 block17_6_ac\n",
      "371 conv2d_214\n",
      "372 batch_normalization_200\n",
      "373 activation_245\n",
      "374 conv2d_215\n",
      "375 batch_normalization_201\n",
      "376 activation_246\n",
      "377 conv2d_213\n",
      "378 conv2d_216\n",
      "379 batch_normalization_199\n",
      "380 batch_normalization_202\n",
      "381 activation_244\n",
      "382 activation_247\n",
      "383 block17_7_mixed\n",
      "384 block17_7_conv\n",
      "385 block17_7\n",
      "386 block17_7_ac\n",
      "387 conv2d_218\n",
      "388 batch_normalization_204\n",
      "389 activation_249\n",
      "390 conv2d_219\n",
      "391 batch_normalization_205\n",
      "392 activation_250\n",
      "393 conv2d_217\n",
      "394 conv2d_220\n",
      "395 batch_normalization_203\n",
      "396 batch_normalization_206\n",
      "397 activation_248\n",
      "398 activation_251\n",
      "399 block17_8_mixed\n",
      "400 block17_8_conv\n",
      "401 block17_8\n",
      "402 block17_8_ac\n",
      "403 conv2d_222\n",
      "404 batch_normalization_208\n",
      "405 activation_253\n",
      "406 conv2d_223\n",
      "407 batch_normalization_209\n",
      "408 activation_254\n",
      "409 conv2d_221\n",
      "410 conv2d_224\n",
      "411 batch_normalization_207\n",
      "412 batch_normalization_210\n",
      "413 activation_252\n",
      "414 activation_255\n",
      "415 block17_9_mixed\n",
      "416 block17_9_conv\n",
      "417 block17_9\n",
      "418 block17_9_ac\n",
      "419 conv2d_226\n",
      "420 batch_normalization_212\n",
      "421 activation_257\n",
      "422 conv2d_227\n",
      "423 batch_normalization_213\n",
      "424 activation_258\n",
      "425 conv2d_225\n",
      "426 conv2d_228\n",
      "427 batch_normalization_211\n",
      "428 batch_normalization_214\n",
      "429 activation_256\n",
      "430 activation_259\n",
      "431 block17_10_mixed\n",
      "432 block17_10_conv\n",
      "433 block17_10\n",
      "434 block17_10_ac\n",
      "435 conv2d_230\n",
      "436 batch_normalization_216\n",
      "437 activation_261\n",
      "438 conv2d_231\n",
      "439 batch_normalization_217\n",
      "440 activation_262\n",
      "441 conv2d_229\n",
      "442 conv2d_232\n",
      "443 batch_normalization_215\n",
      "444 batch_normalization_218\n",
      "445 activation_260\n",
      "446 activation_263\n",
      "447 block17_11_mixed\n",
      "448 block17_11_conv\n",
      "449 block17_11\n",
      "450 block17_11_ac\n",
      "451 conv2d_234\n",
      "452 batch_normalization_220\n",
      "453 activation_265\n",
      "454 conv2d_235\n",
      "455 batch_normalization_221\n",
      "456 activation_266\n",
      "457 conv2d_233\n",
      "458 conv2d_236\n",
      "459 batch_normalization_219\n",
      "460 batch_normalization_222\n",
      "461 activation_264\n",
      "462 activation_267\n",
      "463 block17_12_mixed\n",
      "464 block17_12_conv\n",
      "465 block17_12\n",
      "466 block17_12_ac\n",
      "467 conv2d_238\n",
      "468 batch_normalization_224\n",
      "469 activation_269\n",
      "470 conv2d_239\n",
      "471 batch_normalization_225\n",
      "472 activation_270\n",
      "473 conv2d_237\n",
      "474 conv2d_240\n",
      "475 batch_normalization_223\n",
      "476 batch_normalization_226\n",
      "477 activation_268\n",
      "478 activation_271\n",
      "479 block17_13_mixed\n",
      "480 block17_13_conv\n",
      "481 block17_13\n",
      "482 block17_13_ac\n",
      "483 conv2d_242\n",
      "484 batch_normalization_228\n",
      "485 activation_273\n",
      "486 conv2d_243\n",
      "487 batch_normalization_229\n",
      "488 activation_274\n",
      "489 conv2d_241\n",
      "490 conv2d_244\n",
      "491 batch_normalization_227\n",
      "492 batch_normalization_230\n",
      "493 activation_272\n",
      "494 activation_275\n",
      "495 block17_14_mixed\n",
      "496 block17_14_conv\n",
      "497 block17_14\n",
      "498 block17_14_ac\n",
      "499 conv2d_246\n",
      "500 batch_normalization_232\n",
      "501 activation_277\n",
      "502 conv2d_247\n",
      "503 batch_normalization_233\n",
      "504 activation_278\n",
      "505 conv2d_245\n",
      "506 conv2d_248\n",
      "507 batch_normalization_231\n",
      "508 batch_normalization_234\n",
      "509 activation_276\n",
      "510 activation_279\n",
      "511 block17_15_mixed\n",
      "512 block17_15_conv\n",
      "513 block17_15\n",
      "514 block17_15_ac\n",
      "515 conv2d_250\n",
      "516 batch_normalization_236\n",
      "517 activation_281\n",
      "518 conv2d_251\n",
      "519 batch_normalization_237\n",
      "520 activation_282\n",
      "521 conv2d_249\n",
      "522 conv2d_252\n",
      "523 batch_normalization_235\n",
      "524 batch_normalization_238\n",
      "525 activation_280\n",
      "526 activation_283\n",
      "527 block17_16_mixed\n",
      "528 block17_16_conv\n",
      "529 block17_16\n",
      "530 block17_16_ac\n",
      "531 conv2d_254\n",
      "532 batch_normalization_240\n",
      "533 activation_285\n",
      "534 conv2d_255\n",
      "535 batch_normalization_241\n",
      "536 activation_286\n",
      "537 conv2d_253\n",
      "538 conv2d_256\n",
      "539 batch_normalization_239\n",
      "540 batch_normalization_242\n",
      "541 activation_284\n",
      "542 activation_287\n",
      "543 block17_17_mixed\n",
      "544 block17_17_conv\n",
      "545 block17_17\n",
      "546 block17_17_ac\n",
      "547 conv2d_258\n",
      "548 batch_normalization_244\n",
      "549 activation_289\n",
      "550 conv2d_259\n",
      "551 batch_normalization_245\n",
      "552 activation_290\n",
      "553 conv2d_257\n",
      "554 conv2d_260\n",
      "555 batch_normalization_243\n",
      "556 batch_normalization_246\n",
      "557 activation_288\n",
      "558 activation_291\n",
      "559 block17_18_mixed\n",
      "560 block17_18_conv\n",
      "561 block17_18\n",
      "562 block17_18_ac\n",
      "563 conv2d_262\n",
      "564 batch_normalization_248\n",
      "565 activation_293\n",
      "566 conv2d_263\n",
      "567 batch_normalization_249\n",
      "568 activation_294\n",
      "569 conv2d_261\n",
      "570 conv2d_264\n",
      "571 batch_normalization_247\n",
      "572 batch_normalization_250\n",
      "573 activation_292\n",
      "574 activation_295\n",
      "575 block17_19_mixed\n",
      "576 block17_19_conv\n",
      "577 block17_19\n",
      "578 block17_19_ac\n",
      "579 conv2d_266\n",
      "580 batch_normalization_252\n",
      "581 activation_297\n",
      "582 conv2d_267\n",
      "583 batch_normalization_253\n",
      "584 activation_298\n",
      "585 conv2d_265\n",
      "586 conv2d_268\n",
      "587 batch_normalization_251\n",
      "588 batch_normalization_254\n",
      "589 activation_296\n",
      "590 activation_299\n",
      "591 block17_20_mixed\n",
      "592 block17_20_conv\n",
      "593 block17_20\n",
      "594 block17_20_ac\n",
      "595 conv2d_273\n",
      "596 batch_normalization_259\n",
      "597 activation_304\n",
      "598 conv2d_269\n",
      "599 conv2d_271\n",
      "600 conv2d_274\n",
      "601 batch_normalization_255\n",
      "602 batch_normalization_257\n",
      "603 batch_normalization_260\n",
      "604 activation_300\n",
      "605 activation_302\n",
      "606 activation_305\n",
      "607 conv2d_270\n",
      "608 conv2d_272\n",
      "609 conv2d_275\n",
      "610 batch_normalization_256\n",
      "611 batch_normalization_258\n",
      "612 batch_normalization_261\n",
      "613 activation_301\n",
      "614 activation_303\n",
      "615 activation_306\n",
      "616 max_pooling2d_13\n",
      "617 mixed_7a\n",
      "618 conv2d_277\n",
      "619 batch_normalization_263\n",
      "620 activation_308\n",
      "621 conv2d_278\n",
      "622 batch_normalization_264\n",
      "623 activation_309\n",
      "624 conv2d_276\n",
      "625 conv2d_279\n",
      "626 batch_normalization_262\n",
      "627 batch_normalization_265\n",
      "628 activation_307\n",
      "629 activation_310\n",
      "630 block8_1_mixed\n",
      "631 block8_1_conv\n",
      "632 block8_1\n",
      "633 block8_1_ac\n",
      "634 conv2d_281\n",
      "635 batch_normalization_267\n",
      "636 activation_312\n",
      "637 conv2d_282\n",
      "638 batch_normalization_268\n",
      "639 activation_313\n",
      "640 conv2d_280\n",
      "641 conv2d_283\n",
      "642 batch_normalization_266\n",
      "643 batch_normalization_269\n",
      "644 activation_311\n",
      "645 activation_314\n",
      "646 block8_2_mixed\n",
      "647 block8_2_conv\n",
      "648 block8_2\n",
      "649 block8_2_ac\n",
      "650 conv2d_285\n",
      "651 batch_normalization_271\n",
      "652 activation_316\n",
      "653 conv2d_286\n",
      "654 batch_normalization_272\n",
      "655 activation_317\n",
      "656 conv2d_284\n",
      "657 conv2d_287\n",
      "658 batch_normalization_270\n",
      "659 batch_normalization_273\n",
      "660 activation_315\n",
      "661 activation_318\n",
      "662 block8_3_mixed\n",
      "663 block8_3_conv\n",
      "664 block8_3\n",
      "665 block8_3_ac\n",
      "666 conv2d_289\n",
      "667 batch_normalization_275\n",
      "668 activation_320\n",
      "669 conv2d_290\n",
      "670 batch_normalization_276\n",
      "671 activation_321\n",
      "672 conv2d_288\n",
      "673 conv2d_291\n",
      "674 batch_normalization_274\n",
      "675 batch_normalization_277\n",
      "676 activation_319\n",
      "677 activation_322\n",
      "678 block8_4_mixed\n",
      "679 block8_4_conv\n",
      "680 block8_4\n",
      "681 block8_4_ac\n",
      "682 conv2d_293\n",
      "683 batch_normalization_279\n",
      "684 activation_324\n",
      "685 conv2d_294\n",
      "686 batch_normalization_280\n",
      "687 activation_325\n",
      "688 conv2d_292\n",
      "689 conv2d_295\n",
      "690 batch_normalization_278\n",
      "691 batch_normalization_281\n",
      "692 activation_323\n",
      "693 activation_326\n",
      "694 block8_5_mixed\n",
      "695 block8_5_conv\n",
      "696 block8_5\n",
      "697 block8_5_ac\n",
      "698 conv2d_297\n",
      "699 batch_normalization_283\n",
      "700 activation_328\n",
      "701 conv2d_298\n",
      "702 batch_normalization_284\n",
      "703 activation_329\n",
      "704 conv2d_296\n",
      "705 conv2d_299\n",
      "706 batch_normalization_282\n",
      "707 batch_normalization_285\n",
      "708 activation_327\n",
      "709 activation_330\n",
      "710 block8_6_mixed\n",
      "711 block8_6_conv\n",
      "712 block8_6\n",
      "713 block8_6_ac\n",
      "714 conv2d_301\n",
      "715 batch_normalization_287\n",
      "716 activation_332\n",
      "717 conv2d_302\n",
      "718 batch_normalization_288\n",
      "719 activation_333\n",
      "720 conv2d_300\n",
      "721 conv2d_303\n",
      "722 batch_normalization_286\n",
      "723 batch_normalization_289\n",
      "724 activation_331\n",
      "725 activation_334\n",
      "726 block8_7_mixed\n",
      "727 block8_7_conv\n",
      "728 block8_7\n",
      "729 block8_7_ac\n",
      "730 conv2d_305\n",
      "731 batch_normalization_291\n",
      "732 activation_336\n",
      "733 conv2d_306\n",
      "734 batch_normalization_292\n",
      "735 activation_337\n",
      "736 conv2d_304\n",
      "737 conv2d_307\n",
      "738 batch_normalization_290\n",
      "739 batch_normalization_293\n",
      "740 activation_335\n",
      "741 activation_338\n",
      "742 block8_8_mixed\n",
      "743 block8_8_conv\n",
      "744 block8_8\n",
      "745 block8_8_ac\n",
      "746 conv2d_309\n",
      "747 batch_normalization_295\n",
      "748 activation_340\n",
      "749 conv2d_310\n",
      "750 batch_normalization_296\n",
      "751 activation_341\n",
      "752 conv2d_308\n",
      "753 conv2d_311\n",
      "754 batch_normalization_294\n",
      "755 batch_normalization_297\n",
      "756 activation_339\n",
      "757 activation_342\n",
      "758 block8_9_mixed\n",
      "759 block8_9_conv\n",
      "760 block8_9\n",
      "761 block8_9_ac\n",
      "762 conv2d_313\n",
      "763 batch_normalization_299\n",
      "764 activation_344\n",
      "765 conv2d_314\n",
      "766 batch_normalization_300\n",
      "767 activation_345\n",
      "768 conv2d_312\n",
      "769 conv2d_315\n",
      "770 batch_normalization_298\n",
      "771 batch_normalization_301\n",
      "772 activation_343\n",
      "773 activation_346\n",
      "774 block8_10_mixed\n",
      "775 block8_10_conv\n",
      "776 block8_10\n",
      "777 conv_7b\n",
      "778 conv_7b_bn\n",
      "779 conv_7b_ac\n",
      "780 global_average_pooling2d_8\n",
      "781 dense_15\n",
      "782 dropout_8\n",
      "783 dense_16\n"
     ]
    }
   ],
   "source": [
    "for i,layer in enumerate(inres_model.layers):\n",
    "    print(i,layer.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Construct the ensemble model using the last \"dense layer\" of each base CNN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\n",
    "model1=Model(inputs=[xception_model.layers[0].get_input_at(0)],outputs=xception_model.get_layer('dense_6').output,name='xception')\n",
    "model2=Model(inputs=[vgg_model.layers[0].get_input_at(0)],outputs=vgg_model.get_layer('dense_8').output,name='vgg')\n",
    "model3=Model(inputs=[vgg19_model.layers[0].get_input_at(0)],outputs=vgg19_model.get_layer('dense_10').output,name='vgg19')\n",
    "model4=Model(inputs=[incep_model.layers[0].get_input_at(0)],outputs=incep_model.get_layer('dense_14').output,name='incep')\n",
    "model5=Model(inputs=[inres_model.layers[0].get_input_at(0)],outputs=inres_model.get_layer('dense_16').output,name='inres')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#plot the figures\n",
    "class LossHistory(keras.callbacks.Callback):\n",
    "    def on_train_begin(self, logs={}):\n",
    "        self.losses = {'batch':[], 'epoch':[]}\n",
    "        self.accuracy = {'batch':[], 'epoch':[]}\n",
    "        self.val_loss = {'batch':[], 'epoch':[]}\n",
    "        self.val_acc = {'batch':[], 'epoch':[]}\n",
    "    def on_batch_end(self, batch, logs={}):\n",
    "        self.losses['batch'].append(logs.get('loss'))\n",
    "        self.accuracy['batch'].append(logs.get('acc'))\n",
    "        self.val_loss['batch'].append(logs.get('val_loss'))\n",
    "        self.val_acc['batch'].append(logs.get('val_acc'))\n",
    "    def on_epoch_end(self, batch, logs={}):\n",
    "        self.losses['epoch'].append(logs.get('loss'))\n",
    "        self.accuracy['epoch'].append(logs.get('acc'))\n",
    "        self.val_loss['epoch'].append(logs.get('val_loss'))\n",
    "        self.val_acc['epoch'].append(logs.get('val_acc'))\n",
    "    def loss_plot(self, loss_type):\n",
    "        iters = range(len(self.losses[loss_type]))\n",
    "        plt.figure()\n",
    "        plt.plot(iters, self.losses[loss_type], 'g', label='train loss')\n",
    "        if loss_type == 'epoch':\n",
    "            # acc\n",
    "            plt.plot(iters, self.accuracy[loss_type], 'r', label='train acc')\n",
    "            # loss\n",
    "            plt.plot(iters, self.losses[loss_type], 'g', label='train loss')\n",
    "            # val_acc\n",
    "            plt.plot(iters, self.val_acc[loss_type], 'b', label='val acc')\n",
    "            # val_loss\n",
    "            plt.plot(iters, self.val_loss[loss_type], 'k', label='val loss')\n",
    "        plt.grid(True)\n",
    "        plt.xlabel(loss_type)\n",
    "        plt.ylabel('acc-loss')\n",
    "        plt.legend(loc=\"upper right\")\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ensemble_history= LossHistory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 23382 images belonging to 5 classes.\n",
      "Found 5845 images belonging to 5 classes.\n"
     ]
    }
   ],
   "source": [
    "#generate training and test images\n",
    "TARGET_SIZE=(224,224)\n",
    "INPUT_SIZE=(224,224,3)\n",
    "BATCHSIZE=128\t#could try 128 or 32\n",
    "\n",
    "#Normalization\n",
    "train_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "test_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        './train_224/',\n",
    "        target_size=TARGET_SIZE,\n",
    "        batch_size=BATCHSIZE,\n",
    "        class_mode='categorical')\n",
    "validation_generator = test_datagen.flow_from_directory(\n",
    "        './test_224/',\n",
    "        target_size=TARGET_SIZE,\n",
    "        batch_size=BATCHSIZE,\n",
    "        class_mode='categorical')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def lr_decay(epoch):\n",
    "    lrs = [0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.0001,0.00001,0.000001,\n",
    "           0.000001,0.000001,0.000001,0.000001,0.0000001,0.0000001,0.0000001,0.0000001,0.0000001,0.0000001\n",
    "          ]\n",
    "    return lrs[epoch]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "auto_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=1, verbose=0, mode='auto', epsilon=0.0001, cooldown=0, min_lr=0)\n",
    "my_lr = LearningRateScheduler(lr_decay)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def ensemble(num_class,epochs,savepath='./ensemble.h5'):\n",
    "    img=Input(shape=(224,224,3),name='img')\n",
    "    feature1=model1(img)\n",
    "    feature2=model2(img)\n",
    "    feature3=model3(img)\n",
    "    x=concatenate([feature1,feature2,feature3])\n",
    "    x=Dropout(0.5)(x)\n",
    "    x=Dense(64,activation='relu')(x)\n",
    "    x=Dropout(0.25)(x)\n",
    "    output=Dense(num_class,activation='softmax',name='output')(x)\n",
    "    model=Model(inputs=img,outputs=output)\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\n",
    "    model.compile(loss='categorical_crossentropy',\n",
    "                  optimizer=opt,\n",
    "                  metrics=['accuracy'])\n",
    "    #train model\n",
    "    earlyStopping=kcallbacks.EarlyStopping(monitor='val_acc',patience=2, verbose=1, mode='auto')\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(filepath=savepath, monitor='val_acc', verbose=1, save_best_only=True, mode='auto')\n",
    "    hist=model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        callbacks=[earlyStopping,saveBestModel,ensemble_history,auto_lr],\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.5297 - acc: 0.8809Epoch 00001: val_acc improved from -inf to 1.00000, saving model to ./ensemble.h5\n",
      "183/183 [==============================] - 204s 1s/step - loss: 0.5278 - acc: 0.8814 - val_loss: 0.0590 - val_acc: 1.0000\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.1287 - acc: 0.9770Epoch 00002: val_acc did not improve\n",
      "183/183 [==============================] - 184s 1s/step - loss: 0.1286 - acc: 0.9770 - val_loss: 0.0103 - val_acc: 1.0000\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0975 - acc: 0.9778Epoch 00003: val_acc did not improve\n",
      "183/183 [==============================] - 184s 1s/step - loss: 0.0973 - acc: 0.9778 - val_loss: 0.0033 - val_acc: 1.0000\n",
      "Epoch 00003: early stopping\n"
     ]
    }
   ],
   "source": [
    "ensemble_model=ensemble(num_class=5,epochs=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ensemble_model=load_model('./ensemble.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 ./test_224/0\\100015.png\n"
     ]
    }
   ],
   "source": [
    "#read images from validation folder\n",
    "rootdir = './test_224/'\n",
    "test_laels = []\n",
    "test_images=[]\n",
    "for subdir, dirs, files in os.walk(rootdir):\n",
    "    for file in files:\n",
    "        if not (file.endswith(\".jpeg\"))|(file.endswith(\".jpg\"))|(file.endswith(\".png\")):\n",
    "            continue\n",
    "        test_laels.append(subdir.split('/')[-1])\n",
    "        test_images.append(os.path.join(subdir, file))\n",
    "        \n",
    "print(test_laels[0],test_images[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The testing time is :54.200418 seconds\n"
     ]
    }
   ],
   "source": [
    "#test the averaging model on the validation set\n",
    "import time\n",
    "predict=[]\n",
    "length=len(test_images)\n",
    "t1 = time.time()\n",
    "for i in range((length//127)+1):\n",
    "    inputimg=test_images[127*i:127*(i+1)]\n",
    "    test_batch=[]\n",
    "    for path in inputimg:\n",
    "        thisimg=np.array(Image.open(path))/255\n",
    "        test_batch.append(thisimg)\n",
    "    #print(i, np.array(test_batch).shape)\n",
    "    ensemble_model_batch=ensemble_model.predict(np.array(test_batch))\n",
    "    predict_batch=list(np.argmax(ensemble_model_batch,axis=1))\n",
    "    predict_batch=[label[con] for con in predict_batch]\n",
    "    predict.append(predict_batch)\n",
    "\n",
    "predict=sum(predict,[])\n",
    "\n",
    "t2 = time.time()\n",
    "print('The testing time is :%f seconds' % (t2-t1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Concatenation accuracy:1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\n",
    "acc=accuracy_score(test_laels,predict)\n",
    "print('Concatenation accuracy:%s'%acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5052    0    0    0    0]\n",
      " [   0  225    0    0    0]\n",
      " [   0    0  200    0    0]\n",
      " [   0    0    0  197    0]\n",
      " [   0    0    0    0  171]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00      5052\n",
      "           1       1.00      1.00      1.00       225\n",
      "           2       1.00      1.00      1.00       200\n",
      "           3       1.00      1.00      1.00       197\n",
      "           4       1.00      1.00      1.00       171\n",
      "\n",
      "    accuracy                           1.00      5845\n",
      "   macro avg       1.00      1.00      1.00      5845\n",
      "weighted avg       1.00      1.00      1.00      5845\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "print(confusion_matrix(test_laels, predict))\n",
    "target_names = ['0', '1','2','3','4']\n",
    "print(classification_report(test_laels, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "tf36cnn",
   "language": "python",
   "name": "tf36cnn"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}