{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Implementation of Topic Extraction and Document Clustering\n",
    "\n",
    "- Author:      Johannes Maucher\n",
    "- Last update: 14.12.2021\n",
    "\n",
    "This notebook demonstrates how [gensim](http://radimrehurek.com/gensim/) can be applied for *Latent Semantic Indexing (LSI)*. In LSI a set of abstract topics (features), which are latent in a set of simple texts, is calculated. Then the documents are described and visualised with respect to these abstract features. The notebook is an adoption of the corresponding [gensim LSI tutorial](http://radimrehurek.com/gensim/tut2.html). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Collect and filter text documents\n",
    "A list of very small documents is defined. From the corresponding BoW (Bag of Words) representation all stopwords and all words, which appear only once are removed. The resulting cleaned BoW models of all documents are printed below.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install --upgrade gensim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['human', 'interface', 'computer']\n",
      "['survey', 'user', 'computer', 'system', 'response', 'time']\n",
      "['eps', 'user', 'interface', 'system']\n",
      "['system', 'human', 'system', 'eps']\n",
      "['user', 'response', 'time']\n",
      "['trees']\n",
      "['graph', 'trees']\n",
      "['graph', 'minors', 'trees']\n",
      "['graph', 'minors', 'survey']\n"
     ]
    }
   ],
   "source": [
    "from gensim import corpora, models, similarities\n",
    "\n",
    "documents = [\"Human machine interface for lab abc computer applications\",\n",
    "              \"A survey of user opinion of computer system response time\",\n",
    "              \"The EPS user interface management system\",\n",
    "              \"System and human system engineering testing of EPS\",\n",
    "              \"Relation of user perceived response time to error measurement\",\n",
    "              \"The generation of random binary unordered trees\",\n",
    "              \"The intersection graph of paths in trees\",\n",
    "              \"Graph minors IV Widths of trees and well quasi ordering\",\n",
    "              \"Graph minors A survey\"]\n",
    "# remove common words and tokenize\n",
    "stoplist = set('for a of the and to in'.split())\n",
    "texts = [[word for word in document.lower().split() if word not in stoplist] for document in documents]\n",
    "# remove words that appear only once\n",
    "all_tokens=[]\n",
    "for t in texts:\n",
    "    for w in t:\n",
    "        all_tokens.append(w)\n",
    "tokens_once = set(word for word in set(all_tokens) if all_tokens.count(word) == 1)\n",
    "texts = [[word for word in text if word not in tokens_once]\n",
    "          for text in texts]\n",
    "for t in texts:\n",
    "    print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dictionaries and Corpora\n",
    "The words of the cleaned documents constitute a dictionary, which is persistently saved in the file *deerwester.dict*. The dictionary-method *token2id* displays the dictionary indes of each word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'computer': 0, 'human': 1, 'interface': 2, 'response': 3, 'survey': 4, 'system': 5, 'time': 6, 'user': 7, 'eps': 8, 'trees': 9, 'graph': 10, 'minors': 11}\n"
     ]
    }
   ],
   "source": [
    "dictionary = corpora.Dictionary(texts)\n",
    "dictionary.save('../Data/deerwester.dict') # store the dictionary, for future reference\n",
    "print(dictionary.token2id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, a corpus is generated, which is a very efficient representation of the cleaned documents. In the corpus each word is represented by it's index in the dictionary. The corpus is persistently saved to file *deerwester.mm*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "corpus = [dictionary.doc2bow(text) for text in texts]\n",
    "corpora.MmCorpus.serialize('../Data/deerwester.mm', corpus) # store to disk, for later use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 1), (1, 1), (2, 1)]\n",
      "[(0, 1), (3, 1), (4, 1), (5, 1), (6, 1), (7, 1)]\n",
      "[(2, 1), (5, 1), (7, 1), (8, 1)]\n",
      "[(1, 1), (5, 2), (8, 1)]\n",
      "[(3, 1), (6, 1), (7, 1)]\n",
      "[(9, 1)]\n",
      "[(9, 1), (10, 1)]\n",
      "[(9, 1), (10, 1), (11, 1)]\n",
      "[(4, 1), (10, 1), (11, 1)]\n"
     ]
    }
   ],
   "source": [
    "for c in corpus:\n",
    "    print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code snippet demonstrates how a dictionary and a corpus can be loaded into the python program."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 1.0), (1, 1.0), (2, 1.0)]\n",
      "[(0, 1.0), (3, 1.0), (4, 1.0), (5, 1.0), (6, 1.0), (7, 1.0)]\n",
      "[(2, 1.0), (5, 1.0), (7, 1.0), (8, 1.0)]\n",
      "[(1, 1.0), (5, 2.0), (8, 1.0)]\n",
      "[(3, 1.0), (6, 1.0), (7, 1.0)]\n",
      "[(9, 1.0)]\n",
      "[(9, 1.0), (10, 1.0)]\n",
      "[(9, 1.0), (10, 1.0), (11, 1.0)]\n",
      "[(4, 1.0), (10, 1.0), (11, 1.0)]\n"
     ]
    }
   ],
   "source": [
    "dictionary = corpora.Dictionary.load('../Data/deerwester.dict')\n",
    "corpus = corpora.MmCorpus('../Data/deerwester.mm')\n",
    "for c in corpus:\n",
    "    print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TF-IDF Model of the corpus\n",
    "A tf-idf model is generated from the cleaned documents of the corpus and all corpus documents are represented by the vector of tf-idf values of their words."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TF-IDF Model without document-vector normalisation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 2.1699250014423126), (1, 2.1699250014423126), (2, 2.1699250014423126)]\n",
      "[(0, 2.1699250014423126), (3, 2.1699250014423126), (4, 2.1699250014423126), (5, 1.5849625007211563), (6, 2.1699250014423126), (7, 1.5849625007211563)]\n",
      "[(2, 2.1699250014423126), (5, 1.5849625007211563), (7, 1.5849625007211563), (8, 2.1699250014423126)]\n",
      "[(1, 2.1699250014423126), (5, 3.1699250014423126), (8, 2.1699250014423126)]\n",
      "[(3, 2.1699250014423126), (6, 2.1699250014423126), (7, 1.5849625007211563)]\n",
      "[(9, 1.5849625007211563)]\n",
      "[(9, 1.5849625007211563), (10, 1.5849625007211563)]\n",
      "[(9, 1.5849625007211563), (10, 1.5849625007211563), (11, 2.1699250014423126)]\n",
      "[(4, 2.1699250014423126), (10, 1.5849625007211563), (11, 2.1699250014423126)]\n"
     ]
    }
   ],
   "source": [
    "tfidf = models.TfidfModel(corpus,normalize=False)  # generate a transformation object and fit it to the corpus documents\n",
    "corpus_tfidf = tfidf[corpus] # apply the transformation to all corpus documents\n",
    "for doc in corpus_tfidf:\n",
    "    print(doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform a new document to tf-idf vector. The new document in this example consists of the words \n",
    "* *computer (index 0)*, \n",
    "* *human (index 1)* \n",
    "* 2 times the word *system (index 5)*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 2.1699250014423126), (1, 2.1699250014423126), (5, 3.1699250014423126)]\n"
     ]
    }
   ],
   "source": [
    "newDoc=[(0,1),(1,1),(5,2)]\n",
    "newTFIDF=tfidf[newDoc]\n",
    "print(newTFIDF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify that log2 is applied in the tf-idf calculation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.169925001442312"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.log2(9/2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TF-IDF Model with document-vector normalisation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general it is better to normalise the document vectors, such that each vector has a length of $1$. By applying document normalisation the obtained vectors are *independent* of document length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 0.5773502691896257), (1, 0.5773502691896257), (2, 0.5773502691896257)]\n",
      "[(0, 0.44424552527467476), (3, 0.44424552527467476), (4, 0.44424552527467476), (5, 0.3244870206138555), (6, 0.44424552527467476), (7, 0.3244870206138555)]\n",
      "[(2, 0.5710059809418182), (5, 0.4170757362022777), (7, 0.4170757362022777), (8, 0.5710059809418182)]\n",
      "[(1, 0.49182558987264147), (5, 0.7184811607083769), (8, 0.49182558987264147)]\n",
      "[(3, 0.6282580468670046), (6, 0.6282580468670046), (7, 0.45889394536615247)]\n",
      "[(9, 1.0)]\n",
      "[(9, 0.7071067811865475), (10, 0.7071067811865475)]\n",
      "[(9, 0.5080429008916749), (10, 0.5080429008916749), (11, 0.695546419520037)]\n",
      "[(4, 0.6282580468670046), (10, 0.45889394536615247), (11, 0.6282580468670046)]\n"
     ]
    }
   ],
   "source": [
    "tfidf = models.TfidfModel(corpus,normalize=True)  # generate a transformation object and fit it to the corpus documents\n",
    "corpus_tfidf = tfidf[corpus] # apply the transformation to all corpus documents\n",
    "for doc in corpus_tfidf:\n",
    "    print(doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LSI Model of the corpus\n",
    "A Latent Semantic Indexing (LSI) model is generated from the given documents. The number of topics that shall be extracted is selected to be two in this example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0,\n",
       "  '0.703*\"trees\" + 0.538*\"graph\" + 0.402*\"minors\" + 0.187*\"survey\" + 0.061*\"system\" + 0.060*\"time\" + 0.060*\"response\" + 0.058*\"user\" + 0.049*\"computer\" + 0.035*\"interface\"'),\n",
       " (1,\n",
       "  '0.460*\"system\" + 0.373*\"user\" + 0.332*\"eps\" + 0.328*\"interface\" + 0.320*\"time\" + 0.320*\"response\" + 0.293*\"computer\" + 0.280*\"human\" + 0.171*\"survey\" + -0.161*\"trees\"')]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lsi = models.LsiModel(corpus_tfidf, id2word=dictionary, num_topics=2) # initialize an LSI transformation\n",
    "corpus_lsi = lsi[corpus_tfidf]\n",
    "lsi.print_topics(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown below, each document is described in the new 2-dimensional space. The dimensions represent the two extracted topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Document  0: \t [(0, 0.06600783396090117), (1, 0.5200703306361857)]\n",
      "Document  1: \t [(0, 0.19667592859142272), (1, 0.7609563167700056)]\n",
      "Document  2: \t [(0, 0.08992639972446137), (1, 0.7241860626752515)]\n",
      "Document  3: \t [(0, 0.07585847652177857), (1, 0.6320551586003436)]\n",
      "Document  4: \t [(0, 0.10150299184979987), (1, 0.5737308483002962)]\n",
      "Document  5: \t [(0, 0.7032108939378321), (1, -0.16115180214025504)]\n",
      "Document  6: \t [(0, 0.8774787673119843), (1, -0.1675890686465903)]\n",
      "Document  7: \t [(0, 0.9098624686818588), (1, -0.1408655362871861)]\n",
      "Document  8: \t [(0, 0.6165825350569285), (1, 0.05392907566389654)]\n"
     ]
    }
   ],
   "source": [
    "x=[]\n",
    "y=[]\n",
    "i=0\n",
    "for doc in corpus_lsi: # both bow->tfidf and tfidf->lsi transformations are actually executed here, on the fly\n",
    "    print(\"Document %2d: \\t\"%i,doc)\n",
    "    x.append(doc[0][1])\n",
    "    y.append(doc[1][1])\n",
    "    i+=1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The documents can be plotted in the new 2-dimensional space. In this space the documents are clearly partitioned into 2 clusters, each representing one of the 2 topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuQAAAJcCAYAAACxPbF9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA11ElEQVR4nO3dfbheZX0n+u+PN5nUgFihRgLEVKi8JRgDUieH+IItdBycrSJytgO2KtihM/XqyIxTzmFCOzlKi1Nr67xkpqVMjKbFTiPUaFtfKlFEkugOo7SpQQkQRalNaDSIgPf5Yz8Jm0AgmOx9Z2d/PteV63nWvdZzr9969rp2vrlzr7WqtRYAAKCPA3oXAAAAU5lADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5MCkV1V/VFX/qXcdvVXVsVX1vao6cC/1t6iqPrg3+gJg1wRygH1IVc2qqlZVBz3dz7bW7mqtPbO19siPsd+XVdU9T/dzAOw5gRwAADoSyIFJp6peVFVfqqqtVfXHSQ7daf3bqmpDVf1DVd1QVc8bs+7kqvqrwbpvV9WvD9ofM+1l5xHjqrqzqi6vqtuq6vtV9QdV9VNV9fFBHZ+sqiPGbH9mVd1cVVuqal1VvWzMur+uqt+sqs8PPvuXVfWcweqbBq9bBtNPfraqXlBVn62q+6vq7wfH/ETfy2NG159iP2M/9xNJPp7keYN9fm/Md3ZIVf2vwee/WlXzx3zueVX1p1V1X1V9o6r+zZP8zP6oqj5QVR8b9PXFqvrpMetfOObnsr6q3jBof/7gOzxgsPw/quo7Yz63tKresYt9/vuq2jTY3/qqeuWgfVFVfaSq/niw7ktVNXfM595VVXcM1t1eVUM79fu2qvqbMevnPd3vA2AsgRyYVKrqkCQrkixN8uwk1yd53Zj1r0jy7iRvSDIjycYkywfrpif5ZJJPJHlekhck+dTT2P3rkrwqyQlJ/nlGQ+yvJzkyo79P/81gP0cn+ViS/zSo8Z1J/rSqjhzT1/+d5BeTHJXkkME2SXLW4PVZg+knX0jym0n+MskRSWYm+b2nUfOu9rNDa+37Sc5N8s3BPp/ZWvvmYPV5Gf3+npXkhiS/PzjGA5LcmGRdkqOTvDLJO6rq55+kljcmuWpwHBuSLB709RNJ/irJhwZ1vjHJf6mqk1pr30jyj0leNOjjrCTfq6oTB8sLk3x25x1V1c8k+ZUkp7fWpif5+SR3jtnkNRk9d5492O+Kqjp4sO6OJP9XksMH9X6wqmYM+j0/yaIkFyU5bPD9fPfH/D4AkgjkwORzZpKDk7yvtfZQa+0jSVaPWT+c5A9ba19qrT2Y5D8k+dmqmpXk1Unuba29t7X2g9ba1tbaF5/Gvn+vtfbt1tqmJKuSfLG19uXW2g+S/FkeDY1vSrKytbaytfaj1tpfJVmT5BfG9HVta+3vWmsPJPmTJKc9yX4fSnJckucN6v7c06j56ezniXxucByPZPQfQdtHkk9PcmRr7Tdaaz9srX09yf/IaJjelT9rrd3aWns4ybIxtbw6yZ2ttWtbaw+31r6c5E+TnD9Y/9kkC6vquYPljwyWn5/RULzuCfb1SJJnJDmpqg5urd3ZWrtjzPq1rbWPtNYeSvKfM/q/LGcmSWvt+tbaNwc/uz9O8rUkZww+99Ykv9VaW91GbWitbfwxvw+AJAI5MPk8L8mm1lob07Zxp/U7lltr30vy3YyOWh6T0dHPH9e3x7x/4AmWnzl4f1yS8wdTLbZU1ZYkCzI6Yr/dvWPebxvz2Sfy75JUklsH00Z+6WnU/HT2szufP3QwJea4jE5xGXuMv57kp36MWo5L8pKd+hpOsj2AfzbJyzI6On5Tkr/O6Mj4wiSrWms/2nlHrbUNSd6R0dHs71TV8jHTcJLk7jHb/ijJPRk9d1JVF1XVyJhaTkmyfarPrs6hH+f7AEiSPO2r+AE6+1aSo6uqxoTyY/NoSPpmRsNRkh3TIX4yyaaMhrBdjVh+P8m0McvP3cV2u+PuJEtba2/7MT7bHtfQ2r1J3pYkVbUgySer6qZB6NxbHrffp3B3km+01o7fC/u+O8lnW2uv2sX6zyb57YyG5s8m+VyS/5bkB3mC6SrbtdY+lORDVXVYkv+e5Ook/3Kw+pjt2w2mm8xM8s2qOi6jI9uvTPKF1tojVTWS0X8Qba91x9z3nY5hb30fwBRjhByYbL6Q5OEk/6aqDq6q1+bR6QRJ8uEkv1hVp1XVM5L8fxmdWnJnkj9PMqOq3lFVz6iq6VX1ksHnRpL8QlU9ezA14h17UOMHk/zzqvr5qjqwqg6t0YtEZ+7GZ+9L8qMks7c3VNX5Yz67OaPh+XGjwnvo20l+sqoO383tb02ydXDh5D8ZHOcpVXX6j7HvP09yQlX9y8HP9OCqOn37PPHW2tcy+j8Qb8pocP/HQb2vyy4CeVX9TFW9YnAO/GDw+bHf2Yur6rWD0f53JHkwyS1JfiKj3+99g35+MaMj5Nv9zyTvrKoX16gXDEL83vw+gClGIAcmldbaD5O8Nsmbk/xDkguS/O8x6z+Z5P/N6Bzkb2V0NPONg3VbM3pR5j/P6PSJryV5+eCjSzM6F/nOjF5A+YR3MtnNGu/O6EWDv57RYHd3ksuzG79zW2vbMnqx4+cHUx/OzOj85C9W1fcyemHlrw7mKO81rbW/zeg/Zr4+2O/znmL7RzI69/u0JN9I8vcZDau7G+jH9rU1yc9l9Of0zYz+bK7O6Bzw7T6b5LuD73b7ciX50i66fUaS9wzqujejF4v+hzHrP5rRc2dzRkfNXzu4JuH2JO/N6D/8vp3k1CSfH1Pr9Rn9+XwoydaMXmD87L35fQBTTz12GiYA7N+qalGSF7TW3tS7FoDECDkAAHQlkAMAQEemrAAAQEdGyAEAoKP97j7kz3nOc9qsWbN6lwEAwH5u7dq1f99aO3JP++kayKvqnCS/m+TAJP+ztfaendYfm+S6JM8abPOu1trKJ+tz1qxZWbNmzfgUDAAAA1W18am3emrdpqxU1YFJPpDk3CQnJbmwqk7aabP/J8mftNZelNH70/6Xia0SAADGV8855Gck2dBa+/rgQR/LM/ogjbFaksMG7w/P6AMjAABgv9EzkB+d0afXbXfPoG2sRUneVFX3JFmZ5F8/UUdVdUlVramqNffdd9941AoAAONiX7/LyoVJ/qi1NjPJLyRZWlWPq7m1tqS1Nr+1Nv/II/d4Xj0AAEyYnoF8U5JjxizPHLSN9ZYkf5IkrbUvJDk0yXMmpDoAAJgAPQP56iTHV9Xzq+qQjF60ecNO29yV5JVJUlUnZjSQm5PCuFm0aFGuueaaPe7nl37pl3LUUUfllFNO2QtVAQD7s26BvLX2cJJfSfIXSf4mo3dT+WpV/UZVnTfY7N8meVtVrUvy4SRvbh4tyiTw5je/OZ/4xCd6lwEATAJd55C31la21k5orf10a23xoO3K1toNg/e3t9b+aWttbmvttNbaX/asl/3T4sWLc8IJJ2TBggVZv379jvaRkZGceeaZmTNnToaGhrJ58+YkyYYNG3L22Wdn7ty5mTdvXu64447H9XnWWWfl2c9+9oQdAwAwee3rF3XCuFq7dm2WL1+ekZGRrFy5MqtXr96x7qKLLsrVV1+d2267LaeeemquuuqqJMnw8HAuu+yyrFu3LjfffHNmzJjRq3wAYD/Q9Umd0NuqVasyNDSUadOmJUnOO290ttT999+fLVu2ZOHChUmSiy++OOeff362bt2aTZs2ZWhoKEly6KGH9ikcANhvGCEHAICOBHKmtLPOOisrVqzIAw88kK1bt+bGG29Mkhx++OE54ogjsmrVqiTJ0qVLs3DhwkyfPj0zZ87MihUrkiQPPvhgtm3b1qt8AGA/IJAzpc2bNy8XXHBB5s6dm3PPPTenn376jnXXXXddLr/88syZMycjIyO58sork4yG8/e///2ZM2dOXvrSl+bee+99XL8XXnhhfvZnfzbr16/PzJkz8wd/8AcTdkwAwORS+9tdBOfPn9/WrFnTuwwAAPZzVbW2tTZ/T/sxQg4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQM7ktWxZMmtWcsABo6/LlvWuCADgaTuodwHwY1m2LLnkkmT7UzI3bhxdTpLh4X51AQA8TUbImZyuuOLRML7dtm2j7QAAk4hAzuR0111Prx0AYB8lkDM5HXvs02sHANhHCeT7sUWLFuWaa67Zoz7uvvvuvPzlL89JJ52Uk08+Ob/7u7+7l6rbQ4sXJ9OmPbZt2rTRdgCASUQg50kddNBBee9735vbb789t9xySz7wgQ/k9ttv713W6IWbS5Ykxx2XVI2+Llnigk4AYNIRyPczixcvzgknnJAFCxZk/fr1O9pHRkZy5plnZs6cORkaGsrmzZuTJBs2bMjZZ5+duXPnZt68ebnjjjse09+MGTMyb968JMn06dNz4oknZtOmTRN3QE9meDi5887kRz8afRXGAYBJSCDfj6xduzbLly/PyMhIVq5cmdWrV+9Yd9FFF+Xqq6/ObbfdllNPPTVXXXVVkmR4eDiXXXZZ1q1bl5tvvjkzZszYZf933nlnvvzlL+clL3nJuB8LAMBU4T7k+5FVq1ZlaGgo0wZzq88777wkyf33358tW7Zk4cKFSZKLL744559/frZu3ZpNmzZlaGgoSXLooYfusu/vfe97ed3rXpf3ve99Oeyww8b5SAAApg4j5Dylhx56KK973esyPDyc1772tb3LAQDYrwjk+5GzzjorK1asyAMPPJCtW7fmxhtvTJIcfvjhOeKII7Jq1aokydKlS7Nw4cJMnz49M2fOzIoVK5IkDz74YLbt9LCd1lre8pa35MQTT8yv/dqvTejxAABMBQL5fmTevHm54IILMnfu3Jx77rk5/fTTd6y77rrrcvnll2fOnDkZGRnJlVdemWQ0nL///e/PnDlz8tKXvjT33nvvY/r8/Oc/n6VLl+bTn/50TjvttJx22mlZuXLlhB4XAMD+rFprvWvYq+bPn9/WrFnTuwwAAPZzVbW2tTZ/T/sxQg4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeT7imXLklmzkgMOGH1dtqx3RQAATICDehdARsP3JZck2x/Ks3Hj6HKSDA/3qwsAgHFnhHxfcMUVj4bx7bZtG20HAGC/JpDvC+666+m1AwCw3xDI9wXHHvv02gEA2G8I5PuCxYuTadMe2zZt2mg7AAD7NYF8XzA8nCxZkhx3XFI1+rpkiQs6AQCmAHdZ2VcMDwvgAABTkBFyAADoSCAHAICOBHIAAOhIIAcAgI4EcgAA6EggBwCAjgRyAADoSCAHAICOBHIAAOhIIAcAgI4EcgAA6EggBwCAjgRyAADoSCAHAICOBHIAAOhIIAcAgI4EcgAA6EggBwCAjgRyAADoSCAHAICOBHIAAOhIIAcAgI4EcgAA6EggBwCAjgRyAADoSCAHAICOBHIAAOhIIJ9kFi1alGuuuWaP+vjBD36QM844I3Pnzs3JJ5+c//gf/+Neqg4AgKfroN4FMPGe8Yxn5NOf/nSe+cxn5qGHHsqCBQty7rnn5swzz+xdGgDAlGOEfBJYvHhxTjjhhCxYsCDr16/f0T4yMpIzzzwzc+bMydDQUDZv3pwk2bBhQ84+++zMnTs38+bNyx133PGY/qoqz3zmM5MkDz30UB566KFU1cQdEAAAOwjk+7i1a9dm+fLlGRkZycqVK7N69eod6y666KJcffXVue2223LqqafmqquuSpIMDw/nsssuy7p163LzzTdnxowZj+v3kUceyWmnnZajjjoqr3rVq/KSl7xkwo4JAIBHCeT7uFWrVmVoaCjTpk3LYYcdlvPOOy9Jcv/992fLli1ZuHBhkuTiiy/OTTfdlK1bt2bTpk0ZGhpKkhx66KGZNm3a4/o98MADMzIyknvuuSe33nprvvKVr0zcQQEAsINAPsU961nPystf/vJ84hOf6F0KAMCUJJDv484666ysWLEiDzzwQLZu3Zobb7wxSXL44YfniCOOyKpVq5IkS5cuzcKFCzN9+vTMnDkzK1asSJI8+OCD2bZt22P6vO+++7Jly5YkyQMPPJC/+qu/ygtf+MIJOyYAAB7lLiv7uHnz5uWCCy7I3Llzc9RRR+X000/fse66667L29/+9mzbti2zZ8/Otddem2Q0nF966aW58sorc/DBB+f666/P7Nmzd3zuW9/6Vi6++OI88sgj+dGPfpQ3vOENefWrXz3hxwYAQFKttd417FXz589va9as6V0GAAD7uapa21qbv6f9mLICAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUA+3pYtS2bNSg44YPR12bLeFQEAsA/xYKDxtGxZcsklyfYnZW7cOLqcJMPD/eoCAGCfYYR8PF1xxaNhfLtt20bbAQAgAvn4uuuup9cOAMCUI5CPp2OPfXrtAABMOQL5eFq8OJk27bFt06aNtgMAQATy8TU8nCxZkhx3XFI1+rpkiQs6AQDYwV1WxtvwsAAOAMAuGSEHAICOBHIAAOhIIAcAgI4EcgAA6EggBwCAjgRyAADoSCAHAICOBHIAAOhIIJ/CFi1alGuuuWav9PXII4/kRS96UV796lfvlf4AAKaKroG8qs6pqvVVtaGq3rWLbd5QVbdX1Ver6kMTXSO753d/93dz4okn9i4DAGDS6RbIq+rAJB9Icm6Sk5JcWFUn7bTN8Un+Q5J/2lo7Ock7JrrO/c3ixYtzwgknZMGCBVm/fv2O9pGRkZx55pmZM2dOhoaGsnnz5iTJhg0bcvbZZ2fu3LmZN29e7rjjjsf1ec899+RjH/tY3vrWt07YcQAA7C96jpCfkWRDa+3rrbUfJlme5DU7bfO2JB9orW1Oktbadya4xv3K2rVrs3z58oyMjGTlypVZvXr1jnUXXXRRrr766tx222059dRTc9VVVyVJhoeHc9lll2XdunW5+eabM2PGjMf1+453vCO/9Vu/lQMOMAMKAODp6pmgjk5y95jlewZtY52Q5ISq+nxV3VJV5zxRR1V1SVWtqao199133ziVO/mtWrUqQ0NDmTZtWg477LCcd955SZL7778/W7ZsycKFC5MkF198cW666aZs3bo1mzZtytDQUJLk0EMPzbRp0x7T55//+Z/nqKOOyotf/OKJPRgAgP3Evj6keVCS45O8LMmFSf5HVT1r541aa0taa/Nba/OPPPLIia1wivv85z+fG264IbNmzcob3/jGfPrTn86b3vSm3mUBAEwaPQP5piTHjFmeOWgb654kN7TWHmqtfSPJ32U0oPNjOOuss7JixYo88MAD2bp1a2688cYkyeGHH54jjjgiq1atSpIsXbo0CxcuzPTp0zNz5sysWLEiSfLggw9m27Ztj+nz3e9+d+65557ceeedWb58eV7xilfkgx/84IQeFwDAZNYzkK9OcnxVPb+qDknyxiQ37LTNioyOjqeqnpPRKSxfn8Aa9yvz5s3LBRdckLlz5+bcc8/N6aefvmPdddddl8svvzxz5szJyMhIrrzyyiSj4fz9739/5syZk5e+9KW59957e5UPALBfqtZav51X/UKS9yU5MMkfttYWV9VvJFnTWruhqirJe5Ock+SRJItba8ufrM/58+e3NWvWjHPlAABMdVW1trU2f4/76RnIx4NADgDARNhbgXxfv6gTAAD2awI5AAB0JJADAEBHAjkAAHQkkAMAQEcC+WSxbFkya1ZywAGjr8uW9a4IAIC94KDeBbAbli1LLrkk2f6UzI0bR5eTZHi4X10AAOwxI+STwRVXPBrGt9u2bbQdAIBJTSCfDO666+m1AwAwaQjkk8Gxxz69dgAAJg2BfDJYvDiZNu2xbdOmjbYDADCpCeSTwfBwsmRJctxxSdXo65IlLugEANgPuMvKZDE8LIADAOyHjJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwL5PmjRokW55ppr9rifT3ziE/mZn/mZvOAFL8h73vOevVAZAAB7m0C+n3rkkUdy2WWX5eMf/3huv/32fPjDH87tt9/euywAAHYikO8jFi9enBNOOCELFizI+vXrd7SPjIzkzDPPzJw5czI0NJTNmzcnSTZs2JCzzz47c+fOzbx583LHHXc8pr9bb701L3jBCzJ79uwccsgheeMb35iPfvSjE3pMAAA8NYF8H7B27dosX748IyMjWblyZVavXr1j3UUXXZSrr746t912W0499dRcddVVSZLh4eFcdtllWbduXW6++ebMmDHjMX1u2rQpxxxzzI7lmTNnZtOmTRNzQAAA7DZP6twHrFq1KkNDQ5k2bVqS5LzzzkuS3H///dmyZUsWLlyYJLn44otz/vnnZ+vWrdm0aVOGhoaSJIceemifwgEA2GNGyPdTRx99dO6+++4dy/fcc0+OPvrojhUBAPBEBPJ9wFlnnZUVK1bkgQceyNatW3PjjTcmSQ4//PAcccQRWbVqVZJk6dKlWbhwYaZPn56ZM2dmxYoVSZIHH3ww27Zte0yfp59+er72ta/lG9/4Rn74wx9m+fLlO0beAQDYd5iysg+YN29eLrjggsydOzdHHXVUTj/99B3rrrvuurz97W/Ptm3bMnv27Fx77bVJRsP5pZdemiuvvDIHH3xwrr/++syePXvH5w466KD8/u//fn7+538+jzzySH7pl34pJ5988oQfGwAAT65aa71r2Kvmz5/f1qxZ07sMAAD2c1W1trU2f0/7MWUFAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4F8b1i2LJk1KznggNHXZct6VwQAwCThwUB7atmy5JJLku1Pyty4cXQ5SYaH+9UFAMCkYIR8T11xxaNhfLtt20bbAQDgKQjke+quu55eOwAAjCGQ76ljj3167QAAMIZAvqcWL06mTXts27Rpo+0AAPAUBPI9NTycLFmSHHdcUjX6umSJCzoBANgt7rKyNwwPC+AAAPxYjJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHTUNZBX1TlVtb6qNlTVu55ku9dVVauq+RNZHwAAjLdugbyqDkzygSTnJjkpyYVVddITbDc9ya8m+eLEVggAAOOv5wj5GUk2tNa+3lr7YZLlSV7zBNv9ZpKrk/xgIosDAICJ0DOQH53k7jHL9wzadqiqeUmOaa197Mk6qqpLqmpNVa2577779n6lAAAwTvbZizqr6oAk/znJv32qbVtrS1pr81tr84888sjxLw4AAPaSnoF8U5JjxizPHLRtNz3JKUn+uqruTHJmkhtc2AkAwP6kZyBfneT4qnp+VR2S5I1Jbti+srV2f2vtOa21Wa21WUluSXJea21Nn3IBAGDv6xbIW2sPJ/mVJH+R5G+S/Elr7atV9RtVdV6vugAAYCId1HPnrbWVSVbu1HblLrZ92UTUBAAAE2mfvagTAACmAoEcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoKNdBvKqOrWqbqmqu6tqSVUdMWbdrRNTHgAA7N+ebIT8vyZZlOTUJH+X5HNV9dODdQePc10AADAlHPQk66a31j4xeH9NVa1N8omq+pdJ2viXBgAA+78nC+SpqsNba/cnSWvtM1X1uiR/muTZE1EcAADs755sysrVSU4c29Bauy3JK5P87/EsCgAApopdjpC31j60i/a7krxt3CoCAIApxG0PAQCgI4EcAAA6EsgBAKCjpwzkVXVdVT1rzPIRVfWH41oVAABMEbszQj6ntbZl+0JrbXOSF41bRQAAMIXsTiA/oKqO2L5QVc/OU9y/HAAA2D27E6zfm+QLVXV9kkry+iSLx7UqAACYIp4ykLfW/ldVrUnyikHTa1trt49vWQAAMDXsMpBX1WGttX8cTFG5N8mHxqx7dmvtHyaiQAAA2J892Qj5h5K8OsnaJC2j01W2a0lmj2NdAAAwJewykLfWXj14ff7ElQMAAFPLbt0tpapem2RBRkfGV7XWVoxnUQAAMFXszoOB/kuStyf5P0m+kuTtVfWB8S4MAACmgt0ZIX9FkhNbay0ZfXJnkq+Oa1UAADBF7M6DgTYkOXbM8jGDNgAAYA/tzgj59CR/U1W3DpZPT7Kmqm5IktbaeeNVHAAA7O92J5BfOe5VAADAFLU7T+r8bFX9VEZHxpPk1tbad8a3LAAAmBp25y4rb0hya5Lzk7whyRer6vXjXRgAAEwFuzNl5Yokp28fFa+qI5N8MslHxrMwAACYCnbnLisH7DRF5bu7+TkAAOAp7M4I+Seq6i+SfHiwfEGSj49fSQAAMHXszkWdl1fVa5MsGDQtaa392fiWBQAAU8NTBvKqurq19u+T/O8naAMAAPbA7swFf9UTtJ27twsBAICpaJcj5FX1y0n+VZLZVXXbmFXTk3x+vAsDAICp4MmmrHwooxdvvjvJu8a0b22t/cO4VgUAAFPELgN5a+3+JPcnuXDiygEAgKnF/cQBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOuoayKvqnKpaX1UbqupdT7D+16rq9qq6rao+VVXH9agTAADGS7dAXlUHJvlAknOTnJTkwqo6aafNvpxkfmttTpKPJPmtia0SAADGV88R8jOSbGitfb219sMky5O8ZuwGrbXPtNa2DRZvSTJzgmsEAIBx1TOQH53k7jHL9wzaduUtST7+RCuq6pKqWlNVa+677769WCIAAIyvSXFRZ1W9Kcn8JL/9ROtba0taa/Nba/OPPPLIiS0OAAD2wEEd970pyTFjlmcO2h6jqs5OckWSha21ByeoNgAAmBA9R8hXJzm+qp5fVYckeWOSG8ZuUFUvSvLfk5zXWvtOhxoBAGBcdQvkrbWHk/xKkr9I8jdJ/qS19tWq+o2qOm+w2W8neWaS66tqpKpu2EV3AAAwKfWcspLW2sokK3dqu3LM+7MnvCgAAJhAk+KiTgAA2F8J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAwiS1atCjXXHPNHvfzO7/zOzn55JNzyimn5MILL8wPfvCDvVAdsDsEcgCY4jZt2pT3v//9WbNmTb7yla/kkUceyfLly3uXBVOGQA4Ak8zixYtzwgknZMGCBVm/fv2O9pGRkZx55pmZM2dOhoaGsnnz5iTJhg0bcvbZZ2fu3LmZN29e7rjjjsf1+fDDD+eBBx7Iww8/nG3btuV5z3vehB0PTHUCOQBMImvXrs3y5cszMjKSlStXZvXq1TvWXXTRRbn66qtz22235dRTT81VV12VJBkeHs5ll12WdevW5eabb86MGTMe0+fRRx+dd77znTn22GMzY8aMHH744fm5n/u5CT0umMoEcgCYRFatWpWhoaFMmzYthx12WM4777wkyf33358tW7Zk4cKFSZKLL744N910U7Zu3ZpNmzZlaGgoSXLooYdm2rRpj+lz8+bN+ehHP5pvfOMb+eY3v5nvf//7+eAHPzixBwZTmEAOAFPcJz/5yTz/+c/PkUcemYMPPjivfe1rc/PNN/cuC6YMgRwAJpGzzjorK1asyAMPPJCtW7fmxhtvTJIcfvjhOeKII7Jq1aokydKlS7Nw4cJMnz49M2fOzIoVK5IkDz74YLZt2/aYPo899tjccsst2bZtW1pr+dSnPpUTTzxxQo8LprKDehcAAOy+efPm5YILLsjcuXNz1FFH5fTTT9+x7rrrrsvb3/72bNu2LbNnz861116bZDScX3rppbnyyitz8MEH5/rrr8/s2bN3fO4lL3lJXv/612fevHk56KCD8qIXvSiXXHLJhB8bTFXVWutdw141f/78tmbNmt5lAACwn6uqta21+XvajykrAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0JFADgATadmyZNas5IADRl+XLetdEdCZ+5ADwERZtiy55JJk+4N5Nm4cXU6S4eF+dQFdGSEHgIlyxRWPhvHttm0bbQemLIEcACbKXXc9vXZgShDIAWCiHHvs02sHpgSBHAAmyuLFybRpj22bNm20HZiyBHIAmCjDw8mSJclxxyVVo69LlrigE6Y4d1kBgIk0PCyAA49hhBwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKAjgRwAADoSyAEAoCOBHAAAOuoayKvqnKpaX1UbqupdT7D+GVX1x4P1X6yqWR3KBACAcdMtkFfVgUk+kOTcJCclubCqTtpps7ck2dxae0GS30ly9cRWCQAA46vnCPkZSTa01r7eWvthkuVJXrPTNq9Jct3g/UeSvLKqagJrBACAcdUzkB+d5O4xy/cM2p5wm9baw0nuT/KTO3dUVZdU1ZqqWnPfffeNU7kAALD37RcXdbbWlrTW5rfW5h955JG9ywEAgN3WM5BvSnLMmOWZg7Yn3KaqDkpyeJLvTkh1AAAwAXoG8tVJjq+q51fVIUnemOSGnba5IcnFg/evT/Lp1lqbwBoBAGBcHdRrx621h6vqV5L8RZIDk/xha+2rVfUbSda01m5I8gdJllbVhiT/kNHQDgAA+41ugTxJWmsrk6zcqe3KMe9/kOT8ia4LAAAmyn5xUScAAExWAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAdCSQAwBARwI5AAB0JJADAEBHAjkAAHQkkAMAQEcCOQAAJFm0aFGuueaaPeqjqn6mqkbG/PnHqnrHk33moD3aIwAAsENrbX2S05Kkqg5MsinJnz3ZZ4yQAwAwZS1evDgnnHBCFixYkPXr1+9oHxkZyZlnnpk5c+ZkaGgomzdvTpJs2LAhZ599dubOnZskJ1bVTz9J969MckdrbeOT1SCQAwAwJa1duzbLly/PyMhIVq5cmdWrV+9Yd9FFF+Xqq6/ObbfdllNPPTVXXXVVkmR4eDiXXXZZ1q1blyR/m+RbT7KLNyb58FPVIZADADAlrVq1KkNDQ5k2bVoOO+ywnHfeeUmS+++/P1u2bMnChQuTJBdffHFuuummbN26NZs2bcrQ0ND2LlprbdsT9V1VhyQ5L8n1T1WHQA4AAHvfuUm+1Fr79lNtKJADADAlnXXWWVmxYkUeeOCBbN26NTfeeGOS5PDDD88RRxyRVatWJUmWLl2ahQsXZvr06Zk5c2ZWrFixvYuqqmm76P7C7MZ0lcRdVgAAmKLmzZuXCy64IHPnzs1RRx2V008/fce66667Lm9/+9uzbdu2zJ49O9dee22S0XB+6aWX5sorr0ySFyZ5bpKvj+23qn4iyauSXLo7dVRrbe8c0T5i/vz5bc2aNb3LAABgP1dVa1tr8/e0H1NWAACgI4EcAAA6MoccAJiUFi1alGc+85l55zvfuUf9zJo1K9OnT8+BBx6Ygw46KKa+MtEEcgBgyvvMZz6T5zznOb3LYIoyZQUAmDT25DHn8+bNyx133NGrdNglgRwAmBT29DHnN998c2bMmPG4fqsqP/dzP5cXv/jFWbJkyYQdDz+eRYsW5ZprrtnjfrZs2ZLXv/71eeELX5gTTzwxX/jCF/ZCdT8egRwAmBT29DHnhx56aKZNe/wzXD73uc/lS1/6Uj7+8Y/nAx/4QG666aaJOyj2zLJlyaxZyQEHjL4uW7bbH/3VX/3VnHPOOfnbv/3brFu3LieeeOK4lflUBHIAYEo7+uijkyRHHXVUhoaGcuutt3auiJ094VSlZcsy8ta35syNGzOntQxt3JjNb3tbsmzZU05Vuv/++3PTTTflLW95S5LkkEMOybOe9awJPqpHCeQAwKSwp485f/DBB7Nt27bH9Pn9738/W7du3fH+L//yL3PKKadM3EHxlHY5VemKK3LRD36Qq5PcluTUJFc98EByxRVPOVXpG9/4Ro488sj84i/+Yl70ohflrW99a77//e9P9KHtIJADAJPC2Mecn3vuuY97zPnll1+eOXPmZGRkZPtjzbN06dK8//3vz5w5c/LSl740995772P6/Pa3v50FCxZk7ty5OeOMM/LP/tk/yznnnDOhx8WT2+VUpY0bsyXJwsF2Fye5KcnWjRufcqrSww8/nC996Uv55V/+5Xz5y1/OT/zET+Q973nPRB3S47jtIQAwaVxxxRW54oorHtd+2mmn5ZZbbnlc+/HHH59Pf/rTu+xv9uzZWbdu3V6tkQlyzDHJ3Xc/cfuPfvSkH505c2ZmzpyZl7zkJUmS17/+9V0DuRFyAAD2WbucqvTud+eIqqwabLc0ycKDDsr0d7/7KacqPfe5z80xxxyzYz76pz71qZx00kkTdESPZ4QcAIB91tipSkcdddSjU5WGh3Pdxo15+1VXZdsPf5jZ/+Sf5Nr3vS8ZHs7SM87IpZdemiuvvDIHH3xwrr/++syePfsx/f7e7/1ehoeH88Mf/jCzZ8/OtddeO/EHN1CttW47Hw/z589vHnkLAMB4q6q1rbX5e9qPKSsAANCRQA4AAB0J5AAA0JFADgBMjD14zDnsz9xlBQAYf8uWJZdckmy//dzGjaPLSTI83K8u2AcYIQcAxt8VVzwaxrfbtm20HaY4gRwAGH933fX02tk/mKa0WwRyAGD8HXvs02tn8ts+TWnjxqS1R6cpCeWPI5ADAONv8eJk2rTHtk2bNtrO/sk0pd0mkAMA4294OFmyJDnuuKRq9HXJEhd07s9MU9pt7rICAEyM4WEBfCo59tjRaSpP1M5jGCEHAGDvM01ptwnkAADsfaYp7TZTVgAAGB+mKe0WI+QAANCRQA4AAB0J5AAA0JFADgAAHQnkAADQkUAOAAAdCeQAANCRQA4AAB0J5AAA0FGXQF5Vz66qv6qqrw1ej3iCbU6rqi9U1Ver6raquqBHrQAAMJ56jZC/K8mnWmvHJ/nUYHln25Jc1Fo7Ock5Sd5XVc+auBIBAGD89Qrkr0ly3eD9dUn+xc4btNb+rrX2tcH7byb5TpIjJ6pAAACYCL0C+U+11r41eH9vkp96so2r6owkhyS5YxfrL6mqNVW15r777tu7lQIAwDg6aLw6rqpPJnnuE6y6YuxCa61VVXuSfmYkWZrk4tbaj55om9bakiRLkmT+/Pm77AsAAPY14xbIW2tn72pdVX27qma01r41CNzf2cV2hyX5WJIrWmu3jFOpAADQTa8pKzckuXjw/uIkH915g6o6JMmfJflfrbWPTGBtAAAwYXoF8vckeVVVfS3J2YPlVNX8qvqfg23ekOSsJG+uqpHBn9O6VAsAAOOkWtu/plzPnz+/rVmzpncZAADs56pqbWtt/p7240mdAADQkUAOAAAd7XdTVqrqviQbe9dBN89J8ve9i2DScv6wJ5w/7Annz+R0XGttjx9cud8Fcqa2qlqzN+ZyMTU5f9gTzh/2hPNnajNlBQAAOhLIAQCgI4Gc/c2S3gUwqTl/2BPOH/aE82cKM4ccAAA6MkIOAAAdCeQAANCRQM6kU1XnVNX6qtpQVe96gvW/VlW3V9VtVfWpqjquR53su57qHBqz3euqqlWVW5Gxw+6cP1X1hsHvoa9W1Ycmukb2Xbvxd9ixVfWZqvry4O+xX+hRJxPLHHImlao6MMnfJXlVknuSrE5yYWvt9jHbvDzJF1tr26rql5O8rLV2QZeC2efszjk02G56ko8lOSTJr7TW1kx0rex7dvN30PFJ/iTJK1prm6vqqNbad7oUzD5lN8+fJUm+3Fr7r1V1UpKVrbVZPepl4hghZ7I5I8mG1trXW2s/TLI8yWvGbtBa+0xrbdtg8ZYkMye4RvZtT3kODfxmkquT/GAii2Oftzvnz9uSfKC1tjlJhHHG2J3zpyU5bPD+8CTfnMD66EQgZ7I5OsndY5bvGbTtyluSfHxcK2KyecpzqKrmJTmmtfaxiSyMSWF3fgedkOSEqvp8Vd1SVedMWHXs63bn/FmU5E1VdU+SlUn+9cSURk8H9S4AxktVvSnJ/CQLe9fC5FFVByT5z0ne3LkUJq+Dkhyf5GUZ/R+6m6rq1Nbalp5FMWlcmOSPWmvvraqfTbK0qk5prf2od2GMHyPkTDabkhwzZnnmoO0xqursJFckOa+19uAE1cbk8FTn0PQkpyT566q6M8mZSW5wYScDu/M76J4kN7TWHmqtfSOjc4aPn6D62Lftzvnzloxeg5DW2heSHJrkORNSHd0I5Ew2q5McX1XPr6pDkrwxyQ1jN6iqFyX57xkN4+ZusrMnPYdaa/e31p7TWps1uJDqloyeSy7qJNmN30FJVmR0dDxV9ZyMTmH5+gTWyL5rd86fu5K8Mkmq6sSMBvL7JrRKJpxAzqTSWns4ya8k+Yskf5PkT1prX62q36iq8wab/XaSZya5vqpGqmrnX3ZMYbt5DsET2s3z5y+SfLeqbk/ymSSXt9a+26di9iW7ef782yRvq6p1ST6c5M3NLfH2e257CAAAHRkhBwCAjgRyAADoSCAHAICOBHIAAOhIIAcAgI4EcoBJrqqeVVX/ag/7WFlVz9rNbV9YVV+oqger6p17sl8A3PYQYNKrqllJ/ry1dsoE7e+oJMcl+RdJNrfWrpmI/QLsr4yQA0x+70ny04MHYf12jfrtqvpKVf2fqrogSarqZVV1U1V9rKrWV9V/q6oDBuvuHDxVMlV1UVXdVlXrqmrpzjtrrX2ntbY6yUMTeZAA+6uDehcAwB57V5JTWmunJUlVvS7JaUnmJnlOktVVddNg2zOSnJRkY5JPJHltko9s76iqTk7y/yR5aWvt76vq2RN0DABTlhFygP3PgiQfbq090lr7dpLPJjl9sO7W1trXW2uPZPSx3At2+uwrklzfWvv7JGmt/cNEFQ0wVQnkAFPLzhcOuZAIoDOBHGDy25pk+pjlVUkuqKoDq+rIJGcluXWw7oyqev5g7vgFST63U1+fTnJ+Vf1kkpiyAjD+zCEHmORaa9+tqs9X1VeSfDzJv0vys0nWZXQE/N+11u6tqhcmWZ3k95O8IMlnkvzZTn19taoWJ/lsVT2S5MtJ3jx2m6p6bpI1SQ5L8qOqekeSk1pr/zh+Rwmw/3LbQ4ApoqpeluSdrbVXdy4FgDFMWQEAgI6MkAMAQEdGyAEAoCOBHAAAOhLIAQCgI4EcAAA6EsgBAKCj/x8TEUI02J8JSQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 864x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "from matplotlib import pyplot as plt\n",
    "plt.figure(figsize=(12,10))\n",
    "plt.plot(x,y,'or')\n",
    "plt.title('documents in the new space')\n",
    "plt.xlabel('topic 1')\n",
    "plt.ylabel('topic 2')\n",
    "#plt.xlim([0,1.1])\n",
    "#plt.ylim([-0.9,0.3])\n",
    "s=0.02\n",
    "for i in range(len(x)):\n",
    "    plt.text(x[i]+s,y[i]+s,\"doc \"+str(i))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LSI models can be saved to and loaded from files: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "lsi.save('../Data/model.lsi') # same for tfidf, lda, ...\n",
    "lsi = models.LsiModel.load('../Data/model.lsi')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "nav_menu": {},
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}