Introduction_to_Text_Fabric_Assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Exercises for the Text-Fabric course"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Your name: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We start with some exercises that are useful for working with text-fabric, in which you have to practice with working with for loops, lists and strings."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "You can iterate over the elements in a list using a for loop and then do something with these elements. In the following script 1 is added to each number of first_list and it is appended to second_list. Finally, second_list is printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "first_list = [2, 3, 4, 5, 100]\n",
    "second_list = []\n",
    "\n",
    "for num in first_list:\n",
    "    num += 1\n",
    "    second_list.append(num)\n",
    "    \n",
    "print(second_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In the following exercises, try to write concise code!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*1. With the functions int(), float() and str(), you can convert data into other data types. Use these functions (if needed), a for loop and range() to produce the following list:\n",
    "\n",
    "str_list = ['1', '2', '3', '4', '5', '6', '7']\n",
    "\n",
    "First initialize the empty list str_list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "str_list = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*2. With the functions int(), float() and str(), you can convert data into other data types. Use these functions (if needed), a for loop and range() to produce the following list:\n",
    "\n",
    "int_list = [11, 22, 33, 44, 55, 66]\n",
    "\n",
    "First initialize the empty list int_list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "int_list = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*3. With the functions int(), float() and str(), you can convert data into other data types. Use these functions (if needed), a for loop and range() to produce the following list:\n",
    "    \n",
    "int_list2 = [12, 23, 34, 45, 56, 67]\n",
    "\n",
    "First initialize the empty list int_list2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "int_list2 = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*4. Use a for loop and range() to produce the following list:\n",
    "    \n",
    "int_list3 = [2, 4, 6, 8, 10]\n",
    "\n",
    "First initialize the empty list int_list3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "int_list3 = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*5. Use a while loop and range() to produce the following list:\n",
    "    \n",
    "int_list4 = [2, 4, 6, 8, 10]\n",
    "\n",
    "First initialize the empty list int_list4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "int_list4 = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*6. You start with the following list:\n",
    "\n",
    "book_list = ['Esther', 'Daniel', 'Ezra']\n",
    "\n",
    "Use a for loop to produce book_list2:\n",
    "    \n",
    "book_list2 = ['EstherEsther', 'DanielDaniel', 'EzraEzra']\n",
    "\n",
    "First initialize the empty list book_list2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "book_list = ['Esther', 'Daniel', 'Ezra']\n",
    "book_list2 = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Text-fabric exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Before you can start making the exercises you have to import TF and load the features you need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import sys, collections, os\n",
    "import pprint as pp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from tf.fabric import Fabric\n",
    "\n",
    "DATABASE = '~/github'\n",
    "BHSA = 'bhsa/tf/c'\n",
    "\n",
    "TF = Fabric(locations=[DATABASE], modules=[BHSA], silent=False )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "api = TF.load('''\n",
    "    lex\n",
    "''')\n",
    "#load the features that you need here! lex is given already."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "api.loadLog()\n",
    "api.makeAvailableIn(globals())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*7. Very often it is useful to have a list of the names of the biblical books (in your favorite language, but in Latin script) in the MT order. Write a script that produces such a list named book_list. You start with initializing an empty list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "NB. You may not hardcode the booknames in this exercise. In other words, the following solution is not allowed:\n",
    "\n",
    "book_list = []  \n",
    "book_list.append('Genesis')  \n",
    "book_list.append('Exodus')  \n",
    "book_list.append('Leviticus')  \n",
    "etc.\n",
    "\n",
    "You have to extract the booknames from text-fabric. For clarity, trivial, hardcoded solutions are not allowed throughout the exercises."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "book_list = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*8. Write a script that produces a list with the names of the biblical books in the MT order, but the names have to be abbreviated to the first four characters (so it looks like this: ['Gene', 'Exod', ....]). First initialize an empty list with the name books_list_abbr."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "book_list_abbr = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*9. Write a script that counts the number of words in Genesis. Initialize a variable gen_words_count first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "gen_words = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "*10. Write a script that counts the number of yiqtols in the Song of Songs. Initialize a variable song_yiq_count first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "song_yiq_count = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*11. Write a script that counts the number of words in each biblical books, and store the result in a dictionary with the name word_counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "word_counts = {} # or: word_counts = collections.defaultdict(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*12. Write a script that saves the information from the dictionary word_counts in a csv-file with the name word_counts.csv.\n",
    "The file contains 2 columns. The first contains the booknames in MT order, the second column contains the word counts. The file has a header: 'book', 'word_count'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*13. Write a script that counts the number of times the name of the Lord (JHWH) occurs in each biblical book, and store the result in a dictionary called jhwh_count."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*14. Suppose a certain Hebrew book consists of 10000 words, of which 500 words are the Hebrew article (H). The fraction of words that is the article is 0.05. Write a script that calculates the fraction of Hebrew words that is the article in each Biblical book, and save the result in a dictionary called art_fraction. You may have to use more dictionaries in this exercise to obtain the final result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "art_fraction = {}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*15. Write a script that counts the frequencies of all the lexemes in the book of Judges, and save the result in a dictionary called jud_lex_count."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*16. Write a script that saves the information from the dictionary jud_lex_count in a csv-file with the name jud_lex_count.csv.\n",
    "The file contains 2 columns. The first contains the lexemes (without specific order), the second column contains the word counts. The file has a header: 'lexeme', 'frequency'. You get a bonus-credit if you succeed to save the lexemes in the file in the order of their frequencies. In that case the most frequent lexeme comes directly under the header, then the second most frequent, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### In the following exercises you make some steps in the direction of the dataset for the main assignment of this course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*17. Write a script that counts the number of clauses in each biblical book that contain a verbal predicate and an explicit subject (do not include subject suffixes), and store the result in a dictionary with the name subj_verb_count."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*18. Write a script that that counts the number of clauses in each biblical book that contain a verbal predicate and an explicit subject (do not include subject suffixes), but now you should count for each book 2 numbers: one for the order verb-subject, the other for the order subject-verb. Save the result in a csv file with three columns and the header: 'book', 'order', 'frequency'.\n",
    "The books should be in the MT order, the variable 'order' has two values: 'SV' (subject before verb) and 'VS' (verb before subject), so the file has 78 rows and a header. The csv file has the name sv_order_counts.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "*19. Write a script that creates a csv file with nine columns and a header: 'id', 'book', 'chapter', 'verse', 'discourse', 'discourse_last', 'tense', 'order', 'language'. The name of the csv file is sv_order_complete.csv.  \n",
    "Each row in the dataset should contain information about one clause in the Hebrew Bible, in the order in which they occur in the MT. You include only those clauses that contain a verbal predicate and an explicit subject (do not include subject suffixes). The id is the tf identifier of the clause (this is an integer), book, chapter and verse are self-evident, for discourse you use the feature 'txt' in text-fabric. This is a string consisting if the characters Q, N, D and ?. Discourse_last is the last character of the string of the variable discourse. This means that if the value of the variable discourse is 'QN', discourse_last is 'N', and if the value of the variable discourse is 'Q', discourse_last is 'Q'. The veriable tense contains the tense of the verb in the predicate. The variable order has two values: 'VS' (verb before subject) and 'SV' (subject before verb), and language has two variables as well (Hebrew or Aramaic).  \n",
    "\n",
    "The first row of the csv should look like this (this is information about the first clause in the Hebrew Bible):\n",
    "\n",
    "426582, Genesis, 1, 1, ?, ?, perf, VS, Hebrew\n",
    "\n",
    "The final file should contain several tens of thousands of rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}