ModelBuilding_Predictions.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "PrediccionesTest.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "Zoaap16wrf08",
        "dj2LmP8QsWtQ",
        "luvk4pgfxRDv",
        "ghMDpntG_s0g"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x1DNkdpLme1d"
      },
      "source": [
        "# Sexism Identification in Social networks (EXIST 2021).  \n",
        "\n",
        "## Entrega Final: Trayectoria en el proyecto y exposición de resultados finales: \n",
        "\n",
        "  \n",
        "Autores:\n",
        "Andrea García Pastor  \n",
        "Elizaveta Gilyarovskaya "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GgFR7HmDpS-o"
      },
      "source": [
        "A continuación expondremos los pasos seguidos y los métodos utilizados a lo largo de las últimas sesiones para llegar hasta el modelo final que hemos utilizado para predecir el conjunto test que vamos a enviar a la competición.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cnvEJsvvrhxX"
      },
      "source": [
        "# Descargamos librerías\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import re\n",
        "import string\n",
        "import math\n",
        "\n",
        "# Preprocesamiento\n",
        "import nltk\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.tokenize import word_tokenize\n",
        "#from nltk.stem import PorterStemmer\n",
        "#from nltk.stem import WordNetLemmatizer\n",
        "\n",
        "# Machine Learning \n",
        "from sklearn.feature_extraction.text import TfidfTransformer\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.pipeline import Pipeline\n",
        "from gensim.models import Word2Vec\n",
        "#from sklearn.model_selection import train_test_split \n",
        "from sklearn.metrics import accuracy_score \n",
        "from sklearn.naive_bayes import MultinomialNB \n",
        "#from sklearn.linear_model import LogisticRegression \n",
        "#from sklearn.svm import SVC \n",
        "from sklearn import svm\n",
        "from sklearn.preprocessing import MinMaxScaler, StandardScaler\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.ensemble import StackingClassifier\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "#from mlxtend.regressor import StackingCVRegressor"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9UOK-6synsO2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "776f3663-723a-4c29-f95b-93e14cfebb37"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0wcBj94L5-Hh",
        "outputId": "9844a72b-83fb-4252-f83a-5fb02e02924f"
      },
      "source": [
        "#Para Andrea\n",
        "%cd \"/content/drive/My Drive/lnr\""
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/My Drive/lnr\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sMQaN0oQrCBQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d9c56abe-5461-4ae9-fbe6-1282df3afd07"
      },
      "source": [
        "# Para Lisa\n",
        "%cd \"/content/drive/My Drive/lnr bert\"\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[Errno 2] No such file or directory: '/content/drive/My Drive/lnr bert'\n",
            "/content\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zoaap16wrf08"
      },
      "source": [
        "### 1. Lo primero es leer el training dataset.   \n",
        "Para este proyecto hemos decidido entrenar un clasificador únicamente para los tweets ingleses. Por tanto filtramos por la variable 'language' aquellas filas que eran de tweets españoles. Debido al tiempo reducido, esta decisión viene motivada por el hecho de querer aprender a entrenar un clasificador para el Natural Language Processing independientemente del idioma, nos interesaba dedicar más tiempo a probar diferentes estrategias y modelos de natural language processing que dedicar tiempo a hacer el mismo modelo para dos idiomas distintos."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m-8yikWrsSkI"
      },
      "source": [
        "#Leemos el training dataset:\n",
        "def read_tsv(url, language): \n",
        "    df = pd.read_csv(url, sep='\\t')\n",
        "    df = df[df['language'] == language]\n",
        "    texts = list(df['text']) \n",
        "    ids = list(df['id'])\n",
        "    t1 = list(df['task1']) \n",
        "    t2 = list(df['task2']) \n",
        "    return texts, ids,  t1, t2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dj2LmP8QsWtQ"
      },
      "source": [
        "### 2. Preproceso"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WdPZEQ9nsXnE"
      },
      "source": [
        "El preprocesamiento de los tweets va a consistir en quitar los signos de puntuación, los dígitos, quitar los enlaces a las páginas web, las menciones y los hashtags. Las funciones implementadas para eso son las que se muestran a continuación:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eeiSRKmEuTBc",
        "outputId": "2aa24cf8-56e1-4bfe-9394-b66192e4a144"
      },
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "nltk.download('punkt')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m-Bo9HcHtJHW"
      },
      "source": [
        "stop_words = set(stopwords.words('english'))\n",
        "\n",
        "def regex(tweet):  # Quitamos URL, @, emoticonos, dígitos y #\n",
        "    tweet = tweet.lower()\n",
        "    tweet = re.sub(r\"http\\S+www\\S+|https?://[\\S]+\", '', tweet, flags=re.MULTILINE)  \n",
        "    tweet = re.sub(r'\\@\\w+|\\#','', tweet) \n",
        "    tweet = re.sub(r\"[\\U00010000-\\U0010ffff]|xD|XD|:(|:)|;(|;)|:D\",'', tweet) \n",
        "    tweet = re.sub(r'[0-9]+','', tweet) \n",
        "    tweet = tweet.translate(str.maketrans('','', string.punctuation))\n",
        "    #tweet = re.sub(r\"([\\.]+|[!`¡#$%&\\'´’¿*+,-./(...):;<=*>?@\\^_`~])\", '', tweet) #no va bien del todo\n",
        "    return tweet\n",
        "\n",
        "\n",
        "def preprocessing(tweet): \n",
        "    clean_tweet = regex(tweet)\n",
        "    # Quitamos stopwords \n",
        "    tokens= word_tokenize(clean_tweet) \n",
        "    filtered = [w for w in tokens if not w in stop_words]\n",
        "    return \" \".join(filtered)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "by5BYhPst4k0"
      },
      "source": [
        "Importamos el training set y aplicamos el preproceso, miramos algunos de los tweets:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PbCQkLQft7mv"
      },
      "source": [
        "tweets, ids, t1, t2 = read_tsv('EXIST2021_training.tsv','en') "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0Zec9EQVw0Gy"
      },
      "source": [
        "Veamos el resultado del preproceso:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wb4ewZM1wxV-",
        "outputId": "b6d20bc4-8bcc-48e4-e21b-34b79f25cbaf"
      },
      "source": [
        "print('Tweet real:', tweets[23])\n",
        "print('Tweet preprocesado:', preprocessing(tweets[23]), '\\n')\n",
        "\n",
        "print('Tweet real:', tweets[5])\n",
        "print('Tweet preprocesado:', preprocessing(tweets[5]), '\\n') \n",
        "\n",
        "print('Tweet real:', tweets[3])\n",
        "print('Tweet preprocesado:', preprocessing(tweets[3]), '\\n') \n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Tweet real: @Banyosssss My philosophy: He has a funny mustache what else does a man need\n",
            "Tweet preprocesado: philosophy funny mustache else man need \n",
            "\n",
            "Tweet real: @Smithcouple971 Hello....m raj....m with good size and excellent stamina ....A passionate pussy licker...Love to lick every holes were womens desire to b licked...DoggyMissionaryWomen on topMy best pose to spank over bed\n",
            "Tweet preprocesado: hellom rajm good size excellent stamina passionate pussy lickerlove lick every holes womens desire b lickeddoggymissionarywomen topmy best pose spank bed \n",
            "\n",
            "Tweet real: @AurelieGuiboud Incredible!  Beautiful!But I laughed so much when I read about you drifting in your wheelchair.I can just picture it  https://t.co/uvl5HhbmbR\n",
            "Tweet preprocesado: incredible beautifulbut laughed much read drifting wheelchairi picture \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "luvk4pgfxRDv"
      },
      "source": [
        "### 3. Extracción de características"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vRGiFgzixJLc"
      },
      "source": [
        "A continuación probarémos diferentes metodos de recodificar/transformar el input para futuros modelos. Es decir, el texto procesado de longitud variable se convertirá en vectores de características numéricas de tamaño fijo. Lo harémos mediante cuatro estrategias para transformar datos textuales en una representación numérica: Bolsa de palabras (Bag-of-Words), N-gramas de palabras. N-gramas de caracteres, Codificación TF-IDF y Word embeddings."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zWm6F50MxrBj",
        "outputId": "059c0f85-3feb-4d77-e991-49b49663db29"
      },
      "source": [
        "# Formamos el vector x por 3 tweets aleatorios\n",
        "x1 = preprocessing(tweets[2]) \n",
        "x2 = preprocessing(tweets[3])\n",
        "x3 = preprocessing(tweets[4])\n",
        "x = [x1,x2,x3]\n",
        "print(x)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['wow skirt short length inch', 'incredible beautifulbut laughed much read drifting wheelchairi picture', 'find extremely hard believe kelly yr old mum would believe makes question fuck game especially none apologised tried explain amp immediate response like tweets attacking victims']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FwYXhKhsxymV"
      },
      "source": [
        " #### 3.1 Bolsa de palabras (unigramas) ----------------------------------"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ba002L-HxvPJ",
        "outputId": "2670f40f-e8ab-404c-acb5-4dd3ea4766f5"
      },
      "source": [
        "vectorizer = CountVectorizer()\n",
        "vectorizer.fit(x) #lista de textos\n",
        "X_bag_of_words = vectorizer.transform(x) # Formamos la bolsa con los 3 tweets\n",
        "X_bag_of_words #salida--> (nº frase, orden alfabético) : nº apariciones\n",
        "print(vectorizer.vocabulary_) #diccionario con todas las palabras"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'wow': 36, 'skirt': 30, 'short': 29, 'length': 18, 'inch': 14, 'incredible': 15, 'beautifulbut': 3, 'laughed': 17, 'much': 21, 'read': 27, 'drifting': 5, 'wheelchairi': 34, 'picture': 25, 'find': 9, 'extremely': 8, 'hard': 12, 'believe': 4, 'kelly': 16, 'yr': 37, 'old': 24, 'mum': 22, 'would': 35, 'makes': 20, 'question': 26, 'fuck': 10, 'game': 11, 'especially': 6, 'none': 23, 'apologised': 1, 'tried': 31, 'explain': 7, 'amp': 0, 'immediate': 13, 'response': 28, 'like': 19, 'tweets': 32, 'attacking': 2, 'victims': 33}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RyJhAyokyBEe"
      },
      "source": [
        "La representación numérica queda de la siguiente forma, el inconveniente es que el orden de las palabras en el texto se pierde."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WjDZfnSlyATY",
        "outputId": "ce6c19e3-4eca-4a58-afdd-ef10f278abf4"
      },
      "source": [
        "print(X_bag_of_words[0:2])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  (0, 14)\t1\n",
            "  (0, 18)\t1\n",
            "  (0, 29)\t1\n",
            "  (0, 30)\t1\n",
            "  (0, 36)\t1\n",
            "  (1, 3)\t1\n",
            "  (1, 5)\t1\n",
            "  (1, 15)\t1\n",
            "  (1, 17)\t1\n",
            "  (1, 21)\t1\n",
            "  (1, 25)\t1\n",
            "  (1, 27)\t1\n",
            "  (1, 34)\t1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QA3ZnNP4yaFF"
      },
      "source": [
        "#### 3.2 N-gramas de palabras. N-gramas de caracteres -----------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lz8gRtDzydGN"
      },
      "source": [
        "Es una alternativa a unigramas para considerar el orden de las palabras. Además, es resistente a los errores ortográficos, que en el caso de los tweets es muy acertado."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FvYoVxFNycC_",
        "outputId": "e9f9621f-b181-454f-d5f0-4658b2d4bc85"
      },
      "source": [
        "#Prueba con letras \n",
        "\n",
        "vectorizer = CountVectorizer(analyzer='char_wb', ngram_range=(2, 2))\n",
        "vectorizer.fit([x1]) #utilizamos un solo texto\n",
        "X_bag_of_words = vectorizer.transform([x1])\n",
        "X_bag_of_words #salida: (nº frase, orden alfabético)  nº apariciones\n",
        "print(vectorizer.vocabulary_)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{' w': 3, 'wo': 23, 'ow': 16, 'w ': 22, ' s': 2, 'sk': 19, 'ki': 11, 'ir': 10, 'rt': 17, 't ': 20, 'sh': 18, 'ho': 8, 'or': 15, ' l': 1, 'le': 12, 'en': 5, 'ng': 14, 'gt': 6, 'th': 21, 'h ': 7, ' i': 0, 'in': 9, 'nc': 13, 'ch': 4}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RBIJoGz4y0kQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ea4101ae-d514-4075-f890-ee24a6299f2f"
      },
      "source": [
        "#Prueba 2-n\n",
        "\n",
        "vectorizer = CountVectorizer(ngram_range=(2, 2))\n",
        "vectorizer.fit(x) #lista de textos\n",
        "X_bag_of_words = vectorizer.transform(x)\n",
        "X_bag_of_words #salida: (nº frase, orden alfabético)  nº apariciones\n",
        "print(vectorizer.vocabulary_) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'wow skirt': 34, 'skirt short': 29, 'short length': 28, 'length inch': 18, 'incredible beautifulbut': 15, 'beautifulbut laughed': 3, 'laughed much': 17, 'much read': 21, 'read drifting': 26, 'drifting wheelchairi': 6, 'wheelchairi picture': 32, 'find extremely': 10, 'extremely hard': 9, 'hard believe': 13, 'believe kelly': 4, 'kelly yr': 16, 'yr old': 35, 'old mum': 24, 'mum would': 22, 'would believe': 33, 'believe makes': 5, 'makes question': 20, 'question fuck': 25, 'fuck game': 11, 'game especially': 12, 'especially none': 7, 'none apologised': 23, 'apologised tried': 1, 'tried explain': 30, 'explain amp': 8, 'amp immediate': 0, 'immediate response': 14, 'response like': 27, 'like tweets': 19, 'tweets attacking': 31, 'attacking victims': 2}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_0Trwxluy5rE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "98f5825c-d7cb-410b-a60d-cc10912882f9"
      },
      "source": [
        "#Prueba 5-n\n",
        "\n",
        "vectorizer = CountVectorizer(ngram_range=(5, 5))\n",
        "vectorizer.fit(x) #lista de textos\n",
        "X_bag_of_words = vectorizer.transform(x)\n",
        "X_bag_of_words #salida: (nº frase, orden alfabético)  nº apariciones\n",
        "print(vectorizer.vocabulary_) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'wow skirt short length inch': 25, 'incredible beautifulbut laughed much read': 13, 'beautifulbut laughed much read drifting': 2, 'laughed much read drifting wheelchairi': 15, 'much read drifting wheelchairi picture': 17, 'find extremely hard believe kelly': 8, 'extremely hard believe kelly yr': 7, 'hard believe kelly yr old': 11, 'believe kelly yr old mum': 3, 'kelly yr old mum would': 14, 'yr old mum would believe': 26, 'old mum would believe makes': 20, 'mum would believe makes question': 18, 'would believe makes question fuck': 24, 'believe makes question fuck game': 4, 'makes question fuck game especially': 16, 'question fuck game especially none': 21, 'fuck game especially none apologised': 9, 'game especially none apologised tried': 10, 'especially none apologised tried explain': 5, 'none apologised tried explain amp': 19, 'apologised tried explain amp immediate': 1, 'tried explain amp immediate response': 23, 'explain amp immediate response like': 6, 'amp immediate response like tweets': 0, 'immediate response like tweets attacking': 12, 'response like tweets attacking victims': 22}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vUZBHgu-y-V3"
      },
      "source": [
        "#### 3.3 Codificación TF-IDF ---------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2EKNeYrEzBLO"
      },
      "source": [
        "Expresa cuán relevante es una palabra para un texto en un\n",
        "corpus. De esta forma, el valor de un término aumenta proporcionalmente al número de veces\n",
        "que aparece en el texto, y es compensado por su frecuencia en el corpus."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "heU6m4E6z-mT",
        "outputId": "4d4609db-759f-46ce-b564-4e850082cac1"
      },
      "source": [
        "#TfidfTransformer: Transforma una matriz de conteo en una representación TF-IDF.\n",
        "\n",
        "pipe = Pipeline([('count', CountVectorizer()),('tfid', TfidfTransformer())])\n",
        "pipe.fit(x)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(memory=None,\n",
              "         steps=[('count',\n",
              "                 CountVectorizer(analyzer='word', binary=False,\n",
              "                                 decode_error='strict',\n",
              "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
              "                                 input='content', lowercase=True, max_df=1.0,\n",
              "                                 max_features=None, min_df=1,\n",
              "                                 ngram_range=(1, 1), preprocessor=None,\n",
              "                                 stop_words=None, strip_accents=None,\n",
              "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
              "                                 tokenizer=None, vocabulary=None)),\n",
              "                ('tfid',\n",
              "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
              "                                  sublinear_tf=False, use_idf=True))],\n",
              "         verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Hk_w7tdL0Dhv",
        "outputId": "c489bf35-3fed-42ff-8217-12190c7e15dc"
      },
      "source": [
        "#TfidfVectorizer: Convierte una colección de textos sin procesar en una matriz TF-IDF\n",
        "\n",
        "tfidf_vectorizer = TfidfVectorizer()\n",
        "tfidf_vectorizer.fit([tweets[0],tweets[1]])\n",
        "tfidf_vectorizer.transform([tweets[0],tweets[1]]).toarray()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.17674221, 0.17674221, 0.        , 0.17674221, 0.        ,\n",
              "        0.        , 0.        , 0.17674221, 0.        , 0.        ,\n",
              "        0.        , 0.        , 0.17674221, 0.17674221, 0.        ,\n",
              "        0.17674221, 0.17674221, 0.        , 0.17674221, 0.17674221,\n",
              "        0.        , 0.        , 0.17674221, 0.        , 0.        ,\n",
              "        0.        , 0.17674221, 0.        , 0.        , 0.17674221,\n",
              "        0.        , 0.17674221, 0.        , 0.17674221, 0.        ,\n",
              "        0.17674221, 0.12575354, 0.17674221, 0.        , 0.        ,\n",
              "        0.        , 0.        , 0.17674221, 0.        , 0.35348442,\n",
              "        0.        , 0.17674221, 0.17674221, 0.        , 0.17674221,\n",
              "        0.17674221, 0.        , 0.        , 0.12575354, 0.        ,\n",
              "        0.        , 0.        , 0.        , 0.17674221, 0.17674221,\n",
              "        0.        , 0.17674221, 0.        , 0.        , 0.17674221,\n",
              "        0.17674221],\n",
              "       [0.        , 0.        , 0.38244897, 0.        , 0.12748299,\n",
              "        0.12748299, 0.12748299, 0.        , 0.12748299, 0.12748299,\n",
              "        0.12748299, 0.12748299, 0.        , 0.        , 0.12748299,\n",
              "        0.        , 0.        , 0.12748299, 0.        , 0.        ,\n",
              "        0.12748299, 0.12748299, 0.        , 0.12748299, 0.12748299,\n",
              "        0.12748299, 0.        , 0.12748299, 0.12748299, 0.        ,\n",
              "        0.12748299, 0.        , 0.12748299, 0.        , 0.12748299,\n",
              "        0.        , 0.0907052 , 0.        , 0.12748299, 0.12748299,\n",
              "        0.25496598, 0.25496598, 0.        , 0.12748299, 0.        ,\n",
              "        0.12748299, 0.        , 0.        , 0.12748299, 0.        ,\n",
              "        0.        , 0.25496598, 0.12748299, 0.18141039, 0.12748299,\n",
              "        0.25496598, 0.12748299, 0.25496598, 0.        , 0.        ,\n",
              "        0.12748299, 0.        , 0.12748299, 0.12748299, 0.        ,\n",
              "        0.        ]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yt4cbbyD0HBH"
      },
      "source": [
        "#### 3.4 Word embeddings ---------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d8KVkvI90KCn"
      },
      "source": [
        "Es un modelo más reciente que crea vectores para cada una de las palabras usando redes neuronales. El resultado es un conjunto de vectores de palabras donde los vectores cercanos en el espacio vectorial tienen significados similares basados ​​en el contexto, y los vectores de palabras distantes entre sí tienen significados diferentes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vwMZE2LY0Mll"
      },
      "source": [
        "import gensim\n",
        "from gensim.models.word2vec import Word2Vec\n",
        "import gensim.downloader as api\n",
        "\n",
        "def word2Vec(word):\n",
        "    model = api.load(\"glove-twitter-25\")\n",
        "    vec = model.get_vector(word)\n",
        "    return vec"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kQx-i12w0QaO",
        "outputId": "e82ab533-eeeb-4257-a222-799505f78d0a"
      },
      "source": [
        "print(word2Vec(x1[0])) # para la palabra \"incredible\""
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[-0.13495   0.90243   0.39679  -0.53241  -0.47991  -0.31209  -0.10571\n",
            "  1.0359    0.91999   0.90929  -0.93797   0.41424  -4.2842   -0.080945\n",
            " -1.0815   -1.0538    0.32917   0.39186  -0.34039   0.86967  -0.32634\n",
            " -0.80405   0.031681  0.70784   0.2564  ]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aipduiZE0bm7"
      },
      "source": [
        "Ahora entrenamos un dos tipos de modelo Word2Vec con el corpus formado por los 3 tweets aleatorios que cogimos al principio, para no sobrecargar el notebook. La parte principal del modelo es model.wv, donde “wv” significa “word vectors”."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TSKrHyME0fgI"
      },
      "source": [
        "> Modelo 1: palabras"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yencJFIM0hen",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "976b5dd4-b91e-4399-866b-194b5290493c"
      },
      "source": [
        "import multiprocessing\n",
        "cores = multiprocessing.cpu_count()\n",
        "print('Nº de cores: ', cores)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Nº de cores:  2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uKQDQhv60mgM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8504953c-0c7f-44e9-f9ca-aece65e3ad5f"
      },
      "source": [
        "#Creamos una lista de este estilo para poder pasársela al modelo\n",
        "x2 = [word_tokenize(i) for i in x]\n",
        "print(x2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[['wow', 'skirt', 'short', 'length', 'inch'], ['incredible', 'beautifulbut', 'laughed', 'much', 'read', 'drifting', 'wheelchairi', 'picture'], ['find', 'extremely', 'hard', 'believe', 'kelly', 'yr', 'old', 'mum', 'would', 'believe', 'makes', 'question', 'fuck', 'game', 'especially', 'none', 'apologised', 'tried', 'explain', 'amp', 'immediate', 'response', 'like', 'tweets', 'attacking', 'victims']]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kPj_gC_3069G"
      },
      "source": [
        "from gensim.models import Word2Vec\n",
        "model = Word2Vec(sentences=x2, window=5, min_count=1, workers=cores-1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "In5JPMZk0-4e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "72dfe23e-dfbd-4e72-f6c3-a44bbf1c02c2"
      },
      "source": [
        "model.train(x, total_examples=model.corpus_count, epochs=30, report_delay=1) #pasamos la lista de tweets\n",
        "model.wv.similarity('believe', 'game')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.10731904"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "76nD38O21ETz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "129e6e3e-bdeb-4f18-9d18-e97cfb742ac0"
      },
      "source": [
        "model.wv.most_similar(positive=[\"mum\"])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('picture', 0.2748515009880066),\n",
              " ('fuck', 0.1919270157814026),\n",
              " ('hard', 0.14750374853610992),\n",
              " ('attacking', 0.13505050539970398),\n",
              " ('yr', 0.12198891490697861),\n",
              " ('especially', 0.10104119777679443),\n",
              " ('extremely', 0.0963210016489029),\n",
              " ('question', 0.09349123388528824),\n",
              " ('short', 0.09020862728357315),\n",
              " ('wow', 0.0766962543129921)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ob9OicIk1GxI"
      },
      "source": [
        "> Modelo 2: bigramas"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J27TQEWE1KqA",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "dec8fcfb-40fb-4d4b-fcba-6851cf7aecab"
      },
      "source": [
        "from gensim.models import Phrases\n",
        "from gensim.models.phrases import Phraser\n",
        "# Train a bigram detector.\n",
        "bigram_transformer = Phrases(x2)\n",
        "\n",
        "# Apply the trained MWE detector to a corpus, using the result to train a Word2vec model.\n",
        "model2 = Word2Vec(bigram_transformer[x2], min_count=1)\n",
        "model2.train(x, total_examples=model.corpus_count, epochs=30, report_delay=1) #pasamos la lista de tweets"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/gensim/models/phrases.py:598: UserWarning: For a faster implementation, use the gensim.models.phrases.Phraser class\n",
            "  warnings.warn(\"For a faster implementation, use the gensim.models.phrases.Phraser class\")\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0, 8220)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fiDoiWaL1NYe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7caf9c98-0bdc-4151-d61d-3067ff0f0256"
      },
      "source": [
        "bigram_phraser = Phraser(bigram_transformer)\n",
        "print(bigram_phraser)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<gensim.models.phrases.Phraser object at 0x7f4c590ca050>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dz31PnMj1Qve"
      },
      "source": [
        "Como podemos observar, el modelo no identifica ningún bigrama importante en el conjunto de tweets seleccionados."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AdsYq9vw1TeR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4f7af6a4-6793-4245-b59d-0ed6ce96f1f4"
      },
      "source": [
        "model2.wv.similarity('believe', 'game')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.10731904"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8v4plqYm1XjK"
      },
      "source": [
        "Los resultados varían ligeramente de un modelo a otro."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "--IcJbEI2GZ6"
      },
      "source": [
        "### 4. Construcción de clasificador SVM."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Huq_iym1dla"
      },
      "source": [
        "Antes de meternos en construcción de modelos hay que comprobar si se trata de una tarea de clasificación de clases balanceadas o no, en este caso consideramos que la clase 'sexista' y 'no-sexista' están balanceadas."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i_wKRRWX1oW9",
        "outputId": "4cf17395-20fe-4562-dcca-1ecefefcaba1"
      },
      "source": [
        "# Vemos que las clases son  balanceadas \n",
        "from collections import Counter\n",
        "Counter(t1)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Counter({'non-sexist': 1800, 'sexist': 1636})"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hcgwZzT619ES"
      },
      "source": [
        "Aplicamos el preproceso anterior:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kTb6sdPG2PRC",
        "outputId": "7238b4e6-a783-4078-b0bc-b6af6ebabd0e"
      },
      "source": [
        "clean_tweets=[preprocessing(i) for i in tweets]\n",
        "print(len(clean_tweets)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3436\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8JVy9WCZ2TFQ"
      },
      "source": [
        "Para recodificar el input de nuestro futuro clasificador hemos optado por la codificación TF-IDF con TfidfVectorizer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aRoKNYlr2bha"
      },
      "source": [
        "tfidf_vectorizer = TfidfVectorizer()\n",
        "x = tfidf_vectorizer.fit_transform(clean_tweets)\n",
        "y = [ 1 if i=='sexist' else 0 for i in t1 ]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WFde6G2u28lT"
      },
      "source": [
        "Vamos a entrenar un modelo SVM probando difentes kernels y valores de c para ver cuál es el más óptimo de todos. Esta es la función que comprueba las diferentes opciones. Para no hacer un triple bucle hemos decidido suponer que en el caso de kernel=poly cogerémos el degree por defecto, es decir 3, ya que iterando diferentes grados para ese kernel tarda demasiado en ejecutarse."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OfolLBIr2kQW"
      },
      "source": [
        "def entrenar_SVM(X,Y, kernel, cv, c):  \n",
        "    clf = svm.SVC(kernel=kernel, C=c,\n",
        "                    gamma='scale')  \n",
        "    args = {'kernel': kernel}\n",
        "    clf.fit(X, Y)\n",
        "    scores = cross_val_score(clf, X, Y, cv=cv)\n",
        "    score_medio = scores.mean()\n",
        "    errores=[]\n",
        "    for score in scores:\n",
        "        errores.append(1-score)\n",
        "    mse = np.array(errores).mean()\n",
        "    \n",
        "\n",
        "    aux = 1.0 * np.array(errores)\n",
        "    media_errores = aux.mean()\n",
        "    # intervalo de confianza\n",
        "    #epsilon = 1.96 * math.sqrt((media_errores * \n",
        "                  #(1 - media_errores)) / len(X)) \n",
        "\n",
        "    return clf, score_medio, mse, args"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vc8Ijo972pVf"
      },
      "source": [
        "A continuación vemos que el modelo que mejor accuracy da es el SVM con kernel lineal, C=1 y validación cruzada con con fold=5.También habíamos probado con el perceptrón multicapa y regresión logística pero ambos modelos daban un accuracy mucho menor y la red neuronal además tardaba muchísimo en entrenarse.     \n",
        "\n",
        "\n",
        "No se ha añadido ningún parámetro de class weight en el modelo ni se ha hecho ningún preproceso de over o undersampling ya que las clases están balanceadas."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qtAWV59B2rUD",
        "outputId": "7d1fcc3f-08c0-482e-b0a9-4e6e63741edc"
      },
      "source": [
        "kernels = ['linear', 'rbf', 'poly', 'sigmoid']\n",
        "C=[0.01, 0.5, 1, 10]\n",
        "CV=5 \n",
        "current_score = 0\n",
        "for kernel in kernels:\n",
        "    for c in C:\n",
        "        clf, score_medio, mse, args = entrenar_SVM(x, y, kernel, CV, c)\n",
        "        \n",
        "        if score_medio > current_score:\n",
        "            current_score = score_medio\n",
        "            clf_mejor = clf\n",
        "            score_medio_mejor = score_medio\n",
        "            mse_mejor = mse\n",
        "            args_best = args\n",
        "            \n",
        "print(f'Mejor modelo {clf_mejor} con {args_best}')\n",
        "print(f'El error obtenido del mejor modelo es: {1-score_medio_mejor}')\n",
        "\n",
        "print(\n",
        "    f'Mejor resultado (accuracy): {score_medio_mejor} '\n",
        ")\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mejor modelo SVC(C=1, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
            "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='linear',\n",
            "    max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
            "    tol=0.001, verbose=False) con {'kernel': 'linear'}\n",
            "El error obtenido del mejor modelo es: 0.2753232795098338\n",
            "Mejor resultado (accuracy): 0.7246767204901662 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NYJPVQfjTzlo"
      },
      "source": [
        "Ahora aplicamos la misma función para encontrar un modelo de SVM que mejor prediga los niveles de sexismo, para eso recodificamos a numérica la variable task2 y aplicamos la misma función:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n2VOz3WWTwdZ",
        "outputId": "ea408a92-8850-49bf-91e7-7e86431d5f96"
      },
      "source": [
        "pd.value_counts(t2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "non-sexist                      1800\n",
              "ideological-inequality           386\n",
              "stereotyping-dominance           366\n",
              "sexual-violence                  344\n",
              "misogyny-non-sexual-violence     284\n",
              "objectification                  256\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X5swU_BoUpnW"
      },
      "source": [
        "y2=[]\n",
        "for i in t2:\n",
        "  if i == 'non-sexist':\n",
        "    y2.append(0)\n",
        "  elif i == 'ideological-inequality':\n",
        "    y2.append(1)\n",
        "  elif i == 'stereotyping-dominance':\n",
        "    y2.append(2)\n",
        "  elif i ==  'sexual-violence' :\n",
        "    y2.append(3)\n",
        "  elif i == 'misogyny-non-sexual-violence':\n",
        "    y2.append(4)\n",
        "  else:\n",
        "    y2.append(5)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l0RIpuOxVuW3",
        "outputId": "428a405a-ca75-4d16-82a5-fa2676b1a4a7"
      },
      "source": [
        "kernels = ['linear', 'rbf', 'poly', 'sigmoid']\n",
        "C=[0.01, 0.5, 1, 10]\n",
        "CV=5 \n",
        "current_score = 0\n",
        "for kernel in kernels:\n",
        "    for c in C:\n",
        "        clf, score_medio, mse, args = entrenar_SVM(x, y2, kernel, CV, c)\n",
        "        \n",
        "        if score_medio > current_score:\n",
        "            current_score = score_medio\n",
        "            clf_mejor = clf\n",
        "            score_medio_mejor = score_medio\n",
        "            mse_mejor = mse\n",
        "            args_best = args\n",
        "            \n",
        "print(f'Mejor modelo {clf_mejor} con {args_best}')\n",
        "print(f'El error obtenido del mejor modelo es: {1-score_medio_mejor}')\n",
        "\n",
        "print(\n",
        "    f'Mejor resultado (accuracy): {score_medio_mejor} '\n",
        ")\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mejor modelo SVC(C=1, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
            "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='linear',\n",
            "    max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
            "    tol=0.001, verbose=False) con {'kernel': 'linear'}\n",
            "El error obtenido del mejor modelo es: 0.39580413662367564\n",
            "Mejor resultado (accuracy): 0.6041958633763244 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "53HW0MkCXhqY"
      },
      "source": [
        "Vemos que para predecir los niveles de sexismo el accuracy es bastante peor. La mejor opción sigue siendo kernel = 'linear' y C=1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Z1UAc_E3be6"
      },
      "source": [
        "A continuación vamos a probar GridSearch también por si a caso para ver si con otra combinación de parámetros conseguimos mejor accuracy medio."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TB4HmAqU3fmn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "42b3b507-51ac-40a8-9ee1-c1e401c6a2e0"
      },
      "source": [
        "from sklearn import svm\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "\n",
        "clf = GridSearchCV(estimator=svm.SVC(), cv=5, param_grid={'C': [1, 10], 'kernel': ('linear', 'rbf')})\n",
        "clf.fit(x, y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=5, error_score=nan,\n",
              "             estimator=SVC(C=1.0, break_ties=False, cache_size=200,\n",
              "                           class_weight=None, coef0=0.0,\n",
              "                           decision_function_shape='ovr', degree=3,\n",
              "                           gamma='scale', kernel='rbf', max_iter=-1,\n",
              "                           probability=False, random_state=None, shrinking=True,\n",
              "                           tol=0.001, verbose=False),\n",
              "             iid='deprecated', n_jobs=None,\n",
              "             param_grid={'C': [1, 10], 'kernel': ('linear', 'rbf')},\n",
              "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
              "             scoring=None, verbose=0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eSkZHqvpcLxZ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f3e38231-1832-4b63-b8ea-e99a4a5ad3cd"
      },
      "source": [
        "clf.cv_results_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'mean_fit_time': array([1.11858964, 1.34735632, 1.29625525, 1.35825481]),\n",
              " 'mean_score_time': array([0.23969054, 0.28445454, 0.22347679, 0.29165301]),\n",
              " 'mean_test_score': array([0.72467672, 0.71071011, 0.68684032, 0.71739955]),\n",
              " 'param_C': masked_array(data=[1, 1, 10, 10],\n",
              "              mask=[False, False, False, False],\n",
              "        fill_value='?',\n",
              "             dtype=object),\n",
              " 'param_kernel': masked_array(data=['linear', 'rbf', 'linear', 'rbf'],\n",
              "              mask=[False, False, False, False],\n",
              "        fill_value='?',\n",
              "             dtype=object),\n",
              " 'params': [{'C': 1, 'kernel': 'linear'},\n",
              "  {'C': 1, 'kernel': 'rbf'},\n",
              "  {'C': 10, 'kernel': 'linear'},\n",
              "  {'C': 10, 'kernel': 'rbf'}],\n",
              " 'rank_test_score': array([1, 3, 4, 2], dtype=int32),\n",
              " 'split0_test_score': array([0.73546512, 0.71075581, 0.70348837, 0.73255814]),\n",
              " 'split1_test_score': array([0.72925764, 0.72634643, 0.69432314, 0.72343523]),\n",
              " 'split2_test_score': array([0.72197962, 0.70596798, 0.71179039, 0.71906841]),\n",
              " 'split3_test_score': array([0.71615721, 0.69723435, 0.66812227, 0.70451237]),\n",
              " 'split4_test_score': array([0.72052402, 0.713246  , 0.65647744, 0.70742358]),\n",
              " 'std_fit_time': array([0.01257114, 0.00669891, 0.00776336, 0.00865786]),\n",
              " 'std_score_time': array([0.0024085 , 0.00397889, 0.00489284, 0.00323887]),\n",
              " 'std_test_score': array([0.006848  , 0.00953951, 0.02110889, 0.01034018])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KFUqu0Wwd8qH"
      },
      "source": [
        "Vemos que la mejor combinación de parámetros que devuelve GridSearch es la misma con la que dimos nosotras mediante nuestra función y efectivamente da el mismo accuracy que teníamos hasta ahora:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H0DWb5HwdYLr",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f72c327d-d9dd-490a-b3cc-7b444805030e"
      },
      "source": [
        "clf.best_estimator_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(C=1, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
              "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='linear',\n",
              "    max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
              "    tol=0.001, verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hDE_vLwZeGna",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7ed271db-d8f7-4078-886d-41b5fb008042"
      },
      "source": [
        "clf.best_score_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7246767204901662"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 68
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BdeeALTGeS3k"
      },
      "source": [
        "### 5. BERT"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cDr1NWQ_38l0"
      },
      "source": [
        "A partir de ahora vamos a probar el modelo de Bert para ver si mejoramos el accuracy, para eso volvemos a importar los datos sin preprocesarlos. Será necesario convertir la variable target en una variable numérica. Además al ser posible utilizar el modelo Bert_multilingual, en este apartado utilizaremos tanto los tweets ingleses como los tweets españoles para entrenar el modelo."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bVhUCoT7fLqr"
      },
      "source": [
        "**Utilizaremos la librería Transformers de hugging face para nuestro objetivo:**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uxjaOcIQ4ybQ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "783eccaf-2bf2-4fc6-a23d-b8e80917397a"
      },
      "source": [
        "!pip install transformers\n",
        "import transformers\n",
        "from transformers import BertModel, BertTokenizer, AdamW, get_linear_schedule_with_warmup\n",
        "import torch\n",
        "from transformers import pipeline #The easiest way to use a pretrained model on a given task is to use pipeline().\n",
        "!pip install pytorch_lightning\n",
        "import pytorch_lightning as pl \n",
        "from pytorch_lightning import Trainer\n",
        "from torch import nn, optim\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "from sklearn.model_selection import train_test_split\n",
        "from textwrap import wrap"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting transformers\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/b0/9e/5b80becd952d5f7250eaf8fc64b957077b12ccfe73e9c03d37146ab29712/transformers-4.6.0-py3-none-any.whl (2.3MB)\n",
            "\r\u001b[K     |▏                               | 10kB 16.1MB/s eta 0:00:01\r\u001b[K     |▎                               | 20kB 23.4MB/s eta 0:00:01\r\u001b[K     |▍                               | 30kB 27.3MB/s eta 0:00:01\r\u001b[K     |▋                               | 40kB 28.5MB/s eta 0:00:01\r\u001b[K     |▊                               | 51kB 30.8MB/s eta 0:00:01\r\u001b[K     |▉                               | 61kB 27.0MB/s eta 0:00:01\r\u001b[K     |█                               | 71kB 28.2MB/s eta 0:00:01\r\u001b[K     |█▏                              | 81kB 29.0MB/s eta 0:00:01\r\u001b[K     |█▎                              | 92kB 30.1MB/s eta 0:00:01\r\u001b[K     |█▍                              | 102kB 30.8MB/s eta 0:00:01\r\u001b[K     |█▌                              | 112kB 30.8MB/s eta 0:00:01\r\u001b[K     |█▊                              | 122kB 30.8MB/s eta 0:00:01\r\u001b[K     |█▉                              | 133kB 30.8MB/s eta 0:00:01\r\u001b[K     |██                              | 143kB 30.8MB/s eta 0:00:01\r\u001b[K     |██                              | 153kB 30.8MB/s eta 0:00:01\r\u001b[K     |██▎                             | 163kB 30.8MB/s eta 0:00:01\r\u001b[K     |██▍                             | 174kB 30.8MB/s eta 0:00:01\r\u001b[K     |██▌                             | 184kB 30.8MB/s eta 0:00:01\r\u001b[K     |██▋                             | 194kB 30.8MB/s eta 0:00:01\r\u001b[K     |██▉                             | 204kB 30.8MB/s eta 0:00:01\r\u001b[K     |███                             | 215kB 30.8MB/s eta 0:00:01\r\u001b[K     |███                             | 225kB 30.8MB/s eta 0:00:01\r\u001b[K     |███▎                            | 235kB 30.8MB/s eta 0:00:01\r\u001b[K     |███▍                            | 245kB 30.8MB/s eta 0:00:01\r\u001b[K     |███▌                            | 256kB 30.8MB/s eta 0:00:01\r\u001b[K     |███▋                            | 266kB 30.8MB/s eta 0:00:01\r\u001b[K     |███▉                            | 276kB 30.8MB/s eta 0:00:01\r\u001b[K     |████                            | 286kB 30.8MB/s eta 0:00:01\r\u001b[K     |████                            | 296kB 30.8MB/s eta 0:00:01\r\u001b[K     |████▏                           | 307kB 30.8MB/s eta 0:00:01\r\u001b[K     |████▍                           | 317kB 30.8MB/s eta 0:00:01\r\u001b[K     |████▌                           | 327kB 30.8MB/s eta 0:00:01\r\u001b[K     |████▋                           | 337kB 30.8MB/s eta 0:00:01\r\u001b[K     |████▊                           | 348kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████                           | 358kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████                           | 368kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████▏                          | 378kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████▎                          | 389kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████▌                          | 399kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████▋                          | 409kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████▊                          | 419kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████                          | 430kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████                          | 440kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████▏                         | 450kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████▎                         | 460kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████▌                         | 471kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████▋                         | 481kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████▊                         | 491kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████▉                         | 501kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████                         | 512kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████▏                        | 522kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████▎                        | 532kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████▍                        | 542kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████▋                        | 552kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████▊                        | 563kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████▉                        | 573kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████                        | 583kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████▏                       | 593kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████▎                       | 604kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████▍                       | 614kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████▋                       | 624kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████▊                       | 634kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████▉                       | 645kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████                       | 655kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████▏                      | 665kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████▎                      | 675kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████▍                      | 686kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████▌                      | 696kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████▊                      | 706kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████▉                      | 716kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████                      | 727kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████                      | 737kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████▎                     | 747kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████▍                     | 757kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████▌                     | 768kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████▋                     | 778kB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████▉                     | 788kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████                     | 798kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████                     | 808kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████▎                    | 819kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████▍                    | 829kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████▌                    | 839kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████▋                    | 849kB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████▉                    | 860kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████                    | 870kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████                    | 880kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████▏                   | 890kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████▍                   | 901kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████▌                   | 911kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████▋                   | 921kB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████▊                   | 931kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████                   | 942kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████                   | 952kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████▏                  | 962kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████▎                  | 972kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████▌                  | 983kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████▋                  | 993kB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████▊                  | 1.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████                  | 1.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████                  | 1.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████▏                 | 1.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████▎                 | 1.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████▌                 | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████▋                 | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████▊                 | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████▉                 | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████                 | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████▏                | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████▎                | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████▍                | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████▋                | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████▊                | 1.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████▉                | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████                | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████▏               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████▎               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████▍               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████▋               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████▊               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████▉               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████               | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▏              | 1.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▎              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▍              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▌              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▊              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▉              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████              | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████▎             | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████▍             | 1.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████▌             | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████▋             | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████▉             | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████             | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████             | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████▎            | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████▍            | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████▌            | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████▋            | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████▉            | 1.4MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████            | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████            | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████▏           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████▍           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████▌           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████▋           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████▊           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████           | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████▏          | 1.5MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████▎          | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████▌          | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████▋          | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████▊          | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████▉          | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████          | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████▏         | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████▎         | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████▌         | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████▋         | 1.6MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████▊         | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████▉         | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████         | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████▏        | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████▎        | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████▍        | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████▋        | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████▊        | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████▉        | 1.7MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████        | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████▏       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████▎       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████▍       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████▌       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████▊       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████▉       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████       | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▏      | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▎      | 1.8MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▍      | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▌      | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▊      | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▉      | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████      | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████      | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▎     | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▍     | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▌     | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▋     | 1.9MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▉     | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████     | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████     | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████▏    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████▍    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████▌    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████▋    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████▉    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████    | 2.0MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▏   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▍   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▌   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▋   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▊   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████   | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▏  | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▎  | 2.1MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▌  | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▋  | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▊  | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▉  | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████  | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▏ | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▎ | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▌ | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▋ | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▊ | 2.2MB 30.8MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▉ | 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████ | 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▏| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▎| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▍| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▋| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▊| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▉| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 2.3MB 30.8MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 2.3MB 30.8MB/s \n",
            "\u001b[?25hRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2019.12.20)\n",
            "Collecting tokenizers<0.11,>=0.10.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ae/04/5b870f26a858552025a62f1649c20d29d2672c02ff3c3fb4c688ca46467a/tokenizers-0.10.2-cp37-cp37m-manylinux2010_x86_64.whl (3.3MB)\n",
            "\u001b[K     |████████████████████████████████| 3.3MB 47.6MB/s \n",
            "\u001b[?25hCollecting huggingface-hub==0.0.8\n",
            "  Downloading https://files.pythonhosted.org/packages/a1/88/7b1e45720ecf59c6c6737ff332f41c955963090a18e72acbcbeac6b25e86/huggingface_hub-0.0.8-py3-none-any.whl\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.41.1)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from transformers) (4.0.1)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
            "Collecting sacremoses\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/75/ee/67241dc87f266093c533a2d4d3d69438e57d7a90abb216fa076e7d475d4a/sacremoses-0.0.45-py3-none-any.whl (895kB)\n",
            "\u001b[K     |████████████████████████████████| 901kB 49.3MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.19.5)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from transformers) (20.9)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.0.12)\n",
            "Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->transformers) (3.7.4.3)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->transformers) (3.4.1)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2020.12.5)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (1.0.1)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (7.1.2)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (1.15.0)\n",
            "Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->transformers) (2.4.7)\n",
            "Installing collected packages: tokenizers, huggingface-hub, sacremoses, transformers\n",
            "Successfully installed huggingface-hub-0.0.8 sacremoses-0.0.45 tokenizers-0.10.2 transformers-4.6.0\n",
            "Collecting pytorch_lightning\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/c2/a1/a991780873b5fd760fb99dfda01916fe9e5b186f0ba70a120e6b4f79cfaa/pytorch_lightning-1.3.1-py3-none-any.whl (805kB)\n",
            "\u001b[K     |████████████████████████████████| 808kB 36.1MB/s \n",
            "\u001b[?25hRequirement already satisfied: torch>=1.4 in /usr/local/lib/python3.7/dist-packages (from pytorch_lightning) (1.8.1+cu101)\n",
            "Requirement already satisfied: tensorboard!=2.5.0,>=2.2.0 in /usr/local/lib/python3.7/dist-packages (from pytorch_lightning) (2.4.1)\n",
            "Requirement already satisfied: numpy>=1.17.2 in /usr/local/lib/python3.7/dist-packages (from pytorch_lightning) (1.19.5)\n",
            "Collecting pyDeprecate==0.3.0\n",
            "  Downloading https://files.pythonhosted.org/packages/14/52/aa227a0884df71ed1957649085adf2b8bc2a1816d037c2f18b3078854516/pyDeprecate-0.3.0-py3-none-any.whl\n",
            "Requirement already satisfied: tqdm>=4.41.0 in /usr/local/lib/python3.7/dist-packages (from pytorch_lightning) (4.41.1)\n",
            "Collecting future>=0.17.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829kB)\n",
            "\u001b[K     |████████████████████████████████| 829kB 43.2MB/s \n",
            "\u001b[?25hCollecting torchmetrics>=0.2.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/3b/e8/513cd9d0b1c83dc14cd8f788d05cd6a34758d4fd7e4f9e5ecd5d7d599c95/torchmetrics-0.3.2-py3-none-any.whl (274kB)\n",
            "\u001b[K     |████████████████████████████████| 276kB 41.6MB/s \n",
            "\u001b[?25hCollecting fsspec[http]>=2021.4.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e9/91/2ef649137816850fa4f4c97c6f2eabb1a79bf0aa2c8ed198e387e373455e/fsspec-2021.4.0-py3-none-any.whl (108kB)\n",
            "\u001b[K     |████████████████████████████████| 112kB 38.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from pytorch_lightning) (20.9)\n",
            "Collecting PyYAML<=5.4.1,>=5.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7a/a5/393c087efdc78091afa2af9f1378762f9821c9c1d7a22c5753fb5ac5f97a/PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636kB)\n",
            "\u001b[K     |████████████████████████████████| 645kB 42.7MB/s \n",
            "\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch>=1.4->pytorch_lightning) (3.7.4.3)\n",
            "Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (3.12.4)\n",
            "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (2.23.0)\n",
            "Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.32.0)\n",
            "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.15.0)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.0.1)\n",
            "Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (0.12.0)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (0.4.4)\n",
            "Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (0.36.2)\n",
            "Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.30.0)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (3.3.4)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.8.0)\n",
            "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (56.1.0)\n",
            "Collecting aiohttp; extra == \"http\"\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/88/c0/5890b4c8b04a79b7360e8fe4490feb0bb3ab179743f199f0e6220cebd568/aiohttp-3.7.4.post0-cp37-cp37m-manylinux2014_x86_64.whl (1.3MB)\n",
            "\u001b[K     |████████████████████████████████| 1.3MB 39.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->pytorch_lightning) (2.4.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (2020.12.5)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.24.3)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (3.0.4)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (1.3.0)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (0.2.8)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3.6\" in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (4.7.2)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (4.2.2)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (4.0.1)\n",
            "Collecting yarl<2.0,>=1.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/f1/62/046834c5fc998c88ab2ef722f5d42122230a632212c8afa76418324f53ff/yarl-1.6.3-cp37-cp37m-manylinux2014_x86_64.whl (294kB)\n",
            "\u001b[K     |████████████████████████████████| 296kB 51.6MB/s \n",
            "\u001b[?25hCollecting multidict<7.0,>=4.5\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7c/a6/4123b8165acbe773d1a8dc8e3f0d1edea16d29f7de018eda769abb56bd30/multidict-5.1.0-cp37-cp37m-manylinux2014_x86_64.whl (142kB)\n",
            "\u001b[K     |████████████████████████████████| 143kB 50.0MB/s \n",
            "\u001b[?25hCollecting async-timeout<4.0,>=3.0\n",
            "  Downloading https://files.pythonhosted.org/packages/e1/1e/5a4441be21b0726c4464f3f23c8b19628372f606755a9d2e46c187e65ec4/async_timeout-3.0.1-py3-none-any.whl\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp; extra == \"http\"->fsspec[http]>=2021.4.0->pytorch_lightning) (21.2.0)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (3.1.0)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (0.4.8)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard!=2.5.0,>=2.2.0->pytorch_lightning) (3.4.1)\n",
            "Building wheels for collected packages: future\n",
            "  Building wheel for future (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for future: filename=future-0.18.2-cp37-none-any.whl size=491058 sha256=2a4df517b0005031189d1f1e7f197e819df866c46281710717355af888f7c51f\n",
            "  Stored in directory: /root/.cache/pip/wheels/8b/99/a0/81daf51dcd359a9377b110a8a886b3895921802d2fc1b2397e\n",
            "Successfully built future\n",
            "Installing collected packages: pyDeprecate, future, torchmetrics, multidict, yarl, async-timeout, aiohttp, fsspec, PyYAML, pytorch-lightning\n",
            "  Found existing installation: future 0.16.0\n",
            "    Uninstalling future-0.16.0:\n",
            "      Successfully uninstalled future-0.16.0\n",
            "  Found existing installation: PyYAML 3.13\n",
            "    Uninstalling PyYAML-3.13:\n",
            "      Successfully uninstalled PyYAML-3.13\n",
            "Successfully installed PyYAML-5.4.1 aiohttp-3.7.4.post0 async-timeout-3.0.1 fsspec-2021.4.0 future-0.18.2 multidict-5.1.0 pyDeprecate-0.3.0 pytorch-lightning-1.3.1 torchmetrics-0.3.2 yarl-1.6.3\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "yaml"
                ]
              }
            }
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ANi-Q-yUsmIk"
      },
      "source": [
        "df = pd.read_csv('EXIST2021_training.tsv', sep='\\t')\n",
        "dftest = pd.read_csv('EXIST2021_test.tsv', sep='\\t')\n",
        "#df = df[df['language'] == 'en'] \n",
        "df['task1'] = df['task1'].replace({'sexist': 1, 'non-sexist':  0})  # Codificamos la variable target a numérica\n",
        "df['task2'] = df['task2'].replace({'non-sexist': 0, 'ideological-inequality':  1, 'stereotyping-dominance': 2, 'sexual-violence': 3, 'misogyny-non-sexual-violence': 4, 'objectification': 5 }) \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8juZp9EK5_zM"
      },
      "source": [
        " #Se importa el conjunto test para pasarlo posteriormente al modelo entrenado y guardar las predicciones las que se enviarán al concurso\n",
        "\n",
        "\n",
        "#df = pd.read_csv('EXIST2021_test.tsv', sep='\\t')\n",
        "#df = df[df['language'] == 'en'] \n",
        "#df['task1'] = df['task1'].replace({'sexist': 1, 'non-sexist':  0})  # Codificamos la variable target a numérica"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "whWqKUbOfAYP"
      },
      "source": [
        "Implementación del modelo preentrenado BERT de Transformers con PyTorch con ayuda del tutorial (https://www.youtube.com/watch?v=mvh7DV84mr4):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X6lsMOT5YZEw",
        "outputId": "d8638fa0-82f1-4b44-f7b8-57507184a54e"
      },
      "source": [
        "# Inicialización parámetros.\n",
        "\n",
        "\n",
        "RANDOM_SEED = 58\n",
        "MAX_LEN = 160 # basándonos en la longitud de los tokens tras el proceso de tokenización\n",
        "BATCH_SIZE = 16\n",
        "NCLASSES = 6 # 2 clases en la variable respuesta task1, 6 clases en la variable task2\n",
        "\n",
        "np.random.seed(RANDOM_SEED)\n",
        "torch.manual_seed(RANDOM_SEED)\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "\n",
        "\n",
        "print(device)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "cuda:0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mBANW0lpYvzC"
      },
      "source": [
        "# Importamos el modelo de interes y su tokenizador. Hemos elegido la versión de Bert que distingue entre mayúsculas y minúsculas.\n",
        "PRE_TRAINED_MODEL_NAME = 'bert-base-multilingual-cased'\n",
        "tokenizer = BertTokenizer.from_pretrained(PRE_TRAINED_MODEL_NAME)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Szh-0SAZQrA"
      },
      "source": [
        "# CREACIÓN DATASET\n",
        "\n",
        "class IMDBDataset(Dataset):\n",
        "\n",
        "  def __init__(self,reviews,labels,tokenizer,max_len):\n",
        "    self.reviews = reviews\n",
        "    self.labels = labels\n",
        "    self.tokenizer = tokenizer\n",
        "    self.max_len = max_len\n",
        "\n",
        "  def __len__(self):\n",
        "      return len(self.reviews)\n",
        "    \n",
        "  def __getitem__(self, item):\n",
        "    review = str(self.reviews[item])\n",
        "    label = self.labels[item]\n",
        "    encoding = tokenizer.encode_plus(\n",
        "        review,\n",
        "        max_length = self.max_len,\n",
        "        truncation = True,\n",
        "        add_special_tokens = True,\n",
        "        return_token_type_ids = False,\n",
        "        pad_to_max_length = True,\n",
        "        return_attention_mask = True,\n",
        "        return_tensors = 'pt'\n",
        "        )\n",
        "\n",
        "    return {\n",
        "          'review': review,\n",
        "          'input_ids': encoding['input_ids'].flatten(),\n",
        "          'attention_mask': encoding['attention_mask'].flatten(),\n",
        "          'label': torch.tensor(label, dtype=torch.long)\n",
        "      }"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k7wy0WS0Zltz"
      },
      "source": [
        "# Data loader:\n",
        "\n",
        "def data_loader(df, tokenizer, max_len, batch_size):\n",
        "  dataset = IMDBDataset(\n",
        "      reviews = df.text.to_numpy(),\n",
        "      labels = df.task2.to_numpy(),\n",
        "      tokenizer = tokenizer,\n",
        "      max_len = MAX_LEN\n",
        "  )\n",
        "\n",
        "  return DataLoader(dataset, batch_size = BATCH_SIZE, num_workers = 4)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "me5AVykwZ2_i",
        "outputId": "46610cc2-463c-41b1-c3a7-c28b88f3a6bd"
      },
      "source": [
        "# Realizamos la partición train - validación.\n",
        "df_train, df_test = train_test_split(df, test_size = 0.2, random_state=RANDOM_SEED)\n",
        "\n",
        "train_data_loader = data_loader(df_train, tokenizer, MAX_LEN, BATCH_SIZE)\n",
        "test_data_loader = data_loader(df_test, tokenizer, MAX_LEN, BATCH_SIZE)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n",
            "  cpuset_checked))\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jiq7mGfBa5in"
      },
      "source": [
        "# EL MODELO!\n",
        "\n",
        "class BERTSentimentClassifier(nn.Module):\n",
        "\n",
        "  def __init__(self, n_classes):\n",
        "    super(BERTSentimentClassifier, self).__init__()\n",
        "    self.bert = BertModel.from_pretrained(PRE_TRAINED_MODEL_NAME)\n",
        "    self.drop = nn.Dropout(p=0.1)\n",
        "    self.linear = nn.Linear(self.bert.config.hidden_size, n_classes)\n",
        "\n",
        "  def forward(self, input_ids, attention_mask):\n",
        "    _, cls_output = self.bert(\n",
        "        return_dict=False,\n",
        "        input_ids = input_ids,\n",
        "        attention_mask = attention_mask\n",
        "    )\n",
        "    drop_output = self.drop(cls_output)\n",
        "    output = self.linear(drop_output)\n",
        "    return output\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hnEi0lcaeaWQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d00c34f8-0c61-4c36-e2c0-483c86211e53"
      },
      "source": [
        "model = BERTSentimentClassifier(NCLASSES)\n",
        "model = model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at bert-base-multilingual-cased were not used when initializing BertModel: ['cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.decoder.weight']\n",
            "- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n923ZFJfegRl"
      },
      "source": [
        "# ENTRENAMIENTO\n",
        "# (For a typical Pytorch training cycle, we need to implement the loop for epochs, iterate through the mini-batches, \n",
        "# perform feedforward pass for each mini-batch, compute the loss, \n",
        "# perform backpropagation for each batch and then finally update the gradients.)\n",
        "\n",
        "EPOCHS = 5\n",
        "optimizer = AdamW(model.parameters(), lr=1e-5, correct_bias=False)\n",
        "total_steps = len(train_data_loader) * EPOCHS\n",
        "scheduler = get_linear_schedule_with_warmup(\n",
        "    optimizer,\n",
        "    num_warmup_steps = 0,\n",
        "    num_training_steps = total_steps\n",
        ")\n",
        "loss_fn = nn.CrossEntropyLoss().to(device)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UExED9kIemLu"
      },
      "source": [
        "# Iteración entrenamiento\n",
        "def train_model(model, data_loader, loss_fn, optimizer, device, scheduler, n_examples):\n",
        "  model = model.train()\n",
        "  losses = []\n",
        "  correct_predictions = 0\n",
        "  for batch in data_loader:\n",
        "    input_ids = batch['input_ids'].to(device)\n",
        "    attention_mask = batch['attention_mask'].to(device)\n",
        "    labels = batch['label'].to(device)\n",
        "    outputs = model(input_ids = input_ids, attention_mask = attention_mask)\n",
        "    _, preds = torch.max(outputs, dim=1)\n",
        "    loss = loss_fn(outputs, labels)\n",
        "    correct_predictions += torch.sum(preds == labels)\n",
        "    losses.append(loss.item())\n",
        "    loss.backward()\n",
        "    nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
        "    optimizer.step()\n",
        "    scheduler.step()\n",
        "    optimizer.zero_grad()\n",
        "  return correct_predictions.double()/n_examples, np.mean(losses)\n",
        "\n",
        "def eval_model(model, data_loader, loss_fn, device, n_examples):\n",
        "  model = model.eval()\n",
        "  losses = []\n",
        "  correct_predictions = 0\n",
        "  with torch.no_grad():\n",
        "    for batch in data_loader:\n",
        "      input_ids = batch['input_ids'].to(device)\n",
        "      attention_mask = batch['attention_mask'].to(device)\n",
        "      labels = batch['label'].to(device)\n",
        "      outputs = model(input_ids = input_ids, attention_mask = attention_mask)\n",
        "      _, preds = torch.max(outputs, dim=1)\n",
        "      loss = loss_fn(outputs, labels)\n",
        "      correct_predictions += torch.sum(preds == labels)\n",
        "      losses.append(loss.item())\n",
        "  return correct_predictions.double()/n_examples, np.mean(losses)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kqhzYXiCerjI",
        "outputId": "acc9580f-2513-4da5-9dc2-2651f6869098"
      },
      "source": [
        "# Entrenamiento!!!\n",
        "\n",
        "for epoch in range(EPOCHS):\n",
        "  print('Epoch {} de {}'.format(epoch+1, EPOCHS))\n",
        "  print('------------------')\n",
        "  train_acc, train_loss = train_model(\n",
        "      model, train_data_loader, loss_fn, optimizer, device, scheduler, len(df_train)\n",
        "  )\n",
        "  test_acc, test_loss = eval_model(\n",
        "      model, test_data_loader, loss_fn, device, len(df_test)\n",
        "  )\n",
        "  print('Entrenamiento: Loss: {}, accuracy: {}'.format(train_loss, train_acc))\n",
        "  print('Validación: Loss: {}, accuracy: {}'.format(test_loss, test_acc))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1 de 5\n",
            "------------------\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n",
            "  cpuset_checked))\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Entrenamiento: Loss: 1.299458673570764, accuracy: 0.5547392940333273\n",
            "Validación: Loss: 1.15576252409003, accuracy: 0.5616045845272206\n",
            "Epoch 2 de 5\n",
            "------------------\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Entrenamiento: Loss: 0.9559501386474402, accuracy: 0.6620677297975274\n",
            "Validación: Loss: 1.144628637216308, accuracy: 0.5773638968481375\n",
            "Epoch 3 de 5\n",
            "------------------\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Entrenamiento: Loss: 0.7493488409030745, accuracy: 0.7412650062712776\n",
            "Validación: Loss: 1.1533515107902614, accuracy: 0.6060171919770774\n",
            "Epoch 4 de 5\n",
            "------------------\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Entrenamiento: Loss: 0.6038289842513366, accuracy: 0.7994982977960939\n",
            "Validación: Loss: 1.1794308677993037, accuracy: 0.6181948424068767\n",
            "Epoch 5 de 5\n",
            "------------------\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Entrenamiento: Loss: 0.5169852087781217, accuracy: 0.8342590933524457\n",
            "Validación: Loss: 1.1962187757546252, accuracy: 0.6131805157593123\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ESLXHEyf7vxu"
      },
      "source": [
        "# Para guardar el modelo previamente entrenado\n",
        "\n",
        "#torch.save(model.state_dict(), 'bertMultilingual.bin')\n",
        "#modelo = torch.load('bertMultilingual.bin')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aeq9zXviFNDw"
      },
      "source": [
        "def classifySentiment(review_text):\n",
        "  encoding_review = tokenizer.encode_plus(\n",
        "      review_text,\n",
        "      max_length = MAX_LEN,\n",
        "      truncation = True,\n",
        "      add_special_tokens = True,\n",
        "      return_token_type_ids = False,\n",
        "      pad_to_max_length = True,\n",
        "      return_attention_mask = True,\n",
        "      return_tensors = 'pt'\n",
        "      )\n",
        "  \n",
        "  input_ids = encoding_review['input_ids'].to(device)\n",
        "  attention_mask = encoding_review['attention_mask'].to(device)\n",
        "  output = model(input_ids, attention_mask)\n",
        "  _, prediction = torch.max(output, dim=1)\n",
        "  #print(\"\\n\".join(wrap(review_text)))\n",
        "  return (prediction)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TZlc_emdbSUU"
      },
      "source": [
        "Predecimos y guardamos las predicciones en el formato solicitado por la competición:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-em5Lv4jFq3D",
        "outputId": "74b6c442-06dc-48b7-fd61-6127492a7f92"
      },
      "source": [
        "dftest['predicciones'] = dftest['text'].map(classifySentiment)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py:2110: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  FutureWarning,\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 200
        },
        "id": "KM13MRDXdOsm",
        "outputId": "a54519f9-7b15-48dd-df58-2a98f8f00a10"
      },
      "source": [
        "dftest.tail()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>test_case</th>\n",
              "      <th>id</th>\n",
              "      <th>source</th>\n",
              "      <th>language</th>\n",
              "      <th>text</th>\n",
              "      <th>predicciones</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4363</th>\n",
              "      <td>EXIST2021</td>\n",
              "      <td>11341</td>\n",
              "      <td>twitter</td>\n",
              "      <td>es</td>\n",
              "      <td>@IreneMontero Se llama nota de corte, y es lo ...</td>\n",
              "      <td>[tensor(2, device='cuda:0')]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4364</th>\n",
              "      <td>EXIST2021</td>\n",
              "      <td>11342</td>\n",
              "      <td>twitter</td>\n",
              "      <td>es</td>\n",
              "      <td>@freckles887 Osea todo atack of titan parte de...</td>\n",
              "      <td>[tensor(0, device='cuda:0')]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4365</th>\n",
              "      <td>EXIST2021</td>\n",
              "      <td>11343</td>\n",
              "      <td>twitter</td>\n",
              "      <td>es</td>\n",
              "      <td>@alabadomango Cuéntame más!!Es por androcentri...</td>\n",
              "      <td>[tensor(0, device='cuda:0')]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4366</th>\n",
              "      <td>EXIST2021</td>\n",
              "      <td>11344</td>\n",
              "      <td>gab</td>\n",
              "      <td>es</td>\n",
              "      <td>Que duro es ser tan atractiva como Jaba de Hutt.</td>\n",
              "      <td>[tensor(0, device='cuda:0')]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4367</th>\n",
              "      <td>EXIST2021</td>\n",
              "      <td>11345</td>\n",
              "      <td>twitter</td>\n",
              "      <td>es</td>\n",
              "      <td>@elmundoes A Pablo es que ya no le hacen caso ...</td>\n",
              "      <td>[tensor(0, device='cuda:0')]</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      test_case  ...                  predicciones\n",
              "4363  EXIST2021  ...  [tensor(2, device='cuda:0')]\n",
              "4364  EXIST2021  ...  [tensor(0, device='cuda:0')]\n",
              "4365  EXIST2021  ...  [tensor(0, device='cuda:0')]\n",
              "4366  EXIST2021  ...  [tensor(0, device='cuda:0')]\n",
              "4367  EXIST2021  ...  [tensor(0, device='cuda:0')]\n",
              "\n",
              "[5 rows x 6 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_n2p4CoaJxIw"
      },
      "source": [
        "def recodificar(x):\n",
        "  if str(x)[8] == '0':\n",
        "    return 'non-sexist'\n",
        "  elif str(x)[8] == '1':\n",
        "    return 'ideological-inequality'\n",
        "  elif str(x)[8] == '2':\n",
        "    return 'stereotyping-dominance'\n",
        "  elif str(x)[8] == '3':\n",
        "    return 'sexual-violence'\n",
        "  elif str(x)[8] == '4':\n",
        "    return 'misogyny-non-sexual-violence'\n",
        "  else:\n",
        "    return 'objectification'\n",
        "\n",
        "def id(x):\n",
        "  return str(x).zfill(6)\n",
        "\n",
        "#df['task2'] = df['task2'].replace({'non-sexist': 0, 'ideological-inequality':  1, 'stereotyping-dominance': 2, 'sexual-violence': 3, 'misogyny-non-sexual-violence': 4, 'objectification': 5 }) \n",
        "\n",
        "\n",
        "dftest['predicciones'] = dftest['predicciones'].apply(recodificar)\n",
        "dftest['id'] = dftest['id'].apply(id)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IpTp7z0WLFmT"
      },
      "source": [
        "#dftest.head(100)\n",
        "dftest = dftest.drop(['source','language', 'text'], axis=1)\n",
        "dftest.to_csv('task2_Andrea_Lisa_1.tsv',sep='\\t', header=None, index=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jQmJEYOvX4zq"
      },
      "source": [
        "def id(x):\n",
        "  return str(x).zfill(6)\n",
        "task1 = pd.read_csv('task1_Andrea_Lisa_1.tsv', sep='\\t', header=None)\n",
        "task1[1] = task1[1].apply(id)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "emzFgROXax3c",
        "outputId": "95b1b6de-3a8d-4d16-ae99-9e586d55e2b6"
      },
      "source": [
        "task1[1] = task1[1].astype(str)\n",
        "task1[1]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0       006978\n",
              "1       006979\n",
              "2       006980\n",
              "3       006981\n",
              "4       006982\n",
              "         ...  \n",
              "4363    011341\n",
              "4364    011342\n",
              "4365    011343\n",
              "4366    011344\n",
              "4367    011345\n",
              "Name: 1, Length: 4368, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O9Yp3zv42SMX"
      },
      "source": [
        "#task1 = pd.read_csv('task1_Andrea_Lisa_1.tsv', sep='\\t', header=None)\n",
        "#task1[2] = task1[2].replace({'sexist': 'non-sexist', 'non-sexist':  'sexist'})\n",
        "#task1.head()\n",
        "task1.to_csv('task1_Andrea_Lisa_1.tsv',sep='\\t', header=None, index=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ixc3BGTVzcAy"
      },
      "source": [
        "dftest.to_csv('task1_Andrea_Lisa_1.tsv',sep='\\t', header=None, index=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "00J80j4S6Wf_"
      },
      "source": [
        "Vemos que por ahora es el mejor accuracy obtenido."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ghMDpntG_s0g"
      },
      "source": [
        "\n",
        "\n",
        "### 7. StackingClassifier"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HniIMAKce7m5"
      },
      "source": [
        "La idea de Stacking implica el entrenamiento de un modelo para combinar las predicciones de otros modelos. \n",
        "\n",
        "Se comienza entrenando varios aprendices sobre los datos de entrenamient (modelos de primer nivel), y se termina entrenando el modelo final (modelo de segundo nivel o stacking model) sobre los datos originales considerando como características adicionales las predicciones de los primeros. Este modelo final es frecuentemente una regresión logística.  \n",
        "\n",
        "El rendimiento de este método de ensamblado aumenta cuanto más diversos sean los modelos de primer nivel, ya que cada uno de estos modelos explicará una parte diferente de la varianza de los datos originales."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5SZBHGtfAAjj"
      },
      "source": [
        "from sklearn.svm import SVC\n",
        "from sklearn import model_selection\n",
        "\n",
        "svm = SVC() \n",
        "lr = LogisticRegression()\n",
        "cart = DecisionTreeClassifier() \n",
        "rf = RandomForestClassifier()\n",
        "mlp = MLPClassifier(random_state=1, max_iter=300)\n",
        "knn = KNeighborsClassifier()\n",
        "\n",
        "estimators = [('svm', svm),('lr', lr),('cart', cart),('rf', rf),('mlp',mlp),('knn',knn)]\n",
        "\n",
        "sclf = StackingClassifier(estimators=estimators, final_estimator=lr)\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NEro_W6KgspH"
      },
      "source": [
        "Comparamos los modelos para Task1:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ddFLOEf7LB6G",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6713bd50-6150-4187-a59f-135db17fb2a0"
      },
      "source": [
        "for clf, label in zip([svm, lr, cart, rf, mlp, knn, sclf], ['SVC', 'Logistic Regression','Decision Tree','Random Forest', 'MLPClassifier','KNN', 'StackingClassifier']):\n",
        "    scores = model_selection.cross_val_score(clf, x, y, cv=3, scoring='accuracy')\n",
        "    print(\"Accuracy: %f [%s]\"  % (scores.mean(), label))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy: 0.699359 [SVC]\n",
            "Accuracy: 0.699651 [Logistic Regression]\n",
            "Accuracy: 0.679574 [Decision Tree]\n",
            "Accuracy: 0.712166 [Random Forest]\n",
            "Accuracy: 0.648137 [MLPClassifier]\n",
            "Accuracy: 0.628634 [KNN]\n",
            "Accuracy: 0.726716 [StackingClassifier]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CEM5Ylvzgk4e"
      },
      "source": [
        "Comparamos los modelos para Task2:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LmruTRfHWnUq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "31243b76-e17a-4379-d9ec-452ec07131fa"
      },
      "source": [
        "for clf, label in zip([svm, lr, cart, rf, mlp, knn, sclf], ['SVC', 'Logistic Regression','Decision Tree','Random Forest', 'MLPClassifier','KNN', 'StackingClassifier']):\n",
        "    scores = model_selection.cross_val_score(clf, x, y2, cv=3, scoring='accuracy')\n",
        "    print(\"Accuracy: %f [%s]\"  % (scores.mean(), label))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy: 0.561990 [SVC]\n",
            "Accuracy: 0.569850 [Logistic Regression]\n",
            "Accuracy: 0.550641 [Decision Tree]\n",
            "Accuracy: 0.601572 [Random Forest]\n",
            "Accuracy: 0.557625 [MLPClassifier]\n",
            "Accuracy: 0.555586 [KNN]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/neural_network/_multilayer_perceptron.py:573: UserWarning: Training interrupted by user.\n",
            "  warnings.warn(\"Training interrupted by user.\")\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/neural_network/_multilayer_perceptron.py:573: UserWarning: Training interrupted by user.\n",
            "  warnings.warn(\"Training interrupted by user.\")\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Accuracy: 0.621074 [StackingClassifier]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "496Hk2M50VL_"
      },
      "source": [
        "Vamos a evaluar las predicciones de task2 con la métrica f1-macro ya que es más robusta que el accuracy en este tipo de clasificaciones:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b9ukpXnUmqZy",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "54d3a298-8063-4470-d956-c4c960583bf7"
      },
      "source": [
        "for clf, label in zip([svm, lr, cart, rf, mlp, knn, sclf], ['SVC', 'Logistic Regression','Decision Tree','Random Forest', 'MLPClassifier','KNN', 'StackingClassifier']):\n",
        "    scores = model_selection.cross_val_score(clf, x, y2, cv=3, scoring='f1_macro')\n",
        "    print(\"F1 score: %f [%s]\"  % (scores.mean(), label))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "F1 score: 0.252657 [SVC]\n",
            "F1 score: 0.286128 [Logistic Regression]\n",
            "F1 score: 0.416182 [Decision Tree]\n",
            "F1 score: 0.386311 [Random Forest]\n",
            "F1 score: 0.375267 [MLPClassifier]\n",
            "F1 score: 0.391629 [KNN]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "F1 score: 0.450914 [StackingClassifier]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4y5wv77XXMNk"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}