Quora-Intent-segregation

This repository aims to segregate questions with similar intent into the same group .This is useful for sites like Quora.Here I explore various techniques to predict whether any given pair of questions are similar ?

Quora is a place to gain and share knowledge—about anything. It’s a platform to ask questions and connect with people who contribute unique insights and quality answers. This empowers people to learn from each other and to better understand the world.

Over 100 million people visit Quora every month, so it's no surprise that many people ask similarly worded questions. Multiple questions with the same intent can cause seekers to spend more time finding the best answer to their question, and make writers feel they need to answer multiple versions of the same question. Quora values canonical questions because they provide a better experience to active seekers and writers, and offer more value to both of these groups in the long term.

> Credits: Kaggle

Problem Statement

• Identify which questions asked on Quora are duplicates of questions that have already been asked.
• This could be useful to instantly provide answers to questions that have already been answered.
• We are tasked with predicting whether a pair of questions are duplicates or not.

SOURCE/USEFUL LINKS:

• Source : https://www.kaggle.com/c/quora-question-pairs
  
  ____ Useful Links ____
• Discussions : https://www.kaggle.com/anokas/data-analysis-xgboost-starter-0-35460-lb/comments
• Kaggle Winning Solution and other approaches: https://www.dropbox.com/sh/93968nfnrzh8bp5/AACZdtsApc1QSTQc7X0H3QZ5a?dl=0
• Blog 1 : https://engineering.quora.com/Semantic-Question-Matching-with-Deep-Learning
• Blog 2 : https://towardsdatascience.com/identifying-duplicate-questions-on-quora-top-12-on-kaggle-4c1cf93f1c30

This repo contains following notebooks alongside corresponding pdf versions:

1.Quora:Contains exploratory data analysis

2.Quora_Preprocessing:Contains some preprocessing done for ML pipeline.

2.ML_models: Extracted Features along side tfidf features of the data is run through various models ending with xgboost.

3.Quora_Mean_w2v:Here weighted w2v features of the data are used.

Bonus:A neural network model is also included....