wootware_script

Please note: The script is for learning purposes and not for any malicious intent

Script working as of 6 Jan 2021

• Built a script to retrieve prices of current PC components on hand at Wootware.co.za
• Also added the ability to:
  • Add items to your cart to purchase at a later date
  • Display total cost at the end of adding items

---

Packages

• Selenium : https://pypi.org/project/selenium/
• BeautifulSoup : https://pypi.org/project/beautifulsoup4/
• Pandas: https://pypi.org/project/pandas/
• Python-decouple: https://pypi.org/project/python-decouple/

---

Table of content

1. How it works
2. Installation
3. How to use script(Demo)
4. Troubleshoot

---

How it works

• add_components() function prompts user which component/s it needs to retrieve from website to be stored in shopping_list

def add_components():
    comp = {1:'Internal Hard Drives', 2:'Processors / CPUs', 3:'Graphics Cards', 4:'Memory / RAM'}
    print("** Please enter components needed at Wootware.co.za**\n\n1. Internal Hard Drives\n2. Processors / CPUs\n3. Graphics Cards\n4. Memory / RAM\n")
    user_components= list(map(int,input("Please enter component needed (Spaces between them e.g 1 2 3 ) : ").split())) 
    user_components=list(dict.fromkeys(user_components)) # removes duplicates
    shopping_list =[comp[x] for x, y in zip(user_components,comp)]

• navigate_pages() creates list on which pages it needs to navigate to based on shopping_list.

def navigate_pages(value):
   comp = {
       'Internal Hard Drives':'https://bit.ly/3ahXX66', 
       'Processors / CPUs':'https://bit.ly/3r4bfsS',
       'Graphics Cards':'https://bit.ly/37rmDY7', 
       'Memory / RAM':'https://bit.ly/37qO5oU'
   }
   links=[comp[k] for k in value]

• website_nav() handles the selenium automation aswell as use get_scrap() to webscrape pages on Wootware.co.za to create list of conponents(e.g list of CPUs) into a dataframe called dataframe_results

dataframe_results=get_scrap(driver.current_url)
df_list.append(dataframe_results)  

• cart_components returns a list of items the users want to purchase based on code (e.g e-0,e-1,e-2 etc.)

list(input("Please enter component for cart (Spaces between them e.g e-1 e12 e-33 ) : ").split())

• In for_scrapping.py the get_link() function handles automating adding to cart based on cart_components which then displays the cart total at the end

cart_button=driver.find_element_by_id('product-addtocart-button').click()
y=driver.find_element_by_css_selector('div.subtotal span.price')
print('------------------------\nTotal in cart: ' ,y.get_attribute('innerHTML'),'\n------------------------\nBye!')  

Installation

1. Install pipenv

pip install pipenv

2. Clone repo
3. Open .env (it will be hidden in terminal) and enter registered user details for Wootware.co.za (email and password) to be used to sign in

//Example
email=test@gmail.com
password=123

4. Create pipenv

pipenv --three

5. Activate virtual environment

pipenv shell

6. To install packages

pip install -r requirements.txt

7. Install additional dependencies

sudo apt-get install -y unzip xvfb libxi6 libgconf-2-4

8. To deactivate virtual environment when you are done with the script

deactivate

How to use script

You can have two options on how to use the script:

a) Headless state (default)

b) User-interface state

To run Selenium with a user-interface(i.e browser interaction) comment out these lines in main.py:

Run

To run in Terminal

python3 src/main.py

To run in VScode:

1. Open VScode
2. Press Ctrl + Shift + P and select 'Python interpreter'
3. Choose python 'Wootware_script:pipenv'
4. Open main.py and run

Troubleshoot

If you have any issue with geckodriver try installing

sudo install firefox-geckodriver