Only pulling first page of results

[shadyKatie]

It seems that only the first page of results is appearing in the export (~14 jobs). Is there a way to pull all results for a given search?

[haasr]

Good point... I should fix that. I'm guessing I could just add some logic to check if there is a next page by looking for the x-paths of these buttons at the bottom and then keep clicking them as long as there are more and repeating the scrape. [cid:b18adc9a-42b4-4e53-a06d-31b98a3764f5] I'd like to work on this soon but I am hella busy right now. If you figure this out before I get to it, send me a pull request! I'm really shit with x-paths and Selenium but ChatGPT was helpful in re-writing this last time Indeed changed their page structure. Sincerely, Ryan Haas • he/him/his Professor of Using Information Technology East Tennessee State University Department of Computing

…

________________________________ From: shadyKatie ***@***.***> Sent: Tuesday, May 28, 2024 2:34 PM To: haasr/indeed-jobs-searcher ***@***.***> Cc: Subscribed ***@***.***> Subject: [haasr/indeed-jobs-searcher] Only pulling first page of results (Issue #1) It seems that only the first page of results is appearing in the export (~14 jobs). Is there a way to pull all results for a given search? — Reply to this email directly, view it on GitHub<#1>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALYCXYDRRGGTIV4BWWCICW3ZETE3RAVCNFSM6AAAAABINO36R6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDCNRYHA3DOMQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>