data.json file access error [Bad Response]

[shivaganesht]

Installation method

PyPI (via pip)

Description

I encountered an issue while trying to access the data.json file from the Sherlock project. Below are the details to help reproduce the bug:

What I was trying to do: I was running the Sherlock tool to collect information about specific usernames on various platforms.

What I expected to happen: I expected the tool to download and access the data.json file successfully, allowing the program to run without any interruptions.

What actually happened: Instead of successfully accessing the data.json file, I received an error message stating: "Bad response while accessing data file URL https://raw.githubusercontent.com/sherlock-project/sherlock/master/sherlock/resources/data.json.This suggests that the file may not be accessible or may have been moved or deleted.

Comment: I checked the URL in my browser, and it seems to return an error, which indicates that the file may no longer be available. This issue could impact users trying to run the tool, as it relies on this data file. Please investigate the status of the data.json file and consider providing an alternative or a local copy option to ensure functionality.

Steps to reproduce

Install Sherlock:
Open your terminal or command prompt.
Run the command: pip install sherlock.

Run Sherlock:
In the terminal, execute the command: sherlock , replacing with any username you wish to search for.

Observe the Output:
Monitor the terminal output as the tool attempts to access the data.json file.

Error Encounter:
Note the error message that appears, specifically: "Bad response while accessing data file URL https://raw.githubusercontent.com/sherlock-project/sherlock/master/sherlock/resources/data.json.

Check URL Manually:
Open a web browser and navigate to https://raw.githubusercontent.com/sherlock-project/sherlock/master/sherlock/resources/data.json.
Confirm if the file is accessible or if an error (e.g., 404 Not Found) is displayed.

Additional information

Bug Reported by: Shiva Ganesh Talikota

Code of Conduct

• I agree to follow this project's Code of Conduct

[ppfeister]

What version of the tool are you running? (--version)

[Danila-Kovalenko]

Same here, my version is 0.14.4, and, sadly, it will not work

[ppfeister]

@Danila-Kovalenko 0.14.4 is long out of date and that behavior is expected due to a refactor. You should update your install.

Note that the Parrot repositories are out of date, so I would recommend using any other source. Trying to get an update pushed there sometime soon

[AshleyDownBelow]

@ppfeister both pipx and pip say 0.14.4 is latest despite updated package on PyPi

[ppfeister]

That would seem to be a local issue unfortunately and not one with Sherlock. I would however recommend trying --no-cache for pipx or --no-cache-dir for pip. @AshleyDownBelow Also applicable to @Danila-Kovalenko and @shivaganesht

Failing that, you'd have to set a higher verbosity and read through the logs. Setting no-cache may very well solve it though.

[shivaganesht]

What version of the tool are you running? (--version)

sherlock 0.14.4
Python HTTP for Humans.: 2.31.0
Python: 3.7.6

[shivaganesht]

@ppfeister both pipx and pip say 0.14.4 is latest despite updated package on PyPi

Yes, I am having the same issue here.

[shivaganesht]

That would seem to be a local issue unfortunately and not one with Sherlock. I would however recommend trying --no-cache for pipx or --no-cache-dir for pip. @AshleyDownBelow Also applicable to @Danila-Kovalenko and @shivaganesht

Failing that, you'd have to set a higher verbosity and read through the logs. Setting no-cache may very well solve it though.

I'm getting this error now. did I do it correctly? or guide me on how to execute this?

[shivaganesht]

That would seem to be a local issue unfortunately and not one with Sherlock. I would however recommend trying --no-cache for pipx or --no-cache-dir for pip. @AshleyDownBelow Also applicable to @Danila-Kovalenko and @shivaganesht
Failing that, you'd have to set a higher verbosity and read through the logs. Setting no-cache may very well solve it though.

I'm getting this error now. did I do it correctly? or guide me on how to execute this?

I'm not getting how to do it...

[ppfeister]

Let me be more specific --

# If using pipx
pipx upgrade sherlock-project --pip-args="--no-cache-dir"
# If using pip
pip install sherlock-project --upgrade --no-cache-dir

Can't say for sure if this will be what sets you right, but it's at least worth trying. I also have no idea what oddities might be present on the Windows side of things.

[shivaganesht]

Let me be more specific --

# If using pipx
pipx upgrade sherlock-project --pip-args="--no-cache-dir"
# If using pip
pip install sherlock-project --upgrade --no-cache-dir

Can't say for sure if this will be what sets you right, but it's at least worth trying. I also have no idea what oddities might be present on the Windows side of things.

I tried. No Luck. 🥲

[bytexenon]

sherlock 0.14.4 Python HTTP for Humans.: 2.31.0 Python: 3.7.6

Hello,

This issue seems to be related to your Python version. The latest versions of Sherlock no longer supports Python 3.7, so Pip automatically selects the latest compatible version, which unfortunately uses a broken GitHub URL for the data.json file.

To resolve this, try upgrading your Python version by running:

sudo apt update
sudo apt install python3

(or an alternative depending on your distribution). After upgrading, reinstall the package and it should work

[ppfeister]

Oh god, good catch @bytexenon

Didn't realize he was on 3.7. That went EOL almost a year and a half ago.

Those commands only work for deb + derivs (as you mention) but he can probably just get the latest for Windows from their website https://www.python.org/downloads/windows/