Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scraper Returns Error: {"errors":[{"code":366,"message":"Missing data."}]} when using the login() function #86

Open
aurmaza opened this issue May 7, 2024 · 6 comments

Comments

@aurmaza
Copy link

aurmaza commented May 7, 2024

When using the login function with username, password and email on a local instance of scraper, it works and I am able to use functions that need the login function. Furthermore, isLoggedIn also returns true.
However, when pushing the same code to a server on the cloud, I get the error below.
Screenshot 2024-05-06 202546
I am using the same exact inputs that I used on local on the cloud server
This code is not run client side so I am not using the proxy but I tried anyway and it gave further errors on both local and the cloud server.
Is there any configurations needed that I am unaware of?
Im also using express.js

@witherBattler
Copy link

Same exact issue with my app

@witherBattler
Copy link

Hey, what hosting service do you use? Also, do you do app.enable("trust proxy") in your code or not?

@witherBattler
Copy link

witherBattler commented Jul 25, 2024

I did some logging of the onboardingTaskUrl on line 266 of auth-user.js (as well as headers and data (body) that is send)

Here's what's being sent locally (with cleared cookies):

https://api.twitter.com/1.1/onboarding/task.json {} {"flow_name":"login","input_flow_data":{"flow_context":{"debug_overrides":{},"start_location":{"location":"splash_screen"}}}}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:0","subtask_inputs":[{"subtask_id":"LoginJsInstrumentationSubtask","js_instrumentation":{"response":"{}","link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:1","subtask_inputs":[{"subtask_id":"LoginEnterUserIdentifierSSO","settings_list":{"setting_responses":[{"key":"user_identifier","response_data":{"text_data":{"result":"allttt1807501"}}}],"link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:7","subtask_inputs":[{"subtask_id":"LoginEnterPassword","enter_password":{"password":"hopefullythismakemoney","link":"next_link"}}]} // this part is different from production 
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:8","subtask_inputs":[{"subtask_id":"AccountDuplicationCheck","check_logged_in_account":{"link":"AccountDuplicationCheck_false"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:14","subtask_inputs":[]}

Here is what's being sent in production (render.com)

https://api.twitter.com/1.1/onboarding/task.json {} {"flow_name":"login","input_flow_data":{"flow_context":{"debug_overrides":{},"start_location":{"location":"splash_screen"}}}}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191939981972814:-1721919400102:I3rJn2VppOBHfAaRmxNHT97d:0","subtask_inputs":[{"subtask_id":"LoginJsInstrumentationSubtask","js_instrumentation":{"response":"{}","link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191939981972814:-1721919400102:I3rJn2VppOBHfAaRmxNHT97d:1","subtask_inputs":[{"subtask_id":"LoginEnterUserIdentifierSSO","settings_list":{"setting_responses":[{"key":"user_identifier","response_data":{"text_data":{"result":"allttt1807501"}}}],"link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} 
{"flow_token":"g;172191939981972814:-1721919400102:I3rJn2VppOBHfAaRmxNHT97d:6","subtask_inputs":[{"subtask_id":"LoginEnterAlternateIdentifierSubtask","enter_text":{"link":"next_link"}}]} // different from when it runs locally

// and the error:
/opt/render/project/src/twitter-scraper/dist/auth-user.js:274
            return { status: 'error', err: new Error(await res.text()) };
                                           ^
Error: {"errors":[{"code":366,"message":"Missing data."}]}
    at TwitterUserAuth.executeFlowTask (/opt/render/project/src/twitter-scraper/dist/auth-user.js:274:44)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async TwitterUserAuth.handleEnterAlternateIdentifierSubtask (/opt/render/project/src/twitter-scraper/dist/auth-user.js:143:16)
    at async TwitterUserAuth.login (/opt/render/project/src/twitter-scraper/dist/auth-user.js:65:24)
    at async Scraper.login (/opt/render/project/src/twitter-scraper/dist/scraper.js:282:9)
    at async startTwitter (file:///opt/render/project/src/twitter.js:35:7)

So the beginning of the login process is the same, but not the end: locally, it does subtask_id LoginEnterPassword, while production subtask_id is LoginEnterAlternateIdentifierSubtask...

It seems like it tries to enter an alternate identifier subtask but it doesn't exist, hence why it says "missing data" (missing alternate identifier/alternate identifier subtask)

@karashiiro would it be possible to provide us with a quick fix for this?

@karashiiro
Copy link
Collaborator

Not sure what that subtask actually represents, it sounds like a 2FA challenge? I've been incredibly busy lately with some service outages in another project I maintain, but if there's any info on the flow floating around I can see what I can do.

@karashiiro
Copy link
Collaborator

Actually, that seems to have been added in this PR? #93

@witherBattler
Copy link

You were right, all I had to do was add the "email" parameter (used a phone number instead, though)
I was afraid it would be required to also obtain the verification code, but for some reason there was no need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants