Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable credentials using browser obtained token and cookie. #162

Closed
wants to merge 2 commits into from

Conversation

ghost
Copy link

@ghost ghost commented Jul 20, 2022

The way it works is that you need two tokens from the browser.
Currently, getting them is inconvenient, but let's start with the
basics, and we can improve later.

  1. Get the main token by entering the browser devtools while logged
    into Slack (normally F12). From there, select "Storage", then
    "Local Storage", "https://app.slack.com" and "localConfig_v2". From
    inside this value, fish out the value of the "token" key, which
    starts with "xoxc-". Make sure to get the whole token, and not the
    surrounding quotes. Paste this value in some temporary place.

  2. Now move from "Local Storage" to "Cookies", and again
    "https://app.slack.com". Copy the value of the "d" cookie, which
    starts with "xoxd-". Paste this value after the one from the
    previous step, with exactly one space in between.

  3. Take the two concatenated values, and paste them into the password
    field of the Slack account. The value should look like this:
    "xoxc-12345 xoxd-67890" (but much longer obviously).

About the implementation: This requires using form based submission,
because the JSON based API is broken and does not return the correct
data, even with the same parameters (see issue #123). Therefore the
way the token is passed, even the single token which already worked,
is now changed to form data.

Since slack_api_get is no longer a valid call when form based
submission is used (it requires post), this function has been removed.

Signed-off-by: Kristian Amlie kristian.amlie@northern.tech

@ghost
Copy link
Author

ghost commented Jul 20, 2022

I haven't had much time to test this yet, so use at your own risk.

Also, we need to find a better way of obtaining the tokens to begin with. But one step at a time, the above steps work for me to log in and send messages at least.

@ghost ghost mentioned this pull request Jul 20, 2022
@moozhub
Copy link

moozhub commented Jul 21, 2022

I haven't had much time to test this yet, so use at your own risk.

Also, we need to find a better way of obtaining the tokens to begin with. But one step at a time, the above steps work for me to log in and send messages at least.

It's not working for me unfortunately. If I had to guess, it has something to do with the token encoding. If I try to put in the URL encoded version of the xoxd- cookie, it at least logs me out of my current slack session, otherwise it doesn't appear to be doing anything aside from returning "invalid_auth".

I'm not sure if it makes a difference or not either, but I noticed you're using content dispositions instead of posting directly to the http://slack.com/api/rtm.connect?token=<xoxc-token>&batch_presence_aware=1&presence_sub=true like I am with curl ... maybe that as something to do with it too?

@moozhub
Copy link

moozhub commented Jul 21, 2022

Also, as mentioned in another thread, at least during my testing with curl, the d-s cookie was required, or else it always resulted in "invalid_auth". Unless I'm missing something I didn't see it being passed in the debug window.

@ghost
Copy link
Author

ghost commented Jul 22, 2022

Yes, I was wondering if this would show up. It was working for me, so I thought maybe it would be enough. I wonder what makes the difference...

For the purposes of testing, can you simply try to hardcode the d-s cookie here:

-		g_string_append_printf(request, "Cookie: d=%s\r\n", sa->d_cookie);
+		g_string_append_printf(request, "Cookie: d=%s; d-s=12345\r\n", sa->d_cookie);

If that does not work, can you post the debug log of the failing call? Make sure you get the "api call" and "api response" in particular. And make sure to scrub out your tokens so you don't share them to the public.

Just FYI, it might take me a while to get back to you, since I'll be travelling after today.

I'm not sure if it makes a difference or not either, but I noticed you're using content dispositions instead of posting directly to the http://slack.com/api/rtm.connect?token=<xoxc-token>&batch_presence_aware=1&presence_sub=true like I am with curl ... maybe that as something to do with it too?

I tried, but URL parameters do not work together with the d cookie.

@moozhub
Copy link

moozhub commented Jul 28, 2022

Yes, I was wondering if this would show up. It was working for me, so I thought maybe it would be enough. I wonder what makes the difference...

For the purposes of testing, can you simply try to hardcode the d-s cookie here:

-		g_string_append_printf(request, "Cookie: d=%s\r\n", sa->d_cookie);
+		g_string_append_printf(request, "Cookie: d=%s; d-s=12345\r\n", sa->d_cookie);

If that does not work, can you post the debug log of the failing call? Make sure you get the "api call" and "api response" in particular. And make sure to scrub out your tokens so you don't share them to the public.

Just FYI, it might take me a while to get back to you, since I'll be travelling after today.

I'm not sure if it makes a difference or not either, but I noticed you're using content dispositions instead of posting directly to the http://slack.com/api/rtm.connect?token=<xoxc-token>&batch_presence_aware=1&presence_sub=true like I am with curl ... maybe that as something to do with it too?

I tried, but URL parameters do not work together with the d cookie.

@kacf Adding in the d-s cookie did indeed allow me to get passed the initial authentication issues. However, in my case since I think the workspace that I'm connecting to is a bit large, the connection appears to be closing before the entire API response is received at the client, and is therefore returning a "Invalid JSON response".

Edit:

I've confirmed it's when the plugin is retrieving the data from http://slack.com/api/user.list

Edit2:

So I think it may have had something to do with the debug window being opened, because the API response was bogging down pidgin. I was able to connect successfully and send / receive messages. Good work!

@raul-klg
Copy link

Hi! Just for you to know, I've followed the instructions from the PR post and it looks that it's authenticating. For instance, here is this random excerpt:
(17:49:22) slack: api response: {"ok":true,"members":[{"id":"USLACKBOT","team_id":"PRIVATE","name":"slackbot",...
The problem here is that after the (lengthy) pidgin conversation with slack I have no real feedback in the pidgin GUI, ie: no channels, no users, no conversations..

@ghost
Copy link
Author

ghost commented Aug 25, 2022

@moozhub: Good that it worked for you!

If you are sure the d-s cookie is needed, then I will probably aim at making it optional, since it doesn't seem to be needed for me, and I'd rather keep the number of tokens to a minimum if possible.

@raul-klg: Could you save the whole debug window log during a login session? Remember to scrub all your tokens before posting.

@ghost
Copy link
Author

ghost commented Aug 26, 2022

If you are sure the d-s cookie is needed, then I will probably aim at making it optional, since it doesn't seem to be needed for me, and I'd rather keep the number of tokens to a minimum if possible.

I started working on this, and I have some news. It turns out that d-s is a session cookie. In other words, the browser does not store this cookie, so we should not either. What we should do instead, is to receive it in the first response, store it in memory, and then use it in subsequent requests.

However, again I'm not able to test any of this, so I ask @moozhub: If you revert to my original version without the d-s cookie, and look at the Debug Log after attempting to log in, can you confirm these things:

  1. The first occurrence of "api call" in the log is this:
    api call: https://XXXX.slack.com/api/rtm.connect
    
  2. The first occurrence of "Response headers" after the "api call" line contains 200 OK, and one of the headers shortly after that contains this:
    set-cookie: d-s=1661256789 ...
    

If so I believe we can fetch the cookie from there.

@teddywing
Copy link

@kacf I tested 8a09a6d and can confirm both of your questions:

  1. Yes, the first occurrence of “api call” is to https://XXXX.slack.com/api/rtm.connect.
  2. Yes, the first occurrence of “Response headers” is a 200 and includes the set-cookie header.

Here is a copy of my debug log (slightly redacted), beginning at the Slack account connection:

...
(00:58:38) account: Connecting to account me@example.com%example.slack.com.
(00:58:38) connection: Connecting. gc = 0x6000001bc300
(00:58:38) signals: Signal data for chat-conversation-typing not found!
(00:58:38) slack: api call: https://example.slack.com/api/rtm.connect
POST /api/rtm.connect HTTP/1.0
Host: example.slack.com
Content-Type: multipart/form-data; boundary=---------------------------13173217157072942672

[Headers]

(00:58:38) util: requesting to fetch a URL
(00:58:38) dnsquery: Performing DNS lookup for example.slack.com
(00:58:38) dns: Created new DNS child 9395, there are now 1 children.
(00:58:38) dns: Successfully sent DNS request to child 9395
(00:58:39) dns: Got response for 'example.slack.com'
(00:58:39) dnsquery: IP resolved for example.slack.com
(00:58:39) proxy: Attempting connection to 18.134.215.41
(00:58:39) proxy: Connecting to example.slack.com:443 with no proxy
(00:58:39) proxy: Connection in progress
(00:58:39) proxy: Connecting to example.slack.com:443.
(00:58:39) proxy: Connected to example.slack.com:443.
(00:58:39) gnutls: Starting handshake with example.slack.com
(00:58:39) gnutls: Handshake complete

[gnutls certs]

(00:58:39) certificate/x509/tls_cached: Starting verify for example.slack.com
(00:58:39) certificate/x509/tls_cached: Checking for cached cert...
(00:58:39) certificate/x509/tls_cached: ...Found cached cert
(00:58:39) gnutls: Attempting to load X.509 certificate from $HOME/.purple/certificates/x509/tls_peers/example.slack.com
(00:58:39) certificate/x509/tls_cached: Peer cert matched cached
(00:58:39) util: Writing file $HOME/.purple/certificates/x509/tls_peers/example.slack.com
(00:58:39) certificate: Successfully verified certificate for example.slack.com
(00:58:39) util: request constructed
(00:58:39) util: Response headers: 'HTTP/1.0 200 OK
date: Tue, 30 Aug 2022 22:58:39 GMT
server: Apache
x-powered-by: HHVM/4.153.1
access-control-allow-origin: *
referrer-policy: no-referrer
x-slack-backend: r
x-slack-unique-id: [...]
strict-transport-security: max-age=31536000; includeSubDomains; preload
access-control-allow-headers: slack-route, x-slack-version-ts, x-b3-traceid, x-b3-spanid, x-b3-parentspanid, x-b3-sampled, x-b3-flags
access-control-expose-headers: x-slack-req-id, retry-after
x-oauth-scopes: identify,read,post,client,apps
x-accepted-oauth-scopes: rtm:stream,client
expires: Mon, 26 Jul 1997 05:00:00 GMT
cache-control: private, no-cache, no-store, must-revalidate
pragma: no-cache
x-xss-protection: 0
x-content-type-options: nosniff
x-slack-req-id: [...]
vary: Accept-Encoding
set-cookie: d-s=[...]; path=/; domain=.slack.com; secure; httponly; SameSite=Lax
content-type: application/json; charset=utf-8
x-envoy-upstream-service-time: 133
x-backend: main_normal main_bedrock_normal_with_overflow main_canary_with_overflow main_bedrock_canary_with_overflow main_control_with_overflow main_bedrock_control_with_overflow
x-server: slack-www-hhvm-main-iad-hizd
x-slack-shared-secret-outcome: no-match
via: envoy-www-iad-kl5j, envoy-edge-lhr-2opp
x-edge-backend: envoy-www
x-slack-edge-shared-secret-outcome: no-match
connection: close

'
(00:58:40) slack: api response: {"ok":true,"url":"wss:\/\/ ...

@ghost
Copy link
Author

ghost commented Aug 31, 2022

Thanks @teddywing! But did your login ultimately work? Or did it fail later in the process?

@teddywing
Copy link

My login was successful and the Buddy List window populated with Slack channels.

@ghost
Copy link
Author

ghost commented Aug 31, 2022

Ok. Unfortunately I need the log from someone whose login doesn't work.

@teddywing
Copy link

Ah, I missed that sorry.

@raul-klg
Copy link

raul-klg commented Sep 5, 2022

@raul-klg: Could you save the whole debug window log during a login session? Remember to scrub all your tokens before posting.

@kacf I have sent you an email with the logging. I hope it is helpful.

@ghost
Copy link
Author

ghost commented Sep 7, 2022

Thanks @raul-klg! I've analyzed the log and I have a few questions:

  1. Is it correct that you manually shut down the connection after about 40 seconds or so? I can't see any errors, and the first message about shutdown appears to be user initiated. I just want to make sure I understand the log correctly.
  2. Does your organization by any chance have a huge number of archived channels?
    • From what I can read in the log, all the requests do actually work, but the Slack API is quite stupid in the sense that it forces you to go through every single channel, even archived ones, when doing API pagination. This likely causes it to be extremely slow to connect.
  3. Does it eventually connect if you leave it long enough?

@raul-klg
Copy link

raul-klg commented Sep 7, 2022

Great! Thanks!

1. Is it correct that you manually shut down the connection after about 40 seconds or so? I can't see any errors, and the first message about shutdown appears to be user initiated. I just want to make sure I understand the log correctly.

Yes. I issued Ctrl-Q after that time.

2. Does your organization by any chance have a huge number of archived channels?

That's possible but I don't have access to all organization channels. What I see from the web interface is ~6 connections, 7 channels and ~10 direct messages.

3. Does it eventually connect if you leave it long enough?

I can try this again.

@raul-klg
Copy link

raul-klg commented Sep 7, 2022

I have to say that I got pidgin to sync with slack... after 10 minutes :)
After this I would say the code works. Let me know if I can help some other way. Thanks!

@ghost
Copy link
Author

ghost commented Sep 7, 2022

Alright, that's good! Then I have increased confidence that this is working.

The issue with the slowness is then unrelated to this pull request. One quick way to get some improvement is to raise this to 1000 (the maximum), but I think the effect will still only be moderate at best. Other than that we may have to look at different ways of querying for conversations.

@serj-p
Copy link

serj-p commented Sep 16, 2022

you're genius! i was able to connect to my organization's workspase which uses SSO.

@ghost
Copy link
Author

ghost commented Sep 19, 2022

@dylex: Sounds like this is working for quite a few people. We might want to think about merging it soon.

The procedure for obtaining the token still needs a bit of cleaning up though. I have not really found a more robust way to get it than the instructions written above. I did look briefly into trying to extract them automatically using sqlite on browser cookie databases. This works, but it's hacky, and will only work on Linux. So I'm not really advocating for that.

So then I suppose putting the manual instructions in the README is the best approach. Unless you have a better idea?

@teddywing
Copy link

One note for Bitlbee: I needed to manually set my token/password in a config file due to the way it’s formatted. Since Bitlbee treats spaces as word separators, only the first token was saved as my password with:

account slack set password xoxc-… xoxd-…

I’m not sure if there’s a way to quote the password for Bitlbee, but I ended up manually setting the password by encrypting it:

$ bitlbee -x enc <identify-password> 'xoxc-… xoxd-…'

then pasting the result in /var/lib/bitlbee/<account-name>.xml under //user/account/@password.

@ghost
Copy link
Author

ghost commented Sep 20, 2022

That's a good point, we should probably separate them with a comma instead of a space.

@ghost
Copy link
Author

ghost commented Jan 20, 2023

Rebased branch to get the fix for #174.

@frozencemetery
Copy link

For what it's worth, bitlbee accepted them quoted for me - account 0 set password "xoxc-whatever xoxd-whatever"

@peci1
Copy link

peci1 commented Feb 9, 2023

I've tried the finduserhack branch, but I still get connection errors.

(11:11:04) slack: api call: https://xxx.slack.com/api/rtm.connect?token=xoxc-xxx-xxx-xxx xoxd-xxx%2Bxxx%2Fxxx%2Bxxx%2Fxxx&batch_presence_aware=1&presence_sub=true
...
(11:11:04) nss: Trusting CN=slack.com,O="Slack Technologies, Inc.",L=San Francisco,ST=California,C=US
(11:11:04) certificate: Successfully verified certificate for sunset-frisbee.slack.com
(11:11:04) util: request constructed
(11:11:04) util: Response headers: 'HTTP/1.1 400 Bad Request
content-length: 11
content-type: text/plain
date: Thu, 09 Feb 2023 10:11:04 GMT
server: envoy
via: envoy-edge-fra-vpsc
connection: close

'
(11:11:04) util: parsed 11
(11:11:04) slack: api response: Bad Request
(11:11:04) connection: Connection error on 0x55d5c7f92c20 (reason: 0 description: Invalid JSON response)
(11:11:04) account: Disconnecting account xxx@xxx.cz%xxx.slack.com (0x55d5c5b2bf90)
(11:11:04) connection: Disconnecting connection 0x55d5c7f92c20
(11:11:04) GLib-GObject: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
(11:11:04) connection: Destroying connection 0x55d5c7f92c20

Do you have any idea what's wrong? I've noticed the xoxd token is urlencoded and I also tried with the decoded token, but the result was the same...

@peci1
Copy link

peci1 commented Feb 9, 2023

Hmm, I see the previous error when I choose to save the password in Pidgin. If I don't choose to store it, I get a different error:

(11:23:50) util: request constructed
(11:23:50) util: Response headers: 'HTTP/1.1 200 OK
date: Thu, 09 Feb 2023 10:23:50 GMT
server: Apache
x-powered-by: HHVM/4.153.1
access-control-allow-origin: *
referrer-policy: no-referrer
x-slack-backend: r
x-slack-unique-id: xxx
strict-transport-security: max-age=31536000; includeSubDomains; preload
access-control-allow-headers: slack-route, x-slack-version-ts, x-b3-traceid, x-b3-spanid, x-b3-parentspanid, x-b3-sampled, x-b3-flags
access-control-expose-headers: x-slack-req-id, retry-after
expires: Mon, 26 Jul 1997 05:00:00 GMT
cache-control: private, no-cache, no-store, must-revalidate
pragma: no-cache
x-robots-tag: noindex,nofollow
x-xss-protection: 0
x-content-type-options: nosniff
x-slack-req-id: xxx
vary: Accept-Encoding
content-type: application/json; charset=utf-8
x-envoy-upstream-service-time: 103
x-backend: main_normal main_canary_with_overflow main_control_with_overflow
x-server: slack-www-hhvm-main-iad-vbyz
x-slack-shared-secret-outcome: no-match
via: envoy-www-iad-ihwo, envoy-edge-fra-lhao
x-edge-backend: envoy-www
x-slack-edge-shared-secret-outcome: no-match
connection: close
transfer-encoding: chunked

'
(11:23:51) slack: api response: {"ok":false,"error":"invalid_auth"}

@peci1
Copy link

peci1 commented Mar 2, 2023

Now I have successfully set up a new Slack account using this branch. So the previous errors were probably spurious...

@dchmelik
Copy link

After pressing F12 I didn't see anywhere called storage, so these instructions seem incomplete.

@peci1
Copy link

peci1 commented Mar 30, 2023

Which browser?

@dchmelik
Copy link

I used Chromium/Chrome then Mozilla/Firefox so was able to try but it didn't work in BitlBee... I saw someone mention a newer/better way to do this but they never actually gave the link.

The way it works is that you need two tokens from the browser.
Currently, getting them is inconvenient, but let's start with the
basics, and we can improve later.

1. Get the main token by entering the browser devtools while logged
   into Slack (normally F12). From there, select "Storage", then
   "Local Storage", "https://app.slack.com" and "localConfig_v2". From
   inside this value, fish out the value of the "token" key, which
   starts with "xoxc-". Make sure to get the whole token, and not the
   surrounding quotes. Paste this value in some temporary place.

2. Now move from "Local Storage" to "Cookies", and again
   "https://app.slack.com". Copy the value of the "d" cookie, which
   starts with "xoxd-". Paste this value after the one from the
   previous step, with exactly one space in between.

3. Take the two concatenated values, and paste them into the password
   field of the Slack account. The value should look like this:
   "xoxc-12345 xoxd-67890" (but much longer obviously).

About the implementation: This requires using form based submission,
because the JSON based API is broken and does not return the correct
data, even with the same parameters (see issue #123). Therefore the
way the token is passed, even the single token which already worked,
is now changed to form data.

Since `slack_api_get` is no longer a valid call when form based
submission is used (it requires post), this function has been removed.

Signed-off-by: Kristian Amlie <kristian.amlie@northern.tech>
@ghost
Copy link
Author

ghost commented Sep 19, 2023

Rebased to latest main branch, and added fix for new requirement, d-cookie in WebSocket requests.

@ghost
Copy link
Author

ghost commented Sep 19, 2023

@dylex: I cannot check whether this affects the non-2FA version of slack-libpurple. But if it does, feel free to cherry-pick that patch.

slack-rtm.c Outdated Show resolved Hide resolved
slack-rtm.c Outdated Show resolved Hide resolved
Started being required by Slack around 2023-09-19.

Signed-off-by: Kristian Amlie <kristian.amlie@northern.tech>
@ghost ghost mentioned this pull request Sep 20, 2023
@jleightcap
Copy link

jleightcap commented Oct 3, 2023

Thanks for this work!

I seem to be having an issue with a Slack account; I've follow the instructions for the formatted "${localConfig_v2 token} ${d cookie}" password, verified in ~/.purple/accounts.xml.
This gives the following error:

(00:22:19) slack: RTM 10:                                                                                                                   
(00:22:19) websocket: message 81 len 90                                                                                                     
(00:22:19) slack: RTM 1: {"type":"error","error":{"msg":"invalid_auth","code":401,"source":"gatewayserver-iad-22"}}                         
(00:22:19) slack: Unhandled RTM type error

I added a patch to hard-code a d-s cookie as described. This caused no change.

Let me know if/how to provide futher logs, if it would help! I'm only aware of checking pidgin -d , currently.

@ghost
Copy link
Author

ghost commented Oct 6, 2023

@jleightcap: Which client are you using?

@jleightcap
Copy link

@kacf Pidgin!

@ghost
Copy link
Author

ghost commented Oct 9, 2023

If you check the log in the Pidgin Debug window (you need to keep it open while connection), can you post the log of:

  1. slack: api call: https://<xyz>.slack.com/api/rtm.connect and the associated data below it.
  2. The first util: Response headers which occurs after that.
  3. The first few slack/RTM messages after point 2.

Beware of private tokens in the output before posting.

@jleightcap
Copy link

jleightcap commented Oct 17, 2023 via email

@ghost
Copy link
Author

ghost commented Oct 27, 2023

Thanks, @jleightcap. Honestly I can't say. I've been comparing logs between what you posted and myself, and everything looks identical up to the point where it gives you authentication error.

At this point I think your only bet is to open the browser debugging tools to see what the web client does, and try to replicate it. This might reveal a whole new set of issues though, since Slack has apparently redirected Pidgin to the legacy endpoint for some time, and the web client will probably try to use a newer endpoint. I don't know what the changes in the new protocol are.

@jleightcap
Copy link

I appreciate the debugging effort, @kacf. I only have one Slack account to test with, if I find another I'll report results here. I'm unfortunately not well equipped to debug this further...

@berrange
Copy link

The way it works is that you need two tokens from the browser. Currently, getting them is inconvenient, but let's start with the basics, and we can improve later.

1. Get the main token by entering the browser devtools while logged
   into Slack (normally F12). From there, select "Storage", then
   "Local Storage", "https://app.slack.com" and "localConfig_v2". From
   inside this value, fish out the value of the "token" key, which
   starts with "xoxc-". Make sure to get the whole token, and not the
   surrounding quotes. Paste this value in some temporary place.

2. Now move from "Local Storage" to "Cookies", and again
   "https://app.slack.com". Copy the value of the "d" cookie, which
   starts with "xoxd-". Paste this value after the one from the
   previous step, with exactly one space in between.

3. Take the two concatenated values, and paste them into the password
   field of the Slack account. The value should look like this:
   "xoxc-12345 xoxd-67890" (but much longer obviously).

These steps are easily automated and weechat's slack plugin has a python script todo pretty much that. I've modified that python script a little, so it prints out tokens in the format you specified in bullet point (3), and matches up the team name. Then added some code on top of this pull request, that can automatically invoke the python script. End result, is that the user just needs point to the location of the python script, and all the token stuff magically "just works", assuming you have a valid slack login in your browser. The code is berrange@60b72b2

@kacf feel free to add the above commit in this pull request if you desire, or I can submit this separately as a follow on pull req, as & when this one eventually merges.

@ghost ghost closed this by deleting the head repository Nov 27, 2024
@peci1
Copy link

peci1 commented Nov 27, 2024

Continuation in #190.

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.