Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在遇到瑞数的情况,stealth无法起作用,将一直卡住 #28

Open
cdsjsz opened this issue May 19, 2023 · 14 comments
Open

在遇到瑞数的情况,stealth无法起作用,将一直卡住 #28

cdsjsz opened this issue May 19, 2023 · 14 comments
Labels
help wanted Extra attention is needed question Further information is requested

Comments

@cdsjsz
Copy link

cdsjsz commented May 19, 2023

复现代码如下

package main

import (
	"crypto/md5"
	"fmt"
	"github.com/go-rod/rod/lib/launcher"
	"time"

	"github.com/go-rod/rod"
	"github.com/go-rod/stealth"
)

func main() {
	path, _ := launcher.LookPath()
	url := launcher.New().Bin(path).Headless(false).MustLaunch()
	browser := rod.New().
		ControlURL(url).Timeout(time.Minute).MustConnect()
	defer browser.MustClose()

	// You can also use stealth.JS directly without rod
	fmt.Printf("js: %x\n\n", md5.Sum([]byte(stealth.JS)))

	page := stealth.MustPage(browser)

	page.MustNavigate("http://lib.cqvip.com/")
	time.Sleep(10 * time.Minute)
	
}

@cdsjsz cdsjsz changed the title 在遇到瑞数的情况,stealth无法起作用 在遇到瑞数的情况,stealth无法起作用,将一直卡住 May 19, 2023
@cdsjsz
Copy link
Author

cdsjsz commented May 19, 2023

newusermode也同样无法绕过检测

package main

import (
	"crypto/md5"
	"fmt"
	"github.com/go-rod/rod/lib/launcher"
	"time"

	"github.com/go-rod/rod"
	"github.com/go-rod/stealth"
)

func main() {
	path, _ := launcher.LookPath()
	url := launcher.NewUserMode().Bin(path).Headless(false).MustLaunch()
	browser := rod.New().
		ControlURL(url).Timeout(time.Minute).MustConnect()
	defer browser.MustClose()

	// You can also use stealth.JS directly without rod
	fmt.Printf("js: %x\n\n", md5.Sum([]byte(stealth.JS)))

	//page := stealth.MustPage(browser)
	page := browser.MustPage()

	page.MustNavigate("https://lib.cqvip.com/")
	time.Sleep(10 * time.Minute)

}

@ysmood
Copy link
Member

ysmood commented May 21, 2023

你不需要 stealth

go-rod/rod#744

package main

import (
	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	u := launcher.NewUserMode().
		Leakless(true).
		UserDataDir("tmp/t").
		Set("disable-default-apps").
		Set("no-first-run").
		MustLaunch()

	rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://lib.cqvip.com/")

	utils.Pause()
}

@cdsjsz
Copy link
Author

cdsjsz commented May 22, 2023

感谢您的回复,在有头模式下没有问题,但在无头模式下还是会卡住,请问如果想要在无头模式下不卡白屏,应该如何操作呢,谢谢

@ysmood
Copy link
Member

ysmood commented May 23, 2023

可以利用 docker 加 XVFB 来实现有头

https://github.com/go-rod/rod/blob/927baba50a3086dcc82906b585d98b182318a677/lib/examples/launch-managed/main.go#L28

@cdsjsz
Copy link
Author

cdsjsz commented May 24, 2023

可以利用 docker 加 XVFB 来实现有头

https://github.com/go-rod/rod/blob/927baba50a3086dcc82906b585d98b182318a677/lib/examples/launch-managed/main.go#L28

感谢您的帮助,在docker模式下也会卡白屏,复现代码如下

package main

import (
	"fmt"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
	"github.com/go-rod/rod/lib/utils"
)

func main() {

	l := launcher.MustNewManaged("")
	// Launch with headful mode
	l.Headless(false).XVFB("--server-num=5", "--server-args=-screen 0 1600x900x16").Set("disable-default-apps").
		Set("no-first-run").Set("disable-gpu")

	browser := rod.New().Client(l.MustClient()).MustConnect()

	// You may want to start a server to watch the screenshots of the remote browser.
	launcher.Open(browser.ServeMonitor(""))
	page := browser.MustPage("https://lib.cqvip.com/")
	fmt.Println(
		page.HTML(),
	)

	utils.Pause()
}

可能是由于不是newusermode模式下启动浏览器导致的?在docker模式下,我们能让他以newusermode启动么,谢谢

@cdsjsz
Copy link
Author

cdsjsz commented May 24, 2023

如果仅将manager的改为newusermode,则返回websocket bad handshake: 502 Bad Gateway.
图片

@ysmood
Copy link
Member

ysmood commented May 24, 2023

l.Flags = launcher.NewUserMode().Flags

@cdsjsz
Copy link
Author

cdsjsz commented May 24, 2023

l.Flags = launcher.NewUserMode().Flags

感谢您的回复,加入该参数后会发生panic,显示:websocket bad handshake: 502 Bad Gateway. 复现代码如下

package main

import (
	"fmt"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	l := launcher.MustNewManaged("wss://example.com")
	l.Headless(false).XVFB("--server-num=5", "--server-args=-screen 0 1600x900x16").Set("disable-default-apps").
		Set("no-first-run").Set("disable-gpu")
	l.Flags = launcher.NewUserMode().Flags

	browser := rod.New().Client(l.MustClient()).MustConnect()

	// You may want to start a server to watch the screenshots of the remote browser.
	launcher.Open(browser.ServeMonitor(""))
	page := browser.MustPage("https://lib.cqvip.com/")
	fmt.Println(
		page.HTML(),
	)

	utils.Pause()
}

@ysmood ysmood added help wanted Extra attention is needed question Further information is requested labels May 24, 2023
@ysmood
Copy link
Member

ysmood commented May 24, 2023

你可以自己慢慢调试 flag,不过大概率是不行的。建议你提交 ticket 给 chromium 团队来处理。这已经超出 rod 应该处理的范围了。

@cdsjsz
Copy link
Author

cdsjsz commented May 24, 2023

你可以自己慢慢调试 flag,不过大概率是不行的。建议你提交 ticket 给 chromium 团队来处理。这已经超出 rod 应该处理的范围了。

请问一下这是chromium哪方面出了问题啊?看这个报错怎么像是ssl的问题

@ysmood
Copy link
Member

ysmood commented May 24, 2023

NewUserMode 部分的评论可以都忽略,这本来就不是 rod 的合理用法,只是可能能成功的 hack,不应该寄希望于此。
我意思是这个应该 work,但是白屏了,这是 chromium 的 bug:#28 (comment)

你就算不用 rod,也会白屏的,这显然就不是 rod 的问题了。

@cdsjsz
Copy link
Author

cdsjsz commented May 24, 2023

感谢您的解答,主要问题是该网站在正常浏览器下没有问题,NewUserMode有头模式下也没有问题,在New下无论有头无头都卡白屏,能否请教一下NewUserMode和New的关键差异在哪么?我尝试看了下相关的代码,主要是flag的不同?

@ysmood
Copy link
Member

ysmood commented May 24, 2023

是的,我前面的回答也提到了

你可以自己慢慢调试 flag

@cdsjsz
Copy link
Author

cdsjsz commented May 25, 2023

好的,感谢您的帮助

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants