Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate http_parser upgrade or replacement #196

Open
nicolasff opened this issue May 10, 2021 · 1 comment
Open

Investigate http_parser upgrade or replacement #196

nicolasff opened this issue May 10, 2021 · 1 comment

Comments

@nicolasff
Copy link
Owner

I noticed that http_parser is no longer maintained and users are advised to migrate to llhttp. The README there mentions significant performance improvements compared to http_parser, and this might be worth looking into.

Simply upgrading http_parser to the latest version is not as simple as dropping in the new .c and .h files: the on_query_string callback no longer exists so both endpoint and query string parameters are now passed in using a single callback. It looks like there might be a parser available in http_parser, there's a mention of it in the old README.

@jessie-murray if you're interested in giving it a try, this might be a good upgrade.

@jessie-murray
Copy link
Collaborator

I integrated llhttp on a branch and ran bench.sh without LRANGE and with REQUESTS set to 1000000 to compare llhttp and the current http-parser. Webdis was configured with stderr logging at level 3:

Command http-parser llhttp Change
PING 97,356.88 96,413.49 -0.97%
SET 96,286.78 97,759.95 +1.53%
GET 96,504.08 97,525.44 +1.06%
INCR 96,779.34 96,428.75 -0.36%
LPUSH 96,331.24 95,862.91 -0.49%

(numbers in requests/second as reported by ab).

Just running the same test several times on the same parser is enough to get results that change by more than the amount listed here. I don't know how to make it more stable, if it's just due to the few other processes running on my laptop or if there is a more fundamental reason for it. I don't know how much variance should be expected is what I'm trying to say. For example, here are 3 runs of just PING with http-parser followed by 3 runs of PING with llhttp, both kept at 10x the REQUESTS to try to keep the numbers stable:

http-parser:

PING: 97653.00 requests/sec.
PING: 98317.27 requests/sec.
PING: 98012.81 requests/sec.

(average: 97,994.36)

llhttp:

PING: 98091.05 requests/sec.
PING: 97292.51 requests/sec.
PING: 96314.97 requests/sec.

(average: 97,232.84)

I didn't find llhttp to show "significant performance improvements" compared to http-parser, it was instead very slightly slower in some cases. There might be a few reasons for this, one being that Webdis may spend most of its time waiting on Redis rather than parsing HTTP requests, so even if the request parsing is made faster this has a very small impact on the overall request latency.

I found a few places in Webdis where request processing seems sub-optimal, so I will try to look at those instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants