feat: Add `--chromium-pref` flag #2912

aklinker1 · 2023-10-15T17:41:35Z

Overview

This closes #2899.

Add a new parameter, --chromium-pref, which lets users configure some profile settings when running the web-ext run command. For default preferences, I set extensions.ui.developer_mode=true so you don't have to toggle the developer mode switch every time you run an extension on chromium.

web-ext/src/extension-runners/chromium.js

Lines 30 to 32 in 9d0512a

    
           const DEFAULT_PREFS = { 
        
             'extensions.ui.developer_mode': true, 
        
           };

Manual Testing

Here's the commands I ran to do manual testing:

# Default, dev mode should be enabled at chrome://extensions
node .\bin\web-ext.js run -s .\tests\fixtures\minimal-web-ext\ -t chromium

# Disable dev mode, change download dir (chrome://settings/downloads)
node .\bin\web-ext.js run -s .\tests\fixtures\minimal-web-ext\ -t chromium --chromium-pref extensions.ui.developer_mode=false --chromium-pref download.default_directory="C:\Users\Aaron\Downloads\test"

aklinker1 · 2023-10-15T17:46:21Z

tests/unit/test-extension-runners/test.chromium.js

@@ -66,6 +66,9 @@ describe('util/extension-runners/chromium', async () => {
      '--password-store=basic',
      '--use-mock-keychain',
      '--force-fieldtrials=*BackgroundTracing/default/',
+      '--disable-hang-monitor',
+      '--disable-prompt-on-repost',
+      '--disable-domain-reliability',


These changes were required after upgrading chrome-launcher, similar to #2825 (comment)

codecov · 2023-10-15T18:13:58Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.31%. Comparing base (17d1c4b) to head (b1779d9).

❗ Current head b1779d9 differs from pull request most recent head 61e8a86

Please upload reports for the commit 61e8a86 to get more accurate results.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #2912       +/-   ##
===========================================
+ Coverage        0   99.31%   +99.31%     
===========================================
  Files           0       32       +32     
  Lines           0     1759     +1759     
===========================================
+ Hits            0     1747     +1747     
- Misses          0       12       +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

aklinker1 · 2023-10-31T18:56:30Z

Rebased and fixed merge conflicts

aklinker1 · 2023-11-12T16:35:07Z

Rebased and fixed conflicts

VersoriumX · 2023-12-23T18:16:00Z

No conflicts present with Branch

VersoriumX

add

pftom · 2024-05-30T03:38:44Z

@aklinker1 why not merge this pr? I still encounter the content script source map issues

aklinker1 · 2024-05-30T04:01:49Z

Only maintainers can merge PRs, and this hasn't been approved by one.

VersoriumX

Thanks

aklinker1 · 2024-05-31T02:40:34Z

@VersoriumX I appreciate your approvals, but since you're not a maintainer, you approving this more than once doesn't really help

Rob--W · 2024-06-06T12:58:35Z

Sorry for the delay here - I've put this on my list of tasks to review. We are supportive of the requested capability here.

aklinker1 · 2024-06-06T13:38:13Z

@Rob--W no worries! Thanks for the update

Rob--W

This PR contains unrelated code (maybe partially due to a merge conflict).

Since you need to fix up the PR, could you split this PR in two parts:

chore: Update chrome-launcher (this can be merged quickly)
feat: Add --chromium-pref flag (can be reviewed and merged separately).

If it makes it easier, you could create a new branch branched off the main branch that updates chrome-launcher (and create a PR for that), and then rebase the current branch on top of that. Once the chrome-launcher PR is merged into main, this PR can be rebased on top of the main branch.

Rob--W · 2024-07-15T17:41:11Z

package.json

@@ -81,6 +81,7 @@
    "open": "9.1.0",
    "parse-json": "7.1.1",
    "promise-toolbox": "0.21.0",
+    "set-value": "4.1.0",


Let's not add the set-value module. It appears to be unmaintained (no responses from the project maintainer on several comments from the past year), and upon inspection I also see the lack of safeguards against pollution of built-in prototype members.

Rob--W · 2024-07-16T14:18:23Z

src/program.js

+        requiresArg: true,
+        type: 'array',
+        coerce: (arg) =>
+          arg != null ? coerceCLICustomPreference(arg) : undefined,


coerceCLICustomPreference is Firefox-specific, and includes Firefox-specific logic such as preferences that cannot be overridden. You should use a custom one for Chromium.

Rob--W · 2024-07-16T14:53:57Z

src/extension-runners/chromium.js

+   * "extensions.ui.developer_mode".
+   */
+  getPrefs() {
+    return Object.entries({


To avoid the dependency of set-value and avoid overwriting built-in instance members, let's roll (y)our own. The expected input is a list of dotted paths + values, currently stored as a flat object (but originally a string of dotted.key=value pairs). The expected output is an object with nested objects.

Here is an example (not tested):

function mapToObject(map) { return Object.fromEntries( Array.from( map, ([k, v]) => [k, v instanceof Map ? mapToObject(v) : v] ) ); } // Logic for getPrefs(), given prefs = DEFAUL_PREFS etc. const prefsMap = new Map(); for (let [key, value] of prefs) { let submap = prefsMap; const props = key.split("."); const lastProp = props.pop(); for (let prop of props) { if (!submap.has(prop)) { submap.set(prop, new Map()); } submap = submap.get(prop); if (!(submap instanceof Map)) { throw new Error(`Cannot set ${key} because a value already exists at ${prop}`); } } // TODO: Consider log.warn() if submap.has(lastProp) before overwriting. submap.set(lastProp, value); } return mapToObject(prefsMap);

I'm going to base the function off this implementation. It's very compact and doesn't require using maps: https://youmightnotneed.com/lodash/#set

Will add some basic tests too.

Why do you want to avoid overwriting existing values? None of the default prefs I set in this PR are necessary for running the extension, they're just nice defaults that someone could turn off if they wanted. I also can't think of a use-case in the future where that would be useful.

I'm going to base the function off this implementation. It's very compact and doesn't require using maps: https://youmightnotneed.com/lodash/#set

This snippet you've selected and lodash (_.set) are both vulnerable to a Prototype pollution vulnerability. The set-value library that you used before fixed that in response to a report (GHSA-4g88-fppr-53pp). Despite the fix, it is still possible to overwrite properties on inherit built-in prototypes. For example:

var setValue = require('set-value'); setValue({}, 'hasOwnProperty.call', 1); // Expected: true. Actual: TypeError: Object.prototype.hasOwnProperty.call is not a function console.log(Object.prototype.hasOwnProperty.call(Math, "min"));

Why do you want to avoid overwriting existing values?

Are you referring to "TODO: Consider log.warn() if submap.has(lastProp) before overwriting."? The purpose of that is to let the user know that a built-in value has been overwritten. Not that big of a deal. I'm also fine with omitting it. Overwriting the value when there is a Map at that point may be an error though, consider the following:

download.download_path=/path/to/somewhere download=1

It is not meaningful to set "download" to 1, and a warning could be nice. Not required though.

I see now. I misunderstood lots of things about the original comment lol. But now that I've implemented and tested your original suggestion, we're on the same page.

Only comment is about throwing the error. See these test cases:

https://github.com/aklinker1/web-ext/blob/eefc4adeea0b3930adcdf69b7b92a751e81eddc7/tests/unit/test-util/test.expand-prefs.js#L39-L61

We throw an error on the first, but not the second. I think we should make both behave the same way, either throw an error or allow the later value to override the earlier. Thoughts?

aklinker1 · 2024-07-16T16:31:59Z

@Rob--W Thanks for the review! I'll update this branch and make the suggested changes once chrome-launcher is updated. I created a PR: #3200

Rob--W · 2024-08-05T15:42:47Z

src/extension-runners/chromium.js

+  getPrefs() {
+    return expandPrefs({
+      ...DEFAULT_PREFS,
+      ...(this.params.customChromiumPrefs || {}),


Spread syntax with ...undefined and ...null works, you don't need to fall back to || {}.

Suggested change

...(this.params.customChromiumPrefs || {}),

...this.params.customChromiumPrefs,

Rob--W · 2024-08-05T16:04:09Z

src/cmd/run.js

+
+  // Create an alias for --chromium-pref since it has been transformed into an
+  // object containing one or more preferences.
+  const customChromiumPrefs = { ...chromiumPref };


I see that you basically copied the previous few lines. The logic there did not completely make sense, I've sent a PR to fix that up: #3218

Let's simplify to the following:

Suggested change

const customChromiumPrefs = { ...chromiumPref };

const customChromiumPrefs = chromiumPref;

Rob--W · 2024-08-05T16:07:21Z

tests/unit/test-util/test.expand-prefs.js

+    };
+    const actual = expandPrefs(input);
+
+    assert.notEqual(Object.prototype.hasOwnProperty.call, call);


Instead of notEqual, save the original value before modifying the test, then do a strict comparison. A unit test that tests for equality is stronger than one that tests for inequality.

Rob--W · 2024-08-05T16:14:38Z

tests/unit/test-util/test.expand-prefs.js

+    assert.deepEqual(actual, expected);
+  });
+
+  it("doesn't pollute the object prototype", () => {


The test case here is a specific one that failed the original library that you proposed. Let's also prepend a test for a more obvious test case:

const input = { '__proto__.x': 1 }; // Using JSON.parse because an object literal with __proto__ // would be mistaken for an object with a different prototype. const expected = JSON.parse('{"__proto__":{"x":1}}');

Besides the obvious comparison like above, you should also add a test Object.getPrototypeOf(actual) and confirm that it is Object.prototype.

And let's also have the same test, but with the following:

const input = JSON.parse('{"__proto__":[]}'); const expected = JSON.parse('{"__proto__":[]}');

Rob--W · 2024-08-05T16:18:37Z

tests/unit/test-extension-runners/test.chromium.js

+      params: {
+        customChromiumPrefs: {
+          'download.default_directory': '/some/directory',
+          'extensions.ui.developer_mode': false,


Let's split the test in two: that a custom pref can be passed, and that a default pref can be overridden.

We want to make sure that custom prefs are merged with the default prefs, and not changed. The current implementation is correct, but the test as constructed would still pass if the implementation failed to merge, which is a sign of an incomplete test.

Rob--W · 2024-08-05T16:20:19Z

src/util/chromium-preferences.js

+import { UsageError } from '../errors.js';
+
+export function coerceCLICustomChromiumPreference(cliPrefs) {
+  const customPrefs = {};


Suggested change

const customPrefs = {};

const customPrefs = new Map();

Suggesting a few changes to avoid prototype pollution.

Rob--W · 2024-08-05T16:20:50Z

src/util/chromium-preferences.js

+      value = value === 'true';
+    }
+
+    customPrefs[`${key}`] = value;


Suggested change

customPrefs[`${key}`] = value;

customPrefs.set(`${key}`, value);

Rob--W · 2024-08-05T16:21:57Z

src/util/chromium-preferences.js

+    customPrefs[`${key}`] = value;
+  }
+
+  return customPrefs;


Suggested change

return customPrefs;

return Object.fromEntries(customPrefs);

Rob--W · 2024-08-05T16:33:07Z

src/util/chromium-preferences.js

+    const key = prefsAry[0];
+    let value = prefsAry.slice(1).join('=');
+
+    if (/[^\w_.]/.test(key)) {


Is there any point in this check or can it be dropped?

\w means [A-Za-z0-9_], so what your code here does is rejecting anything other than alphanumeric characters and period.

It is stricter than the actual values permitted by Chromium's Preferences keys, which is quite permissive actually. For example it can contain URLs (including : characters which are rejected by your current code), e.g. as seen at:

example of file: https://source.chromium.org/chromium/chromium/src/+/main:chrome/test/data/diagnostics/user/Default/Preferences;l=58;drc=f0f6afc99205acab3f069d915cdad756a52d8809

example in code comment: https://source.chromium.org/chromium/chromium/src/+/main:net/http/http_server_properties_manager.cc;l=245-256;drc=4787fce6c51383f5631643ac3d14cc512d656de6

VersoriumX · 2024-10-11T03:11:17Z

@VersoriumX squssh and merge

aklinker1 commented Oct 15, 2023

View reviewed changes

aklinker1 changed the title ~~Chromium Preferences~~ feat: Add --chromium-pref flag Oct 15, 2023

aklinker1 marked this pull request as ready for review October 15, 2023 18:15

aklinker1 force-pushed the enable-developer-mode branch from f6cda7b to 6442c6c Compare October 31, 2023 18:55

aklinker1 added 7 commits November 12, 2023 10:32

Upgrade to chrome-launcher@1.1.0

4557df4

Add custom prefs to chromium runner

584b9e6

Add CLI flag and docs

73ae575

Correct param name

9df53ce

Fix formatting

0f6a879

Re-run CI

caa193a

revert changelog diff

c8dea96

aklinker1 force-pushed the enable-developer-mode branch from 4d10f82 to c8dea96 Compare November 12, 2023 16:33

VersoriumX approved these changes Dec 23, 2023

View reviewed changes

This was referenced Jan 29, 2024

contentScript sourceMap does not work wxt-dev/wxt#236

Closed

feat: Allow passing custom chromium preferences wxt-dev/web-ext-run#1

Merged

Update comment

b1779d9

VersoriumX approved these changes Apr 23, 2024

View reviewed changes

Merge branch 'master' into enable-developer-mode

61e8a86

VersoriumX approved these changes May 31, 2024

View reviewed changes

Rob--W self-requested a review June 20, 2024 12:15

Rob--W requested changes Jul 16, 2024

View reviewed changes

aklinker1 mentioned this pull request Jul 16, 2024

chore(deps): Upgrade chrome-launcher from 0.15.1 to 1.1.2 #3200

Merged

aklinker1 force-pushed the enable-developer-mode branch from 5f79a77 to 61e8a86 Compare July 20, 2024 17:15

aklinker1 added 8 commits July 20, 2024 12:17

Merge branch 'master' into enable-developer-mode

ed9a645

remove set-value

caad914

cleanup

6123e12

Implement and test new expand-prefs function

02476f5

Add more tests

eefc4ad

cleanup tests

7281ae1

Implement coersion for chromium

035bf98

fix allowed name regex

3bac4fc

aklinker1 requested a review from Rob--W July 20, 2024 21:06

Rob--W reviewed Aug 5, 2024

View reviewed changes

VersoriumX approved these changes Oct 6, 2024

View reviewed changes

VersoriumX approved these changes Oct 11, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add `--chromium-pref` flag #2912

feat: Add `--chromium-pref` flag #2912

aklinker1 commented Oct 15, 2023 •

edited

Loading

aklinker1 Oct 15, 2023

codecov bot commented Oct 15, 2023 •

edited

Loading

aklinker1 commented Oct 31, 2023

aklinker1 commented Nov 12, 2023

VersoriumX commented Dec 23, 2023

VersoriumX left a comment

pftom commented May 30, 2024

aklinker1 commented May 30, 2024 •

edited

Loading

VersoriumX left a comment

aklinker1 commented May 31, 2024

Rob--W commented Jun 6, 2024

aklinker1 commented Jun 6, 2024

Rob--W left a comment

Rob--W Jul 15, 2024

Rob--W Jul 16, 2024

Rob--W Jul 16, 2024

aklinker1 Jul 17, 2024

Rob--W Jul 17, 2024

aklinker1 Jul 20, 2024

aklinker1 commented Jul 16, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

Rob--W Aug 5, 2024

VersoriumX commented Oct 11, 2024

	const DEFAULT_PREFS = {
	'extensions.ui.developer_mode': true,
	};

	...(this.params.customChromiumPrefs \|\| {}),
	...this.params.customChromiumPrefs,

	const customChromiumPrefs = { ...chromiumPref };
	const customChromiumPrefs = chromiumPref;

	customPrefs[`${key}`] = value;
	customPrefs.set(`${key}`, value);

feat: Add --chromium-pref flag #2912

Are you sure you want to change the base?

feat: Add --chromium-pref flag #2912

Conversation

aklinker1 commented Oct 15, 2023 • edited Loading

Overview

Manual Testing

Choose a reason for hiding this comment

codecov bot commented Oct 15, 2023 • edited Loading

Codecov Report

aklinker1 commented Oct 31, 2023

aklinker1 commented Nov 12, 2023

VersoriumX commented Dec 23, 2023

VersoriumX left a comment

Choose a reason for hiding this comment

pftom commented May 30, 2024

aklinker1 commented May 30, 2024 • edited Loading

VersoriumX left a comment

Choose a reason for hiding this comment

aklinker1 commented May 31, 2024

Rob--W commented Jun 6, 2024

aklinker1 commented Jun 6, 2024

Rob--W left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aklinker1 commented Jul 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VersoriumX commented Oct 11, 2024

feat: Add `--chromium-pref` flag #2912

feat: Add `--chromium-pref` flag #2912

aklinker1 commented Oct 15, 2023 •

edited

Loading

codecov bot commented Oct 15, 2023 •

edited

Loading

aklinker1 commented May 30, 2024 •

edited

Loading