[5.x]: Applying project config changes will sometimes revert the config folder to an early state

[MoritzLost]

What happened?

Description

I'm still seeing the issue described in #15012. I can't reliably reproduce it, but sometimes, running up or project-config/apply will not apply the config, but will instead reset the config to an earlier state (essentially running project-config/write).

I've encountered this in two scenarios:

1. When switching between branches with different config states in a local dev environment (in dev mode with allowAdminChanges set to true).
2. When deploying to a live server (in production mode with allowAdminChanges set to false).

Not sure if those are the same issue, or two unrelated issues.

I don't know what causes this or how to reproduce it. Just now I saw this while deploying an update from Craft 4 to Craft 5. The environment doesn't allow admin changes, and the deployment script only applies the most recent state from GitHub, here's the relevant part:

# Flush all caches
$FORGE_PHP craft cache/flush-all --interactive=0
$FORGE_PHP craft clear-caches/all --interactive=0

# Update source code from repository
git pull --ff-only origin $FORGE_SITE_BRANCH

# Install Composer dependencies
$FORGE_COMPOSER install --no-dev --no-progress --no-interaction --prefer-dist --optimize-autoloader

# Apply project config and any pending migrations
$FORGE_PHP craft migrate/all --no-content --interactive=0
$FORGE_PHP craft project-config/apply
$FORGE_PHP craft migrate --track=content --interactive=0

Now I can see in the deployment log that the Composer installation ran successfully, and the first set of migrations (before the project-config/apply step) were also executed correctly. But after that, the project-config/apply command resulted in an error:

29 migrations were applied.

Migrated up successfully.
Your project config files were created for different versions of Craft and/or plugins than what’s currently installed.

Craft CMS is installed with schema version of 5.3.0.2 while 4.5.3.0 was expected.
Navigation is installed with schema version of 2.1.0 while 2.0.7 was expected.
Super Table is installed with schema version of 4.0.0 while 3.0.0 was expected.

Try running `composer install` from your terminal to resolve.

Checking the files on the server, I could see that the config/ folder was reset to the state before the deployment started. After running git reset --hard and git clean -df, I restarted the deployment. This time, it finished successfully.

So something caused Craft to write out the existing config instead of applying it. Either as part of the migrate/all --no-content step, or at the beginning of the project-config/apply step.

Steps to reproduce

Unclear.

Expected behavior

• Running project-config/apply should never, in any circumstances, change any files in the config/ folder. If the config can't be applied for any reason, it should throw an error, and possibly advice on how to fix the issue.
• While allowAdminChanges is set to false, Craft should never, ever write anything to the config/ folder.

Actual behavior

In some situations, running up or project-config/apply will have unexpectedly revert the config to the current state in the database, acting like project-config/write.

Craft CMS version

5.4.10

PHP version

8.3

Operating system and version

No response

Database type and version

MySQL 8

Image driver and version

No response

Installed plugins and versions

No response

[brandonkelly]

The next time this happens, can you capture the resulting (“original”) project config that gets output, and compare it with the actual original project config? I’m curious if there are any changes that stick around from the deployed project config changes.

[MoritzLost]

@brandonkelly Thanks for the reply, I'll try to check that next time it happens. Pretty sure there were no changes between the previous config and the resulting config, though. At the moment, we're seeing this issue frequently in dev – we'll switch branches, apply the config, and instead the config is reset to the state of the previous branch. Anything we can check in that state to see where it went wrong, or provide more debug info?

Though some instances of this were in outdated Craft versions, so this might also muddy the waters. I'm trying to get all our current projects updated to 5.5+ to see if it's still an issue there.

[MoritzLost]

@brandonkelly Some more context – I'm noticing that this happens at least every time there's any other error that causes up or project-config/apply to fail. Issues like #16150 or #16145 result in an error during this step. After that, everything looks fine and git doesn't show any changes. But running project-config/apply again after that will reset the config and write out the previous config to the config/ folder.

That's definitely not the only time this happens, but it's definitely a pain point.