After the last release all my articles from Zottero got duplicated

[piojanu]

As in the title. I have set the cronjob for 3 am every day. And on 20 Oct (after the new release on 19 Oct) all the articles got pushed to the Zotero second time. I tried to simply remove them, but they got pushed again. Can we somehow fix this issue?

[e-alizadeh]

@piojanu Sorry for the issue. Have you tried to delete items you first time pushed to Readwise? and not the new ones?

[pcuellar]

I am having exactly the same issue, second week I get duplicates. Is the suggestion to delete originals?

[piojanu]

@e-alizadeh This is not a real solution as I’ve already edited many of the highlights and created mastery cards from them.

[e-alizadeh]

@piojanu @pcuellar I just found out what's going on. A recent bugfix resulted in a change in a parameter (source_url) that Readwise uses to de-duplicate the highlights.

Short-term solution

So, as a short-term solution, I would suggest pinning Zotero2Readwise version (pip install zotero2readwise==0.2.4). If you've forked from Zotero2Readwise-Sync, you need to pin the PyPi version in the GitHub Actions step (💿 Install Zotero2Readwise Python package in .github/workflows/automation.yml file).

Long-term solution

The reason this is happened is the way my library fetches data and pushes them to Readwise. Each time all highlights are sent to Readwise, and Readwise de-duplication algorithm ignores duplicated highlights. So, I opened another issue to work on partial push of highlights, that should resolve this as well.

[pcuellar]

Thanks for the suggestions and for looking into it. Just pinned 0.2.4 (FYI I've always been in 0.2.4). Current PyPi version is 0.2.6, should I pin that in the sync or 0.2.4?

[e-alizadeh]

Thanks for the suggestions and for looking into it. Just pinned 0.2.4 (FYI I've always been in 0.2.4). Current PyPi version is 0.2.6, should I pin that in the sync or 0.2.4?

Please pin it to 0.2.4 for now IF YOU WANT TO KEEP THE ORIGINAL HIGHLIGHTS and not the newly duplicated highlights. In this case, please wait for the partial upload to Readwise issue to be implemented.

Since your highlights got duplicated, if you are OK to keep the new ones and delete the old ones, then you can pin the most recent version (0.2.6).

Sorry for the inconvenience here.