Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement OneDrive Business Shared Folders Support (Issue #459) #473

Merged
merged 123 commits into from
Jun 27, 2020

Conversation

abraunegg
Copy link
Owner

  • Initial work on supporting OneDrive Business Shared Folders

* Inital work on supporting OneDrive Business Shared Folders
@abraunegg abraunegg added this to the v2.4.0 milestone Apr 23, 2019
@abraunegg abraunegg changed the title Inital work on supporting OneDrive Business Shared Folders (Issue #459) WIP: Initial work on supporting OneDrive Business Shared Folders (Issue #459) Apr 23, 2019
@abraunegg
Copy link
Owner Author

abraunegg commented Apr 23, 2019

How to configure

  1. List shared folders:
./onedrive --confdir '~/.config/onedrive-preining' --list-shared-folders
Initializing the Synchronization Engine ...

Listing available OneDrive Business Shared Folders:
Shared Folder: SomeRubbishFolder
Shared Folder: AnotherSharedFolder
Shared Folder: TestSharedFolder
  1. Create a new file called 'business_shared_folders' in your config directory
  2. Each new line in 'business_shared_folders' is a 'Shared Folder' from step 1 above that you want to sync
  3. Validate your configuration with ./onedrive --confdir '~/.config/onedrive-preining' --display-config:
onedrive version                    = v2.3.3-1-gc8e47a4
Config path                         = /home/alex/.config/onedrive-preining
Config file found in config path    = true
Config option 'check_nosync'        = false
Config option 'sync_dir'            = /home/alex/OneDrivePreining
Config option 'skip_dir'            = 
Config option 'skip_file'           = ~*|.~*|*.tmp
Config option 'skip_dotfiles'       = false
Config option 'skip_symlinks'       = false
Config option 'monitor_interval'    = 45
Config option 'min_notify_changes'  = 5
Config option 'log_dir'             = /var/log/onedrive/
Selective sync configured           = false
Selective Business Shared Folders configured = true
business_shared_folders contents:
TestSharedFolder
ThisDoesNotExist
SomeRubbishFolder
  1. Perform a sync:
./onedrive --confdir '~/.config/onedrive-preining' --synchronize --verbose --sync-shared-folders
Using Config Dir: /home/alex/.config/onedrive-preining
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/alex/OneDrivePreining
Initializing the Synchronization Engine ...
Account Type: business
Default Drive ID: b!eRoTwf855UO30XBlm3Yz-NYBVmMu24pHvgrM4OUip-8F1LO13vWcT5o0bEQr4zit
Default Root ID: 01WOGXO2N6Y2GOVW7725BZO354PWSELRRZ
Remaining Free Space: 1099329560663
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 01WOGXO2N6Y2GOVW7725BZO354PWSELRRZ
Syncing OneDrive Business Shared Folder: SomeRubbishFolder
Applying changes of Path ID: 01DBFNO5QIQCS5F3EUOVAKDH7TL7ROL6BM
Syncing OneDrive Business Shared Folder: TestSharedFolder
Applying changes of Path ID: 01DBFNO5VLLTCOGVRW6ZBYFBKAXHJI5IGF
Uploading differences of /home/alex/OneDrivePreining
Processing root
The directory has not changed
Processing Cygwin.zip
The file has not changed
Processing local_dir
The directory has not changed
Processing asdf.txt
The file has not changed
Uploading new items of /home/alex/OneDrivePreining
Applying changes of Path ID: 01WOGXO2N6Y2GOVW7725BZO354PWSELRRZ
Syncing OneDrive Business Shared Folder: SomeRubbishFolder
Applying changes of Path ID: 01DBFNO5QIQCS5F3EUOVAKDH7TL7ROL6BM
Syncing OneDrive Business Shared Folder: TestSharedFolder
Applying changes of Path ID: 01DBFNO5VLLTCOGVRW6ZBYFBKAXHJI5IGF

What has been tested

  1. File uploads / downloads / deletes to non shared folders
  2. File uploads / downloads to shared folders
  3. Delete's on local folders for where an object is on a shared folder

What is not working yet
Unsure ... this needs a lot of regression testing

* Handle local deletes where item resides on a different drive id
* Update --single-directory handling for OneDrive Business Shared Folders
* Update debugging output
* Update logging output as to why a 'change' was discarded
* Fix bug where 'item path' contains 'syncFolderChildPath' would always be true as bad isItemRemote test was being used as qualifier
@abraunegg
Copy link
Owner Author

Further testing

  • validated with --sync-shared-folders --single-directory using OneDrive Personal, OneDrive Business & OneDrive Office365 account types

@abraunegg abraunegg requested a review from norbusan April 24, 2019 22:51
@abraunegg
Copy link
Owner Author

@norbusan
This should be good to go for full review / regression testing. This passes all of my sanity checks.

* Add readme on how to configure
* Code cleanup
* Update shared folders readme
* Fix crash when 5xx error is received when getDefaultRoot() is called
* Update Business Shared Folder logic to handle when a local directory already exists but not synced
* Update Business Shared Folder logic to handle namespace conflicts (2 different users sharing the same folder name)
* Update Business Shared Folder logic when handling existing items
* Update readme
* Some shared folders dont detail who it is shared by - not good for conflict resolution
* validate that parent reference id also exists before using
* add logic to look for shared folders where the shared folder is not the root folder being shared with us
* clean up merge
* update last ditch effort to match this onedrive shared folder
@abraunegg abraunegg removed the request for review from norbusan May 7, 2019 09:52
@abraunegg abraunegg removed this from the v2.4.0 milestone May 8, 2019
@abraunegg abraunegg changed the title WIP: Initial work on supporting OneDrive Business Shared Folders (Issue #459) WIP: Initial work on supporting OneDrive Business Shared Folders (Issue #459) (on-hold) May 9, 2019
* Update to handle feature via config file
@abraunegg abraunegg requested a review from norbusan June 19, 2020 06:36
@abraunegg
Copy link
Owner Author

@norbusan
Zero change for OneDrive Personal Shared Folders either:

./onedrive --confdir '~/.config/onedrive-personal/' --synchronize --verbose
Using 'user' Config Dir: /home/alex/.config/onedrive-personal/
Using 'system' Config Dir: 
Configuration file successfully loaded
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Opening the item database ...
All operations will be performed in: /home/alex/OneDrivePersonal
Application version: v2.4.2-120-gcbda90d
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5357147516
Fetching details for OneDrive Root
OneDrive Root exists in the database
Initializing the Synchronization Engine ...
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Number of items from OneDrive to process: 1
Syncing OneDrive Shared Folder: Shared Sub Folder - 3 Deep
Applying changes of Path ID: BC7D88EC1F539DCF!1442
Syncing OneDrive Shared Folder: shared
Applying changes of Path ID: BC7D88EC1F539DCF!1288
Uploading differences of ~/OneDrivePersonal
Processing root
The directory has not changed
Processing random_videos
The directory has not changed
Processing random_images
The directory has not changed
Processing target
The directory has not changed
Processing random_files
The directory has not changed
Processing Pictures
The directory has not changed
Processing 3f44KciNSizldUZUMl6ivqzBclQCgew9
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image3.png
The file has not changed
Processing image2.png
The file has not changed
Processing image4.png
The file has not changed
Processing oHq6r8cMXNOsVlpNaP76Ip7TNjy7gr2Q
The directory has not changed
Processing image0.png
The file has not changed
Processing image2.png
The file has not changed
Processing image3.png
The file has not changed
Processing image4.png
The file has not changed
Processing image1.png
The file has not changed
Processing QcTtuBtAbyHI5BUVyMeUQ5NGfWn0vPnP
The directory has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image4.png
The file has not changed
Processing image0.png
The file has not changed
Processing image3.png
The file has not changed
Processing WFHPuOizV6RvEdba57FFaafeJbEIlYvi
The directory has not changed
Processing image1.png
The file has not changed
Processing image3.png
The file has not changed
Processing image0.png
The file has not changed
Processing image4.png
The file has not changed
Processing image2.png
The file has not changed
Processing yLmlZaKpiZ18h3iWHpJOCX8bhMNOoemq
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image3.png
The file has not changed
Processing image4.png
The file has not changed
Processing 51ijWUoAFJeRaPePV1FpxaqQCDfrEwcC
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image4.png
The file has not changed
Processing image3.png
The file has not changed
Processing EyoZp52OOBUivXV2pkQPnJWxqnLGPcpU
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image4.png
The file has not changed
Processing image3.png
The file has not changed
Processing hQbLdkm5CnWcrvqNtcTgwpR1r9MUvkx3
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image3.png
The file has not changed
Processing image4.png
The file has not changed
Processing WbA29Hri2H7NkKN7Ov9QJuEJQ4Jh1Gvn
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image3.png
The file has not changed
Processing image4.png
The file has not changed
Processing xnh0Yrh3x6pDd3C2CpIvkeYSUkibU711
The directory has not changed
Processing image0.png
The file has not changed
Processing image1.png
The file has not changed
Processing image2.png
The file has not changed
Processing image3.png
The file has not changed
Processing image4.png
The file has not changed
Processing Shared Sub Folder - 3 Deep
The directory has not changed
Processing Shared Sub Folder - 3 Deep
The directory has not changed
Processing Architecture Layres
The directory has not changed
Processing architecture layers v0.4 - AB.vsd
The file has not changed
Processing Cisco VDI Whitepaper
The directory has not changed
Processing white_paper_c11-494994.pdf
The file has not changed
Processing campover.pdf
The file has not changed
Processing SMPP
The directory has not changed
Processing SMPP-IF-SPEC_v3_3-24858.pdf
The file has not changed
Processing SMPP_v3_4_Issue1_2-24857.pdf
The file has not changed
Processing Images
The directory has not changed
Processing JPG
The directory has not changed
Processing IMG_2919.JPG
The file has not changed
Processing IMG_2921.JPG
The file has not changed
Processing IMG_2920.JPG
The file has not changed
Processing IMG_2922.JPG
The file has not changed
Processing IMG_2923.JPG
The file has not changed
Processing IMG_2924.JPG
The file has not changed
Processing IMG_2926.JPG
The file has not changed
Processing IMG_2925.JPG
The file has not changed
Processing PNG
The directory has not changed
Processing CentOS - iSCSI & NFS Mount Example System-2014-10-24-11-48-10.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-08-24-18-09-36.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-08-24-18-12-09.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-07-12.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-08-19.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-20-06.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-32-50.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-31-43.png
The file has not changed
Processing asdf
The directory has not changed
Processing random_files
The directory has not changed
Processing 1JbwHOy4eD3anIugNrbIhcPxhJmjr3ZX
The directory has not changed
Processing file4.data
The file has not changed
Processing file0.data
The file has not changed
Processing file1.data
The file has not changed
Processing file2.data
The file has not changed
Processing file3.data
The file has not changed
Processing 7mU8vo1a8SDpltie8ZLKsZ8ZoyXRnYgY
The directory has not changed
Processing file0.data
The file has not changed
Processing file3.data
The file has not changed
Processing file1.data
The file has not changed
Processing file2.data
The file has not changed
Processing file4.data
The file has not changed
Processing 8oMG8PdeRimidWGx8Q5ZFMBw7lSDzGGf
The directory has not changed
Processing file0.data
The file has not changed
Processing file3.data
The file has not changed
Processing file1.data
The file has not changed
Processing file2.data
The file has not changed
Processing file4.data
The file has not changed
Processing jalhLqB40ECdX1QcGhadRAZ9YozHxZ9r
The directory has not changed
Processing file1.data
The file has not changed
Processing file4.data
The file has not changed
Processing file0.data
The file has not changed
Processing file2.data
The file has not changed
Processing file3.data
The file has not changed
Processing SUsRJc8mCE1HhpEtNTAkGhPowDzjjZBH
The directory has not changed
Processing file3.data
The file has not changed
Processing file4.data
The file has not changed
Processing file0.data
The file has not changed
Processing file1.data
The file has not changed
Processing file2.data
The file has not changed
Processing shared
The directory has not changed
Processing shared
The directory has not changed
Processing Cisco VDI Whitepaper
The directory has not changed
Processing white_paper_c11-494994.pdf
The file has not changed
Processing campover.pdf
The file has not changed
Processing 10-Files
The directory has not changed
Processing Images
The directory has not changed
Processing JPG
The directory has not changed
Processing IMG_2919.JPG
The file has not changed
Processing IMG_2920.JPG
The file has not changed
Processing IMG_2921.JPG
The file has not changed
Processing IMG_2922.JPG
The file has not changed
Processing IMG_2923.JPG
The file has not changed
Processing IMG_2926.JPG
The file has not changed
Processing IMG_2924.JPG
The file has not changed
Processing IMG_2925.JPG
The file has not changed
Processing PNG
The directory has not changed
Processing MyNAS Test Install for Documentation-2014-08-24-18-09-36.png
The file has not changed
Processing CentOS - iSCSI & NFS Mount Example System-2014-10-24-11-48-10.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-07-12.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-08-24-18-12-09.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-08-19.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-20-06.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-31-43.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-32-50.png
The file has not changed
Processing Cisco VDI Whitepaper
The directory has not changed
Processing white_paper_c11-494994.pdf
The file has not changed
Processing campover.pdf
The file has not changed
Processing SMPP
The directory has not changed
Processing SMPP-IF-SPEC_v3_3-24858.pdf
The file has not changed
Processing SMPP_v3_4_Issue1_2-24857.pdf
The file has not changed
Processing Architecture Layres
The directory has not changed
Processing architecture layers v0.4 - AB.vsd
The file has not changed
Processing Exchange_Visio_Stencil_and_Template
The directory has not changed
Processing MicrosoftExchangeServer2007_Icons_README.txt
The file has not changed
Processing MicrosoftExchangeServer2007_Icons.vss
The file has not changed
Processing MicrosoftExchangeServer2007_Icons.vst
The file has not changed
Processing Architecture Layres
The directory has not changed
Processing architecture layers v0.4 - AB.vsd
The file has not changed
Processing Images
The directory has not changed
Processing PNG
The directory has not changed
Processing MyNAS Test Install for Documentation-2014-08-24-18-09-36.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-08-24-18-12-09.png
The file has not changed
Processing CentOS - iSCSI & NFS Mount Example System-2014-10-24-11-48-10.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-07-12.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-08-19.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-20-06.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-31-43.png
The file has not changed
Processing MyNAS Test Install for Documentation-2014-09-11-15-32-50.png
The file has not changed
Processing JPG
The directory has not changed
Processing IMG_2923.JPG
The file has not changed
Processing IMG_2922.JPG
The file has not changed
Processing IMG_2926.JPG
The file has not changed
Processing IMG_2924.JPG
The file has not changed
Processing IMG_2925.JPG
The file has not changed
Processing IMG_2919.JPG
The file has not changed
Processing IMG_2920.JPG
The file has not changed
Processing IMG_2921.JPG
The file has not changed
Processing SMPP
The directory has not changed
Processing SMPP-IF-SPEC_v3_3-24858.pdf
The file has not changed
Processing SMPP_v3_4_Issue1_2-24857.pdf
The file has not changed
Uploading new items of ~/OneDrivePersonal
Applying changes of Path ID: 66D53BE8A5056ECA!101
Syncing OneDrive Shared Folder: Shared Sub Folder - 3 Deep
Applying changes of Path ID: BC7D88EC1F539DCF!1442
Syncing OneDrive Shared Folder: shared
Applying changes of Path ID: BC7D88EC1F539DCF!1288

* Update documentation
src/itemdb.d Show resolved Hide resolved
src/main.d Outdated Show resolved Hide resolved
src/main.d Outdated Show resolved Hide resolved
src/selective.d Outdated Show resolved Hide resolved
* If sync_list contains a comment line, exclude this from sync_list processing
* If business_shared_folders contains a comment line, exclude this from business_shared_folders processing
src/sync.d Outdated Show resolved Hide resolved
}
}
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The very same code appears in line 3481, and maybe other places, can we get rid of these duplications? (Not necessary for this PR, but could be a separate ;-) )

norbusan
norbusan previously approved these changes Jun 20, 2020
Copy link
Collaborator

@norbusan norbusan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the comments I made, I haven't found anything suspicious by reading the code for now.

If you are content with your real-life testing please go ahead, otherwise send be credentials for a business account and a I will test, too.

* Update from 'master' and resolve conflicts
* Only scan the database using the unique list of drive ID's to avoid multiple DB queries if drive ID is present multiple times
* Update Makefile based on change in filename
* Merge 'master' and resolve conflicts
* Update functions to match codin style
* Update how entries are added to driveIDsArray to ensure they are unique
* Update logging output to reflect the correct path based on what is actually shared, not the full underlying parent path
* code simplification
* PR logging tweaks
* Add known issue details of an API issue which prevents shared folders from outside an organisation being configured to sync, as the API does not provide them in the API response.
* Update documentation
* Add response error handling for graphQuery
@abraunegg abraunegg merged commit 9cc72c2 into master Jun 27, 2020
@abraunegg abraunegg deleted the onedrive-business-shared_folders branch June 27, 2020 09:10
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators May 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sync folders shared by others in onedrive business
2 participants