Question: Depth search for Files/Directories

[ghost]

Hey, is there any way of realising a depth search for specific files ?

I tried manually to implement a depth search with recursion and Directory#list(). But this is really really slow on a share with many directories, cause I have to share#open() every new recursion step individually, which takes up some time.

Or can I change any of the required parameters for share#open() to be "faster" e.g. only retrieve It's meta data, or enough to call Directory#list() on it ?
I tried reading through the Specification, and found out that there is no actual SMB method for it, but I may be wrong, or missing something.

Hope someone can help me on this.

[pepijnve]

If I understand you correctly you're trying to do something like find . -name <some name>. If that's the case, no there's no way to accelerate that besides doing a full recursive directory listing. Best you can do is parallelise the process rather than doing the scan single threaded. There's no way to parallelise listing of a single directory, but you can list multiple directories in parallel.

[pepijnve]

BTW, be sure to use Directory#iterator instead of Directory#list for this. Directory listing over SMB is done in chunks. list will block until the entire directory has been listed and will hold all entries for the directory in memory. iterator only holds on to one chunk and will fetch the next chunk when needed. This allows you to stream the results which will be much less resource intensive than list.

[ghost]

That good to know. I'll test that tomorrow. Though biggest performance impact in my test case was definitely the open() calls. :/ Also to your first question. I need to retrieve all results. Not stop on first find, that's what makes this thing a little harder on performance side.

…

On Tue 12. Feb 2019 at 5:27 PM, Pepijn Van Eeckhoudt < ***@***.***> wrote: BTW, be sure to use Directory#iterator instead of Directory#list for this. Directory listing over SMB is done in chunks. list will block until the entire directory has been listed and will hold all entries for the directory in memory. iterator only holds on to one chunk and will fetch the next chunk when needed. This allows you to stream the results which will be much less resource intensive than list. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#430 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFMOJnqdsiU6Caf8k9cXMB18OieYGUNEks5vMutqgaJpZM4a2_0i> .

[rokkakasu]

Hi @hierynomus I have created the implementation for Listing all files recursively with single session.
Present implementation lists files under single directory and creates NTLM authentication for each sub-folders to list files.
Kindly check my PR and approve.
#754

Thanks
@rokkakasu