Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New logging forces user to have a home folder #137

Open
DavideWalder opened this issue Jun 14, 2024 · 5 comments
Open

New logging forces user to have a home folder #137

DavideWalder opened this issue Jun 14, 2024 · 5 comments

Comments

@DavideWalder
Copy link

This was not the case with 0.2.5

To reproduce:

Dockerfile:

FROM python:3.12.3-slim

ARG UID=10001
RUN adduser \
    --disabled-password \
    --gecos "" \
    --shell "/sbin/nologin" \
    --no-create-home \
    --uid "${UID}" \
    appuser

RUN pip install kagglehub==0.2.6
USER $UID

ENTRYPOINT [ "python", "-c", "import kagglehub" ]
docker build -t kagglehub . && docker run --rm kagglehub
@neshdev
Copy link
Member

neshdev commented Jun 14, 2024

We added additional logging capabilities to help troubleshoot user issues. This is the line that is most likely causing the issue.

Can you explain the usecase on why there isn't a user profile?

@neshdev
Copy link
Member

neshdev commented Jun 14, 2024

Other parts of the kagglehub expect the user directory to exists such as downloads. We store the files in ~/.cache. This folder will be created regardless. See code here. The location is also configurable.

@neshdev
Copy link
Member

neshdev commented Jun 14, 2024

@DavideWalder - Would allowing a env variable for the log path be a sufficient solution?

@DavideWalder
Copy link
Author

Here is my use case:
I have a containerized application using tensorflow and keras-nlp, the latter has a dependency on kagglehub (I am not using it directly).
The application is a stateless container deployed on k8s, therefore running as non-root and without a home folder (since it is stateless).

There are a few design points that should be considered:

  • I never had any dependency require a home folder (using multiple ML libraries), the solution adopted for kagglehub might not be standard
  • Shared artefacts/state between projects might be a tricky topic, since it might break project independence.
  • Storing artefacts outside of the project's root folder might be problematic. Caches (the downloads you mentioned) are fine since they are provably invariant to which project populated them, and temporary files are not retrievable from projects other than the one that created them (e.g. done by random suffixes)

The only simple solution I see is storing logs in a folder relative to the project's root. Environment variables make the interface to the library more complex and implicit.

I already have workarounds for this problems, so not pressing for me. I just wanted to make the project aware that this is happening

@DavideWalder
Copy link
Author

Another point:
This is happening on import, having side-effects on imports is in general something I would avoid. Solutions could be:

  • create the folder lazily on first usage
  • have an explicit logging abstraction, that the user intializes on demand

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants