📝 Add storage FAQ draft (#43)

* 📝 Add storage FAQ draft * 🍱 Actually add it * 💚 Fix * 💚 Fix * 💚 Fix * 💚 Fix
laminlabs · Aug 7, 2023 · 1e52912 · 1e52912
1 parent 8135e72
commit 1e52912
Show file tree

Hide file tree

Showing 4 changed files with 41 additions and 2 deletions.
diff --git a/.gitignore b/.gitignore
@@ -20,6 +20,8 @@ lamin_examples_docs/
 lamindb_docs.zip
 pytorch_lamin_mnist_docs.zip
 redun_lamin_fasta_docs.zip
+lamin_usecases_docs*
+docs*
 
 # macOS
 .DS_Store

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -24,7 +24,7 @@ repos:
         language_version: python3
         args:
           - --max-line-length=88
-          - --ignore=E203
+          - --ignore=E203,W503
         exclude: |
           (?x)(
               __init__.py

diff --git a/docs/faq/storage.md b/docs/faq/storage.md
@@ -0,0 +1,19 @@
+# Storage FAQ
+
+1. What is default storage? How do I find out about default storage?
+
+2. Where is my SQLite file? What happens if I move the `.lndb` file around?
+
+3. What is the `.lamindb/` folder? Will there be multiple `.lamindb/` folders if I have multiple storage locations?
+
+4. What happens if I move files around? What should I do if I want to bulk migrate files to another storage (let’s say another s3 bucket)?
+
+5. When should I pass `key=` and when should I rely on cryptic ids to register a file? What’s the recommended process to register a file?
+
+6. Will I never be able to find my file if I don’t give it a description? (should this even be allowed?)
+
+7. What should I do if I have a local file and want to upload it to S3? (Shall I register a File first and upload it with `.save`, or shall I upload outside of Lamin before registering it?)
+
+8. How to update a file in storage? What’s the process to update file records after I moved files around or updated files?
+
+9. How do I version a file? Do I always make a new file record and a new transform if I want to track the parent files?
diff --git a/noxfile.py b/noxfile.py
@@ -33,6 +33,16 @@ def install(session: nox.Session) -> None:
 ```
 """
 
+FAQ_MATCH = """\
+```
+"""
+
+FAQ_APPEND = """\
+faq/storage
+```
+"""
+
+
 OTHER_TOPICS_ORIG = """
 ```{toctree}
 :hidden:
@@ -79,9 +89,17 @@ def pull_artifacts(session):
     pull_from_s3_and_unpack("lamindb_docs.zip")
     Path("lamindb_docs/README.md").rename("README.md")
     for path in Path("lamindb_docs").glob("*"):
-        if path.name == "index.md" or "/storage/" in path.as_posix():
+        if (
+            path.name == "index.md"
+            or "/storage/" in path.as_posix()
+            or path.name == "faq"  # directory treated below
+        ):
             continue
         path.rename(Path("docs") / path.name)
+    # lamindb faq
+    for path in Path("lamindb_docs/faq").glob("*"):
+        path.rename(Path("docs/faq") / path.name)
+    replace_content("docs/faq.md", {FAQ_MATCH: FAQ_APPEND})
     # lamindb guide
     replace_content("docs/guide.md", {OTHER_TOPICS_ORIG: "\n\n"})
     # integrations