Technical Details

Tech Stack

Frontend

JS Framework: SvelteKit meta framework for Svelte, also using TypeScript
Design and Components: TailwindCSS along with Skeleton UI
Query Helper : Tanstack Query for fetching and managing asynchronous data.

Backend

API: trpc for API
Database : PostgreSQL hosted on NeonDB and with Drizzle for ORM.
Authentication : Using custom auth with JWT using jose
Hosting Provider : Vercel for hosting our website and API. As currently our backend is Edge Compatible so we are using Vercel Edge Functions for API.

Data Processing

Transliteration: Lipi Lekhika
Data Source: Valmiki Ramayanam - Wikisource
Scripts Associated the tasks in data/ramaayana/:
- get_raw_data.py: Fetches the data from the Wikisource and stores it in data/ramaayana/raw_data/. It prefers to fetch the data from the cached zip instead also of directly fetching from the Wikisource(as the source might be edited which would cause inconsitent results).
- get_text.py: Processes the raw data and stores the text in data/ramaayana/text_data/.
- get_json.py: Processes the text data and stores the JSON in data/ramaayana/data/.
- make_excel_files.ts: Processes the JSON data and stores the Excel files in data/ramaayana/out/.
- run_tests.py: Runs the tests on the JSON data, and saves the generated test result in data/ramayan/test_out.md.
Raw Data Cached Zip of Extarcted HTML on raw data release