LLM Api

This repo contains the code for running an LLM Api in 2 environments:

1. dev: A development environment running locally on docker
2. prd: A production environment running on AWS ECS

Setup Workspace

1. Clone the git repo

from the llm-api dir:

2. Create + activate a virtual env:

python3 -m venv aienv
source aienv/bin/activate

3. Install phidata:

pip install phidata

4. Setup workspace:

phi ws setup

5. Copy workspace/example_secrets to workspace/secrets:

cp -r workspace/example_secrets workspace/secrets

6. Optional: Create .env file:

cp example.env .env

Run LLM Api locally

1. Install docker desktop

2. Set OpenAI Key

Set the OPENAI_API_KEY environment variable using

export OPENAI_API_KEY=sk-***

OR set in the .env file

3. Start the workspace using:

phi ws up

Open localhost:8000/docs to view the FastApi docs.

4. Stop the workspace using:

phi ws down

Next Steps:

• Run the LLM Api on AWS
• Update the dev application
• Update the production application
• Add python dependencies
• Format & validate your code
• Secret management
• CI/CD
• Add database tables
• Read the LLM Api guide