Skip to content

Commit

Permalink
feat: support gradio client with modelz endpoints (#37)
Browse files Browse the repository at this point in the history
* feat: support gradio client with modelz endpoints

Signed-off-by: Kaiyang-Chen <kaiyang-chen@sjtu.edu.cn>

* doc: gradio client

Signed-off-by: Kaiyang-Chen <kaiyang-chen@sjtu.edu.cn>

* doc

Signed-off-by: Kaiyang-Chen <kaiyang-chen@sjtu.edu.cn>

---------

Signed-off-by: Kaiyang-Chen <kaiyang-chen@sjtu.edu.cn>
  • Loading branch information
Kaiyang-Chen authored Jul 14, 2023
1 parent 6864aec commit 9e41a37
Show file tree
Hide file tree
Showing 5 changed files with 612 additions and 339 deletions.
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,37 @@ modelz --help
```shell
echo "cute cat" | modelz inference $PROJECT --serde msgpack --write-file cat.jpg --read-stdin
```



## Gradio Client on Modelz Endpoints

We provide a lightweight Python library that makes it very easy to use any Gradio app served on modelz as an API. The functionalities of `GradioClient` are completely identical to `Client` in `gradio_client` library provided by Gradio. The only difference is that when initializing the client, you should enter your Modelz serving endpoint URL instead of a Hugging Face space.

### Example Usage

```python
from modelz import GradioClient as Client

# Parameter here is the endpoint of your Modelz deployment
# The format is like https://${DEPOLOYMENT_KEY}.modelz.io/
cli = Client("https://translator-th85ze61tj4n3klc.modelz.io/")

cli.view_api()
# >> Client.predict() Usage Info
# ---------------------------
# Named API endpoints: 1

# - predict(text, api_name="/predict") -> output
# Parameters:
# - [Textbox] text: str
# Returns:
# - [Textbox] output: str


cli.predict("hallo", api_name="/predict")
# >> "Bonjour."


```

2 changes: 2 additions & 0 deletions modelz/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
from .client import ModelzClient
from .gradio_client import GradioClient

__all__ = [
"ModelzClient",
"GradioClient",
]
23 changes: 23 additions & 0 deletions modelz/gradio_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
from gradio_client import Client

from modelz.env import EnvConfig

config = EnvConfig()


class GradioClient:
"""Create a Gradio Client with Modelz endpoints.
Args:
host: Modelz host address
"""

def __init__(
self,
host: str = None,
) -> None:
self.host = host if host else config.host
self._client = Client(self.host)

def __getattr__(self, attr):
"""Delegate access to implement the composition."""
return getattr(self._client, attr)
Loading

0 comments on commit 9e41a37

Please sign in to comment.