Deploying Models on Banana 🍌

The cool people over at Banana give you probably the best way to deploy your models on serverless GPUs...

In this guide we show you how you can leverage our ready-to-go GPU dev environments to prepare and test your model before deploying on Banana. (If it works here it'll work on Banana 🤙)

Hit this link to create a new Brev environment pre-configured with everything you need

it'll come including an Nvidia T4 with 16GB of VRAM

1. If you haven't already, create an account
2. Down at the bottom, hit the link to connect a credit card (we give you 30 mins of free GPU usage 🙂) and
3. Come back to the env setup tab and hit create!

Preparing your model

Check you have the Brev CLI installed:

brev ls

If not, install it with instructions here

Open your new Brev environment with the name of the environment you set:

brev open serverless-template --wait

This'll spawn up a VSCode window right into the Banana serverless template.

You can confirm you have GPUs with:

nvidia-smi

To start the server run:

python3 server.py

Brev will then automatically forward the port to localhost:8000. Then you can edit download.py and app.py to download and prepare your custom model.

Verify the docker build using: (If you're running on Brev containers, put "sudo" in front of each docker command)

docker build -t test-build .

Then run the server via Docker:

docker run --gpus=all -p 8000:8000 test-build

If that works we're off to the races and ready to deploy!

Pushing to Banana

To deploy on Banana, push your local files to a git repo:

git remote add origin [url-of-repo]
git push -u origin main

Then in Banana console's connect your git repo and you're ready to deploy!