The cool people over at Banana give you probably the best way to deploy your models on serverless GPUs...
In this guide we show you how you can leverage our ready-to-go GPU dev environments to prepare and test your model before deploying on Banana. (If it works here it'll work on Banana 🤙)
Hit this link to create a new Brev environment pre-configured with everything you need
it'll come including an Nvidia T4 with 16GB of VRAM
- If you haven't already, create an account
- Down at the bottom, hit the link to connect a credit card (we give you 30 mins of free GPU usage 🙂) and
- Come back to the env setup tab and hit create!
Preparing your model
Check you have the Brev CLI installed:
If not, install it with instructions here
Open your new Brev environment with the name of the environment you set:
brev open serverless-template --wait
This'll spawn up a VSCode window right into the Banana serverless template.
You can confirm you have GPUs with:
To start the server run:
Brev will then automatically forward the port to localhost:8000. Then you can edit download.py and app.py to download and prepare your custom model.
Verify the docker build using:
docker build -t test-build .
Then run the server via Docker:
docker run --gpus=all -p 8000:8000 test-build
If that works we're off to the races and ready to deploy!
Pushing to Banana
To deploy on Banana, push your local files to a git repo:
git remote add origin [url-of-repo] git push -u origin main
Then in Banana console's connect your git repo and you're ready to deploy!