Skip to content

Rewrite Flycast blueprint to drop GPU/Ollama example #2394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kcmartin wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Rewrite Flycast blueprint to drop GPU/Ollama example #2394

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6fa4830
Rewrite Flycast blueprint to drop GPU/Ollama example
kcmartin Apr 29, 2026
06fef08
Fix broken IP addresses reference link
kcmartin Apr 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 29 additions & 75 deletions blueprints/private-applications-flycast.html.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,16 @@
author: xe
categories:
- networking
date: 2024-06-17
date: 2026-04-29
---

<center><iframe width="600" height="315" src="https://www.youtube-nocookie.com/embed/PR0rzkwKwFA" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe></center><br />

## Overview

A lot of the time your applications are made to be public and shared with the world. Sometimes you need to be more careful. When you deploy your apps on Fly.io, you get a private network for your organization. This lets your applications in other continents contact each other like they were in the same room.

Sometimes you need a middle ground between fully public apps and fully private apps. [Flycast](/docs/networking/flycast/) addresses are private but global IPv6 addresses inside your private network that go through the Fly Proxy, so you get all of the load management and Machine waking powers that you get for free with public apps.

This blueprint covers what Flycast is, when and why you’d want to use it, and shows you how to create an instance of Ollama that you can connect to over Flycast.
This blueprint covers what Flycast is, when and why you’d want to use it, and shows you how to deploy a small private HTTP service that you can connect to over Flycast.

## What is Flycast?

Expand All @@ -37,17 +35,13 @@

Just a heads-up. In general, it’s a bad idea to assume that network access barriers like Flycast or NAT are security layers. At best, this is an obfuscation layer that makes it more difficult for attackers to get into private applications. Flycast is not a replacement for authentication in your private applications. With Flycast, you don’t know _who_ a request is coming from, but you do know that it’s coming from _something_ or _someone_ in your private network.

One of the biggest platform features that uses Flycast out of the box is Fly Postgres. Even though Flycast addresses are local to your private network, Fly Postgres still configures usernames and passwords for your database.
One of the biggest platform features that uses Flycast out of the box is [Managed Postgres](/docs/mpg/). Even though Flycast addresses are local to your private network, Managed Postgres still configures usernames and passwords for your database.

## Goal

We'll show Flycast off by setting up an instance of [Ollama](https://ollama.com+external).

Ollama is a program that wraps large language models and gives you an interface like Docker so that you can run open-weights large language models privately on your own device. Large language models are computationally expensive to run, so being able to offload them to a GPU-powered Fly Machine means you can hack all you want without burning up your precious battery life.
We'll show Flycast off by deploying a small HTTP service that's only reachable from inside your organization's private network. The classic example is an internal admin panel: a service you and your apps need to reach, but that should never be exposed to the public internet. Flycast also lets the platform turn the service off when nobody's using it, so you're not paying to keep an idle admin panel running.

Ollama doesn’t ship with authentication by default. When you create an instance of Ollama, anyone can access it without entering in a username, password, or API key. This is fine for running your models on your own computer; but it means that if you expose it to the internet, anyone can use it and run models whenever they want. This could rack up your bill infinitely.

This is where Flycast comes in. Flycast lets you run a copy of Ollama on your private network so that you and your apps can access it, but nobody else. Flycast also lets you have the platform turn off your Ollama server when you’re not using it, which will save you money. This fits into that middle ground case that Flycast covers perfectly.
For this walkthrough we'll use the public [`nginxdemos/hello`](https://hub.docker.com/r/nginxdemos/hello+external) image as a stand-in for any private HTTP service. It's a tiny container that responds to HTTP requests with the server's hostname and address, which makes it easy to confirm that traffic is reaching the right Machine over Flycast.

Check warning on line 44 in blueprints/private-applications-flycast.html.md

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
[Fly.Spelling] Is 'walkthrough' a typo?

Raw Output:
{"message": "[Fly.Spelling] Is 'walkthrough' a typo?", "location": {"path": "blueprints/private-applications-flycast.html.md", "range": {"start": {"line": 44, "column": 10}}}, "severity": "INFO"}

## Prerequisites

Expand All @@ -56,41 +50,29 @@
- [sign up or sign in](/docs/getting-started/sign-up-sign-in/) to Fly.io
- [install flyctl](/docs/flyctl/install/) (the Fly CLI)

If you want to interact with your Flycast apps&mdash;like an Ollama instance&mdash;from your computer, you’ll need to [jack into your private network with WireGuard](/docs/blueprints/connect-private-network-wireguard/).
If you want to interact with your Flycast apps from your computer, you’ll need to [jack into your private network with WireGuard](/docs/blueprints/connect-private-network-wireguard/).

## Steps

Create a new folder on your computer called `ollama`. This is where we’ll put the Ollama configuration. Open a terminal in that folder and run the fly launch command:

```
fly launch --from https://github.com/fly-apps/ollama-demo --no-deploy
```

This command creates a new fly app from the [`ollama-demo` template](https://github.com/fly-apps/ollama-demo+external) and tells the flyctl command to not deploy it after you create the app. If we don’t do this, then the platform will create public IPv4 and IPv6 addresses, which will make this a public app. The name you choose when you create your app will be used to connect to your app over Flycast.

Next, allocate a Flycast address for your app with the `fly ips allocate-v6` command:
Create a new folder on your computer called `flycast-demo` and open a terminal in it. We don't need any source code for this walkthrough — we'll launch the app directly from a public Docker image with the `--flycast` flag, which tells Fly Launch to allocate a private IPv6 address instead of public ones:

Check warning on line 57 in blueprints/private-applications-flycast.html.md

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
[Fly.Spelling] Is 'walkthrough' a typo?

Raw Output:
{"message": "[Fly.Spelling] Is 'walkthrough' a typo?", "location": {"path": "blueprints/private-applications-flycast.html.md", "range": {"start": {"line": 57, "column": 126}}}, "severity": "INFO"}

```
$ fly ips allocate-v6 --private
fly launch --image nginxdemos/hello --flycast
```

Now you can deploy the app with the `fly deploy` command:

```
$ fly deploy
```
The name you choose for your app will be the hostname you use to reach it over Flycast (`<appname>.flycast`). When `fly launch` asks if you want to tweak the settings, you can accept the defaults.

After that finishes, you can see the list of IP addresses associated to an app with `fly ips list`:
After deploy finishes, you can see the list of IP addresses associated to an app with `fly ips list`:

```
$ fly ips list
VERSION IP TYPE REGION CREATED AT
v6 fdaa:3:9018:0:1::7 private global 23h12m ago

Learn more about [Fly.io public, private, shared and dedicated IP addresses](/docs/reference/services/#ip-addresses).
```

This app only has one IP address: a private Flycast IPv6 address. If had public IP addresses, it'd look like this:
Learn more about [Fly.io public, private, shared and dedicated IP addresses](/docs/networking/services/#anycast-ip-addresses).

This app only has one IP address: a private Flycast IPv6 address. If it had public IP addresses, it'd look like this:

```
$ fly ips list -a recipeficator
Expand All @@ -112,88 +94,60 @@
# apt update && apt install -y curl iputils-ping dnsutils
```

My app is named`xe-ollama`, so let’s look up its `.flycast` address with `nslookup xe-ollama.flycast`:
Say my app is named `flycast-demo`. Let’s look up its `.flycast` address with `nslookup flycast-demo.flycast`:

```
# nslookup xe-ollama.flycast
# nslookup flycast-demo.flycast
Server: fdaa::3
Address: fdaa::3#53

Name: xe-ollama.flycast
Name: flycast-demo.flycast
Address: fdaa:3:9018:0:1::7
```

It matches that IP address from earlier. Now let’s see what happens when we ping it:

```
# ping xe-ollama.flycast -c2
PING xe-ollama.flycast (fdaa:3:9018:0:1::7) 56 data bytes
# ping flycast-demo.flycast -c2
PING flycast-demo.flycast (fdaa:3:9018:0:1::7) 56 data bytes
64 bytes from fdaa:3:9018:0:1::7: icmp_seq=1 ttl=63 time=0.138 ms
64 bytes from fdaa:3:9018:0:1::7: icmp_seq=2 ttl=63 time=0.223 ms

--- xe-ollama.flycast ping statistics ---
--- flycast-demo.flycast ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1009ms
rtt min/avg/max/mdev = 0.138/0.180/0.223/0.042 ms
```

Perfect, now let’s make a request to the Ollama app with curl:
Now let’s make a request to the app with curl:

```
# curl http://xe-ollama.flycast
# curl http://flycast-demo.flycast
```

It took a moment for Ollama to spin up, and now we get a happy “Ollama is running” message. Wait a few moments so your Ollama app goes to sleep and run the `time` command to see how long the first request takes:
You should get back an HTML page with the server's hostname and address. Wait a few moments so your app goes to sleep, then run the `time` command to see how long the first request takes:

```
# time curl http://xe-ollama.flycast
Ollama is running
real 0m9.144s
# time curl -o /dev/null -s http://flycast-demo.flycast
real 0m2.411s
user 0m0.003s
sys 0m0.003s
```

It took a few seconds for the platform to wake up Ollama and make sure it was ready for your requests. The next request is a lot faster:
It took a couple of seconds for the platform to wake the Machine up and make sure it was ready for your request. The next request is a lot faster:

```
# time curl http://xe-ollama.flycast
Ollama is running
# time curl -o /dev/null -s http://flycast-demo.flycast
real 0m0.043s
user 0m0.003s
sys 0m0.003s
```

And if you wait a few moments, it’ll spin back down.

### Running Llama 3

Now that we’ve set up Ollama and demonstrated the platform turning it off and on for you, let’s run Llama 3. Exit out of that shell Machine with control-D so we can make a new one with the Ollama client installed.

Create an Ollama shell using `fly machine run`:

```
$ fly machine run --shell ollama/ollama
```

Once that starts up, point the Ollama client to your Flycast app by setting the `OLLAMA_HOST` environment variable:

```
# export OLLAMA_HOST=http://xe-ollama.flycast
```

Then you can ask Llama 3 anything you want:

```javascript
# ollama run llama3 "Why is the sky blue?"
```

It took a moment for Ollama to get ready and download the image, then it downloaded it and answered your question. Once it’s been idle for a moment, the platform will turn Ollama back off.

And there we go! We’ve covered what Flycast is, why you’d want to use it, and set up an instance of Ollama to show it off.
And there we go! We’ve covered what Flycast is, why you’d want to use it, and walked through deploying a private HTTP service to show it off. The same pattern works for any internal app: admin panels, internal APIs, dashboards, or anything else you'd rather keep off the public internet.

## Related reading

We've talked about Flycast in some past blog posts and blueprints:

- [Autostart and autostop private apps](/docs/blueprints/autostart-internal-apps/)
- [Deploy Your Own (Not) Midjourney Bot on Fly GPUs](https://fly.io/blog/not-midjourney-bot/)
- [Scaling Large Language Models to zero with Ollama](https://fly.io/blog/scaling-llm-ollama/)
- [Flycast — private Fly.io services](/docs/networking/flycast/)
- [Private networking](/docs/networking/private-networking/)
Loading