September 2025 Devlog

September brought major improvements to Firezone's reliability, performance, and administrative capabilities. Our focus this month was on making connections more resilient, helping admins manage client versions more effectively, and optimizing our networking stack for real-world conditions.

Smarter Admin Tools

Managing a fleet of Clients just got easier. The admin portal now shows client versions directly in the clients table with a sortable version column.¹ When planning Gateway upgrades, admins can quickly identify which clients might lose connectivity due to version incompatibility.

We've also enhanced the "outdated Gateway" email notification system, which is now enabled by default for all accounts.² These emails now include a count of clients that will be affected by the Gateway upgrade, with a direct link to view them in the portal.

For organizations with large user bases, we added batch upsert operations for directory sync.³ This lays the groundwork for more efficient directory synchronization that can handle thousands of users without overwhelming the system.

A new /account API endpoint provides programmatic access to billing details and seat usage.⁴ This enables better integration with capacity planning and billing automation tools.

Rock-Solid Connections

Connection reliability saw significant improvements across the board. We implemented graceful connection shutdown using a new peer-to-peer control protocol message.⁵ When Gateways restart for maintenance or upgrades, Clients now immediately fail over to alternative Gateways instead of waiting through a 15-second ICE timeout.⁶

This graceful handling also applies when Clients sign out, making it much easier to distinguish between actual network problems and normal disconnections in the logs.

We fixed a synchronization issue where Clients and Gateways could disagree on authorization state.⁷ Now when a Client receives an ICMP "prohibited" error from a Gateway, it automatically re-authorizes access for that Resource. This keeps both sides in sync even when authorization states drift out of alignment.

DNS resource handling got more reliable too. Proxy IP assignments for DNS resources now persist across Client sessions.⁸ Previously, restarting the Client could reassign 100.96.0.1 to a different resource, breaking applications that cached the IP address. Now these mappings stay consistent for the lifetime of the tunnel process, fixing issues with long-lived connections like SSH sessions.

Better DNS Resolution

The Gateway switched from using the system's libc resolver to the hickory-resolver library for A and AAAA record resolution.⁹ This change enables proper TTL-based caching instead of the previous hardcoded 30-second cache. DNS responses are now cached for exactly as long as the authoritative nameserver specifies, reducing unnecessary queries while ensuring timely updates.

Performance Under Load

We optimized the event-loop to prevent starvation of lower-priority inputs.¹⁰ The tunnel now batch-processes input from all sources rather than prioritizing high-traffic sources to completion. This prevents timeout checks and DNS resolution from being delayed when UDP sockets or the TUN device are extremely busy.

Linux systems got a significant performance boost through better socket buffer management.¹¹ Clients and Gateways now attempt to set UDP receive and send buffer maximums to 128 MB and 16 MB respectively at startup. The default 200KB buffers were causing packet drops during high throughput scenarios, directly correlating with buffer errors visible in nstat.

We also limited the number of optimistic ICE candidates to prevent CPU spikes.¹² When clients advertise many IPv6 addresses, the previous unlimited candidate generation could cause performance issues. Optimistic candidates are now disabled entirely for IPv6 and limited to 2 for IPv4, focusing on the scenarios where they're most beneficial.

Relay Infrastructure Advances

Our relay infrastructure received important eBPF improvements. The relay can now handle relay-to-relay candidate pairs in the eBPF kernel, which occur when both Client and Gateway allocate from the same relay.¹³ Previously these packets would need to traverse userspace or fail entirely.

We restructured the eBPF code into modular components that handle all cross-stack translation cases, including IPv4↔IPv6 transformations. A comprehensive integration test with double symmetric NAT validates this behavior in continuous integration.

We also fixed several edge cases in the eBPF layer, including properly handling DNS replies¹⁴ and re-populating the channel map when TURN channels are refreshed.¹⁵

Platform-Specific Improvements

The Android client now launches authentication in a CustomTab instead of the default browser.¹⁶ This fixes a Firefox bug where only the first browser tab could intercept the custom URI scheme, making subsequent sign-ins fail until users manually closed the old tab. CustomTabs ensure only one sandboxed authentication instance exists at a time. It also results in a nice UX improvement where the "You have been signed-in" tabs are no longer lingering around.

macOS developers got quality-of-life improvements making the workspace build correctly on macOS with appropriate stubs and conditional compilation.¹⁷

The macOS client now detects and alerts users when multiple instances are running, preventing interference with tunnel state.¹⁸

Infrastructure and Tooling

All CI runners now use Ubuntu 24.04 to match production relay environments.¹⁹ This ensures builds and tests run on the same kernel version as production, catching platform-specific issues earlier.

The docker-compose test environment now uses realistic network topology with separate subnets for Clients, Gateways, relays, and backend.²⁰ Each component has a dedicated router container performing NAT and firewall rules, enabling proper testing of relayed connections with port randomization.

Database performance improved with the addition of missing indexes on foreign key columns.²¹²² These indexes ensure efficient cascade deletes now that hard-delete has been fully rolled out.

Notable Bug Fixes

Several important bugs were squashed:

Gateway re-joins Phoenix channel topic on send errors to prevent message loss²³
Fixed poll-after-completion panics in Client session event-loop²⁴
Fixed DNS resource NAT reset when Client reassigns proxy IPs after sign out²⁵
Relay filters traces by log filter to respect OTEL configuration²⁶
Internet site no longer counts against Starter plan resource limits²⁷

That's September in a nutshell. See our changelog for a more compact version of the above or view the full diff on GitHub.

About Firezone

Firezone is an open source platform for securely managing remote access to your organization's networks and applications. Unlike traditional VPNs, Firezone takes a granular, least-privileged approach with group-based policies that control access to individual applications, entire subnets, and everything in between. Get started for free or learn more about how Firezone can help secure your organization.