#AlchemyConf 2025 is a two-day event in Braga, Portugal, focused on Elixir development. Taking place on April 2-3, it‘s a chance for the Elixir community to share insights, discuss real-world applications, and connect in person. #ElixirLang #MyElixirStatus

The 2024 Lightning talks from @ElixirConf are out! piped.video/QJCWzN1Vahs #MyElixirStatus #ElixirConf2024 #ElixirConf

I started with @elixirlang and get why people love it. The examples is not only doc, but also test for the code!

.@supabase's Supavisor is a connection pooler written in @elixirlang. The design is pretty neat. Poolers can teach you a lot about how TCP and database connections work. And as you'll see, Elixir/OTP is purpose-built for this stuff. What’s a connection pooler? A connection pooler is an intermediary between client connections and database connections. Clients open up connections with the pooler as opposed to with the database. The pooler maintains connections with the database. When a client connection wants to make a request, the pooler finds an available database connection and routes traffic between the two: A connection is an open TCP socket and some state. Why pool? The primary motivation for Supabase to pool is to better support serverless workflows. In a persistent fullstack web framework like Phoenix, your connection library (e.g. Ecto) pools for you. In serverless, processes are ephemeral. So every time your function boots, it needs to establish a database connection. Setting up and tearing down database connections is actually kind of expensive! When a new connection is established, Postgres needs to authenticate the user and then set up that connection’s environment (allocate memory and load some stuff into it). And this process involves several network round-trips. Furthermore, serverless functions don’t always clean up neatly after themselves. It’s easy to forget to close your database connection. Or if your function crashes, it may erroneously leave the connection open. The secondary motivation was IPv4 address exhaustion. In order for your client to connect to your database, it needs an IP address. Without a pooler, this is a direct connection. Meaning every Supabase database needed a public IP address. IPv4s are not free. And IPv6 isn’t widely supported enough yet to be a viable alternative (different topic, different day). How does it work? So, you need a persistent TCP connection with some state for each client connection. And the same for each server connection. Persistent... stateful... if only OTP had the tools! 😁 The pooler in Supavisor consists of two pools: the client connection pool and the database connection pool. And each of these pools are comprised of GenServers (more specifically a GenStateMachine or GSM), one per connection. For non-Elixir folk, a GenServer is a persistent, stateful process in OTP (the Elixir platform). This process is addressable, and other processes can send/receive messages to/from it. A GSM is the state machine version of a GenServer. Good for this use case, because a database connection has a lifecycle well-modeled as a state machine. So, if there are 1000 client connections and 100 database connections, there will be 1000 `ClientHandler` GSMs and 100 `DbHandler` GSMs running in Supavisor. To create its pool of connections, Supavisor boots up a DbHandler GSM for each database connection. The GSM is responsible for handling the TCP connection with the database (`gen_tcp`). Likewise, when clients initiate a connection, Supavisor spawns a ClientHandler GSM, which is responsible for the client-side TCP connection. So, at a high level, queries work like this: the client sends a query over its TCP connection with Supavisor, `select 'is this thing on?'`. This results in a `handle_event` callback on ClientHandler being invoked with the query. ClientHandler "checks out" a database connection from the pool. What that means in practice: it marries itself to the pid of an available DbHandler. It then forwards the client query along to the DbHandler. The DbHandler, in turn, forwards the message along to the database over its open TCP connection. When the database responds, the process happens in reverse: the database’s response invokes a `handle_event` callback on DbHandler with the TCP payload. The DbHandler routes the response to the ClientHandler it's currently paired up with. And the ClientHandler sends the response over its open TCP connection. Finally, it "releases" the DbHandler back into the pool. This design solves the resource allocation problem: when new ClientHandlers boot up, the startup cost is cheap: Supavisor handles the authentication handshake without involving the database at all (it caches credentials). And if a serverless function doesn't shut down its TCP connection with Supavisor, the cost is low. It just means a ClientHandler will be sitting around idly (idle GenServer is much cheaper than idle Postgres connection). And it solves the IPv4 allocation problem: Supabase can have one big Supavisor cluster per region with a single hostname. And that cluster can contain ClientHandler/DbHandler pools for tens of thousands of dbs. The code With this background knowledge, you’ll probably enjoy scanning through some code. You can find ClientHandler here: github.com/supabase/supaviso… The `handle_event(info, {:tcp, _, bin}, data)` callbacks are the handlers for messages coming through the gen_tcp socket. If you scroll down to the "incoming query" functions, you’ll see the ClientHandler checkout database connections and route queries to the DbHandler. And you can find DbHandler here: github.com/supabase/supaviso… Skip the authentication parts. The `handle_event({:call, from}...` are incoming messages from the ClientHandler, you’ll see the DbHandler forward along via its socket. And again the `handle_event(:info, {:tcp ... parts` are the DbHandler handling responses from the database. It’s a neat project, big shout out to Supabase for creating it. I know a lot of folks have contributed to it, but special shout out to @abc3erl and @chasers who helped me wrap my head around Supavisor in its early days.

Hooray! Thanks to the Erlang Ecosystem Foundation, the next versions of @livebookdev and @elixirlang will have signed installers on Windows. This means a much smoother experience for those getting started. Kudos to @TheErlef and @wojtekmach for doing the hard work!

BTW, this is (finally) the version we are recommending that you can build applications with. API should be almost entirely stable at this point. Today you can build LiveView apps for: * iPhone * iPad * Apple Watch * AppleTV * MacOS * Apple Vision Pro #myelixirstatus

New Demo: Pawsitively – LLM content moderation API in 50 LOC, including the entire Phoenix app. Plus a LiveView front-end to play with it. Implemented and deployed entirely as a livebook! 🧵

Elixir community, i need some help for builtwithphoenix.com please! What tech are people using with Phoenix! Reply like so: ``` name: "Nerves", url: "nerves-project.org/" logo_url: "github.com/nerves-project/ne…" ``` #MyElixirStatus @elixirlang @elixirphoenix

Congrats to the @elixirlang & @elixirphoenix teams for ranking among the most admired languages and frameworks on this year's Stack Overflow Developer Survey! #MyElixirStatus

Most apps need file exports. I'm excited to share my latest video on optimizing handlers in Elixir with chunked responses! Learn how to get more out of your server memory and see the Stream module in action. #MyElixirStatus piped.video/OMVYZziNjhc

The Process 4.2 is out. This episode is about corner cases when calling GenServers, the concepts are key to the stability and reliability of software written in the BEAM ecosystem. piped.video/watch?v=cSqY0zMi… #myelixirstatus #webeamtogether #elixirlang

It is truly incredible how far ~50 lines of code can get you with the right tools. Extremely simple, Livebook-based chat with local model. Took 15min to put together. Kudos to @livebookdev @AIatMeta @ollama @lebrunel for making magic like this possible! #myelixirstatus

2024 StackOverflow Developer Survey results are in. Elixir continues to rank as the second most admired language, Phoenix as the most admired web framework. 💜 (“Admired” here meaning % of people who worked with it and want to continue doing so.) #MyElixirStatus

@elixirphoenix is at #1 in the web framework category

@elixirlang is 2nd most admired language on the @StackOverflow survey, yet again. #elixir

I'm looking for an @elixirlang wiz in the U.S. Remote anywhere in the U.S. is ok. $175k-ish. Wide range depending on your experience. The whole app is @elixirlang LiveView. You'll be engineer #3 at a very fast growing startup. jumpapp.com linkedin.com/jobs/view/39789…

I've been hacking on a new experimental live coding system in collaboration with @sheffielduni. We're giving an introductory kids friendly workshop on the 21st of Sept alongside techno legends @TheBlackDog. Come along and live code some beats with us! eventbrite.co.uk/e/a-techno-…

It's always great to learn about a language directly from the creator. This week I'm joined by @josevalim—the creator of #Elixir—to understand its journey from inception to a type-checked future: piped.video/IGmwiyines0

In the next few days I'll be working on providing precompiled artifacts so that users don't have to go through setting up the IREE compiler compilation As always, this wouldn't be possible without @DockYard, as this was done during work hours!

There's still some work to be done in the way of providing precompiled artifacts but... Metal support for Nx is here! github.com/elixir-nx/nx_iree… contains a working example of how to run Nx functions on NxIREE. Many thanks to the IREE team who've helped a lot in the past few months