Upgrading Hex security | Gleam programming language

Gleam is a type safe and scalable language for the Erlang virtual machine and JavaScript runtimes. Today Gleam v1.15.0 has been published, let's go over what's new.

The Hex package manager

Gleam is a BEAM language, so it uses Hex, the package manager of the Erlang ecosystem. Hex recently introduced a new OAuth2 based authentication system, replacing the old system of exchanging the account's username and password for a long-lived token. This new system has many security advantages:

Multi-factor authentication is used for all endpoints that require write permissions, so stolen credentials or tokens are insufficient to perform sensitive actions.
Hex clients such as Gleam never see your password, so there is less opportunity for an attacker to harvest credentials given access to your workstation.
The access tokens are short lived, so if stolen there is only a small window in which they could be used by the attacker.
The Hex site can integrate with other OAuth providers, enabling companies to use their existing identity management and audit systems to secure Hex.

Gleam now uses exclusively this new system for authentication, and any existing legacy tokens that Gleam has stored locally will be revoked the first time that the new version of Gleam is used with Hex. The legacy system will be disabled for Gleam packages in future.

Lastly, the password used for encrypting local Hex tokens must now be at least 8 characters in length.

This work was done in collaboration with and was sponsored by The Erlang Ecosystem Foundation. A huge thank-you to Jonatan Männchen and the rest of the team there! If you would like to support important infrastructure projects for the whole ecosystem consider becoming a member of the foundation.

Better Hex errors

It's possible for Hex operations to fail, for example, attempting to depend on a package that does not exist, or attempting to publish a new version of a package that you are not the maintainer of. When this happens we typically show the programmer the error message as-is from the Hex API. These error messages are not bad, but compared to the other error messages from Gleam it can be not immediately obvious what the problem is and how the programmer should proceed.

Ameen Radwan and vyacheslavhere have added custom errors for these two scenarios, fixing the two main stumbling points folks have with Hex errors. Thank you both!

Guard clause ergonomics

Adi Salimgereyev has implemented two nice improvements to guard expressions in case expressions. Firstly, the string concatenation operator can now be used:

case message {
  action if version <> ":" <> action == "v1:delete" -> handle_delete()
  _ -> ignore_command()
}

And a helpful custom error message is now shown when int and float binary operators are used incorrectly in case expression guards.

Thank you Adi Salimgereyev!

Additionally, the language server now supports renaming, go to definition, hover, and finding references from expressions in case clause guards. Thank you Surya Rose!

Internal types presentation

Gleam's internal publicity level allows definitions to be technically importable and usable from other modules, but not be part of the actual public API of the module. This may be useful for creating "escape hatch" APIs, or for facilitating shared functionality between modules owned by the same maintainer.

Because they are not public, these internal definitions are not shown in the generated API documentation, and are not suggested for autocompletion by the language server. There was however some compiler and language server functionality where they did not get special treatment, which could make it not immediately obvious that a definition is internal. Giacomo Cavalieri has fixed these now!

The "Add missing patterns" code action will insert a catch all pattern for internal types, the language server will no longer show completions for the fields of internal types, and the compiler no longer shows the structure of internal types when displaying an "Inexhaustive patterns" error. Thank you Jak!

JavaScript FFI additions

Gleam can compile to JavaScript, making use of the wealth of code that exists in the JavaScript ecosystem. To enable JavaScript code to construct and work with Gleam data structures the compiler provides an API for each type in a Gleam codebase. This release brings some improvements to this API:

The BitArray$isBitArray and BitArray$BitArray$data functions have been added, enabling JavaScript to consume the Gleam prelude bit-array type.

import { BitArray$isBitArray, BitArray$BitArray$data } from "../gleam.mjs";

export function writeFile(path, data) {
  if (BitArray$isBitArray(data)) {
    const buffer = BitArray$BitArray$data(data);
    return fs.write(path, buffer);
  }
}

TypeScript types are also generated for this API. Functions such as BitArray$isBitArray that check if a value is of a given type now have the return type value is TypeName, so the TypeScript type checker can use these functions to understand whether or not the value is the expected Gleam type.

Package quality publish checks

We want the Hex package repository to be full of high-quality and production-ready code. It is not a place for sharing or showing-off prototypes, and name-squatting is against the terms of service. We keep an eye on the packages that are published over time, monitoring for any undesirable trends that we wish to discourage, and introducing checks to prevent them occasionally.

One problem we noticed recently was folks opting to not document their package with a README, which is intended to serve as the home-page of the documentation. - The build tool will now refuse to publish any package that has the default README generated by the gleam new command, or is missing any README altogether.

Thank you Giacomo Cavalieri!

Code folding

The language server now supports textDocument/foldingRange, enabling folding for contiguous import blocks and multi-line top-level definitions, such as function bodies, custom types, constants, and type aliases.

import gleam/int
import gleam/list
import gleam/string

This block of imports can now be folded in the editor like so:

import gleam/int ...

Thank you Aayush Tripathi!

CLI documentation improvements

The help command and --help flag can be used with Gleam's command line interface to view a concise overview of various commands. This was helpful for the very basics, but typically the programmer would need to refer to the documentation on the Gleam website for more complex tasks, such as adding a dependency from git, or adding a package with a specific version constraint.

Stopping what you are doing to go and find a web page is annoying, so the gleam help add, gleam help deps, and gleam help docs commands have been improved with much more detailed documentation output, covering these sorts of tasks.

Consistent configuration

Gleam tooling uses the gleam.toml file for configuration. Due to a mistake very early in Gleam's initial development there has been an odd inconsistency with the names of two of the keys in this file: dev-dependencies and tag-prefix. They use sausage-case, but all the others use snake_case!

With this release they are now canonically snake case, making them consistent. The old format continues to be supported, but it is deprecated.

name = "my_app"
version = "2.4.0"

[dependencies]
gleam_stdlib = ">= 0.44.0 and < 2.0.0"

[dev_dependencies]
gleeunit = ">= 1.0.0 and < 2.0.0"

Extract anonymous function

The Extract function code action now has a special-case for extracting anonymous functions, using the body of the anonymous function as the body of the new extracted function.

pub fn double(numbers: List(Int)) -> List(Int) {
  let multiplier = 2
  list.map(numbers, fn(number) { number * multiplier })
  //                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
}

When this anonymous function is extracted the code is refactored like so:

pub fn double(numbers: List(Int)) -> List(Int) {
  let multiplier = 2
  list.map(numbers, function(_, multiplier))
}

fn function(number: Int, multiplier: Int) -> Int {
  number * multiplier
}

Thank you Hari Mohan!

Module usage renaming

The rename action can be triggered on usages of a module, rather than just its import, for example:

import lustre/element
import lustre/element/html
import lustre/event

fn view(model: Int) -> element.Element(Msg) {
  //                     ^  Renaming module to `el` here
  let count = int.to_string(model)

  html.div([], [
    html.button([event.on_click(Incr)], [element.text(" + ")]),
    html.p([], [element.text(count)]),
    html.button([event.on_click(Decr)], [element.text(" - ")]),
  ])
}

Using renaming when hovering on module name would result in the following code:

import lustre/element as el
import lustre/element/html
import lustre/event

fn view(model: Int) -> el.Element(Msg) {
  let count = int.to_string(model)

  html.div([], [
    html.button([event.on_click(Incr)], [el.text(" + ")]),
    html.p([], [el.text(count)]),
    html.button([event.on_click(Decr)], [el.text(" - ")]),
  ])
}

Thank you Vladislav Shakitskiy!

Add missing type parameter code action

The language server now suggests a quick-fix code action for when a custom type definition uses a type parameter in its variants that have not been declared in its header.

pub type Store {
  Store(name: String, data: inner_type)
}

This code doesn't compile as the inner_type type variable hasn't been declared. Running the code action results in the code being fixed like so:

pub type Store(inner_type) {
  Store(name: String, data: inner_type)
}

Thank you Andi Pabst!

String prefix language server triggering

It's now possible to find references and rename variables in string prefix patterns:

case wibble {
  "1" as digit <> rest -> digit <> rest
  //     ^^^^^    ^^^^
  // You can now trigger "Find references" and "Rename" from here
}

Thank you Igor Castejón!

More precise completions

The language server protocol has some ambiguities in places, which means that different editors can have slightly different behaviour for the same actions. One way in which this has caused a problem for Gleam programmers is when using a completion where the suffix already exists in the code. For example, imagine you're updating your code to fully qualify the uses of the Json type:

pub fn payload() -> js|Json
//                    ^ typing the module name

Accepting the json.Json completion in some editors would result in the incorrect code json.JsonJson. To prevent this annoyance the language server now gives more precise instructions to the editor, so all will render the correct json.Json code.

Thank you Giacomo Cavalieri!

Signature help type parameter names

The language server now shows the original names used for type parameters in the signature help information. This can help understand what each of the parameters mean. For example:

pub fn twice(data: input, function: fn(input) -> output) -> output {
  function(function(data))
}

pub fn main() {
  twice( )
  //    ↑ Cursor here
}

This will show this signature help information:

wibble(input, fn(input) -> output)

Thank you Samuel Cristobal!

Other language server improvements

Developer experience and productivity are our main focus, so we have lots of small but impactful quality-of-life improvements in the language server.

The language server now suggests completions for keywords that are expressions, like echo, panic, and todo. Thank you Giacomo Cavalieri.

The "Fill labels" code action now uses variables from scope when they match the label name and expected type, instead of leaving a todo placeholder that the programmer would need to replace with those variables. Thank you Vladislav Shakitskiy.

The "Interpolate String" code action now lets the user "cut out" any portion of a string, regardless of whether it is a valid Gleam identifier or not. Thank you Hari Mohan.

The language server now performs best-effort zero value generation for decode.failure and has support for Nil when using the generate dynamic decoder code action. Thank you Hari Mohan.

The language server now allows extracting the start of a pipeline into a variable. Thank you Giacomo Cavalieri.

Custom type definitions and custom type variants now show their documentation on hover, so you can see what it looks like when rendered. Thank you Giacomo Cavalieri.

And the rest

And thank you to the bug fixers and experience polishers: acandoo, Andrey Kozhev, Christian Widlund, daniellionel01, Etienne Boutet, Gavin Morrow, Giacomo Cavalieri, Hari Mohan, John Downey, Justin Lubin, Surya Rose, vyacheslavhere, and Vladislav Shakitskiy.

For full details of the many fixes and improvements they've implemented see the changelog.

A call for support

Gleam is not owned by a corporation; instead it is entirely supported by sponsors, most of which contribute between $5 and $20 USD per month, and Gleam is my sole source of income.

We have made great progress towards our goal of being able to appropriately pay the core team members, but we still have further to go. Please consider supporting the project or core team members Giacomo Cavalieri and Surya Rose on GitHub Sponsors.

Thank you to all our sponsors! And special thanks to our top sponsors:

Try Gleam