Configuration

Registry URLs, auth tokens, proxy settings, cache behavior. Every package manager has a way to configure these, and they almost always live outside the manifest.

.npmrc is an INI-format file that can live at the project root, in your home directory, or globally. npm and pnpm both read it. It controls the registry URL, auth tokens for private registries, proxy settings, and dozens of install behaviors like legacy-peer-deps and engine-strict. There’s a footgun here: if an .npmrc ends up inside a published package tarball, npm will silently apply those settings when someone installs your package in their project. Less well known are the shell, script-shell, and git settings, which point at arbitrary executables that npm will invoke during lifecycle scripts and git operations. Research by Snyk and Cider Security showed these as viable attack vectors: a malicious .npmrc committed to a repository can redirect script execution without touching package.json at all.

.yarnrc.yml replaced the INI format of Yarn Classic’s .yarnrc. It configures which linker to use (PnP, pnpm-style, or traditional node_modules), registry auth, and the pnpMode setting that controls how strictly Yarn enforces its dependency resolution. The yarnPath setting is security-sensitive: it points to a JavaScript file that Yarn will execute as its own binary, so a malicious .yarnrc.yml can hijack the entire package manager.

bunfig.toml is Bun’s config file, covering registry config, install behavior, and the test runner all in one TOML file.

pip.conf on Unix and pip.ini on Windows, searched at ~/.config/pip/pip.conf, ~/.pip/pip.conf, and /etc/pip.conf. The PIP_CONFIG_FILE environment variable can override all of these or point to /dev/null to disable config entirely. Malformed config files are silently ignored rather than producing errors, so you can have broken configuration for months without realizing it.

uv.toml or the [tool.uv] section in pyproject.toml.

.bundle/config stores Bundler’s per-project config, created by bundle config set. RubyGems has its own .gemrc file, which Bundler deliberately ignores because it calls Gem::Installer directly. The credentials file at ~/.gem/credentials must have 0600 permissions or RubyGems refuses to read it.

.cargo/config.toml is the most interesting of the bunch because it’s hierarchical: Cargo walks up the directory tree merging config files as it goes, so you can have workspace-level settings that individual crates inherit. It controls registries, proxy settings, build targets, and command aliases. A backwards-compatibility quirk means Cargo still reads .cargo/config without the .toml extension, and if both files exist, the extensionless one wins, which is an easy way to have a stale config file shadow your actual settings.

.condarc is searched at six different paths from /etc/conda/.condarc through ~/.condarc to $CONDA_PREFIX/.condarc, plus .d/ directories at each level for drop-in fragments, and you can put one inside a specific conda environment to configure just that environment. Every setting also has a CONDA_UPPER_SNAKE_CASE environment variable equivalent.

~/.m2/settings.xml holds Maven’s repositories and credentials, plus ~/.m2/settings-security.xml stores the master password used to decrypt encrypted passwords in the main settings file. Most developers don’t know settings-security.xml exists. .mvn/maven.config holds per-project default CLI arguments (since Maven 3.9.0, each arg must be on its own line), and .mvn/jvm.config sets JVM options.

gradle.properties lives at both project and user level. Init scripts in ~/.gradle/init.d/ run before every build, which is how enterprises inject internal repository configurations across all projects.

auth.json keeps Composer credentials separate from composer.json (per-project or at ~/.composer/auth.json) so you can gitignore it.

nuget.config is XML searched hierarchically from the project directory up to the drive root, then at the user level. Like pip, malformed XML is silently ignored.

deno.json is both configuration and import map, controlling formatting, linting, test config, lock file behavior, and dependency imports in a single file. If you have a separate import_map.json, Deno reads that too, though the trend is toward folding everything into deno.json.

Publishing

What gets included or excluded when you publish a package. People accidentally ship secrets and accidentally omit files they need in roughly equal measure.

.npmignore works like .gitignore but for npm pack and npm publish. If it doesn’t exist, npm falls back to .gitignore. But if you create an .npmignore, it completely replaces .gitignore for packaging purposes, they are not merged. This means patterns you had in .gitignore to keep .env files or credentials out of version control no longer protect you from publishing them. npm-shrinkwrap.json is identical in format to package-lock.json but gets included inside published tarballs. It’s the only npm lock file that travels with a published package, intended for CLI tools and daemons that want locked transitive dependencies for their consumers rather than letting the consumer’s resolver pick versions.

MANIFEST.in controls what goes into a Python source distribution using directives like include, exclude, recursive-include, graft, and prune. It only matters for sdists, not wheels.

.helmignore controls what gets excluded when packaging a Helm chart, following .gitignore syntax.

Workspaces

Monorepo topology and inter-package relationships. The JavaScript ecosystem has the most options here, which probably says something about the JavaScript ecosystem.

pnpm-workspace.yaml defines workspace membership with a packages: field. Where npm and Yarn put this in a workspaces field in package.json, pnpm requires a separate file.

lerna.json handles versioning and publishing across workspace packages, though Lerna’s remaining value is mostly the publishing workflow (changelogs, version bumps). nx.json and turbo.json configure task pipelines and caching for Nx and Turborepo monorepo builds.

go.work (added in Go 1.18) lists use directives pointing to local module directories so you can develop across multiple modules without replace directives scattered through your go.mod files. It generates a companion go.work.sum checksum file.

settings.gradle / settings.gradle.kts declares all Gradle subprojects with include statements and is mandatory for multi-project builds. Maven uses <modules> in a parent pom.xml.

Overrides and resolution

When a transitive dependency has a bug or a security vulnerability and you can’t wait for every package in the chain to release an update, override files let you force a specific version or patch a package in place. Most developers don’t know these mechanisms exist and spend hours working around dependency conflicts that a single config line would fix.

In the JavaScript ecosystem, npm has overrides, Yarn has resolutions, and pnpm has pnpm.overrides, all fields in package.json that force specific versions of transitive dependencies. Yarn Berry and pnpm also support patching dependencies in place: Yarn’s patch: protocol stores diff files in .yarn/patches/, and pnpm’s pnpm.patchedDependencies references diffs in a patches/ directory, built into the workflow via pnpm patch and pnpm patch-commit.

.pnpmfile.cjs goes further than any of these: the readPackage hook lets you programmatically rewrite any package’s package.json at install time, and afterAllResolved can modify the lockfile after resolution. It’s the nuclear option for dependency problems, living next to the lockfile and running before anything gets installed.

constraints.txt is used via pip install -c constraints.txt to pin versions of packages without triggering their installation. It’s been available since pip 7.1, yet almost nobody uses it despite being exactly what large organizations need for base image management and reproducible environments. uv has override-dependencies in [tool.uv] for the same purpose with better ergonomics.

Directory.Packages.props is worth knowing about if you work in .NET. NuGet’s Central Package Management (6.4+) lets you put a single file at the repo root that sets <PackageVersion> for all projects, so individual .csproj files use <PackageReference> without version numbers. It eliminates version drift across large solutions and is one of the better implementations of centralized version management I’ve seen. Directory.Build.props can inject shared package references into all projects too.

gradle/libs.versions.toml is Gradle’s version catalog, with sections for [versions], [libraries], [bundles], and [plugins], referenced in build files as typed accessors like libs.someLibrary.

cabal.project supports constraints: stanzas for pinning transitive Haskell deps, and cabal.project.freeze locks everything down.

Vendoring and integrity

Beyond lockfiles, some package managers support vendoring all dependency source code into the repository and tracking its integrity.

.cargo-checksum.json lives in each vendored crate directory after running cargo vendor, containing the SHA256 of the original tarball and per-file checksums. If you need to patch vendored source (which you sometimes do for air-gapped builds), setting "files": {} in the checksum file disables integrity checking for that crate, which is the known workaround and also completely defeats the purpose of the checksums.

GONOSUMCHECK and GONOSUMDB are Go environment variables that bypass the checksum database for private modules, which is how enterprises use Go modules without leaking internal module paths to Google’s infrastructure. Go’s vendor/modules.txt (generated by go mod vendor) lists vendored packages and their module versions, and the Go toolchain verifies it matches go.mod. If your repo has a vendor/ directory and go.mod specifies Go 1.14+, vendoring is automatically enabled without any flag, which surprises people who have a stale vendor directory they forgot about.

.yarn/cache/ and .pnp.cjs make up Yarn Berry’s zero-install setup: compressed zip archives of every dependency and the Plug’n’Play loader mapping package names to zip locations, both committed to version control. After git clone, the project works without running yarn install, though your repository size will grow substantially.

.terraform.lock.hcl records Terraform provider version locks with platform-specific hashes, which means a lock file generated on macOS may fail verification on Linux CI unless you’ve run terraform providers lock for multiple platforms.

Hooks and scripts

Lifecycle scripts that run during install, build, or publish. Supply chain attacks often hide here, but so does a lot of useful automation.

.pnpmfile.cjs isn’t just for overrides. pnpm’s hooks API includes readPackage for rewriting manifests, afterAllResolved for modifying the resolved lockfile, and custom fetchers for alternative package fetching logic.

.yarn/plugins/ contains committed plugin files that hook into Yarn Berry’s lifecycle. .yarn/sdks/ holds editor integration files generated by @yarnpkg/sdks to make PnP work with IDEs.

.mvn/extensions.xml loads Maven extensions that hook into the build lifecycle before anything else runs. Gradle’s init scripts in ~/.gradle/init.d/ execute before every build and can inject repositories, apply plugins, or configure all projects. Cargo’s build.rs is a build script that runs before compilation, generating code, linking native libraries, or setting cfg flags. Go’s //go:generate directives in source files run via go generate for code generation, though they’re not part of the build itself.

I’ll keep updating this post as I find more. If you know of package manager magic files I’ve missed or have corrections, reach out on Mastodon or submit a pull request on GitHub.

When an attacker compromises a maintainer’s credentials or takes over a dormant package, they publish a malicious version and wait for automated tooling to pull it into thousands of projects before anyone notices. William Woodruff made the case for dependency cooldowns in November 2025, then followed up with a redux a month later: don’t install a package version until it’s been on the registry for some minimum period, giving the community and security vendors time to flag problems before your build pulls them in. Of the ten supply chain attacks he examined, eight had windows of opportunity under a week, so even a modest cooldown of seven days would have blocked most of them from reaching end users.

The concept goes by different names depending on the tool (cooldown, minimumReleaseAge, stabilityDays, exclude-newer) and implementations vary in whether they use rolling durations or absolute timestamps, whether they cover transitive dependencies or just direct ones, and whether security updates are exempt. But the adoption over the past year has been remarkably fast.

JavaScript

The JavaScript ecosystem moved on this faster than anyone else, with pnpm shipping minimumReleaseAge in version 10.16 in September 2025, covering both direct and transitive dependencies with a minimumReleaseAgeExclude list for packages you trust enough to skip. Yarn shipped npmMinimalAgeGate in version 4.10.0 the same month (also in minutes, with npmPreapprovedPackages for exemptions), then Bun added minimumReleaseAge in version 1.3 in October 2025 via bunfig.toml. npm took longer but shipped min-release-age in version 11.10.0 in February 2026. Deno has --minimum-dependency-age for deno update and deno outdated. Five package managers in six months, which I can’t think of a precedent for in terms of coordinated feature adoption across competing tools.

Python

uv has had --exclude-newer for absolute timestamps since early on and added relative duration support (e.g. 1 week, 30 days) in version 0.9.17 in December 2025, along with per-package overrides via exclude-newer-package. pip shipped --uploaded-prior-to in version 26.0 in January 2026, though it only accepts absolute timestamps and there’s an open issue about adding relative duration support.

Ruby

Bundler and RubyGems have no native cooldown support, but gem.coop, a community-run gem server, launched a cooldowns beta that enforces a 48-hour delay on newly published gems served from a separate endpoint. Pushing the cooldown to the index level rather than the client is interesting because any Bundler user pointed at the gem.coop endpoint gets cooldowns without changing their tooling or workflow at all.

Rust, Go, PHP, .NET

Cargo has an RFC in progress and the registry-side infrastructure for cooldowns is stabilized in Cargo 1.94 (releasing March 5, 2026). Their approach sidesteps the exemption list problem entirely: instead of exempting packages from cooldowns, you explicitly opt in to a new version with cargo update foo --precise 1.5.10, which records the choice in your lockfile. No exclude list to remember to clean up later. In the meantime there’s also cargo-cooldown, a third-party wrapper that enforces a configurable cooldown window on developer machines as a proof-of-concept. Go has an open proposal for go get and go mod tidy, Composer has two open issues, and NuGet has an open issue though .NET projects using Dependabot already get cooldowns on the update bot side since Dependabot expanded NuGet support in July 2025.

Dependency update tools

Renovate has had minimumReleaseAge (originally called stabilityDays) for years, long before the rest of the ecosystem caught on, adding a “pending” status check to update branches until the configured time has passed. Mend Renovate 42 went a step further and made a 3-day minimum release age the default for npm packages in their “best practices” config via the security:minimumReleaseAgeNpm preset, making cooldowns opt-out rather than opt-in for their users. Dependabot shipped cooldowns in July 2025 with a cooldown block in dependabot.yml supporting default-days and per-semver-level overrides (semver-major-days, semver-minor-days, semver-patch-days), with security updates bypassing the cooldown. Snyk takes the most aggressive stance with a built-in non-configurable 21-day cooldown on automatic upgrade PRs. npm-check-updates added a --cooldown parameter that accepts duration suffixes like 7d or 12h.

Checking your config

zizmor added a dependabot-cooldown audit rule in version 1.15.0 that flags Dependabot configs missing cooldown settings or with insufficient cooldown periods (default threshold: 7 days), with auto-fix support. StepSecurity offers a GitHub PR check that fails PRs introducing npm packages released within a configurable cooldown period. OpenRewrite has an AddDependabotCooldown recipe for automatically adding cooldown sections to Dependabot config files. For GitHub Actions specifically, pinact added a --min-age flag, and prek (a Rust reimplementation of pre-commit) added --cooldown-days.

Still waiting

For Go, Bundler, Composer, and pip, cooldown support is still in discussion or only partially landed, which means you’re relying on Dependabot or Renovate to enforce the delay. That covers automated updates, but nothing stops someone from running bundle update or go get locally and pulling in a version that’s been on the registry for ten minutes. I couldn’t find any cooldown discussion at all for Maven, Gradle, Swift Package Manager, Dart’s pub, or Elixir’s Hex, if you know of one, let me know and I’ll update this post.

The feature also goes by at least ten different configuration names across the tools that do support it (cooldown, minimumReleaseAge, min-release-age, npmMinimalAgeGate, exclude-newer, stabilityDays, uploaded-prior-to, min-age, cooldown-days, minimum-dependency-age), which makes writing about it almost as hard as configuring it across a polyglot project.

Language vs. system package managers

On npm, PyPI, and RubyGems, running npm publish or gem push makes a package installable worldwide in seconds, and if Dependabot or Renovate happens to run in that window, the malicious code lands in a project without a human ever seeing it. All of the supply chain attacks William examined exploit this property, where publishing and distribution are the same act and nothing stands between a compromised maintainer account and thousands of downstream projects.

System package managers work differently because they separate those two things. When someone pushes a new version of an upstream library, it doesn’t appear in apt install or brew install until a distribution maintainer has reviewed the change, updated the package definition, and pushed it through a build pipeline. Fedora packages go through review and koji builds, Homebrew requires a pull request that passes CI and gets merged by a maintainer. A compromised upstream tarball still has to survive that process before it reaches anyone’s machine, and the people doing the reviews tend to notice when a patch adds an obfuscated postinstall script that curls a remote payload.

Debian goes further. Even if a maintainer account is compromised, uploads land in unstable first, then automigrate to testing after 2 to 10 days depending on urgency and availability of package tests. Stable only gets updates through a separate release process. That’s effectively a built-in cooldown with human review at multiple stages.

Cooldowns on the language package manager side are trying to retrofit something like that review window onto ecosystems that never had one, giving security researchers a few days to flag a malicious publish before automated tooling pulls it into lockfiles. Asking Homebrew or apt to add the same feature would mean delaying security patches through a process that already has human gatekeepers, which costs more than it saves.

The timestamp problem

pip’s --uploaded-prior-to and npm’s older --before flag both take absolute timestamps, and the discussion about adding relative duration support to pip reveals how these two modes serve different goals that happen to share implementation surface. An absolute timestamp pins your dependency resolution to a moment in time, so running the same install six months from now produces the same result, which is a reproducibility feature. A relative duration like 7 days creates a sliding window that moves forward with you, so you always exclude recently published packages regardless of when you run the build, which is a security feature. uv’s --exclude-newer accepts both forms, and npm has both --before for absolute dates and min-release-age for relative durations. pnpm, Yarn, Bun, and Deno only accept relative durations.

The pip thread also gets into the surprisingly fiddly business of parsing duration strings. ISO 8601 durations (P7D) are unambiguous but nobody wants to type them, human-readable strings like 7 days are friendly but need a parser that pip’s maintainers would rather not write and maintain, and variable-length calendar units like months and years require knowing which month you’re in to convert to a concrete number of days. uv went with ISO 8601 plus friendly strings but excluded months and years entirely, and pip’s maintainers are leaning toward just accepting a bare number of days, which covers nearly every real use case without dragging in leap year arithmetic.

Even the question of what “seven days ago” means gets complicated when your CI server is in UTC, your developer laptop is in US Pacific time, and the registry timestamp uses whatever timezone PyPI’s servers happen to be configured with. A few hours of timezone drift can determine whether a package published six days and twenty-two hours ago passes the cooldown check or not.

Registries

When you run gem install rails, the client needs to know where to look. RubyGems defaults to rubygems.org, pip to pypi.org, npm to registry.npmjs.org, and that default is just a URL baked into the client configuration. You can change it, which is exactly what makes dependency confusion possible: if your client checks a public registry before a private one and the names overlap, an attacker who registers the right name on the public registry wins.

Companies run private registries with different names for the same packages, or the same names for different packages. Nix, Guix, and Spack layer multiple package repositories with their own namespaces on top of each other. Go uses URL-based module paths where the registry name is literally embedded in the package identity. Which registry you’re talking to determines what every other name in the system means, because a registry name is really a lookup context: give it a package name and it hands back a list of versions.

Namespaces

Some registries insert another naming layer between the registry and the package. Packagist requires vendor prefixes (symfony/console), Maven requires reverse-domain group IDs (org.apache.commons:commons-lang3), and npm has optional scopes (@babel/core) that most of the ecosystem’s biggest packages never adopted because they predate the feature. RubyGems and PyPI have flat namespaces where the package name is all there is. Even the separator characters differ: @scope/name on npm, vendor/package on Packagist, group:artifact on Maven, and Cargo’s proposed namespaces use :: because / was already taken by the feature syntax.

A namespace is really a claim of authority over a family of names, which makes questions like who gets to publish under @google/ or who owns the serde namespace in Cargo’s proposed serde::derive scheme into governance problems dressed up as naming problems. They only get harder as registries grow. Zooko’s triangle says you can’t have names that are simultaneously human-readable, decentralized, and secure, and registries exist largely to hold two of those three together. I covered the different namespace models in more detail previously.

Package names

Once you’ve picked a registry and navigated any namespace, you arrive at a package name, and that name resolves to a list of available versions. requests, express, serde, rails. These need to be unique within their registry and namespace, memorable enough to type from recall, and stable enough that renaming doesn’t break everything downstream. Name scarcity in flat registries is why you get python-dateutil because dateutil was taken. PyPI normalizes hyphens, underscores, dots, and case so my-package, my_package, My.Package, and MY_PACKAGE all resolve to the same thing, a decision that prevents some squatting but means four different-looking strings in requirements files can point at the same package. npm used to allow uppercase package names and then banned them, so legacy packages like JSONStream still exist with capital letters that no new package can use. The package called node on npm isn’t Node.js.

Sometimes projects bake a major version into the package name itself, like boto3 or webpack5, effectively creating a new package that has its own version history on top of the version number already embedded in its name. boto3 version 1.34.0 is a different thing from a hypothetical boto4 version 1.0.0, even though the underlying project is the same.

Typosquatting exploits the gap between what you meant to type and what the registry resolved; slopsquatting exploits LLM hallucinations of package names that don’t exist yet but could be registered by an attacker. The registry will resolve whatever string you give it, no questions asked.

Versions

Pick a version from that list and you get a particular snapshot of code, along with its metadata: a list of dependencies, a list of builds, and whatever the maintainer wrote in the changelog. Versions look like numbers but they’re really strings, which becomes obvious as soon as you see 1.0.0-beta.2+build.456 or Python’s 1.0a1.post2.dev3 or the dozens of versioning schemes people have invented over the years. Prerelease tags, build metadata, epoch prefixes, calver date segments all get bolted onto the version string to carry meaning that a simple three-number tuple can’t express, and every ecosystem parses and sorts these strings differently. Debian prepends an epoch (2:1.0.0) so that a repackaged version sorts higher than the original even if the version number is lower. Ruby uses .pre.1 where npm uses -pre.1. Is 1.0.0 the same as v1.0.0? Depends who you ask. 1.2.3 is supposed to communicate something about compatibility relative to 1.2.2 and 2.0.0, but that communication happens entirely through convention around the name, with no mechanism to enforce it. Elm is the rare exception, where the registry diffs APIs and rejects publishes that break compatibility without a major bump.

When a maintainer account is compromised, publishing 1.2.4 with malicious code looks indistinguishable from a routine patch release, because the version name carries no provenance. And when a version gets yanked or deleted, lockfiles that pinned to that exact name suddenly point at nothing.

Dependencies and requirements

Each version carries a list of dependencies, and each dependency is itself a pair of names: a package name and a version constraint. requests >= 2.28 means “the package named requests, at a version whose name satisfies >= 2.28”. So you’re back at the package name layer, looking up another name, getting another list of versions, and the resolver walks this graph recursively trying to find a consistent set of version names that satisfies all the constraints simultaneously. When two packages name the same dependency with incompatible constraints, the resolver has to either find a way through or prove that no path exists.

The same “convention not enforcement” problem from versioning carries over here. The version constraints are a small language for describing sets of version names, and every ecosystem invented its own. ~> 2.0 in Ruby, ^2.0 in npm, >=2.0,<3.0 in Python all use different syntax with subtly different semantics, especially once you hit edge cases around 0.x versions. A broad constraint like >=1.0 names a large and growing set of versions; a pinned ==1.2.3 names exactly one. The choice of constraint syntax determines how much of the version namespace a single declaration covers, and there’s no cross-ecosystem agreement on what the symbols mean.

Some dependencies are themselves hidden behind yet another name. pip has extras (requests[security]), Cargo has features (serde/derive), and Bundler has groups (:development, :test), all of which are named sets of additional dependencies that only activate when someone asks for them by name. pip install requests and pip install requests[security] install different dependency trees from the same package, selected by a string in square brackets that the package author chose.

These constraint languages also compose with the namespace layer. npm’s @types/node@^18.0.0 combines a scope, a package name, and a version constraint into a single expression, while Maven’s org.apache.commons:commons-lang3:3.12.0 encodes group, artifact, and version as three colon-separated names that only make sense when parsed together.

Builds and platforms

Once the resolver has settled on a version, the client needs to pick the right build artifact, and that means matching platform names. Unlike the earlier naming layers, which are mostly human-coordination problems, platform identity is inherently fuzzy: an M1 Mac running Rosetta is simultaneously two platforms depending on who’s asking, and manylinux is a compatibility fiction that keeps getting revised as the definition shifts underneath it. PyPI wheels look like numpy-1.24.0-cp311-cp311-manylinux_2_17_x86_64.whl, packing the package name, version, Python version, ABI tag, and platform into a single filename. RubyGems appends a platform suffix to get nokogiri-1.15.4-x86_64-linux-gnu.gem, and Conda encodes the channel, platform, and build hash.

If the platform name on the artifact doesn’t match the platform name the client computes for its own environment, the package won’t install, or the wrong binary gets selected silently. And as I wrote about in platform strings, the same M1 Mac is aarch64-apple-darwin to LLVM, arm64-darwin to RubyGems, darwin/arm64 to Go, and macosx_11_0_arm64 to Python wheels, so every tool that works across ecosystems ends up maintaining a translation table between naming schemes that each made sense in their original context.

Source repositories

The naming doesn’t stop at the registry. Most packages point back to a source repository, and that’s another stack of names: the host (github.com), the owner or organization (rails), the repository name (rails), branches (main, 7-1-stable), tags (v7.1.3), and commits (a SHA that’s finally content-addressed rather than human-chosen). Go and Swift skip the registry layer entirely and use these repository URLs as the package identity, which means the naming conventions of GitHub or whatever host you’re on become part of your dependency graph directly. Monorepos add another wrinkle: Babel’s source lives at babel/babel on GitHub but publishes dozens of packages under @babel/*, so the mapping from repo name to package name is one-to-many.

Version tags in git are particularly interesting because they’re the bridge between two naming systems. A maintainer creates a git tag called v1.2.3, and the registry or build tool maps that to a version name in its own scheme. But there’s no standard for whether the tag should be v1.2.3 or 1.2.3 or release-1.2.3, so tooling has to guess or be configured. And when an organization renames on GitHub, or a project moves from one owner to another, every downstream reference to the old owner/repo pair breaks unless the host maintains redirects, which GitHub does until someone registers the old name, at which point you have the repo-jacking problem.

Naming and trust

At each of these layers you’re trusting that a name resolves to what you think it does, that the registry URL points to the right service, that the package name belongs to who you think it does, that a version was published legitimately, that a constraint won’t pull in something unexpected, that a platform-tagged binary was built from the same source as the one for your colleague’s machine. That trust is transitive, flowing through your dependencies’ names and their dependencies’ names in a chain where nobody has full visibility. The registry is the authority that makes most of these names meaningful, which is why the question of who governs registries keeps coming back to the surface.

The explosion of open source was built on this kind of transitive trust between maintainers. A package with 800 transitive dependencies works because each maintainer along the way did a reasonable job of choosing and maintaining their own dependencies, and the maintainers they depended on did the same. Nobody designed this trust network or audited it as a whole. It just grew as people built on each other’s work, and it has held up well enough that we’ve come to take it for granted, even as bad actors have started to map its weak points.

We have decent tools now for scanning our own dependency trees. You can run npm audit or Dependabot or Snyk against your lockfile and get a report on known vulnerabilities. But when you do that, you’re trusting that the maintainer of each package in your tree is doing the same: running audits, reviewing what they pull in, dropping dependencies whose maintainers have gone quiet, keeping their build tooling current. And you’re trusting that those maintainers are trusting their own dependencies’ maintainers to do the same, all the way down through a chain of people who mostly don’t know each other and have no visibility into each other’s practices.

Every package you install was also built, tested, and published using dependencies you never see: a JavaScript library’s devDependencies, the build tools that compiled a Rust crate before it was uploaded, the pytest plugins that ran during CI, the GitHub Action that handled publishing. You’re trusting that the maintainer chose those carefully, keeps them updated, and drops them when they go stale, and that the maintainers of those tools are doing the same. A maintainer who never runs npm audit, who has a three-year-old GitHub Action in their publish workflow, who accepted a PR from a stranger adding a new build dependency without much scrutiny, produces an artifact on the registry that looks identical to one from a maintainer who checks everything meticulously.

The event-stream incident is the classic example: the original maintainer handed the project to someone new, that person added a malicious dependency, and nobody upstream noticed. The xz backdoor was more patient and more frightening. A co-maintainer spent two years making legitimate contributions before planting obfuscated code in the build system and test fixtures, targeting a part of the toolchain that almost nobody reads. And then there’s the codecov bash uploader compromise, which didn’t target a library at all but a CI tool that thousands of projects were curling into their build pipelines. I suspect most maintainers who used it never read the script once.

Trusted publishing is an effort to close part of this gap. PyPI, npm, and RubyGems now support publishing flows where packages are built and uploaded directly from CI using short-lived credentials tied to a specific repository and workflow, which creates a verifiable link between the source and the published artifact. But it also means we’re now trusting that each maintainer’s CI configuration is sound, that the GitHub Actions in their workflow are maintained by people who are themselves doing due diligence, that the dev dependencies installed during the build are ones they’ve reviewed. GitHub Actions in particular has almost none of the supply chain protections that language package managers have spent years building, so in practice we’ve traded one unverifiable assumption for a different one.

Semver ranges compound this because npm update or bundle update or cargo update will pull in new versions across your entire tree in seconds, and you’re trusting that every maintainer in the chain shipped something good since the last time your lockfile was generated, including versions built against whatever state their toolchains were in at the time.

Large companies deal with this by vendoring and rebuilding everything from source in controlled environments, effectively verifying each level of the trust chain themselves instead of relying on each maintainer to have done it. But even vendoring just moves the boundary. Those controlled builds still run on compilers and operating systems and hardware that somebody else produced, and at some point you stop verifying and start trusting. The honest version of “we’ve audited our supply chain” is “we’ve audited our supply chain down to a depth we felt comfortable with and then stopped”.

A 2023 study of Maven found that 11.58% of dependency updates contain breaking changes that impact clients, and nearly half arrived in non-major version bumps. Most library maintainers have no way to validate their version number before publishing, so the feedback loop is reactive: release, wait for bug reports, and hope the breakage wasn’t too widespread before you can cut a patch.

Distributions

Debian packages declare test suites following the DEP-8 specification, and when a package is a candidate for migration from unstable to testing, the migration tool Britney triggers autopkgtest for the package and all of its reverse dependencies. A regression blocks migration, so an Expat update that causes test failures in its dependents sits in unstable until someone resolves them, and a Coq update that broke mathcomp-analysis and mathcomp-finmap did the same. The maintainer finds out who they broke and how before the change reaches anyone who didn’t opt into unstable.

Autopkgtest doesn’t check API compatibility. It runs actual test suites of actual consumers, which encode whatever implicit contracts those consumers have built against, including ones the upstream maintainer has never heard of. If library Y changes the sort order of a hash table in a patch release and package X’s tests assumed that order was stable, migration blocks until someone decides whose assumption was wrong.

Fedora’s recent work with tmt, Packit, and Testing Farm runs downstream tests in the PR, before anything is released. The Cockpit project configured it so that opening a PR on their core library automatically runs the test suites of cockpit-podman and other dependents against the proposed change, with results showing up as status checks before merge. As they put it, “it is too late at the distro level anyway: at that point the new upstream release which includes the regression was already done, and the culprit landed possibly weeks ago already.”

When a maintainer discovers breakage in a PR, they’re still inside the change. They remember why they restructured that error path, they know which tests they considered, and the diff is right in front of them. The cost of responding to a downstream failure at this point is a few minutes of thought and maybe a revised approach. When the same breakage surfaces as an issue filed three weeks after release, the maintainer has to reload the context of the change, understand the downstream project’s usage well enough to see why it broke, decide whether to fix forward or revert, cut a new release, and hope that consumers who already pinned away will unpin. The information is the same in both cases, a downstream test failed, but the cost of acting on it scales with the distance from the change that caused it.

Debian’s autopkgtest catches breakage before migration to testing, which is better than catching it after, but the change has already been released upstream by that point. The Fedora approach catches it before the upstream release happens at all, which means the maintainer can fix it before anyone outside their own CI ever encounters it. František Lachman and Cristian Le presented the PTE project at FOSDEM. Downstream feedback that arrives while you’re still writing the code changes how you think about the change itself.

Language ecosystems

Distributions can do this because they have structural properties that language ecosystems lack: a single canonical dependency graph, a standardized test interface (DEP-8 in Debian’s case), a shared execution environment where every package builds and runs the same way, and the authority to block a release based on downstream results. npm, PyPI, and RubyGems have fragmented tooling, no standard way to invoke a package’s tests from outside its own repo, heterogeneous execution environments, and no mechanism to gate a publish on anything other than the maintainer’s own judgement. A few language ecosystems have built partial versions of downstream testing anyway, though they tend to belong to compiler teams with the resources to work around these gaps.

Rust’s crater compiles and tests every crate on crates.io against both the current and proposed compiler, then diffs the results. A recent PR adding impl From<f16> for f32 to the standard library broke 3,143 crates out of 650,587 tested. Adding a trait implementation is unambiguously backwards-compatible by semver’s rules, but it broke type inference in thousands of downstream projects because existing code depended on there being exactly one conversion path between those types. Crater caught it before it shipped, during a run that took five to six days across Linux x86_64. Without it, the Rust team would have discovered the breakage from 3,143 individual bug reports.

Crater also benefits from Rust being compiled: a type inference failure shows up at build time, before any tests run. In Python, Ruby, or JavaScript, the equivalent breakage only surfaces at runtime, so you need downstream test suites that actually exercise the affected code paths, and those code paths need to be covered in the first place. The case for downstream testing is stronger in dynamic ecosystems because there’s no compile step to catch the easy ones, and the signal is harder to get.

Node.js runs CITGM (Canary in the Goldmine), which tests about 80 curated npm packages against proposed Node versions. A refactor in Node 12 moved isFile from Stats.prototype to StatsBase.prototype, changing nothing about the public API but breaking the esm module because it walked the prototype chain directly. In a separate release, a change to the timing of a readable event on EOF broke the dicer module, which depended on that event firing synchronously.

All of these were built by teams with dedicated infrastructure budgets and release processes, and an individual library maintainer who publishes a widely-used package on npm or PyPI or RubyGems has nothing comparable, even though they face the same problem at a different scale.

Merge confidence

Renovate’s Merge Confidence aggregates data from millions of update PRs to tell consumers whether an update is safe: how old the release is, what percentage of Renovate users have adopted it, and what percentage of updates result in passing tests. The signal comes from real test results across real projects, but it arrives after the release and flows to consumers, never back to the maintainer who shipped the change.

The algorithm is private, and the underlying dataset of which updates broke which projects’ tests stays behind Mend’s paywall. Dependabot shows a compatibility score on security update PRs, calculated from CI results across other public repos that made the same update, but only when at least five candidate updates exist, and the data doesn’t flow back to the maintainer either. I’ve started indexing Dependabot PRs at dependabot.ecosyste.ms to build an open version of this signal. It doesn’t have CI data yet, but it already tracks merge percentages per update, which gives a rough proxy for how much trouble a particular version bump is causing across the ecosystem.

Discovery

Registries track which packages declare dependencies on other packages, but applications that consume libraries are mostly invisible: a Rails app that depends on a gem won’t show up in RubyGems’ reverse dependency list, and a company’s internal service using an npm package won’t appear on npmjs.com. The maintainer’s view of their dependents is limited to whatever the registry can see, which skews heavily toward other libraries and misses the applications, which are where the stranger usage patterns and more surprising implicit contracts show up.

ecosyste.ms tracks dependents across both packages and open source repositories, scanning millions of repos on GitHub, GitLab, and other forges for manifest files that declare dependencies. A maintainer can see which applications actually use their library, which is the view you’d need to build a downstream testing system on.

Building it

This is something I want to build on top of ecosyste.ms. A maintainer connects the service to their CI, and on every PR or pre-release branch it queries ecosyste.ms for the top N dependents of the package, both libraries and applications, ranked by some combination of dependent count, download volume, and recency of commits. It clones each one, installs the proposed version of the library in place of the current release, and runs their test suite in an isolated environment. The results come back as a report on the PR: which dependents were tested, which ones regressed, what the stack traces look like, which of the maintainer’s changes likely caused each failure.

A maintainer looking at that report before tagging a release would see things that are currently invisible to them. They’d see that popular applications parse their error messages with regex and will break if the wording changes, that a widely-used wrapper library calls a method they considered internal and were about to remove, that their optimisation to batch database calls changed the callback order in a way that two downstream projects’ integration tests depend on.

Michal Gorny’s catalogue of problems with downstream testing Python packages lays out the failure modes: test suites that modify installed files assuming they’re in a disposable container, pytest plugins in the environment causing unexpected test collection, tests requiring network access or Docker, timing-dependent assertions, floating-point precision differences across architectures, source distributions that omit test files entirely. Any service trying this across a registry would need to handle all of these gracefully, distinguishing genuine regressions from environmental noise, which is a hard problem that Debian has spent years refining with autopkgtest and still hasn’t fully solved.

Developer tools usually fund themselves by selling an enterprise version, but large companies facing similar coordination problems between internal teams already solved them with monorepos. When all your code lives in one tree, downstream testing is just CI: you run every affected test before merging, no special infrastructure needed. Google, Meta, and Microsoft have invested heavily in making that work, and inside their monorepos the problem is already solved. Nobody’s going to buy an enterprise version of downstream testing when their codebase doesn’t have a “downstream,” which leaves open source maintainers as the only audience for a tool like this, and they can’t fund it.

ecosyste.ms already provides the dependent discovery, source repositories are linked from package metadata, test suites follow ecosystem conventions that are well-understood enough to automate, and container infrastructure makes isolated environments cheap. Crater and autopkgtest have proven the approach works at ecosystem scale. The missing piece is stitching these together into something an individual maintainer can point at their package and get results from, without needing a compiler team’s budget or a distro’s infrastructure.

Dear Data Subject,

Thank you for your request under Article 15 of the General Data Protection Regulation (EU) 2016/679 to access all personal data we hold about you.

We apologize for the delay in responding. Your request was initially routed to our dependency resolution system, which spent 47 days attempting to resolve your identity against our user registry before entering a circular reference with GitHub’s SSO provider. A human has since intervened.

1. Categories of Personal Data Processed

• Identity data: Name, email address, username, GitHub handle, two-factor authentication status, and 487 unique IP addresses recorded since account creation.
• Package data: Full publishing history for buttplug (147 versions) and 1 package published at 2:47 AM containing your .env file. You un-published it within four minutes, by which time 14 users had installed it.
• Behavioral data: Every npm install you have ever run, including timestamps and resolved dependency trees. Every npm audit you have run (4 times) and every npm audit you chose not to run (approximately 11,200 times), all of which we log.
• node_modules inventory: Resolved dependency trees, install manifests, and content hashes collected from your local environment during package installation. This constitutes the largest category at 412 pages (see Appendix J).

2. Purposes of Processing

• Service provision: To deliver packages to your machine.
• Dependency graph construction: To build and maintain a complete graph of every package’s relationship to every other package, and by extension, every developer’s relationship to every other developer, though we have not yet determined a use for it.
• Security: To detect anomalous publishing behavior. Our system flagged your 2:47 AM publish as anomalous.
• Legitimate interest: We have a legitimate interest in understanding the full topology of the JavaScript ecosystem. We acknowledge this interest is difficult to distinguish from surveillance.

3. Recipients of Personal Data

• GitHub, Inc.: Our parent company. They hold your data under a separate privacy policy. You will need to submit a separate DSAR to them. They will redirect you to Microsoft.
• GitHub Dependabot: Each of the 147 versions of buttplug you have published generated automated pull requests titled “Bump buttplug” across an estimated 1,247 downstream repositories.
• Microsoft Corporation: Our parent company’s parent company. Their response to your DSAR will be delivered via Microsoft Teams, which you will need to install.
• Cloudflare, Inc.: Our CDN provider. They have observed every package you have ever downloaded. They consider this metadata, not personal data.
• The npm public registry: Your published packages, including their package.json files, are publicly available. Your package.json from the 2:47 AM incident contained your home directory path and your OS username. We cannot un-publish this information, as at least one of the 14 downstream consumers has mirrored it to IPFS.
• GitHub Arctic Code Vault: Your published packages were frozen in February 2020 on archival film in a decommissioned coal mine in Svalbard, Norway.
• An unspecified number of CI/CD pipelines: Your packages are installed approximately 900 times per week in automated build environments. Each of these environments logs the installation. We do not control these logs, nor, as far as we can determine, does anyone else.
• An unknown number of software bills of materials: Under Executive Order 14028, federal software suppliers are required to produce SBOMs listing all components. Your package buttplug is listed as a transitive dependency in an estimated 340 SBOMs submitted as federal records to US government agencies.

4. Retention Periods

• Account data: For the lifetime of your account, plus 7 years after deletion, plus the remaining useful life of physical backup media.
• Package data: Indefinitely. npm’s contract with the ecosystem is that published packages are permanent. Un-publishing is technically possible but discouraged since 2016.
• Behavioral data: 24 months in our primary database, after which it is moved to cold storage, where it remains queryable.
• node_modules inventories: We do not have a retention policy for this data because we did not realize we were collecting it.

5. Your Rights

• Right of access: You are exercising this right now.
• Right to rectification: You may request correction of inaccurate data. If you would like us to update the OS username in the leaked package.json, please note that this would require modifying a published package, which would break the integrity hash, which would cause npm audit to flag it as tampered, which would generate security advisories for the 14 downstream consumers, one of whom has mirrored it to a public Git repository. We advise against rectification at this time.
• Right to erasure: You may request deletion of your personal data where there is no compelling reason for its continued processing. We believe there is a compelling reason: buttplug has 1,247 direct dependents, including 3 production banking applications. Deleting your account would remove it from the registry, breaking its dependents, their dependents, and so on until an estimated 0.003% of the JavaScript ecosystem fails to build. Our legal team considers this a compelling reason.
• Right to data portability: You may request your data in a structured, commonly used, machine-readable format. We have prepared your data as a 2.7 GB JSON file, available for download at a pre-signed URL that expires in 7 days.
• Right to object: You may object to processing based on legitimate interest. If you object to our construction of the global dependency graph, your objection will be noted in the graph.

6. Automated Decision-Making

• Trust score: Our system has assigned you a trust score of 72 out of 100, based on account age, publishing frequency, two-factor authentication status, and whether you have ever mass-transferred package ownership to a stranger. The platform average is 64. The scoring methodology is proprietary.
• Bus factor assessment: Our system has determined that buttplug has a bus factor of 1: You are driving the bus. This assessment has been shared with downstream maintainers who have opted into critical dependency notifications.

7. International Transfers

• United States: Where our servers are located. This transfer is covered by the EU-US Data Privacy Framework, which replaced Privacy Shield, which replaced Safe Harbor.
• 47 additional countries: Your published packages are distributed via a global CDN. We cannot enumerate which edge nodes have cached your package.json at any given time. The full list of jurisdictions is included in Appendix K.

If you have questions about this response, please contact our Data Protection Officer at dpo@npmjs.com. Please allow 30 days for a reply. If our response requires querying the dependency graph, please allow 47 additional days.

Yours faithfully,

Data Protection Officer
npm, Inc.
A subsidiary of GitHub, Inc.
A subsidiary of Microsoft Corporation

Enclosures:
Appendix A: Account metadata (3 pages)
Appendix B: Publishing history including retracted packages (7 pages)
Appendix C: Behavioral telemetry (41 pages)
Appendix D: Dependency graph, your packages only (28 pages)
Appendix E: Dependency graph for buttplug, including transitive dependents (119 pages)
Appendix F: npm audit output (84 pages)
Appendix G: Download logs (31 pages)
Appendix H: IP address history with geolocation (6 pages)
Appendix J: node_modules inventory, deduplicated (412 pages)
Appendix K: List of jurisdictions (2 pages)

Total enclosures: 743 pages
Format: JSON

Matter.js handles the physics and Rough.js gives it the hand-drawn xkcd look. Each reload generates a different tower from a seeded PRNG that picks a taper profile, varies the block sizes and row widths, and drifts the whole thing slightly off-center as it goes up. The project names are randomly assembled from parts that sound like real packages – things like node-flux.js or libcrypt-fast or hyper-mux@3.12.7 – though about one in five times you’ll get an actual name like left-pad or log4j instead. Reload enough times and you might run into some unusual tower shapes, and the Konami code does what you’d hope.

The info button shows the tower’s seed, which you can share as a ?seed= URL parameter, basically a way to say “look at this disaster” and have someone else see the exact same precarious arrangement.

Some ways this could go further:

• Upload an SBOM and build the tower from your actual dependency tree, with block sizes based on how many other packages depend on each one
• Pull real dependency data from ecosyste.ms so you can see what your project’s tower looks like before you start pulling blocks out
• Use the phone’s accelerometer to let you tilt and topple the tower

Source on GitHub.

homebrew-core has one Ruby file per package formula, and every brew update used to clone or fetch the whole repository until it got large enough that GitHub explicitly asked them to stop. Homebrew 4.0 switched to downloading a JSON file over HTTP, because users wanted the current state of a package rather than its commit history. But updating a formula still means opening a pull request against homebrew-core, because git is where the collaboration tooling lives. Instead of using git as a database, what if you used a database as a git?

A git repository is a content-addressable object store where objects go in indexed by the SHA1 of their content, plus a set of named references pointing at specific objects by hash. The on-disk format (loose objects as individual files, packfiles as delta-compressed archives with a separate index, a ref store split between a directory of files and a packed-refs flat file with a locking protocol that breaks on NFS) is an implementation detail. The protocol for synchronising objects and refs between repositories is what actually matters, and since git-the-program is just one implementation of it, you can swap the storage backend without clients noticing.

The whole data model fits in two tables:

CREATE TABLE objects (
    repo_id  integer NOT NULL,
    oid      bytea NOT NULL,
    type     smallint NOT NULL,
    size     integer NOT NULL,
    content  bytea NOT NULL,
    PRIMARY KEY (repo_id, oid)
);

CREATE TABLE refs (
    repo_id  integer NOT NULL,
    name     text NOT NULL,
    oid      bytea,
    symbolic text,
    PRIMARY KEY (repo_id, name)
);

An object’s OID is computed the same way git does it, SHA1("<type> <size>\0<content>"), using pgcrypto’s digest() function, and refs get compare-and-swap updates through SELECT FOR UPDATE. A libgit2 backend registers these tables as its storage layer, and if the protocol really is separable from the format, a normal git client should be able to push to and clone from a Postgres database without knowing the difference.

To test this I built gitgres, about 2,000 lines of C implementing the libgit2 git_odb_backend and git_refdb_backend interfaces against Postgres through libpq, plus roughly 200 lines of PL/pgSQL for the storage functions. libgit2 handles pack negotiation, delta resolution, ref advertisement, and the transport protocol while the backend reads and writes against the two tables, and a git remote helper (git-remote-gitgres) lets you add a Postgres-backed remote to any repo and push or clone with a normal git client that has no idea it’s talking to a database. There’s a Dockerfile in the repo if you want to try it out without building libgit2 and libpq from source.

The objects table contains the same bytes git would store on disk, and a set of SQL functions parse them into tree entries, commit metadata, and parent links that you can join against like any other table.

SELECT r.name AS repo, c.author_name, c.authored_at, i.title AS issue
FROM commits c
JOIN repositories r ON r.id = c.repo_id
JOIN issues i ON i.repo_id = c.repo_id
  AND c.message ILIKE '%#' || i.index || '%'
WHERE c.authored_at > now() - interval '30 days';

That query joins git commit data against Forgejo’s issue tracker, something that currently requires fetching commits through git log, pattern-matching issue references in application code, and then querying the database for the matching issues. With both sides in Postgres it’s one query.

Forgejo

A self-hosted Forgejo or Gitea instance is really two systems bolted together: a web application backed by Postgres, and a collection of bare git repositories on the filesystem. Anything that needs to show git data in the web UI has to shell out to the binary and parse text, which is why something as straightforward as a blame view requires spawning a subprocess rather than running a query. If the git data lived in the same Postgres instance as everything else, that boundary disappears.

Forgejo stores issues, pull requests, users, permissions, webhooks, branch protection rules, and CI status in Postgres already, and git repositories are the one thing left on the filesystem, forcing every deployment to coordinate backups between them, and the two systems scale and fail in different ways. The codebase already shows the strain: Forgejo mirrors branch metadata from git into its own database tables (models/git/branch.go) so it can query branches without shelling out to git every time.

All git interaction goes through modules/git, about 15,000 lines of Go that shells out to the git binary and parses text output. With git data in Postgres, reading an object becomes SELECT content FROM objects WHERE oid = $1 on the database connection Forgejo already holds, and walking commit history is a query against a materialized view rather than spawning git log.

The deployment collapses to a single Postgres instance where pg_dump backs up forge metadata, git objects, and user data together, and replicas handle read scaling for the web UI without NFS mounts or a Gitaly-style RPC layer. The path there is a Forgejo fork replacing modules/git with a package that queries Postgres, where Repository holds a database connection and repo_id instead of a filesystem path and Commit, Tree, Blob become thin wrappers around query results.

Postgres

Postgres has its own primitives for things that forges currently build custom infrastructure around. A trigger on the refs table firing NOTIFY means any connected client learns about a push the moment it happens, which is how forges normally end up building a custom webhook polling layer. Multi-tenant repo isolation becomes a database concern through row-level security on the objects and refs tables, and logical replication lets you selectively stream repositories across Postgres instances, a kind of partial mirroring that filesystem-based git can’t do. Commit graph traversal for ancestry queries and merge-base computation falls to recursive CTEs, and pg_trgm indexes on blob content give you substring search across all repositories without standing up a separate search index.

Diff, merge, blame

Content-level diffs, three-way merge, and blame stay in libgit2 rather than being reimplemented in SQL, since libgit2 already has that support and works against the Postgres backends through cgo bindings. The Forgejo fork would be “replace modules/git with libgit2 backed by Postgres” rather than “replace modules/git with raw SQL,” because the read-side queries only cover the simple cases and anything involving content comparison or graph algorithms still needs libgit2 doing the work with Postgres as its storage layer. That’s a meaningful dependency to carry, though libgit2 is well-maintained and already used in production by the Rust ecosystem and various GUI clients. SQL implementations of some of this using recursive CTEs would be interesting to try eventually but aren’t needed to get a working forge. The remaining missing piece is the server-side pack protocol: the remote helper covers the client side, but a Forgejo integration also needs a server that speaks upload-pack and receive-pack against Postgres, either through libgit2’s transport layer or a Go implementation that queries the objects table directly.

Storage

Git packfiles use delta compression, storing only the diff when a 10MB file changes by one line, while the objects table stores each version in full. A file modified 100 times takes about 1GB in Postgres versus maybe 50MB in a packfile. Postgres does TOAST and compress large values, but that’s compressing individual objects in isolation, not delta-compressing across versions the way packfiles do, so the storage overhead is real. A delta-compression layer that periodically repacks objects within Postgres, or offloads large blobs to S3 the way LFS does, is a natural next step. For most repositories it still won’t matter since the median repo is small and disk is cheap, and GitHub’s Spokes system made a similar trade-off years ago, storing three full uncompressed copies of every repository across data centres because redundancy and operational simplicity beat storage efficiency even at hundreds of exabytes.

gitgres is a neat hack right now, but if open source hosting keeps moving toward federation and decentralization, with ForgeFed, Forgejo’s federation work, and more people running small instances for their communities, the operational simplicity of a single-Postgres deployment matters more than raw storage efficiency. Getting from a handful of large forges to a lot of small ones probably depends on a forge you can stand up with docker compose up and back up with pg_dump, and that’s a lot easier when there’s no filesystem of bare repos to manage alongside the database.

npm and PyPI have both shipped build provenance attestations using Sigstore over the past couple of years. When you publish a package from GitHub Actions with trusted publishing configured, the CI environment signs an in-toto attestation binding the artifact to the source repository, commit, and workflow that built it, and the signature goes into a public transparency log that anyone downstream can verify without trusting the registry. PyPI has had this on by default for trusted publishers since late 2024, npm generates provenance automatically, and the cost to publishers is close to zero. I wrote about how this fits into the broader reproducible builds picture recently.

Meanwhile the EU Cyber Resilience Act, which grew out of product safety regulation originally written for things like toasters, introduced “open source stewards” as a legal concept, and Article 25 gives the European Commission power to create voluntary security attestation programmes for them. At FOSDEM this year, Æva Black presented work with the Eclipse Foundation on what such a programme might look like. The proposed model has manufacturers funding stewards who issue attestations about the projects they support, with a tiered approach where the light tier asks whether a project has functional tests, a vulnerability reporting contact, and an end-of-life policy. Æva noted a maintainer could fill it out in minutes. So this is a checklist about project hygiene, filled out by a human, attesting to things like whether a CONTRIBUTING.md exists, which has almost nothing in common with a cryptographic proof logged in a transparency ledger except that both are called attestations.

Madalin Neag at OpenSSF wrote an excellent piece in January working through the details of how steward attestations relate to the projects they cover, since stewards don’t control technical decisions or releases, and a point-in-time attestation may not reflect the state of a component by the time a manufacturer integrates it. These are the kind of design questions that need working out as the delegated act takes shape.

This isn’t the first time naming has caused confusion at the boundary between open source and compliance. The CRA itself calls anyone who places software on the EU market a “manufacturer,” which is product safety language from the world of toasters and power tools. Daniel Stenberg got a taste of what that framing produces when a company sent him a compliance questionnaire demanding he account for Log4j in curl, treating him as a vendor with SLA obligations for a project that has never used Java.

Both SPDX and CycloneDX have a “supplier” field for each component, and SBOM generators routinely fill it with the maintainer’s name, even though the maintainer has no contractual relationship with the consumer and is not a supplier in any commercial sense. These words carry legal connotations that don’t match the relationships they’re describing, and now that they’re codified in standards and regulation they’re difficult to undo.

What I keep thinking about is the maintainer who enables trusted publishing, whose CI generates Sigstore provenance on every release, and who then gets contacted by a foundation about a CRA attestation programme asking them to fill out a form about whether they have a security policy. The cryptographic attestation infrastructure already exists and already generates machine-verifiable supply chain metadata at scale, and continuous signals like CSAF advisories and VEX documents provide ongoing security posture rather than point-in-time snapshots. The Article 25 delegated act hasn’t been written yet and the Commission is still taking input. It would be nice if the two communities compared notes before then, if only so that maintainers don’t end up navigating two unrelated things with the same name.

Naming is hard, but it matters more than usual when the names carry assumptions about what’s actually in place. “Attested” sounds rigorous whether or not it is, and “supplier” implies a contractual relationship that doesn’t exist. Once these words are in standards and regulations, people downstream build processes around what they think the words mean, and unpicking those assumptions later is much harder than getting the names right in the first place. Toaster regulations at least have the advantage that everyone agrees on what a toaster is.

The Reproducible Builds project has been working on this since 2013, when Lunar (Jérémy Bobbio) organized a session at DebConf13 and began patching Debian’s build tooling. The Snowden disclosures had made software trust an urgent concern, Bitcoin’s Gitian builder had shown the approach was viable for a single project, and the Tor Project had begun producing deterministic builds of Tor Browser. Lunar wanted to apply the same thinking to an entire operating system.

The first mass rebuild of Debian packages in September 2013 found that 24% were reproducible, and by January 2014, after fixing the lowest-hanging fruit in dpkg and common build helpers, that jumped to 67%. Today Debian’s testing infrastructure shows around 96% of packages in trixie building reproducibly under controlled conditions, while reproduce.debian.net runs a stricter test by rebuilding the actual binaries that ftp.debian.org distributes rather than clean-room test builds.

The project grew into a cross-distribution effort as Arch Linux, NixOS, GNU Guix, FreeBSD, and others joined over the following years. Summits have been held most years since 2015, most recently in Vienna in October 2025. Chris Lamb, who served as Debian Project Leader from 2017 to 2019, co-authored an IEEE Software paper on the project that won Best Paper for 2022. Lunar passed away in November 2024. The project’s weekly reports, published continuously since 2015, give a sense of the scale of work involved: each one lists patches sent to individual upstream packages fixing timestamps, file ordering, path embedding, locale sensitivity, one package at a time, hundreds of packages a year. Getting from 24% to 96% was not a single architectural fix but a decade of this kind of janitorial patching across the entire Debian archive.

How verification works

You build the same source twice in different environments and compare the output, and if the bytes match, nobody tampered with the artifact between source and distribution. In practice this requires recording everything about the build environment, which Debian does with .buildinfo files capturing exact versions of all build dependencies, architecture, and build flags. A verifier retrieves the source, reconstructs the environment using tools like debrebuild, builds the package, and compares SHA256 hashes against the official binary.

When hashes don’t match, diffoscope is how you find out why. Originally written by Lunar as debbindiff, it recursively unpacks archives, decompiles binaries, and shows you exactly where two builds diverge across hundreds of file formats: ZIP, tar, ELF, PE, Mach-O, PDF, SQLite, Java class files, Android APKs. Feed it two JARs that should be identical and it’ll dig through the archive, into individual class files, into the bytecode, and show you that one has a timestamp from Tuesday and the other from Wednesday.

The project also maintains strip-nondeterminism for removing non-deterministic metadata from archives after the fact, and reprotest, which builds packages under deliberately varied conditions (different timezones, user IDs, locales, hostnames, file ordering) to flush out hidden assumptions.

What makes builds non-reproducible

Benedetti et al. tested 4,000 packages from each of six ecosystems using reprotest for their ICSE 2025 paper “An Empirical Study on Reproducible Packaging in Open-Source Ecosystems”, varying time, timezone, locale, file ordering, umask, and kernel version between builds. Cargo and npm scored 100% reproducible out of the box because both package managers hard-code fixed values in archive metadata, eliminating nondeterminism at the tooling level. PyPI managed 12.2%, limited to packages using the flit or hatch build backends which fix archive metadata the same way. Maven came in at 2.1%, and RubyGems at 0%.

The dominant cause across all three failing ecosystems was timestamps embedded in the package archive, responsible for 97.1% of RubyGems failures, 92.4% of Maven failures, and 87.7% of PyPI failures. The standard fix is SOURCE_DATE_EPOCH, an environment variable defined by the Reproducible Builds project in 2015, containing a Unix timestamp that build tools should use instead of the current time. GCC, Clang, CMake, Sphinx, man-db, dpkg, and many other tools now honour it, but it’s opt-in, so any build tool that doesn’t check the variable just uses the current time.

Most of this turned out to be fixable with infrastructure changes rather than per-package work. Simply configuring SOURCE_DATE_EPOCH brought Maven from 2.1% to 92.6% and RubyGems from 0% to 97.1%, and small patches to the package manager tools addressing umask handling, file ordering, and locale issues pushed PyPI to 98% and RubyGems to 99.9%. The packages that remained unreproducible were ones running arbitrary code during the build, like setup.py scripts calling os.path.expanduser or gemspecs using Time.now in version strings, which no amount of tooling can fix because the nondeterminism is in the package author’s code.

File ordering causes similar problems because readdir() returns entries in filesystem-dependent order (hash-based on ext4, lexicographic on APFS, insertion order on tmpfs) and tar and zip tools faithfully preserve whatever order they’re given. The project built disorderfs, a FUSE filesystem overlay that deliberately shuffles directory entries to expose ordering bugs during testing. Absolute paths get embedded in compiler debug info and source location macros, so a binary built in /home/alice/project differs from one built in /home/bob/project. Archive metadata carries UIDs, GIDs, and permissions. Locale differences change output encoding. Parallel builds produce output in nondeterministic order, and any single unfixed source is enough to make the whole build non-reproducible.

Go

Since Go 1.21 in August 2023, the toolchain produces bit-for-bit identical output regardless of the host OS, architecture, or build time, after Russ Cox’s team eliminated ten distinct sources of nondeterminism including map iteration order, embedded source paths, file metadata in archives, and ARM floating-point mode defaults.

Go runs nightly verification at go.dev/rebuild using gorebuild, and Andrew Ayer has independently verified over 2,672 Go toolchain builds with every one matching. The Go Checksum Database at sum.golang.org adds a transparency log so that even if a module author modifies a published version, the ecosystem detects it. Anything that calls into C via cgo reintroduces the host C toolchain as a build input and all the nondeterminism that comes with it, but pure Go code is genuinely reproducible across platforms and over time.

Maven

Maven’s official guide documents the steps: set project.build.outputTimestamp in pom.xml, upgrade all plugins to versions that respect it, verify with mvn clean verify artifact:compare. Maven 4.0.0-beta-5 enables reproducible mode by default, and Reproducible Central maintains a list of independently verified releases.

The timestamp only works if every plugin in the chain respects it, though, and many third-party plugins don’t. Different JDK versions produce different bytecode, ZIP entry ordering varies by implementation, and Maven builds are assembled from dozens of plugins that each introduce their own potential nondeterminism. Researchers built Chains-Rebuild to canonicalize six root causes of Java build unreproducibility, which gives a sense of how many separate things can go wrong in a single build system.

Cargo

Rust’s RFC 3127 introduced trim-paths, which remaps absolute filesystem paths out of compiled binaries and is now the default in release builds, replacing paths like /home/alice/.cargo/registry/src/crates.io-abc123/serde-1.0.200/src/lib.rs with serde-1.0.200/src/lib.rs. Embedded paths were the most common source of non-reproducibility in Rust binaries, and the cargo-repro tool lets you rebuild and compare crates byte-for-byte to check for remaining issues.

Procedural macros and build scripts (build.rs) remain a gap since they can do anything at build time: read environment variables, call system tools, generate code based on the hostname. The cc crate, used to compile bundled C code, reintroduces the same C-toolchain nondeterminism that cgo does for Go.

PyPI

The Benedetti et al. study found only 12.2% of PyPI packages reproducible out of the box, and the split came down to build backend: packages using flit or hatch were reproducible because those backends fix archive metadata the way Cargo and npm do, while packages using setuptools (still the majority) were not. With patches to address umask handling and archive metadata the number reached 98%, with the remaining 2% coming from packages running arbitrary code in setup.py or pyproject.toml build hooks.

PyPI has also moved further than most registries on attestations through PEP 740, shipped in October 2024, which adds support for Sigstore-signed digital attestations uploaded alongside packages. These link each artifact to the OIDC identity that produced it, so combined with trusted publishing, PyPI can record that a package was built in a specific CI workflow from a specific commit with a cryptographic signature binding artifact to source.

RubyGems

RubyGems 3.6.7 made the gem building process more reproducible by default, setting a default SOURCE_DATE_EPOCH value and sorting metadata in gemspecs so that building the same gem twice produces the same .gem file without special configuration. Individual gems can still have their own nondeterminism, native extensions like nokogiri compile against host system libraries with all the usual C-toolchain variation, and there’s no independent rebuild verification infrastructure for RubyGems.

npm

The npm registry accepts arbitrary tarballs with no connection to source, no build provenance, and no way to independently rebuild a package and compare it against what’s published. package-lock.json and npm ci give you dependency pinning and integrity hashes that confirm the tarball hasn’t changed since publication, but that says nothing about whether it matches any particular source commit.

Homebrew

Homebrew distributes prebuilt binaries called bottles, built on GitHub Actions and hosted as GitHub release artifacts. The project has a reproducible builds page documenting the mechanisms available to formula authors: SOURCE_DATE_EPOCH is set automatically during builds, build paths are replaced with placeholders like @@HOMEBREW_PREFIX@@ during bottle creation, and helpers like Utils::Gzip.compress produce deterministic gzip output. There’s no systematic testing of what percentage of bottles actually rebuild identically, though.

Since Homebrew 4.3.0 in May 2024, every bottle comes with a Sigstore-backed attestation linking it to the specific GitHub Actions workflow that built it, meeting SLSA Build Level 2 requirements. Users can verify attestations by setting HOMEBREW_VERIFY_ATTESTATIONS=1, though verification isn’t yet the default because it currently depends on the gh CLI and GitHub authentication while the project waits on sigstore-ruby to mature.

Trusted publishing

Traditionally a maintainer authenticates with an API token, builds on their laptop, and uploads. Trusted publishing replaces that with OIDC tokens from CI so that the registry knows the package was built by a specific GitHub Actions workflow in a specific repository, not just uploaded by someone who had the right credentials.

PyPI launched trusted publishing in April 2023, built by Trail of Bits and funded by Google’s Open Source Security Team. RubyGems.org followed in December 2023, npm shipped provenance attestations via Sigstore in 2023 and full trusted publishing in July 2025, crates.io launched in July 2025, and NuGet followed in September 2025. Over 25% of PyPI uploads now use it.

Once provenance tells you that a package was built from commit abc123 of github.com/foo/bar in a specific workflow, anyone can check out that commit and attempt to rebuild, and if the build is reproducible the rebuilt artifact should match the published one. Most of these trusted publishing flows run on GitHub Actions, though, which itself has serious problems as a dependency system: no lockfile, no integrity verification, and mutable tags that can change between runs, meaning the build infrastructure that’s supposed to provide provenance guarantees doesn’t have great provenance properties of its own.

Google’s OSS Rebuild

OSS Rebuild, announced by Google’s Open Source Security Team in July 2025, takes a pragmatic approach to the fact that most builds aren’t bit-for-bit reproducible yet by rebuilding packages from source and performing semantic comparison, normalizing known instabilities like timestamps and file ordering before checking whether the meaningful content matches.

At launch it covered thousands of packages across PyPI, npm, and crates.io, using automation and heuristics to infer build definitions from published metadata, rebuilding in containers, and publishing SLSA Level 3 provenance attestations signed via Sigstore. The stabilize CLI tool handles the normalization by stripping timestamps, reordering archive entries, and removing owner metadata from ZIPs, tars, and wheels. Maven Central, Go modules, and container base images are on the roadmap.

Matthew Suozzo’s FOSDEM 2026 talk pushed beyond pure reproducibility into build observability, adding a network proxy for detecting hidden remote dependencies and eBPF-based build tracing to answer not just whether a build can be reproduced but what the build is actually doing at runtime, which is useful independently of whether the output happens to be deterministic.

Where things stand

Language package managers are years behind Linux distributions on reproducible builds because Debian controls its build infrastructure and can mandate changes to that environment, while language registries accept uploads from anywhere and historically had no way to know how an artifact was produced. Trusted publishing is shifting that by moving builds from laptops into CI where the registry has visibility into the process, and combined with build provenance and SLSA attestations, this creates conditions where independent verification becomes possible even when the build tooling itself hasn’t caught up. Go got there by making the compiler deterministic, which is the cleanest solution but requires controlling the entire toolchain from the start.