Release notes for IRRd 4.3.0

Note

4.3.0 was initially released on June 6th, 2023, but was yanked due to a packaging error on June 9th, leading to the release of IRRd 4.3.0.post1. The main visible issue from the error was that the documentation of 4.4 was published as 4.3.0. If you had previously installed 4.3.0 through PyPI/pip, you can keep running on that version, or update to 4.3.0.post1 without needing restarts or migrations, as the 4.3.0 and 4.3.0post1 PyPI packages have identical code.

IRRd 4.3’s major new features are:

  • Granular options for stricter authentication checks for related objects, such as requiring an ASN prefix in a set name and requiring authentication for the prefixed ASN.

  • Route object preference, where route objects from one source may suppress overlapping route objects from a lower priority source. Similar to RPKI and scope filter, suppression applies to mirrored sources as well.

  • An event stream based on WebSocket, where clients can subscribe to receive updates of all committed changes to IRR objects for all sources, in a structured format. Also includes an initial download endpoint in JSONL format.

  • bcrypt hashes for maintainer passwords, and the option to mark hashers as legacy or disable them entirely.

  • Maintainer suspension, where a mntner along with any objects it maintains can be suspended. This is similar to deletion, except that the objects can be reactivated later.

  • A replacement for irr_rpsl_submit from IRRd v3 to ease migrations. This is an independent Python script that uses IRRd’s HTTP API, not depending on other IRRd code, to make it easy to drop in.

In version 4.2, IRRd required newly created authoritative as-set objects to have a hierarchical name where the first element is an AS number. In 4.3, this feature has been significantly expanded.

For all RPSL set objects, IRRd can now be configured to require:

  • Including an ASN prefix in the name of the set, e.g. AS65537:AS-EXAMPLE being valid, but AS-EXAMPLE being invalid.

  • Passing authentication for the corresponding aut-num, e.g. AS65537 in the example, skipping this check if the aut-num does not exist.

  • Passing authentication for the corresponding aut-num, e.g. AS65537 in the example, failing this check if the aut-num does not exist.

The first two options, requiring a prefix with opportunistic aut-num authentication, is now the default for all set objects. You can change the configuration for specific RPSL set objects, or set your own common configuration that applies to all sets.

The compatibility.permit_non_hierarchical_as_set_name setting has been removed, as it is now covered by the prefix_required setting. The auth.authenticate_related_mntners setting has been renamed to auth.authenticate_parents_route_creation, as this setting exclusively relates to authentication for route(6) objects and needs to be distinct from the configuration for RPSL set objects.

If you were using auth.authenticate_related_mntners or compatibility.permit_non_hierarchical_as_set_name, you need to update your configuration.

All checks are only applied when users create new set objects in authoritative databases. Authoritative updates to existing objects, deletions, or objects from mirrors are never affected. When looking for related objects for authentication, IRRd only looks in the same IRR source.

By default, IRRd 4.3 requires an ASN prefix and uses opportunistic authentication.

Route object preference

IRRd can now use route object preference to suppress overlapping route(6) objects from different sources, based on a configured priority for each source.

By default, this feature is not enabled for any sources.

Event stream

IRRd now offers a WebSocket-based event stream. All changes to IRR objects are published on this endpoint. There is also a new initial download endpoint that publishes all IRR data, with optional filtering, in JSONL format. Both initial downloads and ongoing changes include all data parsed from objects by IRRd, instead of only the raw object text. This makes this format easier to work with and integrate with other tooling.

In their current design, these features are not optimised for high volume use. Access is controlled by the new server.http.event_stream_access_list setting. By default all access is denied.

New password hasher and new password settings

IRRd 4 has always supported CRYPT-PW and MD5-PW hashing for passwords in mntner objects. This version adds support for BCRYPT-PW, using bcrypt, a secure and modern password hashing method.

IRRd 4.3 has a new setting auth.password_hashers to configure which password hashes are enabled. The default is stricter than in older versions: bcrypt-pw and md5-pw are fully enabled, crypt-pw is only enabled in legacy mode. The legacy mode means that IRRd will accept authentication through CRYPT-PW on mntners as before, but will not accept auth lines using CRYPT-PW on new or modified mntner objects in authoritative databases. This requires users to upgrade to a stronger hash when they next update their mntner object.

Maintainer suspension

IRRd now supports suspending a maintainer and all objects it maintains. This feature is only accessible to admins using the override password, and only when enabled on an authoritative source with the suspension_enabled setting.

Suspended objects act as if they have been deleted, but can be restored by an admin at a later time.

By default, this feature is not enabled for any sources.

Replacement for irr_rpsl_submit

IRRd now includes an irr_rpsl_submit script, similar to the submit tool from IRRD v3. The main purpose of this script is to provide (limited) compatibility with existing integrations that called irr_rpsl_submit directly to submit to older IRRd versions. It is a light wrapper that calls the IRRd HTTP API.

Note that this command is not intended to be used for handling incoming email changes - see the deployment guide for the irrd_submit_email instead. This is unchanged from previous versions of IRRd.

This code was contributed by MERIT for use with RADb.

New journal expiry command

IRRD includes a new command: irrd_expire_journal. You can call this to remove journal entries older than a given date, in both authoritative and mirrored sources. Do use caution, as the journal is the only place where IRRd keeps a change history. Journal entries can not be recovered after being expired.

Minimum Python version

The minimum Python version for IRRd is now 3.7. Python 3.6 is end of life and therefore no longer supported. In turn, IRRd 4.3.x will be the last minor version to support Python 3.7.

Other dependency versions

IRRd now requires Redis 5 or newer. PostgreSQL 11 or newer is strongly recommended before upgrading, as it makes database migrations significantly faster.

Other changes compared to 4.2.8

  • The sources.{name}.nrtm_query_serial_range_limit setting was added to limit the load of NRTM queries.

  • IRRd processes will now log a traceback of all their threads when receiving a SIGUSR1 signal. This can be helpful when debugging hanging workers or other complex issues.

  • When configured to drop privileges after starting, IRRd will now check whether the less privileged user is able to write to the log file, before dropping the privileges. Previously, it would drop privileges, then fail to write to the log file, and be unable to report this error.

  • Inserting the journal entries is now much faster when inserting multiple entries per source. This happens in NRTM mirroring or when users submit large sets of changes. IRRd will process these changes faster up to an order of magnitude.

  • IRRd applies stricter checks to the range operator in route-set members in strict validation mode. Values are now checked for appropriate prefix length, rejecting values like 192.0.2.0/26^24-32.

  • The compatibility.irrd42_migration_in_progress setting was removed as it only applied to 4.2 upgrades.

Upgrading to IRRd 4.3.0 from 4.2.x

As with any other major IRRd release, depending on the availability needs of your setup, it is recommended to first test the upgrade and operation on a testing/acceptance setup with similar data size.

The recommended steps to upgrade are:

  • Make sure your Python (minimum 3.7) and Redis (minimum 5) versions are recent enough.

  • Update your configuration, as several configuration options have been removed, as noted above.

  • Disable all cron and e-mail triggered tasks. There should be no calls to any IRRd scripts during the upgrade process.

  • Stop the IRRd process.

  • Upgrade the IRRd package from within the virtualenv with pip install irrd==4.3.0.post1

  • Run the database migrations, using the irrd_database_upgrade command. Important note: some of the migrations change large amounts of data, and may take up to 10-45 minutes to run in total. While the migrations are running, IRRd should not be running and any cron / e-mail triggered tasks must be disabled. There must be no calls to irrd_submit_email, irrd_load_database, or any other scripts.

  • Restart IRRd.

  • Re-enable the cron / e-mail triggered tasks.

  • To support the new event stream, update your nginx or other HTTP service configuration to enable WebSocket connections. See the new nginx example. Note that the recommended gzip_types was also expanded.

The database migrations only add columns, so there is an option to keep your IRRd daemon running while running the database migrations. This is more complex, but will reduce downtime. Load and locks will hold back database changes and may cause delayed responses to queries. Delayed responses may persist for an hour or so after the upgrade.

If you prefer this option, before starting the process, set import_timer for all sources and rpki.roa_import_timer to several hours. This reduces writes and possible deadlocks. Then, follow the process as before, but instead of stopping and starting, only restart IRRd once, after the database migration.

Downgrading from 4.3 to 4.2.x

If you are running IRRd 4.3, and would like to downgrade back to 4.2.x, the database schema needs to be modified. You can either restore an older copy of your database, start with a fresh database, or use the database migrations.

If you want to use the database migrations, run this command before downgrading your local package installation to 4.2.x:

irrd_database_downgrade --version 8744b4b906bb

If you would like to re-upgrade to 4.3 later on, you will need to run the database migrations again, as listed in the upgrade steps.