Release notes for IRRd 4.2.0

IRRd 4.2.0 adds several major features:

  • A new GraphQL query interface. This new interface supports much greater query flexibility than traditional whois, with output in a structured JSON format, over a secure HTTPS connection.

  • A new HTTPS API call for whois queries. This supports the same whois queries as port 43, but using standard HTTPS instead of the more complex raw TCP sockets.

  • A new HTTPS API call for submitting RPSL objects. This allows submitting one or multiple objects, and returns a JSON response with the result of the requested changes.

  • Prefix searches can have up to a 100x performance improvement in several circumstances.

The new HTTP interfaces are intended to be used with HTTPS, to provide secure and authenticated transport. The deployment guide has been updated with an nginx example.

PyPy is now an officially supported deployment platform. It is an alternative to CPython, the most commonly used Python interpreter. PyPy offers improved performance in the order of 10% for some queries, and even higher for some GraphQL queries. CPython remains fully supported, and may be the easier option depending on your deployment platform. IRRd 4.2 was continuously tested against CPython 3.6 and 3.9, and PyPy 3.6 and 3.7.

Other changes compared to 4.1.7

  • IRRd now rejects unknown settings in the config file, i.e. settings that are found in the file, but unknown to IRRd. This reduces unintentional configuration errors.

  • The optional geofeed attribute was added for inetnum and inet6num objects, which can contain a URL which in turn contains geographical information, as defined in draft-ymbk-opsawg-finding-geofeeds-03.

  • Retrieval of files over FTP now follows the ftp_proxy environment variable, if set.

  • New as-set objects created in authoritative databases are required to have a hierarchical name, like AS65540:AS-CUSTOMERS. For example, AS-CUSTOMERS is no longer allowed. This new behaviour can be disabled with the compatibility.permit_non_hierarchical_as_set_name setting.

  • IRRd now supports dropping privileges after binding to the whois port, by setting the user and group settings. This allows you to start IRRd as root, have it bind to port 43, and then drop privileges. This is now the recommended deployment method.

  • The periodic tasks to update the RPKI and scope filter status use considerably less memory while running.

Changes needed to upgrade

IRRd 4.2 from 4.1 is a smaller upgrade than 4.1 from 4.0. If you are upgrading from a 4.0.x version, please make sure to also follow all steps in the 4.1 release notes.

If you are upgrading from an earlier version than 4.1.7, make sure to read the release notes for any intermediate releases as well.

Tip

Depending on the availability requirements of your IRRd deployment, it may be smart to upgrade your own test setup first and verify whether 4.2 works as expected for you. That also ensures you are familiar with the exact upgrade steps in your deployment platform.

Several configuration options have been added or changed:

  • The server.http.access_list setting is now called server.http.status_access_list. The access list only applies to the status page - not the entire HTTP interface. The setting was renamed to reflect this change.

  • The default port and interface for the HTTP interface is now 127.0.0.1:8000. This was changed because the HTTP interface should be run behind an HTTPS proxy.

  • A new setting server.http.workers was added, set to 4 by default. This is the number of HTTP worker processes started. Note that like whois, each HTTP worker process uses up to 200MB of memory. Depending on available resources, a higher number of workers may be appropriate.

  • A new setting server.http.forwarded_allowed_ips was added. This needs to be set when the HTTP proxy in front of IRRd is not running on the local host.

  • The new setting compatibility.inetnum_search_disabled allows IRRd to default to high performance prefix queries. You should enable this setting if you are certain your instance does not process inetnum objects.

As IRRd now rejects unknown settings, it is possible that there are other settings in your configuration file that were always unknown, but will now be rejected by IRRd 4.2.

Other than the settings changes, IRRd 4.2 includes database migrations. There are two upgrade paths for this:

  • A simpler path where IRRd is shut down during all migrations. These take about 15-45 minutes, during which IRRd will not be available.

  • A more complex path where IRRd is only restarted once. This limits the downtime, but consists of more steps.

In either path, performance may be reduced up to half an hour after all database migrations complete, due to PostgreSQL analysing and vacuuming modified tables.

Simpler upgrade

  • Update your settings file as noted above.

  • Disable all cron and e-mail triggered tasks. There should be no calls to any IRRd scripts during the upgrade process.

  • Shut down your IRRd instance.

  • Upgrade the IRRd package from within the virtualenv with pip install irrd==4.2.0

  • Run the database migrations, using the irrd_database_upgrade command. Important note: some of the migrations change large amounts of data, and may take up to 15-45 minutes to run in total. While the migrations are running, IRRd should remain shut down and any cron / e-mail triggered tasks must be disabled. There must be no calls to irrd_submit_email or irrd_load_database.

  • Start IRRd and re-enable the cron / e-mail triggered tasks.

Low downtime upgrade

  • Update your settings file as noted above.

  • Disable all cron and e-mail triggered tasks.

  • Upgrade the IRRd package from within the virtualenv with pip install irrd==4.2.0

  • In your settings file, set compatibility.irrd42_migration_in_progress to true.

  • Run only the first database migration, using the irrd_database_upgrade command as: irrd_database_upgrade --version f4c837d8258c. This should take a few seconds.

  • Shut down your IRRd instance (the running 4.1 instance) and start the 4.2 instance.

  • Re-enable the cron / e-mail triggered tasks.

  • Run the remaining migrations by running irrd_database_upgrade without a version parameter. These may take up to 15-45 minutes to run. Performance may be affected.

  • After all migrations are complete, remove the compatibility.irrd42_migration_in_progress setting.

  • Send a SIGHUP to the running IRRd process.

HTTP service

With either path, you should also install nginx or a similar service to serve as an HTTPS proxy. An nginx example is included in the deployment guide. As no current services depend on this, you can do this after completing all other upgrade steps.

Downgrading from 4.2 to 4.1.x

If you are running IRRd 4.2, and would like to downgrade back to 4.1.x, the database schema needs to be modified. You can either restore an older copy of your database, start with a fresh database, or use the database migrations.

If you want to use the database migrations, run this command before downgrading your local package installation to 4.1.x:

irrd_database_downgrade --version a7766c144d61

If you would like to re-upgrade to 4.2 later on, you will need to run the database migrations again, as listed in the upgrade steps.