Release notes for IRRd 4.1.0¶
IRRd 4.1.0 adds several major new features, including RPKI-aware mode, a new daemon architecture with full multiprocessing, synthetic NRTM, and a scope filter.
The changes between 4.0.x and 4.1.0 are major. Please read these release notes carefully before upgrading. Upgrading to IRRd 4.1.0 requires several changes to the deployment setup. Downgrading back to IRRd 4.0.x, if needed, also requires steps before downgrading the installed IRRd package.
Contents
New features¶
RPKI-aware mode¶
RPKI-aware mode is now available, where IRRd
imports RPKI ROAs and can filter or reject RPSL objects that are
in conflict with ROAs. Pseudo-IRR objects are generated for all ROAs.
When RPKI-aware mode is enabled, RPKI
becomes an invalid as a regular
IRR source name, as it is reserved for pseudo-IRR objects from ROAs.
RPKI-aware mode also affects mirroring.
IRRd 4.1.0 includes several database migrations to support RPKI-aware mode, whether enabled or disabled, and facilitate performance improvements needed for RPKI-aware mode. Running these migrations is required to run IRRd 4.1.0, even if RPKI-aware mode is never enabled.
Danger
The impact of running IRRd in RPKI-aware mode can be dramatic, and it is strongly recommended to read the RPKI integration documentation very carefully before enabling it, to understand all consequences. By default, RPKI-aware mode is enabled. RPKI-aware mode can be disabled entirely, or certain sources can be excluded from RPKI validation.
New daemon architecture¶
IRRd has a new daemon architecture, where all whois queries, HTTP requests, mirror processes and the preloader run in their own process. This improves performance significantly, especially where many processes are running on servers with many cores. Previously, the entire IRRd process was limited to one CPU core.
To communicate between processes, IRRd now requires a running Redis instance.
The commands to start IRRd and several IRRd scripts have also changed.
The --uid
parameter is no longer supported.
Synthetic NRTM¶
The irrd_load_database
command allowed loading RPSL data from local files,
often used to load data generated by scripts or other systems. Data imported
this way could not be mirrored over NRTM.
In IRRd 4.1, the irrd_update_database
command has been added. This
supports periodically updating a source to the state in a particular file,
and automatically generates journal entries for any differences, allowing
NRTM mirroring. See the mirroring documentation
for further details.
Other changes¶
A scopefilter has been added. This allows you to filter RPSL objects matching certain prefixes and AS numbers from your IRRd instance. By default, the scope filter is disabled.
The default for
server.whois.max_connections
has been reduced from 50 to 10. In 4.1, IRRd whois workers use considerably more memory, about 200 MB each, and one worker is started for each permitted connection. Therefore, at the default 10 connections, the whois processes use about 2 GB of memory, at 50 connections, this is about 10 GB.The
last-modified
attribute is set every time an object is created or updated in an authoritative source. You can apply this to all existing authoritative objects with the new irrd_set_last_modified_auth command.Serial handling in IRRd has changed for NRTM mirrors. If you mirror a source over NRTM, and keep a local journal, IRRd used to keep these serials identical. The NRTM ADD from the original source would be stored in the local journal under the same serial number, unless it was ignored by the object class filter. In 4.1, if you enable RPKI-aware mode or the scope filter for a source, or it has been enabled at any point since the last full reload, IRRd keeps its own serial number range in the local journal for that source. This may be out of sync with the NRTM source. Different IRRd instances mirroring from the same NRTM source may have different serial numbers for the same change. If neither RPKI-aware mode nor the scope filter is enabled, and hasn’t been since the last full reload, IRRd synchronises the local serials with the NRTM source, the default behaviour in 4.0. See the :ref`NRTM serial handling documentation <mirroring-nrtm-serials>` for further details.
When users create route(6) objects in authoritative databases, IRRd also verifies authorisation from related object maintainers. This behaviour is enabled by default, but can be disabled with the
auth.authenticate_related_mntners
setting.The
!j
command has changed, and now is exclusively used to check mirroring status. It returns what the most recent serial processed from a mirror is. For more extensive status information, like the local serials in the journal, use the new !J command.IRRd starts a maximum of three mirror processes at the same time, to reduce peak loads. A further three, if needed, are started 15 seconds later, regardless of whether the previous ones have finished.
HTTP(s) downloads are now supported for the
sources.{name}.import_source
andsources.{name}.import_serial_source
settings.A number of new configuration options were added, and some are required. See the configuration documentation for more information on these options.
RIPE style query responses now always end with two empty lines, consistent with the RIPE database.
A custom flexible logging config can now be set with the
log.logging_config_path
.A timeout was added for FTP connections.
Memory usage during large RPSL imports has been reduced.
A bug was fixed where some invalid objects could cause parser exceptions.
Steps required to upgrade¶
The following steps are required to upgrade to IRRd 4.1.0, regardless of whether RPKI-aware mode is enabled or not.
Disable all cron and e-mail triggered tasks. There should be no calls to any IRRd scripts during the upgrade process.
Upgrade the IRRd package from within the virtualenv with
pip install -U irrd
Install a Redis instance as documented in the deployment guide and configure the
redis_url
setting.Note that unix sockets are strongly recommended over TCP sockets for both PostgreSQL and Redis, for improved performance. The effect of this is more significant with the new multi-process daemon architecture.
Set
piddir
to a directory where IRRd can write its PID file,irrd.pid
.Run the database migrations, using the same command used to create the tables initially in deployment. Important note: some of the migrations change large amounts of data, and may take up to 15-45 minutes to run in total. While the migrations are running, IRRd should be shut down and any cron / e-mail triggered tasks must be disabled. There must be no calls to
irrd_submit_email
orirrd_load_database
.Update any startup scripts or systemd for IRRd to call the new daemon process, with the new command line arguments, and use
setcap
to allow IRRd to bind to privileged ports: see the updated deployment guide.Remove the
--irrd_pidfile
parameter from calls toirrd_submit_email
andirrd_load_database
.Ensure that RPKI-aware mode is configured as desired. By default it is enabled.
Start IRRd and re-enable the cron / e-mail triggered tasks.
If you would like to set
last-modified
for existing authoritative objects, use the new irrd_set_last_modified_auth command.
Downgrading from 4.1 to 4.0.x¶
If you are running IRRd 4.1, and would like to downgrade back to 4.0.x, the database schema needs to be modified. You can either restore an older copy of your database, start with a fresh database, or use the database migrations.
If you want to use the database migrations, run this command before downgrading your local package installation to 4.0.x:
/home/irrd/irrd-venv/bin/irrd_database_downgrade --version 28dc1cd85bdc
If you would like to re-upgrade to 4.1 later on, you will need to run
irrd_database_upgrade
again, as noted in the steps above.
The downgrade migration typically takes a few seconds.