e2etest — Framework for writing integration tests for aioxmpp

This subpackage provides utilities for writing end-to-end or intgeration tests for aioxmpp components.

Warning

For now, the API of this subpackage is classified as internal. Please do not test your external components using this API, as it is experimental and subject to change.

Overview

The basic concept is that tests are written like normal unittests. However, tests are written by inheriting classes from aioxmpp.e2etest.TestCase instead of unittest.TestCase. e2etest.TestCase has the provisioner attribute which provides access to a provision.Provisioner instance.

Provisioners are objects which provide a way to obtain a connected XMPP client. The JID to which the client is bound is unspecified; however, each client gets a unique bare JID and the clients are able to communicate with each other. In addition, provisioners provide information about the environment in which the clients act. This includes providing JIDs of entities implementing specific protocols or features. The details are explained in the documentation of the Provisioner base class.

By default, tests which are written with e2etest.TestCase are skipped when using the normal test runners. This is because the provisioners need to be configured; this is handled using a custom nosetests plugin which is not loaded by default (for good reasons). To run the tests, use (instead of the normal nosetests3 binary):

$ python3 -m aioxmpp.e2etest

The command line interface is identical to the one of nosetests3, except that additional options are provided to configure the plugin. In fact, aioxmpp.e2etest is simply a nose test runner with an additional plugin.

By default, the configuration is read from ./.local/e2etest.ini. For details on configuring the provisioners, see the developer guide.

Main API

Decorators for test methods

The following decorators can be used on test methods (including setUp and tearDown):

@aioxmpp.e2etest.require_feature(feature_var, argname=None, *, multiple=False)

Parameters:

• feature_var (str) – XEP-0030 feature var of the required feature
• argname (str or None) – Optional argument name to pass the FeatureInfo to
• multiple (bool) – If true, all peers are returned instead of a random one.

Before running the function, it is tested that the feature specified by feature_var is provided in the environment of the current provisioner. If it is not, unittest.SkipTest is raised to skip the test.

If the feature is available, the FeatureInfo instance is passed to the decorated function. If argname is None, the feature info is passed as additional positional argument. otherwise, it is passed as keyword argument using the argname.

If multiple is true, all peers supporting the given feature are passed in a set. Otherwise, only a random peer is returned.

This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the setUp method.

@aioxmpp.e2etest.skip_with_quirk(quirk)

Parameters:quirk (Quirks) – The quirk to skip on

If the provisioner indicates that the environment has the given quirk, the test is skipped.

This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the setUp method.

General decorators

@aioxmpp.e2etest.blocking

The decorated coroutine function is run using the run_until_complete() method of the current (at the time of call) event loop.

The decorated function behaves like a normal function and is not a coroutine function.

This decorator must be applied to a coroutine function (or method).

@aioxmpp.e2etest.blocking_timed

Like blocking_with_timeout(), the decorated coroutine function is executed using asyncio.AbstractEventLoop.run_until_complete() with a timeout, but the timeout is configured in the end-to-end test configuration (see End-to-end tests (or integration tests)).

This is the recommended decorator for any test function or method, to prevent the tests from hanging when anythin goes wrong. The timeout is under control of the provisioner configuration, which means that it can be adapted to different setups (for example, running against an XMPP server in the internet will be slower than if it runs on localhost).

The decorated function behaves like a normal function and is not a coroutine function.

This decorator must be applied to a coroutine function (or method).

@aioxmpp.e2etest.blocking_with_timeout(timeout)

The decorated coroutine function is run using the run_until_complete() method of the current (at the time of call) event loop.

If the execution takes longer than timeout seconds, asyncio.TimeoutError is raised.

The decorated function behaves like a normal function and is not a coroutine function.

This decorator must be applied to a coroutine function (or method).

Class for test cases

class aioxmpp.e2etest.TestCase(methodName='runTest')

A subclass of unittest.TestCase for end-to-end test cases.

This subclass provides a single additional attribute:

provisioner

This is the configured provision.Provisioner instance.

If no provisioner is configured (for example because the e2etest nose plugin is not loaded), this reads as None.

Note

Under nosetests and the vanilla unittest runner, tests inheriting from TestCase are automatically skipped if provisioner is None.

Provisioners

class aioxmpp.e2etest.provision.Provisioner(logger=<logging.Logger object at 0x7fcf61a7deb8>)

Base class for provisioners.

Provisioners are responsible for providing test cases with XMPP accounts and client objects connected to these accounts, as well as information about the environment the accounts live in.

A provisioner must implement the following methods:

coroutine _make_client(logger)

Parameters:logger – The logger to pass to the client. Returns:Client with a fresh account.

Construct a new aioxmpp.PresenceManagedClient connected to a new account. This method must be re-implemented by subclasses.

configure(section)

Read the configuration and set up the provisioner.

Parameters:section – mapping of config keys to values

Subclasses will implement this to configure their account setup and servers to use.

See also

configure_tls_config()

for a function which extracts TLS-related arguments for aioxmpp.security_layer.make()

configure_quirks()

for a function which extracts a set of Quirk enumeration members from the configuration

configure_blockmap()

for a function which extracts a mapping which allows to block features from specific hosts

The following methods are the API used by test cases:

coroutine get_connected_client(presence=<PresenceState available>, *, services=[], prepare=None)

Return a connected client to a unique XMPP account.

Parameters:

• presence (aioxmpp.PresenceState) – initial presence to emit
• prepare (coroutine receiving the client as argument) – a coroutine run after the services are summoned but before the client connects.

Raises:

• OSError – if the connection failed
• RuntimeError – if a client could not be provisioned due to resource constraints

Returns:

Connected presence managed client

Return type:

aioxmpp.PresenceManagedClient

Each account used by the clients returned from this method is unique; all clients are guaranteed to have different bare JIDs.

The clients and accounts are cleaned up after the tear down of the test runs. Some provisioners may have a limit on the number of accounts which can be used in the same test.

Clients obtained from this function are cleaned up automatically on tear down of the test. The clients are stopped and the accounts deleted or cleared, so that each test starts with a fully fresh state.

A coroutine may be passed as prepare argument. It is called with the client as the single argument after all services in services have been summoned but before the client connects, this is for example useful to connect signals that fire early in the connection process.

get_feature_provider(feature_nses)

Parameters:feature_ns (iterable of str) – Namespace URIs to find a provider for Returns:JID of the entity providing all features Return type:aioxmpp.JID

If there is no entity supporting all requested features, None is returned.

has_quirk(quirk)

Parameters:quirk (Quirk) – Quirk to check for Returns:true if the environment has the given quirk

These methods can be used by provisioners to perform plumbing tasks, such as shutting down clients or deleting accounts:

coroutine initialise()

Called once on test framework startup.

Subclasses may run service discovery code here to detect features of the environment they are connected to.

See also

discover_server_features()

for a function which uses XEP-0030 service discovery to find features.

coroutine finalise()

Called once on test framework shutdown (timeout of 10 seconds applies).

coroutine setup()

Called before each test run.

coroutine teardown()

Called after each test run.

The default implementation cleans up the clients obtained from get_connected_client().

class aioxmpp.e2etest.provision.AnonymousProvisioner(logger=<logging.Logger object at 0x7fcf61a7deb8>)

This provisioner uses SASL ANONYMOUS to obtain accounts.

It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key host, TLS can be configured as documented in configure_tls_config() and quirks are set as described in configure_quirks(). A configuration for a locally running Prosody instance might look like this:

[aioxmpp.e2etest.provision.AnonymousProvisioner]
host=localhost
no_verify=true
quirks=[]

The server configured in host must support SASL ANONYMOUS and must allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.

Note

Make sure to disable PEP (XEP-0163) support on the server, to avoid the …pubsub#publish feature to be bound to the server instead of the pubsub component.

This is unfortunate because it prohibits testing PEP properly. This may be fixed in a future release when anything PEP-specific is implemented.

class aioxmpp.e2etest.Quirk

Enumeration of implementation quirks.

Each enumeration member represents a quirk of an implementation. A quirk is a behaviour of an implementation which does not directly violate standards, but which is unfortunate in a way that it disables some features of aioxmpp.

One example of such a quirk is the rewriting of message stanza IDs which some MUC implementations do when reflecting the messages. This breaks the stanza tracking of aioxmpp.muc.Room.send_tracked_message().

The following quirks are defined:

MUC_REWRITES_MESSAGE_ID https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#muc-id-rewrite

This quirk must be configured when the environment the provisioner provides rewrites the message IDs when they are reflected by the MUC implementation.

The quirk does not need to be set if the environment does not provide a MUC implementation at all.

Helper functions

coroutine aioxmpp.e2etest.provision.discover_server_features(disco, peer, recurse_into_items=True, blockmap={})

Use XEP-0030 service discovery to discover features supported by the server.

Parameters:

• disco (aioxmpp.DiscoClient) – Service discovery client which can query the peer server.
• peer (JID) – The JID of the server to query
• recurse_into_items – If set to true, the XEP-0030 items exposed by the server will also be queried for their features. Only one level of recursion is performed.

Returns:

A mapping which maps XEP-0030 feature vars to the JIDs at which the service is provided.

This uses XEP-0030 service discovery to obtain a set of features supported at peer. The set of features is returned as a mapping which maps the var values of the features to the JID at which they were discovered.

If recurse_into_items is true, a XEP-0030 items query is run against peer. For each JID discovered that way, discover_server_features() is re-invoked (with recurse_into_items set to false). The resulting mappings are merged with the mapping obtained from querying the features of peer (existing entries are not overriden – so peer takes precedence).

aioxmpp.e2etest.provision.configure_tls_config(section)

Generate keyword arguments for use with security_layer.make() from the configuration which control the TLS behaviour of the security layer.

Parameters:section – Configuration section to work on. Returns:Keyword arguments for security_layer.make() Return type:dict

The generated keyword arguments are pin_type, pin_store and no_verify. The options in the config file have the same names and the semantics are the following:

pin_store and pin_type can be used to configure certificate pinning, in case the server you want to test against does not have a certificate which passes the default OpenSSL PKIX tests.

If set, pin_store must point to a JSON file, which consists of a single object mapping host names to arrays of strings containing the base64 representation of what is being pinned. This is determined by pin_type, which can be 0 for Public Key pinning and 1 for Certificate pinning.

There is also the no_verify option, which, if set to true, will disable certificate verification altogether. This does not much harm if you are testing against localhost anyways and saves the configuration nuisance for certificate pinning. no_verfiy takes precedence over pin_store and pin_type.

aioxmpp.e2etest.provision.configure_quirks(section)

Generate a set of Quirk enum members from the given configuration section.

Parameters:section – Configuration section to work on. Returns:Set of Quirk members

This parses the configuration key quirks as a python literal (see ast.literal_eval()). It expects a list of strings as a result.

The strings are interpreted as Quirk enum values. If a string starts with #, it is prefixed with https://zombofant.net/xmlns/aioxmpp/e2etest/quirks for easier manual writing of the configuration. See Quirk for the currently defined quirks.