PolySwarmPolySwarmPolySwarmPolySwarm
Help

Building a File-Submitting Ambassador

polyswarm-client's filesystem.py ambassador expands on the eicar.py ambassador, submitting artifacts from a local filesystem.

It begins in a similar manner:

import logging
import random
import os

from concurrent.futures import CancelledError

from polyswarmartifact import ArtifactType

from polyswarmclient.abstractambassador import AbstractAmbassador
from polyswarmclient.corpus import DownloadToFileSystemCorpus

logger = logging.getLogger(__name__)

ARTIFACT_DIRECTORY = os.getenv('ARTIFACT_DIRECTORY', 'docker/artifacts')
ARTIFACT_BLACKLIST = os.getenv('ARTIFACT_BLACKLIST', 'truth.db').split(',')
BOUNTY_TEST_DURATION_BLOCKS = int(os.getenv('BOUNTY_TEST_DURATION_BLOCKS', 5))

Again, imports are handled, the bounty duration is hard-coded and logging is configured. filesystem.py makes use of polyswarmclient.corpus, a helper class that will download, decrypt and extract an artifact collection. Swarm Technologies uses this class internally during continuous integration to ensure that malicious artifacts are detected as such by microengines.

Continuing:

class Ambassador(AbstractAmbassador):
    """Ambassador which submits artifacts from a directory"""

    def __init__(self, client, testing=0, chains=None, watchdog=0, submission_rate=30):
        """Initialize a filesystem ambassador
        Args:
            client (`Client`): Client to use
            testing (int): How many test bounties to respond to
            chains (set[str]): Chain(s) to operate on
        """
        super().__init__(client, testing, chains, watchdog, submission_rate)

filesystem.py makes use of several more arguments to the AbstractAmbassador class that are useful for testing:

  • testing: when nonzero, this parameter specifies the maximum number of bounties the ambassador will generate before exiting.
  • watchdog: a block interval. Bounties placed by this ambassador are checked against each new block to ensure that the bounty has been successfully placed on-chain.
  • submission_rate: if nonzero, this produces a sleep in the main event loop to prevent the ambassador from overloading polyswarmd during testing.

Moving on:

        self.artifacts = []
        u = os.getenv("MALICIOUS_BOOTSTRAP_URL")
        if u:
            logger.info("Unpacking malware corpus at {0}".format(u))
            d = DownloadToFileSystemCorpus()
            d.download_and_unpack()
            bfl = d.get_benign_file_list()
            mfl = d.get_malicious_file_list()
            logger.info("Unpacking complete, {0} malicious and {1} benign files".format(len(mfl), len(bfl)))
            self.artifacts = bfl + mfl
        else:
            for root, dirs, files in os.walk(ARTIFACT_DIRECTORY):
                for f in files:
                    self.artifacts.append(os.path.join(root, f))

If the environment variable MALICIOUS_BOOTSTRAP_URL is set, the ambassador downloads artifacts from a testing repository. If it's not set, ARTIFACT_DIRECTORY directory is walked relative to the ambassador's current working directory. Files are gathered in preparation for bounty generation.

filesystem.py overrides the generate_bounties method:

    async def generate_bounties(self, chain):
        """Submit bounty from the filesystem
        Args:
            chain (str): Chain sample is being requested from
        """
        amount = await self.client.bounties.parameters[chain].get('bounty_amount_minimum')

        while True:
            try:
                filename = random.choice(self.artifacts)

                logger.info('Submitting file %s', filename)
                ipfs_uri = await self.client.post_artifacts([(filename, None)])
                if not ipfs_uri:
                    logger.error('Error uploading artifact to IPFS, continuing')
                    continue

                await self.push_bounty(ArtifactType.FILE, amount, ipfs_uri, BOUNTY_TEST_DURATION_BLOCKS, chain)
            except CancelledError:
                logger.warning('Cancel requested')
                break
            except Exception:
                logger.exception('Exception in bounty generation task, continuing')
                continue

This is identical to the logic contained withing eicar.py.

filesystem.py builds on eicar.py by building an artifact from on-disk and, optionally, a remote URL. In a real-world ambassador, these artifacts would come from the consumer's submissions to the ambassador.

Test Your Participant

Once everything is in place, let's test our participant:

Unit Testing →

Next Steps

Now that you're familiar with developing several proof of concept ambassadors, it's time to consider what it would take to build an production ambassadors that connects to the PolySwarm marketplace.

Production Ambassadors →