PolySwarmPolySwarmPolySwarmPolySwarm
Go to PolySwarm
Home

Create Your Engine Template

Clone the microengine-webhooks-py repo

The first step is to get a copy of our reference Engine implementation from GitHub. We'll use git to clone the microengine-webhooks-py repo.

(psvenv) $ git clone https://github.com/polyswarm/microengine-webhooks-py.git
(psvenv) $ cd microengine-webhooks-py/

What's in this repo?

This repo uses a standard Python package layout, but here is a quick overview of the contents:

  • chart - Kubernetes chart and values
  • docker - Dockerfiles to build Docker images
  • nginx - Configs for nginx unit
  • src/microenginewebhookspy - Python code for your Engine's web service and worker
  • tests - Unit and Integration tests

Install dependencies

Next, let's install the Python dependencies.

(psvenv) $ pip install -r requirements.txt

When this is done, you have everything you need to build an Engine.

EICAR Engine

As-is, this template is a fully functional Engine that can only detect EICAR. The purpose of a trivial EICAR engine is to understand how an Engine works and to test protocols and connectivity.

The EICAR test file is used by the Antivirus industry to test an engines' ability to detect malware. The file is not malicious, but is flagged as such by most vendors.

The EICAR test file is defined as a file that begins with the following string: X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H* followed by a variable amount of whitespace.

The Code

The three most important files for Engine developers are scan.py, settings.py, and models.py.

scan.py

scan.py is the primary source file defining Engine behavior. The first of the two functions that already exist in scan.py is scan(). scan() takes a Bounty and returns a ScanResult. The primary function of scan() is to execute a scan on the Bounty's artifact and determine whether it is malicious. The result of the scan is stored in the ScanResult's fields: verdict, confidence, and metadata.

  • Verdict can be one of four states: malicious, suspicious, benign, or unknown.
  • Confidence is a measure of how sure the Engine feels about the Verdict. A decimal between 0.0 and 1.0.
  • Metadata is a ScanMetadata object containing a collection of details about the artifact and/or scanning application.

Developers should override scan() and replace the function contents to perform their own scan.

The second function is compute_bid(). compute_bid() takes a Bounty & ScanResult pair, then returns an int as the amount of NCT to bid. compute_bid() is covered in more detail in the section on bidding strategy.

settings.py

settings.py has environment variables for tuning Engine behavior. By default, there are variables for the Webhook secret, Celery broker, and log level. Developers should add more environment variables as needed to configure the behavior of their Engine.

models.py

models.py contains all the definitions for objects like Bounty, ScanResult, and the Verdict enum.

Remaining Python Files

The remainder of the source files can be quickly summarized, and they won't be discussed in any detail in these docs.

  • middleware.py - WSGI middleware that checks the signature with the shared secret
  • tasks.py - Celery task definition that handles the bounty and coordinates scanning, bidding, and responding to the PolySwarm Marketplace
  • utils.py - Helper function definitions
  • views.py - Flask Blueprint and routes to receive, and parse Webhooks
  • wsgi.py - Flask application defintion

Next Steps

Now we have a working Engine template, let's modify it to build our own Engine.

2021 © PolySwarm Pte. Ltd.