The first step is to get a copy of our reference Engine implementation from GitHub.
git to clone the microengine-webhooks-py repo.
(psvenv) $ git clone https://github.com/polyswarm/microengine-webhooks-py.git (psvenv) $ cd microengine-webhooks-py/
This repo uses a standard Python package layout, but here is a quick overview of the contents:
chart- Kubernetes chart and values
docker- Dockerfiles to build Docker images
nginx- Configs for
src/microenginewebhookspy- Python code for your Engine's web service and worker
tests- Unit and Integration tests
Next, let's install the Python dependencies.
(psvenv) $ pip install -r requirements.txt
When this is done, you have everything you need to build an Engine.
As-is, this template is a fully functional Engine that can only detect EICAR. The purpose of a trivial EICAR engine is to understand how an Engine works and to test protocols and connectivity.
The EICAR test file is used by the Antivirus industry to test an engines' ability to detect malware. The file is not malicious, but is flagged as such by most vendors.
The EICAR test file is defined as a file that begins with the following string:
X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H* followed by a variable amount of whitespace.
The three most important files for Engine developers are scan.py, settings.py, and models.py.
scan.py is the primary source file defining Engine behavior.
The first of the two functions that already exist in scan.py is
scan() takes a
Bounty and returns a
The primary function of
scan() is to execute a scan on the
Bounty's artifact and determine whether it is malicious.
The result of the scan is stored in the
- Verdict can be one of four states: malicious, suspicious, benign, or unknown.
- Confidence is a measure of how sure the Engine feels about the
Verdict. A decimal between
- Metadata is a
ScanMetadataobject containing a collection of details about the artifact and/or scanning application.
Developers should override
scan() and replace the function contents to perform their own scan.
The second function is
compute_bid() takes a
ScanResult pair, then returns an
int as the amount of NCT to bid.
compute_bid() is covered in more detail in the section on bidding strategy.
settings.py has environment variables for tuning Engine behavior. By default, there are variables for the Webhook secret, Celery broker, and log level. Developers should add more environment variables as needed to configure the behavior of their Engine.
models.py contains all the definitions for objects like
ScanResult, and the
The remainder of the source files can be quickly summarized, and they won't be discussed in any detail in these docs.
- middleware.py - WSGI middleware that checks the signature with the shared secret
- tasks.py - Celery task definition that handles the bounty and coordinates scanning, bidding, and responding to the PolySwarm Marketplace
- utils.py - Helper function definitions
- views.py - Flask Blueprint and routes to receive, and parse Webhooks
- wsgi.py - Flask application defintion
Now we have a working Engine template, let's modify it to build our own Engine.