PolySwarmPolySwarmPolySwarmPolySwarm
Help

Windows Configuration

System Requirements

Windows has recently added containerization facilities similar to those provided by the Linux kernel. Like Linux containers, Windows containers operating in process-isolation mode share their kernel and core operating system.

This guide will walk you through creating Windows-based PolySwarm participants inside Docker containers.

Make sure your system meets or exceeds the following requirements:

  • A physical computer running Windows Server or Windows 10 Pro / Enterprise, version 1809 or newer.
  • VT-x supported and enabled in BIOS
  • 8GB+ of RAM
  • 2+ CPU cores
  • 21GB+ disk space

These instructions are tested against:

  • Windows 10 Pro 1809 (17763.437)
  • Docker Desktop CE Version 2.0.0.3 (31259), Build: 8858db3

Using older versions may work, but have not been tested and will not be supported.

Some cloud providers such as Azure and Amazon support hosting guest Windows containers as well.


Prerequisites

Install Hyper-V

We'll need Hyper-V, Microsoft's hypervisor, to:

  1. Develop our Windows-based PolySwarm participant (microengine or arbiter).
  2. Conduct integration tests (these tests require Linux virtual machines that will be hosted using Hyper-V).

To install Hyper-V, open an "elevated" / "privileged" PowerShell console:

  • search "PowerShell" in the desktop search bar
  • right click on "Windows PowerShell"
  • select "Run as administrator".

In the elevated PowerShell prompt, run:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All

Restart Windows.

Install Docker

  1. Register / log into Docker Hub
  2. Download Docker for Windows.
  3. Run the installer.

Detailed instructions for less common configurations can be found in the Docker documentation.

After installation is complete, run Docker Desktop, enable Container functionality if prompted, and log into your Docker Hub account.

Configure Docker for Windows Containers

Docker for Windows can either host Linux containers or Windows containers; it cannot host both at the same. Docker will default to Linux container mode; we'll need to switch to Windows container mode.

You can override the default setting during the installation process or at any time after installation using the Docker tray-menu (right-click the icon) and select "Switch to Windows containers...".

Switching to Windows container mode will require a host restart.

Verify that you're in Windows container mode by right-clicking the Docker tray-menu icon and looking for the "Switch to Linux containers..." option. If your menu still says "Switch to Windows containers..." then you have not successfully switched. There appears to be bugs in this switching process; you may need to try switching (and restarting) several times.

A Note on Process Isolation

Whenever possible, we strongly recommend running Windows containers in "Process Isolation" mode. Process Isolation causes Windows containers to share kernel resources with your host, akin to how Docker works on Linux, incurring far less overhead than Hyper-V based container virtualization.

You cannot use Process Isolation if:

  1. Your host Windows installation does not match the kernel version of the polyswarm-client-windows container (1809). We strongly recommend matching host and container versions where possible.
  2. Your participant (microengine, arbiter) requires the use of Windows kernel modules. We strongly recommend building participants without this requirement.

If you cannot use Process Isolation mode, omit --isolation=process from subsequent instructions to run the container using Hyper-V virtualization.

Pull & Run polyswarm-client-windows

Pull the polyswarm-client-windows image:

docker pull polyswarm/polyswarm-client-windows:latest

Test the polyswarm-client-windows image:

docker run --isolation=process -it polyswarm/polyswarm-client-windows powershell

Within the container, run Python and attempt to import polyswarmclient:

PS C:\> python
Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 13:35:33) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import polyswarmclient
>>>

If you're able to import polyswarmclient within the container then your Docker setup is fully working. Run quit() to exit the Python prompt and then exit to exit and destroy the container.


Development Environment

This guide will assume that you'll be conducting development directly on your Windows host. This is the simplest configuration choice.

Advanced users might consider conducting development within a Windows Hyper-V VM. Configuring such a VM is left as an exercise to the reader.

Configure Script Execution

Run the following in an elevated PowerShell prompt.

  1. Permit script execution (necessary for installing Chocolatey & using virtualenvs):

    Set-ExecutionPolicy Bypass -Scope LocalMachine -Force
  2. Force PowerShell to use TLSv2:

    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Install Chocolatey & Prerequisites

Chocolatey is a package manager for Windows. We'll use it to help with installing some prerequisites.

Run the following in an elevated PowerShell prompt.

  1. Install Chocolatey:

    iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
  2. Use Chocolatey to install prerequisites:

    choco install -y git
    choco install -y python --version 3.6.7

Disable Anti-Malware Products

We strongly recommend disabling all anti-malware products in your development environment - including the built-in Windows Defender.

Below, we describe disabling Windows Defender. Disabling third party solutions is left as an exercise for the reader.

PolySwarm engines should expect to come into contact with malware. Existing anti-malware engines, including the built-in Windows Defender, can easily get in our way, quarantining or deleting files during development.

In an elevated PowerShell prompt:

choco install -y disabledefender-winconfig

Reboot Windows.

If you need to re-enable realtime protection in the future, simply uninstall the package:

choco uninstall disabledefender-winconfig

and reboot Windows.

Create a Virtual Environment (virtualenv)

We strongly recommend that you create a PolySwarm virtualenv so as to keep the system-wide Python packages clean:

cd ~
python -m venv polyswarmvenv
./polyswarmvenv/Scripts/Activate.ps1

Prepare for Integration Testing

PolySwarm tools support two kinds of testing:

  • Unit Tests ensure that particular components (units) of your participant function as expected. These tests can be executed directly in your Windows development environment.
  • Integration Tests ensure your participant performs successfully, end to end, in a PolySwarm testnet. Standing up a PolySwarm testnet requires a Linux development environment.

We'll use Hyper-V to create a Linux virtual machine on your Windows development machine that we will use later to host a PolySwarm testnet for integration testing.

In the following sections, we've shortened PowerShell prompts to PS > and Linux prompts to $ to make commands easier to read.

Unless otherwise stated, all commands are to be executed inside your PolySwarm Python virtual environment.

Create a Linux Virtual Machine

We'll use Hyper-V to create a Linux VM for hosting our testnet.

  1. Search "Hyper-V Quick Create" in the desktop search bar.
  2. Run Hyper-V Quick Create.
  3. Select Ubuntu 18.04.2 as the operating system.
  4. Click "More Options".
  5. Name the VM polyswarm_lin.
  6. Choose a network connection that can communicate with both Docker containers and the Internet. Typically, this is called Default Switch, but the name may vary. If in doubt, confirm that your chosen Network appears in docker network list.
  7. Click "Create Virtual Machine".

It will take a few minutes to create the VM. When creation is done, you're offered the option to "Connect" to the VM or "Edit settings...". Choose "Edit settings...".

In the settings window, reconfigure for:

  • Memory: 8192MB
  • Edit the Hard Drive and expand the disk to at least: 40GB

Click "OK" to save and close settings. Click "Connect" to connect to the VM. Click "Start" in the VM window.

Follow the installation prompts and complete the Ubuntu installation.

Once the installation is complete, shut down the VM.

Configure Networking Between Windows Docker Containers and the Linux VM

On your Windows host:

  1. Open the Control Panel.
  2. Select Network & Internet.
  3. Look on the right panel under "Network status".
  4. It should say that your host is connected to the Internet.
  5. In the network diagram above "You're connected to the Internet", look at the Ethernet jack icon.
  6. Note the Ethernet adapter displayed in parens (). This adapter will likely be something similar to Ethernet0.

In an elevated PowerShell prompt, create the transparent Docker network switch, substituting Ethernet0 for the name of your network connect as appropriate:

PS > $networkid=docker network create -d transparent -o com.docker.network.windowsshim.interface="Ethernet0" "PolySwarm Network"

Attach your Linux VM to the same switch (ensure that your Linux VM is powered off before attempting this):

PS> Connect-VMNetworkAdapter -VMName polyswarm_lin -SwitchName $networkid

Next, we need to verify that our Linux VM and Windows Docker container have IP addresses on the same network.

Launch your Linux VM and execute the following:

$ ip a show eth0
eth0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 00:15:5d:07:c9:c1
    inet 192.168.86.31/24 brd 192.168.86.1 scope global dynamic
       valid_lft forever preferred_lft forever

Of note here is 192.168.86.31/24, meaning the Linux VM is on the 192.168.86.0 subnet with a mask of 255.255.255.0. The exact IP address (192.168.86.31) and subnet (192.168.86.0) may differ.

We need to verify the same subnet and mask is allocated to both Windows Docker containers and your Linux VM. Specific IP addresses may differ.

Launch a Windows container (note the --network argument):

PS > docker run --isolation=process --network "PolySwarm Network" -it polyswarm/polyswarm-client-windows

Inside your Windows Docker container, run:

[container] PS > ipconfig

Windows IP Configuration


Ethernet adapter vEthernet (Ethernet):

   Connection-specific DNS Suffix  . : localdomain
   Link-local IPv6 Address . . . . . : fe80::fc35:45f8:8c2e:3032%47
   IPv4 Address. . . . . . . . . . . : 192.168.48.137
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.48.2

It may take several minutes for your Windows Docker container to acquire an IP address. If the IP address listed via ipconfig is not on the same subnet as the Linux VM, wait a few minutes and run ipconfig again.

With both our Linux VM and Windows Docker container on the same subnet, let's verify connectivity.

First, within your Windows Docker container, alias the Linux host's address to polyswarmd (substitute the IP address with the actual address of the Linux VM):

[container] PS > Add-Content C:\Windows\System32\Drivers\etc\hosts -Value "192.168.86.31 polyswarmd"

Ping the IPv4 address of the Linux host polyswarmd:

[container] PS > ping polyswarmd
Pinging 192.168.86.30 with 32 bytes of data:
Reply from 192.168.86.30: bytes=32 time<1ms TTL=128
...

If your Windows Docker container is able to ping your Linux VM, you're all set!

Install Linux Prerequisites

Your Linux VM will need a handful of tools in order to stand up a local PolySwarm testnet. Follow the Linux Configuration instructions inside of your Linux VM.

Next Steps

Now that we have a working development environment, we'll use participant-template to bootstrap our project.

Create Your Participant →