Windows has recently added containerization facilities similar to those provided by the Linux kernel. Like Linux containers, Windows containers operating in process-isolation mode share their kernel and core operating system.
This guide will walk you through creating Windows-based PolySwarm Engines inside Docker containers.
Make sure your system meets or exceeds the following requirements:
- A physical computer running Windows Server or Windows 10 Pro / Enterprise, version 1809 or newer.
- VT-x supported and enabled in BIOS
- 8GB+ of RAM
- 2+ CPU cores
- 21GB+ disk space
These instructions are tested against:
- Windows 10 Pro 1809 (17763.437)
- Docker Desktop CE Version 184.108.40.206 (31259), Build: 8858db3
Using older versions may work, but have not been tested and will not be supported.
We'll need Hyper-V, Microsoft's hypervisor, to:
- Develop our Windows-based Engine.
- Conduct integration tests (these tests require Linux virtual machines that will be hosted using Hyper-V).
To install Hyper-V, open an "elevated" / "privileged" PowerShell console:
- search "PowerShell" in the desktop search bar
- right click on "Windows PowerShell"
- select "Run as administrator".
In the elevated PowerShell prompt, run:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All
Detailed instructions for less common configurations can be found in the Docker documentation.
After installation is complete, run Docker Desktop, enable Container functionality if prompted, and log into your Docker Hub account.
Docker for Windows can either host Linux containers or Windows containers; it cannot host both at the same. Docker will default to Linux container mode; we'll need to switch to Windows container mode.
You can override the default setting during the installation process or at any time after installation using the Docker tray-menu (right-click the icon) and select "Switch to Windows containers...".
Switching to Windows container mode will require a host restart.
Verify that you're in Windows container mode by right-clicking the Docker tray-menu icon and looking for the "Switch to Linux containers..." option. If your menu still says "Switch to Windows containers..." then you have not successfully switched. There appears to be bugs in this switching process; you may need to try switching (and restarting) several times.
Whenever possible, we strongly recommend running Windows containers in "Process Isolation" mode. Process Isolation causes Windows containers to share kernel resources with your host, akin to how Docker works on Linux, incurring far less overhead than Hyper-V based container virtualization.
You cannot use Process Isolation if:
- Your host Windows installation does not match the kernel version of the
1809). We strongly recommend matching host and container versions where possible.
- Your Engine requires the use of Windows kernel modules. We strongly recommend building participants without this requirement.
If you cannot use Process Isolation mode, omit
--isolation=process from subsequent instructions to run the container using Hyper-V virtualization.
docker pull polyswarm/polyswarm-client-windows:latest
docker run --isolation=process -it polyswarm/polyswarm-client-windows powershell
Within the container, run Python and attempt to import
PS C:\> python Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 13:35:33) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import polyswarmclient >>>
If you're able to import
polyswarmclient within the container then your Docker setup is fully working.
quit() to exit the Python prompt and then
exit to exit and destroy the container.
This guide will assume that you'll be conducting development directly on your Windows host. This is the simplest configuration choice.
Advanced users might consider conducting development within a Windows Hyper-V VM. Configuring such a VM is left as an exercise to the reader.
Run the following in an elevated PowerShell prompt.
Permit script execution (necessary for installing Chocolatey & using virtualenvs):
Set-ExecutionPolicy Bypass -Scope LocalMachine -Force
Force PowerShell to use TLSv2:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Chocolatey is a package manager for Windows. We'll use it to help with installing some prerequisites.
Run the following in an elevated PowerShell prompt.
iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
Use Chocolatey to install prerequisites:
choco install -y git choco install -y python --version 3.6.7
We strongly recommend disabling all anti-malware products in your development environment - including the built-in Windows Defender.
Below, we describe disabling Windows Defender. Disabling third party solutions is left as an exercise for the reader.
PolySwarm participants should expect to come into contact with malware. Existing anti-malware engines, including the built-in Windows Defender, can easily get in our way, quarantining or deleting files during development.
In an elevated PowerShell prompt:
choco install -y disabledefender-winconfig
If you need to re-enable realtime protection in the future, simply uninstall the package:
choco uninstall disabledefender-winconfig
and reboot Windows.
We strongly recommend that you create a PolySwarm virtualenv so as to keep the system-wide Python packages clean:
cd ~ python -m venv polyswarmvenv ./polyswarmvenv/Scripts/Activate.ps1 pip install -U pip
PolySwarm tools support two kinds of testing:
- Unit Tests ensure that particular components (units) of your participant function as expected. These tests can be executed directly in your Windows development environment.
- Integration Tests ensure your participant performs successfully, end to end, in a PolySwarm testnet. Standing up a PolySwarm testnet requires a Linux development environment.
We'll use Hyper-V to create a Linux virtual machine on your Windows development machine that we will use later to host a PolySwarm testnet for integration testing.
In the following sections, we've shortened PowerShell prompts to
PS >and Linux prompts to
$to make commands easier to read.
Unless otherwise stated, all commands are to be executed inside your PolySwarm Python virtual environment.
We'll use Hyper-V to create a Linux VM for hosting our testnet.
- Search "Hyper-V Quick Create" in the desktop search bar.
- Run Hyper-V Quick Create.
- Select Ubuntu 18.04.2 as the operating system.
- Click "More Options".
- Name the VM
- Choose a network connection that can communicate with both Docker containers and the Internet.
Typically, this is called
Default Switch, but the name may vary. If in doubt, confirm that your chosen Network appears in
docker network list.
- Click "Create Virtual Machine".
It will take a few minutes to create the VM. When creation is done, you're offered the option to "Connect" to the VM or "Edit settings...". Choose "Edit settings...".
In the settings window, reconfigure for:
- Memory: 8192MB
- Edit the Hard Drive and expand the disk to at least: 40GB
Click "OK" to save and close settings. Click "Connect" to connect to the VM. Click "Start" in the VM window.
Follow the installation prompts and complete the Ubuntu installation.
Once the installation is complete, shut down the VM.
On your Windows host:
- Open the Control Panel.
- Select Network & Internet.
- Look on the right panel under "Network status".
- It should say that your host is connected to the Internet.
- In the network diagram above "You're connected to the Internet", look at the Ethernet jack icon.
- Note the Ethernet adapter displayed in parens ().
This adapter will likely be something similar to
In an elevated PowerShell prompt, create the transparent Docker network switch, substituting
Ethernet0 for the name of your network connect as appropriate:
PS > $networkid=docker network create -d transparent -o com.docker.network.windowsshim.interface="Ethernet0" "PolySwarm Network"
Attach your Linux VM to the same switch (ensure that your Linux VM is powered off before attempting this):
PS> Connect-VMNetworkAdapter -VMName polyswarm_lin -SwitchName $networkid
Next, we need to verify that our Linux VM and Windows Docker container have IP addresses on the same network.
Launch your Linux VM and execute the following:
$ ip a show eth0 eth0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1 link/ether 00:15:5d:07:c9:c1 inet 192.168.86.31/24 brd 192.168.86.1 scope global dynamic valid_lft forever preferred_lft forever
Of note here is
192.168.86.31/24, meaning the Linux VM is on the
192.168.86.0 subnet with a mask of
The exact IP address (
192.168.86.31) and subnet (
192.168.86.0) may differ.
We need to verify the same subnet and mask is allocated to both Windows Docker containers and your Linux VM. Specific IP addresses may differ.
Launch a Windows container (note the
PS > docker run --isolation=process --network "PolySwarm Network" -it polyswarm/polyswarm-client-windows
Inside your Windows Docker container, run:
[container] PS > ipconfig Windows IP Configuration Ethernet adapter vEthernet (Ethernet): Connection-specific DNS Suffix . : localdomain Link-local IPv6 Address . . . . . : fe80::fc35:45f8:8c2e:3032%47 IPv4 Address. . . . . . . . . . . : 192.168.48.137 Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : 192.168.48.2
It may take several minutes for your Windows Docker container to acquire an IP address. If the IP address listed via
ipconfigis not on the same subnet as the Linux VM, wait a few minutes and run
With both our Linux VM and Windows Docker container on the same subnet, let's verify connectivity.
First, within your Windows Docker container, alias the Linux host's address to
polyswarmd (substitute the IP address with the actual address of the Linux VM):
[container] PS > Add-Content C:\Windows\System32\Drivers\etc\hosts -Value "192.168.86.31 polyswarmd"
Ping the IPv4 address of the Linux host
[container] PS > ping polyswarmd Pinging 192.168.86.30 with 32 bytes of data: Reply from 192.168.86.30: bytes=32 time<1ms TTL=128 ...
If your Windows Docker container is able to ping your Linux VM, you're all set!
Your Linux VM will need a handful of tools in order to stand up a local PolySwarm testnet. Follow the Linux Configuration instructions inside of your Linux VM.
Now that we have a working development environment, we'll use
participant-template to bootstrap our project.