Is it possible for AWS Lambda to run a Windows binary via Wine?

Question

I have a Python script with a Windows .exe dependency, which in return relies on a (closed-source) Windows DLL. The Python script runs just fine in Ubuntu via a call to Wine.

Is it possible (and practical) to run this on AWS Lambda?

What would be involved in preparing the code package?

Captivity

I've got Python Lambdas that have a native library dependency and it works pretty well. Ultimately I package the shared library with in the ZIP and within that zip file, in the "lib" directory, I can put shared libraries. Now that is not a full executable, just .so libraries. You'd have to load Wine as a shared library (no, I'm not sure how) and have it run the .exe. Honestly, you might be better off with a small EC2 running Windows - this sounds like a pain. – Lilly

This process would be intermittently run, but would still need to run on demand with minimal delay. Lambda seems like the perfect spot for it (other than this closed dependency issue obviously), since otherwise I'd be paying per hour for a machine which may not take any requests for hours at a time. – Captivity

Totally agree but, for Lambda at least, you've got a pretty challenging setup. The Windows dependency really makes things difficult. Depending on the load a t2.nano is less than USD $5/month. – Lilly

So do you imagine that if I wanted to get this working I'd need to compile Wine binaries for Amazon Linux, and package these along with the script and the exe/DLL? Does Lambda have package size limits...? – Captivity

To answer my second question there, the uncompressed total package size has to be under 250M, and compressed (zip) it has to be under 50M (source). – Captivity

Unless I'm not grabbing all the necessary files (definitely possible), Wine on my (Ubuntu) machine including its documentation etc takes up 125M uncompressed and 27M zipped, so the limits should be OK... – Captivity

@tremby, did you get this to work in the end? – Azimuth

@Azimuth Afraid I can't tell you. I handed it off in the end to another developer. – Captivity

@tremby, I understand. Many thanks for getting back! – Azimuth

Answer 1

Update: the lambda container image feature supports images up to 10gb. I haven't tried it but I think that would be a viable approach, and wouldn't require the hacks I did below to reduce the wine build size.

TL;DR;

Is it Possible? Yes.

Is it practical? The approach I tried is not. A better approach might be to try and put wine into different lambda layers or a custom execution environment.

Will it work for you? It depends, deployment package size and disk space are the limiting factors.

---

Old, somewhat hacky method to fit wine into the regular lambda environment:

I compiled a custom wine with minimal dependencies for lambda, compressed it and then put it onto S3.

Then, in the lambda at runtime, I downloaded the archive, extracted it to /tmp and ran it with a custom empty wine prefix.

My test windows executable was 64bit curl.exe.

1. Compile Wine for Lambda

From https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html, I first tried amzn-ami-hvm-2018.03.0.20181129-x86_64-gp2, but it had an older compilation environment and wouldn't configure.

With AMI amzn2-ami-hvm-2.0.20190313-x86_64-gp2 on a t3.2xlarge ec2, I was able to configure and compile. These are the commands I used, references aws-compile and building-wine:

> sudo yum groupinstall "Development Tools"
> mkdir -p ~/wine-dirs/wine-source
> git clone git://source.winehq.org/git/wine.git ~/wine-dirs/wine-source
> cd ~/wine-dirs/wine-source
> ./configure --enable-win64 --without-x --without-freetype --prefix /opt/wine
> make -j8
> sudo mkdir -p /opt/wine
> sudo chown ec2-user.ec2-user /opt/wine/
> make install
> cd /opt/
> tar zcvf ~/wine-64.tar.gz wine/

This was only a 64-bit build. It also had almost no other optional wine dependencies.

2. Reduce the size of the Wine build further

I removed a lot of optional dependencies from the wine build at compilation time, but it was still too big. /tmp is limited to 500MB.

I deleted files in the package subdirectories, including what looked like optional libs, until I got it down to around 300MB uncompressed.

I verified that wine would still run curl.exe after deleting files from the build.

3. Compress it

I created a tar.bz2 of wine and curl with default bz2 options, it ended up around 80MB. The compressed and extracted files together required about 390MB.

That way there is enough room to both download the archive and extract it to /tmp inside the lambda.

> du -h .
290M    ./wine/lib64/wine
292M    ./wine/lib64
276K    ./wine/share/wine
8.0K    ./wine/share/applications
288K    ./wine/share
5.0M    ./wine/curl-7.66.0-win64-mingw/bin
5.0M    ./wine/curl-7.66.0-win64-mingw
12M     ./wine/bin
308M    ./wine
390M    .

> ls
wine wine.tar.bz2

4. Upload wine.tar.bz2 to S3

Create an S3 bucket and upload the wine.tar.bz2 file to it.

5. Create the Lambda

Create an AWS Lambda using the python 3.7 runtime. While this uses a different underlying AMI than what wine was built on above, it still worked.

In the lambda execution role, grant access to the S3 bucket.

RAM: 1024MB. I chose this because lambda CPU power scales with the memory.

Timeout: 1 min

6. Lambda code:

I needed to follow the advice from this question and answer to change the wine prefix inside the lambda. I also turned off the display as it suggested.

e.g.:

handler():
  ... download from S3 to /tmp, cd to /tmp

  subprocess.call(["tar", "-jxvf", "./wine.tar.bz2"])
  
  os.environ['DISPLAY'] = ''
  os.environ['WINEARCH'] = 'win64'
  os.environ['WINEPREFIX'] = '/tmp/wineprefix'
  
  subprocess.call(["./wine/bin/wine64", "./wine/curl-7.66.0-win64-mingw/bin/curl.exe", "http://www.stackoverflow.com"])

Success!

Government

poida, thanks heaps for your description here, a bit of a lifesaver! I was wondering if you could clarify a couple of things for me? How do you remove optional dependencies at compilation time? And by package subdirectories do you mean the files in /opt/wine/lib64/wine? Cheers – Gasbag

@Gasbag at the > ./configure --enable-win64 --without-x --without-freetype --prefix /opt/wine step. Without x and without freetype were enough for the proof of concept. – Government

By deleting subdirectories and optional libraries, my process was a bit hit and miss, I'd delete one, see if the binary still ran, then repeat. I looked for libraries that I thought wouldn't be needed. In my favour was that curl didn't need many dependencies to run. The more complex program you're trying to run, the harder this will be. – Government

Can you share the exact lambda code you used for this solution. – Bonar

Hi @poida! Thank you so much for sharing these steps. I am looking into something similar. Could you clarify why the compiled package needs to be stored on S3 and can we not package it in a lambda layer? – Pundit

hi @Pundit you're welcome! I'm pretty sure you could try putting it into a lambda layer and not in S3, but I haven't tried it. Looks like atm the total lambda container size limit is 10GB, including layers, so that should be plenty of space to install wine – Government

@Government I followed your steps for compiling wine, zipped the package, and tried to include that in a lambda layer (not in a docker container), and it did throw an error for the package size. Cannot exceed 250MB uncompressed. But deploying as container image would probably work. I am unable to get wine running though with those steps. I posted a question here - #77267903 Would be great if you could help with that. – Pundit