How to run OpenAI Gym .render() over a server

Question

I am running a python 2.7 script on a p2.xlarge AWS server through Jupyter (Ubuntu 14.04). I would like to be able to render my simulations.

Minimal working example

import gym
env = gym.make('CartPole-v0')
env.reset()
env.render()

env.render() makes (among other things) the following errors:

...
HINT: make sure you have OpenGL install. On Ubuntu, you can run 
'apt-get install python-opengl'. If you're running on a server, 
you may need a virtual frame buffer; something like this should work: 
'xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>'")
...
NoSuchDisplayException: Cannot connect to "None"

I would like to some how be able to see the simulations. It would be ideal if I could get it inline, but any display method would be nice.

Edit: This is only an issue with some environments, like classic control.

---

Update I

Inspired by this I tried the following, instead of the xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py> (which I couldn't get to work).

xvfb-run -a jupyter notebook

Running the original script I now get instead

GLXInfoException: pyglet requires an X server with GLX

---

Update II

Issue #154 seems relevant. I tried disabling the pop-up, and directly creating the RGB colors

import gym
env = gym.make('CartPole-v0')
env.reset()

img = env.render(mode='rgb_array', close=True)  
print(type(img)) # <--- <type 'NoneType'>

img = env.render(mode='rgb_array', close=False) # <--- ERROR
print(type(img)) 

I get ImportError: cannot import name gl_info.

---

Update III

With inspiration from @Torxed I tried creating a video file, and then rendering it (a fully satisfying solution).

Using the code from 'Recording and uploading results'

import gym

env = gym.make('CartPole-v0')
env.monitor.start('/tmp/cartpole-experiment-1', force=True)
observation = env.reset()
for t in range(100):
#    env.render()
    print(observation)
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done:
        print("Episode finished after {} timesteps".format(t+1))
        break

env.monitor.close()

I tried following your suggestions, but got ImportError: cannot import name gl_info from when running env.monitor.start(....

From my understanding the problem is that OpenAI uses pyglet, and pyglet 'needs' a screen in order to compute the RGB colors of the image that is to be rendered. It is therefore necessary to trick python to think that there is a monitor connected

---

Update IV

FYI there are solutions online using bumblebee that seem to work. This should work if you have control over the server, but since AWS run in a VM I don't think you can use this.

---

Update V

Just if you have this problem, and don't know what to do (like me) the state of most environments are simple enough that you can create your own rendering mechanism. Not very satisfying, but.. you know.

Weatherley

Have you tried making a virtual screen buffer like the very helpful error message asks? – Candent

That is the thing, I don't know how to do that. In the guides that I have looked at I don't understand how to make it work over the server. – Weatherley

I have updated the post with my attempt – Weatherley

I'm stuck here too, any help would be greatly appreciated :) – Mario

Just a heads up, @TokeFaurby, your solution was deprecated as of December 23, 2016. You need to use a slightly different API and a differnet approach to get this working. I think your approach won't work anymore though, since the VideoRecorder class as part of env.wrapper.Monitor is now calling env.render directly. – Simas

I have solved that using this answer: stackoverflow NameError: name 'base' is not defined, while running open AI gym in GOOGLE COLAB – Lacteal

Answer 1

Got a simple solution working:

If on a linux server, open jupyter with

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

In Jupyter

import matplotlib.pyplot as plt
%matplotlib inline
from IPython import display

After each step

def show_state(env, step=0, info=""):
    plt.figure(3)
    plt.clf()
    plt.imshow(env.render(mode='rgb_array'))
    plt.title("%s | Step: %d %s" % (env._spec.id,step, info))
    plt.axis('off')

    display.clear_output(wait=True)
    display.display(plt.gcf())

Note: if your environment is not unwrapped, pass env.env to show_state.

Reichsmark

Works well, thanks, but the env still opens in an external window as well (I'm running it local, not in a remote server). Any ideas on how to prevent this? – Basketwork

I am getting /bin/xvfb-run: line 181: 0: command not found and /bin/xvfb-run: line 186: kill: (31449) - No such process errors. Any idea? – Profitsharing

What does the "24" in 1400x900x24 mean? I'm guessing 1400 is monitor width and 900 is monitor height – Jan

24 specifies the bit depth of the virtual monitor - see here – Dissoluble

My problem happens at the render stage: env = gym.make('CartPole-v0') ; env.render(mode='rgb_array') ; gives me ValueError: Array length must be >= 0, not -48424951659315200 – Lissalissak

Answer 2

This GitHub issue gave an answer that worked great for me. It's nice because it doesn't require any additional dependencies (I assume you already have matplotlib) or configuration of the server.

Just run, e.g.:

import gym
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0') # insert your favorite environment
render = lambda : plt.imshow(env.render(mode='rgb_array'))
env.reset()
render()

Using mode='rgb_array' gives you back a numpy.ndarray with the RGB values for each position, and matplotlib's imshow (or other methods) displays these nicely.

Note that if you're rendering multiple times in the same cell, this solution will plot a separate image each time. This is probably not what you want. I'll try to update this if I figure out a good workaround for that.

Update to render multiple times in one cell

Based on this StackOverflow answer, here's a working snippet (note that there may be more efficient ways to do this with an interactive plot; this way seems a little laggy on my machine):

import gym
from IPython import display
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
for _ in range(100):
    plt.imshow(env.render(mode='rgb_array'))
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Update to increase efficiency

On my machine, this was about 3x faster. The difference is that instead of calling imshow each time we render, we just change the RGB data on the original plot.

import gym
from IPython import display
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
img = plt.imshow(env.render(mode='rgb_array')) # only call this once
for _ in range(100):
    img.set_data(env.render(mode='rgb_array')) # just update the data
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Caernarvonshire

Did you actually try and run this over a server? Or did you only test this on your local machine? What you are suggesting is essentially the same as in Update II. The problem is that env.render() imports gl_info behind the scenes, even if the window doesn't open. mode=rgb_array doesn't change that – Weatherley

Ah, I didn't try it on an AWS server; the one I used must have something configured differently such that I don't get the same error. Sorry it didn't help – Caernarvonshire

@Caernarvonshire there's an additional problem with your solution. Some envs seem to have ways of bypassing the window creation while others do not. For example, if you want to try this out, replace env = gym.make('Breakout-v0') with env = gym.make('CartPole-v0') in your examples and you'll see what I mean. 'Breakout' happens to be an atari game which uses the atari_py.ALEInterface() as its renderer and returns an image. see github.com/openai/gym/blob/master/gym/envs/atari/… – Simas

Answer 3

I think we should just capture renders as video by using OpenAI Gym wrappers.Monitor and then display it within the Notebook.

Example:

Dependencies

!apt install python-opengl
!apt install ffmpeg
!apt install xvfb
!pip3 install pyvirtualdisplay

# Virtual display
from pyvirtualdisplay import Display

virtual_display = Display(visible=0, size=(1400, 900))
virtual_display.start()

Capture as video

import gym
from gym import wrappers

env = gym.make("SpaceInvaders-v0")
env = wrappers.Monitor(env, "/tmp/SpaceInvaders-v0")

for episode in range(2):
    observation = env.reset()
    step = 0
    total_reward = 0

    while True:
        step += 1
        env.render()
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        total_reward += reward
        if done:
            print("Episode: {0},\tSteps: {1},\tscore: {2}"
                  .format(episode, step, total_reward)
            )
            break
env.close()

Display within Notebook

import os
import io
import base64
from IPython.display import display, HTML

def ipython_show_video(path):
    """Show a video at `path` within IPython Notebook
    """
    if not os.path.isfile(path):
        raise NameError("Cannot access: {}".format(path))

    video = io.open(path, 'r+b').read()
    encoded = base64.b64encode(video)

    display(HTML(
        data="""
        <video alt="test" controls>
        <source src="data:video/mp4;base64,{0}" type="video/mp4" />
        </video>
        """.format(encoded.decode('ascii'))
    ))

ipython_show_video("/tmp/SpaceInvaders-v0/openaigym.video.4.10822.video000000.mp4")

I hope it helps. ;)

Nephogram

I get "NameError: Cannot access: /tmp/SpaceInvaders-v0/openaigym.video.4.10822.video000000.mp4" at the end after running everything. Any idea? – Franchescafranchise

Check the file name under /tmp/SpaceInvaders-v0. The name of the video might be different. Alternatively, provide a known folder for env = wrappers.Monitor(env, "/tmp/SpaceInvaders-v0") – Welby

This also does not work with classic control such as "CartPole-v0": ValueError: Array length must be >= 0, not -48424951659315200 – Lissalissak

I confirm it works with retro import and SpaceInvaders-Atari2600:/nimport retro/n from gym import wrappers/n env = retro.make(game="SpaceInvaders-Atari2600")/n env = wrappers.Monitor(env, "SpaceInvaders-Atari2600")/n ......../n ipython_show_video("./SpaceInvaders-Atari2600/openaigym.video.0.2383.video000000.mp4")/n – Berman

Answer 4

I managed to run and render openai/gym (even with mujoco) remotely on a headless server.

# Install and configure X window with virtual screen
sudo apt-get install xserver-xorg libglu1-mesa-dev freeglut3-dev mesa-common-dev libxmu-dev libxi-dev
# Configure the nvidia-x
sudo nvidia-xconfig -a --use-display-device=None --virtual=1280x1024
# Run the virtual screen in the background (:0)
sudo /usr/bin/X :0 &
# We only need to setup the virtual screen once

# Run the program with vitural screen
DISPLAY=:0 <program>

# If you dont want to type `DISPLAY=:0` everytime
export DISPLAY=:0

Usage:

DISPLAY=:0 ipython2

Example:

import gym
env = gym.make('Ant-v1')
arr = env.render(mode='rgb_array')
print(arr.shape)
# plot or save wherever you want
# plt.imshow(arr) or scipy.misc.imsave('sample.png', arr)

Jaleesa

So just to be sure, this also worked for classic control? – Weatherley

With the classical control like CartPole, you can try "ssh -X username@hostname" and render directly. – Jaleesa

I was getting the error ContextException: Could not create GL context remote and none of the other solutions worked for me, but this worked perfectly. – Staub

Answer 5

There's also this solution using pyvirtualdisplay (an Xvfb wrapper). One thing I like about this solution is you can launch it from inside your script, instead of having to wrap it at launch:

from pyvirtualdisplay import Display
display = Display(visible=0, size=(1400, 900))
display.start()

Dimmer

I got this error: ValueError: invalid literal for int() with base 10:'' – Phenacetin

Answer 6

I ran into this myself. Using xvfb as X-server somehow clashes with the Nvidia drivers. But finally this post pointed me into the right direction. Xvfb works without any problems if you install the Nvidia driver with the -no-opengl-files option and CUDA with --no-opengl-libs option. If you know this, it should work. But as it took me quite some time till I figured this out and it seems like I'm not the only one running into problems with xvfb and the nvidia drivers.

I wrote down all necessary steps to set everything up on an AWS EC2 instance with Ubuntu 16.04 LTS here.

Chung

I followed the guide to the best of my ability, but couldn't get it to work. The only difference is that I didn't build TF from source, but since this problem is unrelated I don't think it matters. – Weatherley

Sorry to hear that. Since the Amazon Deep Learning AMI now contains a Tensor Flow version with GPU support I'd rather recommend starting from there: aws.amazon.com/marketplace/pp/B01M0AXXQB But if you'd still want to setup from scratch, please provide some more information about the problems you encountered, so I can help you. – Chung

Answer 7

Referencing my other answer here: Display OpenAI gym in Jupyter notebook only

I made a quick working example here which you could fork: https://kyso.io/eoin/openai-gym-jupyter with two examples of rendering in Jupyter - one as an mp4, and another as a realtime gif.

The .mp4 example is quite simple.

import gym
from gym import wrappers

env = gym.make('SpaceInvaders-v0')
env = wrappers.Monitor(env, "./gym-results", force=True)
env.reset()
for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done: break
env.close()

Then in a new cell Jupyter cell, or download it from the server onto some place where you can view the video.

import io
import base64
from IPython.display import HTML

video = io.open('./gym-results/openaigym.video.%s.video000000.mp4' % env.file_infix, 'r+b').read()
encoded = base64.b64encode(video)
HTML(data='''
    <video width="360" height="auto" alt="test" controls><source src="data:video/mp4;base64,{0}" type="video/mp4" /></video>'''
.format(encoded.decode('ascii')))

If your on a server with public access you could run python -m http.server in the gym-results folder and just watch the videos there.

Echelon

Answer 8

I encountered the same problem and stumbled upon the answers here. Mixing them helped me to solve the problem.

Here's a step by step solution:

Install the following:

apt-get install -y python-opengl xvfb

Start your jupyter notebook via the following command:

xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

Inside the notebook:

import gym
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('MountainCar-v0') # insert your favorite environment
env.reset()
plt.imshow(env.render(mode='rgb_array')

Now you can put the same thing in a loop to render it multiple times.

from IPython import display

for _ in range(100):
    plt.imshow(env.render(mode='rgb_array'))
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Hope this works for anyone else still facing an issue. Thanks to Andrews and Nathan for their answers.

Layoff

This solution finally worked for me, but I got an error AttributeError: 'ImageData' object has no attribute 'data' and installing pyglet-v1.3.2 fixed that for me from the solution over here. – Kalong

Answer 9

I avoided the issues with using matplotlib by simply using PIL, Python Image Library:

import gym, PIL
env = gym.make('SpaceInvaders-v0')
array = env.reset()
PIL.Image.fromarray(env.render(mode='rgb_array'))

I found that I didn't need to set the XV frame buffer.

Mulkey

I highly doubt that this works over a headless server. The problem is with the env.render function, PIL has nothing to do with this issue. – Weatherley

Sorry didn't notice that you used SpaceInvaders-v0. This issue is only caused by some environment, like classic control. I edited the OP appropriately – Weatherley

Yes, I see now that my answer only works with some simulations. That's a shame, and limitation of gym. – Mulkey

Answer 10

I was looking for a solution that works in Colaboratory and ended up with this

from IPython import display
import numpy as np
import time

import gym
env = gym.make('SpaceInvaders-v0')
env.reset()

import PIL.Image
import io


def showarray(a, fmt='png'):
    a = np.uint8(a)
    f = io.BytesIO()
    ima = PIL.Image.fromarray(a).save(f, fmt)
    return f.getvalue()

imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')

while True:
    time.sleep(0.01)
    env.step(env.action_space.sample()) # take a random action
    display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')

EDIT 1:

You could use xvfbwrapper for the Cartpole environment.

from IPython import display
from xvfbwrapper import Xvfb
import numpy as np
import time
import pyglet
import gym
import PIL.Image
import io    

vdisplay = Xvfb(width=1280, height=740)
vdisplay.start()

env = gym.make('CartPole-v0')
env.reset()

def showarray(a, fmt='png'):
    a = np.uint8(a)
    f = io.BytesIO()
    ima = PIL.Image.fromarray(a).save(f, fmt)
    return f.getvalue()

imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')


for _ in range(1000):
  time.sleep(0.01)
  observation, reward, done, info = env.step(env.action_space.sample()) # take a random action
  display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')


vdisplay.stop()

If you're working with standard Jupyter, there's a better solution though. You can use the CommManager to send messages with updated Data URLs to your HTML output.

IPython Inline Screen Example

In Colab the CommManager is not available. The more restrictive output module has a method called eval_js() which seems to be kind of slow.

Traver

It works with SpaceInvaders-v0, but when I call CartPole-v0 or Pendulum-v0 I get NameError: name 'base' is not defined. Any idea? – Profitsharing

@AfshinOroojlooy try to !apt-get install python-pyglet && pip install pyglet and then restart the runtime – Lole

@shadi, thanks for the reply. I solved this problem a while ago. – Profitsharing

Answer 11

I had the same problem and I_like_foxes solution to reinstall nvidia drivers with no opengl fixed things. Here are the commands I used for Ubuntu 16.04 and GTX 1080ti https://gist.github.com/8enmann/931ec2a9dc45fde871d2139a7d1f2d78

Idolism

Answer 12

This might be a complete workaround, but I used a docker image with a desktop environment, and it works great. The docker image is at https://hub.docker.com/r/dorowu/ubuntu-desktop-lxde-vnc/

The command to run is

docker run -p 6080:80 dorowu/ubuntu-desktop-lxde-vnc

Then browse http://127.0.0.1:6080/ to access the Ubuntu desktop.

Below are a gif showing it the Mario bros gym environment running and being rendered. As you can see, it is fairly responsive and smooth.

Haag

Answer 13

I created this mini-package which allows you to render your environment onto a browser by just adding one line to your code.

Put your code in a function and replace your normal env.render() with yield env.render(mode='rgb_array'). Encapsulate this function with the render_browser decorator.

import gym
from render_browser import render_browser

@render_browser
def test_policy(policy):
    # Your function/code here.
    env = gym.make('Breakout-v0')
    obs = env.reset()

    while True:
        yield env.render(mode='rgb_array')
        # ... run policy ...
        obs, rew, _, _ = env.step(action)

test_policy(policy)    

When you visit your_ip:5000 on your browser, test_policy() will be called and you'll be able to see the rendered environment on your browser window.

Cirrose

Your library doesn't work. When I opened the browser, I couldn't see the environment there. – Hamlen

The library not working usually has to do with your env. The render mode rgb_array might not be implemented. This is esp the case w/ mujoco based non-gym envs. – Cirrose

Answer 14

In my IPython environment, Andrew Schreiber's solution can't plot image smoothly. The following is my solution:

If on a linux server, open jupyter with

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

In Jupyter

import matplotlib.pyplot as plt
%matplotlib inline
%matplotlib notebook
from IPython import display

Display iteration:

done = False
obs = env.reset()

fig = plt.figure()
ax = fig.add_subplot(111)
plt.ion()

fig.show()
fig.canvas.draw()

while not done:
    # action = pi.act(True, obs)[0] # pi means a policy which produces an action, if you have
    # obs, reward, done, info = env.step(action) # do action, if you have
    env_rnd = env.render(mode='rgb_array')
    ax.clear()
    ax.imshow(env_rnd)
    fig.canvas.draw()
    time.sleep(0.01)

Bennett

This solution results in NameError: name 'base' is not defined for me. Any idea? – Profitsharing