Cheat sheet for caffe / pycaffe?

Question

Does anyone know whether there is a cheat sheet for all important pycaffe commands? I was so far using caffe only via Matlab interface and terminal + bash scripts.

I wanted to shift towards using ipython and work through the ipython notebook examples. However I find it hard to get an overview of all the functions that are inside the caffe module for python. (I'm also quite new to python).

Pero

Answer 1

The pycaffe tests and this file are the main gateway to the python coding interface.

First of all, you would like to choose whether to use Caffe with CPU or GPU. It is sufficient to call caffe.set_mode_cpu() or caffe.set_mode_gpu(), respectively.

Net

The main class that the pycaffe interface exposes is the Net. It has two constructors:

net = caffe.Net('/path/prototxt/descriptor/file', caffe.TRAIN)

which simply create a Net (in this case using the Data Layer specified for training), or

net = caffe.Net('/path/prototxt/descriptor/file', '/path/caffemodel/weights/file', caffe.TEST)

which creates a Net and automatically loads the weights as saved in the provided caffemodel file - in this case using the Data Layer specified for testing.

A Net object has several attributes and methods. They can be found here. I will cite just the ones I use more often.

You can access the network blobs by means of Net.blobs. E.g.

data = net.blobs['data'].data
net.blobs['data'].data[...] = my_image
fc7_activations = net.blobs['fc7'].data

You can access the parameters (weights) too, in a similar way. E.g.

nice_edge_detectors = net.params['conv1'].data
higher_level_filter = net.params['fc7'].data

Ok, now it's time to actually feed the net with some data. So, you will use backward() and forward() methods. So, if you want to classify a single image

net.blobs['data'].data[...] = my_image
net.forward() # equivalent to net.forward_all()
softmax_probabilities = net.blobs['prob'].data

The backward() method is equivalent, if one is interested in computing gradients.

You can save the net weights to subsequently reuse them. It's just a matter of

 net.save('/path/to/new/caffemodel/file')

Solver

The other core component exposed by pycaffe is the Solver. There are several types of solver, but I'm going to use only SGDSolver for the sake of clarity. It is needed in order to train a caffe model. You can instantiate the solver with

solver = caffe.SGDSolver('/path/to/solver/prototxt/file')

The Solver will encapsulate the network you are training and, if present, the network used for testing. Note that they are usually the same network, only with a different Data Layer. The networks are accessible with

 training_net = solver.net
 test_net = solver.test_nets[0] # more than one test net is supported

Then, you can perform a solver iteration, that is, a forward/backward pass with weight update, typing just

 solver.step(1)

or run the solver until the last iteration, with

 solver.solve()

Other features

Note that pycaffe allows you to do more stuff, such as specifying the network architecture through a Python class or creating a new Layer type. These features are less often used, but they are pretty easy to understand by reading the test cases.

Hermineherminia

Very nice answer. Thank you. Just googled for the meaning of the three dots in net.blobs['data'].data[...]. Apparently it is a short form of addressing all elements in the matrix which would normally be written as [:,:,:]. – Pero

You're welcome! You can accept the answer if it's work for you. – Hermineherminia

Awesome post. Thank you. One small item: when I access net.params['conv'] (for example) I get that it is a BlobVec, not a Blob, so an access like nets.params['conv1'].data does not make sense. Instead, that vector needs to be indexed (e.g. net.params['conv1'][0].data) in order to get a Blob from which you can extract .data. Maybe someone else can comment on why this might not match the cheatsheet above (maybe newer version?), but it seemed worth sharing. – Darkle

@FlavioFerrara: I want to do real-time augmentation such as rotation, shearing etc. So net.blobs['data'].data[...] = my_altered_image. After I do net.forward(), how do I update the weights of the model without saving it to disk using net.save? – Santee

When you want to update the weights of the model, you probably want a Solver. It handles the forward/backward pass and the weight update for you. Real-time data augmentation can be done by implementing a custom DataLayer, which is pretty easy to do in Python. See here – Hermineherminia

is there a way to specify a solver configuration using python? or do you have to write it out in a text file manually? – Felonious

There seems to be no way to specify a solver through a specific class. Yet, you can use a Temporary File inside Python, like here. IMO, it's not so clear through – Hermineherminia

@FlavioFerrara you mentioned that "more than one test net is supported". Do you know the syntax to specify more than one test net in the solver prototxt file? – Eaglestone

I believe you can simply specificy multiple test_net parameters. – Hermineherminia

After specifying data with net.blobs['data'].data[...] = my_image, shouldn't we forward using something like net.forward(start='conv1')? Otherwise the net would reload data as specified in prototxt descriptor file. – Indefinable

+1 for such a clear answer. Is there way to specify the network architecture for deploy.prototxt within python? More details about my question are here - #40986509 – Proprioceptor

Does somebody have prebuild binaries of caffe/pycaffe as it is unavailable. Please share I was not able to build it properly even after many attempts. – Marbleize

Answer 2

Please note that the answer by Flavio Ferrara has a litte problem which may cause you waste a lot of time:

net.blobs['data'].data[...] = my_image
net.forward()

The code above is noneffective if your first layer is a Data type layer, because when net.forward() is called, it will begin from the first layer, and then your inserted data my_image will be covered. So it will show no error but give you totally irrelevant output. The correct way is to assign the start and end layer, for example:

net.forward(start='conv1', end='fc')

Here is a Github repository of Face Verification Experiment on LFW Dataset, using pycaffe and some matlab code. I guess it could help a lot, especially the caffe_ftr.py file.

https://github.com/AlfredXiangWu/face_verification_experiment

Besides, here are some short example code of using pycaffe for image classification：

http://codrspace.com/Jaleyhd/caffe-python-tutorial/ http://prog3.com/sbdm/blog/u011762313/article/details/48342495

Lashonlashond

On your note on setting image in data. Does it also cause trouble with solver.step(1)? – Petrolic