What's the purpose of tf.app.flags in TensorFlow?

Question

I am reading some example codes in Tensorflow, I found following code

flags = tf.app.flags
FLAGS = flags.FLAGS
flags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')
flags.DEFINE_integer('max_steps', 2000, 'Number of steps to run trainer.')
flags.DEFINE_integer('hidden1', 128, 'Number of units in hidden layer 1.')
flags.DEFINE_integer('hidden2', 32, 'Number of units in hidden layer 2.')
flags.DEFINE_integer('batch_size', 100, 'Batch size.  '
                 'Must divide evenly into the dataset sizes.')
flags.DEFINE_string('train_dir', 'data', 'Directory to put the training data.')
flags.DEFINE_boolean('fake_data', False, 'If true, uses fake data '
                 'for unit testing.')

in tensorflow/tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py

But I can't find any docs about this usage of tf.app.flags.

And I found the implementation of this flags is in the tensorflow/tensorflow/python/platform/default/_flags.py

Obviously, this tf.app.flags is somehow used to configure a network, so why is it not in the API docs? Can anyone explain what is going on here?

Definition

Answer 1

The tf.app.flags module is presently a thin wrapper around python-gflags, so the documentation for that project is the best resource for how to use it argparse, which implements a subset of the functionality in python-gflags.

Note that this module is currently packaged as a convenience for writing demo apps, and is not technically part of the public API, so it may change in future.

We recommend that you implement your own flag parsing using argparse or whatever library you prefer.

EDIT: The tf.app.flags module is not in fact implemented using python-gflags, but it uses a similar API.

Winifredwinikka

"packaged as a convenience for writing demo apps, and is not technically part of the public AP" ... kind of strange that it's used in almost every tutorial, but there is no documentation on it. Leads to plenty of confusion. – Malamut

For a good example of how to use argparse to pass arguments to a TensorFlow model and how to bundle it up into a Python module for the cloud, see task.py in the taxifare module, part of the the training-data-analyst course materials. – Linkous

Is tf.app.run also not part of the public API? Because it relies on tf.app.flags and it has public documentation (tensorflow.org/api_docs/python/tf/app/run), so I assume it is public and supported. If it is recommended to use argparse instead, could you provide a brief example of the recommended way of using it with argparse? – Crowbar

isn't documentation an issue for everything in tensorflow. – Miso

Answer 2

The tf.app.flags module is a functionality provided by Tensorflow to implement command line flags for your Tensorflow program. As an example, the code you came across would do the following:

flags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')

The first parameter defines the name of the flag while the second defines the default value in case the flag is not specified while executing the file.

So if you run the following:

$ python fully_connected_feed.py --learning_rate 1.00

then the learning rate is set to 1.00 and will remain 0.01 if the flag is not specified.

As mentioned in this article, the docs are probably not present because this might be something that Google requires internally for its developers to use.

Also, as mentioned in the post, there are several advantages of using Tensorflow flags over flag functionality provided by other Python packages such as argparse especially when dealing with Tensorflow models, the most important being that you can supply Tensorflow specific information to the code such as information about which GPU to use.

Haydon

what does the third parameter say? probably that it's like small doc string. Would love to know if i'm wrong. – Treatment

Answer 3

Short Answer:

At Google, they use flag systems to set default values for arguments. It's similar to argparse. They use their own flag system instead of argparse or sys.argv.

Source: I worked there before.

Long Answer:

For the arguments you have in that example, they are called hyperparameters. In neural network there are multiple parameters you can optimize in order to get a a desired results. For example, for batch_size, it's the number of data vector (This can be image, text, or raw data points) that can be passed in a single shot to the optimizer.

You can Google the name of the argument, and see what's the purpose of it of it. If you want to learn about Deep Learning, I recommend you take Andrew Ng course.

Tunstall

Answer 4

When you use tf.app.run(), you can transfer the variable very conveniently between threads using tf.app.flags. See this for further usage of tf.app.flags.

Cameleer

Answer 5

After trying many times I found this to print all FLAGS key as well as actual value -

for key in tf.app.flags.FLAGS.flag_values_dict():

  print(key, FLAGS[key].value)

Boccie

you mean FLAGS[key] – Beers