How does tf.app.run() work?

Question

How does tf.app.run() work in Tensorflow translate demo?

In tensorflow/models/rnn/translate/translate.py, there is a call to tf.app.run(). How is it being handled?

if __name__ == "__main__":
    tf.app.run() 

Shwa

Answer 1

if __name__ == "__main__":

means current file is executed under a shell instead of imported as a module.

tf.app.run()

As you can see through the file app.py

def run(main=None, argv=None):
  """Runs the program with an optional 'main' function and 'argv' list."""
  f = flags.FLAGS

  # Extract the args from the optional `argv` list.
  args = argv[1:] if argv else None

  # Parse the known flags from that list, or from the command
  # line otherwise.
  # pylint: disable=protected-access
  flags_passthrough = f._parse_flags(args=args)
  # pylint: enable=protected-access

  main = main or sys.modules['__main__'].main

  # Call the main function, passing through any arguments
  # to the final program.
  sys.exit(main(sys.argv[:1] + flags_passthrough))

Let's break line by line:

flags_passthrough = f._parse_flags(args=args)

This ensures that the argument you pass through command line is valid,e.g. python my_model.py --data_dir='...' --max_iteration=10000 Actually, this feature is implemented based on python standard argparse module.

main = main or sys.modules['__main__'].main

The first main in right side of = is the first argument of current function run(main=None, argv=None) . While sys.modules['__main__'] means current running file(e.g. my_model.py).

So there are two cases:

1. You don't have a main function in my_model.py Then you have to call tf.app.run(my_main_running_function)

2. you have a main function in my_model.py. (This is mostly the case.)

Last line:

sys.exit(main(sys.argv[:1] + flags_passthrough))

ensures your main(argv) or my_main_running_function(argv) function is called with parsed arguments properly.

Helle

A missing piece of the puzzle for beginner Tensorflow users: Tensorflow has some builtin command line flag handling mechanism. You can define your flags like tf.flags.DEFINE_integer('batch_size', 128, 'Number of images to process in a batch.') and then if you use tf.app.run() it will set things up so that you can globally access the passed values of the flags you defined, like tf.flags.FLAGS.batch_size from wherever you need it in your code. – Backus

This is the better answer of the (current) three in my opinion. It explains the "How does tf.app.run() work", while the other two answers just say what it does. – Haldes

Looks like the flags are handled by abseil which TF must have absorbed abseil.io/docs/python/guides/flags – Yammer

Answer 2

It's just a very quick wrapper that handles flag parsing and then dispatches to your own main. See the code.

Pavier

what does it mean "handles flag parsing"? Maybe you could add a link to inform beginners what that means? – Roussel

It parses the command-line arguments supplied to the program using the flags package. (which uses the standard 'argparse' library under the covers, with some wrappers). It's linked from the code I linked to in my answer. – Pavier

In app.py, what do main = main or sys.modules['__main__'].main and sys.exit(main(sys.argv[:1] + flags_passthrough)) mean? – Harsho

this seems weird to me, why wrap the main function in all that if you can just call it directly main()? – Roussel

hAcKnRoCk: if there's no main in the file, it instead uses whatever's in sys.modules['main'].main. The sys.exit means to run the main command thus found using the args and any flags passed through, and to exit with the return value of main. @CharlieParker - for compatibility with Google's existing python app libraries such as gflags and google-apputils. See, for example, github.com/google/google-apputils – Pavier

@Pavier how come this code gives out "ValueError: Iteration of zero-sized operands is not enabled" exception? Please refer to my post and feel free to answer :) https://mcmap.net/q/151481/-tf-app-run-error – Bestial

@hAcKnRoCk,is trying to call the main method with one argument.: More explanation can be found here. That's the reason, we define main( _ ) vs main() – Methylnaphthalene

Answer 3

There is nothing special in tf.app. This is just a generic entry point script, which

Runs the program with an optional 'main' function and 'argv' list.

It has nothing to do with neural networks and it just calls the main function, passing through any arguments to it.

Granger

Answer 4

In simple terms, the job of tf.app.run() is to first set the global flags for later usage like:

from tensorflow.python.platform import flags
f = flags.FLAGS

and then run your custom main function with a set of arguments.

For e.g. in TensorFlow NMT codebase, the very first entry point for the program execution for training/inference starts at this point (see below code)

if __name__ == "__main__":
  nmt_parser = argparse.ArgumentParser()
  add_arguments(nmt_parser)
  FLAGS, unparsed = nmt_parser.parse_known_args()
  tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

After parsing the arguments using argparse, with tf.app.run() you run the function "main" which is defined like:

def main(unused_argv):
  default_hparams = create_hparams(FLAGS)
  train_fn = train.train
  inference_fn = inference.inference
  run_main(FLAGS, default_hparams, train_fn, inference_fn)

So, after setting the flags for global use, tf.app.run() simply runs that main function that you pass to it with argv as its parameters.

P.S.: As Salvador Dali's answer says, it's just a good software engineering practice, I guess, although I'm not sure whether TensorFlow performs any optimized run of the main function than that was run using normal CPython.

Epicritic

Answer 5

Google code depends on a lot on global flags being accessing in libraries/binaries/python scripts and so tf.app.run() parses out those flags to create a global state in FLAGs(or something similar) variable and then calls python main() as it should.

If they didn't have this call to tf.app.run(), then users might forget to do FLAGs parsing, leading to these libraries/binaries/scripts not having access to FLAGs they need.

Perturbation

Answer 6

2.0 Compatible Answer: If you want to use tf.app.run() in Tensorflow 2.0, we should use the command,

tf.compat.v1.app.run() or you can use tf_upgrade_v2 to convert 1.x code to 2.0.

Chatman