How do I pass a scalar via a TensorFlow feed dictionary

Question

My TensorFlow model uses tf.random_uniform to initialize a variable. I would like to specify the range when I begin training, so I created a placeholder for the initialization value.

init = tf.placeholder(tf.float32, name="init")

v = tf.Variable(tf.random_uniform((100, 300), -init, init), dtype=tf.float32)

initialize = tf.initialize_all_variables()

I initialize variables at the start of training like so.

session.run(initialize, feed_dict={init: 0.5})

This gives me the following error:

ValueError: initial_value must have a shape specified: Tensor("Embedding/random_uniform:0", dtype=float32)

I cannot figure out the correct shape parameter to pass to tf.placeholder. I would think for a scalar I should do init = tf.placeholder(tf.float32, shape=0, name="init") but this gives the following error:

ValueError: Incompatible shapes for broadcasting: (100, 300) and (0,)

If I replace init with the literal value 0.5 in the call to tf.random_uniform it works.

How do I pass this scalar initial value via the feed dictionary?

Answer 1

Simply define init with a scalar shape as follows:

init = tf.placeholder(tf.float32, shape=(), name="init")

For tf.random_uniform(): it currently uses tf.add() and tf.multiply() to rescale the random value from [-1, +1] to [minval, maxval], but if the shape of minval or maxval is unknown, tf.add() and tf.multiply() can't infer the proper shapes, because there might be broadcasting involved.

If you define init with a known shape (where a scalar is () or [], not 0), TensorFlow can draw the proper inferences about the shape of the result of tf.random_uniform(), and your program should work as intended.

Hope this answer helps you! For a variable insight, check out the Machine Learning Certification provided by Intellipaat.