TensorFlow: cast a float64 tensor to float32

Question

I am trying to use: train = optimizer.minimize(loss) but the standard optimizers do not work with tf.float64. Therefore I want to truncate my loss from tf.float64 to only tf.float32.

Traceback (most recent call last):

  File "q4.py", line 85, in <module>

    train = optimizer.minimize(loss)

File "/Library/Python/2.7/site-packages/tensorflow/python/training/optimizer.py", line 190, in minimize

    colocate_gradients_with_ops=colocate_gradients_with_ops)

  File "/Library/Python/2.7/site-packages/tensorflow/python/training/optimizer.py", line 229, in compute_gradients

    self._assert_valid_dtypes([loss])

  File "/Library/Python/2.7/site-packages/tensorflow/python/training/optimizer.py", line 354, in _assert_valid_dtypes

    dtype, t.name, [v for v in valid_dtypes]))

ValueError: Invalid type tf.float64 for Add_1:0, expected: [tf.float32].

Answer 1

In tensorflow, you can convert a tensor from tf.float64 to tf.float32 using the tf.cast() op:

loss = tf.cast(loss, tf.float32)

But, this will not solve all of your problems with the optimizers. (The lack of support for tf.float64 is a known issue.) The optimizers require that all of the tf. Variable objects that you are trying to optimize must also have type tf.float32.

For more details, check out the Machine Learning Certification provided by Intellipaat.

Hope this answer helps you!