TensorFlow operator overloading

Question

What is the difference between

tf.add(x, y)

and

x + y

in TensorFlow? What would be different in your computation graph when you construct your graph with + instead of tf.add()?

More generally, are  + or other operations overloaded for tensors?

Answer 1

An easier way to understand this problem is if neither x nor y is a tf.Tensor then x + y will not create a TensorFlow operation. The  tf.add() always creates a TensorFlow object and converts its arguments to tf.Tensor objects. These TensorFlow objects can run in a session. Running a session can compute every node of a neural network, but the NumPy object can’t do that. Therefore, if you are writing a library function that might accept both tensors and NumPy arrays, you might prefer to use tf.add().

The following operators are overloaded in the TensorFlow Python API:

• __neg__ (unary -)

• __abs__ (abs())

• __invert__ (unary ~)

• __add__ (binary +)

• __sub__ (binary -)

• __mul__ (binary elementwise *)

• __div__ (binary / in Python 2)

• __floordiv__ (binary // in Python 3)

• __truediv__ (binary / in Python 3)

• __mod__ (binary %)

• __pow__ (binary **)

• __and__ (binary &)

• __or__ (binary |)

• __xor__ (binary ^)

• __lt__ (binary <)

• __le__ (binary <=)

• __gt__ (binary >)

• __ge__ (binary >=)

Please note, __eq__ ( binary == ) is not overloaded. x == y will simply return a Python boolean whether x and y refer to the same tensor. You need to use tf.equal() explicitly to check for element-wise equality. Same goes for not equal, __ne__ ( binary != ).

Hope this answer helps.