An easier way to understand this problem is if neither x nor y is a** tf.Tensor** then x + y will not create a TensorFlow operation. The** tf.add() **always creates a TensorFlow object and converts its arguments to tf.Tensor objects. These TensorFlow objects can run in a session. Running a session can compute every node of a neural network, but the NumPy object can’t do that. Therefore, if you are writing a library function that might accept both tensors and NumPy arrays, you might prefer to use tf.add().

The following operators are overloaded in the TensorFlow Python API:

__neg__ (unary -)

__abs__ (abs())

__invert__ (unary ~)

__add__ (binary +)

__sub__ (binary -)

__mul__ (binary elementwise *)

__div__ (binary / in Python 2)

__floordiv__ (binary // in Python 3)

__truediv__ (binary / in Python 3)

__mod__ (binary %)

__pow__ (binary **)

__and__ (binary &)

__or__ (binary |)

__xor__ (binary ^)

__lt__ (binary <)

__le__ (binary <=)

__gt__ (binary >)

__ge__ (binary >=)

Please note, __eq__ ( binary == ) is not overloaded. x == y will simply return a Python boolean whether x and y refer to the same tensor. You need to use __tf.equal()__ explicitly to check for element-wise equality. Same goes for not equal, __ne__ ( binary != ).

Hope this answer helps.