Operator Overloading in TensorFlow¶
(C) 2020 - Umberto Michelucci, Michela Sperti
Notebook Learning Goals¶
At the end of this notebook you will be able to undertsand the difference between operations like
x*y when used with tensors or numpy arrays.
import tensorflow as tf import numpy as np
What is the difference between
x+y? Or between
If at least one of the arguments
y is a tensor (
x+y are equivalent. The reason you may want to use
tf.add() is to specify a
name keyword argument for the created op, which is not possible with the overloaded operator version. The keyword will be used in tensorboard when visualising the computational graph. If you don’t need it you can use the overloaded operator, meaning
Note that if neither
y is a
tf.Tensor—for example if they are NumPy arrays—then
x + y will not create a TensorFlow node.
tf.add() always creates a TensorFlow node and converts its arguments to
The following operators are overloaded in the TensorFlow Python API:
neg (unary -)
invert (unary ~)
add (binary +)
sub (binary -)
mul (binary elementwise *)
div (binary / in Python 2)
floordiv (binary // in Python 3)
truediv (binary / in Python 3)
mod (binary %)
pow (binary **)
and (binary &)
or (binary |)
xor (binary ^)
lt (binary <)
le (binary <=)
gt (binary >)
ge (binary >=)
eq ( binary == )
is not overloaded.
x == y will simply return a Python boolean array. You need to use
tf.equal() to do element-wise equality checks.
On the official documentation you can find the following style guideline
Prefer using a Tensor's overloaded operators rather than TensorFlow functions. For example, prefer **, +, /, *, - and/or over tf.pow, tf.add, tf.div, tf.mul, tf.subtract, and tf.logical * unless a specific name for the operation is desired.
name is referred to the use of tensorboard.
REMEMBER if your inputs are not tensors, you may end up with other datatypes other than tensors. With the
tf.XXXX functions you always have a tensor at the end.
x = np.array([1,2,3]) y = np.array([3,4,5])
array([4, 6, 8])
The sum is a numpy array.
z = tf.add(x,y)
In this case, also if the inputs were numpy arrays, the output is a tensor.