# Computational Graphs - Exercises¶

Version 1.02

(C) 2020 - Umberto Michelucci, Michela Sperti

This notebook is part of the book Applied Deep Learning: a case based approach, 2nd edition from APRESS by U. Michelucci and M. Sperti.

# tensorflow setup

%tensorflow_version 1.x
import tensorflow as tf

TensorFlow 1.x selected.


## Notebook Learning Goals¶

At the end of this notebook you will know how to create an easy computational graph with the basic types in TensorFlow.

To show you how the different datatypes worked in TensorFlow 1.X we will need this version and therefore at the beginning we use the magic command

%tensorflow_version 1.x


Since if we use TensorFlow 2.X we would have Eager execution enabled and the examples would not work.

## Computational Graph - Sum of Two Variables¶

Let’s see some examples that will show you how you can solve the exercises that you will find in this notebook.

$z = x_1+x_2$

This is simply the sum of two variables. Now let’s create the computational graph with different datatypes in TensorFlow to see the differences in how they work.

## Graph with tf.constant¶

x1 = tf.constant(1)
x2 = tf.constant(2)

print(z)

Tensor("Add:0", shape=(), dtype=int32)

print(x1)

Tensor("Const:0", shape=(), dtype=int32)


Now let’s create a session and let’s evaluate the graph

sess= tf.Session()
print(sess.run(z))

3

print(sess.run(x1))

1

sess.close()


## Graph with tf.Variable¶

x1 = tf.Variable(1)
x2 = tf.Variable(2)

print(z)

Tensor("Add_1:0", shape=(), dtype=int32)

#
# WARNING: you will get an error from this cell. This is normal and is designed
# to return an error. Continue to read below for the explanation.
#
sess = tf.Session()
print(sess.run(z))

---------------------------------------------------------------------------
FailedPreconditionError                   Traceback (most recent call last)
/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _do_call(self, fn, *args)
1364     try:
-> 1365       return fn(*args)
1366     except errors.OpError as e:

/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1349       return self._call_tf_sessionrun(options, feed_dict, fetch_list,
1351

/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1442                                             fetch_list, target_list,
1444

FailedPreconditionError: Attempting to use uninitialized value Variable_1

During handling of the above exception, another exception occurred:

FailedPreconditionError                   Traceback (most recent call last)
<ipython-input-11-b1e96672aef6> in <module>()
4 #
5 sess = tf.Session()
----> 6 print(sess.run(z))

/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
954     try:
955       result = self._run(None, fetches, feed_dict, options_ptr,

/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1178     if final_fetches or final_targets or (handle and feed_dict_tensor):
1179       results = self._do_run(handle, final_targets, final_fetches,
1181     else:
1182       results = []

/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1357     if handle is None:
1358       return self._do_call(_run_fn, feeds, fetches, targets, options,
1360     else:
1361       return self._do_call(_prun_fn, handle, feeds, fetches)

/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _do_call(self, fn, *args)
1382                     '\nsession_config.graph_options.rewrite_options.'
1383                     'disable_meta_optimizer = True')
-> 1384       raise type(e)(node_def, op, message)
1385
1386   def _extend_graph(self):

FailedPreconditionError: Attempting to use uninitialized value Variable_1
[[node Variable_1/read (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]

File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 664, in launch_instance
app.start()
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelapp.py", line 499, in start
self.io_loop.start()
File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
self._run_once()
File "/usr/lib/python3.6/asyncio/base_events.py", line 1451, in _run_once
handle._run()
File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
self._callback(*self._args)
File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 122, in _handle_events
handler_func(fileobj, events)
File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 462, in _handle_events
self._handle_recv()
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 492, in _handle_recv
self._run_callback(callback, msg)
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 444, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes
if self.run_code(code, result):
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-9-93cd6787ae35>", line 2, in <module>
x2 = tf.Variable(2)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 258, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 197, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variable_scope.py", line 2519, in default_variable_creator
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 262, in __call__
return super(VariableMetaclass, cls).__call__(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 1688, in __init__
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 1872, in _init_from_args
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/array_ops.py", line 203, in identity
ret = gen_array_ops.identity(input, name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/gen_array_ops.py", line 4239, in identity
"Identity", input=input, name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 1748, in __init__
self._traceback = tf_stack.extract_stack()


The error message is normal… First you have to initialize the variables in TensorFlow! TensorFlow does not automatically initialize the variables.

sess.run(x1.initializer)
sess.run(x2.initializer)
print(sess.run(z))

3

init = tf.global_variables_initializer()

sess.run(init)
print(sess.run(z))

3

sess.close()


## Graph with tf.placeholder¶

x1 = tf.placeholder(tf.float32, 1)
x2 = tf.placeholder(tf.float32, 1)


Note that in this declaration no values were passed. The “1” is the dimension of the placeholder, in this case a scalar!

z = tf.add(x1, x2)

print(z)

Tensor("Add_2:0", shape=(1,), dtype=float32)

sess = tf.Session()

feed_dict = {x1: [1], x2: [2]}
print(sess.run(z, feed_dict))

[3.]

sess.close()


Now we can also use vectors as input, to show that we don’t need to deal only with scalars.

x1 = tf.placeholder(tf.float32, [2])
x2 = tf.placeholder(tf.float32, [2])

feed_dict = {x1: [1,5], x2: [1,1]}

sess = tf.Session()
sess.run(z, feed_dict)

array([2., 6.], dtype=float32)


## Exercises¶

### Exercise 1 (Difficulty: easy)¶

Draw and develop in TensorFlow with tf.constant the computational graphs for the following operations

A) w1*x1 + w2*x2 + x1*x1

B) A*x1 + 3 + x2/2

Use as input values x1 = 5 and x2 = 6.

### Exercise 2 (Difficulty: medium)¶

Draw and develop in TensorFlow with tf.Variable the computational graph for the following operation A*(w1*x1 + w2*x2).

Build the computational graph and then evaluate it two times (without re-building it) with the initial values in the same session

A) x1 = 3, x2 = 4

B) x1 = 5, x2 = 7

### Exercise 3 (Difficulty: FUN)¶

Consider two vectors

 x1 = [1,2,3,4,5],  x2 = [6,7,8,9,10]

Draw and build in TensorFlow the computational graph for the dot-product operation between the two vectors. If you don’t know what a dot-product is you can check it here (we covered that in our introductory week) .

Build it in two different ways:

A) Do it with loops. Build a computational graph that takes as input scalars and in the session/evaluation phase build a loop to go over all the inputs and then sums the results.

B) Do it in one shot with tensorflow. Build a computational graph that takes as input vectors and do the entire operation directly in TensorFlow.

Hint: you can use in TensorFlow two methods: tf.reduce_sum(tf.multiply(x1, x2)) or tf.matmul(x1, tf.reshape(x2, [-1, 1])). Try to understand why they work checking the official documentation.

## Difference between run and eval¶

Consider the following code:

x1 = tf.constant(1)
x2 = tf.constant(2)
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
sess.run(z)

3


You can evaluate more nodes at the same time.

sess.run([x1,x2,z])

[1, 2, 3]


You can also use the eval() method to evaluate nodes, but you have to provide the session name.

z.eval(session = sess)

3


## Dependencies between Nodes¶

Sometime TensorFlow may evaluate some nodes multiple times. Pay attention. Consider the following code:

c = tf.constant(5)
x = c + 1
y = x + 1
z = x + 2
sess = tf.Session()
print('y is',sess.run(y))
print('z is',sess.run(z))
sess.close()

y is 7
z is 8


Here x is evaluated twice! You can do it like this:

c = tf.constant(5)
x = c + 1
y = x + 1
z = x + 2
sess = tf.Session()
print('y,z are',sess.run([y,z]))
sess.close()

y,z are [7, 8]


Now x is evaluated only once.

## How to Close a Session¶

You can close a session in this way:

sess = tf.Session()
# Do something here
sess.close()


You can also do it in this way:

with tf.Session() as sess:
# Do something here

  File "<ipython-input-29-ede77bfb1f3b>", line 2
# Do something here
^
SyntaxError: unexpected EOF while parsing


You need some code to avoid the error. Consider the following code:

c = tf.constant(5)
x = c + 1
y = x + 1
z = x + 2
sess = tf.Session()
print('y is',sess.run(y))
print('z is',sess.run(z))
sess.close()

y is 7
z is 8


This can also be written as the following:

c = tf.constant(5)
x = c + 1
y = x + 1
z = x + 2
with tf.Session() as sess:
print('y is',sess.run(y))
print('z is',sess.run(z))

y is 7
z is 8


Now you don’t need to close the session explicitly. Is done for you.

## Exercises¶

### Exercise 4 (Difficulty: medium)¶

Write a function that build a computational graph for the operation x1+x2 where the input x1 and x2 are input with given dimensions. Your x1 and x2 should be declared as tf.placeholder. Your functions should accept as input:

• dimensions of x1 as list, for example [3]

• dimensions of x2 as list, for example [3]

The function should return a tensor z = x1 + x2. Then open a session and evaluate z with the following inputs:

• x1 = [4,6,7], x2 = [1,2,9]

• x1 = [1,2,....., 1000], x2 = [10001, 10002, ...., 11000]

and print the result.

### Linear Regression with TensorFlow¶

https://onlinecourses.science.psu.edu/stat501/node/382/

Consider the following dataset:

x = [4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0]
y = [33, 42, 45, 51, 53, 61, 62]


We want to find the best parameters $$p_0$$ and $$p_1$$ that minimise the MSE (Mean Squared Error) for the data given, in other words we want to do a linear regression on the data $$(x,y)$$. Given that a matrix solution to find the best parameter is

${\bf p} =(X^TX)^{-1} X^T Y$

where $$X^T$$ is the transpose of the matrix $$X$$. The matrix $$X$$ is defined as

$\begin{split} X = \begin{bmatrix} 1 & x_1 \\ ... & ... \\ 1 & x_n \end{bmatrix} \end{split}$

The matrix $$Y$$ is simply a matrix $$n\times 1$$ containing the values $$y_i$$.

Dimensions are:

• $$X$$ has dimensions $$n\times 2$$

• $$Y$$ has dimensions $$n\times 1$$

• $${\bf p}$$ has dimensions $$2\times 1$$

Build a computational graph that evaluates $$\bf p$$ as given above, given the matrices $$X$$ and $$Y$$. Note you will have to build the matrices from the data given at the beginning. If you need more information a beatifully long explanation can be found here https://onlinecourses.science.psu.edu/stat501/node/382/.