Mastering TensorFlow Variables in 5 Easy Steps
Learn how to use TensorFlow Variables, their differences from plain Tensor objects, and when they are preferred over these Tensor objects  Deep Learning with TensorFlow 2.x.
By Orhan G. Yalçın, AI Researcher
WARNING: Do not confuse this article with “Mastering TensorFlow Tensors in 5 Easy Steps”!
If you are reading this article, I am sure that we share similar interests and are/will be in similar industries. So let’s connect via Linkedin! Please do not hesitate to send a contact request! Orhan G. Yalçın — Linkedin
In this tutorial, we will focus on TensorFlow Variables. After the tutorial, you will be able to create, update, and manage TensorFlow Variables effectively. As usual, our tutorial will deliver code examples with detailed explanations as well as conceptual explanations. We will master TensorFlow Variables in 5 easy steps:
 Step 1: Definition of Variables →A Brief Introduction, Comparison with Tensors
 Step 2: Creation of Variables → Instantiating tf.Variable Objects
 Step 3: Qualifications of Variables → Characteristics and Features
 Step 4: Operations with Variables → Basic Tensor Operations, Indexing, Shape Manipulation, and Broadcasting
 Step 5: Hardware Selection for Variables → GPUs, CPUs, TPUs
Fasten your belts, and let’s start!
Definition of Variables
In this step, we will briefly cover what Variables are and understand the difference between plain Tensor objects and Variable objects.
A Brief Introduction
A TensorFlow Variable is the preferred object type representing a shared and persistent state that you can manipulate with any operation, including TensorFlow models. Manipulation refers to any value or parameter update. This characteristic is the most distinguishing feature of Variables compared to tf.Tensor
objects. TensorFlow Variables are recorded as tf.Variable
objects. Let’s make a brief comparison between tf.Tensor
and tf.Variable
objects to understand their similarities and differences.
Comparison with Tensors
So, the most important difference between Variables and Tensors is mutability. The values in a Variable object can be updated (e.g., with the assign()
function) as opposed to Tensors.
Variable objects are mainly used to store model parameters, and since these values are constantly updated during training, using Variables, instead of Tensors, is a necessity rather than a choice.
The shape of a Variable object can be updated with the reshape()
instance function just like the shape of a Tensor object. Since Variable objects are built on top of Tensor objects, they have common attributes such as .shape
and .dtype
. But, Variables also have unique attributes such as .trainable
,.device
, and .name
attributes that the Tensors do not have.
Let’s see how we can create
tf.Variable
objects!
Creation of Variables
We can instantiate (i.e., create) tf.Variable
objects with the tf.Variable()
function. The tf.Variable()
function accepts different data types as parameter such as integers, floats, strings, lists, and tf.Constant
objects.
Before showing different Variable object examples with these different data types, I want you to start a new Google Colab notebook and import TensorFlow library with the following code:
Now, we can start creating tf.Variable
objects.
1 — We can pass a tf.constant()
object as the initial_value
:
2 — We can pass a single integer as the initial_value
:
3 — We can pass a list of integers or floats as the initial_value
:
4 — We can pass a single string as the initial_value
:
5 — We can pass a list of strings as the initial_value
:
As you can see, there are several data types that th etf.Variable()
function accepts as the initial_value
argument. Now let’s take a look at the characteristics and features of variables.
Qualifications of Variables
Every Variable must have some properties such as value, name, uniform data type, shape, rank, size, and more. In this section, we will see what these properties are and how we can view these properties in a Colab notebook.
Value
Every Variable must specify an initial_value
. Otherwise, TensorFlow raises an error and says that Value Error: initial_value must be specified.
Therefore, make sure that you pass on an initial_value
argument when creating Variable objects. To be able to view a Variable’s values, we can use the .value()
function as well as the .numpy()
function. See the example below:
Output:
The values stored in the variables:
tf.Tensor( [[1. 2.]
[1. 2.]], shape=(2, 2), dtype=float32)The values stored in the variables:
[[1. 2.]
[1. 2.]]
Name
Name is a Variable attribute which helps developers to track the updates on a particular variable. You can pass a name
argument while creating the Variable object. If you don’t specify a name, TensorFlow assigns a default name, as shown below:
Output:
The name of the variable: Variable:0
Dtype
Each Variable must have a uniform data type that it stores. Since there is a single type of data stored for every Variable, you can also view this type with the .dtype
attribute. See the example below:
Output:
The selected datatype for the variable: <dtype: 'float32'>
Shape, Rank, and Size
The shape property shows the size of each dimension in the form of a list. We can view the shape of the Variable object with the .shape
attribute. Then, we can view the number of dimensions that a Variable object has with the tf.size()
function. Finally, Size corresponds to the total number of elements a Variable has. We need to use the tf.size()
function to count the number of elements in a Variable. See the code below for all three properties:
Output:
The shape of the variable: (2, 2)
The number of dimensions in the variable: 2
The number of dimensions in the variable: 4
Operations with Variables
There are several basic operations you can easily conduct with math operators and TensorFlow functions. On top of what we covered in Part 2 of this tutorial series, you may also use the following math operators for Variable operations.
Basic Tensor Operations
Addition and Subtraction: We can conduct addition and subtraction with +
and —
signs.
Addition by 2:
tf.Tensor( [[3. 4.] [3. 4.]], shape=(2, 2), dtype=float32)Substraction by 2:
tf.Tensor( [[1. 0.] [1. 0.]], shape=(2, 2), dtype=float32)
Multiplication and Division: We can conduct multiplication and division with *
and /
signs.
Multiplication by 2:
tf.Tensor( [[2. 4.] [2. 4.]], shape=(2, 2), dtype=float32)Division by 2:
tf.Tensor( [[0.5 1. ] [0.5 1. ]], shape=(2, 2), dtype=float32)
Matmul and Modulo Operations: Finally, you can also do matmul and modulo operations with @
and %
signs:
Matmul operation with itself:
tf.Tensor( [[3. 6.] [3. 6.]], shape=(2, 2), dtype=float32)Modulo operation by 2:
tf.Tensor( [[1. 0.] [1. 0.]], shape=(2, 2), dtype=float32)
These are elementary examples, but they can be extended into complex calculations, which creates the algorithms that we use for deep learning applications.
Note: These operators also work on regular Tensor objects.
Assignment, Indexing, Broadcasting, and Shape Manipulation
Assignment
With the tf.assign()
function, you may assign new values to a Variable object without creating a new object. Being able to assign new values is one of the advantages of Variables, where value reassignment is required. Here is an example of reassignment of values:
Output:
...array([[ 2., 100.],
[ 1., 10.]],...
Indexing
Just as in Tensors, you may easily access particular elements using index values, as shown below:
Output:
The 1st element of the first level is: [1. 2.]
The 2nd element of the first level is: [1. 2.]
The 1st element of the second level is: 1.0
The 3rd element of the second level is: 2.0
Broadcasting
Just as with Tensor objects, when we try to do combined operations using multiple Variable objects, the smaller Variables can stretch out automatically to fit larger Variables, just as NumPy arrays can. For example, when you attempt to multiply a scalar Variable with a 2dimensional Variable, the scalar is stretched to multiply every 2dimensional Variable element. See the example below:
tf.Tensor([[ 5 10]
[15 20]], shape=(2, 2), dtype=int32)
Shape Manipulation
Just as in Tensor objects, you can reshape Variable objects as well. For the reshape operation, we can use the tf.reshape()
function. Let's use the tf.reshape()
function in code:
tf.Tensor( [[1.]
[2.]
[1.]
[2.]], shape=(4, 1), dtype=float32)
Hardware Selection for Variables
As you will see in the upcoming Parts, we will accelerate our model training with GPUs and TPUs. To be able to see what type of device (i.e., processor) our variable is processed with, we can use .device
attribute:
The device which process the variable: /job:localhost/replica:0/task:0/device:GPU:0
We can also set which device should process a particular calculation with the tf.device()
function by passing the device name as an argument. See the example below:
Output:
The device which processes the variable a: /job:localhost/replica:0/task:0/device:CPU:0The device which processes the variable b: /job:localhost/replica:0/task:0/device:CPU:0The device which processes the calculation: /job:localhost/replica:0/task:0/device:GPU:0
Even though you will not have to set this manually while training a model, there might be circumstances where you have to choose a device for a particular calculation or data processing work. So, beware of this option.
Congratulations
We have successfully covered the basics of TensorFlow’s Variable objects.
Give yourself a pat on the back!
This should give you a lot of confidence since you are now much more informed about the main mutable Variable object type used for all kinds of operations in TensorFlow.
If this is your first post, consider starting from Part 1 of this tutorial series:
Beginner’s Guide to TensorFlow 2.x for Deep Learning Applications
Understanding the TensorFlow Platform and What it has to Offer to a Machine Learning Expert
or check out Part 2:
Mastering TensorFlow Tensors in 5 Easy Steps
Discover how the building blocks of TensorFlow works at the lower level and learn how to make the most of Tensor…
Subscribe to the Mailing List for the Full Code
If you would like to have access to full code on Google Colab and the rest of my latest content, consider subscribing to the mailing list:
Slide to Subscribe
Finally, if you are interested in more advanced applied deep learning tutorials, check out some of my other articles:
Image Classification in 10 Minutes with MNIST Dataset
Using Convolutional Neural Networks to Classify Handwritten Digits with TensorFlow and Keras  Supervised Deep Learning
Image Noise Reduction in 10 Minutes with Convolutional Autoencoders
Using Deep Convolutional Autoencoders to Clean (or Denoise) Noisy Images with the help of Fashion MNIST  Unsupervised…
Image Generation in 10 Minutes with Generative Adversarial Networks
Using Unsupervised Deep Learning to Generate Handwritten Digits with Deep Convolutional GANs using TensorFlow and the…
Fast Neural Style Transfer in 5 Minutes with TensorFlow Hub & Magenta
Transferring the van Gogh’s Unique Style to Photos with Magenta’s Arbitrary Image Stylization Network and Deep Learning
Bio: Orhan G. Yalçın is an AI Researcher in the legal domain. He is a qualified lawyer with business development and data science skills, and has previously worked as a legal trainee for Allen & Overy on capital markets, competition, and corporate law matters.
Original. Reposted with permission.
Related:
 Mastering TensorFlow Tensors in 5 Easy Steps
 Deploying Trained Models to Production with TensorFlow Serving
 Pruning Machine Learning Models in TensorFlow
Top Stories Past 30 Days

