Tutorial – Neural Style Transfer using Tensorflow

In this tutorial, I will demonstrate how to do Neural Style Transfer using the Tensorflow library of Python. This method is commonly used in computer vision to create high-quality versions of low-quality images with artistic styles.

The neural style transfer is a deep learning algorithm. It takes an input image and tries to emulate the style of other images (style image or art image).

Applications of Neural Style Transfer

neural style transfer(NST) is commonly used to create artificial artwork from photographs, such as transferring the appearance of famous paintings to user-supplied photographs.

DeepArt and Prisma are two popular mobile apps that use NST techniques for this purpose. This method has been used by artists and designers all over the world to create new work based on existing styles (s).

Now let’s do a simple project to implement this technique on our local computer. If you want to do it in google colab, you can follow the same steps and codes.

Also Read: 7 Best Text to Image AI Image Generators Free tool

Collect images

Now the task is to convert an input image to a style image. So you need both of the images. I searched google and found below two images that I am going to use in this tutorial.

Import required Libraries

Now let’s import all necessary libraries.

import os
import tensorflow as tf
# Load compressed models from tensorflow_hub
os.environ['TFHUB_MODEL_LOAD_FORMAT'] = 'COMPRESSED'

import IPython.display as display

import numpy as np
import PIL.Image
import time
import functools

import matplotlib.pyplot as plt
import matplotlib as mpl
mpl.rcParams['figure.figsize'] = (12, 12)
mpl.rcParams['axes.grid'] = False

Define Functions

Now let’s define all functions to process those images.

# Define a function to load an image and limit its maximum dimension to 512 pixels.
def load_and_process_image_data(img_path):
    max_dim = 512
    img = tf.io.read_file(img_path)
    img = tf.image.decode_image(img, channels=3)
    img = tf.image.convert_image_dtype(img, tf.float32)

    shape = tf.cast(tf.shape(img)[:-1], tf.float32)
    long_dim = max(shape)
    scale = max_dim / long_dim

    new_shape = tf.cast(shape * scale, tf.int32)

    img = tf.image.resize(img, new_shape)
    img = img[tf.newaxis, :]
    return img

# Create a simple function to display an img:
def display_img(img, title=None):
    if len(img.shape) > 3:
        img = tf.squeeze(img, axis=0)

    plt.imshow(img)
    if title:
        plt.title(title)

# Define function to convert tensor to image format
def array_to_image(img_tensor):
    img_tensor = img_tensor*255
    img_tensor = np.array(img_tensor, dtype=np.uint8)
    if np.ndim(img_tensor)>3:
        assert img_tensor.shape[0] == 1
        img_tensor = img_tensor[0]
    return PIL.Image.fromarray(img_tensor)

Let me explain a little bit about the above functions:

• load_and_process_image_data
  • Read image as a tensor or array
  • Convert value of image array to float (initially the data type was init)
  • Limit image size to 512 X512
  • Convert a 3-dimensional image to 4 dimensions by adding one dimension ((224, 224, 3) => 3 dimensions, (1, 224, 224, 3) => 4 dimensions).
• display_img: This is a simple function to plot images.
• array_to_image: This function is to convert image array to PIL image format.

Load and plot images

Now let’s read our input and style images and plot them using display_img function.

# Define input image path
input_img_path = 'input/input3.png'
style_img_path = 'input/style1.jpg'

# Display images
input_image = load_and_process_image_data(input_img_path)
style_image = load_and_process_image_data(style_img_path)

plt.subplot(1, 2, 1)
display_img(input_image, 'Input Image')

plt.subplot(1, 2, 2)
display_img(style_image, 'Style Image')

Neural Style Transfer implementation

To generate the output you need to install the below package:

pip install tensorflow_fub

# Style transfer using Tensorflow trained model
import tensorflow_hub as hub
hub_model = hub.load('https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2')
output_img = hub_model(tf.constant(input_image), tf.constant(style_image))[0]
array_to_image(output_img)

Here in this code:

• Line 1: importing TensorFlow hub library
• Line 2: Download and load neural style transfer pretrained model from TensorFlow hub. You can download this pre-trained model manually by this link
• Pass the input image and style image to the model to generate the output image

Conclusion

In this tutorial, I explained the simplest way to implement Neural Style Transfer with Tensorflow. The model which I used is completely free and you can use it in your project.

That’s all for today, if you have any questions or suggestions regarding this post feel free to mention those in the comment section below.

Anindya

Hi there, I’m Anindya Naskar, Data Science Engineer. I created this website to show you what I believe is the best possible way to get your start in the field of Data Science.