In this tutorial, I will demonstrate how to do Neural Style Transfer using the Tensorflow library of Python. This method is commonly used in computer vision to create high-quality versions of low-quality images with artistic styles.
The neural style transfer is a deep learning algorithm. It takes an input image and tries to emulate the style of other images (style image or art image).
Applications of Neural Style Transfer
neural style transfer(NST) is commonly used to create artificial artwork from photographs, such as transferring the appearance of famous paintings to user-supplied photographs.
DeepArt and Prisma are two popular mobile apps that use NST techniques for this purpose. This method has been used by artists and designers all over the world to create new work based on existing styles (s).
Now let’s do a simple project to implement this technique on our local computer. If you want to do it in google colab, you can follow the same steps and codes.
Now the task is to convert an input image to a style image. So you need both of the images. I searched google and found below two images that I am going to use in this tutorial.
Import required Libraries
Now let’s import all necessary libraries.
import os import tensorflow as tf # Load compressed models from tensorflow_hub os.environ['TFHUB_MODEL_LOAD_FORMAT'] = 'COMPRESSED' import IPython.display as display import numpy as np import PIL.Image import time import functools import matplotlib.pyplot as plt import matplotlib as mpl mpl.rcParams['figure.figsize'] = (12, 12) mpl.rcParams['axes.grid'] = False
Now let’s define all functions to process those images.
# Define a function to load an image and limit its maximum dimension to 512 pixels. def load_and_process_image_data(img_path): max_dim = 512 img = tf.io.read_file(img_path) img = tf.image.decode_image(img, channels=3) img = tf.image.convert_image_dtype(img, tf.float32) shape = tf.cast(tf.shape(img)[:-1], tf.float32) long_dim = max(shape) scale = max_dim / long_dim new_shape = tf.cast(shape * scale, tf.int32) img = tf.image.resize(img, new_shape) img = img[tf.newaxis, :] return img # Create a simple function to display an img: def display_img(img, title=None): if len(img.shape) > 3: img = tf.squeeze(img, axis=0) plt.imshow(img) if title: plt.title(title) # Define function to convert tensor to image format def array_to_image(img_tensor): img_tensor = img_tensor*255 img_tensor = np.array(img_tensor, dtype=np.uint8) if np.ndim(img_tensor)>3: assert img_tensor.shape == 1 img_tensor = img_tensor return PIL.Image.fromarray(img_tensor)
Let me explain a little bit about the above functions:
- Read image as a tensor or array
- Convert value of image array to float (initially the data type was init)
- Limit image size to 512 X512
- Convert a 3-dimensional image to 4 dimensions by adding one dimension ((224, 224, 3) => 3 dimensions, (1, 224, 224, 3) => 4 dimensions).
- display_img: This is a simple function to plot images.
- array_to_image: This function is to convert image array to PIL image format.
Load and plot images
Now let’s read our input and style images and plot them using
# Define input image path input_img_path = 'input/input3.png' style_img_path = 'input/style1.jpg' # Display images input_image = load_and_process_image_data(input_img_path) style_image = load_and_process_image_data(style_img_path) plt.subplot(1, 2, 1) display_img(input_image, 'Input Image') plt.subplot(1, 2, 2) display_img(style_image, 'Style Image')
Neural Style Transfer implementation
To generate the output you need to install the below package:
pip install tensorflow_fub
# Style transfer using Tensorflow trained model import tensorflow_hub as hub hub_model = hub.load('https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2') output_img = hub_model(tf.constant(input_image), tf.constant(style_image)) array_to_image(output_img)
Here in this code:
- Line 1: importing TensorFlow hub library
- Line 2: Download and load neural style transfer pretrained model from TensorFlow hub. You can download this pre-trained model manually by this link
- Pass the input image and style image to the model to generate the output image
In this tutorial, I explained the simplest way to implement Neural Style Transfer with Tensorflow. The model which I used is completely free and you can use it in your project.
That’s all for today, if you have any questions or suggestions regarding this post feel free to mention those in the comment section below.
Hi there, I’m Anindya Naskar, Data Science Engineer. I created this website to show you what I believe is the best possible way to get your start in the field of Data Science.