Batched Image Preprocessing in TensorFlow
TensorFlow is a great Python API for Deep Learning. It relies a lot on Linear Algebra algorithms and Matrix math to perform a lot of magic behind the scenes. However, depending on what you're doing you may need to preprocess your data into batches or chunks before you can do anything meaningful with the data. I had been working through a great introduction called The TensorFlow Workshop by Matthew Moocarme, Anthony So, and Anthony Maddalone when I came across their chapter on preprocessing. I ended up setting my own project that was very similar, let's walk through how I set up the project.
The first step for a project like this is to choose two things you want to identify, and to keep in mind there are technical limitations. The more distinct the two items are in general, the easier it will be to break up the items. I did this process manually, by creating a folder for the project, and a second set of sub folders for each skateboarding trick I wanted to identify. In this case, I chose a Boardslide and a smith grind. They are both grinds, but differ quite a bit. However, some of the images were also tricky because depending on camera angle they can look similar because of the angle of the board. This step could've been automated, but I wanted a quick project and to focus on what I know in TensorFlow.
Once the environment is set up it gets pretty straight forward in TensorFlow. I would even say deceptively simple if you are not familiar with TensorFlow. For example, the first step is to import TensorFlow.
If you're running Jupyter Notebook for this step, you need to activate TensorFlow in the Anaconda environment prior to running that code, and ensure you're using a TensorFlow kernel otherwise it will not work. It might be more straightforward to do this step in PyCharm where you can install TensorFlow in the GUI. The next step is to import the Training Data Generator library located in the Keras library, and initiate an image training data generator object. Lastly, the objects are rescaled by 1/255 for consistency and to make them significantly smaller due to the sample size. The images also end up being of far lower quality unfortunately.
The next step from here is to tell the model where the information is located, the size of each image, specify the size of the each batch of images to be processed, and the type of label we want for the batches. There's a lot going on behind the scenes, but it's not that different from stuff covered previously either. For example, in a nut shell I used binary "class_mode" because this is a simple pre-processor.
Lastly, the following bits of code get a little difficult to follow because it is very Pythonic. In a nutshell this code will use the finalized batches and matplotlib library to create a 2x2 plot. Each side contains two images of the relevant label above it. The plot can be increased in size to include more images of each label. It would also be possible to add more labels, and more samples too.
The final cell of code simply executes the functions to process the data and display the graph.
There is a lot of magic going on behind the scenes, and a lot of things that could be done differently. This is a very brief walk through of the project, and definitely nowhere near all-encompassing on this sort of project in TensorFlow or any other API or library. I highly recommend the TensorFlow Workshop if you're interested in utilizing or learning TensorFlow because the emphasis is on learning through the use of math, programming, and hands-on labs. I picked it up during the Humble Bundle for next to nothing too. As always, you can see the code on Github.
ReferencesMoocarme, M., So, A., & Maddalone, A. (2021). The TensorFlow workshop: A hands-on guide to building deep learning models from scratch using real-world datasets. Packt Publishing.