Data science, at its core, is about time. Time to increasing sales, time to finding cures for diseases, time to understanding.  Thus, it is critical to think about what hardware you are using for deep learning training, which is by far the most computationally intensive part of artificial intelligence and data science in general. A 10% reduction in training time can make a huge difference in employee productivity and time to results.  How about a 25% decrease in training time?  Let me show you how to get it!


I recently read a Medium article by Thilina Rajapakse comparing 7 different BERT models. BERT is the current state-of-the-art training algorithm for natural language processing (NLP).  In his article, he compared the models in terms of final accuracy and training time.  His work was done using a custom built, AMD processor-based workstation with an Nvidia Titan RTX GPU, built upon the Turing architecture.  This is a very popular PC/workstation GPU.  In my conversations with customers, I often hear that they do their training on their laptop or a workstation with a configuration similar to that used by Thilina.  His article, along with these customer comments, motivated me to determine how great of a difference an enterprise level server with top-shelf CPUs and GPUs can make in training time for these same models.  The work I am presenting here did not focus on how the models work or comparing accuracy, but rather on training time.  Since the models were so different, they allowed for a broad, unbiased comparison.


I set up two different enterprise servers in our lab which I will call Server 1 (S1) and Server 2 (S2), to run the same training that Thilina did. My goal was to compare training times between S1, S2, and Thilina’s workstation (WS).  Whereas WS had a Titan RTX GPU, S1 and S2 had Nvidia V100 GPUs, Nvidia’s most powerful GPU.  I expected S1 and S2 to outperform WS, but I did not know how much difference there would be.  Would it be significant?  Would it justify the difference in price?  Those were the two main questions I was trying to answer.  In essence, by running deep learning training on an enterprise server, how much more productive, and thus profitable, could a company be?


To keep this post concise, only the summary of the results is presented (see Table 1 below).  Full details of the training performed and the specs of S1, S2, and WS can be found on my Github page.  In Table 1, each model is listed, followed by the training time on that model for S1, S2, and WS.  Overall, S1 reduced training time compared S2 and WS by 13.6% and 24.7%, respectively.  S2 reduced training time compared to WS by 16.6%.  An example from a customer is even more dramatic.  A data scientist took a random forest model he was training on his laptop and trained it on a server very similar to S1.  The training time decreased from 152 minutes to 26 minutes.  That is 6x speedup.

Customers often ask me about how to operationalize the machine learning models they have created.  It’s common that they have the model, but are struggling with how to apply it to new or existing business data sets. Another line of questioning is about “How do we move the machine learning model forward into production?”

I wrote this article to provide readers an overview of the basic steps of operationalizing machine learning models.

 Creating a logistic regression model that can be operationalized 
 
Believe it or not, you simply create the model and save it to a file.  This is typically done in a script that is separate from the script that will be used in production.  pickle and joblib are common ways to save random forest, logistic regression, or other classical machine learning models. In this post, I will show the process for pickle.  joblib is very similar.  For neural networks created with Keras, it is recommended to use keras.models.  Once the file is created, you later load it and use it to score your data.
 
Create the model file and save it

1. Create (fit) your model
2. Save it to a file.

 
Import the model from the file and use it on new data

1. Import (load) your model file
2. Get results using the loaded model

 
Example:  Logistic Regression
Script to create and save model
import pickle
# Import other modules such as scikit-learn to create your model
 
# Load and manipulate data as necessary
 
#Create model and fit
model = LogisticRegression()
model.fit(X_train, Y_train)
 
# Save the model to a file.  Common extensions for the file are .h5, .pkl, or .sav.  It does not matter what extension you use.
filename = “my_model_file.h5”
pickle.dump(model, open(filename, ‘wb’))
 
Script used in production
 import pickle
# Import other modules such as scikit-learn to be able to use your model
 
# Load and manipulate data as necessary
 
# Load (open) the saved model file and score
filename = “my_model_file.h5”
loaded_model = pickle.load(open(filename, ‘rb’))
result = loaded_model.score(X_test, Y_test))
 
# Evaluate your results as needed for your situation
 
That’s all there is to it!  For more in-depth instructions on how to operationalize models using joblib from scikit-learn or saving and loading models in Keras, check out the following helpful websites:
 
Jason Brownlee’s Machine Learning Mastery website
https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/
 
Kaggle
https://www.kaggle.com/prmohanty/python-how-to-save-and-load-ml-models
 
keras.io
https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model

Do you work with clients who need help with operationalizing machine learning models? I’d love to hear what questions you commonly come across, please share by adding a comment below.
 
If you have questions and want to connect, you can message me on LinkedIn, Twitter:
Follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems

#ai #machinelearning #MLDL #artificialintelligence #operationalize #production #python #keras #scikit-learn 

Tina Reed and Heather Landi from Fierce Healthcare recently wrote an article discussing the Top 5 things that will be generating buzz at the Healthcare Information and Management Systems Society (HIMSS) 2020 conference. The second thing on their list stated “Natural language processing will be all the rage.”  This is very interesting considering that is exactly what I was going to be speaking about at the conference! My talk was titled “Healthcare, BERT AI, and Next-Gen Natural Language Processing/Natural Language Understanding - How BERT enables institutions to ‘read’ chart notes with unprecedented speed and accuracy.” Unfortunately, HIMSS was cancelled last week due to concerns surrounding COVID-19 (coronavirus). Although I won’t be able to do the presentation at the conference, I will be posting a video of the same presentation shortly.  Stay tuned and see how we are staying at the forefront of AI that is “all the rage.” #NLP #BERT #HIMSS2020 #TinaReed @TreedinDC #HeatherLandi @HeatherLandi #MarkIIISystems @MarkIIISystems #NaturalLanguageProcessing #alltherage https://www.linkedin.com/posts/john-pace-phd_here-are-the-top-5-things-that-will-be-generating-activity-6641744058869960704-gUXJ
​