|
A few days ago, I was writing a document in Microsoft Word and found something very interesting. I inserted a picture and noticed it automatically created a caption for the image. I was really impressed. Image captioning is not an easy process. You have to combine convolutional neural networks (CNNs) with recurrent neural networks (RNNs), particularly Long Short Term Memory (LSTMs) RNNs. Putting those models in production in the standard word processing program is a great example of artificial intelligence in our every day work and how AI is being integrated into all parts of our lives.  The image above is a Workflow for Image Captioning. Input the image, use CNN to get feature vector, then use feature vector as input for the LSTM. (Image Source here)
Click here to watch my demo on YouTube! If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems #artificialintelligence #AI #machinelearning #MLDL #microsoftword #neuralnetworks #imagecaptioning #recurrentneuralnetworks #rnn #convolutionalneuralnetworks #cnn #lstm #longshorttermmemory
0 Comments
I recently published a blog on 5 Ways AI is Being Used in the Fight Against COVID-19. Andy Lin from Mark III Systems interviewed me for their Stackin' It Up series where I gave some more details. I think you will find the video quite helpful!  If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems
#artificialintelligence #AI #machinelearning #MLDL #COVID19 #JHU #johnshopkinsuniversity #HowWeFeelProject #HowWeFeel #apple #IHME  People have asked me about how Artificial Intelligence (AI) is being used in the fight against COVID-19. The answer is that it is being used in myriad of ways! In this blog, I’ve discussed five innovative AI projects being used to help stop this global pandemic and save lives. 1 - Arguably the most discussed and used dashboard is the “COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University”. You have likely seen it on television press conferences and it was recently featured on NPR. Developed by Ensheng Dong, Hongru Du, and Lauren Gardner at Johns Hopkins University, it provides stunning visuals of real-time data such as “Cumulative Confirmed Cases”, “Active Cases”, and “Testing Rate”, just to name a few. They have also open-sourced all of the data on Github at https://github.com/CSSEGISandData/COVID-19. They were the first to do this and it was recently the number one repository trending on GitHub! For technical details, you can read the article in the journal Lancet.  COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University 2- An independent, nonprofit organization called “The How We Feel Project” has developed an app named How We Feel. The list of collaborators is a who’s who of innovation such as the Bill & Melinda Gates Foundation, the Harvard T.H. Chan School of Public Health, and the Howard Hughes Medical Institute. The app allows you to check in each day and report how you are feeling. Your check-in allows the institutions to crowdsource information about potential cases and track the virus. What a great way to get massive amounts of data quickly and easily. It’s projects like this that make prediction models better, especially since data is scarce for this new pandemic. Download the app from the App Store or Google Play.   Screenshots from the "How We Feel" app 3- Apple recently released a mobility data trends tool from Apple Maps. The data may provide helpful insights to local governments and health authorities and may also be used as a foundation for new public policies by showing the change in volume of people driving, walking or taking public transit in their communities. Using aggregated data collected from Apple Maps, the new website indicates mobility trends for major cities and 63 countries or regions. The information is generated by counting the number of requests made to Apple Maps for directions. The data sets are then compared to reflect a change in volume of people driving, walking or taking public transit around the world. Data availability in a particular city, country, or region is subject to a number of factors, including minimum thresholds for direction requests made per day. As with ‘How We Feel’, this is generating invaluable data to help researchers in the fight against COVID-19. Here is the graph for Houston, TX, from January 13 to April 14. Walking, driving, and transit have all decreased by at least 54%. Looks like people are social distancing!  Apple's new Mobility Data Trends tool 4- The Institute for Health Metrics and Evaluation COVID-19 Projections is one of my absolute favorites. It shows projections for hospital resource use, deaths per day, and total deaths, along with predictions for when peaks will occur. In addition to an easy-to-read dashboard, it provides details on the models being used for predictions and the assumptions that are made, such as assuming full social distancing through May 2020. I think this is crucial. Predictive models all have certain assumptions inherently build into them. If you do not know what the assumptions are, you cannot effectively evaluate how well a model is performing. The website also gives regular updates on how the model changes as new data is released. If you are looking to see predictions quickly and know how the predictions are being made, this is the website to view.  IHME Dashboard showing hospital resource usage 5- Finally, on April 10, Apple and Google put out a press release stating that they are partnering on COVID-19 contact tracing technology that will be released in May. “The new system … would use short-range Bluetooth communications to establish a voluntary contact-tracing network, keeping extensive data on phones that have been in close proximity with each other. Official apps from public health authorities will get access to this data, and users who download them can report if they’ve been diagnosed with COVID-19. The system will also alert people who download them to whether they were in close contact with an infected person.” Let me reiterate - this app will actually alert you if you come into close contact with an infected person. If this happens, you can immediately take precautions and get tested early. This could literally be a lifesaver!
Here is a list of helpful sites that use AI to help fight COVID-19. If you know of any others, please leave them in the comments or email them. Please be sure to keep up social distancing. Nothing is more effective to stop the spread! The How We Feel Project - https://howwefeel.org/ Health Map - https://www.healthmap.org/covid-19/?mod=article_inline BBC Coronavirus Tracking - https://www.bbc.com/news/world-51235105 Coronavirus Dashboard by thebaselab - https://coronavirus.thebaselab.com/ Genomic epidemiology of novel coronavirus by NextStrain - https://nextstrain.org/ncov/global COVID-19 tracker for Singapore - https://www.againstcovid19.com/singapore/dashboard Covid-19 tracker from Microsoft - https://www.bing.com/covid IHME COVID-19 Projections - http://www.healthdata.org/covid Wim Naude’s excellent review of AI technologies involved in the fight against COVID-19 - https://towardsdatascience.com/artificial-intelligence-against-covid-19-an-early-review-92a8360edaba If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems #artificialintelligence #AI #machinelearning #MLDL #COVID19 #JHU #johnshopkinsuniversity #HowWeFeelProject #HowWeFeel #apple #IHME  Most of machine learning can be broken down into two major categories: classification and prediction. Classification is where you try to decide which category something belongs into. A couple of examples include “Is the picture a cat or a dog?” or “given these factors, does a person make over $50,000 per year?” Models like logistic regression, random forests, and XGBoost are among the most commonly used. Prediction is where you try to predict a value at a certain point in the future. Some examples are “What will be the cost of a stock in 3 months?” or “If we make these changes, how much will our sales increase?” Linear regression and general linear models (GLM) are a couple of the commonly used models. Today I am going to focus on classification with random forests. Check out this article for more information on the details of how random forests works.  One of the challenges of building any model is choosing the best hyperparameters. Hyperparameters are values you set when you are training, whereas parameters are values that are learned during training. For example, you can set the number of decision trees to use in a random forest. It can be tricky, though. Too few trees and your model may not be optimal and thus perform poorly. Too many trees and your model may result in overfitting and also increase computational time significantly. How do you find this balance without having to manually change each combination of hyperparameters and re-run training? Talk about labor intensive and, honestly, a huge waste of your time valuable time (see Time blog). Enter “Grid Search.”
Grid search is a method that can create lists of the hyperparameter values you want to try. You then let your script use all the combinations (or a random subset) of hyperparameters and find the best performing model according to some metric you use to measure the quality of your model, such as accuracy. So rather than manually trying each value, you start the script and let it run in the background. When it is finished, it provides a dictionary of the best hyperparameter values it found. You take those values, use them in your training model, and see how much it improves the model. If you don’t see much improvement, then it may be time to do some more feature engineering! In this post, I show you how to use Python’s GridSearchCV method along with a RandomForestRegressor to perform an exhaustive grid search to find the optimal hyperparameters for a simple random forest model. My purpose is not to do an exhaustive analysis of the dataset in order to get the absolute best classification results, but rather to demonstrate the process and provide a template script you can use with your own data. Feel free to take this dataset, manipulate it, and see if you get better results. If you do, please be sure to share them. The script, in the form of a Jupyter Notebook found on my Github, takes census data and tries to predict if the person in each row makes less or greater than $50,000 per year. The data I used came from the UCI Machine Learning Repository at https://archive.ics.uci.edu/ml/datasets/Census+Income. In the script, the following is done:
The script is straightforward and will hopefully allow you to be more productive in your work. Instead of a lot of manual labor, you can focus on the things you love about data science and making your business more efficient and profitable. If you have questions and want to connect, you can message me on LinkedIn, Twitter: Follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems #machinelearning #mldl #randomforest #hyperparameters #logisticregression #XGBoost #linearregression #generallinearmodels #GLM #GridSearchCV #RandomForestRegressor #overfitting #gridsearch #ai #artificialintelligence  Data science, at its core, is about time. Time to increasing sales, time to finding cures for diseases, time to understanding. Thus, it is critical to think about what hardware you are using for deep learning training, which is by far the most computationally intensive part of artificial intelligence and data science in general. A 10% reduction in training time can make a huge difference in employee productivity and time to results. How about a 25% decrease in training time? Let me show you how to get it! I recently read a Medium article by Thilina Rajapakse comparing 7 different BERT models. BERT is the current state-of-the-art training algorithm for natural language processing (NLP). In his article, he compared the models in terms of final accuracy and training time. His work was done using a custom built, AMD processor-based workstation with an Nvidia Titan RTX GPU, built upon the Turing architecture. This is a very popular PC/workstation GPU. In my conversations with customers, I often hear that they do their training on their laptop or a workstation with a configuration similar to that used by Thilina. His article, along with these customer comments, motivated me to determine how great of a difference an enterprise level server with top-shelf CPUs and GPUs can make in training time for these same models. The work I am presenting here did not focus on how the models work or comparing accuracy, but rather on training time. Since the models were so different, they allowed for a broad, unbiased comparison. I set up two different enterprise servers in our lab which I will call Server 1 (S1) and Server 2 (S2), to run the same training that Thilina did. My goal was to compare training times between S1, S2, and Thilina’s workstation (WS). Whereas WS had a Titan RTX GPU, S1 and S2 had Nvidia V100 GPUs, Nvidia’s most powerful GPU. I expected S1 and S2 to outperform WS, but I did not know how much difference there would be. Would it be significant? Would it justify the difference in price? Those were the two main questions I was trying to answer. In essence, by running deep learning training on an enterprise server, how much more productive, and thus profitable, could a company be? To keep this post concise, only the summary of the results is presented (see Table 1 below). Full details of the training performed and the specs of S1, S2, and WS can be found on my Github page. In Table 1, each model is listed, followed by the training time on that model for S1, S2, and WS. Overall, S1 reduced training time compared S2 and WS by 13.6% and 24.7%, respectively. S2 reduced training time compared to WS by 16.6%. An example from a customer is even more dramatic. A data scientist took a random forest model he was training on his laptop and trained it on a server very similar to S1. The training time decreased from 152 minutes to 26 minutes. That is 6x speedup.  Do these results demonstrate that using an enterprise level server rather than a workstation or laptop is significantly better? If decreasing your training time by 25% makes you more productive, then it is significantly better. In addition to being able to decrease training times, S1 had 8 GPUs. S2 and WS each had 1 GPU, although S2 can support up to 4 GPUs. (In this testing, only one GPU was utilized.) This means that 8 different people can be training concurrently on S1. Imagine 8 data scientists being able to train models at the same time and do it 25% faster than they can on their workstation or laptop. So not only is the training time decreased, it is decreased for as many as 8 people at the same time. This is a big deal.
To find out about how you can implement an enterprise level-server and have it set up to be able to perform your training, contact me. I love helping companies utilize deep learning to its full potential and make a fundamental shift in their operations. For details on the dataset used and the code, check out Thalina’s excellent article here. If you have questions and want to connect, you can message me on LinkedIn, Twitter: Follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems #artificialintelligence #ai #deeplearning #DL #machinelearning #ML #markiiisystems #BERT #servers  Customers often ask me about how to operationalize the machine learning models they have created. It’s common that they have the model, but are struggling with how to apply it to new or existing business data sets. Another line of questioning is about “How do we move the machine learning model forward into production?”
I wrote this article to provide readers an overview of the basic steps of operationalizing machine learning models. Creating a logistic regression model that can be operationalized Believe it or not, you simply create the model and save it to a file. This is typically done in a script that is separate from the script that will be used in production. pickle and joblib are common ways to save random forest, logistic regression, or other classical machine learning models. In this post, I will show the process for pickle. joblib is very similar. For neural networks created with Keras, it is recommended to use keras.models. Once the file is created, you later load it and use it to score your data. Create the model file and save it
Import the model from the file and use it on new data
Example: Logistic Regression Script to create and save model import pickle # Import other modules such as scikit-learn to create your model # Load and manipulate data as necessary #Create model and fit model = LogisticRegression() model.fit(X_train, Y_train) # Save the model to a file. Common extensions for the file are .h5, .pkl, or .sav. It does not matter what extension you use. filename = “my_model_file.h5” pickle.dump(model, open(filename, ‘wb’)) Script used in production import pickle # Import other modules such as scikit-learn to be able to use your model # Load and manipulate data as necessary # Load (open) the saved model file and score filename = “my_model_file.h5” loaded_model = pickle.load(open(filename, ‘rb’)) result = loaded_model.score(X_test, Y_test)) # Evaluate your results as needed for your situation That’s all there is to it! For more in-depth instructions on how to operationalize models using joblib from scikit-learn or saving and loading models in Keras, check out the following helpful websites: Jason Brownlee’s Machine Learning Mastery website https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/ Kaggle https://www.kaggle.com/prmohanty/python-how-to-save-and-load-ml-models keras.io https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model Do you work with clients who need help with operationalizing machine learning models? I’d love to hear what questions you commonly come across, please share by adding a comment below. If you have questions and want to connect, you can message me on LinkedIn, Twitter: Follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems #ai #machinelearning #MLDL #artificialintelligence #operationalize #production #python #keras #scikit-learn Tina Reed and Heather Landi from Fierce Healthcare recently wrote an article discussing the Top 5 things that will be generating buzz at the Healthcare Information and Management Systems Society (HIMSS) 2020 conference. The second thing on their list stated “Natural language processing will be all the rage.” This is very interesting considering that is exactly what I was going to be speaking about at the conference! My talk was titled “Healthcare, BERT AI, and Next-Gen Natural Language Processing/Natural Language Understanding - How BERT enables institutions to ‘read’ chart notes with unprecedented speed and accuracy.” Unfortunately, HIMSS was cancelled last week due to concerns surrounding COVID-19 (coronavirus). Although I won’t be able to do the presentation at the conference, I will be posting a video of the same presentation shortly. Stay tuned and see how we are staying at the forefront of AI that is “all the rage.” #NLP #BERT #HIMSS2020 #TinaReed @TreedinDC #HeatherLandi @HeatherLandi #MarkIIISystems @MarkIIISystems #NaturalLanguageProcessing #alltherage https://www.linkedin.com/posts/john-pace-phd_here-are-the-top-5-things-that-will-be-generating-activity-6641744058869960704-gUXJ  Photo Credit: Tina Reed, fiercehealthcare.com, https://www.fiercehealthcare.com/tech/here-are-top-things-will-be-generating-buzz-at-himss-2020
The NVIDIA DGX Workstation is a high-performance AI workstation that enables your Data Science team to get started quickly with the power of a data center in your office. With 4 NVIDIA V100 32GB GPUs and NVIDIA Docker preloaded, there is plenty of computing power to allow multiple users to run their most challenging AI training projects. Additionally, the DGX Workstation allows you to download preconfigured and optimized Docker containers from the NVIDIA GPU Cloud to further decrease the time it takes to get started. Our customers use their DGX Workstations for projects involving Natural Language Processing, classification with algorithms such as XGBoost, image analysis of x-rays and MRIs, and predictive analytics.
The setup for the DGX Workstation (DGX-WS) is a quick process, typically less than 30 minutes from power on until your first nvidia-docker run command. As you do your install, here are a few tips that can ensure the process is as smooth as possible. These tips come from our experience with installing and configuring DGX-WS. 1. The Setup Guide says the ethernet ports are configured for DHCP by default. We have found this is typically not the case, but that they are set to “manual”. Be sure to edit the /etc/network/interfaces file (or configure the ports using the GUI) and either set the ports to DHCP or a static IP address. As with any server, best practice is static IP. Timesaving tip: Make sure you have a monitor, keyboard, and mouse handy when first booting so you can configure the interfaces. After that, everything can be done with ssh. 2. Sometimes the DGX-WS will ship with nvidia-docker v1. If so, it needs to be upgraded to nvidia-docker v2. You can check you nvidia-docker version using: nvidia-docker version If the command returns 2.0.x, then your system already contains the upgrade to the NVIDIA Container Runtime for Docker and no further action is needed. If it does not return 2.0.x, then perform the steps listed here. 3. One of the most “creative” errors that can occur is this. It can happen when trying to pull new docker containers from NGC: user@DGX-WS:~/ docker pull nvcr.io/nvidia/pytorch:20.01-py3 (or any other container) Error response from daemon: Get https://nvcr.io/v2/: dial tcp: lookup nvcr.io on 127.0.0.53:53: server misbehaving “server misbehaving” – someone had fun with that error. This is typically caused by a DNS issue. Make sure the DNS servers are correctly configured and it should resolve the problem. Setting the DNS server to 8.8.8.8 is the most common fix. 4. Finally, make sure you remove the piece of foam that is inside the DGX-WS that keeps the GPUs from moving during shipping. It’s easy to overlook this step! Be sure to follow me on Twitter (@pacejohn), on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/), and follow my company, Mark III Systems, on Twitter as well (@markiiisystems).  At the Re-Work Deep Learning Summit in San Francisco in January, I learned about some new, very large image datasets. I have heard of and used the Cifar 10 and 100 datasets, as well as ImageNet. However, the “80 Million Tiny Images Dataset” and the “CelebA Dataset” were discussed. These were new to me. All of these are very large collections of images that are labeled for classification tasks. I often benchmark servers against each other to figure out which perform best in different situations. Since computer vision is such a big topic today, benchmarks on large image datasets are particularly useful and informative. Even a 10% increase in speed can have a large impact on business. In the next few weeks, I will be comparing the different buses in the IBM POWER9 AC922 server versus several Intel-based servers. All servers will have the same NVIDIA V100 GPUs and similar CPUs. However, the servers have different buses that move data at different speeds between the CPU and GPU. The faster the bus, the quicker data can be moved from the CPU to GPU and vice versa. The AC922 uses NVIDIA's NVLink technology between the CPU and GPU while Intel-based servers use PCIe Gen3. The NVLink bus in the AC922’s moves data at 150GB/sec versus the PCIe Gen3 bus in Intel-bus server which moves data at 32GB/sec. The question I am trying to answer is if the image classification tasks will run faster on the AC922 since the bus speed is approximately 5x faster than the Intel-based servers. Benchmarks from other groups using different types of dataset have shown this is the case, but I do not think this has been done with these large datasets of very small images. After the bus benchmarks, I will be testing disk I/O, particularly the IBM Elastic Storage Server (ESS) using Spectrum Scale against other external storage systems. The results should be published in the next few weeks. I am excited to see how they come out. The details on the different datasets are below. Be sure to follow me on Twitter (@pacejohn), on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/), and follow my company, Mark III Systems, on Twitter as well (@markiiisystems).  Last week, I attended the Re-work Deep Learning Conference in San Francisco. The speakers were a plethora of the top AI researchers and practitioners in the world - Facebook AI Research (FAIR), Google Brain, Netflix, Uber, MIT, UC-Berkeley, Amazon, and Pandora, just to name a few. The topics ranged from academic topics like pruning neural networks to putting AI into practice in retail. As with most conferences, there seemed to be an overarching theme in the sessions I attended. At this conference, the theme was “you have to use multiple AI methods to solve your problem.” Consumers are relying on AI more each day and they expect the AI to evolve. For the evolution to occur, new techniques must be invented. In addition, techniques must be combined. On Day 1, I saw a talk by Walmart AI Labs entitled “Personalizing the Online Grocery Substitution Experience.” My family uses Walmart online grocery ordering and pickup. At times, items are out-of-stock and we get substituted items. Sometimes we wonder why in the world we got some of the substitutions because they seem unrelated to what we ordered. Other times we get items that are significantly greater than what we ordered. For example, we may order a 50-count bottle of Advil. It is out-of-stock, so they substitute it with a 250-count bottle of Advil for the same price. Seems strange, but we definitely don’t complain. This talk fascinated me because they discussed how they determine what to substitute. As you can see from the pictures below, determining what to substitute is not a trivial problem. Walmart combines multiple AI and mathematical techniques to solve it, including constraint-based optimizations (CBO) and graph convolutional networks (GCN). These two techniques are important because they address two different problems that must be solved. The GCN is used to determine which item to substitute and the probability for acceptance of the substitution. However, that must also be combined with a cost analysis, which is where CBO comes into play. A substitution that is typically accepted may cost Walmart a significant amount of money, while substitutions that are declined make no money for Walmart. A balance has to be found that makes the customer happy and produces profit for Walmart. I’m looking forward to analyzing our substitutions on our grocery order this weekend to see if I can figure out the reasoning. Just another example of AI making life easier for consumers while still providing profits for sellers. I welcome your feedback and suggestions. Follow me on Twitter (@pacejohn) and on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/). Also, follow my company, Mark III Systems on Twitter (@markiiisystems).   |
RSS Feed
