The NVIDIA DGX Workstation is a high-performance AI workstation that enables your Data Science team to get started quickly with the power of a data center in your office. With 4 NVIDIA V100 32GB GPUs and  NVIDIA Docker preloaded, there is plenty of computing power to allow multiple users to run their most challenging AI training projects. Additionally, the DGX Workstation allows you to download preconfigured and optimized Docker containers from the NVIDIA GPU Cloud to further decrease the time it takes to get started.  Our customers use their DGX Workstations for projects involving Natural Language Processing, classification with algorithms such as XGBoost, image analysis of x-rays and MRIs, and predictive analytics.


The setup for the DGX Workstation (DGX-WS) is a quick process, typically less than 30 minutes from power on until your first nvidia-docker run command.  As you do your install, here are a few tips that can ensure the process is as smooth as possible.  These tips come from our experience with installing and configuring DGX-WS.


1.  The Setup Guide says the ethernet ports are configured for DHCP by default. We have found this is typically not the case, but that they are set to “manual”.  Be sure to edit the /etc/network/interfaces file (or configure the ports using the GUI) and either set the ports to DHCP or a static IP address.  As with any server, best practice is static IP.  Timesaving tip: Make sure you have a monitor, keyboard, and mouse handy when first booting so you can configure the interfaces.  After that, everything can be done with ssh.


2.  Sometimes the DGX-WS will ship with nvidia-docker v1. If so, it needs to be upgraded to nvidia-docker v2.  You can check you nvidia-docker version using:

      nvidia-docker version

If the command returns 2.0.x, then your system already contains the upgrade to the NVIDIA Container Runtime for Docker and no further action is needed.  If it does not return 2.0.x, then perform the steps listed here.

3.  One of the most “creative” errors that can occur is this. It can happen when trying to pull new docker containers from NGC:

     user@DGX-WS:~/  docker pull nvcr.io/nvidia/pytorch:20.01-py3 (or any other container)
     
  Error response from daemon: Get https://nvcr.io/v2/: dial tcp: lookup nvcr.io on 127.0.0.53:53: server misbehaving

“server misbehaving” – someone had fun with that error.  This is typically caused by a DNS issue.  Make sure the DNS servers are correctly configured and it should resolve the problem.  Setting the DNS server to 8.8.8.8 is the most common fix.

​4.  Finally, make sure you remove the piece of foam that is inside the DGX-WS that keeps the GPUs from moving during shipping. It’s easy to overlook this step!

Be sure to follow me on Twitter (@pacejohn), on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/), and follow my company, Mark III Systems, on Twitter as well (@markiiisystems).

At the Re-Work Deep Learning Summit in San Francisco in January, I learned about some new, very large image datasets.  I have heard of and used the Cifar 10 and 100 datasets, as well as ImageNet.  However, the “80 Million Tiny Images Dataset” and the “CelebA Dataset” were discussed.  These were new to me.  All of these are very large collections of images that are labeled for classification tasks.

I often benchmark servers against each other to figure out which perform best in different situations.  Since computer vision is such a big topic today, benchmarks on large image datasets are particularly useful and informative.  Even a 10% increase in speed can have a large impact on business.
​
In the next few weeks, I will be comparing the different buses in the IBM POWER9 AC922 server versus several Intel-based servers.  All servers will have the same NVIDIA V100 GPUs and similar CPUs. However, the servers have different buses that move data at different speeds between the CPU and GPU.  The faster the bus, the quicker data can be moved from the CPU to GPU and vice versa.  The AC922 uses NVIDIA's NVLink technology between the CPU and GPU while Intel-based servers use PCIe Gen3.  The NVLink bus in the AC922’s moves data at 150GB/sec versus the PCIe Gen3 bus in Intel-bus server which moves data at 32GB/sec.  The question I am trying to answer is if the image classification tasks will run faster on the AC922 since the bus speed is approximately 5x faster than the Intel-based servers.  Benchmarks from other groups using different types of dataset have shown this is the case, but I do not think this has been done with these large datasets of very small images.  After the bus benchmarks, I will be testing disk I/O, particularly the IBM Elastic Storage Server (ESS) using Spectrum Scale against other external storage systems.  The results should be published in the next few weeks.  I am excited to see how they come out.
The details on the different datasets are below.

Be sure to follow me on Twitter (@pacejohn), on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/), and follow my company, Mark III Systems, on Twitter as well (@markiiisystems).

Last week, I attended the Re-work Deep Learning Conference in San Francisco.   The speakers were a plethora of the top AI researchers and practitioners in the world - Facebook AI Research (FAIR), Google Brain, Netflix, Uber, MIT, UC-Berkeley, Amazon, and Pandora, just to name a few.  The topics ranged from academic topics like pruning neural networks to putting AI into practice in retail.  As with most conferences, there seemed to be an overarching theme in the sessions I attended.  At this conference, the theme was “you have to use multiple AI methods to solve your problem.”  Consumers are relying on AI more each day and they expect the AI to evolve.  For the evolution to occur, new techniques must be invented.  In addition, techniques must be combined.


On Day 1, I saw a talk by Walmart AI Labs entitled “Personalizing the Online Grocery Substitution Experience.”  My family uses Walmart online grocery ordering and pickup.  At times, items are out-of-stock and we get substituted items.  Sometimes we wonder why in the world we got some of the substitutions because they seem unrelated to what we ordered.  Other times we get items that are significantly greater than what we ordered.  For example, we may order a 50-count bottle of Advil.  It is out-of-stock, so they substitute it with a 250-count bottle of Advil for the same price.  Seems strange, but we definitely don’t complain.  This talk fascinated me because they discussed how they determine what to substitute.
​

As you can see from the pictures below, determining what to substitute is not a trivial problem.  Walmart combines multiple AI and mathematical techniques to solve it, including constraint-based optimizations (CBO) and graph convolutional networks (GCN).  These two techniques are important because they address two different problems that must be solved.  The GCN is used to determine which item to substitute and the probability for acceptance of the substitution.  However, that must also be combined with a cost analysis, which is where CBO comes into play.  A substitution that is typically accepted may cost Walmart a significant amount of money, while substitutions that are declined make no money for Walmart.  A balance has to be found that makes the customer happy and produces profit for Walmart.

I’m looking forward to analyzing our substitutions on our grocery order this weekend to see if I can figure out the reasoning.  Just another example of AI making life easier for consumers while still providing profits for sellers.

I welcome your feedback and suggestions. Follow me on Twitter (@pacejohn) and on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/).  Also, follow my company, Mark III Systems on Twitter (@markiiisystems).​

On January 15, I attended the Watson Warriors event in snowy Seattle, hosted by Tech Data.  Watson Warriors is a multi-challenge game, developed by Launch Consulting, that allows data scientists to compete against each other to solve AI problems using Watson Studio Cloud.  In this first Warrior challenge, we used weather data and images to predict where fires were occurring.  Several more challenges will be hosted around the country in the coming months.

I was very pleased to see so many people at the event.  In my estimation, there were probably about 80 people there.   I teamed up with a couple of friends from IBM and another business partner in the competition.  We didn’t win, but we did have a great time.  I’m looking forward to the next one in San Antonio in March!

​If you read my blog from December 20 about answering questions from long passages using BERT, you know how excited I am about how BERT is having a huge impact on natural language processing.  BERT, or Bidirectional Encoder Representations from Transformers, which was developed by Google, is a new method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks. 
 
The BERT pre-trained models can be used for more than just question/answer tasks.  They can also be used to determine how similar two sentences are to each other.  In this post, I am going to show how to find these similarities using a measure known as cosine similarity.  I do some very simple testing using 3 sentences that I have tokenized manually.  If using a larger corpus, you will definitely want to have the sentences tokenized using something like nltk.tokenize.  The first two sentences (0 and 1) come from the same blog entry, while the third (2) comes from a separate blog entry.  The similarity between sentences 0 and 1 should be higher with each other than with sentence 2 (I’ll explain later why I used the numbers 0-2 instead of 1-3).  Let’s see if that is the case.  The sentences are:

1. BERT was developed by Google and Nvidia has created an optimized version that uses TensorRT
2. One drawback of BERT is that only short passages can be queried
3. I attended a conference in Denver

 
In this testing, I used the BERT-as-a-Service server and client.  The testing was done two different ways.  The first was where the BERT server and BERT client were both running on the same physical server.  The second was where the BERT server was on a different physical server than the BERT client was on.  Where the two differ, I will list the syntax needed for each.  Note:  If running the BERT server in a Docker container on one physical server and the BERT client on a different physical server or PC, you must specify that ports 5555 and 5556 are explicitly used for the container the BERT server is running in.  If you do not, the client won’t be able to connect.  To do this, be sure to add the -p switch to your docker run command (-p 5555:5555 -p 5556:5556).  This is not necessary if the BERT server and BERT client are both running on the same physical server.
 
Installing the BERT Server and Client
 
The BERT server and client require TensorFlow version 1.10 or greater.  I used version 1.14.0-rc0 in this testing.  To install each, run the following:

• pip install bert-serving-server bert-serving-client
• pip install tensorflow-gpu==1.14.0-rc0

 
Create a directory for the models to be saved in:

• mkdir /home/username/bert_models

 
Get the pre-trained models (https://github.com/google-research/bert#pre-trained-models).  I used BERT-Base, both the Uncased and Cased versions for testing.

• BERT-Base, Uncased: 12-layer, 768-hidden, 12-heads, 110M parameters
  • wget https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip
  • unzip uncased_L-12_H-768_A-12.zip

• BERT-Base, Cased: 12-layer, 768-hidden, 12-heads , 110M parameters
  • wget https://storage.googleapis.com/bert_models/2018_10_18/cased_L-12_H-768_A-12.zip
  • unzip cased_L-12_H-768_A-12.zip

 
Starting the BERT Server
 
When starting the BERT server, you must specify which pre-trained model to use.  The switch specifies how many clients can concurrently connect to the server at one time.  The switch is not necessary, but I use it here so the encoded sentences can be seen.
 
If using the Uncased model, run:

• bert-serving-start -model_dir /home/username/bert_models/uncased_L-12_H-768_A-12/ -num_worker=1 -show_tokens_to_client

 
If using the Cased model, run:

• bert-serving-start -model_dir /home/username/bert_models/cased_L-12_H-768_A-12/ -num_worker=1 -show_tokens_to_client

 
When the server is started, you will see something similar to this:

​Connecting to the BERT Server with the BERT Client and Finding Sentence Similarity
 
The following script (BERT_sentence_similarity.py) can be found on my Github at https://github.com/pacejohn/BERT-Cosine-Similarities.
 
# Connect to a BERT server from a BERT client and determine the cosine similarity between 2 sentences

# Imports
from bert_serving.client import BertClient
from sklearn.metrics.pairwise import cosine_similarity
 
# Uncomment the following line if the BERT server that is running locally (on the same physical server that the client will be running on).
#client = BertClient()
 
# Uncomment the following line if the BERT server is running remotely (on a different physical server than the client will be running on).
# You must specify the IP of the remote server and the ports
client = BertClient(ip='10.3.50.61', port=5555, port_out=5556)
 
# Save tokenized sentences to variables.  This makes it easier later.
# I started the numbering at 0 rather than 1 so it matches the indexes of the arrays that are created when the encoding happens.
sentence0 = ['bert', 'was', 'developed', 'by', 'google', 'and', 'nvidia', 'has', 'created', 'an', 'optimized', 'version', 'that', 'uses', 'tensorrt']
sentence1 = ['one', 'drawback', 'of', 'bert', 'is', 'that', 'only', 'short', 'passages', 'can', 'be', 'queried']
sentence2 = ['i', 'attended', 'a', 'conference', 'in', 'denver']
 
# Specify which 2 sentences to compare.
first_sentence = 0
second_sentence = 1
 
# Encode the sentences using the BERT client.
sentences = client.encode([sentence0, sentence1, sentence2], show_tokens=True, is_tokenized=True)
# If you print ‘sentences’, it will show the arrays along with the encoded sentences.  This can be interesting because it shows which words it did not recognize.  They are denoted by [UNK].
sentences
 
# Calculate cosine similarity between the 2 sentences you specified and display it
cos_sim = cosine_similarity(sentences[0][first_sentence][:].reshape(1,-1),sentences[0][ second_sentence][:].reshape(1,-1))
 
# Show the sentences and their cosine similarity, but leave off the [CLS] at the beginning and [SEP] at the end
if first_sentence == 0 and second_sentence == 1:
           print("\n\nThe cosine similarity between sentence:\n" + str(' '.join(sentence0)) + "\n\nand sentence:\n\n" + str(' '.join(sentence1)) + "\nis " + str(cos_sim[0][0]))
if first_sentence == 0 and second_sentence == 2:
           print("\n\nThe cosine similarity between sentence:\n" + str(' '.join(sentence0)) + "\n\nand sentence:\n\n" + str(' '.join(sentence2)) + "\nis " + str(cos_sim[0][0]))
if first_sentence == 1 and second_sentence == 2:
           print("\n\nThe cosine similarity between sentence:\n" + str(' '.join(sentence1)) + "\n\nand sentence:\n\n" + str(' '.join(sentence2)) + "\nis " + str(cos_sim[0][0]))
 
# Show the encoded versions of the sentences for comparison
print("\n******\nThe encoded sentences are")
print(*sentences[1][first_sentence][1:len(sentences[1][first_sentence][:])-1])
print("\nand\n")
print(*sentences[1][second_sentence][1:len(sentences[1][second_sentence][:])-1])
print("\n")
 
As a final note, Tables 1 to 4 below show the differences in the cosine similarity between the sentences when capital letters are used or not used and when the Uncased or Cased model is used.  In my next post, I will discuss these differences in more detail.
 
I welcome your feedback and suggestions. Follow me on Twitter (@pacejohn) and on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/).  Also, follow my company, Mark III Systems on Twitter (@markiiisystems).
 
This post is also on Medium at ​https://medium.com/@johnpace_32927/finding-cosine-similarity-between-sentences-using-bert-as-a-service-6bbcd02defcf​.

Thanks to Anirudh S and Han Xiao​ for code snippets and ideas:
​
https://towardsdatascience.com/word-embedding-using-bert-in-python-dd5a86c00342

https://github.com/hanxiao/bert-as-service#building-a-qa-semantic-search-engine-in-3-minutes

BERT, or Bidirectional Encoder Representations from Transformers, is a new method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks.  It’s safe to say it is taking the NLP world by storm. BERT was developed by Google and Nvidia has created an optimized version that uses TensorRT. (https://github.com/google-research/bert and https://devblogs.nvidia.com/nlu-with-tensorrt-bert/)

One drawback of BERT is that only short passages can be queried when performing Question & Answer. After the passages reach a certain length, the correct answer cannot be found.  To run a Question & Answer query, you have to provide the passage to be queried and the question you are trying to answer from the passage.

I have created a script that allows you to query longer passages and get the correct answer.  I take an input passage and break it into paragraphs that are delimited by \n. Each paragraph is then queried to try to find the answer. All answers that are returned are put into a list. The list is then analyzed to find the answer with the highest probability.  This is returned as the final answer. When you run the script, you will want to change the paths to correspond with your setup.  All code and files are on my GitHub HERE.

In order to run the script properly, you need to make sure that a Docker container is created. Before running the query, be sure to start the TensorRT engine. Here are the steps Nvidia says to do and that I am doing.

From home directory, run the following. It takes a while.

• Clone the TensorRT repository and navigate to BERT demo directory
  • git clone --recursive https://github.com/NVIDIA/TensorRT && cd TensorRT/demo/BERT
    
• Create and launch the docker image
  • sh python/create_docker_container.sh
    
• Build the plugins and download the fine-tuned models
  • cd TensorRT/demo/BERT && sh python/build_examples.sh base fp16 384
    
• Build the TensorRT runtime engine and start it. If you don't nohup this, you won't be able to do anything else.
  • nohup python python/bert_builder.py -m /workspace/models/fine-tuned/bert_tf_v2_base_fp16_384_v2/model.ckpt-8144 -o bert_base_384.engine -b 1 -s 384 -c /workspace/models/fine-tuned/bert_tf_v2_base_fp16_384_v2 > tensorrt.out &
    
• After you have started the engine, you can then run the Q&A query in the script
  • python python/bert_inference_loop.py
    
• One caveat is that the TensorRT engine will terminate after a period of time.  Be sure it is running before you perform you query.

Files I am using for queries

There are 3 files that can I am using as input passages.  Feel free to try it with your own passages.

22532_Full_Document.txt - this is the full document I am using. If you ask a question about the first part, it will return the correct answer. If you ask a question about a later part, it will not find the answer.

22532_Short_Document_With_Answers.txt - this is a shortened passage that has answers to the query. If you use the same query as I did for the question, it will find 2 answers. The one with the higher probability is the correct answer.
22532_Short_Document_Without_Answers.txt - this is a shortened passage that does not have the answers to the query. If you use the same query as I did for the question, it will not find any answers.

The question that is asked is "How many patients experienced recurrence at 12 years of age?" Feel free to experiment.

I welcome your feedback and suggestions. Follow me on Twitter (@pacejohn) and on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/).  Also, follow my company, Mark III Systems on Twitter (@markiiisystems).

This post is also on Medium at https://medium.com/analytics-vidhya/question-and-answer-for-long-passages-using-bert-dfc4fe08f17f.

​If you have ever used Pandas, you know that it is great tool for creating and working with data frames.  As with any software, optimizations can be done to make it perform better.  Nvidia has developed software called RAPIDS, a data science framework that includes a collection of libraries for executing end-to-end data science pipelines completely on the GPU (www.rapids.ai).  Included in RAPIDS is cuDF, which allows data frames to be loaded and manipulated on a GPU.  In this post, I am going to discuss some benchmarking I have done with RAPIDS, particularly cuDF.  I conducted multiple experiments where I created data frames that ran on the CPU using Pandas and data frames that ran on the GPU using cuDF, then executed common methods on those data frames.  I will be using the term “processes” to describe the execution of the methods.  I will also be using the convention that Nvidia has used, in which the Pandas data frames are named PDF, for Pandas data frame, and the cuDF data frames are named GDF, for GPU data frame.

For my benchmarking data, I used a CSV file from MIMIC-III, a freely accessible critical care database that was created by the MIT Lab for Computational Physiology (https://mimic.physionet.org/).  The file was named MICROBIOLOGYEVENTS.csv.  It consisted of 631,726 rows and 16 columns of data.  I duplicated the records in the file to create new files that consisted of 5 million, 10 million, 20 million, and 40 million rows and 16 columns of data, respectively.  Experiments were then conducted using each of these 5 files.  An individual experiment is defined as running a processes on one of the 5 versions of the MICROBIOLOGYEVENTS files as input.  Each experiment was repeated 5 times and the results averaged together to give the final results that I list.

The benchmarking was done on both an Nvidia DGX-1 and an IBM POWER Systems AC922 using a single GPU in each.  The GPUs in the servers were both Nvidia V100 models, with the DGX-1 GPU having the model with 32GB of RAM and the AC922 having the 16GB model.

​For the benchmarking, I ran some common processes on both PDF and GDF data frames and calculated the amount of time it took to run.  The processes were done in the following order using a Jupyter Notebook that can be found on my Github (https://github.com/pacejohn/RAPIDS-Benchmarks).  

• Loading the file from a csv into PDF and GDF data frames.
  
• Finding the number of unique values in one column.
  
• Finding the number of rows with unique values in one column.
  
• Finding the 5 smallest and largest values in one column.
  
• Selecting 5 specific rows in a column by index.
  
• Sorting the data frame by values in one column.
  
• Creating a new column with no data in it.
  
• Creating a new column that was populated with a calculated value.  It took the value of a preexisting column and multiplied it by 2 to get the calculated value.
  
• Dropping a column.

In addition, I created an additional Jupyter Notebook that was used to concatenate 2 data frames.  In this experiment, the MICROBIOLOGYEVENTS.csv, which has 631,726 rows, was concatenated onto each of the 5 MICROBIOLOGYEVENTS input files.

In 4 of the 9 experiments, the GDF outperformed the PDF regardless of the input file that was used.  In 3 experiments, the PDF outperformed the GDF.  Interestingly, in 2 experiments the PDF outperformed the GDF on small data frames but not on the larger ones.  In the concatenation experiments, the GDF always outperformed the PDF.  The results for the processes that were run on the AC922 are below.  The results for the DGX-1 are similar.  For complete results, including the actual times for the processes to run and the DGX-1 results, see my Github.

The most remarkable differences in performance were in the following processes.

GDF Outperforms PDF

• For time to load the input file, the GDF outperformed the PDF by an average of 8.3x faster (range 4.3x-9.5x).  For the input file with 40 million records, the GDF was created and loaded in 5.87 seconds while the PDF took 56.03 seconds.
  
• When sorting the data frame by values in one column, the GDF outperformed the PDF by an average of 15.5x faster (range 2.1x-23.4x).  Due to the GPU in the AC922 only having 16GB of RAM, the 40 million row data frame was not able to be sorted so these number include the results of the sort on the DGX-1 for the 40 million row data frame.
  
• When creating a new column that was populated with a calculated value, the GDF outperformed the PDF by an average of 4.8x faster (range 2.0x-7.1x).
  
• The most remarkable performance difference was seen when dropping a single column.  Amazingly, the GDP outperformed the PDF by an average of 3,979.5x faster (range 255.7x-9,736.9x).  Performance scaled linearly as the size of the data frame became larger.
  
• When concatenating the 631,726 row data frame onto another data frame, the GDF outperformed the PDF by an average of 10.4x faster (range 1.2x-29.0x).  As with sorting, the 16GB GPU ran out of memory when trying to append the data frame onto the 40 million row data frame sorted so these number include the results of the sort on the DGX-1 for the 40 million row data frame.

PDF Outperforms GDF
When the PDF outperformed the GDF, the results were astonishing.

• When finding the number of unique values in one column, the PDF outperformed the GDF by an average of 74.1x (range 5.7x-286.0x).  However, as the size of the data frame increased, the performance difference reduced dramatically, from 285.9x to 5.7x.  This suggests that at a point the GDF would most likely perform better, but additional experiments would need to be conducted to demonstrate that.  This trend is seen in the processes where the PDF outperforms the GDF on smaller data frames. 
  
• The most remarkable performance difference was seen when selecting 5 specific rows in a column by index.  In this case, the PDF outperformed the GDF by an average of 427.4x faster (range 32.2x-735.0x).  

PDF Outperforms GDF on Smaller Data Frames

• When selecting the 5 smallest and largest values for a column, the PDF outperformed the GDF on the 0.631 million, 5 million, and 10 million row data frames with the performance decreasing as the data frame became larger (range 1.1x to 12.7x).  On the larger data frames, the GDF performed best and the performance increased as the data frame became larger (range 1.8x-3.3x).
  
• When adding a blank column, the PDF outperformed the GDF only on the 0.631 million row data frame.

As shown above, data frames that run on the GPU can often speed up processes that manipulate the data frame by 10x to over 1,000x when compared to data frames that run on the CPU, but this is not always the case.  There is also a tradeoff in which smaller data frames perform better on the CPU while larger data frames perform better on the GPU.  The syntax for using a GDF is slightly different than using a PDF, but the learning curve is not steep, and the effort is worth the reward.  I’m going to try the same benchmarking on some other data sets and use some other methods to see how the results compare.  Stay tuned for the next installation.  

Follow me on Twitter (@pacejohn) and on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/).  Also, follow my company, Mark III Systems on Twitter (@markiiisystems).

​On December 11 and 12, I attended the AI Summit New York City.  AI Summit conferences are always top notch and showcase a wide variety of companies that are innovating with their AI technologies.  This year was no different.  There were companies showing products that ranged from using AI to determine which makeup is best for your skin type to augmenting datasets by creating synthetic data to optimizing data science workflows.  The breadth of technologies was definitely interesting.

​At my booth, my co-worker, Lisa Stone, and I showed off the latest version my American Sign Language transcription software.  The response was even bigger than at other conferences.  I had people from major banks, hospital groups, and universities who discussed possible use cases for their industry.  A total of 25 people allowed me to gather more training data by filming them signing the alphabet as well as the words “hello”, “good”, “bad”, “I love you”, “you”, “me”, and “your”.  Adding words that have motions is the next step of the project.  (Spoiler alert:  I have that version done for some words, but it needs a little more testing before it moves to prime time.)  One exciting experience was when a student from a university in New Jersey came to see the demo.  He was interested and we had a good discussion.  Later that day, he brought his entire team of 10 developers, data scientists, other AI practitioners to see the demo.  You can imagine how surprised I was to see 10 people all show up at once because one man was so excited and wanted them to see it.  Finally, I was interviewed on camera by the organizers of the conference because they thought the project was highly innovative and can help make people’s lives better.  I call it “AI for Good”.

​Just as exciting, I had AI practitioners who discussed the technology with me from a very technical perspective.  For me, that is the beauty of these types of conferences.  I am able to plunge deeply into the technology and see how other people are doing AI.  This information can be used to improve my product.  The sharing of ideas is how industries are advanced.  As the saying goes “All of us is better than one of us.”  How many times do we get to talk to people about the hyperparameters we are using when training our models and how we can optimize them?  Not everyone is interested in that, but it’s the type of stuff that gets me excited.  We also talked about the hardware I am running the training on.  Let’s just say the hardware is quite impressive!

One final note, while in NYC, I got to visit Pace University.  No, I’m not the founder and I don’t think I am directly related, but how many people have a university that has their name?

I am looking forward to the conference next year.  If you would like to discuss more, let me know.  Be sure to follow me on Twitter (@pacejohn), on LinkedIn (https://www.linkedin.com/in/john-pace-phd-20b87070/), and follow my company, Mark III Systems, on Twitter as well (@markiiisystems).

I attended the Super Computing 2019 (SC19) conference in Denver.  SC is a very large conference that highlights some of the most cutting-edge work in high-performance computing (HPC), AI, computational fluid dynamics, and other fields where massive computation is required.  Groups like NASA, the National Laboratories, including Sandia, Oak Ridge, and Lawrence Livermore, and major research universities all highlighted their work.  For someone who loves to see how far computing can be pushed, this is the place to be.
At the conference, I presented my software that uses AI to transcribe American Sign Language (ASL) in real-time as a person signs in front of a camera.  As letters of the alphabet are signed, they are written across the screen so they can be read.  The ultimate goal of the project is to develop both phone and web apps that allow someone who does not speak ASL to be able to understand what someone who does speak ASL is signing.
Since the project started in August of this year, I have been able to make significant progress.  One of the main things I need, and that is needed in any computer vision project, is lots and lots of training data.  At several other conferences this year I have asked people to allow me to take a video of them performing the ASL alphabet.  I was able to get video of 31 new people performing the alphabet.
At SC19, I added a new twist to the video collection.  I had a screen that played a 10-minute loop of various backgrounds, including random colors, a person walking through a castle and a forest, concert video footage, and highlights from the 2019 World Series (sorry Astros fans).  People stood in front of the screen as they performed the alphabet.  The reason for the screen is very simple.  When someone signs, they will rarely be standing in front of a solid, non-moving background.  Instead, they will be in a classroom or restaurant or outside or somewhere else where the environment is not static.  For the AI software to be generalizable, the training must be done using myriad backgrounds.  By adding 10-minutes of different backgrounds, I was able to ensure that each letter that was signed would be on a different background.  As I did at previous conferences, I made every attempt to get people with varying hand shapes, colors, sizes, fingernails (both painted and unpainted), and jewelry to sign the alphabet.  This will also make the models generalizable to the maximum number of people and reduce bias as much as possible.
As at other conference, the response to the software was very good.  I had many, many people come by and try the demo.  In fact, I had my first deaf person try it.  Honestly, I was quite nervous when she tried.  I have been careful to make it as useful as possible, but I wouldn’t know for sure how successful it is until a deaf person tried it.  She was very impressed and said it will be very helpful for the deaf community.  She even asked how she can help with the development.  I am very much looking forward to working with her.
Finally, I was able to do 2 oral presentations of the challenges I face in developing the software.  I had several people ask me questions and we had good discussions about some ways to overcome some of the challenges.

If you haven’t had a chance, look at my Twitter feed (@pacejohn) to see the posts I made during the conference and to stay up to date on my latest research.  Follow my company on Twitter as well (@markiiisystems) and www.markiiisys.com.

The Nvidia GTC conferences – my favorite times of the year!  Some of the greatest minds in AI come together to show off the work they are doing on Nvidia GPUs or the research into AI that they are conducting.  For an AI fanatic, it’s a brain overload!
In November, I attend the GTC conference in Washington, DC.  Though this conference is significantly smaller than its sister conference in San Jose, it was still packed full of fantastic presentations.  The main terms I heard over and over were BERT and PyTorch.  BERT is taking the NLP world by storm and PyTorch is quickly gaining momentum and market share among AI/ML/DL frameworks.  In this post, I’m going to highlight one of the sessions that stood out to me. In fact, it was possibly the most meaningful and even life-changing presentation I have ever seen at a conference.
Robert Pless, Department Chair & Patrick & Donna Martin Professor of Computer Science at George Washington University, presented a talk entitled “Explainable Deep Learning to Fight Sex Trafficking.”  In the talk, he discussed a project he is conducting with the FBI and that is currently in use by the National Center for Missing and Exploited.  The project is very simple and elegant, and is having a major impact in helping to stop or prevent crimes against children.
The goal of the project is to be able to identify, down to the specific hotel room, locations where sex trafficking is happening.  His team has developed an app called “TraffickCam.”  With the app, you can take 4 pictures of a hotel room that are then stored in an image database.  These images can then be compared to images that law enforcement officials gather from the Internet or other sources in their trafficking investigations.  By identifying distinctive regions that occur in both images, they can identify the specific room where the photograph was taken.  From this information, law enforcement can target their investigation much more specifically.  As of the presentation time, there are ~356,000 images from ~32,000 hotels in the database and this number is growing.  However, this dataset is only able to identify the correct hotel room in 8-35% of the searches.  How can this number increase -- more data!  I have already added images from 3 hotel rooms since I heard the talk.  You can help increase the number, too.
While this topic is quite disturbing, I am incredibly excited to see AI being used in such a helpful and life-changing manner.  AI sometimes gets a bad rap, particularly about privacy, but this project is proof that it can change the world for the better.  If you haven’t already, please download the app from the App Store or Google Play.  It takes less than one minute to take the photos of your hotel room.  This one minute can change someone’s life forever.  For more information on the project, you can click here or visit their Facebook page.  Be sure to watch the YouTube video for more info on how the project started https://www.youtube.com/watch?v=zhfHOR6yc98 and read the paper at https://arxiv.org/abs/1901.11397.
Follow me on Twitter (@pacejohn) for more updates!  Also, be sure to follow my company, Mark III Systems (@markiiisystems) and watch their blog at https://www.markiiisys.com/blog/