Does Microsoft Word use Deep Learning?

Hi, I'm Clippy. Remember me?

​A few days ago, I was writing a document in Microsoft Word and found something very interesting. I inserted a picture and noticed it automatically created a caption for the image. I was really impressed. Image captioning is not an easy process. You have to combine convolutional neural networks (CNNs) with recurrent neural networks (RNNs), particularly Long Short Term Memory (LSTM) RNNs. Putting those models in production in the standard word processing program is a great example of artificial intelligence in our every day work and how AI is being integrated into all parts of our lives.

Image from https://raw.githubusercontent.com/yunjey/pytorch-tutorial/master/tutorials/03-advanced/image_captioning/png/model.png

​The image above is a workflow for image captioning. First, you input the image. Next, use a CNN to get a feature vector. Finally, use the feature vector as input for the LSTM. That's all there is to it (too bad each of those steps is not trivial).

Here is a fun video I made to demonstrate how the process works in Microsoft Word. Try it out for yourself!
​​

If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn, LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/, and follow my company, Mark III Systems, on Twitter @markiiisystems
​

#artificialintelligence #AI #machinelearning #MLDL #microsoftword #neuralnetworks #imagecaptioning #recurrentneuralnetworks #rnn #convolutionalneuralnetworks #cnn #lstm #longshorttermmemory #microsoft #microsoftword #microsoftoffice #office365