This post is the last of my 4 part series on Keras Callbacks.  A callback is an object that can perform actions at various stages of training and is specified when the model is trained using model.fit(). In today's post, I give a very brief overview of the LearningRateScheduler callback.

So, what is a learning rate anyways?  In the context of neural networks, the learning rate is a hyperparameter that is used when calculating the new weights during back propagation.  For more details and the math involved, check out Matt Mazur's excellent discussion.  The learning rate helps determine how much the weights change.  Adjusting the learning rate can play a huge role in helping the network converge and lowering the loss value.  If the learning rate is too high, the weights can fluctuate significantly between updates, causing the model to converge quickly resulting in a suboptimal solution.  If the learning rate is too small, the weights will not fluctuate enough, causing the process to get stuck.  In either case, your model will suffer.  For a more detailed explanation of the impact of learning rate on neural network performance, check out Jason Brownlee's great post.

One powerful technique for training an optimal model is to adjust the learning rate as training progresses.  Start with a somewhat high learning rate, then reduce it as the training progresses.  Think of sanding wood.  You start with coarse grit sandpaper to do some initial smoothing.  You then continue to use finer and finer grit sandpaper until you have very smooth surface.  If you just used coarse grit sandpaper, you would get a sanded surface, but it would not be smooth.  If you used very fine grit sandpaper, you may or may not get a smooth surface, but it would have taken forever! 

Keras provides a nice callback called LearningRateScheduler that takes care of the learning rate adjustments for you.  Simply define your schedule and Keras does the rest.  At a predetermined epoch of the training, the learning rate is adjusted by a factor that you decide.  For example, at epoch 100, your learning rate is adjusted by a factor of 0.1.  At epoch 200, it is adjusted again by a factor of 0.1 and so on.  That's all there is to it.  You can easily make adjustments to this schedule by updating the callback.  You could even do a grid search to test multiple schedules and find the best model.  As with most of the features of Keras, it is easy and straightforward.  

The syntax for the callback is below as well as some example code from the keras.io website.

I hope you have enjoyed this series on Keras Callbacks.  Be sure to check out all 4 parts.
​
Part 1 - Keras EarlyStopping Callback
Part 2 - Keras ModelCheckpoint Callback
Part 3 - Keras Tensorboard Callback

​If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn and LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/.

There is an enormous amount of information floating around about COVID and SARS-CoV-2, the virus that causes COVID.  Unfortunately, much of the information is filtered through a lens of bias or reported by non-scientists.  The definitive source of information is the peer-reviewed scientific literature that is being published.  These papers are thoroughly reviewed by experts in the field to make sure the science is solid and the research was conducted appropriately.  I can tell you from experience, the peer-review process is not easy.  Your manuscripts are read with a critical eye to make sure you are not making unfounded conclusions, that you followed accepted protocols, and overall your findings are valid.  There is a very good repository called LitCovid, part of the National Library of Medicine, part of the National Center for Biotechnology Information (NCBI), the organization that manages PubMed.  As of November 10, 2020, there are over 68,000 peer-reviewed, published articles in the database. The articles are searchable by subject, journal name, and other criteria.  If you want to know what has really been discovered and the directions that the research is headed, LitCovid is the source.

Link to the LitCovid publication and citation
https://academic.oup.com/nar/advance-article/doi/10.1093/nar/gkaa952/5964074

Qingyu Chen, Alexis Allot, Zhiyong Lu, LitCovid: an open database of COVID-19 literature, Nucleic Acids Research, , gkaa952, https://doi.org/10.1093/nar/gkaa952

​​If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn and LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/.

A few months ago, I recorded a podcast titled "Managing Data in Research" with my coworker Holly Newman, US Healthcare Client Leader, at Mark III Systems (https://soundcloud.com/user-438874054/heathcare-powerchat-106-managing-data-in-research-h-newman-j-pace-mark-iii-systems).  The podcast is a part of the Dell Technologies Healthcare Power Chat series.  In the podcast, I start by defining data that is used for clinical settings versus data used for research. Holly and I then discuss the flow of data between the clinical and research sides of healthcare and the challenges of data access. I then explain how discoveries made on the research side feed the clinical side and how AI and ML are used to drive insights. Holly and I conclude by sharing Mark III Systems' approach to making clinical data available to the research side, how they help clients optimize the infrastructure used to analyze and process the data, how research groups can get started, where to find more information, and final thoughts.  I hope you'll take a few minutes and give it a listen.  I would love to hear your feedback!

​If you have questions and want to connect, you can message me on LinkedIn or Twitter. Also, follow me on Twitter @pacejohn and LinkedIn https://www.linkedin.com/in/john-pace-phd-20b87070/.

#ai #aiinhealthcare #research #machinelearning #datascience #structureddata #unstructureddata #dell #delltechnologies #healthcarepowerchat #clinicaldata