https://shivammehta25.github.io/Machine Learning, Deep Learning, AI and Speech Researcher 2023-01-27T12:20:40+01:00 Shivam Mehta https://shivammehta25.github.io/ Jekyll © 2023 Shivam Mehta /assets/img/favicons/favicon.ico /assets/img/favicons/favicon-96x96.png 2023-01-27T11:31:00+01:00 2023-01-27T12:20:13+01:00 https://shivammehta25.github.io/posts/temperature-in-language-models-open-ai-whisper-probabilistic-machine-learning/ Shivam Mehta

Introduction When I published my new work OverFlow to arXiv and added the model to the Coqui-TTS framework. Someone on the discord channel asked me that our system OverFlow had a parameter called sampling temperature what is that temperature? And recalled that OpenAI’s whisper and GPT-3 also have a sampling temperature. So I explained to him in the discord chat. So, I thought I might as well w...

2023-01-10T09:51:00+01:00 2023-01-10T11:18:59+01:00 https://shivammehta25.github.io/posts/deriving-categorical-cross-entropy-and-softmax/ Shivam Mehta

Introduction Recently, on the Pytorch discussion forum, someone asked the question about the derivation of categorical cross entropy and softmax. So I thought it would be a good idea to write a blog post about it with more details to it. Cross-Entropy from information theory The idea of this section is to understand the categorical cross entropy from a bit of information theory background. T...

2022-12-28T13:54:00+01:00 2023-01-09T08:17:07+01:00 https://shivammehta25.github.io/posts/welcome-to-the-blog/ Shivam Mehta

Hello and welcome to the blog (again)! Again because I used to post on shivammehta.me/blog but I decided to move to a new platform. I will try to migrate my old posts to here to keep everything in one place. Since the start of my PhD journey around two years ago, I’ve focused on learning new skills or sharpening the existing ones. Going forward, I will post new blog entries on this platfo...

2021-06-10T18:02:00+02:00 2023-01-10T11:02:22+01:00 https://shivammehta25.github.io/posts/universal-approximation-theory-the-intuition/ Shivam Mehta

(Migrated from old blog) Recently, I attended a course on Deep Learning and found this very nice intuition for how the Universal Approximation Theorem works. What is Universal Approximation Theorem? Universal Approximation Theorem states that a neural network with a single hidden layer and finite number of neurons can approximate any continuous functions in $ \mathbb{R} ^n$ (under mild condi...

2020-12-06T16:55:00+01:00 2020-12-06T16:55:00+01:00 https://shivammehta25.github.io/posts/pytorch-computation-graph/ Shivam Mehta

(Migrated from old blog) I started with Deep Learning when PyTorch was already a big name and with extensive community support, which is just growing every day the more I use it the more I fall in love with it as a whole package to do my thing and the best part is, even if there is no functionality already available for my specific use case I can always write something myself because all of Py...