THIS IS A DRAFT v1 as of 9 June 2022. Posting it early. Beware of bugs/errors. Will remove this notice once they’re cleaned up. Will also post full code soon.

I present the transformer architecture from ground up in Julia in this post. While my main purpose is to understand it in all detail without relying on any framework, I’m also hoping to try this out a way to teach machine learning … i.e. assuming whole program gradient calculation as a primitive. I won’t be explaining the why of the transformer architecture – for which I refer to Jay Alammar’s awesome tutorial – but mostly the how.

Read More ...

This is a post by my kid, His first post! The rest of the text below is his attempt at explaining odd and even numbers.

Read More ...

My kid’s math teacher recently posted a puzzle for the parents that got me down a rabbit hole of sorts and made me reflect on a number of things. This post is about that … and I mix the math and programming bits. I describe the puzzle in the first section and then the rest reveal aspects of the solution. So if you want to avoid the spoilers, you should stop reading after the first section. Read More ...

When talking about vector spaces, the notion of a dual vector space seems to often be introduced via postulates, such as in the excellent theoretical minimum series by eminent physicist Dr. Leonard Susskind. While that’s fine, I like an approach using linear functions, which I’ll present here. I learnt this from somewhere obviously, though I don’t recall where from. First a brief recap of vectors using Dirac’s notation. If \(\ket{a}\) and \(\ket{b}\) are vectors, then under a chosen operation “\(+\)”, \(\ket{a} + \ket{b} = \ket{c}\) is also a vector. Read More ...

As a TLA+ newbie, I found the fairness (weak and strong) formulae in TLA+ a bit hard going initially, so sharing the way I managed to wrap my head around it in case it is of help to others. What I’m hoping to get to is to be able to “read” the formulae in a chunked manner so that they make logical sense to me.

Read More ...

(originally posted on the Imaginea Labs blog - https://labs.imaginea.com/inai-rest-in-the-small/) The Inai project (forked here) began as a light hearted attempt to explore the consequences of bringing REST principles to organizing components within a single server node. It started off with the following question - Given we know that REST principles are useful to organize the development, deployment and evolution of large collections of services, would those principles be useful in the small - i. Read More ...

The Matrix Calculus You Need For Deep Learning (MCYNDL) is a great summary of what you need and expands starting from pretty much the basics. However, I’ve always felt that introducing the index notation used in physics can help simplify what looks like complicated math, even in that simplified intro. So here is my attempt at re-presenting the material in that paper, but using the simplified index notation. I think good notations are technology for the mind because they let you think and work with things that you otherwise might find too cumbersome or complicated to deal with. Read More ...

Tools in the “Scientific Machine Learning” (SciML) space have been getting more sophisticated and general purpose. While going through some material regarding “universal differential equations”, I couldn’t help dream up potential applications to tough problems in physical modeling synthesis of musical instruments .. particuarly interested in the vina family of instruments. Noting down some speculative thoughts here that I hope will refine and be replaced with more accurate ones as I learn more. Read More ...

The async/await facility in ES5 is perhaps the single biggest readability boost to async code in Javascript - both in NodeJS and in client-side JS code. With async/await, code that does async activities starts to read like synchronous code with the keywords async and await thrown in at appropriate points. But don’t lose sight of opportunities for concurrency while settling into the comfort zone of synchronous-looking code. One way to think of and use async/await usage in JS is like this - Read More ...

This is a stub post linking to material presented during my talk titled “I’m LisP. I’m inevitable.” at the first Chennai Clojure meetup at Pramati Technologies. Downloads View slides online PDF slides PowerPoint slides Video of talk