Let’s Read: Transformer Models, Part 1

Large Language Models (LLMs) are a hot topic today, but few people know even the basics of how they work. I work in data science, but I also didn’t really know how they work. In this series, I’d like to go through the foundational paper that defined the Transformer model on which LLMs are based.

“Attention is all you need” by Ashish Vaswani et al. from the Proceedings of the 31st International Conference on Neural Information Processing Systems, December 2017. https://dl.acm.org/doi/10.5555/3295222.3295349 (publicly accessible)

This series aims to be understandable to a non-technical audience, but will discuss at least some of the technical details. If the technical parts are too difficult, please ask for clarification in the comments. You’re also welcome to just read the TL;DR parts, which should contain the essential points.

[Read more…]

Origami: Stars

origami stars

Five- and Six-pointed Stars. Designer unknown.

Back in 2019, we had a small wedding celebration–we didn’t actually hold a wedding reception, and that’s a story that I’ve already told.  As decorations for the celebration, I made a dozen giant paper cranes (actually Tsuru Roses) from wrapping paper, and you can see a photo of those at the bottom of my story.  I also made 50 origami stars from foil paper and holographic paper, seen above.

We’ve officially reached our 5th anniversary!  I am not inclined to be sentimental, but I am grateful for how incredibly fortunate we are.

My history in music

At the end of each year, I have a tradition of sharing a bunch of music. It started out as a little antidote to Christmas music, but then I started to focus on a different theme each year.

Today, I’d like to share music that represents my personal trajectory. I started listening to music in high school–and by that I mean actually choosing music for myself instead of just listening to whatever’s playing on the radio. At first, I gravitated towards electronic and techno. From there I started following a few individual rock bands. Then I went into more niche genres.  There were several different threads running in parallel, and I’d like to share a few examples of each.

[Read more…]

Laudatory categories

Theseus’ ship is a philosophical thought experiment that asks what happens if you take a ship, replace it piece by piece until none of the original pieces are left. Is it the same ship, or is it a different one?

Now imagine the following response: “It depends. Is the ship seaworthy?”

This response is a bit absurd, because clearly the question does not depend on whether the ship is seaworthy. A ship may still be the same ship while falling into disrepair, or perhaps the ship was never seaworthy in the first place. And on the other hand, you could have another ship which is also seaworthy but is nonetheless a different ship. We may disagree on how to answer the question about Theseus’ ship, but surely whether the ship is seaworthy is besides the point.

Nonetheless, this seems to be the way people think about many categories. A laudatory category is one whose definition has become intertwined with the question of “is it good?” A pejorative category is one whose definition has become intertwined with the question “is it bad?” Let’s talk about a few examples.

[Read more…]

Link Roundup: December 2023

Just a short link roundup this month, a few days late.

Plagiarism and You(Tube) | H. Bomberguy (video, 3:51 hours) – It’s alright if you don’t watch this one, I’ll give a quick summary.  Harris demonstrates the extensive plagiarism of a few specific YouTubers: Illuminaughtii, Internet Historian, and James Somerton.  Somerton is the big one because he was an established gay youtuber.

This drama has been all over the internet and back again at this point, but I thought I’d highlight a few other responses I’ve seen.  Todd in the Shadows (normally a music critic) had another long video about Somerton’s made up lies.  Verity Ritchie talked about Somerton ripping off their video without quite copying it.  The Ace Couple talked about donating to Somerton’s bogus film production, and dealing with his ignorance on asexuality.  I also recommend this (text) article on Plagiarism Today, which provides further expertise on plagiarism.

Harm and Justice | Leftist Cooks (video, 2:04 hours) – It’s a video about restorative justice, basically consisting of a series of shorter video essays about each step in the process.  I particularly like the part about unrecognized rape (“Naming the harm”).  I guess most people wouldn’t remember me talking about it, but that happened to me a long time ago, and it’s very much on point.  I also love the part about apologies, and then later about how forgiveness isn’t an obligation.

Youtubes I didn’t like

Since it was revealed that gay youtuber James Somerton was engaging in extensive plagiarism in his videos, many of us are reflecting on how it relates to us personally. My first reaction was that I never liked Somerton to begin with. His videos were very disjointed, and they took forever to get to the point, or never got to the point at all. I also spotted one or two factual errors that made me think he didn’t know anything.

But I feel for his former fans, and I don’t blame them for liking and trusting Somerton. How could they have known?

And I never said out loud that I disliked Somerton, and why would I? I just thought he was a bad essayist, I didn’t think he had actually done anything wrong. Maybe if more people spoke up, stuff like this would come to light sooner?

I’m grasping at straws, I’m not convinced for a minute that this is the right answer. But let’s try it for a moment. Let’s talk about a few youtubes I didn’t like and why.

[Read more…]

Critiques of Train

cn: Discussion of Holocaust media, while avoiding discussion of the Holocaust itself.

Train (2009) is a board game slash art installation by Brenda Romero.  It’s the most famous game in her series The Mechanic is the Message. There is only one copy of the game. In the game, players are tasked with transporting meeples by train. Over the course of the game, the players learn that they are in fact taking the role of Nazis transporting people to concentration camps.

Train makes a provocative and valuable statement, and I am by no means outraged by the game. However, I do have a few bones to pick with it.

[Read more…]