Everyone knew this: Abhishek wanted people to think he was smart. This wasn’t connected to how smart he actually was, which was reasonably. Nor was this secret drive something he ever let himself in on; it presented itself to others as a wild desperation when faced with a situation or topic he knew nothing about, and presented itself internally as a churning and prickling all over his body that drove his reward seeking system towards something, anything, that would stop the sensation. More often than not, the method of alleviation was simply to hold someone’s gaze and attention for, at minimum, thirty seconds of expounding, and remain unchallenged after. Abhishek was proud of his curiosity and desire to learn. He didn’t realize that he never really learned anything from other people, but instead just followed daily paths from Reddit and HackerNews until he’d run through three related Wikipedia articles. This was the way he was, and eventually, post-college, he reached an equilibrium where this core need of his was more or less satisfied most of the time.
It was September 2023 when he was invited to Pawan’s birthday hang in Prospect Park. He’d known Pawan since they were kids in Bergen Township together; their moms were friends for a while, but eventually the families drifted while Pawan and Abhishek stayed tight. They’d both gone to Bergen Academies for high school. Abhishek Rutgers. Pawan Princeton. They’d stayed in touch through the years and hung out when they were home together. They had an easy friendship: video games, tennis, talking about girls. When they went out together in the early days of being in the city, they’d get drunk and hug and clumsily say “I fucking love you dude, can you believe we’ve known each other so long.” This was their way for a while.
After years, they hung out less. Pawan had gotten weird about Abhishek’s job at Palantir. He spent a lot of time when they caught up talking fervently and earnestly about politics, even though he himself worked at Meta. Abhishek didn’t work on weapons or surveillance, he worked on building drop-in software for large chemical manufacturers to make materials ordering more efficient. There was a sense he had that the thorny problems of reality were, in fact, the only type of work that actually mattered, and that everything else was froufrou bullshit. This was also the opinion of everyone else at Palantir, and most of the people he’d gotten to know over the years in New York. This was his New York: Williamsburg, Dumbo, Spicy Moon, Málà Project, Downtown Brooklyn High Rise, friends with pools in their buildings in LIC, Instagram Reels, Strava obsession, nights first at Le Bain and Hotel Chantelle and eventually Public Records and Knockdown.
Abhishek was happy to go to Pawan’s birthday, even though he didn’t know a lot of Pawan’s new friends. Someone “dropped a pin” in the groupchat full of people he didn’t know, and he traipsed to the part of the Long Meadow under the big tree on the hill. He didn’t come to Prospect Park much. He got there an hour late, which to him felt fine, but everyone else was already there and sitting in a big circle. Lots of white people. He immediately clocked one shockingly pretty woman and looked away instantly. There was a pile of Trader Joe’s snacks in the middle. Abhishek didn’t know what to do with the White Claws he’d brought. He awkwardly set them down on the edge of the blanket.
“Hey,” he said.
“Hey,” said everyone, friendly.
“Is this—” he asked.
“Pawan’s?” said a mustache guy. Mustache guy was the most accurate term Abhishek had for making this guy internally legible. Short sleeved rugby shirt. Mustache. “Yeah. He’s in the bathroom. I think the line’s pretty long or something.”
“Yeah, I just went,” said the girl next to him, pale, beautiful, and resplendent in a floral skirt. “It was really long. I needed to pee so bad too, it was horrible.” Everyone laughed, the ice melted by the mention of piss.
“Dude, one time I needed to pee so bad,” ventured another girl across the circle, “that I thought I could fart quietly when waiting in line for the bathroom, and I just ended up fully pissing myself in front of everyone and couldn’t stop it. I just kept yelling ‘I’m sorry. I’m sorry.’ Once you release one, it all goes, y’know?” Everyone was laughing hard. One guy who was laughing really loudly kept laughing after all the others. He quietly repeated “I’m sorry,” sending him into further fits of glee.
Abhishek was taken aback. These people were all hot, well-dressed. They had glowing skin. There was an ease of well-being to them that seemed to flow around them, a comfort in their body. Though there were no specific visual indicators that tipped him off, Abhishek had the sneaking suspicion that half or more of them were gay, a rarity in his own friend group. He’d often thought of this type of white privileged yuppie as being sanctimonious, hyper woke, pretentious, and unfunny. But these people were charming, friendly, vulgar.
He sat down on the blanket next to the pretty girl that he first noticed. He felt guilty for looking directly at her, because her face exuded joy and beauty. He wanted to be near her but didn’t want to deal with the discomfort of looking at her.
“Hey,” he said. “I’m Abhishek.”
“Hey,” said one of the guys across the circle. “I’m David. This is Anna, Shae, Margaret, Hope, Esther, Dylan, Phil, and The Dog.” He said “The Dog” when pointing directly at the beautiful woman sitting next to Abhishek. Abhishek made a half smile, indicating that he understood that there was a joke happening. “We were just talking about Alex Garland. Have you seen Warfare?”
“Uh, no, is that a movie?” said Abhishek, the prickling and churning feeling creeping across his body. The not knowing feeling.
“Yeah, it’s out right now,” said David, in a warm tone that made it clear that Abhishek was not penalized for not knowing this information.
“He made Civil War too. And that movie Men. And Annihilation. And Ex Machina.” piped in Shae. She pronounced Ex Machina “Ex Ma-kee-na.” Abhishek assumed he’d been wrong for pronouncing it “Ex Masheena” all these years.
“Oh yeah, I love Ex Machina,” he said, overjoyed to have one of his favorite topics to talk about. “It’s such an interesting movie about AI. I really feel like we’ll have to worry about those kinds of suboptimal outcomes if we don’t figure out the alignment issue before AGI comes around.”
“Ugh, I hate AI,” burst out Esther. “It’s so ugly.”
“Yeah,” said Dylan, nodding. “All those images and videos are so fake and ugly. And it can’t write for shit.”
“Ten bottles of water to write the worst sonnet I’ve ever read,” joked Hope. “I’m so tired of hearing about this stupid thing.”
“I mean, it’s getting a lot better,” said Abhishek. “It can write code really well. It’s honestly inevitable. There are too many people working on it.”
“They need to regulate it,” said Margaret. “They need to protect artists.”
Something about this way of talking was shocking to Abhishek, whose circles adopted every new AI tool as soon as they came out, whose friends worked in AI or on LLM-wrapper startups, whose employer had built version after version of its own proprietary software that fell under the broad label of “AI.” No one in his world had ever really questioned that it was the future; everyone talked about the hype but no one ever really believed that the core underlying thing wasn’t valuable, and every single person he knew was trying their hardest to grab their slice of the pie before it was all snatched up. He felt surrounded by enemies.
The conversation moved on and AI was left behind. Eventually it splintered. Pawan was nowhere to be seen. The girl next to Abhishek was quiet. He decided to try.
“Hey, I don’t think I got your name earlier,” he said to her.
“Hey,” she said, brightly, “I’m The Dog.”
“The Dog?” he said.
“Yeah, I guess it’s kind of weird for new people. I mostly just spend my time with my friends and family so I always forget.”
“Like, is that short for something,” Abhishek said, trying to act like it was a normal nickname.
“No, that’s just my first name,” said The Dog.
“What’s your last name,” asked Abhishek stupidly. He still couldn’t look at her. He wanted to touch her skin to see if it was soft, but he knew this was wrong.
“Vasquez,” she said.
“The Dog Vasquez,” said Abhishek, becoming stupider every second. “Can I—is it—um—did you like, change it?”
“Yeah,” she said. “When I was eighteen.”
“Oh,” said Abhishek, reaching a nadir—or was it a zenith?—of internal stupidity. “And was that, um, because of like, gender? Or like, trans…” he trailed off, his last brain cell shutting down.
“No,” she said. “What gender would that name even be?”
“Non-um,” started Abhishek. He knew the word “nonbinary,” but he didn’t really know anyone who identified as such. He was worried that saying the word out loud, for some reason, would be intensely cringe, and he wanted so desperately to nonchalantly process that this beautiful Latina woman’s name was “The Dog,” accept it, not question it, and move onto the green pastures of conversation he could see over the hill, the verdant bursting forests and streams and brooks of his life together with The Dog, but he just couldn’t. He couldn’t understand why anyone’s name would be “The Dog.”
“Nonbinary?” said The Dog, laughing. “No, I’m cisgender. Boring.” She made a face.
“I don’t think you’re boring,” said Abhishek, way too fast.
“Thanks, I guess. Are you Pawan’s boy from high school?” asked The Dog. “He mentioned one of them was coming.”
“Yeah, I am. Love that kid. Damn, he’s been in the bathroom forever,” said Abhishek. He was unwilling to let it go. “Wait, so why did you change your name to The Dog?”
The Dog shrugged. “I thought it was funny.” This was the coolest thing Abhishek had ever heard. He tried to picture his mother’s response to him changing his name.
“So your parents are just like, cool with it?” he asked.
The Dog shrugged again. “They were mad at first, but like, it’s been ten years. They don’t care anymore.” It was clear she was tired of this topic. Abhishek felt like he had to stop time for a moment and clean out his brain to be able to process the idea of The Dog, the way The Dog lived, the way The Dog did whatever The Dog wanted. He paused for a second to allow himself to do this.
“So like,” he started hesitantly, “how do you know Pawan?”
“We’ve been dating for like a year!” she said chirpily. “You haven’t heard of me?” at this, her face fell a bit.
“Oh, I guess we’ve kind of been out of touch,” he said lamely, feeling like he’d been struck by lightning. Pawan was dating The Dog. That’s why there was an empty spot on the blanket next to The Dog. That’s why Pawan had been so flaky for a year. Pawan was dating The Dog. Pawan was kissing The Dog. Pawan was fucking The Dog. Pawan was cuddling with The Dog. It had been a year. They were probably in love. The Dog loved Pawan and Pawan loved The Dog. “Damn, that’s crazy.”
“Heyyy, Pawan,” yelled Dylan. Pawan was walking up behind Abhishek.
The Dog put on an affect. “Damn big boy, why you take so long in the bathroom? Takin a huge shit or something? Takin a big dump?”
“Why, you like that kinda thing?” Pawan said with a huge grin.
“You know I do,” said The Dog, giggling, and popped up super fast and gave Pawan a kiss.
Abhishek was struck with misery in that moment that never lifted for the rest of his life. He felt like he saw through The Matrix. The truth of this life was that you can date a beautiful funny woman named The Dog, and no one cares that her name’s weird. On his deathbed, he finally let the memory go and thought, “Well, I’ve lived a good life,” right before he passed. But that was only because to really believe the alternative would have just been too sad.
Nabeel’s Footnote
Stop dude. Just stop
I’ve been to a lot of these idyllic picnics with beautiful souls who love to talk about pee and poop and cum. But usually at these things there are both very hot people and very ugly people. And when you get to know these people better, it always turns out that most of them—hot and ugly—are completely miserable, despite the fact that they’re also all on anti-depressants (there’s usually one or two hot gay dudes who really do seem to have found happiness). Still probably better than being a generic South Asian techie, although I really have no idea what that’s like.
Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment?
The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt.
What are folks’ theories on the version increment? Is the architecture significantly different (not talking about adding more experts to the MoE or fine tuning on 3.7’s worst failures. I consider those minor increments rather than major).
One way that it could be different is if they varied several core hyperparameters to make this a wider/deeper system but trained it on the same data or initialized inner layers to their exact 3.7 weights. And then this would “kick off” the 4 series by allowing them to continue scaling within the 4 series model architecture.