Do 60-Minute Coding Tasks in 60 Seconds—With AI - Ep. 41 with Steve Krouse

[00:00] It's just so cheap to make software that we can make an app for this podcast and then never use it again. So there are three parts. One's a timer. Another one is for notes. And then the last one is a chat GPT integration where it'll read the notes and then generate more questions. And I'm laughing because it finished the app before you were done describing what the app was to me. [00:23] That's actually super cool. I love this. [00:30] soon. That's been a long-term ambition. There are people who really don't know the first thing about coding and come on to Valtown and make stuff like this in a minute or two and their mind is blown and they're so excited and that's amazing. Steve, welcome to the show. [01:00] to be on your podcast. So for people who don't know, Steve and I have been friends for many years at this point. We met in college. Steve is the co-founder and CEO of Valtown. And Steve will be able to describe Valtown better than I can. But it's sort of like a social coding site. Steve, how do you describe Valtown? Yeah, it's, I think, harder than most companies because it

[01:30] we have in our website, I think holds up remarkably well. If you're a programmer, it's if GitHub Gist were fun and AWS Lambda, no, sorry, if GitHub Gist could run and AWS Lambda were fun. Exactly. So, I mean, basically, it lets you code in your browser and share your code snippets and build and compose code snippets together. And you recently released Townie, which is a very cool AI coding agent. And also, just full disclosure, I'm an investor, [02:00] all town. I think it's awesome. Um, you're doing really, really amazing stuff. Um, [02:04] And I will also say, you are actually one of the reasons, one of the biggest reasons I started this podcast because you were like, you know, Dan, I'm a fan of the podcast. [02:12] You write all the time, but I don't have time for that. I want to just listen to what you have to say. And you had sort of a vision for what this could be. And so I really appreciate you pushing me to do it because it's been super fun. And also just an honor to have you on the show. [02:29] Thank you. Yeah, thanks for doing what I want. I think it's important for your audience to know that I started out as an audience member of yours in the first place. That's how we met. I stumbled upon Dan's blog when I was a freshman in college, and he was a junior at Penn. And I reached out, you know, I should have found that email, because we probably still have it. Like, they're very cute. You know, like, I'm a little freshman, please take pity on me and mentor me. I want to be like you one day. So, yeah. [02:58] Um, [02:59] It's wonderful that I've found myself in the content that I was an audience member of.

[03:29] kind of stuff. Do you want to give people just a little bit of a... [03:33] overview of that part of your brain and where Valtown came from and why it's important to you? That'd be great. Yeah, it is a longer history, I think, than most startups. And there are a lot of startups, I think, that look like Valtown, [03:46] today, but hopefully we have some deeper historical perspective because of my background. So I think where I'll start the story is I went to this kind of life-changing after-school program that taught me how to code when I was in middle school. And I, through that program, fell in love with mathematics and fell in love with programming. And overall, I just felt like a smarter human. And I knew it had something to do with this coding program. And in college, I went to a hackathon, [04:16] which was a whole nother eye-opening kind of experience. But one of the things that happened at that hackathon [04:22] was that I was pointed at the work of Brad Victor and Seymour Papert. And I read Seymour Papert. And it like turns out that it was all on purpose. [04:30] Who is that? I know Brent Victor, but I don't know Seymour Papert. Yeah, yeah. So Seymour Papert was a... [04:40] a mathematician and a... [04:43] like educational theorist. So he studied, he was like a mathematician who then decided to study with John Piaget and [04:50] like one of the fathers of like the stages of development kind of educational theories. And Seymour Papper had this question, which is like, why are some kids bad at math, but no kids are bad at French? Like if you're, if you're bad at French, it's because you didn't grow up in France. But we know that if you grew up in France, you'd speak French just fine. Like some kids are just genetically bad at math. That doesn't, that didn't make sense to him. He, his conviction was that those kids who seem like they're bad at math genetically just didn't grow up in math land,

[05:20] Yes. [05:20] had some sort of math land experience growing up. And his experience was like that. He got obsessed with gears growing up and other mathematical objects that were just in his house. [05:30] Got it. Okay, cool. So that makes a lot of sense. So continue with the story. Yeah. So he wanted to see, can we make kids good at math? Can we make a virtual math land on this new computer thing to make kids good at math? And what he came up with was actually a programming language, but it was never to teach kids to code. There were no jobs for coding back in the 60s and 70s. It was like, can we trick kids into being good at math? [05:52] Through programming. [05:54] And it totally worked for me. And so ever since I [05:58] lived it and then read about how it was all on purpose, it totally blew my mind. And I became obsessed with the power of programming languages, programming environments, developer tools for, for not only the power that they give human beings to like create stuff, but the way that they change the way you think and like actually make you a smarter human in other aspects of your life. This is, this has been the thread. So like that moment in when I was like 18 years old, has followed me for the last 10 years. How do they change how you think? [06:28] One of my favorite examples is in logo geometry, the way that the logo turtle works. So the programming language Seymour Papert made was called Logo. It's the predecessor to Scratch. So if you've ever used Scratch, it was made by his student, and there are a lot of similarities. And in both programming languages, the model is that you have a character on the screen, and you can give instructions to move forward or turn and move and turn and move.

[06:58] you would have a pen on your feet. So you could say, put the pen down and then give it instructions on how to move and then pick the pen up and then there'd be a shape on the screen. So an example of... [07:07] how like I learned mathematical concepts. I grew up in math land through this is that like, how do you draw a square? How do you draw a hexagon? How do you draw a circle? Like you, you, you build up as you kind of intuit how to do these things, you build up an intuition for, um, what later in life you'll learn as like a derivative in mathematics. So like, I don't know, five years after I like was, was drawing circles with my feet, you know, mentally you learn to [07:37] problems, like putting yourself into the code. Like imagine yourself. Okay. So if I was like this, like programmers do that all the time. So you have to, you have to morphize the code. [07:45] In this specific example, it's very geometric. So then a couple of years later when I was learning derivatives, I was like, oh, the derivative is just the way you're pointing if you're walking along the curve. That's very easy. I've walked on curves for a long time. And I remember in the class, I was tutoring kids after that class. And everyone was like, oh, my God, Steve's a genius. He got it so fast. I was helping the teacher on the board. But I knew I wasn't born this way. I had these very powerful experiences that nobody else in the class had. [08:12] That's really interesting. So let me make sure I understand the example you just gave. So basically, if you're programming in this environment and you're trying to make a circle, the way that you end up having to do that is by breaking the circle into small little segments that are like the angle of the segment changes just a little bit each time so that you end up making a circle because circles aren't actually round.

[08:42] then becomes a derivative. And so the concept of derivative was intuitive to you because of that. [08:48] Yes, yes. It was so easy for me to put myself on the paper, like my feet are on the paper and I'm walking along the curves. [08:54] That makes sense. I mean, I think it's such an interesting... This is a whole rabbit hole we can go down, but one of the really interesting things about the history of math is it's only in the 19th century that we start to get formalism in math, where you're using all the equations and stuff that get people intimidated. If you read Newton or Galileo, they're interspersing information. [09:20] Geometry with like actual written text, like there's a lot more of a it's like where the sort of Renaissance man idea is like comes from like there's a lot more of an integration between all the different. [09:33] senses and ways of thinking to help you understand math. And it was only later on that we formalized it, which is helpful because the formalism is really powerful, but [09:45] You end up losing a lot of the intuition unless you're like a super math genius or you're lucky enough to use a programming environment like this. And I think that's what makes people, makes it much harder and less appealing for people, which is interesting. I will also say, though, like as someone who like I taught myself how to code, I'm still terrible at math. I think there's something it's possible I haven't learned. Right. But like there's something for me about and this shows up in other areas of my life, too.

[10:15] me about just sequential processing, processing sequences where each step of the sequence depends on the previous step. I'm very likely to, A, it costs me a lot to do that. And B, I'm very likely to skip a step or get them in the wrong order or do all that kind of stuff. So any math that I can do that's more intuitive where I'm almost sizing things in my head as opposed to following [10:45] sequence of steps, that I can do. But the sequence thing really messes me up. And that's why I kind of like the AI stuff because I kind of get things intuitively. So I kind of know what I want to do. And the AI stuff just figures out how to write the code or do things step by step so I don't have to... It fills in that part of my brain. Totally. Yeah. Yeah. It makes a lot of sense. Yeah. So you're thinking a lot about the future programming. That's kind of what you're building [11:15] And I think like a really interesting, um, [11:18] Intro to this episode is about a year ago when this podcast was first starting, I had this guy, Jeffrey Lidon, who I know is a friend of yours and also someone that you really look up to and respect. [11:30] The cool thing about that episode with him is we live coded an app together using ChatGPT and Replit. And at the time, that was pretty new and pretty cool. And to me, it was wild that you could make an app on a podcast just while talking. That was previously totally impossible. But it took us about an hour-ish...

[11:58] to actually do it. And there was false starts and whatever. So we got it done, but it was not super easy. And I think it would be fun to see how far we've progressed. That was your prompt for me in this episode is to see the difference between last year with Jeffrey lit about a year ago and where we are now as a benchmark of AI progress. And I think you're going to use Townie, [12:28] product to do this. Do you want to introduce it for people? Yeah, sounds great. So yeah, Valtown is a website to write code and share code and deploy code, very importantly. And [12:43] One way to think about Townie is it's taking what Dan and Jeffrey did a year ago and just automating the steps. We were seeing people doing this with Valtown for the last year. They would go to an LM and ask for some code and then paste it and then get some error and paste it back. It was a crazy thing. It seems like over the last year, we've all agreed that that's crazy and that they should be deeply integrated into a single product. That's what Townie is. [13:07] We've... [13:08] work through a number of different iterations of how Tiny works. The current iteration I think is most similar to [13:14] anthropic artifacts if you if your audience is familiar with that definitely i think maybe let's start with just like the valetine home page so people can get a sense for like what the product is i read everyone the poem but i think um [13:27] I could start with the description I give people when I meet them on the street and they say, what do you do? I say that I make a website for programmers to make websites. I think that kind of anchors it really well. It's a tool really designed for programmers, and it's designed for you to make web things, particularly server-side where you can do front-end and full-stack stuff. And I think one of the most important distinctions is that it's in the browser, you write the code, and then it also scales with you to deploy and scale.

[13:57] off to... [13:57] then redeploy it somewhere else. So it's like all integrated and drastically simplifies the programming experience. [14:03] God. [14:04] Cool. Okay, great. So this is Townie where we can describe in English what we want to build. I'm going to paste in and start running the app that you and Jeffrey made. And like as it's coding, I'll describe what's going on. [14:18] So... [14:20] We're making an app for this very podcast we're on. The idea from your episode with Jeffrey was that it's, [14:26] throw away software, disposable software. It's just so cheap to make software that we can make an app for this podcast and then never use it again. So there are three parts. One's a timer. Another one is for notes. And then the last one is a chat GPT integration where it'll read the notes and then generate more questions. And I'm laughing because it finished the app [14:56] I love this. [14:59] Man, I haven't had one of those real AI wow moments in a while. I feel like I'm getting a little jaded, but that was definitely a wow. [15:08] Because, you know, yeah, we made this app with Jeffrey... [15:11] a year ago, it took us about an hour. And there was a lot of copy-pasting back and forth or whatever, and you just made it with just pasting in a description of what we did in the episode. And it's just here, and it works. Wait, can you make sure all the stuff works? Does the generate questions button work? Yeah, yeah. So I'm talking to Dan Shipper, or I guess I'm Dan Shipper talking to... Crass.

[15:40] founder of Valtown fast platform for, I don't know, whatever, whatever. That's fine. Generate questions. So you typed into the interview notes. We have this whole podcast set up. You typed into the interview notes. I'm Dan Schipper talking to Steve Krauss, founder of Valtown. And then you press generate question. And then it generated, it used GPT-4, I assume, [16:04] some AI model to generate questions. And it says like, you know, here are three questions to ask as a founder of Alltown. What was the original vision? So like, it's actually there, you know, reasonable questions. It's not totally, not like totally off. And what's really interesting about this is like, so, you know, obviously we just saw the whole thing just like was made in with, with a single prompt. But what's interesting is this requires a client and a server to be working together and, [16:34] Because, like, in order to, I assume, in order to have the generate questions button work, it's like sending it from the client to the server, the server sending it to OpenAI, and then OpenAI sending it back. That's hard. Like, there's some complexity here. So, I don't know. It's cool. Thank you. Thank you. [16:54] Yeah, I can kind of explain any part of it. Maybe you direct me, like, what you're curious about, or I could just go piece by piece. [17:04] like what is... [17:07] How does this work and how is it able to fill in all the gaps there so there's no copy and pasting? And am I right? Does it have a client and a server? How does it all work? Okay, great. So from a high level, the way it works is...

[17:23] We take your prompt and then a big-ass system prompt that explains how Val 10 works, and we send that off to Cloud 3.5 Sonnet. And I think it's very important to underscore how much of an enabler Cloud 3.5 Sonnet is. When you and Jeff were doing this with, I don't know, ChatGP3 or 3.5 or even the current ChatGPT models, [17:43] We wouldn't get something this good probably, or it would take a lot longer. [17:48] Yeah, Cloud 3.5 Sonnet is like, what's changed since when you and Jeff did it? Like a big part of it is it's all integrated in one tool. We spent a lot of time getting a good system prompt. And then the most important thing that happened was Cloud 3.5 Sonnet. [18:02] Right. Right. That makes sense. Oh yeah. Let's look at the system prompt. So like, what are the core components of the system prompt that like, that made it start to work? [18:13] Yeah, there are a lot of iterations. And if you want, if anyone wants a really deep dive, here's like how we built the original prototype. And it's changed a lot since then. And then we have, we just published a new article about like how we're running it in production. So we have a lot of notes for people who want to build their own. But yeah. [18:31] At a high level, how this system prompt works is it explains [18:37] where the code that we're asking it to write is going to live. We tell it that it's running in Deno JavaScript, not Node.js, not client-side JavaScript. We tell it that it doesn't have to worry about starting the server, deploying the server. We give it all the formatting. We have random bugs in our platform. We tell it about the bugs, about things it shouldn't do, because there are bugs in our platform. And then we give it all of the...

[19:01] Um... [19:03] all the included platform features. So for example, how to do [19:07] OpenAI, which we just saw. We explained that it's available to it and here's how to use it. [19:15] Hmm. That's really interesting. Um, and it's nice, it's nice to have that stuff cause it's, that's what I would normally have to paste into, um, you know, chat to your cloud or whatever. So it has the most update up-to-date docs. Um, yeah. [19:28] I guess, how has the usage been or what have people been doing? If the underlying thing that gets you excited has been... [19:37] How programming changes the way that people think and what they can make, like what has been the effect of. [19:45] Townie on that for you. [19:47] Yeah, it's been fantastic and also a little bit confusing. I didn't expect... [19:55] have non-programmers be able to use Valtown so soon. [20:00] That's been a long-term ambition, but there are people who really don't know the first thing about coding and come on to Valtown and make stuff like this [20:09] in a minute or two and their mind is blown and they're so excited. And that's amazing. I love that. Um, [20:16] Like, you know, yesterday, someone made it like, or recently now that, [20:21] Bitcoin is doing so well. People are making like Bitcoin price trackers, wallet things, like all sorts of like analyzers for crypto. And like Valentine's really good at that. It could just plug and play with the APIs.

[20:31] And on the one hand, I'm really excited about it. On the other hand, I'm getting... [20:37] questions that are so bad because they're not pro like it's a, it's a tool for programmers and then I'm getting, and then like, [20:45] You know, like here, I'll just like the one I got last night was this, like, you know, let's just say we remove this and save it. And by this, you mean basically what you did is you went into the code and you removed a semicolon from the JSON object, or I guess it's CSS. I removed like a quote of the CSS. Yeah, I purposely put a syntax error in the code, a very simple syntax error that like a programmer would have no problem solving. [21:15] but if you're not a programmer you're like this is the worst experience i've ever had in any software ever totally yeah i mean like what are you going to do with that if it looks so intimidating like type error you know and if you're a programmer you're like okay i can fix that or whatever yeah i think that makes a lot of sense that's that's really interesting so like what has that been like for you because [21:39] I've watched you over the last 10 years or so be thinking a lot about the [21:46] like the ways to create a programming language that would be powerful, but also simple enough that people can like people who are non-technical can like easily sort of like onboard and learn to use it. [22:01] And it seems like a lot of those sort of language design techniques.

[22:07] initiatives are just completely changed by the fact that you can just code with English now. What has that been like for you? Yeah, it's been very surprising. I didn't expect this would be how things are. On the one hand, [22:22] Yeah, like you kind of rethink things from scratch. Like, for example, this type error is totally nonsensical to you and me. But like, look, look at this little button here. Like, ask Townie to fix it. And then it just sends it to Townie and Townie. [22:38] immediately figured out the issue. And then the only crappy part about this experience is that it's going to regenerate all the code from scratch. But besides that, the example I gave isn't necessarily... [22:54] a condemning thing. Like, you know, they're like the problem for LLMs is just, or the solution to these problems is often just throw more LLMs at it. So I guess given that you like made this for programmers and now it seems like you have like a whole wave of like non-programmers starting to use it, that seems like a big question for like the business of like, who do you want to serve? Like, yeah. What are you thinking? I'm really tempted to go after people who are non-programmers. [23:24] because it seems like there's just like incredible pent up demand and there's just so many of them and they're not served by any other products. And at the same time, [23:33] I... [23:35] I really like serving the smartest customers and, and like sophisticated customers and like building a pro tool. It's like, you could think of the difference between like Figma and Canva. Are you going like kind of a tool for everyone or are you building a tool for professionals? I think, I think that's kind of where I'm drawing the line. And like the advice I gave to the guy who emailed me about this, this thing right here, uh, like the first day he asked the question,

[24:05] the same exact question. He was like, what's happening? Why do I have this error again? And so my response the second time was, here's how I would solve it. I guess where I'm going at for people, I think there's a whole class of people who are allergic to code. They don't even want to look at it. They don't even want to know it exists. Totally abstracted away from me. And [24:25] Valentine is not the product for those people. The product for those people is... [24:30] Anthropic Cloud Artifacts, I think, or GitHub Sparks, maybe. I think there are products for people who don't want to look at the code, but Valtown is not that product. Yeah. First of all, I think it's really great that you have a sense for the kind of customer that you like to serve. I think that's actually quite rare. And even if you have that sense, it's often quite hard to allow yourself to be guided by it. But overall, I think you will make [25:00] Because it sucks to not like your customers or not necessarily even not like them. But it's much better to really love your customers and want to interact with them and all that kind of stuff. And so I think that makes sense. It's funny because I was kind of, while you're answering, my response to this is like, [25:22] And I know very little about your business. So, you know, take this with a grain of salt. But like, my feeling is like, I think that there's this new type of programmer emerging. [25:34] which is like, I think a lot of people who are,

[25:39] There's already really great programmers. Some of them are using AI, but a lot of them aren't. I can go faster just doing it myself. And there's this new version of programmers who are really AI native and programming with AI is almost a different skill set than programming without it. Yeah. [26:00] And right now, those people look very unsophisticated, but they will be very, very sophisticated in like 10 years. [26:11] And so the example I would draw is like, if you were talking to Mr. Beast like 15 years ago when he first started making YouTube videos, you'd be like, this guy is dumb. Like he's 12 or whatever. [26:30] in the world with some of the biggest budgets. [26:34] And so my feeling about some of this stuff is like... [26:40] And if you reach those people now and grow with them and help kind of instill in them some of the taste that you've built up and experience you've built up over many years, I think that there's just a chance to make a platform for programming that's sort of like YouTube, but it's for programming, it's for building stuff. And those people are not going to be sophisticated programmers. They're just going to be like kids with dreams.

[27:10] - Good. [27:11] that I can get the future Mr. Beasts, in your analogy, who want to put in the work. [27:17] uh to me that's the distinction uh like i think there are people who want to turn off their brain and like just use an llm and do you know what i'm talking about like i don't want to disparage because i really do believe with you that there's a new kind of ai engineer who's unbelievably powerful and and i love watching those people and then there's another kind of llm engineer who will like ask the llm to do something and then leave their computer go do something else and then come back and see an error and just hit the fix it button and go and leave and do something [27:47] They're allergic to the code, and I'm skeptical that those people are giving themselves the right feedback loops. [27:56] to become the next Mr. Beast? Maybe. I mean, I think like, I'll just speak for myself. Like sometimes I do that because like one of the things that I think is, is interesting about these tools. And I've talked about this a couple of times before on the show is like, um, you know, [28:12] Right now, programming requires a lot of focus and attention. And it still does. But what's really interesting about these tools is you can actually make progress even when you have fractured attention.

[28:42] is like, I'm busy. I have a lot of work going on today, but for me, I have this idea I want to make and it's kind of fun to be able to jump in, spend five minutes fixing a thing and then let it go off and do its thing and come back. And that's not to say... [28:57] you should also get rid of focused work. It's just that programming becomes possible in a fractured attention state, which was... [29:07] It was always possible if you had enough money to hire people. But now everyone can do it, which I think is kind of cool. And I also think, I don't know, if we extend the YouTube analogy, to get one Mr. Beast, you had to have 100 million people uploading just random stuff that they put no thought into. So yeah, I think it's kind of interesting. I totally see what you mean. [29:37] who like care. And, um, it also seems like this is, we're at such an early stage of this that like, in order to get those people, you have to like make it so accessible that like tons and tons of people can try it. Um, but maybe I'm, you know, again, you've thought about this way more than I have. [29:55] Yeah. I think a big variable that we're like not, that we haven't yet talked about is what's happening to the underlying models, like on what timescale. So like Sonnet 3.5 has made all of this, this whole conversation possible. And if we don't have another step change from that in the next year or two, then I think maybe like what I'm talking, like people are going to be more limited by human focus and care.

[30:25] relative than, than what you're talking about. It makes a lot more sense. Like beginners are, are, are, [30:30] get even more leverage. So like the rate is gonna, but eventually I think we'll get to a point where [30:38] Um, [30:39] where you're talking about makes a lot more sense. Like you'll just have agents running off in the background. Like they'll be coming up with ideas. They'll be talking to your customers for you, coming up with ideas to make things better. Who knows? [30:50] Well, I guess what world are you planning for? Are you planning for a world where progress is starting to asymptote a bit or are you planning for a world where we're going to continue to see the same kind of jumps in progress that we've been seeing? [31:01] I would like to be like neutral to underlying LM changes and be prepared to like, if the LM is the same, then, you know, our tool continues to work. And if the LM gets better, in theory, we just like change the model name and we get the benefits like everybody else did. And I think we saw this happen with Cursor and WebSim when Sonot 3.5 came out, they were ready and they jumped. And so I think now Valton is ready for the next model jump if it happens. That's really interesting. [31:31] What have you learned about software development and running software teams in the AI age? Because it's very different from programming before. And there's different sets of... [31:46] ideas and like methodologies and it's all it's all different to have like squishy software i was talking to um simon last about this and i i'm curious about like your perspective on it like what has changed for you it for like the the valtown team like how we're working with like like a serious engineering team serious software versus like software you make in valtown that's like a lot squishier yeah so right now there's such a huge gulf uh like when you write code in

[32:16] and... [32:18] Like every week or two, you have to make a tweak to that. And so like, I... [32:23] And the one person on the team who doesn't have that set up on my, like, I can't make a change to our production app because it's just like... [32:31] too much to keep that local software up to date with the team. So you make your change locally, you get all your databases, everything running locally, uh, [32:40] You have to do automated testing. There's just so much paperwork. It's really important stuff to keep our service running smoothly. You submit a pull request, you get a review on the pull request, you do feedback, you deploy it, deploying takes 10 minutes. It's much slower. [32:58] Yeah, it's a totally different feel than Val 10 where every save is a deploy. It happens in 50 milliseconds and you're alive. You're off into the races. Right. I guess what I'm saying is like doing the programming and ensuring a quality product and running tests and all that kind of stuff is... [33:16] it's significantly different when the code you're running is deterministic versus stochastic. [33:25] And I'm curious how that has unfolded for you or what you're learning about, like keeping an LLM app running well in production. Got it. Got it. Yeah. Yeah. [33:36] It's like the word... [33:39] There is evals. Like that's like the, how we as an industry have gotten some amount of predictability or, or understandability about our, how LMs are performing. We didn't have real evals for the first three, four months of Townie. And we would just like YOLO changes. And like, we would like...

[34:04] make a change, we'd test it out ourselves a bunch, we'd deploy it to production, and then it ended up, it totally seemed better, and then we'd get anecdotal reports that it's worse, and we wouldn't really know, and then we would YOLO another change. These things, I feel like it really brings you face-to-face in contact with what a truly stochastic thing is like, because there have been multiple times when [34:31] I go to Twitter and say, "Our users are all reporting X." And does anyone else see X? Has the model gotten stupider? Has the model gotten smarter? Is the model doing this for other people or is it just Belltown? [34:42] Like there'd be, I feel like I was part of, I don't know if you saw this on Twitter. There was like a couple of months ago, this like weird panic where everyone was like, Claude 35 Sonnet has gotten this dumber this weekend. And like everyone in the therapy was trying to figure it out. And I think at the end of it all, it was just a panic that people like me helped contribute to. We were, where we all just tried to, we all just kind of convinced each other was super smart and then had a couple of bad, like interactions with it. And then there was a panic that it was dumb. [35:12] And that's why evals, I feel like evals are important for your mental health. [35:15] Yeah, that's really interesting. I hadn't even thought about the social contagion aspect of the perception of quality. [35:24] You know, that's wild. That's so, I mean, I guess it's like you can draw analogies to like the stock market. [35:33] Um, but your product quality being like the stock market is like a very new, it's a new thing. Very new. Yeah. Yeah. And yeah. So like, yeah.

[35:44] But evals let you kind of like point to something and just like feel good about yourself. Because customers come to me all the time and they like complain about a specific thing. And then I... [35:56] can look at the evals and see if I can find what they're talking about. And if I can't, I can say, I'm sorry, try again one more time. It'll probably work with confidence. Yeah. Yeah. Yeah. I guess then that runs into the problem of then you only take seriously things that you can measure or changes that you can measure. How do you think about that? You're starting to only see things where there is an eval for it. [36:26] very squishy product. And I think customers or users are constantly having the, [36:31] less than optimal experiences, [36:33] And hopefully the tool allows for error correction. [36:38] Yeah, like all the time, I'll talk to Jackson, our team, who does the LLM, the eval stuff. And he's like, yeah, we don't even have an eval for that. Like when Claw 3.5 Sonnet New showed up, we were so excited to run evals on it. And like the evals weren't any different because like we didn't have any evals that were hard enough that like it would show up that it got smarter. Right, right, right, right. That's interesting. Yeah. [37:03] I definitely vibe with that. I think like... [37:07] For any product that we've made that I've been the one to start, there are no evals and we just sort of yellowed it for a while.

[37:16] The official term, yellowing it. Yeah, I think that's great. I want to make yellowing happen. [37:24] And I think we've just now started to add evals, like to spiral. And we have a couple of internal incubations that are not released yet that definitely have evals. [37:37] for us is with Spiral, for example, when you change the model, we also use the new model to change the default prompt that we use to generate new things. But then we didn't want to go and update the prompt for everyone. So we just have a thing that's like, do you want to upgrade the Spiral to the new version of the prompt and people press upgrade, which has been an interesting thing to try. That's really cool. We've struggled with that internally where if you... [38:06] we let you edit the system prompt, but then when we push an update to the system prompt, [38:12] you don't get it. Yeah. It's complicated. And if they've changed it, what do you do? Do you diff it or whatever? But I think at least letting people opt in, letting people know, hey, there's a new prompt and letting people opt in, it seems to work so far. And then you basically make a copy of the spiral so you don't lose anything. And then maybe you can go back and forth if you want to. That's cool. Yeah. So you put the system prompt as a property of a spiral, [38:42] Now, your system prompt in Valetian is a property of your account or of your settings. Yes. Each Spiral has its own prompt, basically. Because Spiral is effectively a prompt builder, like a fancy, fancy prompt builder. So yeah, each one has its own prompt. One question I have for you is, you said previously that what you want to be doing is building an application where you're neutral to the rate of progress. And if the progress is

[39:12] is to the upside, then it's like a really nice surprise. How do you think about doing that? How do you think about architecting your product and your systems for that? Yeah, it's a good question. And it's interesting because it just happened. Like we built a system for Sonic 3.5 and then Sonic 3.5 new came out and... [39:28] and Haku came out. [39:31] In some sense, we're very well positioned for it. In beta flags for us internally, we've been playing around with the different models, and they just work. But in practice, we haven't actually deployed them to customers yet. We're working on it now. [39:46] So I think like a model switcher is, is like some pretty good infrastructure. Another thing we're building, we recently got a bunch of new townie users, uh, which has, uh, has made it like extremely important to get pricing, like proper usage-based pricing in for townie, because like these things are really expensive and we just haven't taken the time because it's hard to build a whole like new pricing model in place. So like, like those sorts of infra pieces, like being able to switch models, being able [40:16] more quickly like you're doing. Like when you add a new model, do you have the infrastructure to like tweak the prompt for that model? We don't... [40:24] like in practice that hasn't been great. Like Claude's pre-fives on it. New is doing things, did a lot of bad stuff. Um, [40:32] And we had to tweak the prompt a lot for it. And ultimately we just rolled back to cloud three, five sonnet old because we couldn't get the prompting right for the newer model. Interesting. What about like strategically? Cause there's this like, there's this tension here, right? Where like you're relying on cloud three, five sonnet, but also cloud has artifacts and like artifacts is like,

[40:54] there's some overlap with Townie. And same is true for like all these other apps or whatever. And so I think like one of the games of being a startup is that's benefiting from AI is sort of strategically thinking about how to benefit from the gains of these companies without also being eaten by them because they all have consumer or B2B like offerings that are not just API offerings. So yeah, how do you think about that? [41:20] Yeah. Our... [41:24] differentiator has always been that we run back, back end compute, we're like a backend as a service provider and, um, [41:32] I think... [41:34] these LLM companies won't want to run their own functions as a service infrastructure internally. They're going to want to partner with someone. And that's the outsized dream that they partner with us to help them run back and compute for their customers somehow. Even now, with the front end apps that they make, it's pretty crazy that Anthropic lets you share the actual front end app. You can kind of deploy from within Anthropic. [42:03] Um, um, [42:05] But yeah, where are these companies going? Yeah, it's unclear, but I think we're like as safe as one could be in backend, in backend function of service infrastructure. I think that's pretty unique. [42:17] Yeah, you think that they're not going to build that. They're going to partner. And yeah, that's interesting. How are you feeling about things? You've been in this company, I guess it's been like a year and a half-ish. I think it has been a year and a half. How long have you been running this?

[42:32] Two years. Two years. You raised around in February, March. Yeah, how are things feeling? What has gone how you expected and what has not gone as you expected? And yeah, where are you right now? Yeah, I think things have been... [42:48] Great, like have gotten better than expected in some departments and worse in others. [42:55] We just are bringing on like three. So we've been a team of four for like the whole year and we just hired three more people who are starting in January. So like I'm kind of sitting on the edge of my seat, like waiting to see how things like the team dynamic shifts because that's like almost doubling. [43:14] Uh... [43:17] In terms of, things are growing well, but not as fast as I want. And it's interesting in this space to see, there are a lot of competitors who nobody knows about or talks about. And we're excited to be doing better than them. And there are crazy success stories that are taking off to the moon overnight. And it's hard to not be one of those. We're in the middle, we're doing well, we're having steady progress. Things aren't going as well as they could. [43:43] What's an example, like Replit or something? [43:46] Replit isn't what I had on my mind. Bolt.new just announced that they did like $4 million in revenue in their first four months of existence. So that's like, wow, amazing, so cool. [44:01] Uh, like cursor, I think is another, I kind of runaway success. Like those are the two that I'd be jealous of, uh, replet agents, I think got a lot of hype, but I don't really know people who use it a lot. Did you use it? Oh, I haven't not really used it. Honestly. Um, I used replet a lot when I was doing my course. Um, I, like I had this, you know, how to build an AI chatbot course and it was like amazing for that. Um, because being able to like set up a project and like

[44:31] And students could like fork it and then press run and just works was like, it was amazing. But like for my day-to-day programming stuff, I'm just, I just use cursor right now. I found that like there were at least back in the back when I was using it, this is before ReplAgent, like there was just some frustrating things about it and like, and the AI was like not as good. So I, but I don't have an, I don't have an updated opinion on it. [44:55] Yeah, yeah. Reblit, I think, originally and still has product market fit mostly in education. That's, like, where it shines. So, it makes sense that it worked for you in a course setting. Yeah, I mean, I know that feeling. It's going well, but it's not, like, blowing up, but it's also not dead. And sort of, like, being in the middle can be... [45:19] I think everyone spends a while in there. It's called the trough of sorrow or whatever reason. I don't know if you would call it the trough of sorrow because things are going well. But sometimes things going well but not incredible is harder than things not going well at all because you can at least just be like, well, I'm not... At least you can just be straight sad as opposed to being like... Yeah. [45:43] Like mostly happy. [45:49] Definitely. I feel like... I feel like... [45:52] We're in this sort of uptick right now. [45:57] Every or one of your apps? Every overall app.

[46:01] Um, and I would say like, I would say growth is like, like, like the last couple of months has been really good. This month is like sort of like leveling off a little bit, but I think we'll have a couple more launches in the next couple of weeks that, that will sort of change that. And we're, we're not really doing any like paid marketing or whatever. Um, but I, I, it just feels like things are happening, which is really, really fun. But there were years. Um, [46:26] where it was not like that. So I, I feel you. Um, uh, and, and I, I totally recognize the like sort of team dynamic thing. Like we've, we've grown the team a lot and [46:40] Is this your office? This is the office I work out of, but it's not mine. You know Jesse Beirutti, right? No. He's a really good friend of mine. Also went to Penn. [46:54] IA Ventures. He's a partner at IA, and this is their office. And they graciously let us work out of here, crash here, until hopefully one day we'll get an office of our own. [47:05] Nice. Have you been to our office in downtown Brooklyn? Oh yes. I, I, you walked up for one for once. I walked up, we, we actually launched sparkle there. If you remember. Um, yes, a very auspicious day. Uh, but we were not able to spend that much time together. Cause I was like running around like, like a chicken with my head cut off. Um, but yeah, I mean, things, things, things always break when you add new people and, um, it's always, it's always fun. How have you thought

[47:35] team sizing in the kind of AI age, like how many people you need to hire and who you need to hire and what that says about like your runway and your budget and all that kind of stuff. [47:48] I haven't really changed my thinking about it because like my experience [47:57] Role models have always been like Instagram and Notion, like a team of on the order of 10. Like it's like anywhere from five to 15, uh, even cursor is like, I think like a, like a [48:08] 15, 20 person team or it was when it exploded. I think like this, like small dense, but mighty engineering teams have always been attractive to me and I'm [48:18] I think AI just makes that more tractable for us mere mortals to be able to do that. But I don't think it means that I can hire like four instead of seven. I think I still need the seven. [48:32] Hopefully the four of us will have an easier, sorry, the seven of us will have an easier go of it than without AI. [48:37] That makes sense. I agree. I feel like the last 10 years, there are the examples of the Instagrams. Notion's a huge team now, but they were pretty small for a while. Notion was like six people when they did database. In 2019, when I was using Notion, when we were all using Notion, it was a real company then. It was like six people, something crazy.

[49:07] I really love having a small team. I think it's super fun. And everyone knows each other and everyone's friends. And it changes things when it's 40 or 50 or 100 or 1,[redacted address] that's less personal. But obviously, you can get less done. But now... [49:28] Depending on what you're doing, you can actually get really far with a smaller team, which sounds just more fun to me. [49:34] Yeah, I don't think a small team is holding, you know, software is one of those things more people don't make. [49:41] it happened any faster, the mythical man month and everything like having a really tight context. Like everyone on the team knows what's going on. Iterate really fast. Because what we're doing right now, for example, is [49:55] adding or creating like a virtual file systems kind of setup. So right now, Valtown is so good at writing single file. [50:04] apps, but then you want to break it up into a couple of different files and have the AI just like edit one of them for you while the other ones they put. [50:12] Ciao. [50:13] But Valtaine can't handle that. It's just like we haven't built that infra. But now we're building it and there's no... [50:21] like there's no parallelizability of it. Like they're like, they're just like core in for decisions. Like the whole team has to kind of be on board with, and we can't add more people to make it go faster. More people would honestly make it go slower. Right. Right. Right. How do you think about like, once you're now you're getting into multifile edits, like, how do you think about what the appropriate unit of work is for the AI to do before it comes back and is like, I did this. And what does that imply for like, how much in the loop should you be versus not?

[50:51] For better or for worse, we've set ourselves up to be spectators in this race. And we watch Cursor and Anthropic. [50:59] And, you know, Bolt and whoever, like, come up with these new UI interaction patterns and AI interaction patterns. And then we take the best ones and we, like, adopt them. I think that's what's worked best for us to be, like, fast copycatters. Because, like, our core business is, and, like, what we want to be best at and, like, innovate on is, like, a functions-as-a-service platform that's, like, the simplest and fastest and, like, most pleasurable to use. And then we, like, layer on these AI features that, like, other people will innovate for us. [51:29] Uh, so that's interesting. [51:32] Yeah, it's weird to be like, because I feel like my instinct is like, no, no, no, we have to be the best. We're going to come up with this stuff. But the industry is just moving so fast. [51:42] If we just wait a month or two, we can see what other people do and copy it. That's what we did with Townie. Like Townie was just like, we looked at before Anthropic Artifacts, Townie was like a tool use thing that looked just like ChatGPT at the time. We were like just kind of copying ChatGPT. And it was pretty bad. And then Anthropic Artifacts launched. And we're like, ah, there we go. That's the correct UI pattern. And now we're like on that. But now we're watching... [52:11] you know, cursor invent all these new UI patterns. And we're like, ah, yes, those are the ones we want now. That's interesting. Does it, like... [52:20] How does it sit with you? And how do you stick to your strategy? It seems you have a very crisp idea of your strategy of what you want to be best at, which is backend as a service functions.

[52:33] Um, is what you said. And that like strategy was formulated, um, even before all this stuff came out or was like really popular. Um, and it seems like you're getting this pull with Townie from people who like, you know, are maybe not even programmers and maybe don't really understand what backend as a service even is. Um, yeah. [52:53] And so how are you dealing with like – [52:58] the like excitement of that and, and, but also you're able to say like, that's, that's important. Um, but we're going to be sort of fast, fast followers because like the core strategy is this other thing. Like, yeah. How are you thinking about that? It's, you really hit the [53:15] In the last week or two, I've had a lot of discussions with our main investor, Dan Levine from Excel. He's... [53:23] like a spiritual co-founder for this company. Uh, [53:27] And like a lot of the core strategy pieces about like what we are and what we aren't come from him. [53:31] And we had a meeting last week where I was like kind of showing him, um, [53:36] like this alternative vision, like we could pivot from the backend as a strategy to a like full stack app development tool. Like maybe you're like YouTube for programmers vision or like Shopify kind of a, like a full stack app platform. [53:51] uh like shopify for sass apps you could is like one idea like we could rebrand we could call ourselves sassify or something um and like go after like bolt and like try and get like hundreds of thousands of non-programmers to just be like buying credits and credits and credits uh to like make make apps like i think that's a real pivot we could plausibly do um and we could have the database included and like all sorts of other services included and then we become

[54:21] We're for everyone and unsophisticated folks, and they're making... [54:29] apps, you know, like, so, you know, kind of like small, medium sized apps, like no serious engineer would, you [54:36] would use us. And Dan and I had a long chat about it and the various trade-offs. We had a long chat and I wrote this like long strategy document about what our strategy is, and very importantly, what it isn't and why we're giving it up. [54:50] And like how we get pigeonholed if we do the other strategy, like one of the big downsides of being like, [54:58] a full stack app is that you have to do the whole stack. And I think it seems simple and it's always seems simple and it's just so much more complicated than everybody thinks. [55:08] in order to do front-end and back-end and monitoring and errors and logging and databases and database migrations and database backups. Just like we're talking about like a mature, back to your question from earlier, a mature engineering organization. Yeah. [55:22] has like a dozen or two dozen tools in order to, including like a local text editor and like local version control, get up like so many, so many tools. And so I want to replace that whole stack of like, [55:37] two dozen tools, it just necessarily has to be [55:42] a toy version of all of them and like a kind of bad version of all of them. And then that customer is necessarily become like unsophisticated. It leads, leads to a weird local maxima that we're scared of. And then the alternative is that we do fewer things, but do them really sharply and better and integrate with all those other things. Like we, we could become just the functions of service piece for a lot of people. And then we can, once that's kind of nailed, we can expand more confidently while remaining quite good at each thing we do instead of,

[56:12] mediocre at all of them. That makes a ton of sense. What, um, what makes you think that that's like the [56:20] part of the stack that you want to sit in? Why choose that one? [56:25] Why backend as a service or why function as a service? It's a really, really good question. Like in some sense, the most honest answer is that... [56:36] Uh, it's, it's what like Dan Levine thought there was an opportunity for. It's like what our investors had as an opportunity for, or it's like what was left, uh, like Dan's also an investor in Vercel and they're doing front end and they're like doing such a good job at front end. Uh, and then there's like a back in shape opportunity. So that's, that's like, that's one way to look at it. Uh, yeah. [56:56] I've always, like another way to look at it is I've, [57:00] Like as long as I've been a programmer, I've, [57:03] really wanted something like this to exist. I've like always hated how complicated back-end stuff is and [57:11] There are like three dozen tools that I've used over the years trying to make my life easier on the back end. [57:18] and never found one that was good enough. [57:22] And so like wanting it to exist, like nobody, well, I guess we could talk about who else is in this market. There's AWS Lambda, which... [57:31] Nobody would ever use directly. You have to use it like intermediated to all sorts of stuff. It's like a terrible, terrible user experience compared to [57:40] normal programming. And then there's Cloudflow Workers, which is actually a wonderful product. And,

[57:46] And I've learned a lot from the Cloudflare team. But still, it's not social. It's not GitHub. It's not like collaborative, open source, productive coding like GitHub is. Like when you make a Cloudflare worker, it's... [58:00] It's kind of invisible code that nobody else can then leverage and use and like and fork. It's not an artifact and it's not a browser-based thing. It's a thing you develop... [58:10] Like a normal engineering artifact. Yeah, it's sort of a last generation type. [58:16] thing, um, like for like people working in more enterprise environments where there's like less flexibility and there's, you know, like, yeah, I think, I think that that makes sense. Um, I, I, I just gotta say, like, I think it's awesome that, um, that you're thinking through these things like so crisply and you have a really like clean perspective on it. Um, which, uh, I think it's awesome one. Cause it like, um, yeah. [58:43] it's just, that's just rare. And two, like, you're giving yourself the ability to be wrong, which also gives you the ability to be right. Um, whereas I think it's like, there's a tendency to like try to do everything and be everything. So as the man who like has the studio where you have like five different products or whatever, it's literally called every, trying to do everything. [59:13] admire that because like, um, I know internally how hard that is for me. Um,

[59:21] And I just really appreciate getting to hear it and that you're willing to share it. I think a lot of people are not willing to share it. I think you're just being incredibly honest and I think you have a very clear vision for what you're doing and I'm just psyched to see what you do with it, what happens in the next year or two. [59:38] Thank you. Yeah, me too. I think it's the vision's been the same since day one, but it's remarkable how many twists and turns there are even staying true to an original vision. [59:51] Totally. Well, this is awesome. I had a great time. I'm really glad you came on the show. We got to do it more often. If people want to find you and find Valtown, where can they find you on the internet? Man, I normally say Twitter, but I'm trying to move off of it these days, vaguely. But yeah, I'm Steve Krause, kind of everywhere on the internet and Valtown is where you can find Valtown. Awesome. Thanks, Steve. Thanks, Dan. [1:00:21] Oh my gosh, folks. You absolutely positively have to smash that like button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard. But instead of gold, it's filled with pure unadulterated knowledge bombs about chat GPT. [1:00:51] leave you on the edge of your seat.

[1:00:52] craving for more it's not just a show it's a journey into the future with dan shipper as the captain of the spaceship so do yourself a favor hit like smash subscribe and strap in for the ride of your life [1:01:05] And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.