The Wisdom Wall

“I think there's gonna be a generation of software engineers that are, like, good enough because it's so easy to retool there with, with all these cogent tools. Like, the barrier is so low. You're gonna be good enough engineers. There's gonna be tons of those. But to be exceptional, like the top one percent, I think…”

“startups are doing as well, they are actually arbitraging a lot of the models. I had some conversations with a number of founders where before they might have been loyalists to, let's say, OpenAI models or Anthropic, and I just had some conversations recently with them, and these are founders that are running larger…”

“I think that's a common pattern sometimes for companies when they need to get, um, responses from elements, elements in their product a lot quicker. They do the meta-prompting with a bigger, beefier model, any of the, I don't know, hundreds of billions of parameters plus models like, uh, I guess, cloud four, 3.7, or…”

“It's gonna actually slow you down in terms of launching quickly, because if you're hiring from a pool of people and engineers that you don't know, it takes over a month or more to find someone good, and it's hard to find people at this stage where it's very nebulous and chaotic, so it's gonna make you move slowly. And…”

“if you build a company and it works and you get users good enough, the tech choices don't matter as much. You can solve your way out of it.”

“Tech debt is totally fine. You gotta get comfortable a little bit with that heat of your tech burning. Totally okay. You're gonna fear the right things, and that is towards getting you product market fit.”

“It is personally thinking, betting more on the installation phase kind of, um, style of companies, which have to do more with our core technology that gets built out. Because a lot of those, there's tons to do. There's so many problems to solve, ah, before any of that, before, before going into the applications, right?…”

“And the system of record, there's, there's opportunity for something that's kind of agentic first, because right now we're still kind of integrate very much with databases and SQL or NoSQL queries at a very low level. But imagine something that generates all the data that you need for all the different views for custom…”

“the way you get a good prompt is all test-driven, just like evals, right? In a sense, the test cases are your evals.”

“We found that it makes it a lot easier for LLMs to follow, because a lot of elements were post-trained in RLHF with kind of XML type of input, and it turns out to produce better results.”

“I think this pattern of sometimes when it's too hard to even kind of write a prose around it, let's just give you an example that turns out to work really well, because it helps LLMs to Reason around complicated tasks and steer it better because you can't quite kind of put exact parameters and it's almost like, um,…”

“In order to do good vibe coding, you still need to have the taste, and you still need That kind of classical, maybe not necessarily classical train, but enough knowledge to judge what's good versus bad. And you only become good with enough practice.”

“I think this is actually a very common pattern that we're seeing a lot of the interesting products built with AI. You use different kinds of models. So, yes, four O mini is for PDF extraction, and then O one for the reasoning, because it's actually very hard to select the components for parts.”

“GPT is all generative based on predicting the next token and patterns and then getting those results to check that they're correct. So I think a lot of it is you had to have a lot of data that was factually correct and Fed into probably the model and the training and having a reward function that get it to reason a bit…”

“I think the other thing that's interesting about one is that it makes a lot of the GPU needs even bigger because it's moving a lot of the computation needs a lot higher for inference because it's taking a lot more time to do a lot of the inference. So I think it's going to change also a lot of dynamics underneath for a…”

“There's this aspect of the best people at the top of their careers. They're actually very not well-rounded. They're very quirky people. For good reason. And that's what makes them outlier. Like, by definition, if you're average, Then it's like, okay, you're not going to build a great company because building a…”

“And then founders think, oh, I did this at a big company. Of course, this is how my product needs to be done, because when I was an engineer at Apple, I had to work for, let's say, the Vision Pro project. I was like, I don't know. Eight years, a decade of work, and it's like, one shot. But that's a myth for startups.…”

“That forcing function is so effective, and this is how YC companies have a higher chance of success. They don't shoot themselves in the foot by dying of being anonymous.”

“I don't think we've seen startups die because of that reason. That's not, that is actually a hundred percent solvable problem. And you're not gonna die from it because the incentives are so high to fix it, and you're on the path to build a large company at that point.”

“I think where is exactly that, where I think is having more legs is when these companies need to customize it to private data sets. So you have the open, general, big foundation model, but then you have to tune it up to specific data sets that, for example, a healthcare or fintech can give out, can give out, and they…”

“A technical founder is a partner in this whole journey of a startup, and it requires really intense level of commitment, and you're in just a dev.”

“And you're going to have a bias towards building a good enough versus the perfect architecture, because if you worked at a big company, you might have been rewarded for the perfect architecture, but not for a startup.”

“The first stage is the ideating stage where you just have an idea of what you want to build, and the goal here is to build a prototype as soon as possible with the singular focus to build something to show and demo to users, and it doesn't even have to work fully.”

“The only tech choices that matter are the ones tied to your customer promises.”

“between two to five, you still get time to code about 70 When you get to five to 10, you only have less than 50%, and beyond 10, you probably won't really have time to code”

“So AR is kind of in that phase of really building the, the installation phase of solving all the hard problems so that you get to a very expressive world in the future that I imagine where AR programming in AR could be just as simple as spinning up a website.”

“You could have a great idea, you could have a great team, and also great investors, but sometimes the timing is just something you don't control, and you could have all those other things be amazing, but the timing is so hard, and that's what gives you sometimes that multiplier effect.”

“that was one of the big things, is like really driving the technology to have provable points with emerging tech. You kind of have to show that it's doable and where it's going to give a sense. Instead of, so it was actually building it at like a minimum. A demo. Which is a bit different than I, I know other founders…”

“it's a lot more efficient for us to write the code and all of the algorithms once, and then cross-compile it across all the architectures for Android or iOS, and even the backend, so that we run some of the CV algorithms, we could prototype it in the phone, and then we could easily move them into the server, because…”

“One of the tricks that, uh, I think founders use is you put like a cannery at the beginning of the context. There's something very esoteric that it would only help. It's like something really funny. It's like, I don't know. My name is Calvin and blah, blah, blah. I drink tea at eight AM. Some random fact. And then as…”

“One big thing about the best prompts is they outline how to reason about the task, and then a big thing is giving it, giving it an example, and this is what it does.”

“So there's this concept of, uh, defining the prompt in the system prompt, then there's the developer prompt, and then there's the user prompt. So what this mean is, uh, the system prompt is basically almost like defining, uh, sort of the high level API of how your company operates.”

“The thing that I really enjoyed about AR is about being engaged in reality instead of escaping it, where imagination to show or tell a story is really used to engage even more so, more deeply, with more layers of information represented, rather than really escaping it, where the magic in AR is not, it's not just…”