Prompt Engineering

“Prompt engineering is how you ask the AI for a better question. And context engineering is how you give AI better information. But graph engineering is how you design the work around the AI so the whole thing stops living inside one messy giant AI chat.”

“I think the skill nowadays is less about prompt engineering and more about figuring out how do you give Cloud a hard task that seems a little bit too hard? And then how do you make it possible for Cloud to verify its work along the way? And the verification, I…”

“People say prompt engineering is gone. Bullshit. Like, the underlying challenge is, how do I work out how I would tell the model how I want to do this? And that's not, it's not like you have to say clever things to understand. Get the model to work in a differ…”

“We made a bet that number of people who need social media video, number of people who want to sell is probably tens of millions of users, but number of people who want to learn prompt engineering is probably hundreds of thousands.”

“Guardrails is, you know, better than prompt engineering.”

“After the Gen AI revolution, like the way now we build intelligent applications is we take models off the shelf. These are general purpose models. They can reason on a variety of tasks, and then we condition the models to work a specific way by doing prompt en…”

“Which I actually think the prompt Engineering, in some ways, may be even more vibrant than before.”

“Prompt engineering used to be the big thing. It was like, here's the ultimate prompt for going viral on social media, or use this prompt for this. And now it's all about context engineering. It's about how well can you load up your agent with all the informati…”

“The way you get a good prompt is all test-driven, just like evals, right? In a sense, the test cases are your evals.”

“The challenge is really articulating and refining prompts to reflect. Reflect the execution of the SOP and like reflect all the sort of institutional knowledge that isn't written down so that agents can properly replace like the humans or the contractors that …”

“And prompt engineering is really just delegating.”

“But I think the name of the game now is, is less on like, Prompt engineering as we had thought about it two years ago. It's more of like, it's like the context engineering side where it's like, what are the tools you give it? What is like the data that it pull…”

“What you'll find without any, these are for fine tuning, without any like technical fine tuning, you can go so far with just prompting. If you're being really careful about this, you will find that the AI gets things wrong predictably.”

“The brilliant thing about leveraging retrieval augmented generation is we can just curate the data. And if we're confident about the currency of the tax database and the content in the database, and we're confident about our retrieval algorithms, our Prompt en…”

“We invented prompt engineering, the first prompt right here at Salesforce Research, commercialized by others, of course.”

“One term I really dislike is prompt engineering. You know, engineering used to mean sending a man to the moon or providing five nines reliability. Prompt engineering is prompt twiddling.”

“Well, I think that, you know, prompt engineering is kind of a subset of context engineering.”

“There's this skill of being able to write evals. I know everybody can write prompts, prompt engineering. You can try and focus the LLM so that it can offer better insights and offer better results. But even as your LLM actually Provide, produces results. You n…”

“Prompt engineering, which by the way was came out of our Salesforce AI research team”

“What we have found is the more that we customized a prompt for our needs and the way that we customize that for our needs, the results are much, much better.”

“My perspective, and this has been validated over and over again, is that people will kind of always be saying it's dead or it's going to be dead with the next model version, but then it comes out and it's not.”

“Notably, that is not where the classical concept of prompt engineering came from. It actually came a bit earlier from a more, I guess, AI engineer perspective, where you're like, I have this product I'm building. I have this one prompt or a couple different pr…”

“A lot of these techniques are judged based off of data sets of like math or reasoning questions. Simply because you're going to evaluate the accuracy programmatically, as opposed to something like generating interview questions, which is no less valuable, but …”

“But I also agree that, you know, maybe in a month or two months or three months, you won't need this anymore because, you know, the bitter lesson always wins.”

“We invented prompt engineering which was majorly rejected publicly on open review an idea that made no sense to the reviewers.”

“I want to kill the idea that you have to be a good prompt engineer. I think if we do our jobs, that stops being true. You know, it's just one of those, like, sharp edges of models that experts can learn, but then you just, over time, you shouldn't need to know…”

“I think prompting is overrated. Like, just language as a unique interface will not get you too far because, like, storytelling and films, particularly in video, it's just so hard to just describe words, you know.”

“In talking to, you know, founders who are actually building, it seems like prompt engineering can get you quite far, right? So like, it's widely variable, but let's say you can get to like, 70, 90% of where you need to be from a performance perspective.”

“I think prompt engineering is going to unlock a huge swath of human society. There's 1.4000000000 people in Africa who know how to speak, and if you were to give them access to a tool that They could create applications live just by speaking to it. That would …”

“I do think that it can be very systematic. You know, you should need to make it very systematic. So if you consider like each prompt is experiment, it should be watching like versions of prompt. It should be able to systematically track your progress with diff…”

“Well, if it took a really complicated prompt and 1600 iterations, that's a failure of my product, not a failure of the candidate.”

“And I would say most people have spent less than an hour prompt engineering. Or like really trying to dial any of this stuff in.”

“And I think we're going back to fine tuning where we realize these models are really cosplay. It's better to use just a small model. We try to specialize it. So I think it's a little bit of a cycle and we're going to start to see like more of fine tuning and l…”

“The model is like this sort of like 10 X multiplier. You're, how good the bottom line model is, huge, huge swing. And then kind of what you can do on top of that, you can squeeze out three, four X kind of more. And so that's kind of where the realm of, you kno…”

“You need to iterate on tools rather than just a prompt. And I think a lot of people, when they make tools for an LLM, they kind of treat it like they're just writing an API for a computer and it's sort of very minimal. It's sort of just the bare bones of what …”

“Even prompt engineering was invented at Salesforce, which we're very proud of.”

“On the research side this was in 2018, we finally culminated all my ideas when we invented prompt engineering, and we didn't quite call it prompt engineering, but the basic idea was there, namely that you can prompt a single model with different questions, and…”

“I feel like in some sense, I feel like prompt engineering, even it's like a slightly negative word at the time, because it refers to all those kind of weird tricks that you have to apply. But I think we don't have to do that anymore. Like given today's progres…”

“I have always viewed prompt engineering as a skill that everybody should and will have, rather than a specialized role to hire for. That being said, there are definitely times where you do need just a prompt engineer. I think for AI companies, it's definitely …”

“Today when people are running RAG or prompt engineering those are search. That's not AI. It's like keeping the AI frozen and fixed.”

“Knowing how to interface with AI systems is an incredibly critical capability that is, I think is very akin to software engineering over the past few decades.”

“I don't think that prompt engineering is going to be a kind of durable differential skill that people will hold. I do think that the way that you set up the ML problem to kind of ask the right questions, if you see what I mean, rather than the specific phrasin…”

“Our experience has been that You can get really, really high quality reasoning from roughly an agentic system without needing to be too cute about it. You can describe the task and you know, within well-defined bounds you don't need to treat the LLM like a per…”

“Well, I think we'll have a whole swath of jobs that are in the kind of prompt engineering, you know, kind of, AI research assistant, you know, kind of directors.”

“You could imagine prompting crafts in many different domains such that you're kind of unlocking the latent super genius.”

“Prompt matters. Like, you know, we went through this whole cycle of like prompt engineering is a thing. It's not really a thing. Fine tuning is like prompt really does matter.”

“My thinking is that even that will become a bot that will start to prompt based on the domain area that, that I am, that it's able to go and feed that information. So right now, obviously prompt engineering, because then you can get the answer the way that you…”

“I think Google is essentially prompt engineering. When you edit your query to get the results that you want, that's prompt engineering.”

“And so the idea that there's something called prompt engineering, engineering in the sense of you're applying an approach because it's going to give you the output that you desire. Is a complete fallacy. At best it's trial and error. It's prompt trial and erro…”

“I'd like to think of it more as prompt directing rather than prompt engineering. Think of yourself like a movie director. Rather than telling the actor, say the line exactly this way, leave breathing room for creativity by the actor. You might say, try it more…”

“And I think a lot of the value for AI companies now, you know, is in prompt engineering. I think the early days was actually just Can you build a model? Can you get the model running? Can you get the model working in production? Where I think a lot of it is, c…”

“When we invented prompt engineering, we actually did it at Salesforce research when I was chief scientist there.”

“It seems to me incredibly obvious that what's going to happen is that this stuff will get abstracted inside and away. And then the actual technology of it may be running a prompt that gets inserted into an LLM, but the user will probably very often won't see t…”

“As part of the research team, we actually invented, ah, prompt engineering in 2018”

“A lot of people believe that prompt engineering is incredibly crucial. I don't necessarily buy into that. I don't know if you remember, but people, like, thought that, like you know query building for Google would be a profession too, right? And that didn't qu…”

“I actually, I'm not as bullish on the idea of a prompt engineer as most, right? I think it'll be like knowing how to use Google or Excel. It's just like a particular skill that'll be helpful. But no, I think the pro I mean, like one of our main ambitions too, …”

“I think prompting is more an expression of your life experience and knowing what would happen. Like if Nikhil was making a new app tomorrow, you were making another version of Koo tomorrow, you already know what works, what doesn't. So you'll be able to prompt…”

“We're all being given the same tools, but you're, that creativity behind the prompt engineering is what's going to make the absolute difference here, which is fascinating.”

“There's obviously every incentive for the people that make these foundational tools to make prompt engineering, for instance, not A thing because they want everyone to be able to do it.”

“I don't think it will become, like, this necessary skill that everyone needs to have, but I do think it will become, you know, like, some people are expert wood whittlers or, you know, really good at Animating hair or whatever, you know, the people that develo…”