The Exchanges

Q that helps you with deciding what action to take next, like TreeSearch, can kind of help you with reasoning. Any learnings from like, kind of going through all the decomposition ones? Are there state of the art ones? Are there ones that are like I don't know what skeleton of thought is, you know, there's a lot of funny names. Uh, what's the state of the art in the composition?

A Yeah. So the skeleton of thought is actually a bit of a different technique. It has to deal with how to parallelize and improve efficiency of prompts. So not very related to the other ones, but in terms of state of the art, I think something like true thought is state of the art on a number of tasks. Of course, the complexity of implementation and the time it takes can be restrictive. My My favorite simple things to do here are just like in a, let's think, step-by-step, say, like, make sure to break the problem down into sub-problems and then solve each of those sub-problems individually. Something like that, which is just like a zero-shot decomposition prompt, often works pretty well. It becomes more clear how to build a more complicated system, which you could bring in API calls to solve each sub-problem individually and then put them all back in the main prompt. Stuff like that, but starting off simple with decomposition is always good. The other thing that I think is quite notable is the similarity between decomposition and thought generation, because they're kind of both generating intermediate reasoning, and actually over the course of this research paper process, I would sometimes come back to the paper like a couple days later, and someone would have moved all of the decomposition techniques into the thought generation, Section. At some point, I did not agree with this,…

AI assessment note: “in terms of state of the art, I think something like true thought is state of the art”

Q Cool. We cannot leave this section without talking a little bit about automatic prompt engineering. You have some sections in here, but I don't think it's like a big focus of prompts, the prompt report. DSPy is up and coming sort of approach. You explored that in your self study or case study. What do you think about Ape and DSPy?

A Yeah. Before this paper, I thought it's really going to keep being a human thing for quite a while. And that like any optimized prompting approach is just sort of too difficult. And then I spent 20 hours prompt engineering for a task, and Dyspy beat me in 10 minutes, and that's when I changed my mind. I would absolutely recommend using these, Dyspy in particular, because it's just so easy to set up. Really great Python library experience. One limitation, I guess, is that you really need ground truth labels, so it's harder, if not impossible currently, to optimize open generation tasks, so like Writing, writing newsletters, I suppose. It's harder to automatically optimize those, and I'm actually not aware of any approaches that do other than sort of meta-prompting where you go and you say to ChatGPG, here's my prompt, improve it for me. I've seen those. I don't know how well those work. Do you do that?

AI assessment note: “Dyspy beat me in 10 minutes, and that's when I changed my mind.”

Q does omniscience or omniscience, which also does generated with papers. I've never heard of this Prisma process that you followed. This is a common literature review process. Like you pull all these papers and then you like filtered them very studiously, like just describe like why you picked this process. Is it a normal thing to do? Is it, was it the best fit for what you wanted to do?

A Yeah, it is a commonly used process in research when people are performing systematic literature reviews and across, I think really all fields. And as far as why we did it, it lends a couple of things. So first of all, this enables us to really be holistic in our approach and lends credibility to our ability to say, okay, well, for the most part, we didn't miss anything important because it's like a very well vetted, again, commonly used technique. I think it was suggested by the PI on the project. I, Unsurprisingly, don't have experience doing systematic literature reviews for this paper. It takes so long to do, although some people, apparently there are researchers out there who just specialize in systematic literature reviews, and they just spend years grinding these out. It was really helpful, and a really interesting part, what we did, we actually used AI as part of that process, so whereas usually researchers would sort of divide all the papers up amongst themselves and read through it, We used a prompt to read through a number of papers to decide whether they were relevant or irrelevant. Of course, we were very careful to test the accuracy, and we have all the statistics on that, comparing it against human performance on evaluation in the paper, but overall, very helpful technique. I would recommend it, and it does take additional time to do because there's just this sor…

AI assessment note: “Yeah, it is a commonly used process in research when people are performing systematic literature reviews”

Q like not super intuitive. Like if you get it wrong, like the, the, the model will actually not give you a score. It just gives you what is like the most likely next token. So like your general thoughts on like structured output prompting, right? Like even now with OpenAI having like, you know, a hundred percent unstructured outputs, I think it's like becoming more and more of a thing.

A All right. Yeah. Let me answer those separately. I'll start with structured outputs. So for the most part, when I'm doing prompting tasks and rolling my own, I don't build a framework. I just use the API and build code around it. And my reasons for that, it's often quicker for my task. There's a lot of invisible prompts at work on a lot of these frameworks. I hate that. So like, you'll have Oh, this function summarizes input. But if you look behind the scenes, it's using some special summarization instruction. And if you don't have visibility on that, you can get confused by the outputs. And also for research papers, you need to be able to say, oh, this is how I did that task. And if you don't know that, then you're going to be misleading other researchers. It's not reproducible. It's a whole mess. But when it comes to structured output prompting, I'm actually really excited about that OpenAI release. I have a project right now that I hope to use it on. Funnily enough, when, at the same day that came out, another, or a paper came out that said, when you force the model to structure its outputs, the, like, performance, the accuracy, creativity is lessened, and that was really interesting. That wasn't something I would have thought about at all, and I guess it remains to be seen how the OpenAI structured output functionality affects that, because maybe they've trained their model…

AI assessment note: “when it comes to structured output prompting, I'm actually really excited about that OpenAI release”

Q But like, okay, fine. Let's get into this. Like, what does AI generated mean, right? Like, if I had ChatGPT rephrase some words.

A No, so they had ChatGPT write the entire paper. And worse, it was a survey paper of, I think, prompting. And I was looking at it, I was like, okay, great. Here's a resource that will probably be useful to us. And I'm reading it, and it's making no sense. And at some point in the paper, they did say like, oh, and this was written in part, or, or we use, I think they're like, we use ChatGPT to generate the paragraphs. I was like, well, what other information is there other than the paragraphs? But it was very clear in reading it that it was completely AI generated. You know, there's like the AI scientist paper that came out recently where they're using AI to generate papers, but their paper itself is not AI generated. But as a matter of where to draw the line, I think if you're using AI to change the entire paper, that's very well past the line.

AI assessment note: “if you're using AI to change the entire paper, that's very well past the line.”

Q if a lot of the training data has, for example, Q semicolon and then A semicolon, it's better to put it that way versus if the training data is a different format, it's better to do it. Maybe run people through that and then how do they figure out what's in the training data and how to best prompt these things? What's a good way to benchmark that? All right.

A Basically we read a bunch of papers and assembled six pieces of design advice about creating few shot prompts. One of my favorite is the ordering one. So how you order your exemplars in the prompt is super important. And we've seen this move accuracy from like zero percent to 90%, like Zero to state of the art on some tasks, which is just ridiculous, and I expect this to change over time in the sense that models should get robust to the order of few shot exemplars, but it's still something to absolutely keep in mind when you're designing prompts, and so that means trying out different orders, making sure you have a random order of exemplars for the most part, because if you have something like all your negative examples first, and then all your positive examples, The model might read into that too much and be like, okay, I just saw a ton of positive examples, so the next one's just probably positive, and there's other biases that you can accidentally generate. I guess you talked about the format, so let me talk about that as well. So how you are formatting your exemplars, whether that's Q colon, A colon, or just input colon output, there's a lot of different ways of doing it, and we recommend sticking to common formats as LLMs have likely seen them the most and are most comfortable with them. Basically what that means is that they're sort of more stable when using those formats…

AI assessment note: “we recommend sticking to common formats as LLMs have likely seen them the most”