Brainstorm AI Singapore 2024: How to write effective AI prompts

Fortune

Teodora DANILOVIC, Prompt Engineer, Autogenai

Transcript

00:00So, in the short time I have with you all today, I want to demonstrate how smart prompting

00:04leads to smart outputs and hopefully give you a few practical tips and techniques that

00:09you can take away and use for your own prompt creation process.

00:14So I work at Autogen AI, where we help organizations around the world write more winning bids,

00:19tenders, and proposals by leveraging large language models and linguistic engineering.

00:24I get a lot of questions around what prompt engineering actually is and what I do in my

00:28role, so I thought it would be useful to clarify that.

00:30Can I get a quick show of hands, who here has used ChatGPT or any other similar platform

00:36for their prompting?

00:38Wonderful, everyone.

00:39Perfect.

00:40So, what most of you have been doing in those instances is called prompt crafting.

00:46Prompt crafting is when you interact real time with a model and give it a prompt for

00:50that individual instance.

00:52You receive useful and relevant responses, but you wouldn't necessarily expect that prompt

00:56to work on any other piece of text that anyone else would use.

01:00So prompt engineering is curating prompts which produce replicable, reliable outputs

01:05to fulfill a specific function, whilst continuously objectively measuring and improving them.

01:11It's about setting up frameworks that scale well in the future with any unknown input

01:15all the time.

01:19There are many prompting techniques out there, but these are some of the most popular.

01:23Today, I will demonstrate a few of these as we consider their benefits and their drawbacks,

01:28and we'll do this by focusing on one task, which is extracting and classifying things

01:32from a data set.

01:36I'll be using AutoGenerate's platform today to demonstrate these prompts, so the feature

01:40that I'll use, it offers multiple options of outputs for each prompt, so you'll see

01:45three options on the right-hand side.

01:47Up here, we have a zero-shot prompt.

01:50It's an instruction with no examples.

01:52It's what everyone does the first time they interact with the large language model.

01:56It works well most of the time, but it does have a few drawbacks.

02:00Sometimes it can lack a nuanced understanding of the task we're trying to achieve, and we'll

02:04see that in practice today.

02:06So in this example, I've asked it to classify a piece of text, and that piece of text has

02:12a debatable output.

02:17So as a human, I would expect the product arrived late, but the quality is excellent,

02:23to generally be a positive statement.

02:25Although it has some positive and some negatives in it, it's weighted heavier on the positive

02:29statement.

02:31We can see that the model has given neutral for all three outputs.

02:37So for this case, zero-shot prompting doesn't encourage that nuanced understanding, and

02:41I probably wouldn't trust it for even larger sets of data.

02:44Here, the model lacks understanding of what I mean by each of those sentiments.

02:48What does positive mean for me?

02:49What does neutral mean for me?

02:53In the previous example, I didn't have enough context.

02:55So one way of providing it with more context is offering examples of what you want.

02:59This is called multi-shot prompting, a shot being an example.

03:04Chain-of-thought prompting means asking the model to think step-by-step and show its reasoning.

03:13So I've given it pretty much the same prompt, but I've given it three examples of what

03:17I think a positive, negative, and neutral statement is.

03:20You can see, as we transform the text, the outputs are far more nuanced.

03:24The first one says positive.

03:26The second one, after explaining its chain of thought, where I've instructed it to think

03:31step-by-step, also concludes to be positive in the end.

03:34It's far closer to what we're looking for.

03:38One thing I will say is that you should be wary of bias with multi-shot prompting.

03:43In this example, the model might take it to mean that positive statement must always

03:47talk about product quality, or it always must mention that the site was confusing, or that

03:51it always has to have one positive and one negative side to it.

03:55If you're using multi-shot prompting for larger sets of data, you have to make sure your examples

03:59cover all bases.

04:01This can be quite difficult to do, as you have to think of every way that something

04:04can be interpreted.

04:06Chain-of-thought also helps in this scenario, because if I can see the model's thought process,

04:10I can see where it went wrong, and it can help with my model debugging.

04:14Multi-shot here was really great for improving this task and giving the model some more understanding

04:18of my intention.

04:19But sometimes I want to do something a bit more complicated, and I don't want it to give

04:23me just a one-word classification for something.

04:26Let's look at how we can introduce even more complexity with some prompting techniques.

04:33Prompt chaining, or multi-step prompting, is best for complex reasoning tasks that cannot

04:37be instructed in one go.

04:39It ensures you're working on the best piece of text at each stage, and it doesn't leave

04:42room for model inconsistency.

04:45It makes sure that the potentially conflicting instructions don't interfere with one another.

04:50Let's say I want a more complex analysis of some sentiment on a larger body of text.

04:54As a human, I might break this process down into a few different steps.

04:58First, classifying the statements as a whole, then extracting themes from the statement,

05:03and then grouping those themes.

05:04This type of breakdown also works great with prompting.

05:10For the sake of brevity, I'm just showing the output of the first prompt that I gave

05:13the model.

05:15The prompt instructed it to classify a list of customer feedback into sentiments using

05:19the same multi-shot prompt that I showed you guys before, except on a larger set of data.

05:24There's around 25, 26 pieces of customer feedback here.

05:27This is the first prompt in this chain.

05:33The second prompt in the chain is a really good zero-shot prompt.

05:36Up at the top here, you can see we've asked it to identify all the themes in the following

05:41list of customer feedback.

05:43Your response should be an exhaustive list of accurate and relevant themes with a number

05:47beside them indicating how many times it appeared.

05:50Do not write the same theme out twice.

05:53Feedback colon.

05:55So I think it's important to note some key parts of that prompt.

06:01So I've asked it to identify all the themes, and then I've repeated myself and said it

06:05should be an exhaustive list.

06:07Repetition is always good.

06:08I've said that it's a list of customer feedback, which provides the model with context of what

06:12I'm talking about.

06:13I've asked it to be accurate and relevant, so it's clear that I want the themes only

06:17to do with customer feedback.

06:19And then I've said how I want the structure to look.

06:22You can see with all of the options, the structure is exactly how I want it to look.

06:26The answers are exactly what I'm looking for.

06:28So I'm happy with this.

06:30Let me bring it over into the editor.

06:33So as we move on to the third prompt in the chain, you can see how with each step, the

06:37output is a combination of everything that came before it.

06:41Context which is implicitly woven into the answer gets carried on to the next prompt.

06:49The prompt that I've given it here, the third prompt in the chain, is classify the following

06:52themes into positive, negative, neutral, or other categories.

06:56The response should have the sentiment as a heading followed by the themes which fall

07:00under that sentiment.

07:01Under each theme, write a brief justification of why it falls under that sentiment.

07:05And then I've given the list of themes.

07:07As you can see on the right-hand side, this is far more nuanced, accurate, and useful

07:13than the first thing we looked at.

07:16This output is actually a combination of all the techniques we've looked at so far.

07:19And we could not have gotten this type of information which is this thorough and this

07:23accurate through only using one of the techniques.

07:29Once you've got an output that you're happy with, the possibilities really are endless.

07:34You can translate it into JSON.

07:38You can play around with the tone.

07:42You can turn it into a PowerPoint presentation and much, much more.

07:48So to conclude this short segment, simplicity is always best.

07:53Although a single-shot prompt wasn't nuanced or accurate enough for our example today,

07:57it is most often the best choice.

07:59The last three examples I showed you of turning it into JSON or translating the tone, all

08:04of those used a really good, successful zero-shot prompt.

08:08A prompt should be direct, unambiguous, and relevant.

08:12And each of the techniques we've gone through today are just different ways of ensuring

08:15that the prompt meets those requirements.

08:17I really hope this offered some insight into what prompt engineering actually is and gave

08:21you some tools that you can take away and implement into your prompt crafting.

08:26I think we might have time for one quick question, if there are any questions in the audience.

08:32Yes, we have someone here.

08:49How do you think about it, and do you use any tools to refine your prompts?

08:55Yeah, so that's actually a really interesting question.

09:00You can prompt models with how you want them to refine your prompts, interestingly.

09:05So you have to have the techniques in order to instruct the models in how you want your

09:09prompts to look anyway.

09:11Sometimes if I'm lacking inspiration on how to begin writing a prompt, I will definitely

09:15use a model.

09:16I'll be direct, unambiguous, relevant, clear about what my parameters are, clear about

09:20what my instructions are, and often it gives me really good framework.

09:25The only problem with models are it's sometimes a bit more nuanced than that, and you're looking

09:29to write a prompt for a specific use case.

09:30So I know my target audience, I know my customers, and I know what they're looking for.

09:35And I can try to put that into prompting, but ultimately it's your subjective opinion

09:39on whether you think it meets these metrics that you've set out.

09:42So yes, absolutely, you can use other models, not only Claude, but any of the providers

09:48to give you a first draft of a good prompt, and to better your own prompts.

09:55Awesome.

09:56I think that's all the time we have today, but if you have any more questions, I'll be

09:59around afterwards, and please feel free to add me on LinkedIn and ask me any questions

10:04there.

10:05Thank you so much.

Category

Transcript

Recommended