News

Jul 19, 2023

Prompt Engineering Embraces New Technique Called Skeleton

Here's the scoop on skeleton-of-thought (SoT) as used in prompt engineering for generative AI. Sometimes less is more. Another oft-used proverb is that at times you ought to prime the pump. We will be

Here's the scoop on skeleton-of-thought (SoT) as used in prompt engineering for generative AI.

Sometimes less is more.

Another oft-used proverb is that at times you ought to prime the pump.

We will be using those valued pieces of sage advice so please keep them in mind.

In today’s column, I am further extending my ongoing series about the latest advances in prompt engineering. My focus this time will be on a plain sailing but amazingly powerful new advance known as skeleton-of-thought (SoT), a creative adaptation of the exceedingly popular chain-of-thought (CoT) prompting technique. I’ll explain what this is and why it is a crucial method that you ought to include in your prompt engineering strategies and tactics.

The use of skeleton-of-thought can substantively boost your generative AI results, though please be aware that it is a specialized technique and has its own right time and place for being exercised.

As a quick background, SoT builds upon the greatly popular chain-of-thought approach that often is used by those aiming to get generative AI to stepwise showcase its presumed logic when answering a question or solving a problem. You merely instruct generative AI to explain step-by-step what it is doing. This is easy-peasy to request. Why do so? Well, remarkedly, research studies have indicated that this is not only insightful for you (i.e., being able to see detailed explanations produced by AI), but it also tends to get generative AI to produce seemingly more reliable and on-target answers.

I’ve covered the basics of chain-of-thought approaches previously, see the link here. Readers have ardently requested more details and seem eager to know more about the latest advances regarding this fundamental technique.

I am pleased to oblige.

Before I dive into the crux of the innovative skeleton-of-thought method, let’s make sure we are all on the same page when it comes to the keystones of prompt engineering and generative AI.

Prompt Engineering Is A Cornerstone For Generative AI

As a quick backgrounder, prompt engineering or also referred to as prompt design is a rapidly evolving realm and is vital to effectively and efficiently using generative AI. Anyone using generative AI such as the widely and wildly popular ChatGPT by AI maker OpenAI, or akin AI such as GPT-4 (OpenAI), Bard (Google), Claude 2 (Anthropic), etc. ought to be paying close attention to the latest innovations for crafting viable and pragmatic prompts.

For those of you interested in prompt engineering or prompt design, I’ve been doing an ongoing series of insightful looks at the latest in this expanding and evolving realm, including this coverage:

Anyone stridently interested in prompt engineering and improving their results when using generative AI ought to be familiar with those notable techniques.

Moving on, here’s a bold statement that pretty much has become a veritable golden rule these days:

If you provide a prompt that is poorly composed, the odds are that the generative AI will wander all over the map and you won’t get anything demonstrative related to your inquiry. Being demonstrably specific can be advantageous, but even that can confound or otherwise fail to get you the results you are seeking. A wide variety of cheat sheets and training courses for suitable ways to compose and utilize prompts has been rapidly entering the marketplace to try and help people leverage generative AI soundly. In addition, add-ons to generative AI have been devised to aid you when trying to come up with prudent prompts, see my coverage at the link here.

AI Ethics and AI Law also stridently enter into the prompt engineering domain. For example, whatever prompt you opt to compose can directly or inadvertently elicit or foster the potential of generative AI to produce essays and interactions that imbue untoward biases, errors, falsehoods, glitches, and even so-called AI hallucinations (I do not favor the catchphrase of AI hallucinations, though it has admittedly tremendous stickiness in the media; here’s my take on AI hallucinations at the link here).

There is also a marked chance that we will ultimately see lawmakers come to the fore on these matters, possibly devising and putting in place new laws or regulations to try and scope and curtail misuses of generative AI. Regarding prompt engineering, there are likely going to be heated debates over putting boundaries around the kinds of prompts you can use. This might include requiring AI makers to filter and prevent certain presumed inappropriate or unsuitable prompts, a cringe-worthy issue for some that borders on free speech considerations. For my ongoing coverage of these types of AI Ethics and AI Law issues, see the link here and the link here, just to name a few.

With the above as an overarching perspective, we are ready to jump into today’s discussion.

When Outlines Rule The World

The skeleton-of-thought method relies on the same premises that you would encounter when being urged to craft an outline on a given topic or any open matter at hand. A skeleton in this instance is simply another way of referring to an outline. A skeleton or an outline is a veritable guide or map. It is the axiomatic forest for the trees.

We copiously make use of outlines or skeletons in all facets of our daily lives.

Imagine that at work you are asked to prepare a memo that discusses your latest activities and accomplishments. You begin with a blank sheet of paper (or, more likely, an online document that is empty). Do you rush into pounding out the words that fully describe all of the wonderful things you’ve been doing? You might, but on the other hand, sometimes starting by creating an outline or skeleton is a better route.

Why make use of an outline or skeleton?

I think we all know that the beauty of first devising an outline or skeleton is that you can pencil out the crucial aspects of what you want to say. This will hopefully allow you to organize your thoughts into a coherent overall structure. If you impatiently start by impulsively babbling and opt to write wantonly, off the top of your head, the odds are that your essay or narrative will be kludgy, muddled, and confounding.

Now then, one supposes that there are in fact some people that can spotlessly spew out a full-on lengthy essay as though being produced akin to flowing crystal clean water. Good for them. And, if the essay or narrative is relatively short and sweet, once again the top of the head wing-it approach might be suitable. Anyway, all I’m saying is that outlines or skeletons are a useful way to get things done and that though it isn’t a cure-all can heroically be advantageous at times (a lot of the time, I brazenly proclaim).

Famed mystery and crime writer Jeffery Deaver has said this about outlines: “I spend eight months outlining and researching a novel before I begin to write a single word of the prose.”

Imagine the kind of steely nerves required when sketching out an outline or skeleton. You are in some sense delaying the task in front of you. The temptation to get into the zone and start your writing right away is enormous. Each moment that you devote to the outline or skeleton is seemingly a moment stolen from the act of writing out fully the essay or narrative being devised.

But we also have to realize that dashing madcap into the act of writing can immensely backfire. You write a bunch of words and then realize that they aren’t going to be relevant to the matter at hand. Ouch, you now delete those words and have wasted precious time composing them. Or maybe you write a bunch of stuff and it is unorganized and meandering. You then spend an inordinate amount of time trying to straighten things out accordingly.

The gist is that doing an outline or skeleton upfront can be highly beneficial. You are making a bet that the time and effort of constructing the sketch will be worthwhile and overall outdo the potential woes and downsides of just leaping into the fray. Sometimes this is a truly smart bet. You need to determine for any particular situation whether the outline or skeleton will be a payoff.

Let’s also agree that the inherent benefits of outlines apply to a lot of settings, beyond merely writing an essay or narrative. Suppose you are going camping. If you do so without first identifying what you need to take with you, there is a possibility that you’ll be empty-handed or have difficulties when you arrive at your camping destination. So, the astute thing to do is make a list of what you need to take along. The list might begin as an outline or skeleton. After the list is drafted, you then fill in the details such as whether you own the item or need to purchase it, whether each item is in suitable shape for camping, and so on.

A few other salient considerations about outlining come to mind too.

You often will do an outline or skeleton based on an initial high-level perspective. Upon mulling over and inspecting the outline, there is a leading chance you might move items around or reword them. In that sense, you are likely to iterate, make changes, and evolve the structure. There’s nothing wrong with doing so, which I mention because some people seem to get out of sorts if their outline isn’t pristine at the get-go. Chill out and let the process play out, you’ll undoubtedly get a better result (well, not always, but enough of the time).

Sometimes you can opt to craft an outline or skeleton and call it a day. You don’t need to do anything else to it. Stamp it as accomplished and completed. More likely, though, will be that you will take a subsequent outing at expanding the outline. You will make use of the skeleton to then pour out your words of wisdom onto the page. Ergo, the overt and likely expansion of an outline or skeleton is part-and-parcel of this process.

You usually do not ardently lock yourself into your outline. Safer is the realization that you need to remain flexible once you get into the throes of doing the writing. During the expansion phase of an outline, you might be triggered to rethink and rejigger the skeleton. If you doggedly refuse to adjust, you are potentially undermining the genuine spirit of an outline. Normally, an outline is a guide and not an irrefutable inarguable concrete wall that is set in stone.

J. K. Rowling, the esteemed author of the Harry Potter series, notably indicated this about outlining: “I always have a basic plot outline, but I like to leave some things to be decided while I write.”

An effort to create an outline will nearly always have an underlying basis or purpose. Rarely would you out of thin air just decide to piece together such an artifact. The crux is that the skeleton has a foundational goal or aspiration as to what it is supposed to encompass. These aren’t just random wordings or random making of lists. It is all undertaken with a purpose in mind.

In recap, we can assert these precepts about outlines and skeletons:

At this juncture, I am sure that you are waiting with bated breath to see how those principles apply to generative AI amid the formulation and use of the new prompt engineering technique known as skeleton-of-thought.

Wait no more, we are next heading into that exciting elucidation.

Skeleton-Of-Thought Brings Smart Outlining To Generative AI

Here’s the deal.

Those using generative AI are usually bent on getting the AI to spew out an answer or essay in its entirety. You can also tell the AI app to use chain-of-thought reasoning as an approach to prod the generative AI toward showcasing the poured-out answer or essay on a stepwise or step-at-a-time basis. All in all, you are still seeking to get the whole kit-and-kaboodle done all at once. The big bang attitude toward using generative AI.

Sometimes less is more, namely that we might be willing to have the generative AI first produce an outline or skeleton of what the essay or answer is likely to consist of (you might recall, I mentioned at the start of this discussion that the catchphrase of less is more would be worthy of remembrance).

Doing an outline or skeleton in generative AI has several vital benefits.

First, you never really know beforehand whether the generative AI will sufficiently or properly grasp whatever question or instruction you are giving to it. You might ask for an essay about banks and be surprised that the AI app generates a thousand words about the banks of a river or stream, rather than about financial institutions and banking. Generative AI is like a box of chocolates. You never know precisely how it is going to respond.

This can be exasperating to get an essay or narrative that is off base what you wanted. Furthermore, it can be costly if you are paying for your use of generative AI. You just tossed away dough on a useless and irrelevant generated result. It is unlikely that the AI maker will offer you a refund.

What can we do about this overall box of chocolate dilemma?

Please tighten your seat belt and get ready for the big reveal (actually, I’m sure you’ve garnered the drift of where I am headed).

Here’s what you can do:

Voila, you can then inspect the skeleton to see if the generative AI is on-target or off-target of your interests. Assuming that the generative AI is on-target, you can tell it to expand the outline and thus get the rest of your verbiage. If the generative AI is off-target, you can instruct it to change direction or maybe start clean if things are really fouled up.

Another plus to this skeleton issuance is that you’ll presumably avoid those costly wrong-topic essays or narratives that the generative AI might inadvertently produce for you. You will nip things in the bud. Admittedly, that being said, there is the cost of the outline being generated and then a second cost to do the expansion, but the odds are that this will be roughly the same as having requested the entire essay at the get-go. The savings come from averting the generation of content that you didn’t intend to get.

There is a potential hidden added plus to using the skeleton-of-thought approach. I earlier noted that the chain-of-thought technique seems to improve the AI-generated results. The same seems to be feasible for the skeleton-of-thought technique. Research so far tentatively suggests that the production of an outline or skeleton will prime the pump for the generative AI (I trust that you noticed I used the proverb of prime the pump, a tasty hint mentioned at the start of this discussion). Once the generative AI has generated the skeleton, it seems to be likelier to stay on course and produce the rest of the answer or essay as befits the now-produced skeleton.

I’m not asserting that the SoT will always be meritorious, which can similarly be said about the use of CoT. They both on-the-balance seem to be quite helpful. Whether this is always the case is certainly debatable. You will need to judge based on your own efforts in using CoT and using SoT.

One aspect to keep in mind is that the outline or skeleton alone might have keen merit for you.

Suppose you are asked by your boss to create a plan for revamping the enterprise system network (let’s assume that your company approves of the use of generative AI for work purposes; some don’t, so be careful about using generative AI in a work context). You are to come up with the key steps of the plan at a high level. This will then be reviewed by your boss. All manner of timing, budgeting, and the rest will need to be done, long before the plan is carried out.

You turn to generative AI and ask the AI app to generate a plan for this purpose, instructing the AI to do so in a skeleton form. The generated outline is all that you need in this instance. You aren’t anticipating expanding it, at least not at this time. Notice that you can ergo use the skeleton-of-thought technique to produce solely a skeleton or outline, doing so without asking for an expansion. Sometimes you’ll seek an expansion, sometimes not.

I urge you to consider adding SoT to your prompt engineering repertoire.

Here is my recommended set of 10 steps to undertake when making use of the skeleton-of-thought (SoT) method while using generative AI:

Follow those recommended steps carefully and mindfully.

Here’s why. Those that opt to toss around the skeleton-of-thought (SoT) method and do so willy-nilly are bound to get willy-nilly results. Your best course of action is to be systematic. Use this new technique judiciously and in a classic Goldilocks frame of mind, not being overly hot or overly cold.

I’ve got yet another trick up my sleeve as to why the skeleton-of-thought can be advantageous. It has to do with the speed of processing that underlies the execution of your requests to a generative AI app. The typical generative AI app will be devised to work on your request on a serial basis, generating a result on a do-one-thing and then do-the-next-thing basis.

AI makers and AI researchers aim to speed up how generative AI functions under the hood. If possible, it would be nifty if parallelism could be enacted. The idea is that your request might be parceled out to multiple computer processors and be acted on in a simultaneous or parallel fashion.

A somewhat simplified viewpoint is that the generative AI might conventionally use a computer processor by inching along a long pipeline of generating your result. If the generating activities could be undertaken simultaneously, this could immensely speed up getting your results produced. Most of us might not notice this faster response time, especially if doing rudimentary actions with generative AI. Nonetheless, this parallelism could be significant for heftier uses of generative AI. There is also something to be said about being able to across the board be more efficient when having thousands or millions of users that are seeking to use a generative AI app.

I mention this because one clever trick would be for the generative AI to do the expansion of the produced skeleton or outline by expanding each of the identified elements on a parallel basis. Let’s imagine that I have an outline that was generated by AI and the outline has eight major elements. I ask the AI to expand the eight elements. The usual internal mechanics might be that the first item gets expanded, then the second item, then the third, and so on. A generative AI with a parallelism capacity might be able to expand all eight simultaneously. The expansion will likely happen behind the scenes and you won’t see anything happen other than that the expanded version shows up on your screen. You’ll have no clue whether it was done on a serial basis or a parallel basis (unless this is revealed to you by the generative AI app).

A hitch to this parallelism is that it might not be viable if the elements of the skeleton are co-dependent or interdependent of each other. Depending upon the circumstance, serial internal processing might be the only suitable recourse.

AI Research Explores Those Dancing Skeletons

With the fundamentals of SoT now on the table for you, we can do a deeper dive into the nature of skeleton-of-thought prompts, along with exploring a recent research paper on some important nuances of this new technique for generative AI.

The research study is entitled “Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding” by Xuefei Ning, Zinan Lin, Zixuan Zhou, Huazhong Yang, and Yu Wang, and was posted online on July 28, 2023. I’ll be citing various excerpts from the study. Make sure to consider reading the full paper to get all the intricacies of this latest AI research.

Here’s what they opted to explore in their research:

I trust that you can see from those excerpts that they are on the hunt to ascertain some key advantages for using this technique, especially the leveraging of parallelism to speed-up processing.

In a nutshell, here’s their overall results:

The speed-up can occur, though if the type of questions being asked do not lend to answers that are readily parallelized, you might not get the speed-up bump.

They provided the various prompts that they used as part of their experimentation, which also kindly illustrates how to craft prompts to get generative AI into a skeleton-of-thought mode.

Consider for example this prompt:

Note that the prompt tries to give guidance to the generative AI about how to compose a skeleton response. This is important, or else the generative AI might go in some unexpected direction. Another thing to keep in mind is that you can potentially do the SoT prompt as a primer or establisher, whereby you might say this at the start of each conversation and then invoke it later on in a conversation via a trigger keyword or catchphrase, see my coverage at the link here.

When you want to do the expansion of a generated skeleton, here’s an example of how they did so:

You can express the same nuances about the expansion in other styles as befits your approach to writing prompts. The research study made use of some AI industry-established prompts too.

For example, here’s a prompt expressing a preference for coherence in the expansion:

Here’s a prompt indicating a preference for a multitude of perspectives to be encompassed (diversity):

Yet another prompt that seeks to ensure relevancy in the response:

You can play around with those prompts and use them when the situation seems applicable.

One question that requires noodling on is whether you want the skeleton to be repeated verbatim during the expansion or whether you want the expansion to potentially be changed up by the generative AI. Usually, the generative AI will do a verbatim repeat of the outline. This seems befitting since you can then see exactly what the original structure was, and that the expansion appears to be abiding by the original structure. Changes to the original structure might be annoying and cause suspicion that the generative AI has lost its way. The counterargument is that if the generative AI is allowed to vary the original structure, this might perhaps be useful, assuming that the generative AI is adding value as a result of doing the expansion.

Mull that over and decide case-by-case what works for you.

An Example Of Skeleton-Of-Thought To Showcase How It Works

The research encompassed a number of experiments to gauge how SoT seems to do. The prompts and questions that were entered into the generative AI apps were generally somewhat short and relatively straightforward. In their conclusion remarks, they call upon AI researchers to go in additional directions to see how SoT functions under a variety of supplementary settings.

I opted to do a series of akin experiments using a somewhat complex problem. In prior columns, I had taken a look at prompt engineering techniques showcased via a legal scenario involving questions of fiduciary duty by corporate officers, see the link here. Specifically, I examined legal matters involving company duty of loyalty and company duty of care.

I repeated those efforts by using SoT. Due to space limitations herein, I am not able to include the details, but please know that I did discern that the SoT approach adds value and has utility in the right situations and when used in the right way.

Let’s take a look at an example from the research paper to see how their use of generative AI turned out. I already showed you the SoT prompt that sets the stage for the generative AI to go into a skeleton-of-thought mode, so I won’t repeat that here. Assume that the generative AI is now engaged in the SoT mode. The example I’ve excerpted involves a question about how to determine if someone is genuine or simply being polite.

Here we go.

The prompt question that was entered into generative AI consisted of:

Generative AI generated this answer as a skeleton or outline:

Generative AI subsequently was asked to generate an expansion of the answer:

Here, in contrast, is the conventional use of generative AI when producing a non-SoT essay answer:

The example above shows the skeleton that was derived, and shows the expansion of the skeleton, plus provides a handy-dandy comparison via a conventional essay answer that was generated. Each of those variations has merit. None is intrinsically better or worse than the other. Your circumstance at hand ought to signal which method is applicable for your generative AI use.

Conclusion

I hope that this discussion about this new technique sparks you into considering making use of it, from time to time and as relevant to do so. If you were intending to preferably get a conventional essay, the SoT would probably not be the direct way to go. If an outline is needed, and potentially an expansion of the outline, the SoT is your erstwhile course of action.

A few final remarks for now.

Outlines are great. They serve a useful purpose. Most people use them quite frequently. You don’t always use outlines or skeletons. I am reminded of the famous line that if all that you have is a hammer, everything in the world is treated like it is a nail. You go around hammering things that don’t require hammering. Worse still, you might hammer something that via another tool would serve a useful purpose and achieve whatever aim it has.

Skeleton-of-thought is an interesting and useful tool for prompt engineering. Use it well. Use it wisely. I’d advise that you have SoT in your toolbox of prompt engineering techniques and methods. Leverage SoT as seems befitting.

Sometimes less is more and priming the pump is grandly fruitful.