LLM Handbook: Strategies and Techniques for Practitioners

Image by Author

Large Language Models (LLMs) have revolutionized the way machines interact with humans. They are a sub-category of Generative AI, with a focus on text-based applications, while Generative AI is much broader including text, audio, video, images, and even, code!

AWS summarizes it well – “Generative artificial intelligence (generative AI) is a type of AI that can create new content and ideas, including conversations, stories, images, videos, and music. It reuses training data to solve new problems.”

Generative AI has opened up new frontiers in the AI landscape!

LLMs come with their ability to generate human-like responses, but how should AI practitioners use them? Is there a guide or an approach to help the industry build confidence with this cutting-edge technology?

That’s precisely what we will discuss in this article. So, let’s get started.
 

An assistant to get started !!!

LLMs are essentially generators, so it is advised to use them for purposes, such as generating summaries and providing explanations, and answers to a wide range of questions. Typically, AI is used to assist human experts. Similarly, LLMs can augment your understanding of complex topics.

Industry experts consider LLMs as good sounding boards – yes, they are good for asking validation questions, brainstorming ideas, creating drafts, or even checking whether there is a better way to articulate the existing content. Such recommendations provide developers and AI enthusiasts the playground to test this powerful technology.

Not just text, LLMs help generate and debug code, as well as explain complex algorithms in an easy-to-understand manner, highlighting their role in demystifying the jargon to provide a tailor-made conceptual understanding for different personas.  
 

Benefits!!

Now, let’s discuss some of the cases underscoring the role of LLMs in bringing efficiencies. The examples below focus on generating reports and insights, and simplifying business processes.

Collaboration Tools: Creating summary reports of data shared across applications such as Slack, is a very effective way to stay informed about projects’ progress. It can include details like the topic, its current status, the development thus far, the participants, action items, due dates, bottleneck, next steps, etc.

Image by Author

Supply Chain: The supply chain planners are mostly in a fire-fighting situation to meet the demand orders. While supply chain planning helps a lot, the last mile delivery requires experts to come together in the war room to keep the supply chain plan intact. A lot of information, often in the form of text gets exchanged, including insights that are helpful for future purposes too. Plus, the summary of such conversations keeps all the stakeholders informed of the real-time status. 
 

Adopting LLMs

With rapidly evolving advancements in technology, it is crucial to not give under the fear of missing out, but instead approach with the business-first mindset. 

In addition to suggestions proposed above, the users must keep themselves updated and regularly check for new techniques, and best practices to ensure the effective use of these models.
 

Separate Facts from Fiction

Having discussed the benefits of LLMs, it is time to understand the other side. We all know there is no free lunch. So, what does it require to make responsible use of LLMs? There are a lot of concerns like model bias, potential misuse such as deepfakes, and their repercussions, requiring increased awareness of the ethical implications of LLMs. 

Image by Author

The situation has worsened to the extent that it has become increasingly difficult to segregate human-generated responses from that of a machine. 

So, it is advised to not consider the information from such tools at face value, instead, consider these tips:

• Refer to models as efficiency-enhancing tools and not as a single point of truth.
• Crowdsource information from multiple sources and cross-check it before taking action – the ensemble works great by bringing together different viewpoints. 
• While you consider the importance and the trust factor of information coming from multiple sources, always check the source of information and the citations, preferably the ones with a higher reputation.
• Do not assume the given information is true. Look for contrarian views, i.e. what if this were wrong? Gather evidence that helps you refute that information is incorrect, rather than trying to support its validity.
• The model response often has gaps in its reasoning, read well, question its relevancy, and nudge it to get to the appropriate response

Tips to Consider while Prototyping LLMs

Let’s get straight to the practical applications of LLMs to know their capabilities as well as limitations. To start with, be prepared for multiple experiments and iteration cycles. Always stay informed about the latest industry developments to get the maximum benefits of the models.

The golden rule is to start from business objectives and set clear goals and metrics. Quite often, the performance metrics include multiple goals in terms of not just accuracy, but also speed, computational resources, and cost-effectiveness. These are the non-negotiables that must be decided beforehand.

The next important step is to choose the right LLM tool or platform that suits the business needs, which also includes the consideration of the closed or open source model. 

Image by Author

The size of the LLMs is another key deciding factor. Does your use-case demand a large model or small approximator models, which are less hungry on compute requirements, make a good trade-off for the accuracy they provide? Note that the larger models provide improved performance at the cost of consuming more computational resources, and in turn the budget.

Given the security and privacy risks that come with the large models, businesses need robust guardrails to ensure their end users' data is safe. It is equally important to understand the prompting techniques to convey the query and get the information from the model.

These prompting techniques are refined over time with repeated experiments, such as by specifying the length, tone, or style of the response, to ensure the response is accurate, relevant, and complete.
 

Summary

LLM is, indeed, a powerful tool for an array of tasks, including summarizing information to explaining complex concepts and data. However, successful implementation requires a business-first mindset to avoid getting into AI hype and find a real valid end-use. Furthermore, awareness of ethical implications such as verifying information, questioning the validity of responses, and being cognizant of potential biases and risks associated with LLM-generated content promotes responsible utilization of these models.
 
 

Vidhi Chugh is an AI strategist and a digital transformation leader working at the intersection of product, sciences, and engineering to build scalable machine learning systems. She is an award-winning innovation leader, an author, and an international speaker. She is on a mission to democratize machine learning and break the jargon for everyone to be a part of this transformation.