LMQL: A Programming Language for LLM Interaction

LMQL, developed by the SRI Lab at ETH Zurich, is a specialized programming language tailored for interacting with large language models (LLMs). It offers a robust and modular approach to prompting, enabling developers to create complex queries with types, templates, constraints, and an optimizing runtime.

Key Features

• Modular Prompting: LMQL allows for the creation of modular prompts, making it easier to manage and reuse components.
• Type Safety: With typed variables, developers can ensure that the output format meets specific requirements, enhancing reliability.
• Constraint Enforcement: The language supports hard constraints that are enforced by the runtime, ensuring that generated content adheres to predefined rules.
• Portability: LMQL code can be executed across multiple backends, including OpenAI, Hugging Face Transformers, and llama.cpp, with minimal changes.

Use Cases

LMQL is particularly useful in scenarios where precise control over LLM outputs is necessary. This includes:

• Research: Facilitating detailed queries for academic and scientific research.
• Development: Streamlining the development process by providing a structured way to interact with LLMs.
• Automation: Automating tasks that require complex interactions with language models.

Getting Started

To start using LMQL, you can define a query function and execute it directly from Python. The language's syntax is designed to be intuitive, allowing developers to quickly grasp its capabilities.

Example

Here's a simple example of how LMQL can be used to query the meaning of life:

@lmql.query
def meaning_of_life():
    '''lmql
    # top-level strings are prompts
    "Q: What is the answer to life, the universe and everything?"
    # generation via (constrained) variables
    "A: [ANSWER]" where len(ANSWER) < 120 and STOPS_AT(ANSWER, ".")
    # results are directly accessible
    print("LLM returned", ANSWER)
    # use typed variables for guaranteed output format
    "The answer is [NUM: int]"
    # query programs are just functions
    return NUM
    '''

# so from Python, you can just do this
meaning_of_life()  # 42

Conclusion

LMQL represents a significant step forward in the interaction with large language models, offering a structured and efficient way to manage complex queries. Its features make it a valuable tool for researchers and developers alike, particularly those working on projects that require precise control over LLM outputs.