How to get started with Rigging
Generator
object. We’ll use get_generator
which will resolve an identifier string to the underlying generator class object.
,api_key=[sk-1234]
to the end of any of the generator IDs to specify them inline.litellm!anthropic/claude-3-sonnet-20240229
, where litellm
is the provider. We just default to that generator and you don’t have to be explicit. You can find more information about this in the generators docs.await
to trigger generation and get your results, or use await_
.chat()
method which you’ll use to initiate the conversations. You can supply messages in many different forms from dictionary objects, full Message
rigging.message.Message] classes, or a simple str
which will be converted to a user message.
generator.chat
is actually just a helper for chat(generator, ...)
, they do the same thing.
chat()
as pipeline
. The naming might be confusing,
but chats go through 2 phases. We first stage them into a pipeline, where we operate
and prepare them before we actually trigger generation with run()
.Calling .chat()
doesn’t trigger any generation, but calling any of these run methods will:rigging.chat.ChatPipeline.run
rigging.chat.ChatPipeline.run_many
rigging.chat.ChatPipeline.run_batch
rigging.chat.ChatPipeline.run_over
.run()
to execute the generation process and collect our final Chat
object.
ChatPipeline
and Chat
objects provide freedom
for forking off the current state of messages, or continuing a stream of messages after generation has occurred.
In general:
ChatPipeline.fork
will clone the current chat pipeline and let you maintain both the new and original object for continued processing.Chat.fork
will produce a fresh ChatPipeline
from all the messages prior to the previous generation (useful for “going back” in time).Chat.continue_
is similar to fork
(actually a wrapper) which tells fork
to include the generated messages as you move on (useful for “going forward” in time).Chat.continue_
after each round of generation.
fork
has created a clone of our chat pipeline.str
objects in place of full messages, which underneath will be converted to a Message
object with the user
role.FunFact
model which we’ll have the LLM fill in. Rigging exposes a Model
base class which you should inherit from when defining structured inputs. This is a lightweight wrapper around pydantic-xml’s BaseXMLModel
with some added features and functionality to make it easy for Rigging to manage. However, everything these models support (for the most part) is also supported in Rigging.
.xml_example()
class method which all models support. By default this will simple emit empty XML tags of our model:
tag=[value]
into your class definition like this:
.parse()
on the last message of our generated chat. This will process the contents of the message, extract the first matching model which parses successfully, and return it to us as a python object.
FunFact
as a class, the result if .parse()
is type-hinted object of that class. In our code, all the properties of FunFact
will be available just like we created the object directly.
Notice that we don’t have to worry about the model being verbose in it’s response, as we’ve communicated that the text between the <fun-fact></fun-fact>
tags is the relevant place to put it’s answer. If the model includes a thought process, supplemental information, or anything else, we can simply ignore it.
parse
will result in an exception from rigging. Rigging is designed at it’s core to manage this process, and we have a few options:
.until_parsed_as()
which will cause the run()
function to internally check if parsing is succeeding before returning the chat back to you..try_parse()
. The type of the return value with automatically switch to #!python FunFact | None
and you can handle cases where parsing failed..parse()
on the message despite using .until_parsed_as()
. This is a limitation of type hinting as we’d have to turn ChatPipeline
and Chat
into generic types, which could carry that information forward. It’s a small price we pay for big code complexity savings. However, the use of .until_parsed_as()
will cause the generated messages to have parsed models in their .parts
. So if you don’t need to access the typed object immediately, you can be confident serializing the chat object and the model will be there when you need it..until_parsed_as()
, a .then()
callback is registered internally to operate during generation. When model output is received, the callback will attempt to parse, and if it fails, it will re-trigger generation. This process will repeat until the model produces a valid output or the maximum “depth” is reached.
Often you might find yourself constantly getting MaxDepthError
exceptions. This is usually a sign that the LLM doesn’t have enough information about the desired output, or complexity in your model is too high. You have a few options for graceful handling these situations:
max_depth
to a higher value. This will allow the model to try more times before failing.allow_failed
to your run()
method and check the .failed
property after generation.then()
to get more external control over the process.run_many()
and generate N examples individuallyPrompt
functions can make parsing even easier. We can refactor our previous example and have rigging parse out FunFacts directly for us:
Message
objects, or even content parts for multi-modal generation (ContentImageUrl
)
Check out Tools for more information.
generate_jokes
prompt will be presented as an available tool when gpt-4o
is working on tasks, and rigging with handle all the inference and type processing for you.