AI output reviews
Last updated
Was this helpful?
Last updated
Was this helpful?
Relay.app's built-in Human-in-the-Loop tools make it very easy to ensure that a human first reviews the output of an AI step before the workflow proceeds.
In the Behavior section of every AI step, there's a question 'Should a person review the AI output?' that lets you (conditionally) turn on Human-in-the-Loop AI output reviews.
By default, the option to require a human review is turned off for newly created steps. To turn it on, toggle this setting to either
Yes, always if you want the AI output to be reviewed every time by a person before continuing the run. Note that if this workflow runs very frequently, you'll also get many review requests that need to be acted on
Only if at least one output field is empty
Only if all output fields are empty
Only if AI cannot generate a valid output
Next, select the medium the reviewer should receive the notification on.
Finally, you may personalize the notification message that's sent every time a human review is needed. You may also set a due date and configure reminder notifications for when the assignee didn't review the AI output in time.
Every time someone is notified to review the output of the AI, they have the following options:
Approve the AI output as-is. After the approval, the run will continue with the next step(s) in the workflow.
End the workflow run right away. If based on the AI output the assignee decides the workflow run shouldn't continue, they can end it right away, skipping all following steps.
Manually edit/override the AI output (i.e. to tweak values)
Re-prompt the AI. A new ouput will be genearted based on updated one-off instructions/clarifications in the prompt. The assignee will be notified again when the new output is ready for review (they'll have the same options again).
The assignee for the reviews should be a collaborator in your workflow. .
