# AI output reviews

Relay.app's built-in Human-in-the-Loop tools make it very easy to ensure that a human first reviews the output of an AI step before the workflow proceeds.

## Setting up AI output review in the Editor

In the **Behavior** section of every AI step, there's a question '**Should a person review the AI output?**' that lets you (conditionally) turn on Human-in-the-Loop AI output reviews.

### 1. Select the review conditions

<div align="left"><figure><img src="https://647519108-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXkKoEy3xMIYaIwd4mRAB%2Fuploads%2Fgit-blob-3a53111d611223f0ab2386c2ffb76b1c0857029e%2FCleanShot%202025-04-29%20at%2015.51.59.png?alt=media" alt="" width="375"><figcaption><p>Human AI Output reviews are available for all AI steps</p></figcaption></figure></div>

By default, the option to require a human review is turned off for newly created steps. To turn it on, toggle this setting to either

* **Yes, always** if you want the AI output to be **reviewed every time** by a person before continuing the run. Note that if this workflow runs very frequently, you'll also get many review requests that need to be acted on
* **Only if at least one output field is empty**
* **Only if all output fields are empty**
* **Only if AI cannot generate a valid output**

### 2. Choose the mechanism to notify the reviewer

<div align="left"><figure><img src="https://647519108-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXkKoEy3xMIYaIwd4mRAB%2Fuploads%2Fgit-blob-c3a9a12bfb49d90b2e298d01b5ceaca4c245fc75%2FCleanShot%202025-04-29%20at%2015.53.53.png?alt=media" alt="" width="375"><figcaption><p>Select the medium to notify the reviewer</p></figcaption></figure></div>

Next, select the medium the reviewer should receive the notification on.

### 3. Select the reviewer (assignee)

<div align="left"><figure><img src="https://647519108-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXkKoEy3xMIYaIwd4mRAB%2Fuploads%2Fgit-blob-5b51e8f9c4d65c85669480508660391e68eacc7f%2FCleanShot%202025-04-29%20at%2015.55.41.png?alt=media" alt="" width="351"><figcaption><p>The assignee should be a collaborator of the Workflow</p></figcaption></figure></div>

The assignee for the reviews should be a collaborator in your workflow. [Learn more about Collaborators here](https://docs.relay.app/workflows/sharing-workflows#collaborators).

### 4. Personalize the notifications and reminders

<div align="left"><figure><img src="https://647519108-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXkKoEy3xMIYaIwd4mRAB%2Fuploads%2Fgit-blob-36d4424819554de062f7a070afe39a3bee3e9f05%2FCleanShot%202025-04-29%20at%2015.57.42.png?alt=media" alt="" width="375"><figcaption><p>Optionally, personalize the review notifications and reminders</p></figcaption></figure></div>

Finally, you may personalize the notification message that's sent every time a human review is needed. You may also set a due date and configure reminder notifications for when the assignee didn't review the AI output in time.

## Reviewing AI outputs

Every time someone is notified to review the output of the AI, they have the following options:

* **Approve the AI output as-is**. After the approval, the run will continue with the next step(s) in the workflow.
* **End the workflow run right away**. If based on the AI output the assignee decides the workflow run shouldn't continue, they can end it right away, skipping all following steps.
* **Manually edit/override the AI output** (i.e. to tweak values)
* **Re-prompt the AI**. A new output will be generated based on updated one-off instructions/clarifications in the prompt. The assignee will be notified again when the new output is ready for review (they'll have the same options again).