Automatic queries
An automatic query is a rule that raises a query for you when a submission matches it. Use one to raise a query on a value that is out of range, an answer that needs a clinician's attention, or a score above a threshold, without waiting for someone to spot it. It is assigned to the right people and lands in their inbox minutes after the data arrives.
Find them under Queries > Automatic queries. The tab has two parts: Rule-based automatic queries, which you define, and "Smart" automatic queries, which ask Trialflare AI to look for anything anomalous.
A query raised automatically is an ordinary query in every other respect. It appears in the queries list marked as raised by Trialflare, its assignees are emailed, and it is worked on and closed in the same way. See Queries for that side of it.
Creating an automatic query
Click Create a new automatic query. The form has three sections.
General
- Name and Description are for your team. They identify the rule on the Automatic queries tab, and participants and assignees never see them.
- Data type is the field the rules are about, and the field the query is raised on. Choose No data type for a query about the submission as a whole, such as a total score. See Queries about the whole submission.
Rules
Add one or more rules with Add a rule. All of them must be met for a query to be raised: three rules mean "this, and this, and this". There is no "or". For "either of these", create two automatic queries.
| Rule type | Is met when |
|---|---|
| Data type (field) value | The field's answer meets a condition. Every field offers Exists and Does not exist. A choice field adds Is one of and Is none of. A number, slider, rating, Likert, weight or height field adds Equals, Does not equal, Greater than and Less than. |
| Score (field) | The points the field scored on this submission meet a condition. |
| Score (whole response) | The submission's total score meets a condition. See Scoring. |
| Stage name | The submission is of this stage. For a repeating stage, you can narrow it to one recurrence or leave it on All recurrences. |
| Participant group | The participant is in this group. |
| Participant site | The participant is at this site. |
| PROMIS T score or PROPr score | Offered for PROMIS fields only: the calculated score meets a condition. |
Rules that need a field (Data type (field) value and Score (field)) can only be chosen once a data type is set. A comparison on a number field (Equals, Greater than and so on) needs a number to compare against.
A few field types have their own rules:
- Choice: Is one of raises the query when any of the options you pick was selected, and Is none of when none of them was. On a multiple-selection field, a rule naming Nausea matches whether nausea was picked alone or with other options. See Data types: choice.
- Likert matrix: a number rule compares the total across its statements, so no single statement can trip it. Exists is met as soon as any statement has an answer, including Not applicable.
- Table: only Exists and Does not exist are available, since a table's answer is a set of rows rather than one value.
Query template
What the raised query will say, and who it goes to:
- Query title, required, and Query description. These are copied into every query the rule raises exactly as written. They can't include the participant's ID or the value; the query itself shows both, along with a link to the submission.
-
Auto-assign query to: one or more members of your team. Each is emailed as soon as a query is raised, and gets a to-do if your team has Tasks enabled. Leave it on Do not auto-assign and the query is still raised, but nobody is told. It waits in the queries list until someone looks.
If someone on the list leaves your team, queries keep being raised and go to whoever remains. With nobody left, they are raised unassigned. They're dropped from the list the next time the rule is saved, so it's worth checking a rule's assignees when someone leaves.
Click Save. The rule applies to submissions from then on.
When a rule is checked
A rule is checked every time a submission is saved, whoever saves it: a participant in the app, a member of staff entering or correcting data, or someone answering a questionnaire link. Drafts are not checked. The query appears a few moments after the save, not instantly.
The submission is judged as it is stored, with every answer on it, not only the ones that were just changed.
Rules only look forward. A new or edited rule is not applied to data already in the study. Submissions that already match only raise a query if they are saved again.
A query stays open until someone closes it. Correcting the answer that raised a query doesn't close the query, because the query is the record of what was asked about and how it was resolved. Close it once the correction is made.
How often a query is raised
That depends on whether the rule is about the whole submission:
- A rule that includes a Score (whole response) rule is raised at most once per submission. Correcting or adding to that submission later doesn't raise it again, even though its score is recalculated. The query records the score at the moment it was raised.
- Any other rule is raised once for each answer. Saving the submission again with the same answer, or correcting a different field on it, doesn't raise the query again, whether the first one is still open or has been resolved. If the answer itself is changed to another value that still meets the rule, a new query is raised about the new value. So is a value that was queried, corrected, and later entered again.
Hidden and blank fields
A field hidden by a Show if condition was never asked, so rules about it are never met, not even Does not exist.
A field that was shown and left blank meets Does not exist, which is how you raise a query on a missing answer. It meets no other rule: a blank answer is not "not equal" to anything, and is "none of" nothing.
On a stage that allows a partial submission, the submission is judged as it stands. A part-finished form is judged on what it scores so far, with the fields still to come counted as blank.
Queries about the whole submission
Set Data type to No data type when the query is about the submission as a whole, typically because its total score crosses a threshold: "PHQ-9 total greater than 19". The query is raised against the submission rather than one field.
A rule like this needs two things, and can't be saved without them:
- a Score (whole response) rule, since there's no field to judge;
- a Stage name rule, so it only applies to that stage. Without one it would be judged against every submission in the study, including forms that award no points at all, which score zero. A Less than rule would then raise a query on all of them.
It can't include rules that need a field. Group and site rules are fine.
See also Stages: Automatic queries on a score.
Routing queries to the right people
The assignees on a rule are a fixed list. To send queries about participants at different sites to each site's own team, create one automatic query per site, each with a Participant site rule and that site's staff as assignees. The same works for groups: a Participant group rule and the group's clinicians.
Questionnaire submissions
Rules are checked on questionnaire submissions too, so screening answers can raise queries before anyone is enrolled. Until a submission is allocated, it belongs to no participant. The query is recorded against the submission, and shows Questionnaire submission in place of a participant ID.
For the same reason, Participant group and Participant site rules are never met by a submission that hasn't been allocated. A submission allocated by a questionnaire rule is allocated before automatic queries are checked, so those rules do apply to it. Allocating a submission by hand later doesn't check the rules again.
"Smart" automatic queries
Where your team has the feature, Trialflare AI can look at each submission for values that look anomalous: a weight of 7kg, a date of birth next year, an answer that doesn't fit with the rest. No rule describes them, so this catches what you didn't think to write a rule for.
Tick Enable smart automatic queries on the Automatic queries tab, then:
- Additional instructions, optional: tell Trialflare what to pay attention to or ignore in this study, such as "weights below 40kg are expected in this cohort". Click Save instructions.
- Assign new smart automatic queries to: who is notified. Left empty, smart queries are raised unassigned. As with rule-based queries, someone who leaves the team simply stops being assigned.
A smart query's title says what Trialflare found, such as Event end date is before its start date or Serious adverse event reported, and its description explains why. Like a rule-based one, it is marked as raised by Trialflare, counts as automatic in your activity figures, and is flagged as automatic in exports. Things to know:
- It can be wrong. Treat a smart query as a prompt to look, not a finding.
- Fields marked PII or restricted are never sent to the model.
- It is charged to the trial's wallet for each submission it checks. If the balance runs out, smart queries pause until it is topped up, and submissions made in the meantime are not checked later. Rule-based automatic queries are free and keep running whatever the balance.
- It only looks at new answers. Each save is checked for the fields it gave a new or changed answer. A submission saved again unchanged isn't checked or charged again, and correcting one field doesn't raise smart queries on the others.
Editing and deleting
Edit reopens the rule. Changes apply to submissions saved from then on. Queries the rule has already raised are left as they are.
To stop a rule, delete it with the bin icon. There is no way to switch one off temporarily. Deleting a rule doesn't affect queries it has already raised.
Automatic queries can't be created or edited while the trial's database is locked. While locked, no data can be submitted either, so nothing would be checked.
Creating, changing and deleting automatic queries is recorded in the trial's event log under Settings > Logs.
Copying a trial doesn't copy its automatic queries. Recreate them in the copy.
Permissions
| To do this | You need |
|---|---|
| See the automatic query rules | trial.readAutomaticQueries (or trial-wide read access) |
| Create, edit and delete them | trial.writeAutomaticQueries (or trial-wide write access) |
| Switch smart automatic queries on and change their settings | Permission to manage the trial |
The queries the rules raise follow the ordinary query permissions.