Harsh_jhamtani Natural Language Decomposition and Interpretation of Complex Utterances 2023

[TOC]

Title: Natural Language Decomposition and Interpretation of Complex Utterances
Author: Jacob Andreas
Publish Year: 15 May 2023
Review Date: Mon, May 22, 2023
url: https://arxiv.org/pdf/2305.08677.pdf

Summary of paper

Motivation

natural language interface often require supervised data to translate user request into structure intent representations
however, during data collection, it can be difficult to anticipate and formalise the full range of user needs
we introduce an approach for equipping a simple language to code model to handle complex utterances via a process of hierarchical natural language decomposition.

Contribution

Experiments show that the proposed approach enables the interpretation of complex utterances with almost no complex training data, while outperforming standard few-shot prompting approaches.

Some key terms

Methodology

an approach called DECOMP, which decomposes complex utterances into a sequence of simpler natural language steps, each resembling an elementary utterance that an existing parser can handle, then uses this parser to map each elementary utterance to a sub-program.

Good things about the paper (one paragraph)

introduce new dataset that map utterances to programs : DeCU
DeCU contains 872 elementary utterances paired with programs.

Example prompt

Top of the prompt is elementary utterances similar to the task
Bottom of the prompt is the complex utterance decomposition examples
- the decomposition examples provide a demonstration of how to generate program fragments for a step conditioned on previous steps and help bridge any possible domain shift from elementary to complex utterances.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145


Utterance : Find all meetings happening this week
Program : val s1 = findEvents ( queryAt (` this `[ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : How many days in the next week do I have meetings ?
Program : val s = List [ DayOfWeek ]( Sunday , Monday , Tuesday , Wednesday , Thursday , Friday , Saturday ). filter ( day = > findEvents ( queryAt ( day /& ( next [ Interval [ Date ]] and isWeek ))). nonEmpty ). size
<EOS >
Utterance : Find the All hands that is on next Monday .
Program : val s2 = theEvent ( called (" All hands ") and queryAt ( theDate ( next [ Date ] /& Monday )))
<EOS >
Utterance : Who all are on the Vacation meet ?
Program : val s = theEvent ( called (" Vacation ")). attendees . all
<EOS >
Utterance : Reschedule all of list s2 to next week
Program : val s3 = s2 . map (x = > modifyEvent (x , startsAt (( next [ Interval [ Date ]] and isWeek ))))
<EOS >
Utterance : Create a list of the days in a week
Program : val s1 = List [ DayOfWeek ]( Monday , Tuesday , Wednesday , Thursday , Friday , Saturday , Sunday )
<EOS >
Utterance : Do we have vacation all day this time ?
Program : val s = theEvent ( called (" vacation ")). isAllDay
<EOS >
Utterance : Find event next week titled ' sync call '
Program : val s1 = theEvent ( called (" sync call ") and queryAt ( next [ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Find all meetings with my reports this week that are an hour long .
Program : val s1 = findEvents ( with_ ( me . directReports ) and queryAt (` this `[ Interval [ Date ]] and isWeek ) and lastsFor (1. hours ))
<EOS >
Utterance : Find all meetings on Monday .
Program : val s1 = findEvents ( queryAt ( Monday )). head
<EOS >
Utterance : Find all meetings on Tuesday .
Program : val s2 = findEvents ( queryAt ( Tuesday )). head
<EOS >
Utterance : Find all meetings on calendar
Program : val s1 = findEvents0
<EOS >
Utterance : Find all of the user 's reports .
Program : val s1 = me . directReports
<EOS >
Utterance : Find all one on one events from last week and return the size of that list .
Program : val s1 = findEvents ( queryAt ( last [ Interval [ Date ]] and isWeek ) and isOneOnOne ). size
<EOS >
Utterance : What do I have next week ?
Program : val s = findEvents ( queryAt ( next [ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Find emails from last week
Program : val s = findEmails ( messageSentAt ( last [ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Find events with Abby this week
Program : val s1 = findEvents ( with_ (" Abby ") and queryAt (` this `[ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Find meeting with Hao this week .
Program : val s1 = theEvent ( with_ (" Hao ") and queryAt (` this `[ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Find the next meeting with Jack
Program : val s1 = theEvent ( next [ Event ] and with_ (" Jack ") )
<EOS >
Utterance : Find meetings called All - Hands .
Program : val s = findEvents ( called (" All - Hands ") )
<EOS >
Utterance : Find all events happening today
Program : val s2 = findEvents ( queryAt ( today ))
<EOS >
Utterance : What all day events do I have this week ?
Program : val s = findEvents ( isAllDay and queryAt (` this `[ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Check my availability for next week
Program : val s = availabilityIncludingMe ( createAt ( next [ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Create a meeting with Jenn for next week
Program : val s = createEvent ( with_ (" Jenn ") and createAt ( next [ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Can you find the emails from Lisa from last week
Program : val s = findEmails ( messageWithSender (" Lisa ") and messageSentAt ( last [ Interval [ Date ]] and isWeek ))
<EOS >
Utterance : Make me a meeting at 3 pm mountain time and refuse any meetings that happen at the same time
Step 1: Grab 3 pm mountain time
Program 1: val s1 = 3. pm inZone TimeZone (" MST ")
Step 2: Create an event using time and timezone s1
Program 2: val s2 = createEvent ( createAt ( s1 ))
Step 3: Find events using time and timezone s1
Program 3: val s3 = findEvents ( queryAt ( s1 ))
Step 4: Decline each event in list s3
Program 4: val s4 = s3 . map (x = > respond (x , withResponse ( ResponseStatusType . declined )))
<EOS >
Utterance : Change my meetings with Abby and those with Dan this week to start 5 minutes later
Step 1: Find events with Abby this week
Program 1: val s1 = findEvents ( with_ (" Abby ") and queryAt (` this `[ Interval [ Date ]] and isWeek ))
Step 2: Find events with Dan and without Abby this week
Program 2: val s2 = findEvents ( with_ (" Dan ") and not ( with_ (" Abby ") ) and queryAt (` this `[ Interval [ Date ]] and isWeek ))
Step 3: Set all meetings from s1 to start 5 minutes later
Program 3: val s3 = s1 . map (x = > modifyEvent (x , startsAt (x. start . local . time + 5. minutes )))
Step 4: Set all meetings from s2 to start 5 minutes later
Program 4: val s4 = s2 . map (x = > modifyEvent (x , startsAt (x. start . local . time + 5. minutes )))
<EOS >
Utterance : Can you decline all the meetings where at least one attendee reports to Jeff .
Step 1: Grab Jeff 's reports
Program 1: val s1 = thePerson (" Jeff "). directReports
Step 2: Decline all events in which each person in list s2 is invited to
Program 2: val s2 = s1 . map (x = > findEvents ( with_ (x)). map (z = > respond (z , withResponse ( ResponseStatusType . declined ))))
<EOS >
Utterance : Can you tell me how many emails I received on Monday ?
Step 1: Query for how many emails were received on last Monday .
Program 1: val s1 = findEmails ( messageSentAt ( last [ Date ] /& Monday )). size
<EOS >
Utterance : Do I have a meeting with Ben on Monday or with Charlie on Wednesday ?
Step 1: Do I have a meeting with Ben on Monday or with Charlie on Wednesday .
Program 1: val s1 = findEvents ( with_ (" Ben ") and queryAt ( Monday )). nonEmpty || findEvents ( queryAt ( Wednesday ) and with_ (" Charlie ")). nonEmpty
<EOS >
Utterance : how long do i have before my next appointment ?
Step 1: Describe duration of time from now until next Meeting
Program 1: val s1 = durationFromNow ( theEvent ( next [ Event ]) . start )
<EOS >
Utterance : How many 1/1 meetings in total I had in the last week ?
Step 1: Find all one on one events from last week
Program 1: val s1 = findEvents ( queryAt ( last [ Interval [ Date ]] and isWeek ) and isOneOnOne )
Step 2: Get the size of that list s1 .
Program 2: val s2 = s1 . size
<EOS >
Utterance : If I have a meeting with Henry this week , change it to be with Nick
Step 1: Find meeting with Henry this week .
Program 1: val s1 = theEvent ( with_ (" Henry ") and queryAt (` this `[ Interval [ Date ]] and isWeek ))
Step 2: Update s1 to replace Henry with Nick , otherwise do nothing
Program 2: val s2 = modifyEvent (s1 , with_ (" Nick ") and not ( with_ (" Henry ")))
<EOS >
Utterance : I need to swap the calls that are on Monday and Tuesday .
Step 1: Find all meetings on Monday .
Program 1: val s1 = findEvents ( queryAt ( Monday )). head
Step 2: Find all meetings on Tuesday .
Program 2: val s2 = findEvents ( queryAt ( Tuesday )). head
Step 3: Swap the date for all meetings found in s1 with Tuesday .
Program 3: val s3 = modifyEvent (s1 , startsAt ( Tuesday ))
Step 4: Swap the start date for all meetings found in s2 with Monday .
Program 4: val s4 = modifyEvent (s2 , startsAt ( Monday ))
<EOS >
Utterance : Change the duration of the team sync call today to two hours , and schedule an event with identical title 10 days from now .
Step 1: Change the duration of the team sync call today to 2 hours
Program 1: val s1 = modifyEvent ( theEvent ( called (" team sync call ") and queryAt ( today )) , lastsFor (2. hours ))
Step 2: create an event with the same title as s1 10 days from now
Program 2: val s2 = createEvent ( createAt ( today + 10. days ) and called (" team sync call "))
<EOS >
Utterance : Check the upcoming meetings to find out any conflicts with vacation next week
Step 1: Find all meetings next week
Program 1: val s1 = findEvents ( queryAt ( next [ Interval [ Date ]] and isWeek ))
Step 2: Find vacation next week

Summary of paper#

Motivation#

Contribution#

Some key terms#

Good things about the paper (one paragraph)#

Example prompt#

Summary of paper

Motivation

Contribution

Some key terms

Good things about the paper (one paragraph)

Example prompt