Added some extra readme and new folder structures
This commit is contained in:
@@ -63,6 +63,11 @@ sudo apt-get update
|
||||
sudo apt-get install bellsoft-java8-runtime-full
|
||||
```
|
||||
|
||||
Also download the following distsim tagger file:
|
||||
```
|
||||
https://github.com/amiangshu/SentiSE/blob/master/edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger
|
||||
```
|
||||
And see the following file for needed modifications to paths: `/LLPTE/rimay_verification.py`
|
||||
Also, there is no error when the directory of preproccesing tooling is not found, be aware. And use full paths for everything.
|
||||
|
||||
|
||||
Also, there is no error when the directory of preproccesing tooling is not found, be aware. And use full paths for everything.
|
||||
I precompiled the jar files for Paska and DSL Rimay.
|
||||
+1
-1
@@ -63,7 +63,7 @@ class GherkinData(ProcessTextFile, LLM_prompt_data):
|
||||
self.gherkin_list = self.load()
|
||||
|
||||
def load(self):
|
||||
return super()._load_xml_file("input_dataset.xml")
|
||||
return super()._load_xml_file("input_dataset/input_dataset.xml")
|
||||
|
||||
#save not implemented
|
||||
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
# Dataset Gherkin
|
||||
|
||||
The dataset in `input_dataset.xml` is custom made and derrived from the `Open_Source_Projects_GitHub_US_AC_Analysis.xlsx` file
|
||||
|
||||
There is an original tag and an simplified tag in the dataset xml file
|
||||
|
||||
The simplified tag is used to make it easer for the LLM to translate Gherkin into Rimay. I made some adjustments to the sentence.
|
||||
@@ -0,0 +1,142 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Chain-of-thought
|
||||
* Datetime: 03_19_2024__22:53:14
|
||||
* UniqueID: 5ee2bdf3-bf47-45be-a60a-27490ca410f1
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Chain of Thought
|
||||
|
||||
|
||||
Learn from the following example, the output information for the CNL Rimay is:
|
||||
This part is the start of the example you have to learn from:
|
||||
|
||||
The input information is as follows:
|
||||
|
||||
Acceptance Criteria 1 in Gherkin, input:
|
||||
Gherkin:
|
||||
Scenario: Create an Order
|
||||
Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer
|
||||
When OI Create Order,
|
||||
Then Order exists in OI
|
||||
And the property settlement_method of Order is equal to FOP
|
||||
|
||||
|
||||
Rimay output 1:
|
||||
When the Order_Issuer (OI ) executes Create a Subscription_Order, if the Order does not exist, then the Order is created and its settlement_method is set to FOP.
|
||||
|
||||
Rimay uses the following structure:
|
||||
When the? Actor <Action> (every "Frequency")? ,|then the? Actor must <Action> (every "Text")?.
|
||||
|
||||
Ending for example
|
||||
|
||||
Translate the following input acceptance criteria Gherkin, according to the example above, to Rimay:
|
||||
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
Given a verification email has already been sent to "valkyrie@cain.com", when the user tries to verify their account using the link from this email, then the user should be notified that the verification was successful.
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:no viable alternative at input 'a' (line : 3 column : 7)
|
||||
ERROR:missing '.' at 'verification' (line : 3 column : 9)
|
||||
ERROR:no viable alternative at input 'email' (line : 3 column : 22)
|
||||
ERROR:missing '.' at 'already' (line : 3 column : 32)
|
||||
ERROR:no viable alternative at input 'been' (line : 3 column : 40)
|
||||
ERROR:missing '.' at 'sent' (line : 3 column : 45)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 50)
|
||||
ERROR:no viable alternative at input 'when' (line : 3 column : 74)
|
||||
ERROR:no viable alternative at input 'tries' (line : 3 column : 88)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 94)
|
||||
ERROR:no viable alternative at input 'verify' (line : 3 column : 97)
|
||||
ERROR:no viable alternative at input 'their' (line : 3 column : 104)
|
||||
ERROR:missing '.' at 'account' (line : 3 column : 110)
|
||||
ERROR:no viable alternative at input 'using' (line : 3 column : 118)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 124)
|
||||
ERROR:mismatched input 'link' expecting RULE_ID (line : 3 column : 128)
|
||||
ERROR:missing '.' at 'this' (line : 3 column : 138)
|
||||
ERROR:no viable alternative at input 'email' (line : 3 column : 143)
|
||||
ERROR:missing '.' at 'the' (line : 3 column : 155)
|
||||
ERROR:no viable alternative at input 'should' (line : 3 column : 164)
|
||||
ERROR:missing '.' at 'be' (line : 3 column : 171)
|
||||
ERROR:no viable alternative at input 'notified' (line : 3 column : 174)
|
||||
ERROR:missing '.' at 'the' (line : 3 column : 188)
|
||||
ERROR:no viable alternative at input 'was' (line : 3 column : 205)
|
||||
ERROR:extraneous input 'successful' expecting '.' (line : 3 column : 209)
|
||||
ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 9)
|
||||
ERROR:Couldn't resolve reference to Actor 'already'. (line : 3 column : 32)
|
||||
ERROR:Couldn't resolve reference to Actor 'sent'. (line : 3 column : 45)
|
||||
ERROR:Couldn't resolve reference to Actor 'user'. (line : 3 column : 83)
|
||||
ERROR:Couldn't resolve reference to Actor 'tries'. (line : 3 column : 88)
|
||||
ERROR:Couldn't resolve reference to Actor 'verify'. (line : 3 column : 97)
|
||||
ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 110)
|
||||
ERROR:Couldn't resolve reference to Actor 'this'. (line : 3 column : 138)
|
||||
ERROR:Couldn't resolve reference to Actor 'user'. (line : 3 column : 159)
|
||||
ERROR:Couldn't resolve reference to Actor 'be'. (line : 3 column : 171)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 192)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 57
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 1
|
||||
Not precise verb: 0
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 96
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 76.5
|
||||
@@ -0,0 +1,142 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Chain-of-thought
|
||||
* Datetime: 03_19_2024__22:57:31
|
||||
* UniqueID: d6283b96-ded7-4bbd-a9c9-d6f25fc23b06
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Chain of Thought
|
||||
|
||||
|
||||
Learn from the following example, the output information for the CNL Rimay is:
|
||||
This part is the start of the example you have to learn from:
|
||||
|
||||
The input information is as follows:
|
||||
|
||||
Acceptance Criteria 1 in Gherkin, input:
|
||||
Gherkin:
|
||||
Scenario: Create an Order
|
||||
Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer
|
||||
When OI Create Order,
|
||||
Then Order exists in OI
|
||||
And the property settlement_method of Order is equal to FOP
|
||||
|
||||
|
||||
Rimay output 1:
|
||||
When the Order_Issuer (OI ) executes Create a Subscription_Order, if the Order does not exist, then the Order is created and its settlement_method is set to FOP.
|
||||
|
||||
Rimay uses the following structure:
|
||||
When the? Actor <Action> (every "Frequency")? ,|then the? Actor must <Action> (every "Text")?.
|
||||
|
||||
Ending for example
|
||||
|
||||
Translate the following input acceptance criteria Gherkin, according to the example above, to Rimay:
|
||||
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
Given a verification email has already been sent to "valkyrie@cain.com", when the user tries to verify their account using the link from this email, then the user should be notified that the verification was successful.
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:no viable alternative at input 'a' (line : 3 column : 7)
|
||||
ERROR:missing '.' at 'verification' (line : 3 column : 9)
|
||||
ERROR:no viable alternative at input 'email' (line : 3 column : 22)
|
||||
ERROR:missing '.' at 'already' (line : 3 column : 32)
|
||||
ERROR:no viable alternative at input 'been' (line : 3 column : 40)
|
||||
ERROR:missing '.' at 'sent' (line : 3 column : 45)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 50)
|
||||
ERROR:no viable alternative at input 'when' (line : 3 column : 74)
|
||||
ERROR:no viable alternative at input 'tries' (line : 3 column : 88)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 94)
|
||||
ERROR:no viable alternative at input 'verify' (line : 3 column : 97)
|
||||
ERROR:no viable alternative at input 'their' (line : 3 column : 104)
|
||||
ERROR:missing '.' at 'account' (line : 3 column : 110)
|
||||
ERROR:no viable alternative at input 'using' (line : 3 column : 118)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 124)
|
||||
ERROR:mismatched input 'link' expecting RULE_ID (line : 3 column : 128)
|
||||
ERROR:missing '.' at 'this' (line : 3 column : 138)
|
||||
ERROR:no viable alternative at input 'email' (line : 3 column : 143)
|
||||
ERROR:missing '.' at 'the' (line : 3 column : 155)
|
||||
ERROR:no viable alternative at input 'should' (line : 3 column : 164)
|
||||
ERROR:missing '.' at 'be' (line : 3 column : 171)
|
||||
ERROR:no viable alternative at input 'notified' (line : 3 column : 174)
|
||||
ERROR:missing '.' at 'the' (line : 3 column : 188)
|
||||
ERROR:no viable alternative at input 'was' (line : 3 column : 205)
|
||||
ERROR:extraneous input 'successful' expecting '.' (line : 3 column : 209)
|
||||
ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 9)
|
||||
ERROR:Couldn't resolve reference to Actor 'already'. (line : 3 column : 32)
|
||||
ERROR:Couldn't resolve reference to Actor 'sent'. (line : 3 column : 45)
|
||||
ERROR:Couldn't resolve reference to Actor 'user'. (line : 3 column : 83)
|
||||
ERROR:Couldn't resolve reference to Actor 'tries'. (line : 3 column : 88)
|
||||
ERROR:Couldn't resolve reference to Actor 'verify'. (line : 3 column : 97)
|
||||
ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 110)
|
||||
ERROR:Couldn't resolve reference to Actor 'this'. (line : 3 column : 138)
|
||||
ERROR:Couldn't resolve reference to Actor 'user'. (line : 3 column : 159)
|
||||
ERROR:Couldn't resolve reference to Actor 'be'. (line : 3 column : 171)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 192)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 57
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 1
|
||||
Not precise verb: 0
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 96
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 76.5
|
||||
@@ -0,0 +1,125 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Chain-of-thought
|
||||
* Datetime: 03_19_2024__22:55:23
|
||||
* UniqueID: 3b9d1772-f5d3-4d15-8d53-a2b47e43f0ef
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: I am editing the address of "Fletcher Ren"
|
||||
When: I remove the street
|
||||
Then: I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Chain of Thought
|
||||
|
||||
|
||||
Learn from the following example, the output information for the CNL Rimay is:
|
||||
This part is the start of the example you have to learn from:
|
||||
|
||||
The input information is as follows:
|
||||
|
||||
Acceptance Criteria 1 in Gherkin, input:
|
||||
Gherkin:
|
||||
Scenario: Create an Order
|
||||
Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer
|
||||
When OI Create Order,
|
||||
Then Order exists in OI
|
||||
And the property settlement_method of Order is equal to FOP
|
||||
|
||||
|
||||
Rimay output 1:
|
||||
When the Order_Issuer (OI ) executes Create a Subscription_Order, if the Order does not exist, then the Order is created and its settlement_method is set to FOP.
|
||||
|
||||
Rimay uses the following structure:
|
||||
When the? Actor <Action> (every "Frequency")? ,|then the? Actor must <Action> (every "Text")?.
|
||||
|
||||
Ending for example
|
||||
|
||||
Translate the following input acceptance criteria Gherkin, according to the example above, to Rimay:
|
||||
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: I am editing the address of "Fletcher Ren"
|
||||
When: I remove the street
|
||||
Then: I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
Given: When editing the address of "Fletcher Ren", if the street is removed, then I should remain on the "Fletcher Ren" address edit page.
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:no viable alternative at input ':' (line : 3 column : 6)
|
||||
ERROR:no viable alternative at input 'When' (line : 3 column : 8)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 21)
|
||||
ERROR:no viable alternative at input 'of' (line : 3 column : 33)
|
||||
ERROR:no viable alternative at input 'if' (line : 3 column : 52)
|
||||
ERROR:no viable alternative at input ',' (line : 3 column : 76)
|
||||
ERROR:no viable alternative at input 'should' (line : 3 column : 85)
|
||||
ERROR:missing '.' at 'remain' (line : 3 column : 92)
|
||||
ERROR:no viable alternative at input 'on' (line : 3 column : 99)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 102)
|
||||
ERROR:extraneous input '"Fletcher Ren"' expecting RULE_ID (line : 3 column : 106)
|
||||
ERROR:no viable alternative at input 'edit' (line : 3 column : 129)
|
||||
ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
|
||||
ERROR:Couldn't resolve reference to Actor 'editing'. (line : 3 column : 13)
|
||||
ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 25)
|
||||
ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'street'. (line : 3 column : 59)
|
||||
ERROR:Couldn't resolve reference to Quantifier 'removed'. (line : 3 column : 69)
|
||||
ERROR:Couldn't resolve reference to Actor 'I'. (line : 3 column : 83)
|
||||
ERROR:Couldn't resolve reference to Actor 'remain'. (line : 3 column : 92)
|
||||
ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 121)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 74
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 2
|
||||
Incomplete system response: 0
|
||||
Passive voice: 1
|
||||
Not precise verb: 0
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 92
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 83.0
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Few-shot learning
|
||||
* Datetime: 03_19_2024__22:56:48
|
||||
* UniqueID: 09643107-0bc8-43c8-9a9c-2d60d5d3df0e
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Few-Shot Learning
|
||||
|
||||
|
||||
### Context
|
||||
The following syntax structure is called Rimay, this is a CNL (Controlled Natural Language):
|
||||
|
||||
$ACTOR is for example a thing or a person.
|
||||
$MODAL_VERB is for example the following word: must and shall.
|
||||
$TRIGGER is for example a trigger action, something what a system does.
|
||||
$WHEN_STRUCTURE is the word 'When' followed by the variable defined $TRIGGER.
|
||||
$SYSTEM_RESPONSE is something that the syst
|
||||
|
||||
$ACTOR, $MODAL_VERB and a $SYSTEM_RESPONSE is mandatory in all Rimay requirements.
|
||||
Replace the variables with the right text according the input acceptance criteria.
|
||||
In the following order:
|
||||
|
||||
Rimay CNL definition: $WHEN_STRUCTURE $ACTOR $MODAL_VERB $SYSTEM_RESPONSE
|
||||
|
||||
really important rule, make only use of the following word list:
|
||||
|
||||
This list of variables is important and can be used to formulete Rimay:
|
||||
Only make use of the following words:
|
||||
|
||||
is, are, equal, equals, greater than, less than, has, have, contain, contains, do, does, every,
|
||||
before, after, if, and, or, not, of, permit, permits, accept, acepts, allow, allows, is, are, shall, must,
|
||||
a, an, the, do, the, following, actions, exclude, excludes, instruct, instructs,
|
||||
allow, allows, authorize, authorizes, request, requests,
|
||||
start, starts, begin, begins, cancel, cancels, hide, hides,
|
||||
restore, restores, compute, computes, publish, publishes, enable, disable,
|
||||
enforce, enforces, create, creates, generate, generates, replace, replaces,
|
||||
prevent, prevents, download, downloads, aggregate, aggregates,
|
||||
interrupt, interrupts, include, includes, store, stores, property, properties, value, values,
|
||||
limit, limits, restrict, restricts, reduce, reduces, migrate, migrates,
|
||||
add, adds, link, links, neglect, neglects, ignore, ignores, accept, accepts, receive, receives, retrieve, retrieves,
|
||||
close, closes, reverse, reverses, insert, inserts, display, displays, show, shows,
|
||||
extract, extracts, remove, removes, delete, deletes, deduct, deducts, report, reports, propose, proposes,
|
||||
detect, detects, select, selects, unselect, unselects, return, returns, send, sends, forward, forward, pass, passes, export, exports,
|
||||
concatenate, concatenates, synchronize, synchronizes, discard, discards, copy, copies,
|
||||
convert, converts, change, changes, transform, transforms, update, updates, set, sets,
|
||||
use, uses, apply, applies, validate, validates, check, checks, calculate, calculates, recalculate, recalculates
|
||||
establish, establishes, search, searches, split, splits, stop, stops, finish, finishes
|
||||
subscribe, subscribes, upload, uploads
|
||||
|
||||
|
||||
Important extra rule: if the word is not defined in the list above, put if between quotes.
|
||||
DO NOT WRITE ANY OTHER WORDS THAN THE DEFINED WORD LIST!
|
||||
|
||||
The following example is a valid Rimay CNL:
|
||||
|
||||
When SystemB receives an "email alert" from SystemA, SystemB must send an Instruction to SystemC.
|
||||
|
||||
With the usage of the following extra information:
|
||||
Actors: actor SystemA, actor SystemB, actor SystemC, actor UserX, actor UserA, actor UserC
|
||||
Classes: class Instruction := description record, class Y
|
||||
|
||||
THIS IS NOT VALID RIMAY:
|
||||
When UserA navigates to the login page, UserA clicks on button, SystemA must refresh the page.
|
||||
|
||||
The following example would be valid:
|
||||
When UserA "navigates to the login page", UserA "clicks on button", SystemA must "refresh the page".
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
When User "I" tries to verify their account using the link from the verification email sent to "valkyrie@cain.com", User "I" must be notified that the verification was successful.
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:no viable alternative at input 'tries' (line : 3 column : 15)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 21)
|
||||
ERROR:no viable alternative at input 'verify' (line : 3 column : 24)
|
||||
ERROR:no viable alternative at input 'their' (line : 3 column : 31)
|
||||
ERROR:missing '.' at 'account' (line : 3 column : 37)
|
||||
ERROR:no viable alternative at input 'using' (line : 3 column : 45)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 51)
|
||||
ERROR:mismatched input 'link' expecting RULE_ID (line : 3 column : 55)
|
||||
ERROR:missing '.' at 'the' (line : 3 column : 65)
|
||||
ERROR:no viable alternative at input 'email' (line : 3 column : 82)
|
||||
ERROR:missing '.' at 'sent' (line : 3 column : 88)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 93)
|
||||
ERROR:no viable alternative at input 'User' (line : 3 column : 117)
|
||||
ERROR:no viable alternative at input '"I"' (line : 3 column : 122)
|
||||
ERROR:no viable alternative at input 'be' (line : 3 column : 131)
|
||||
ERROR:no viable alternative at input 'notified' (line : 3 column : 134)
|
||||
ERROR:missing '.' at 'the' (line : 3 column : 148)
|
||||
ERROR:no viable alternative at input 'was' (line : 3 column : 165)
|
||||
ERROR:extraneous input 'successful' expecting '.' (line : 3 column : 169)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 6)
|
||||
ERROR:Couldn't resolve reference to Actor 'tries'. (line : 3 column : 15)
|
||||
ERROR:Couldn't resolve reference to Actor 'verify'. (line : 3 column : 24)
|
||||
ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 37)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 69)
|
||||
ERROR:Couldn't resolve reference to Actor 'sent'. (line : 3 column : 88)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 117)
|
||||
ERROR:Couldn't resolve reference to Actor 'be'. (line : 3 column : 131)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 152)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 66
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 1
|
||||
Not precise verb: 0
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 96
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 81.0
|
||||
+162
@@ -0,0 +1,162 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Few-shot learning
|
||||
* Datetime: 03_19_2024__22:54:40
|
||||
* UniqueID: 083e8c75-643c-4688-a16a-0dd018d10e41
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: I am editing the address of "Fletcher Ren"
|
||||
When: I remove the street
|
||||
Then: I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Few-Shot Learning
|
||||
|
||||
|
||||
### Context
|
||||
The following syntax structure is called Rimay, this is a CNL (Controlled Natural Language):
|
||||
|
||||
$ACTOR is for example a thing or a person.
|
||||
$MODAL_VERB is for example the following word: must and shall.
|
||||
$TRIGGER is for example a trigger action, something what a system does.
|
||||
$WHEN_STRUCTURE is the word 'When' followed by the variable defined $TRIGGER.
|
||||
$SYSTEM_RESPONSE is something that the syst
|
||||
|
||||
$ACTOR, $MODAL_VERB and a $SYSTEM_RESPONSE is mandatory in all Rimay requirements.
|
||||
Replace the variables with the right text according the input acceptance criteria.
|
||||
In the following order:
|
||||
|
||||
Rimay CNL definition: $WHEN_STRUCTURE $ACTOR $MODAL_VERB $SYSTEM_RESPONSE
|
||||
|
||||
really important rule, make only use of the following word list:
|
||||
|
||||
This list of variables is important and can be used to formulete Rimay:
|
||||
Only make use of the following words:
|
||||
|
||||
is, are, equal, equals, greater than, less than, has, have, contain, contains, do, does, every,
|
||||
before, after, if, and, or, not, of, permit, permits, accept, acepts, allow, allows, is, are, shall, must,
|
||||
a, an, the, do, the, following, actions, exclude, excludes, instruct, instructs,
|
||||
allow, allows, authorize, authorizes, request, requests,
|
||||
start, starts, begin, begins, cancel, cancels, hide, hides,
|
||||
restore, restores, compute, computes, publish, publishes, enable, disable,
|
||||
enforce, enforces, create, creates, generate, generates, replace, replaces,
|
||||
prevent, prevents, download, downloads, aggregate, aggregates,
|
||||
interrupt, interrupts, include, includes, store, stores, property, properties, value, values,
|
||||
limit, limits, restrict, restricts, reduce, reduces, migrate, migrates,
|
||||
add, adds, link, links, neglect, neglects, ignore, ignores, accept, accepts, receive, receives, retrieve, retrieves,
|
||||
close, closes, reverse, reverses, insert, inserts, display, displays, show, shows,
|
||||
extract, extracts, remove, removes, delete, deletes, deduct, deducts, report, reports, propose, proposes,
|
||||
detect, detects, select, selects, unselect, unselects, return, returns, send, sends, forward, forward, pass, passes, export, exports,
|
||||
concatenate, concatenates, synchronize, synchronizes, discard, discards, copy, copies,
|
||||
convert, converts, change, changes, transform, transforms, update, updates, set, sets,
|
||||
use, uses, apply, applies, validate, validates, check, checks, calculate, calculates, recalculate, recalculates
|
||||
establish, establishes, search, searches, split, splits, stop, stops, finish, finishes
|
||||
subscribe, subscribes, upload, uploads
|
||||
|
||||
|
||||
Important extra rule: if the word is not defined in the list above, put if between quotes.
|
||||
DO NOT WRITE ANY OTHER WORDS THAN THE DEFINED WORD LIST!
|
||||
|
||||
The following example is a valid Rimay CNL:
|
||||
|
||||
When SystemB receives an "email alert" from SystemA, SystemB must send an Instruction to SystemC.
|
||||
|
||||
With the usage of the following extra information:
|
||||
Actors: actor SystemA, actor SystemB, actor SystemC, actor UserX, actor UserA, actor UserC
|
||||
Classes: class Instruction := description record, class Y
|
||||
|
||||
THIS IS NOT VALID RIMAY:
|
||||
When UserA navigates to the login page, UserA clicks on button, SystemA must refresh the page.
|
||||
|
||||
The following example would be valid:
|
||||
When UserA "navigates to the login page", UserA "clicks on button", SystemA must "refresh the page".
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: I am editing the address of "Fletcher Ren"
|
||||
When: I remove the street
|
||||
Then: I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
When User "I" edits the address of "Fletcher Ren", User "I" removes the street, User "I" should still be on the "Fletcher Ren" address edit page.
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:no viable alternative at input 'edits' (line : 3 column : 15)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 21)
|
||||
ERROR:missing '.' at 'address' (line : 3 column : 25)
|
||||
ERROR:no viable alternative at input 'of' (line : 3 column : 33)
|
||||
ERROR:no viable alternative at input 'User' (line : 3 column : 52)
|
||||
ERROR:no viable alternative at input '"I"' (line : 3 column : 57)
|
||||
ERROR:mismatched input 'User' expecting 'or' (line : 3 column : 81)
|
||||
ERROR:no viable alternative at input '"I"' (line : 3 column : 86)
|
||||
ERROR:no viable alternative at input 'should' (line : 3 column : 90)
|
||||
ERROR:no viable alternative at input 'still' (line : 3 column : 97)
|
||||
ERROR:missing '.' at 'be' (line : 3 column : 103)
|
||||
ERROR:no viable alternative at input 'on' (line : 3 column : 106)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 109)
|
||||
ERROR:extraneous input '"Fletcher Ren"' expecting RULE_ID (line : 3 column : 113)
|
||||
ERROR:no viable alternative at input 'edit' (line : 3 column : 136)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 6)
|
||||
ERROR:Couldn't resolve reference to Actor 'edits'. (line : 3 column : 15)
|
||||
ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 25)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 52)
|
||||
ERROR:Couldn't resolve reference to PropertyOrLabel 'street'. (line : 3 column : 73)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 81)
|
||||
ERROR:Couldn't resolve reference to Actor 'should'. (line : 3 column : 90)
|
||||
ERROR:Couldn't resolve reference to Actor 'be'. (line : 3 column : 103)
|
||||
ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 128)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 70
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 0
|
||||
Not precise verb: 0
|
||||
Number Req with smells: 0
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 100
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 85.0
|
||||
+161
@@ -0,0 +1,161 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Few-shot learning
|
||||
* Datetime: 03_19_2024__22:52:17
|
||||
* UniqueID: dc906f92-4048-431b-9723-5ac5ee37c5a3
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Few-Shot Learning
|
||||
|
||||
|
||||
### Context
|
||||
The following syntax structure is called Rimay, this is a CNL (Controlled Natural Language):
|
||||
|
||||
$ACTOR is for example a thing or a person.
|
||||
$MODAL_VERB is for example the following word: must and shall.
|
||||
$TRIGGER is for example a trigger action, something what a system does.
|
||||
$WHEN_STRUCTURE is the word 'When' followed by the variable defined $TRIGGER.
|
||||
$SYSTEM_RESPONSE is something that the syst
|
||||
|
||||
$ACTOR, $MODAL_VERB and a $SYSTEM_RESPONSE is mandatory in all Rimay requirements.
|
||||
Replace the variables with the right text according the input acceptance criteria.
|
||||
In the following order:
|
||||
|
||||
Rimay CNL definition: $WHEN_STRUCTURE $ACTOR $MODAL_VERB $SYSTEM_RESPONSE
|
||||
|
||||
really important rule, make only use of the following word list:
|
||||
|
||||
This list of variables is important and can be used to formulete Rimay:
|
||||
Only make use of the following words:
|
||||
|
||||
is, are, equal, equals, greater than, less than, has, have, contain, contains, do, does, every,
|
||||
before, after, if, and, or, not, of, permit, permits, accept, acepts, allow, allows, is, are, shall, must,
|
||||
a, an, the, do, the, following, actions, exclude, excludes, instruct, instructs,
|
||||
allow, allows, authorize, authorizes, request, requests,
|
||||
start, starts, begin, begins, cancel, cancels, hide, hides,
|
||||
restore, restores, compute, computes, publish, publishes, enable, disable,
|
||||
enforce, enforces, create, creates, generate, generates, replace, replaces,
|
||||
prevent, prevents, download, downloads, aggregate, aggregates,
|
||||
interrupt, interrupts, include, includes, store, stores, property, properties, value, values,
|
||||
limit, limits, restrict, restricts, reduce, reduces, migrate, migrates,
|
||||
add, adds, link, links, neglect, neglects, ignore, ignores, accept, accepts, receive, receives, retrieve, retrieves,
|
||||
close, closes, reverse, reverses, insert, inserts, display, displays, show, shows,
|
||||
extract, extracts, remove, removes, delete, deletes, deduct, deducts, report, reports, propose, proposes,
|
||||
detect, detects, select, selects, unselect, unselects, return, returns, send, sends, forward, forward, pass, passes, export, exports,
|
||||
concatenate, concatenates, synchronize, synchronizes, discard, discards, copy, copies,
|
||||
convert, converts, change, changes, transform, transforms, update, updates, set, sets,
|
||||
use, uses, apply, applies, validate, validates, check, checks, calculate, calculates, recalculate, recalculates
|
||||
establish, establishes, search, searches, split, splits, stop, stops, finish, finishes
|
||||
subscribe, subscribes, upload, uploads
|
||||
|
||||
|
||||
Important extra rule: if the word is not defined in the list above, put if between quotes.
|
||||
DO NOT WRITE ANY OTHER WORDS THAN THE DEFINED WORD LIST!
|
||||
|
||||
The following example is a valid Rimay CNL:
|
||||
|
||||
When SystemB receives an "email alert" from SystemA, SystemB must send an Instruction to SystemC.
|
||||
|
||||
With the usage of the following extra information:
|
||||
Actors: actor SystemA, actor SystemB, actor SystemC, actor UserX, actor UserA, actor UserC
|
||||
Classes: class Instruction := description record, class Y
|
||||
|
||||
THIS IS NOT VALID RIMAY:
|
||||
When UserA navigates to the login page, UserA clicks on button, SystemA must refresh the page.
|
||||
|
||||
The following example would be valid:
|
||||
When UserA "navigates to the login page", UserA "clicks on button", SystemA must "refresh the page".
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
When User "valkyrie@cain.com" receives a "verification email", User must try to "verify account" using the link from this email. System should notify User that the verification was successful.
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:no viable alternative at input 'receives' (line : 3 column : 31)
|
||||
ERROR:missing ',' at 'a' (line : 3 column : 40)
|
||||
ERROR:mismatched input '"verification email"' expecting RULE_ID (line : 3 column : 42)
|
||||
ERROR:missing '.' at 'User' (line : 3 column : 64)
|
||||
ERROR:no viable alternative at input 'try' (line : 3 column : 74)
|
||||
ERROR:no viable alternative at input 'to' (line : 3 column : 78)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 104)
|
||||
ERROR:mismatched input 'link' expecting RULE_ID (line : 3 column : 108)
|
||||
ERROR:missing '.' at 'this' (line : 3 column : 118)
|
||||
ERROR:no viable alternative at input 'email' (line : 3 column : 123)
|
||||
ERROR:no viable alternative at input 'should' (line : 3 column : 137)
|
||||
ERROR:missing '.' at 'User' (line : 3 column : 151)
|
||||
ERROR:no viable alternative at input 'that' (line : 3 column : 156)
|
||||
ERROR:no viable alternative at input 'the' (line : 3 column : 161)
|
||||
ERROR:no viable alternative at input 'was' (line : 3 column : 178)
|
||||
ERROR:extraneous input 'successful' expecting '.' (line : 3 column : 182)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 6)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 64)
|
||||
ERROR:Couldn't resolve reference to Actor 'try'. (line : 3 column : 74)
|
||||
ERROR:Couldn't resolve reference to Actor 'this'. (line : 3 column : 118)
|
||||
ERROR:Couldn't resolve reference to Actor 'System'. (line : 3 column : 130)
|
||||
ERROR:Couldn't resolve reference to Actor 'User'. (line : 3 column : 151)
|
||||
ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 165)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 71
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 0
|
||||
Not precise verb: 0
|
||||
Number Req with smells: 0
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 100
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 85.5
|
||||
@@ -0,0 +1,10 @@
|
||||
# Log directory
|
||||
|
||||
This directory contains all the logging with input data, output data and prompt send to the Large-Language-Model ChatGPT.
|
||||
|
||||
Also the Paska output and DSL Rimay output is logged.
|
||||
|
||||
|
||||
The online progressbar api is used to render a visible progressbar.
|
||||
|
||||
## Todo add score explanation.
|
||||
@@ -0,0 +1,107 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Role play
|
||||
* Datetime: 03_19_2024__22:56:06
|
||||
* UniqueID: 6e325bd9-501c-4d15-bfde-c73c8e3ecd91
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: I am editing the address of "Fletcher Ren"
|
||||
When: I remove the street
|
||||
Then: I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Role Play
|
||||
|
||||
|
||||
Behave like a person who writes the Controlled Natural Language (CNL) Rimay.
|
||||
Rimay is a language that is defined in different components.
|
||||
For now we only highlight the following components:
|
||||
|
||||
Mandatory:
|
||||
|
||||
$ACTOR = SystemA
|
||||
$MODAL_VERB = should, can, could, might, will
|
||||
$SYSTEM_RESPONSE = SYSTEM_RESPONSE_EXPRESSION | RESPONSE_BLOCK_ITEMIZED
|
||||
$SYSTEM_RESPONSE_EXPRESSION: ATOMIC_SYSTEM_RESPONSE ((,|, and|, or|and|or)? ATOMIC_SYSTEM_RESPONSE)* ATOMIC_SYSTEM_RESPONSE: ACTION_PHRASE (every TEXT )? RESPONSE_BLOCK_ITEMIZED: do the following actions (in sequence)? : BULLET ATOMIC_SYSTEM_RESPONSE ((,|, and|, or|and|or)? BULLET ATOMIC_SYSTEM_RESPONSE)*
|
||||
Optional:
|
||||
$WHEN_STRUCTURE : When TRIGGER
|
||||
|
||||
Use for ACTOR for example SystemA and use for MODAL_VERB for example the following words: should, can, could, might, will
|
||||
|
||||
|
||||
$ACTOR, $MODAL_VERB and $SYSTEM_RESPONSE is mandatory in all Rimay requirements.
|
||||
|
||||
Rimay CNL: $WHEN_STRUCTURE $ACTOR $MODAL_VERB $SYSTEM_RESPONSE
|
||||
|
||||
Now I want you to answer as the person who writes Rimay to translate this acceptance criteria into Rimay
|
||||
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: I am editing the address of "Fletcher Ren"
|
||||
When: I remove the street
|
||||
Then: I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
$ACTOR SystemA $MODAL_VERB should $SYSTEM_RESPONSE do the following actions in sequence:
|
||||
- I am editing the address of "Fletcher Ren"
|
||||
- I remove the street
|
||||
- I should still be on the "Fletcher Ren" address edit page
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:missing EOF at '$' (line : 3 column : 1)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 63
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 1
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 0
|
||||
Not precise verb: 1
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 94
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 78.5
|
||||
@@ -0,0 +1,110 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Role play
|
||||
* Datetime: 03_19_2024__22:53:57
|
||||
* UniqueID: 71f168d7-5242-4c4c-a828-6cd3b74b3d0b
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Role Play
|
||||
|
||||
|
||||
Behave like a person who writes the Controlled Natural Language (CNL) Rimay.
|
||||
Rimay is a language that is defined in different components.
|
||||
For now we only highlight the following components:
|
||||
|
||||
Mandatory:
|
||||
|
||||
$ACTOR = SystemA
|
||||
$MODAL_VERB = should, can, could, might, will
|
||||
$SYSTEM_RESPONSE = SYSTEM_RESPONSE_EXPRESSION | RESPONSE_BLOCK_ITEMIZED
|
||||
$SYSTEM_RESPONSE_EXPRESSION: ATOMIC_SYSTEM_RESPONSE ((,|, and|, or|and|or)? ATOMIC_SYSTEM_RESPONSE)* ATOMIC_SYSTEM_RESPONSE: ACTION_PHRASE (every TEXT )? RESPONSE_BLOCK_ITEMIZED: do the following actions (in sequence)? : BULLET ATOMIC_SYSTEM_RESPONSE ((,|, and|, or|and|or)? BULLET ATOMIC_SYSTEM_RESPONSE)*
|
||||
Optional:
|
||||
$WHEN_STRUCTURE : When TRIGGER
|
||||
|
||||
Use for ACTOR for example SystemA and use for MODAL_VERB for example the following words: should, can, could, might, will
|
||||
|
||||
|
||||
$ACTOR, $MODAL_VERB and $SYSTEM_RESPONSE is mandatory in all Rimay requirements.
|
||||
|
||||
Rimay CNL: $WHEN_STRUCTURE $ACTOR $MODAL_VERB $SYSTEM_RESPONSE
|
||||
|
||||
Now I want you to answer as the person who writes Rimay to translate this acceptance criteria into Rimay
|
||||
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
$ACTOR = SystemA
|
||||
$MODAL_VERB = should
|
||||
|
||||
$ACTOR should $SYSTEM_RESPONSE do the following actions in sequence:
|
||||
- Verify that a verification email has already been sent to "valkyrie@cain.com"
|
||||
- Try to verify the account using the link from this email
|
||||
- Notify that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:missing EOF at '$' (line : 3 column : 1)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 63
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 0
|
||||
Not precise verb: 1
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 96
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 79.5
|
||||
@@ -0,0 +1,107 @@
|
||||
|
||||
|
||||
# LLM Experiment Information
|
||||
* Prefix: Role play
|
||||
* Datetime: 03_19_2024__22:58:14
|
||||
* UniqueID: 400b7b21-d4bb-4d07-9f8e-de4468a6af15
|
||||
|
||||
|
||||
|
||||
## Gherkin Input
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
## Input prompt, technique: Role Play
|
||||
|
||||
|
||||
Behave like a person who writes the Controlled Natural Language (CNL) Rimay.
|
||||
Rimay is a language that is defined in different components.
|
||||
For now we only highlight the following components:
|
||||
|
||||
Mandatory:
|
||||
|
||||
$ACTOR = SystemA
|
||||
$MODAL_VERB = should, can, could, might, will
|
||||
$SYSTEM_RESPONSE = SYSTEM_RESPONSE_EXPRESSION | RESPONSE_BLOCK_ITEMIZED
|
||||
$SYSTEM_RESPONSE_EXPRESSION: ATOMIC_SYSTEM_RESPONSE ((,|, and|, or|and|or)? ATOMIC_SYSTEM_RESPONSE)* ATOMIC_SYSTEM_RESPONSE: ACTION_PHRASE (every TEXT )? RESPONSE_BLOCK_ITEMIZED: do the following actions (in sequence)? : BULLET ATOMIC_SYSTEM_RESPONSE ((,|, and|, or|and|or)? BULLET ATOMIC_SYSTEM_RESPONSE)*
|
||||
Optional:
|
||||
$WHEN_STRUCTURE : When TRIGGER
|
||||
|
||||
Use for ACTOR for example SystemA and use for MODAL_VERB for example the following words: should, can, could, might, will
|
||||
|
||||
|
||||
$ACTOR, $MODAL_VERB and $SYSTEM_RESPONSE is mandatory in all Rimay requirements.
|
||||
|
||||
Rimay CNL: $WHEN_STRUCTURE $ACTOR $MODAL_VERB $SYSTEM_RESPONSE
|
||||
|
||||
Now I want you to answer as the person who writes Rimay to translate this acceptance criteria into Rimay
|
||||
|
||||
|
||||
|
||||
### Input Data
|
||||
Use the following Gherkin acceptance criteria input:
|
||||
```
|
||||
Given: a verification email has already been sent to "valkyrie@cain.com"
|
||||
When: I try to verify my account using the link from this email
|
||||
Then: I should be notified that the verification was successful
|
||||
```
|
||||
|
||||
### Question:
|
||||
Can you translate the input above into the specified language Rimay CNL?
|
||||
|
||||
### Output indicator
|
||||
Output only your translated Rimay CNL text and nothing else!
|
||||
|
||||
|
||||
## Rimay Output
|
||||
```
|
||||
$ACTOR SystemA $MODAL_VERB should $SYSTEM_RESPONSE do the following actions in sequence:
|
||||
- Verify that a verification email has already been sent to "valkyrie@cain.com"
|
||||
- Try to verify the account using the link from this email
|
||||
- Notify that the verification was successful
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Rimay DSL Verification
|
||||
```
|
||||
ERROR:missing EOF at '$' (line : 3 column : 1)
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 63
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
|
||||
## Rimay Paska Verification
|
||||
```
|
||||
File generated_rimay.csv.csv
|
||||
Smells found in this document:
|
||||
Non-atomic requirement: 0
|
||||
Incomplete requirement: 0
|
||||
Incorrect order requirement: 0
|
||||
Coordination ambiguity: 0
|
||||
Not requirement: ,0
|
||||
Incomplete condition: 0
|
||||
Incomplete system response: 0
|
||||
Passive voice: 0
|
||||
Not precise verb: 1
|
||||
Number Req with smells: 1
|
||||
|
||||
```
|
||||
### Score
|
||||
Score: 96
|
||||
Progressbar: 
|
||||
|
||||
|
||||
|
||||
### Final Score
|
||||
Final_score = 79.5
|
||||
Reference in New Issue
Block a user