Chain of thought resultaten ingecheckt

cot.sh

@@ -3,7 +3,7 @@
 
 
 
-python3 main.py standalone 1 2 Chain-of-thought 1.3
+python3 main.py standalone 1 2 Chain-of-thought 0.2
 
 #
-python3 main.py results Chain-of-thought_1.3temp
+#python3 main.py results Chain-of-thought_1.3temp

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 10: Logging out the user doesn't log out my admin account_Incorrect___04_08_2024__21:34:33__85.0_score.md

@@ -0,0 +1,148 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:34:33
+* UniqueID: f5d609f6-54fc-49da-b8b7-f2ceaa69c56e
+* Gherkinscenario name: Gherkinscenario 10: Logging out the user doesn't log out my admin account_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA impersonates "remnant@london.uk" 
+then logs out from SystemA 
+system must allow UserA to access administration dashboard.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'then' (line : 4 column : 1)
+ERROR:missing ',' at 'logs' (line : 4 column : 6)
+ERROR:no viable alternative at input 'out' (line : 4 column : 11)
+ERROR:no viable alternative at input 'SystemA' (line : 4 column : 20)
+ERROR:no viable alternative at input 'system' (line : 5 column : 1)
+ERROR:missing '.' at 'UserA' (line : 5 column : 19)
+ERROR:no viable alternative at input 'to' (line : 5 column : 25)
+ERROR:no viable alternative at input 'access' (line : 5 column : 28)
+ERROR:no viable alternative at input 'administration' (line : 5 column : 35)
+ERROR:extraneous input 'dashboard' expecting '.' (line : 5 column : 50)
+ERROR:Couldn't resolve reference to Actor 'impersonates'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'logs'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'access'. (line : 5 column : 28)
+
+```
+### DSL-Rimay Score
+DSL_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 85.0
+
+Progressbar: ![85.0%](https://progress-bar.dev/85.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 10: Logging out the user doesn't log out my admin account__04_08_2024__21:33:15__100_score.md

@@ -0,0 +1,159 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:33:15
+* UniqueID: e31e49e8-0806-4814-b75d-f99c6f5da64e
+* Gherkinscenario name: Gherkinscenario 10: Logging out the user doesn't log out my admin account
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA impersonates the customer "remnant@london.uk", 
+When UserA logs out from SystemA, 
+then UserA must still have access to the administration dashboard.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 26)
+ERROR:no viable alternative at input '"remnant@london.uk"' (line : 3 column : 39)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'out' (line : 4 column : 17)
+ERROR:extraneous input 'SystemA' expecting ',' (line : 4 column : 26)
+ERROR:no viable alternative at input 'still' (line : 5 column : 17)
+ERROR:no viable alternative at input 'have' (line : 5 column : 23)
+ERROR:no viable alternative at input 'access' (line : 5 column : 28)
+ERROR:no viable alternative at input 'to' (line : 5 column : 35)
+ERROR:no viable alternative at input 'the' (line : 5 column : 38)
+ERROR:no viable alternative at input 'dashboard' (line : 5 column : 57)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'impersonates'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'customer'. (line : 3 column : 30)
+ERROR:Couldn't resolve reference to Actor 'logs'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'still'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'access'. (line : 5 column : 28)
+ERROR:Couldn't resolve reference to Actor 'administration'. (line : 5 column : 42)
+
+```
+### DSL-Rimay Score
+DSL_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 11: Seeing disabled code field while editing country_Incorrect___04_08_2024__21:36:47__78.0_score.md

@@ -0,0 +1,146 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:36:47
+* UniqueID: 316bb2c9-640d-48b3-9523-cd3f9cdd4f8c
+* Gherkinscenario name: Gherkinscenario 11: Seeing disabled code field while editing country_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA modifies the country in SystemA to "United States", 
+then the code field must be deactivated.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 21)
+ERROR:no viable alternative at input 'in' (line : 3 column : 33)
+ERROR:no viable alternative at input 'SystemA' (line : 3 column : 36)
+ERROR:no viable alternative at input 'to' (line : 3 column : 44)
+ERROR:no viable alternative at input 'the' (line : 4 column : 6)
+ERROR:no viable alternative at input 'field' (line : 4 column : 15)
+ERROR:missing '.' at 'be' (line : 4 column : 26)
+ERROR:no viable alternative at input 'deactivated' (line : 4 column : 29)
+ERROR:Couldn't resolve reference to Actor 'modifies'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 3 column : 25)
+ERROR:Couldn't resolve reference to Actor 'code'. (line : 4 column : 10)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 4 column : 26)
+
+```
+### DSL-Rimay Score
+DSL_Score: 81
+
+Progressbar: ![81%](https://progress-bar.dev/81)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 78.0
+
+Progressbar: ![78.0%](https://progress-bar.dev/78.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 11: Seeing disabled code field while editing country__04_08_2024__21:35:57__99.5_score.md

@@ -0,0 +1,160 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:35:57
+* UniqueID: 184d706b-d0f0-4806-a47d-d319b430a95f
+* Gherkinscenario name: Gherkinscenario 11: Seeing disabled code field while editing country
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While SystemA has "United States" as the country,
+When UserA intends to modify this country,
+then the code field must be deactivated.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:mismatched input 'as' expecting ',' (line : 3 column : 35)
+ERROR:missing '.' at 'the' (line : 3 column : 38)
+ERROR:no viable alternative at input ',' (line : 3 column : 49)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 20)
+ERROR:missing ',' at 'modify' (line : 4 column : 23)
+ERROR:no viable alternative at input 'this' (line : 4 column : 30)
+ERROR:missing '.' at 'country' (line : 4 column : 35)
+ERROR:no viable alternative at input ',' (line : 4 column : 42)
+ERROR:no viable alternative at input 'the' (line : 5 column : 6)
+ERROR:no viable alternative at input 'field' (line : 5 column : 15)
+ERROR:missing '.' at 'be' (line : 5 column : 26)
+ERROR:no viable alternative at input 'deactivated' (line : 5 column : 29)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 3 column : 42)
+ERROR:Couldn't resolve reference to Actor 'intends'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'modify'. (line : 4 column : 23)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Actor 'code'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 26)
+
+```
+### DSL-Rimay Score
+DSL_Score: 74
+
+Progressbar: ![74%](https://progress-bar.dev/74)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 99.5
+
+Progressbar: ![99.5%](https://progress-bar.dev/99.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 12: Changing the currency of my cart_Incorrect___04_08_2024__21:39:22__74.0_score.md

@@ -0,0 +1,150 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:39:22
+* UniqueID: 000b0038-5d7e-4ace-a9c3-0fce72f39800
+* Gherkinscenario name: Gherkinscenario 12: Changing the currency of my cart_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA has "The Pug Mug" in the cart,
+When UserA changes to "EUR" currency,
+Then the total amount should be "€6.82".
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'the' (line : 3 column : 34)
+ERROR:no viable alternative at input ',' (line : 3 column : 42)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 20)
+ERROR:no viable alternative at input 'currency' (line : 4 column : 29)
+ERROR:no viable alternative at input 'the' (line : 5 column : 6)
+ERROR:missing '.' at 'total' (line : 5 column : 10)
+ERROR:no viable alternative at input 'amount' (line : 5 column : 16)
+ERROR:missing '.' at 'should' (line : 5 column : 23)
+ERROR:no viable alternative at input 'be' (line : 5 column : 30)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 3 column : 38)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 23)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 74.0
+
+Progressbar: ![74.0%](https://progress-bar.dev/74.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 12: Changing the currency of my cart__04_08_2024__21:38:20__95.0_score.md

@@ -0,0 +1,154 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:38:20
+* UniqueID: 7f3c3a1e-7a9e-4d3d-829c-00854c32182e
+* Gherkinscenario name: Gherkinscenario 12: Changing the currency of my cart
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA has "The Pug Mug" product in the cart, 
+When UserA changes to "EUR" currency, 
+then the total value should be "€6.82".
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:missing ',' at 'product' (line : 3 column : 31)
+ERROR:no viable alternative at input 'in' (line : 3 column : 39)
+ERROR:no viable alternative at input 'the' (line : 3 column : 42)
+ERROR:no viable alternative at input ',' (line : 3 column : 50)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 20)
+ERROR:no viable alternative at input 'currency' (line : 4 column : 29)
+ERROR:no viable alternative at input 'value' (line : 5 column : 16)
+ERROR:no viable alternative at input 'should' (line : 5 column : 22)
+ERROR:no viable alternative at input 'be' (line : 5 column : 29)
+ERROR:Couldn't resolve reference to Actor 'product'. (line : 3 column : 31)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 3 column : 46)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 22)
+
+```
+### DSL-Rimay Score
+DSL_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 95.0
+
+Progressbar: ![95.0%](https://progress-bar.dev/95.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 13: Viewing information about empty cart after channel switching_Incorrect___04_08_2024__21:43:01__77.0_score.md

@@ -0,0 +1,143 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:43:01
+* UniqueID: 1262d313-31f9-4135-9aef-b56cd03291ae
+* Gherkinscenario name: Gherkinscenario 13: Viewing information about empty cart after channel switching_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA switches their current channel from "Poland" to "France", 
+then UserA's carts should be empty.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'their' (line : 3 column : 21)
+ERROR:no viable alternative at input 'current' (line : 3 column : 27)
+ERROR:missing '.' at 'channel' (line : 3 column : 35)
+ERROR:no viable alternative at input 'from' (line : 3 column : 43)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 6)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 4 column : 11)
+ERROR:Couldn't resolve reference to Actor 'switches'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'their'. (line : 3 column : 21)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 35)
+
+```
+### DSL-Rimay Score
+DSL_Score: 74
+
+Progressbar: ![74%](https://progress-bar.dev/74)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 77.0
+
+Progressbar: ![77.0%](https://progress-bar.dev/77.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 13: Viewing information about empty cart after channel switching__04_08_2024__21:40:14__99.0_score.md

@@ -0,0 +1,156 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:40:14
+* UniqueID: 0c670e01-e68d-43fd-a1a2-de82bb3c6378
+* Gherkinscenario name: Gherkinscenario 13: Viewing information about empty cart after channel switching
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA simplifies his current channel to "Poland", 
+When UserA switches the current channel to "France", 
+then UserA's carts should be empty.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'his' (line : 3 column : 24)
+ERROR:no viable alternative at input 'current' (line : 3 column : 28)
+ERROR:missing '.' at 'channel' (line : 3 column : 36)
+ERROR:no viable alternative at input 'to' (line : 3 column : 44)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 21)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 33)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 5 column : 11)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'simplifies'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'his'. (line : 3 column : 24)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 36)
+ERROR:Couldn't resolve reference to Actor 'switches'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'current'. (line : 4 column : 25)
+
+```
+### DSL-Rimay Score
+DSL_Score: 68
+
+Progressbar: ![68%](https://progress-bar.dev/68)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 99.0
+
+Progressbar: ![99.0%](https://progress-bar.dev/99.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 14: Displaying default shop homepage_Incorrect___04_08_2024__21:45:39__78.5_score.md

@@ -0,0 +1,150 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:45:39
+* UniqueID: de48af71-be1a-4139-afb1-06e4e77cf192
+* Gherkinscenario name: Gherkinscenario 14: Displaying default shop homepage_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA visits the homepage of channel "United States", 
+then UserA should not see a homepage from "maverick/meerkat" theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 19)
+ERROR:no viable alternative at input 'of' (line : 3 column : 32)
+ERROR:no viable alternative at input 'channel' (line : 3 column : 35)
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 43)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'should' (line : 4 column : 12)
+ERROR:missing '.' at 'see' (line : 4 column : 23)
+ERROR:no viable alternative at input 'a' (line : 4 column : 27)
+ERROR:missing '.' at 'homepage' (line : 4 column : 29)
+ERROR:no viable alternative at input 'from' (line : 4 column : 38)
+ERROR:no viable alternative at input 'theme' (line : 4 column : 62)
+ERROR:Couldn't resolve reference to Actor 'visits'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 3 column : 23)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 35)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 4 column : 23)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 4 column : 29)
+
+```
+### DSL-Rimay Score
+DSL_Score: 77
+
+Progressbar: ![77%](https://progress-bar.dev/77)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 78.5
+
+Progressbar: ![78.5%](https://progress-bar.dev/78.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 14: Displaying default shop homepage__04_08_2024__21:44:43__94.0_score.md

@@ -0,0 +1,166 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:44:43
+* UniqueID: 2e1792db-1f40-4eee-92ee-2c8922013b7b
+* Gherkinscenario name: Gherkinscenario 14: Displaying default shop homepage
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the channel "United States" is not using any theme, 
+When UserA visits the homepage of this channel, 
+then UserA should not see a homepage from the "maverick/meerkat" theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 19)
+ERROR:missing ',' at 'any' (line : 3 column : 48)
+ERROR:no viable alternative at input 'theme' (line : 3 column : 52)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 19)
+ERROR:no viable alternative at input 'of' (line : 4 column : 32)
+ERROR:no viable alternative at input 'this' (line : 4 column : 35)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 40)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'should' (line : 5 column : 12)
+ERROR:missing '.' at 'see' (line : 5 column : 23)
+ERROR:no viable alternative at input 'a' (line : 5 column : 27)
+ERROR:missing '.' at 'homepage' (line : 5 column : 29)
+ERROR:no viable alternative at input 'from' (line : 5 column : 38)
+ERROR:no viable alternative at input 'the' (line : 5 column : 43)
+ERROR:extraneous input '"maverick/meerkat"' expecting RULE_ID (line : 5 column : 47)
+ERROR:no viable alternative at input '.' (line : 5 column : 71)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'channel'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Actor 'any'. (line : 3 column : 48)
+ERROR:Couldn't resolve reference to Actor 'visits'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 4 column : 23)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 23)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 5 column : 29)
+ERROR:Couldn't resolve reference to Actor 'theme'. (line : 5 column : 66)
+
+```
+### DSL-Rimay Score
+DSL_Score: 68
+
+Progressbar: ![68%](https://progress-bar.dev/68)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 94.0
+
+Progressbar: ![94.0%](https://progress-bar.dev/94.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 15: Displaying themed shop homepage_Incorrect___04_08_2024__21:47:51__64.0_score.md

@@ -0,0 +1,155 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:47:51
+* UniqueID: 1b6d6794-8901-4d79-b586-50949ddc48c9
+* Gherkinscenario name: Gherkinscenario 15: Displaying themed shop homepage_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given channel "United States" uses theme "maverick/meerkat"
+When UserA visits homepage of this channel
+Then UserA must see homepage with that theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'channel' (line : 3 column : 7)
+ERROR:missing '.' at 'theme' (line : 3 column : 36)
+ERROR:no viable alternative at input '"maverick/meerkat"' (line : 3 column : 42)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'homepage' (line : 4 column : 19)
+ERROR:no viable alternative at input 'of' (line : 4 column : 28)
+ERROR:no viable alternative at input 'this' (line : 4 column : 31)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 36)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'homepage' (line : 5 column : 21)
+ERROR:extraneous input 'theme' expecting '.' (line : 5 column : 40)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'theme'. (line : 3 column : 36)
+ERROR:Couldn't resolve reference to Actor 'visits'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 4 column : 19)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 31)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+
+```
+### DSL-Rimay Score
+DSL_Score: 73
+
+Progressbar: ![73%](https://progress-bar.dev/73)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+### Final Score
+Final_score = 64.0
+
+Progressbar: ![64.0%](https://progress-bar.dev/64.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 15: Displaying themed shop homepage__04_08_2024__21:46:57__73.5_score.md

@@ -0,0 +1,167 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:46:57
+* UniqueID: 4564a148-8c96-47b6-a4ee-686b393b96cd
+* Gherkinscenario name: Gherkinscenario 15: Displaying themed shop homepage
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given the channel "United States" uses "maverick/meerkat" theme,
+When UserA visits the homepage of this channel,
+Then UserA should see a homepage with that theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 7)
+ERROR:missing '.' at 'channel' (line : 3 column : 11)
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 19)
+ERROR:missing '.' at 'theme' (line : 3 column : 59)
+ERROR:no viable alternative at input ',' (line : 3 column : 64)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 19)
+ERROR:no viable alternative at input 'of' (line : 4 column : 32)
+ERROR:no viable alternative at input 'this' (line : 4 column : 35)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 40)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'should' (line : 5 column : 12)
+ERROR:no viable alternative at input 'see' (line : 5 column : 19)
+ERROR:missing '.' at 'a' (line : 5 column : 23)
+ERROR:no viable alternative at input 'with' (line : 5 column : 34)
+ERROR:no viable alternative at input 'theme' (line : 5 column : 44)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Actor 'theme'. (line : 3 column : 59)
+ERROR:Couldn't resolve reference to Actor 'visits'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 4 column : 23)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 5 column : 25)
+
+```
+### DSL-Rimay Score
+DSL_Score: 67
+
+Progressbar: ![67%](https://progress-bar.dev/67)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 73.5
+
+Progressbar: ![73.5%](https://progress-bar.dev/73.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 16: Choosing shipping address from address book_Incorrect___04_08_2024__21:50:33__73.0_score.md

@@ -0,0 +1,157 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:50:33
+* UniqueID: 2e94c9a9-908b-4bba-ba06-f843fc33424f
+* Gherkinscenario name: Gherkinscenario 16: Choosing shipping address from address book_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA has "PHP T-Shirt" in cart
+When UserA selects "Seaside Fwy" for shipping address
+Then address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'cart' (line : 3 column : 34)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'selects' (line : 4 column : 12)
+ERROR:no viable alternative at input '"Seaside Fwy"' (line : 4 column : 20)
+ERROR:missing '.' at 'shipping' (line : 4 column : 38)
+ERROR:no viable alternative at input 'address' (line : 4 column : 47)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'address' (line : 5 column : 6)
+ERROR:missing '.' at 'should' (line : 5 column : 94)
+ERROR:no viable alternative at input 'be' (line : 5 column : 101)
+ERROR:missing '.' at 'filled' (line : 5 column : 104)
+ERROR:no viable alternative at input 'as' (line : 5 column : 111)
+ERROR:no viable alternative at input 'shipping' (line : 5 column : 114)
+ERROR:no viable alternative at input 'address' (line : 5 column : 123)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 3 column : 34)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 4 column : 38)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 94)
+ERROR:Couldn't resolve reference to Actor 'filled'. (line : 5 column : 104)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 5 column : 114)
+
+```
+### DSL-Rimay Score
+DSL_Score: 71
+
+Progressbar: ![71%](https://progress-bar.dev/71)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 73.0
+
+Progressbar: ![73.0%](https://progress-bar.dev/73.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 16: Choosing shipping address from address book__04_08_2024__21:49:30__99.5_score.md

@@ -0,0 +1,160 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:49:30
+* UniqueID: 55be2f5a-80e3-4b86-8993-a2fa969e5535
+* Gherkinscenario name: Gherkinscenario 16: Choosing shipping address from address book
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA has "PHP T-Shirt" in the cart, 
+When UserA selects "Seaside Fwy" street for shipping address, 
+then the shipping address should be filled as "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas".
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:mismatched input 'in' expecting ',' (line : 3 column : 31)
+ERROR:missing '.' at 'the' (line : 3 column : 34)
+ERROR:no viable alternative at input ',' (line : 3 column : 42)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:mismatched input 'for' expecting ',' (line : 4 column : 41)
+ERROR:missing '.' at 'shipping' (line : 4 column : 45)
+ERROR:no viable alternative at input 'address' (line : 4 column : 54)
+ERROR:missing '.' at 'the' (line : 5 column : 6)
+ERROR:no viable alternative at input 'address' (line : 5 column : 19)
+ERROR:missing '.' at 'should' (line : 5 column : 27)
+ERROR:no viable alternative at input 'be' (line : 5 column : 34)
+ERROR:missing '.' at 'filled' (line : 5 column : 37)
+ERROR:no viable alternative at input 'as' (line : 5 column : 44)
+ERROR:no viable alternative at input '.' (line : 5 column : 126)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 3 column : 38)
+ERROR:Couldn't resolve reference to UIComponent 'street'. (line : 4 column : 34)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 4 column : 45)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 27)
+ERROR:Couldn't resolve reference to Actor 'filled'. (line : 5 column : 37)
+
+```
+### DSL-Rimay Score
+DSL_Score: 74
+
+Progressbar: ![74%](https://progress-bar.dev/74)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 99.5
+
+Progressbar: ![99.5%](https://progress-bar.dev/99.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 17: Seeing the unpaid order in the list_Incorrect___04_08_2024__21:52:28__61.0_score.md

@@ -0,0 +1,156 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:52:28
+* UniqueID: 6c827e1d-26a2-4a5b-b6f0-12e1fec53818
+* Gherkinscenario name: Gherkinscenario 17: Seeing the unpaid order in the list_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA is logged in as "john@example.com"
+When UserA checks order history
+Then UserA must see one order with a total of $29.99.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'logged' (line : 3 column : 16)
+ERROR:no viable alternative at input 'in' (line : 3 column : 23)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:mismatched input 'Then' expecting 'by' (line : 5 column : 1)
+ERROR:no viable alternative at input 'must' (line : 5 column : 12)
+ERROR:no viable alternative at input 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'one' (line : 5 column : 21)
+ERROR:missing '.' at 'order' (line : 5 column : 25)
+ERROR:no viable alternative at input 'with' (line : 5 column : 31)
+ERROR:no viable alternative at input 'a' (line : 5 column : 36)
+ERROR:no viable alternative at input 'of' (line : 5 column : 44)
+ERROR:no viable alternative at input '.' (line : 5 column : 50)
+ERROR:missing EOF at '99' (line : 5 column : 51)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'logged'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Quantifier 'order'. (line : 4 column : 19)
+ERROR:Couldn't resolve reference to ClassOrLabel 'history'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Quantifier 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'order'. (line : 5 column : 25)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 38)
+
+```
+### DSL-Rimay Score
+DSL_Score: 62
+
+Progressbar: ![62%](https://progress-bar.dev/62)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,1
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+### Final Score
+Final_score = 61.0
+
+Progressbar: ![61.0%](https://progress-bar.dev/61.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 17: Seeing the unpaid order in the list__04_08_2024__21:51:24__72.0_score.md

@@ -0,0 +1,160 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:51:24
+* UniqueID: acdd3fba-a40e-48d4-8641-da0d9df7b99f
+* Gherkinscenario name: Gherkinscenario 17: Seeing the unpaid order in the list
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA is logged in as "john@example.com", 
+When UserA checks the order history, 
+then UserA should see one order with a total of $29.99.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'logged' (line : 3 column : 16)
+ERROR:no viable alternative at input 'in' (line : 3 column : 23)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:mismatched input 'history' expecting 'by' (line : 4 column : 29)
+ERROR:no viable alternative at input 'should' (line : 5 column : 12)
+ERROR:missing '.' at 'see' (line : 5 column : 19)
+ERROR:no viable alternative at input 'one' (line : 5 column : 23)
+ERROR:missing '.' at 'order' (line : 5 column : 27)
+ERROR:no viable alternative at input 'with' (line : 5 column : 33)
+ERROR:no viable alternative at input 'a' (line : 5 column : 38)
+ERROR:no viable alternative at input 'of' (line : 5 column : 46)
+ERROR:no viable alternative at input '.' (line : 5 column : 52)
+ERROR:missing EOF at '99' (line : 5 column : 53)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'logged'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to ClassOrLabel 'order'. (line : 4 column : 23)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 19)
+ERROR:Couldn't resolve reference to Actor 'order'. (line : 5 column : 27)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 40)
+
+```
+### DSL-Rimay Score
+DSL_Score: 64
+
+Progressbar: ![64%](https://progress-bar.dev/64)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 72.0
+
+Progressbar: ![72.0%](https://progress-bar.dev/72.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 18: Being unable to change currencies_Incorrect___04_08_2024__21:54:52__84.0_score.md

@@ -0,0 +1,149 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:54:52
+* UniqueID: 89b3d005-3c0b-4c9a-8238-ba4b18e441c5
+* Gherkinscenario name: Gherkinscenario 18: Being unable to change currencies_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA modifies the exchange rate between "US Dollar" and "British Pound", 
+then UserA must observe that the source currency is not editable.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 21)
+ERROR:no viable alternative at input 'rate' (line : 3 column : 34)
+ERROR:missing '.' at 'between' (line : 3 column : 39)
+ERROR:no viable alternative at input '"US Dollar"' (line : 3 column : 47)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'observe' (line : 4 column : 17)
+ERROR:no viable alternative at input 'that' (line : 4 column : 25)
+ERROR:no viable alternative at input 'the' (line : 4 column : 30)
+ERROR:no viable alternative at input 'currency' (line : 4 column : 41)
+ERROR:extraneous input 'editable' expecting '.' (line : 4 column : 57)
+ERROR:Couldn't resolve reference to Actor 'modifies'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'exchange'. (line : 3 column : 25)
+ERROR:Couldn't resolve reference to Actor 'between'. (line : 3 column : 39)
+ERROR:Couldn't resolve reference to Actor 'observe'. (line : 4 column : 17)
+ERROR:Couldn't resolve reference to Actor 'source'. (line : 4 column : 34)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 84.0
+
+Progressbar: ![84.0%](https://progress-bar.dev/84.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 18: Being unable to change currencies__04_08_2024__21:53:37__100_score.md

@@ -0,0 +1,146 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:53:37
+* UniqueID: a5c27bb1-f4fe-4e4b-ae53-50bb985f1e77
+* Gherkinscenario name: Gherkinscenario 18: Being unable to change currencies
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the exchange rate of "US Dollar" to "British Pound" is 1.30,
+When UserA intends to modify this exchange rate,
+then UserA must observe that the source currency is deactivated.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'rate' (line : 3 column : 20)
+ERROR:no viable alternative at input 'of' (line : 3 column : 25)
+ERROR:no viable alternative at input '.' (line : 3 column : 63)
+ERROR:missing EOF at '30' (line : 3 column : 64)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'exchange'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Actor 'rate'. (line : 3 column : 20)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 19: Inability to impersonate a customer with no account_Incorrect___04_08_2024__21:57:10__81.0_score.md

@@ -0,0 +1,140 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:57:10
+* UniqueID: 8d34ec67-0381-4538-87c0-3cf8cc71381e
+* Gherkinscenario name: Gherkinscenario 19: Inability to impersonate a customer with no account_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA accesses the details of SystemA's customer "harold@thrasher.ie" with first name "Harold", 
+then UserA must not be able to impersonate them.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 21)
+ERROR:no viable alternative at input 'of' (line : 3 column : 33)
+ERROR:no viable alternative at input 'SystemA' (line : 3 column : 36)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 3 column : 43)
+ERROR:Couldn't resolve reference to Actor 'accesses'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'details'. (line : 3 column : 25)
+
+```
+### DSL-Rimay Score
+DSL_Score: 82
+
+Progressbar: ![82%](https://progress-bar.dev/82)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 81.0
+
+Progressbar: ![81.0%](https://progress-bar.dev/81.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 19: Inability to impersonate a customer with no account__04_08_2024__21:55:54__100_score.md

@@ -0,0 +1,157 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:55:54
+* UniqueID: 7ee42cba-bcea-482c-a932-e7e345669d49
+* Gherkinscenario name: Gherkinscenario 19: Inability to impersonate a customer with no account
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While SystemA has a customer "harold@thrasher.ie" with the first name "Harold", 
+When UserA checks their details, 
+then UserA should not be able to impersonate them.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'customer' (line : 3 column : 21)
+ERROR:no viable alternative at input '"harold@thrasher.ie"' (line : 3 column : 30)
+ERROR:no viable alternative at input 'the' (line : 3 column : 56)
+ERROR:no viable alternative at input 'name' (line : 3 column : 66)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:mismatched input ',' expecting 'by' (line : 4 column : 32)
+ERROR:no viable alternative at input 'should' (line : 5 column : 12)
+ERROR:missing '.' at 'be' (line : 5 column : 23)
+ERROR:no viable alternative at input 'able' (line : 5 column : 26)
+ERROR:missing '.' at 'impersonate' (line : 5 column : 34)
+ERROR:no viable alternative at input 'them' (line : 5 column : 46)
+ERROR:Couldn't resolve reference to Actor 'customer'. (line : 3 column : 21)
+ERROR:Couldn't resolve reference to Actor 'first'. (line : 3 column : 60)
+ERROR:Couldn't resolve reference to Quantifier 'their'. (line : 4 column : 19)
+ERROR:Couldn't resolve reference to ClassOrLabel 'details'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 23)
+ERROR:Couldn't resolve reference to Actor 'impersonate'. (line : 5 column : 34)
+
+```
+### DSL-Rimay Score
+DSL_Score: 77
+
+Progressbar: ![77%](https://progress-bar.dev/77)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 1: Requesting the cash from a creditworthy account_Incorrect___04_08_2024__21:14:35__84.5_score.md

@@ -0,0 +1,138 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:14:35
+* UniqueID: 1fb97d37-a8e6-440d-b765-4c8a6f3e8bdb
+* Gherkinscenario name: Gherkinscenario 1: Requesting the cash from a creditworthy account_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA requests cash, 
+then account must be debited.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'be' (line : 4 column : 19)
+ERROR:no viable alternative at input 'debited' (line : 4 column : 22)
+ERROR:Couldn't resolve reference to Actor 'cash'. (line : 3 column : 21)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 4 column : 19)
+
+```
+### DSL-Rimay Score
+DSL_Score: 94
+
+Progressbar: ![94%](https://progress-bar.dev/94)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 84.5
+
+Progressbar: ![84.5%](https://progress-bar.dev/84.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 1: Requesting the cash from a creditworthy account__04_08_2024__21:13:41__83.0_score.md

@@ -0,0 +1,153 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:13:41
+* UniqueID: 0b0d8e93-4089-4d51-b573-b632e631a7a2
+* Gherkinscenario name: Gherkinscenario 1: Requesting the cash from a creditworthy account
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the account is considered creditworthy,
+When UserA asks for cash,
+then make sure to debit the account.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'creditworthy' (line : 3 column : 33)
+ERROR:no viable alternative at input 'asks' (line : 4 column : 12)
+ERROR:no viable alternative at input ',' (line : 4 column : 25)
+ERROR:no viable alternative at input 'make' (line : 5 column : 6)
+ERROR:no viable alternative at input 'sure' (line : 5 column : 11)
+ERROR:missing '.' at 'debit' (line : 5 column : 19)
+ERROR:no viable alternative at input 'the' (line : 5 column : 25)
+ERROR:extraneous input 'account' expecting '.' (line : 5 column : 29)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'account'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Quantifier 'considered'. (line : 3 column : 22)
+ERROR:Couldn't resolve reference to Actor 'cash'. (line : 4 column : 21)
+ERROR:Couldn't resolve reference to Actor 'make'. (line : 5 column : 6)
+ERROR:Couldn't resolve reference to Actor 'debit'. (line : 5 column : 19)
+
+```
+### DSL-Rimay Score
+DSL_Score: 81
+
+Progressbar: ![81%](https://progress-bar.dev/81)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 1
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+### Final Score
+Final_score = 83.0
+
+Progressbar: ![83.0%](https://progress-bar.dev/83.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 2: Money withdraw from bank account_Incorrect___04_08_2024__21:06:19__69.5_score.md

@@ -0,0 +1,169 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:06:19
+* UniqueID: 44df80ed-cca4-40a6-bf12-47f9be270e93
+* Gherkinscenario name: Gherkinscenario 2: Money withdraw from bank account_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA has a credit balance in their bank account,
+When UserA requests to withdraw an amount below their card limit,
+Then the withdrawal should be processed without any errors or warnings.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'a' (line : 3 column : 17)
+ERROR:no viable alternative at input 'balance' (line : 3 column : 26)
+ERROR:missing '.' at 'their' (line : 3 column : 37)
+ERROR:no viable alternative at input 'bank' (line : 3 column : 43)
+ERROR:missing '.' at 'account' (line : 3 column : 48)
+ERROR:no viable alternative at input ',' (line : 3 column : 55)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:extraneous input 'to' expecting RULE_ID (line : 4 column : 21)
+ERROR:no viable alternative at input 'below' (line : 4 column : 43)
+ERROR:missing '.' at 'their' (line : 4 column : 49)
+ERROR:no viable alternative at input 'card' (line : 4 column : 55)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'the' (line : 5 column : 6)
+ERROR:missing '.' at 'withdrawal' (line : 5 column : 10)
+ERROR:no viable alternative at input 'should' (line : 5 column : 21)
+ERROR:missing '.' at 'be' (line : 5 column : 28)
+ERROR:no viable alternative at input 'processed' (line : 5 column : 31)
+ERROR:missing '.' at 'without' (line : 5 column : 41)
+ERROR:no viable alternative at input 'any' (line : 5 column : 49)
+ERROR:missing '.' at 'errors' (line : 5 column : 53)
+ERROR:no viable alternative at input 'or' (line : 5 column : 60)
+ERROR:no viable alternative at input 'warnings' (line : 5 column : 63)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'credit'. (line : 3 column : 19)
+ERROR:Couldn't resolve reference to Actor 'their'. (line : 3 column : 37)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 48)
+ERROR:Couldn't resolve reference to Actor 'withdraw'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'amount'. (line : 4 column : 36)
+ERROR:Couldn't resolve reference to Actor 'their'. (line : 4 column : 49)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'withdrawal'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 28)
+ERROR:Couldn't resolve reference to Actor 'without'. (line : 5 column : 41)
+ERROR:Couldn't resolve reference to Actor 'errors'. (line : 5 column : 53)
+
+```
+### DSL-Rimay Score
+DSL_Score: 59
+
+Progressbar: ![59%](https://progress-bar.dev/59)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 1
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 69.5
+
+Progressbar: ![69.5%](https://progress-bar.dev/69.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 2: Money withdraw from bank account__04_08_2024__21:05:04__73.5_score.md

@@ -0,0 +1,177 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:05:04
+* UniqueID: 27224088-5e4b-40b5-827f-dcb057517da0
+* Gherkinscenario name: Gherkinscenario 2: Money withdraw from bank account
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA has a bank account with a positive balance,
+When UserA tries to withdraw an amount below their card limit,
+Then the withdrawal should be successful without any errors or warnings.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'a' (line : 3 column : 17)
+ERROR:no viable alternative at input 'account' (line : 3 column : 24)
+ERROR:missing '.' at 'a' (line : 3 column : 37)
+ERROR:no viable alternative at input 'balance' (line : 3 column : 48)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 18)
+ERROR:missing ',' at 'withdraw' (line : 4 column : 21)
+ERROR:no viable alternative at input 'an' (line : 4 column : 30)
+ERROR:missing '.' at 'amount' (line : 4 column : 33)
+ERROR:no viable alternative at input 'below' (line : 4 column : 40)
+ERROR:missing '.' at 'their' (line : 4 column : 46)
+ERROR:no viable alternative at input 'card' (line : 4 column : 52)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'the' (line : 5 column : 6)
+ERROR:missing '.' at 'withdrawal' (line : 5 column : 10)
+ERROR:no viable alternative at input 'should' (line : 5 column : 21)
+ERROR:missing '.' at 'be' (line : 5 column : 28)
+ERROR:no viable alternative at input 'successful' (line : 5 column : 31)
+ERROR:missing '.' at 'without' (line : 5 column : 42)
+ERROR:no viable alternative at input 'any' (line : 5 column : 50)
+ERROR:missing '.' at 'errors' (line : 5 column : 54)
+ERROR:no viable alternative at input 'or' (line : 5 column : 61)
+ERROR:no viable alternative at input 'warnings' (line : 5 column : 64)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'bank'. (line : 3 column : 19)
+ERROR:Couldn't resolve reference to Actor 'positive'. (line : 3 column : 39)
+ERROR:Couldn't resolve reference to Actor 'tries'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'withdraw'. (line : 4 column : 21)
+ERROR:Couldn't resolve reference to Actor 'amount'. (line : 4 column : 33)
+ERROR:Couldn't resolve reference to Actor 'their'. (line : 4 column : 46)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'withdrawal'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 28)
+ERROR:Couldn't resolve reference to Actor 'without'. (line : 5 column : 42)
+ERROR:Couldn't resolve reference to Actor 'errors'. (line : 5 column : 54)
+
+```
+### DSL-Rimay Score
+DSL_Score: 57
+
+Progressbar: ![57%](https://progress-bar.dev/57)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 73.5
+
+Progressbar: ![73.5%](https://progress-bar.dev/73.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 3: Requesting the cash from an overdrawn account_Incorrect___04_08_2024__21:17:55__82.5_score.md

@@ -0,0 +1,142 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:17:55
+* UniqueID: d6f09f78-4184-431b-8640-2cf366e9b8f2
+* Gherkinscenario name: Gherkinscenario 3: Requesting the cash from an overdrawn account_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When account is overdrawn
+then system must display rejection message.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'is' (line : 3 column : 14)
+ERROR:missing ',' at 'overdrawn' (line : 3 column : 17)
+ERROR:no viable alternative at input 'then' (line : 4 column : 1)
+ERROR:no viable alternative at input 'system' (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'overdrawn'. (line : 3 column : 17)
+ERROR:Couldn't resolve reference to Actor 'system'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to EObject 'message'. (line : 4 column : 36)
+ERROR:Couldn't resolve reference to Quantifier 'rejection'. (line : 4 column : 26)
+
+```
+### DSL-Rimay Score
+DSL_Score: 85
+
+Progressbar: ![85%](https://progress-bar.dev/85)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 82.5
+
+Progressbar: ![82.5%](https://progress-bar.dev/82.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 3: Requesting the cash from an overdrawn account__04_08_2024__21:17:01__100_score.md

@@ -0,0 +1,143 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:17:01
+* UniqueID: 1aa7ea19-3e99-4b15-9dfa-15ba9bc41085
+* Gherkinscenario name: Gherkinscenario 3: Requesting the cash from an overdrawn account
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the account is "overdrawn", 
+When UserA requests "cash", 
+then system must display "rejection message".
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:mismatched input '"cash"' expecting RULE_ID (line : 4 column : 21)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'account'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Actor 'system'. (line : 5 column : 6)
+
+```
+### DSL-Rimay Score
+DSL_Score: 96
+
+Progressbar: ![96%](https://progress-bar.dev/96)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 4: User presses button._Incorrect___04_08_2024__21:19:56__70.5_score.md

@@ -0,0 +1,150 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:19:56
+* UniqueID: 57d7ebe6-ec6d-4602-9beb-0329f07bde40
+* Gherkinscenario name: Gherkinscenario 4: User presses button._Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA navigates to the login page
+When UserA clicks on button
+Then SystemA refreshes page
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'navigates' (line : 3 column : 13)
+ERROR:no viable alternative at input 'to' (line : 3 column : 23)
+ERROR:no viable alternative at input 'the' (line : 3 column : 26)
+ERROR:no viable alternative at input 'page' (line : 3 column : 36)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:missing RULE_STRING at 'Then' (line : 5 column : 1)
+ERROR:missing ',' at 'SystemA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'refreshes' (line : 5 column : 14)
+ERROR:missing '.' at 'page' (line : 5 column : 24)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'navigates'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Actor 'login'. (line : 3 column : 30)
+ERROR:Couldn't resolve reference to Quantifier 'button'. (line : 4 column : 22)
+ERROR:Couldn't resolve reference to UIComponent 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'page'. (line : 5 column : 24)
+
+```
+### DSL-Rimay Score
+DSL_Score: 71
+
+Progressbar: ![71%](https://progress-bar.dev/71)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,1
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+### Final Score
+Final_score = 70.5
+
+Progressbar: ![70.5%](https://progress-bar.dev/70.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 4: User presses button.__04_08_2024__21:18:48__82.0_score.md

@@ -0,0 +1,153 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:18:48
+* UniqueID: 62ea0b60-ec9b-4cdc-aeed-9d86a8f400af
+* Gherkinscenario name: Gherkinscenario 4: User presses button.
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA navigates to the login page,
+When UserA clicks on the button,
+then SystemA refreshes the page.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'to' (line : 3 column : 23)
+ERROR:missing ',' at 'the' (line : 3 column : 26)
+ERROR:no viable alternative at input 'page' (line : 3 column : 36)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:missing RULE_STRING at 'button' (line : 4 column : 26)
+ERROR:no viable alternative at input 'refreshes' (line : 5 column : 14)
+ERROR:missing '.' at 'the' (line : 5 column : 24)
+ERROR:no viable alternative at input '.' (line : 5 column : 32)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'navigates'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'login'. (line : 3 column : 30)
+ERROR:Couldn't resolve reference to UIComponent 'button'. (line : 4 column : 26)
+ERROR:Couldn't resolve reference to Actor 'page'. (line : 5 column : 28)
+
+```
+### DSL-Rimay Score
+DSL_Score: 74
+
+Progressbar: ![74%](https://progress-bar.dev/74)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 1
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 60
+
+Progressbar: ![60%](https://progress-bar.dev/60)
+
+            
+
+### Final Score
+Final_score = 82.0
+
+Progressbar: ![82.0%](https://progress-bar.dev/82.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 5: Getting verified after clicking the link in the verification message_Incorrect___04_08_2024__21:23:16__83.0_score.md

@@ -0,0 +1,151 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:23:16
+* UniqueID: 609fcc0b-ef48-4d83-9ab6-4b2405c2d290
+* Gherkinscenario name: Gherkinscenario 5: Getting verified after clicking the link in the verification message_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA verifies their account using the link from the email, 
+then UserA should receive a notification confirming successful verification.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'their' (line : 3 column : 21)
+ERROR:no viable alternative at input 'account' (line : 3 column : 27)
+ERROR:missing '.' at 'the' (line : 3 column : 41)
+ERROR:mismatched input 'link' expecting RULE_ID (line : 3 column : 45)
+ERROR:missing '.' at 'the' (line : 3 column : 55)
+ERROR:no viable alternative at input ',' (line : 3 column : 64)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'should' (line : 4 column : 12)
+ERROR:missing '.' at 'a' (line : 4 column : 27)
+ERROR:no viable alternative at input 'confirming' (line : 4 column : 42)
+ERROR:missing '.' at 'successful' (line : 4 column : 53)
+ERROR:no viable alternative at input 'verification' (line : 4 column : 64)
+ERROR:Couldn't resolve reference to Actor 'verifies'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'their'. (line : 3 column : 21)
+ERROR:Couldn't resolve reference to Actor 'email'. (line : 3 column : 59)
+ERROR:Couldn't resolve reference to Actor 'notification'. (line : 4 column : 29)
+ERROR:Couldn't resolve reference to Actor 'successful'. (line : 4 column : 53)
+
+```
+### DSL-Rimay Score
+DSL_Score: 76
+
+Progressbar: ![76%](https://progress-bar.dev/76)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 83.0
+
+Progressbar: ![83.0%](https://progress-bar.dev/83.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 5: Getting verified after clicking the link in the verification message__04_08_2024__21:22:15__95.0_score.md

@@ -0,0 +1,169 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:22:15
+* UniqueID: 6ca8c0a2-9b9d-4cc7-b229-4f791511ff14
+* Gherkinscenario name: Gherkinscenario 5: Getting verified after clicking the link in the verification message
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the "verification email" has been sent to "UserA", 
+When "UserA" attempts to verify his account using the link from this email, 
+then "UserA" should receive a notification confirming the successful verification.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'sent' (line : 3 column : 41)
+ERROR:no viable alternative at input 'to' (line : 3 column : 46)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing ',' at 'attempts' (line : 4 column : 14)
+ERROR:no viable alternative at input 'to' (line : 4 column : 23)
+ERROR:no viable alternative at input 'verify' (line : 4 column : 26)
+ERROR:no viable alternative at input 'his' (line : 4 column : 33)
+ERROR:missing '.' at 'account' (line : 4 column : 37)
+ERROR:no viable alternative at input 'using' (line : 4 column : 45)
+ERROR:no viable alternative at input 'the' (line : 4 column : 51)
+ERROR:mismatched input 'link' expecting RULE_ID (line : 4 column : 55)
+ERROR:missing '.' at 'this' (line : 4 column : 65)
+ERROR:no viable alternative at input 'email' (line : 4 column : 70)
+ERROR:missing '.' at 'should' (line : 5 column : 14)
+ERROR:no viable alternative at input 'receive' (line : 5 column : 21)
+ERROR:missing '.' at 'confirming' (line : 5 column : 44)
+ERROR:no viable alternative at input 'the' (line : 5 column : 55)
+ERROR:missing '.' at 'successful' (line : 5 column : 59)
+ERROR:no viable alternative at input 'verification' (line : 5 column : 70)
+ERROR:Couldn't resolve reference to Quantifier 'been'. (line : 3 column : 36)
+ERROR:Couldn't resolve reference to Actor 'sent'. (line : 3 column : 41)
+ERROR:Couldn't resolve reference to Actor 'attempts'. (line : 4 column : 14)
+ERROR:Couldn't resolve reference to Actor 'verify'. (line : 4 column : 26)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 4 column : 37)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 65)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 14)
+ERROR:Couldn't resolve reference to Class 'notification'. (line : 5 column : 31)
+ERROR:Couldn't resolve reference to Actor 'confirming'. (line : 5 column : 44)
+ERROR:Couldn't resolve reference to Actor 'successful'. (line : 5 column : 59)
+
+```
+### DSL-Rimay Score
+DSL_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 95.0
+
+Progressbar: ![95.0%](https://progress-bar.dev/95.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 6: The selected province stays after validation error_Incorrect___04_08_2024__21:25:48__85.5_score.md

@@ -0,0 +1,146 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:25:48
+* UniqueID: a41baffc-b661-4bd1-b5fb-4f1089a543ca
+* Gherkinscenario name: Gherkinscenario 6: The selected province stays after validation error_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA is editing the address of "Fletcher Ren"
+then UserA should remain on the "Fletcher Ren" address edit page.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 24)
+ERROR:no viable alternative at input 'of' (line : 3 column : 36)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'should' (line : 4 column : 12)
+ERROR:missing '.' at 'remain' (line : 4 column : 19)
+ERROR:no viable alternative at input 'on' (line : 4 column : 26)
+ERROR:no viable alternative at input 'the' (line : 4 column : 29)
+ERROR:extraneous input '"Fletcher Ren"' expecting RULE_ID (line : 4 column : 33)
+ERROR:no viable alternative at input 'edit' (line : 4 column : 56)
+ERROR:Couldn't resolve reference to Quantifier 'editing'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 28)
+ERROR:Couldn't resolve reference to Actor 'remain'. (line : 4 column : 19)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 4 column : 48)
+
+```
+### DSL-Rimay Score
+DSL_Score: 81
+
+Progressbar: ![81%](https://progress-bar.dev/81)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 85.5
+
+Progressbar: ![85.5%](https://progress-bar.dev/85.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 6: The selected province stays after validation error__04_08_2024__21:24:11__100_score.md

@@ -0,0 +1,152 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:24:11
+* UniqueID: 674cb7ca-c256-4aa8-8b64-1d26b3520243
+* Gherkinscenario name: Gherkinscenario 6: The selected province stays after validation error
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA is editing the address of "Fletcher Ren", when UserA removes the street, then UserA should still be on the "Fletcher Ren" address edit page.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 24)
+ERROR:no viable alternative at input 'of' (line : 3 column : 36)
+ERROR:no viable alternative at input 'when' (line : 3 column : 55)
+ERROR:no viable alternative at input 'should' (line : 3 column : 97)
+ERROR:missing '.' at 'still' (line : 3 column : 104)
+ERROR:no viable alternative at input 'be' (line : 3 column : 110)
+ERROR:missing '.' at 'the' (line : 3 column : 116)
+ERROR:extraneous input '"Fletcher Ren"' expecting RULE_ID (line : 3 column : 120)
+ERROR:no viable alternative at input 'edit' (line : 3 column : 143)
+ERROR:Couldn't resolve reference to Quantifier 'editing'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 28)
+ERROR:Couldn't resolve reference to PropertyOrLabel 'street'. (line : 3 column : 78)
+ERROR:Couldn't resolve reference to Actor 'still'. (line : 3 column : 104)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 135)
+
+```
+### DSL-Rimay Score
+DSL_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock_Incorrect___04_08_2024__21:28:24__73.0_score.md

@@ -0,0 +1,162 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:28:24
+* UniqueID: 32f0c59c-b81e-4743-8cc0-7d6a09f95080
+* Gherkinscenario name: Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given the "T-shirt banana" product is not available in stock
+When UserA views the details of this product
+Then UserA must see that it is out of stock
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 7)
+ERROR:missing '.' at 'product' (line : 3 column : 28)
+ERROR:no viable alternative at input 'is' (line : 3 column : 36)
+ERROR:no viable alternative at input 'available' (line : 3 column : 43)
+ERROR:missing '.' at 'stock' (line : 3 column : 56)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'views' (line : 4 column : 12)
+ERROR:missing '.' at 'the' (line : 4 column : 18)
+ERROR:no viable alternative at input 'of' (line : 4 column : 30)
+ERROR:no viable alternative at input 'this' (line : 4 column : 33)
+ERROR:no viable alternative at input 'product' (line : 4 column : 38)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'that' (line : 5 column : 21)
+ERROR:no viable alternative at input 'it' (line : 5 column : 26)
+ERROR:no viable alternative at input 'is' (line : 5 column : 29)
+ERROR:no viable alternative at input 'stock' (line : 5 column : 39)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'product'. (line : 3 column : 28)
+ERROR:Couldn't resolve reference to Actor 'stock'. (line : 3 column : 56)
+ERROR:Couldn't resolve reference to Actor 'details'. (line : 4 column : 22)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 33)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'it'. (line : 5 column : 26)
+ERROR:Couldn't resolve reference to Actor 'stock'. (line : 5 column : 39)
+
+```
+### DSL-Rimay Score
+DSL_Score: 66
+
+Progressbar: ![66%](https://progress-bar.dev/66)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 73.0
+
+Progressbar: ![73.0%](https://progress-bar.dev/73.0)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock__04_08_2024__21:27:30__100_score.md

@@ -0,0 +1,158 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:27:30
+* UniqueID: 90e4472a-c1c1-4298-8f1e-1223f1b5c70e
+* Gherkinscenario name: Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the product "T-shirt banana" is not available, 
+When UserA views the details of this product, 
+then UserA must see that it is out of stock.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input '"T-shirt banana"' (line : 3 column : 19)
+ERROR:no viable alternative at input 'the' (line : 4 column : 18)
+ERROR:no viable alternative at input 'of' (line : 4 column : 30)
+ERROR:no viable alternative at input 'this' (line : 4 column : 33)
+ERROR:no viable alternative at input 'product' (line : 4 column : 38)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'that' (line : 5 column : 21)
+ERROR:no viable alternative at input 'it' (line : 5 column : 26)
+ERROR:no viable alternative at input 'is' (line : 5 column : 29)
+ERROR:no viable alternative at input 'stock' (line : 5 column : 39)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'product'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Actor 'views'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'details'. (line : 4 column : 22)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 33)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'it'. (line : 5 column : 26)
+
+```
+### DSL-Rimay Score
+DSL_Score: 76
+
+Progressbar: ![76%](https://progress-bar.dev/76)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 8: Disabling country_Incorrect___04_08_2024__21:30:03__80.5_score.md

@@ -0,0 +1,146 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:30:03
+* UniqueID: af638b95-1b27-4b44-848f-8a7d4d6cbbba
+* Gherkinscenario name: Gherkinscenario 8: Disabling country_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA modifies the country to "United States", 
+then UserA must receive a notification confirming the successful edit.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 21)
+ERROR:no viable alternative at input 'to' (line : 3 column : 33)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 6)
+ERROR:missing '.' at 'confirming' (line : 4 column : 40)
+ERROR:no viable alternative at input 'the' (line : 4 column : 51)
+ERROR:missing '.' at 'successful' (line : 4 column : 55)
+ERROR:no viable alternative at input 'edit' (line : 4 column : 66)
+ERROR:Couldn't resolve reference to Actor 'modifies'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 3 column : 25)
+ERROR:Couldn't resolve reference to Class 'notification'. (line : 4 column : 27)
+ERROR:Couldn't resolve reference to Actor 'confirming'. (line : 4 column : 40)
+ERROR:Couldn't resolve reference to Actor 'successful'. (line : 4 column : 55)
+
+```
+### DSL-Rimay Score
+DSL_Score: 81
+
+Progressbar: ![81%](https://progress-bar.dev/81)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 80.5
+
+Progressbar: ![80.5%](https://progress-bar.dev/80.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 8: Disabling country__04_08_2024__21:29:15__72.5_score.md

@@ -0,0 +1,164 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:29:15
+* UniqueID: de0f5b90-f967-4b7e-ae76-629fc379707b
+* Gherkinscenario name: Gherkinscenario 8: Disabling country
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given SystemA has country "United States"
+When UserA edits the country,
+Then UserA should be notified that the edit was successful.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'SystemA' (line : 3 column : 7)
+ERROR:missing '.' at 'country' (line : 3 column : 19)
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 27)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 18)
+ERROR:no viable alternative at input ',' (line : 4 column : 29)
+ERROR:no viable alternative at input 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'should' (line : 5 column : 12)
+ERROR:no viable alternative at input 'be' (line : 5 column : 19)
+ERROR:missing '.' at 'notified' (line : 5 column : 22)
+ERROR:no viable alternative at input 'that' (line : 5 column : 31)
+ERROR:no viable alternative at input 'the' (line : 5 column : 36)
+ERROR:no viable alternative at input 'was' (line : 5 column : 45)
+ERROR:extraneous input 'successful' expecting '.' (line : 5 column : 49)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 3 column : 19)
+ERROR:Couldn't resolve reference to Actor 'edits'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 4 column : 22)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Actor 'notified'. (line : 5 column : 22)
+ERROR:Couldn't resolve reference to Actor 'edit'. (line : 5 column : 40)
+
+```
+### DSL-Rimay Score
+DSL_Score: 70
+
+Progressbar: ![70%](https://progress-bar.dev/70)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 72.5
+
+Progressbar: ![72.5%](https://progress-bar.dev/72.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 9: Resending the verification email as a logged in user_Incorrect___04_08_2024__21:32:09__83.5_score.md

@@ -0,0 +1,140 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:32:09
+* UniqueID: 2b8812de-8189-497f-ae4e-1a48b131a5d3
+* Gherkinscenario name: Gherkinscenario 9: Resending the verification email as a logged in user_Incorrect_
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA resends the verification email 
+then UserA must receive a notification.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 20)
+ERROR:no viable alternative at input 'email' (line : 3 column : 37)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'resends'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'verification'. (line : 3 column : 24)
+ERROR:Couldn't resolve reference to Class 'notification'. (line : 4 column : 27)
+
+```
+### DSL-Rimay Score
+DSL_Score: 87
+
+Progressbar: ![87%](https://progress-bar.dev/87)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 83.5
+
+Progressbar: ![83.5%](https://progress-bar.dev/83.5)
+

output_dataset/Chain-of-thought_0.2temp/Gherkinscenario 9: Resending the verification email as a logged in user__04_08_2024__21:30:51__100_score.md

@@ -0,0 +1,157 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:30:51
+* UniqueID: 66a50116-2684-4f3e-a5e3-6d2a14779d48
+* Gherkinscenario name: Gherkinscenario 9: Resending the verification email as a logged in user
+* LLM-temperature: 0.2
+
+        
+
+## Gherkin Input
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA is logged in as "valkyrie@cain.com", 
+When UserA requests to resend the verification email, 
+then UserA must receive a notification confirming that the verification email has been sent.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'in' (line : 3 column : 23)
+ERROR:extraneous input 'to' expecting RULE_ID (line : 4 column : 21)
+ERROR:no viable alternative at input 'email' (line : 4 column : 48)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'confirming' (line : 5 column : 40)
+ERROR:no viable alternative at input 'that' (line : 5 column : 51)
+ERROR:no viable alternative at input 'the' (line : 5 column : 56)
+ERROR:no viable alternative at input 'email' (line : 5 column : 73)
+ERROR:missing '.' at 'been' (line : 5 column : 83)
+ERROR:no viable alternative at input 'sent' (line : 5 column : 88)
+ERROR:Couldn't resolve reference to Quantifier 'logged'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'resend'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'verification'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Class 'notification'. (line : 5 column : 27)
+ERROR:Couldn't resolve reference to Actor 'confirming'. (line : 5 column : 40)
+ERROR:Couldn't resolve reference to Actor 'verification'. (line : 5 column : 60)
+ERROR:Couldn't resolve reference to Actor 'been'. (line : 5 column : 83)
+
+```
+### DSL-Rimay Score
+DSL_Score: 77
+
+Progressbar: ![77%](https://progress-bar.dev/77)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 10: Logging out the user doesn't log out my admin account_Incorrect___04_08_2024__22:20:30__65.5_score.md

@@ -0,0 +1,157 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:20:30
+* UniqueID: 7ff2ccc1-6248-4ecb-93fa-cfb15ebcf814
+* Gherkinscenario name: Gherkinscenario 10: Logging out the user doesn't log out my admin account_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given: UserA acts as the customer "remnant@london.uk"
+When UserA logs out of SystemA
+Then UserA must still have access to the administration dashboard.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input ':' (line : 3 column : 6)
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 8)
+ERROR:no viable alternative at input 'acts' (line : 3 column : 14)
+ERROR:missing '.' at 'the' (line : 3 column : 22)
+ERROR:no viable alternative at input '"remnant@london.uk"' (line : 3 column : 35)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'out' (line : 4 column : 17)
+ERROR:missing ',' at 'SystemA' (line : 4 column : 24)
+ERROR:no viable alternative at input 'Then' (line : 5 column : 1)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'still' (line : 5 column : 17)
+ERROR:no viable alternative at input 'have' (line : 5 column : 23)
+ERROR:no viable alternative at input 'access' (line : 5 column : 28)
+ERROR:no viable alternative at input 'to' (line : 5 column : 35)
+ERROR:no viable alternative at input 'the' (line : 5 column : 38)
+ERROR:no viable alternative at input 'dashboard' (line : 5 column : 57)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'customer'. (line : 3 column : 26)
+ERROR:Couldn't resolve reference to Actor 'logs'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'still'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'access'. (line : 5 column : 28)
+ERROR:Couldn't resolve reference to Actor 'administration'. (line : 5 column : 42)
+
+```
+### DSL-Rimay Score
+DSL_Score: 71
+
+Progressbar: ![71%](https://progress-bar.dev/71)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,1
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+### Final Score
+Final_score = 65.5
+
+Progressbar: ![65.5%](https://progress-bar.dev/65.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 10: Logging out the user doesn't log out my admin account__04_08_2024__22:19:21__95.5_score.md

@@ -0,0 +1,163 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:19:21
+* UniqueID: 8c1ebd8e-0b4a-41f5-b46d-b796864daea5
+* Gherkinscenario name: Gherkinscenario 10: Logging out the user doesn't log out my admin account
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA impersonates the customer "remnant@london.uk"
+When: UserA logs out from SystemA
+Then: UserA should still be able to access the administration dashboard
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA impersonates the customer "remnant@london.uk", 
+When UserA performs logout from SystemA, 
+then UserA must still have access to the administration dashboard.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 26)
+ERROR:no viable alternative at input '"remnant@london.uk"' (line : 3 column : 39)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'logout' (line : 4 column : 21)
+ERROR:no viable alternative at input 'from' (line : 4 column : 28)
+ERROR:no viable alternative at input 'SystemA' (line : 4 column : 33)
+ERROR:no viable alternative at input ',' (line : 4 column : 40)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'still' (line : 5 column : 17)
+ERROR:no viable alternative at input 'have' (line : 5 column : 23)
+ERROR:no viable alternative at input 'access' (line : 5 column : 28)
+ERROR:no viable alternative at input 'to' (line : 5 column : 35)
+ERROR:no viable alternative at input 'the' (line : 5 column : 38)
+ERROR:no viable alternative at input 'dashboard' (line : 5 column : 57)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'impersonates'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'customer'. (line : 3 column : 30)
+ERROR:Couldn't resolve reference to Actor 'performs'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'logout'. (line : 4 column : 21)
+ERROR:Couldn't resolve reference to Actor 'still'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'access'. (line : 5 column : 28)
+ERROR:Couldn't resolve reference to Actor 'administration'. (line : 5 column : 42)
+
+```
+### DSL-Rimay Score
+DSL_Score: 76
+
+Progressbar: ![76%](https://progress-bar.dev/76)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 1
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 95.5
+
+Progressbar: ![95.5%](https://progress-bar.dev/95.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 11: Seeing disabled code field while editing country_Incorrect___04_08_2024__22:21:58__78.5_score.md

@@ -0,0 +1,156 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:21:58
+* UniqueID: 29303052-3f42-43e9-9f52-36cfd42b8f32
+* Gherkinscenario name: Gherkinscenario 11: Seeing disabled code field while editing country_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given the country of "United States" is set in SystemA
+When UserA attempts to modify the country
+Then the code field must be deactivated.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 7)
+ERROR:missing '.' at 'country' (line : 3 column : 11)
+ERROR:no viable alternative at input 'of' (line : 3 column : 19)
+ERROR:no viable alternative at input 'in' (line : 3 column : 45)
+ERROR:missing '.' at 'SystemA' (line : 3 column : 48)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'attempts' (line : 4 column : 12)
+ERROR:missing '.' at 'modify' (line : 4 column : 24)
+ERROR:no viable alternative at input 'the' (line : 4 column : 31)
+ERROR:missing '.' at 'country' (line : 4 column : 35)
+ERROR:no viable alternative at input 'Then' (line : 5 column : 1)
+ERROR:missing '.' at 'the' (line : 5 column : 6)
+ERROR:no viable alternative at input 'field' (line : 5 column : 15)
+ERROR:missing '.' at 'be' (line : 5 column : 26)
+ERROR:no viable alternative at input 'deactivated' (line : 5 column : 29)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 3 column : 11)
+ERROR:Couldn't resolve reference to Actor 'modify'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Actor 'code'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 26)
+
+```
+### DSL-Rimay Score
+DSL_Score: 72
+
+Progressbar: ![72%](https://progress-bar.dev/72)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 78.5
+
+Progressbar: ![78.5%](https://progress-bar.dev/78.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 11: Seeing disabled code field while editing country__04_08_2024__22:21:09__56.5_score.md

@@ -0,0 +1,166 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:21:09
+* UniqueID: d9b8fc38-c38c-4898-8e48-0b9cda3f31ab
+* Gherkinscenario name: Gherkinscenario 11: Seeing disabled code field while editing country
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: the code field should be disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given the "SystemA" holds country "United States",
+When UserA tries editing current country,
+Then the code field needs to be deactivated.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 7)
+ERROR:missing '.' at 'holds' (line : 3 column : 21)
+ERROR:no viable alternative at input 'country' (line : 3 column : 27)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'editing' (line : 4 column : 18)
+ERROR:no viable alternative at input 'current' (line : 4 column : 26)
+ERROR:missing '.' at 'country' (line : 4 column : 34)
+ERROR:no viable alternative at input ',' (line : 4 column : 41)
+ERROR:no viable alternative at input 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'the' (line : 5 column : 6)
+ERROR:missing '.' at 'code' (line : 5 column : 10)
+ERROR:no viable alternative at input 'field' (line : 5 column : 15)
+ERROR:missing '.' at 'needs' (line : 5 column : 21)
+ERROR:no viable alternative at input 'to' (line : 5 column : 27)
+ERROR:no viable alternative at input 'be' (line : 5 column : 30)
+ERROR:no viable alternative at input 'deactivated' (line : 5 column : 33)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'holds'. (line : 3 column : 21)
+ERROR:Couldn't resolve reference to Actor 'tries'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'editing'. (line : 4 column : 18)
+ERROR:Couldn't resolve reference to Actor 'country'. (line : 4 column : 34)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'code'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'needs'. (line : 5 column : 21)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 30)
+
+```
+### DSL-Rimay Score
+DSL_Score: 68
+
+Progressbar: ![68%](https://progress-bar.dev/68)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,1
+Incomplete condition: 0
+Incomplete system response: 1
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+### Final Score
+Final_score = 56.5
+
+Progressbar: ![56.5%](https://progress-bar.dev/56.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 12: Changing the currency of my cart_Incorrect___04_08_2024__22:23:23__76.0_score.md

@@ -0,0 +1,145 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:23:23
+* UniqueID: eaca24d6-1190-443b-86e9-d1fe923743ed
+* Gherkinscenario name: Gherkinscenario 12: Changing the currency of my cart_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA changes currency to EUR
+then the total value of product "The Pug Mug" should be €6.82.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:mismatched input 'EUR' expecting 'accept' (line : 3 column : 32)
+ERROR:missing ',' at 'the' (line : 4 column : 6)
+ERROR:no viable alternative at input 'value' (line : 4 column : 16)
+ERROR:no viable alternative at input 'product' (line : 4 column : 25)
+ERROR:no viable alternative at input '"The Pug Mug"' (line : 4 column : 33)
+ERROR:no viable alternative at input 'should' (line : 4 column : 47)
+ERROR:no viable alternative at input 'be' (line : 4 column : 54)
+ERROR:missing EOF at '82' (line : 4 column : 60)
+ERROR:Couldn't resolve reference to PropertyId 'currency'. (line : 3 column : 20)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 4 column : 10)
+ERROR:Couldn't resolve reference to Actor 'product'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 4 column : 47)
+
+```
+### DSL-Rimay Score
+DSL_Score: 72
+
+Progressbar: ![72%](https://progress-bar.dev/72)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 76.0
+
+Progressbar: ![76.0%](https://progress-bar.dev/76.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 12: Changing the currency of my cart__04_08_2024__22:22:38__66.5_score.md

@@ -0,0 +1,156 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:22:38
+* UniqueID: eb67389e-8ccd-4a32-8af0-aee7e32363e2
+* Gherkinscenario name: Gherkinscenario 12: Changing the currency of my cart
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "The Pug Mug" in the cart
+When: UserA switches to the "EUR" currency
+Then: the grand total value should be "€6.82"
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+UserA has "The Pug Mug" product in the cart, 
+When UserA changes the currency to "EUR", 
+then the total amount becomes "€6.82".
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'has' (line : 3 column : 7)
+ERROR:no viable alternative at input 'product' (line : 3 column : 25)
+ERROR:no viable alternative at input 'in' (line : 3 column : 33)
+ERROR:no viable alternative at input 'the' (line : 3 column : 36)
+ERROR:no viable alternative at input ',' (line : 3 column : 44)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing 'accept' at '"EUR"' (line : 4 column : 36)
+ERROR:no viable alternative at input 'amount' (line : 5 column : 16)
+ERROR:missing '.' at 'becomes' (line : 5 column : 23)
+ERROR:no viable alternative at input '"€6.82"' (line : 5 column : 31)
+ERROR:no viable alternative at input '.' (line : 5 column : 38)
+ERROR:Couldn't resolve reference to Actor 'product'. (line : 3 column : 25)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 3 column : 40)
+ERROR:Couldn't resolve reference to PropertyId 'currency'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'becomes'. (line : 5 column : 23)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 2
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+### Final Score
+Final_score = 66.5
+
+Progressbar: ![66.5%](https://progress-bar.dev/66.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 13: Viewing information about empty cart after channel switching_Incorrect___04_08_2024__22:24:49__74.0_score.md

@@ -0,0 +1,150 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:24:49
+* UniqueID: 1191a775-3f11-424a-b27f-38ba7b08b4c9
+* Gherkinscenario name: Gherkinscenario 13: Viewing information about empty cart after channel switching_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA simplifies their current channel to "Poland", 
+when UserA changes their current channel to "France", 
+then UserA's carts should be empty.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'simplifies' (line : 3 column : 13)
+ERROR:no viable alternative at input 'their' (line : 3 column : 24)
+ERROR:missing '.' at 'current' (line : 3 column : 30)
+ERROR:no viable alternative at input 'channel' (line : 3 column : 38)
+ERROR:missing '.' at 'when' (line : 4 column : 1)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 34)
+ERROR:no viable alternative at input 'to' (line : 4 column : 42)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 5 column : 11)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'simplifies'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Actor 'current'. (line : 3 column : 30)
+ERROR:Couldn't resolve reference to Quantifier 'their'. (line : 4 column : 20)
+ERROR:Couldn't resolve reference to PropertyId 'current'. (line : 4 column : 26)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 4 column : 34)
+
+```
+### DSL-Rimay Score
+DSL_Score: 68
+
+Progressbar: ![68%](https://progress-bar.dev/68)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 74.0
+
+Progressbar: ![74.0%](https://progress-bar.dev/74.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 13: Viewing information about empty cart after channel switching__04_08_2024__22:24:04__100_score.md

@@ -0,0 +1,159 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:24:04
+* UniqueID: f7349664-b1ab-4d25-9888-cd0674319c72
+* Gherkinscenario name: Gherkinscenario 13: Viewing information about empty cart after channel switching
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA simplifies his current channel to "Poland"
+When: UserA changes current channel to "France"
+Then: UserA carts should be empty
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA simplifies his current channel to "Poland",  
+When UserA switches the current channel to "France",  
+then UserA carts must be emptied.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'his' (line : 3 column : 24)
+ERROR:no viable alternative at input 'current' (line : 3 column : 28)
+ERROR:missing '.' at 'channel' (line : 3 column : 36)
+ERROR:no viable alternative at input 'to' (line : 3 column : 44)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 21)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 33)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'carts' (line : 5 column : 12)
+ERROR:missing '.' at 'be' (line : 5 column : 23)
+ERROR:no viable alternative at input 'emptied' (line : 5 column : 26)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'simplifies'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'his'. (line : 3 column : 24)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 36)
+ERROR:Couldn't resolve reference to Actor 'switches'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'current'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 23)
+
+```
+### DSL-Rimay Score
+DSL_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 14: Displaying default shop homepage_Incorrect___04_08_2024__22:26:35__73.5_score.md

@@ -0,0 +1,145 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:26:35
+* UniqueID: 365cbd73-184e-4534-ab5a-de9e94af2f44
+* Gherkinscenario name: Gherkinscenario 14: Displaying default shop homepage_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When UserA visits channel "United States", 
+then no theme should be used.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'channel' (line : 3 column : 19)
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 27)
+ERROR:no viable alternative at input 'no' (line : 4 column : 6)
+ERROR:no viable alternative at input 'theme' (line : 4 column : 9)
+ERROR:missing '.' at 'should' (line : 4 column : 15)
+ERROR:no viable alternative at input 'be' (line : 4 column : 22)
+ERROR:extraneous input 'used' expecting '.' (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'visits'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 19)
+ERROR:Couldn't resolve reference to Actor 'no'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 4 column : 15)
+
+```
+### DSL-Rimay Score
+DSL_Score: 82
+
+Progressbar: ![82%](https://progress-bar.dev/82)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 85
+
+Progressbar: ![85%](https://progress-bar.dev/85)
+
+            
+
+### Final Score
+Final_score = 73.5
+
+Progressbar: ![73.5%](https://progress-bar.dev/73.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 14: Displaying default shop homepage__04_08_2024__22:25:41__74.0_score.md

@@ -0,0 +1,176 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:25:41
+* UniqueID: 8bff1496-cc9c-429a-a0a6-57cddf28dd16
+* Gherkinscenario name: Gherkinscenario 14: Displaying default shop homepage
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" does not use any theme
+When: UserA visits this channel's homepage
+Then: UserA should not see a homepage from "maverick/meerkat" theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given that the channel "United States" does not utilize any theme,
+When UserA accesses the homepage of this channel,
+Then UserA should not view a homepage using the "maverick/meerkat" theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'that' (line : 3 column : 7)
+ERROR:no viable alternative at input 'the' (line : 3 column : 12)
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 24)
+ERROR:no viable alternative at input 'utilize' (line : 3 column : 49)
+ERROR:no viable alternative at input 'any' (line : 3 column : 57)
+ERROR:missing '.' at 'theme' (line : 3 column : 61)
+ERROR:no viable alternative at input ',' (line : 3 column : 66)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 21)
+ERROR:no viable alternative at input 'of' (line : 4 column : 34)
+ERROR:no viable alternative at input 'this' (line : 4 column : 37)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 42)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'should' (line : 5 column : 12)
+ERROR:no viable alternative at input 'not' (line : 5 column : 19)
+ERROR:no viable alternative at input 'view' (line : 5 column : 23)
+ERROR:no viable alternative at input 'a' (line : 5 column : 28)
+ERROR:missing '.' at 'homepage' (line : 5 column : 30)
+ERROR:no viable alternative at input 'using' (line : 5 column : 39)
+ERROR:no viable alternative at input 'the' (line : 5 column : 45)
+ERROR:extraneous input '"maverick/meerkat"' expecting RULE_ID (line : 5 column : 49)
+ERROR:no viable alternative at input '.' (line : 5 column : 73)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'utilize'. (line : 3 column : 49)
+ERROR:Couldn't resolve reference to Actor 'theme'. (line : 3 column : 61)
+ERROR:Couldn't resolve reference to Actor 'accesses'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 37)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Actor 'view'. (line : 5 column : 23)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 5 column : 30)
+ERROR:Couldn't resolve reference to Actor 'theme'. (line : 5 column : 68)
+
+```
+### DSL-Rimay Score
+DSL_Score: 58
+
+Progressbar: ![58%](https://progress-bar.dev/58)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 74.0
+
+Progressbar: ![74.0%](https://progress-bar.dev/74.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 15: Displaying themed shop homepage_Incorrect___04_08_2024__22:29:02__77.0_score.md

@@ -0,0 +1,164 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:29:02
+* UniqueID: 558b91b4-ed3b-457e-b4b5-de4e5c684fb5
+* Gherkinscenario name: Gherkinscenario 15: Displaying themed shop homepage_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given theme "maverick/meerkat" is used by channel named "United States",
+When UserA visits the homepage of this channel,
+Then UserA must see a homepage designed with the "maverick/meerkat" theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'theme' (line : 3 column : 7)
+ERROR:missing '.' at 'used' (line : 3 column : 35)
+ERROR:no viable alternative at input 'by' (line : 3 column : 40)
+ERROR:no viable alternative at input 'channel' (line : 3 column : 43)
+ERROR:no viable alternative at input 'named' (line : 3 column : 51)
+ERROR:missing '.' at 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'the' (line : 4 column : 19)
+ERROR:no viable alternative at input 'of' (line : 4 column : 32)
+ERROR:no viable alternative at input 'this' (line : 4 column : 35)
+ERROR:no viable alternative at input 'channel' (line : 4 column : 40)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'a' (line : 5 column : 21)
+ERROR:missing '.' at 'homepage' (line : 5 column : 23)
+ERROR:no viable alternative at input 'designed' (line : 5 column : 32)
+ERROR:missing '.' at 'the' (line : 5 column : 46)
+ERROR:extraneous input '"maverick/meerkat"' expecting RULE_ID (line : 5 column : 50)
+ERROR:no viable alternative at input '.' (line : 5 column : 74)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'used'. (line : 3 column : 35)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 43)
+ERROR:Couldn't resolve reference to Actor 'visits'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 4 column : 23)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'homepage'. (line : 5 column : 23)
+ERROR:Couldn't resolve reference to Actor 'theme'. (line : 5 column : 69)
+
+```
+### DSL-Rimay Score
+DSL_Score: 64
+
+Progressbar: ![64%](https://progress-bar.dev/64)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 77.0
+
+Progressbar: ![77.0%](https://progress-bar.dev/77.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 15: Displaying themed shop homepage__04_08_2024__22:27:16__100_score.md

@@ -0,0 +1,145 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:27:16
+* UniqueID: 19984a4f-f446-48e7-b84e-b3c95b88e846
+* Gherkinscenario name: Gherkinscenario 15: Displaying themed shop homepage
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: channel "United States" uses "maverick/meerkat" theme
+When: UserA visit this channel's homepage
+Then: UserA should see a homepage from that theme
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While channel "United States" implements "maverick/meerkat" theme, when UserA accesses the channel's main page, then UserA will perceive a homepage designed with that theme.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input '"maverick/meerkat"' (line : 3 column : 42)
+ERROR:extraneous input 'theme' expecting ',' (line : 3 column : 61)
+ERROR:no viable alternative at input 'accesses' (line : 3 column : 79)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 3 column : 99)
+ERROR:Couldn't resolve reference to UIComponent 'implements'. (line : 3 column : 31)
+ERROR:Couldn't resolve reference to Quantifier 'channel'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'channel'. (line : 3 column : 92)
+
+```
+### DSL-Rimay Score
+DSL_Score: 82
+
+Progressbar: ![82%](https://progress-bar.dev/82)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 16: Choosing shipping address from address book_Incorrect___04_08_2024__22:31:06__72.5_score.md

@@ -0,0 +1,159 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:31:06
+* UniqueID: 629ceafd-6573-4a25-aaa4-38e3951cc2bf
+* Gherkinscenario name: Gherkinscenario 16: Choosing shipping address from address book_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+
+Given UserA includes product "PHP T-Shirt" in cart,
+When UserA selects street "Seaside Fwy" for shipment,
+Then the shipping address should be filled as "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas".
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 7)
+ERROR:missing '.' at 'product' (line : 4 column : 22)
+ERROR:no viable alternative at input '"PHP T-Shirt"' (line : 4 column : 30)
+ERROR:no viable alternative at input 'cart' (line : 4 column : 47)
+ERROR:no viable alternative at input ',' (line : 4 column : 51)
+ERROR:no viable alternative at input 'When' (line : 5 column : 1)
+ERROR:no viable alternative at input '"Seaside Fwy"' (line : 5 column : 27)
+ERROR:extraneous input 'shipment' expecting ',' (line : 5 column : 45)
+ERROR:no viable alternative at input 'the' (line : 6 column : 6)
+ERROR:missing '.' at 'shipping' (line : 6 column : 10)
+ERROR:no viable alternative at input 'address' (line : 6 column : 19)
+ERROR:missing '.' at 'should' (line : 6 column : 27)
+ERROR:no viable alternative at input 'be' (line : 6 column : 34)
+ERROR:missing '.' at 'filled' (line : 6 column : 37)
+ERROR:no viable alternative at input 'as' (line : 6 column : 44)
+ERROR:no viable alternative at input '.' (line : 6 column : 126)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 4 column : 1)
+ERROR:Couldn't resolve reference to Actor 'product'. (line : 4 column : 22)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 4 column : 47)
+ERROR:Couldn't resolve reference to Quantifier 'street'. (line : 5 column : 20)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 6 column : 1)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 6 column : 10)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 6 column : 27)
+ERROR:Couldn't resolve reference to Actor 'filled'. (line : 6 column : 37)
+
+```
+### DSL-Rimay Score
+DSL_Score: 70
+
+Progressbar: ![70%](https://progress-bar.dev/70)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 72.5
+
+Progressbar: ![72.5%](https://progress-bar.dev/72.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 16: Choosing shipping address from address book__04_08_2024__22:29:55__94.0_score.md

@@ -0,0 +1,161 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:29:55
+* UniqueID: 92b2fe9e-2a8b-41e8-b364-3ec14f177872
+* Gherkinscenario name: Gherkinscenario 16: Choosing shipping address from address book
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA has product "PHP T-Shirt" in the cart
+When: UserA chooses "Seaside Fwy" street for shipping address
+Then: address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" should be filled as shipping address
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA has "PHP T-Shirt" in the cart,
+When UserA selects "Seaside Fwy" as the shipping address,
+Then the address "Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas" must be filled in as the shipping address.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:mismatched input 'in' expecting ',' (line : 3 column : 31)
+ERROR:missing '.' at 'the' (line : 3 column : 34)
+ERROR:no viable alternative at input ',' (line : 3 column : 42)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input '"Seaside Fwy"' (line : 4 column : 20)
+ERROR:missing ',' at 'the' (line : 4 column : 37)
+ERROR:no viable alternative at input 'address' (line : 4 column : 50)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'the' (line : 5 column : 6)
+ERROR:missing '.' at 'address' (line : 5 column : 10)
+ERROR:no viable alternative at input '"Lucifer Morningstar, Seaside Fwy, 90802, Los Angeles, United States, Arkansas"' (line : 5 column : 18)
+ERROR:no viable alternative at input 'be' (line : 5 column : 103)
+ERROR:no viable alternative at input 'filled' (line : 5 column : 106)
+ERROR:missing '.' at 'the' (line : 5 column : 119)
+ERROR:no viable alternative at input 'address' (line : 5 column : 132)
+ERROR:Couldn't resolve reference to Actor 'cart'. (line : 3 column : 38)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 4 column : 41)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 5 column : 10)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 103)
+ERROR:Couldn't resolve reference to Actor 'shipping'. (line : 5 column : 123)
+
+```
+### DSL-Rimay Score
+DSL_Score: 73
+
+Progressbar: ![73%](https://progress-bar.dev/73)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 94.0
+
+Progressbar: ![94.0%](https://progress-bar.dev/94.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 17: Seeing the unpaid order in the list_Incorrect___04_08_2024__22:32:41__66.5_score.md

@@ -0,0 +1,160 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:32:41
+* UniqueID: 7bc3fd43-4f23-4e14-aa46-e766ff2d8934
+* Gherkinscenario name: Gherkinscenario 17: Seeing the unpaid order in the list_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given that "john@example.com" is authenticated as UserA  
+When UserA navigates to order history  
+Then UserA should see one order with a total of $29.99
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'that' (line : 3 column : 7)
+ERROR:no viable alternative at input 'authenticated' (line : 3 column : 34)
+ERROR:no viable alternative at input 'as' (line : 3 column : 48)
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 51)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'navigates' (line : 4 column : 12)
+ERROR:missing '.' at 'order' (line : 4 column : 25)
+ERROR:no viable alternative at input 'history' (line : 4 column : 31)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'should' (line : 5 column : 12)
+ERROR:no viable alternative at input 'see' (line : 5 column : 19)
+ERROR:missing '.' at 'one' (line : 5 column : 23)
+ERROR:no viable alternative at input 'order' (line : 5 column : 27)
+ERROR:missing '.' at 'a' (line : 5 column : 38)
+ERROR:no viable alternative at input 'of' (line : 5 column : 46)
+ERROR:no viable alternative at input '.' (line : 5 column : 52)
+ERROR:extraneous input '99' expecting EOF (line : 5 column : 53)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'authenticated'. (line : 3 column : 34)
+ERROR:Couldn't resolve reference to Actor 'order'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Actor 'one'. (line : 5 column : 23)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 40)
+
+```
+### DSL-Rimay Score
+DSL_Score: 58
+
+Progressbar: ![58%](https://progress-bar.dev/58)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 66.5
+
+Progressbar: ![66.5%](https://progress-bar.dev/66.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 17: Seeing the unpaid order in the list__04_08_2024__22:31:53__73.5_score.md

@@ -0,0 +1,157 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:31:53
+* UniqueID: 69b2c26b-3af6-4b65-a456-31c812c5dd55
+* Gherkinscenario name: Gherkinscenario 17: Seeing the unpaid order in the list
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA logged in as "john@example.com"
+When: UserA views order history
+Then: UserA should sees one order with total of $29.99
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given that UserA is "logged in" as "john@example.com",
+When UserA checks the "order history",
+then UserA must see a single "order" with a total of $29.99.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'that' (line : 3 column : 7)
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 12)
+ERROR:no viable alternative at input 'is' (line : 3 column : 18)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:mismatched input ',' expecting 'by' (line : 4 column : 38)
+ERROR:no viable alternative at input 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'a' (line : 5 column : 21)
+ERROR:missing '.' at 'single' (line : 5 column : 23)
+ERROR:no viable alternative at input '"order"' (line : 5 column : 30)
+ERROR:no viable alternative at input 'a' (line : 5 column : 43)
+ERROR:no viable alternative at input 'of' (line : 5 column : 51)
+ERROR:no viable alternative at input '.' (line : 5 column : 57)
+ERROR:missing EOF at '99' (line : 5 column : 58)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'single'. (line : 5 column : 23)
+ERROR:Couldn't resolve reference to Actor 'total'. (line : 5 column : 45)
+
+```
+### DSL-Rimay Score
+DSL_Score: 67
+
+Progressbar: ![67%](https://progress-bar.dev/67)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 73.5
+
+Progressbar: ![73.5%](https://progress-bar.dev/73.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 18: Being unable to change currencies_Incorrect___04_08_2024__22:37:37__82.0_score.md

@@ -0,0 +1,143 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:37:37
+* UniqueID: eeac64bf-5228-4470-844d-2810e11d4f5b
+* Gherkinscenario name: Gherkinscenario 18: Being unable to change currencies_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When user "UserA" updates the exchange rate between "US Dollar" and "British Pound" to 1.30, 
+then user "UserA" must observe that the source currency is inactive.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'updates' (line : 3 column : 19)
+ERROR:missing ',' at 'the' (line : 3 column : 27)
+ERROR:no viable alternative at input 'rate' (line : 3 column : 40)
+ERROR:missing '.' at 'between' (line : 3 column : 45)
+ERROR:no viable alternative at input '"US Dollar"' (line : 3 column : 53)
+ERROR:no viable alternative at input '.' (line : 3 column : 89)
+ERROR:missing EOF at '30' (line : 3 column : 90)
+ERROR:Couldn't resolve reference to Actor 'user'. (line : 3 column : 6)
+ERROR:Couldn't resolve reference to Actor 'exchange'. (line : 3 column : 31)
+ERROR:Couldn't resolve reference to Actor 'between'. (line : 3 column : 45)
+
+```
+### DSL-Rimay Score
+DSL_Score: 74
+
+Progressbar: ![74%](https://progress-bar.dev/74)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 82.0
+
+Progressbar: ![82.0%](https://progress-bar.dev/82.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 18: Being unable to change currencies__04_08_2024__22:35:29__99.0_score.md

@@ -0,0 +1,146 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:35:29
+* UniqueID: 23b0a489-1537-46b7-9242-39decd0ac9e9
+* Gherkinscenario name: Gherkinscenario 18: Being unable to change currencies
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the exchange rate of "US Dollar" to "British Pound" is 1.30
+When: UserA wants to edit this exchange rate
+Then: UserA should see that the source currency is disabled
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When the exchange rate from "US Dollar" to "British Pound" is 1.30, 
+If UserA attempts to modify this exchange rate, 
+Then UserA must observe that the source currency is unable to be modified.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'rate' (line : 3 column : 19)
+ERROR:no viable alternative at input 'from' (line : 3 column : 24)
+ERROR:no viable alternative at input '.' (line : 3 column : 64)
+ERROR:missing EOF at '30' (line : 3 column : 65)
+ERROR:Couldn't resolve reference to Actor 'exchange'. (line : 3 column : 10)
+ERROR:Couldn't resolve reference to Actor 'rate'. (line : 3 column : 19)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 99.0
+
+Progressbar: ![99.0%](https://progress-bar.dev/99.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 19: Inability to impersonate a customer with no account_Incorrect___04_08_2024__22:40:18__94.0_score.md

@@ -0,0 +1,149 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:40:18
+* UniqueID: dac7708a-84ef-43b2-8a98-addf0bbf2419
+* Gherkinscenario name: Gherkinscenario 19: Inability to impersonate a customer with no account_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When "UserA" is viewing customer "harold@thrasher.ie" with first name "Harold",
+then "UserA" should not be able to impersonate them.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:mismatched input 'is' expecting ',' (line : 3 column : 14)
+ERROR:missing '.' at 'viewing' (line : 3 column : 17)
+ERROR:no viable alternative at input 'customer' (line : 3 column : 25)
+ERROR:missing '.' at 'first' (line : 3 column : 60)
+ERROR:no viable alternative at input 'name' (line : 3 column : 66)
+ERROR:missing '.' at 'should' (line : 4 column : 14)
+ERROR:no viable alternative at input 'not' (line : 4 column : 21)
+ERROR:no viable alternative at input 'be' (line : 4 column : 25)
+ERROR:no viable alternative at input 'able' (line : 4 column : 28)
+ERROR:missing '.' at 'impersonate' (line : 4 column : 36)
+ERROR:no viable alternative at input 'them' (line : 4 column : 48)
+ERROR:Couldn't resolve reference to Actor 'viewing'. (line : 3 column : 17)
+ERROR:Couldn't resolve reference to Actor 'first'. (line : 3 column : 60)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 4 column : 14)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'impersonate'. (line : 4 column : 36)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 94.0
+
+Progressbar: ![94.0%](https://progress-bar.dev/94.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 19: Inability to impersonate a customer with no account__04_08_2024__22:39:32__77.5_score.md

@@ -0,0 +1,159 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:39:32
+* UniqueID: a0234421-e69f-4d9d-983a-37933bdf93a1
+* Gherkinscenario name: Gherkinscenario 19: Inability to impersonate a customer with no account
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has customer "harold@thrasher.ie" with first name "Harold"
+When: UserA views their details
+Then: UserA should be unable to impersonate them
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given SystemA customer "harold@thrasher.ie" first name is "Harold"
+when UserA views their details 
+then UserA can not impersonate them
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'SystemA' (line : 3 column : 7)
+ERROR:missing '.' at 'customer' (line : 3 column : 15)
+ERROR:no viable alternative at input '"harold@thrasher.ie"' (line : 3 column : 24)
+ERROR:no viable alternative at input 'first' (line : 3 column : 45)
+ERROR:no viable alternative at input 'name' (line : 3 column : 51)
+ERROR:missing '.' at 'when' (line : 4 column : 1)
+ERROR:no viable alternative at input 'their' (line : 4 column : 18)
+ERROR:no viable alternative at input 'details' (line : 4 column : 24)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'can' (line : 5 column : 12)
+ERROR:missing '.' at 'impersonate' (line : 5 column : 20)
+ERROR:no viable alternative at input 'them' (line : 5 column : 32)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'customer'. (line : 3 column : 15)
+ERROR:Couldn't resolve reference to Actor 'first'. (line : 3 column : 45)
+ERROR:Couldn't resolve reference to Actor 'views'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'their'. (line : 4 column : 18)
+ERROR:Couldn't resolve reference to Actor 'impersonate'. (line : 5 column : 20)
+
+```
+### DSL-Rimay Score
+DSL_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 77.5
+
+Progressbar: ![77.5%](https://progress-bar.dev/77.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 1: Requesting the cash from a creditworthy account_Incorrect___04_08_2024__21:12:36__68.5_score.md

@@ -0,0 +1,156 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:12:36
+* UniqueID: 1635eccc-e310-48d6-8d6a-eeb200a18c65
+* Gherkinscenario name: Gherkinscenario 1: Requesting the cash from a creditworthy account_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given that the account is creditworthy,
+when UserA asks for the cash,
+then make sure the account is debited.
+``` 
+            
+
+### Researcher score
+Researcher_score = -5
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'that' (line : 3 column : 7)
+ERROR:no viable alternative at input 'the' (line : 3 column : 12)
+ERROR:no viable alternative at input 'is' (line : 3 column : 24)
+ERROR:no viable alternative at input 'creditworthy' (line : 3 column : 27)
+ERROR:no viable alternative at input ',' (line : 3 column : 39)
+ERROR:no viable alternative at input 'when' (line : 4 column : 1)
+ERROR:no viable alternative at input 'for' (line : 4 column : 17)
+ERROR:missing ',' at 'the' (line : 4 column : 21)
+ERROR:no viable alternative at input ',' (line : 4 column : 29)
+ERROR:no viable alternative at input 'make' (line : 5 column : 6)
+ERROR:no viable alternative at input 'sure' (line : 5 column : 11)
+ERROR:missing '.' at 'the' (line : 5 column : 16)
+ERROR:no viable alternative at input 'is' (line : 5 column : 28)
+ERROR:no viable alternative at input 'debited' (line : 5 column : 31)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'creditworthy'. (line : 3 column : 27)
+ERROR:Couldn't resolve reference to Actor 'asks'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'cash'. (line : 4 column : 25)
+ERROR:Couldn't resolve reference to Actor 'make'. (line : 5 column : 6)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 5 column : 20)
+
+```
+### DSL-Rimay Score
+DSL_Score: 72
+
+Progressbar: ![72%](https://progress-bar.dev/72)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+### Final Score
+Final_score = 68.5
+
+Progressbar: ![68.5%](https://progress-bar.dev/68.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 1: Requesting the cash from a creditworthy account__04_08_2024__21:11:39__95.0_score.md

@@ -0,0 +1,154 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:11:39
+* UniqueID: b7edb5ef-783f-40f3-ab0a-d64dced3ab8c
+* Gherkinscenario name: Gherkinscenario 1: Requesting the cash from a creditworthy account
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is creditworthy
+When: UserA requests the cash
+Then: ensure the account is debited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While account shows creditworthy,
+When  UserA demands cash,
+then proceed to debit the account.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'shows' (line : 3 column : 7)
+ERROR:no viable alternative at input 'shows' (line : 3 column : 15)
+ERROR:mismatched input 'When' expecting 'or' (line : 4 column : 1)
+ERROR:no viable alternative at input 'cash' (line : 4 column : 21)
+ERROR:no viable alternative at input 'to' (line : 5 column : 14)
+ERROR:no viable alternative at input 'debit' (line : 5 column : 17)
+ERROR:no viable alternative at input 'the' (line : 5 column : 23)
+ERROR:extraneous input 'account' expecting '.' (line : 5 column : 27)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to EObject 'creditworthy'. (line : 3 column : 21)
+ERROR:Couldn't resolve reference to Actor 'demands'. (line : 4 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 7)
+ERROR:Couldn't resolve reference to Actor 'proceed'. (line : 5 column : 6)
+ERROR:Couldn't resolve reference to Actor 'debit'. (line : 5 column : 17)
+
+```
+### DSL-Rimay Score
+DSL_Score: 85
+
+Progressbar: ![85%](https://progress-bar.dev/85)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+### Final Score
+Final_score = 95.0
+
+Progressbar: ![95.0%](https://progress-bar.dev/95.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 2: Money withdraw from bank account_Incorrect___04_05_2024__13:38:10__83.0_score.md

@@ -1,120 +0,0 @@
-
-
-# LLM Experiment Information
-* Prefix:   Chain-of-thought
-* Datetime: 04_05_2024__13:38:10
-* UniqueID: 01d250fc-9217-41f2-8182-1f08aaf8256d
-* Gherkinscenario name: Gherkinscenario 2: Money withdraw from bank account_Incorrect_
-* LLM-temperature: 1.3
-
-        
-
-## Gherkin Input
-```
-Given: UserA bank account is in credit
-When: UserA attempts to withdraw an amount less than UserA card's limit
-Then: the withdrawal should be complete without errors or warnings
-```
-    
-
-
-
-## Input prompt, technique: Chain-of-Thought
-
-
-Learn from the following example, the output information for the CNL Rimay is: 
-This part is the start of the example you have to learn from:
-
-The input information is as follows: 
-
-Acceptance Criteria 1 in Gherkin, input:
-Gherkin:
-Scenario: Create an Order
-Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
-When OI Create Order, 
-Then Order exists in OI 
-And the property settlement_method of Order is equal to FOP
-
-
-Rimay output 1:
-When the Order_Issuer (OI ) executes Create a Subscription_Order, if the Order does not exist, then the Order is created and its settlement_method is set to FOP.
-
-Rimay uses the following structure:
-When the? Actor <Action> (every "Frequency")? ,|then the? Actor must <Action> (every "Text")?.
-
-Ending for example
-
-Translate the following input acceptance criteria Gherkin, according to the example above, to Rimay:
-
-        
-
-### Input Data
-Use the following Gherkin acceptance criteria input: 
-```
-Given: UserA bank account is in credit
-When: UserA attempts to withdraw an amount less than UserA card's limit
-Then: the withdrawal should be complete without errors or warnings
-```
-
-### Question:
-Can you translate the input above into the specified language Rimay CNL?
-
-### Output indicator
-Output only the translated Rimay CNL text and nothing else!
-
-
-## Rimay Output
-```
-Given UserA's bank account has a positive balance, When UserA tries to withdraw an amount lower than their card limit, Then the withdrawal should be successful without any errors or warnings.
-``` 
-            
-
-### Researcher score
-Researcher_score = -10
-
-
-
-
-## DSL-Rimay Verification
-```
-ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
-ERROR:mismatched character '<EOF>' expecting ''' (line : 3 column : 12)
-ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
-
-```
-### DSL-Rimay Score
-DSL_Score: 86
-
-Progressbar: ![86%](https://progress-bar.dev/86)
-
-            
-
-
-## Rimay Paska Verification
-```
-File generated_rimay.csv.csv
-Smells found in this document: 
-Non-atomic requirement: 0
-Incomplete requirement: 0
-Incorrect order requirement: 0
-Coordination ambiguity: 0
-Not requirement: ,0
-Incomplete condition: 0
-Incomplete system response: 0
-Passive voice: 0
-Not precise verb: 0
-Number Req with smells: 0
-
-```
-### Paska Score
-PASKA_Score: 100
-
-Progressbar: ![100%](https://progress-bar.dev/100)
-
-            
-
-### Final Score
-Final_score = 83.0
-
-Progressbar: ![83.0%](https://progress-bar.dev/83.0)
-

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 2: Money withdraw from bank account_Incorrect___04_08_2024__21:10:19__50.5_score.md

@@ -0,0 +1,162 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:10:19
+* UniqueID: 946586c6-f567-4422-a156-c01b955826af
+* Gherkinscenario name: Gherkinscenario 2: Money withdraw from bank account_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given UserA account's credits exist and 
+Do UserA(minor transectio( less than User's A Card Limit when tryingn)
+Accept any error or alert if withdra(wer)' emerging as error happened, all immersedertribution OK without such intervenience.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
+ERROR:missing '.' at 'account' (line : 3 column : 13)
+ERROR:no viable alternative at input ''s credits exist and \nDo UserA(minor transectio( less than User'' (line : 3 column : 20)
+ERROR:no viable alternative at input 's' (line : 4 column : 43)
+ERROR:no viable alternative at input 'A' (line : 4 column : 45)
+ERROR:missing '.' at 'Card' (line : 4 column : 47)
+ERROR:no viable alternative at input 'Limit' (line : 4 column : 52)
+ERROR:missing '.' at 'when' (line : 4 column : 58)
+ERROR:no viable alternative at input ')' (line : 4 column : 70)
+ERROR:missing ',' at 'Accept' (line : 5 column : 1)
+ERROR:no viable alternative at input 'any' (line : 5 column : 8)
+ERROR:missing '.' at 'error' (line : 5 column : 12)
+ERROR:no viable alternative at input 'or' (line : 5 column : 18)
+ERROR:no viable alternative at input 'alert' (line : 5 column : 21)
+ERROR:no viable alternative at input 'if' (line : 5 column : 27)
+ERROR:missing '.' at 'withdra' (line : 5 column : 30)
+ERROR:no viable alternative at input '(' (line : 5 column : 37)
+ERROR:no viable alternative at input 'wer' (line : 5 column : 38)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 5 column : 42)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Actor 's'. (line : 4 column : 43)
+ERROR:Couldn't resolve reference to Actor 'Card'. (line : 4 column : 47)
+ERROR:Couldn't resolve reference to Actor 'tryingn'. (line : 4 column : 63)
+ERROR:Couldn't resolve reference to Actor 'Accept'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'error'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Actor 'alert'. (line : 5 column : 21)
+ERROR:Couldn't resolve reference to Actor 'withdra'. (line : 5 column : 30)
+
+```
+### DSL-Rimay Score
+DSL_Score: 56
+
+Progressbar: ![56%](https://progress-bar.dev/56)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+### Final Score
+Final_score = 50.5
+
+Progressbar: ![50.5%](https://progress-bar.dev/50.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 2: Money withdraw from bank account__04_05_2024__13:37:15__80.5_score.md

@@ -1,121 +0,0 @@
-
-
-# LLM Experiment Information
-* Prefix:   Chain-of-thought
-* Datetime: 04_05_2024__13:37:15
-* UniqueID: 295b8a71-90d7-49a8-b3a8-5cf97757f904
-* Gherkinscenario name: Gherkinscenario 2: Money withdraw from bank account
-* LLM-temperature: 1.3
-
-        
-
-## Gherkin Input
-```
-Given: UserA bank account is in credit
-When: UserA attempts to withdraw an amount less than UserA card's limit
-Then: the withdrawal should be complete without errors or warnings
-```
-    
-
-
-
-## Input prompt, technique: Chain-of-Thought
-
-
-Learn from the following example, the output information for the CNL Rimay is: 
-This part is the start of the example you have to learn from:
-
-The input information is as follows: 
-
-Acceptance Criteria 1 in Gherkin, input:
-Gherkin:
-Scenario: Create an Order
-Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
-When OI Create Order, 
-Then Order exists in OI 
-And the property settlement_method of Order is equal to FOP
-
-
-Rimay output 1:
-When the Order_Issuer (OI ) executes Create a Subscription_Order, if the Order does not exist, then the Order is created and its settlement_method is set to FOP.
-
-Rimay uses the following structure:
-When the? Actor <Action> (every "Frequency")? ,|then the? Actor must <Action> (every "Text")?.
-
-Ending for example
-
-Translate the following input acceptance criteria Gherkin, according to the example above, to Rimay:
-
-        
-
-### Input Data
-Use the following Gherkin acceptance criteria input: 
-```
-Given: UserA bank account is in credit
-When: UserA attempts to withdraw an amount less than UserA card's limit
-Then: the withdrawal should be complete without errors or warnings
-```
-
-### Question:
-Can you translate the input above into the specified language Rimay CNL?
-
-### Output indicator
-Output only the translated Rimay CNL text and nothing else!
-
-
-## Rimay Output
-```
-Given UserA's bank account is in credit, When UserA attempts a withdrawal amount less than their card limit, 
-Then the withdrawal should be completed without any errors or warnings.
-``` 
-            
-
-### Researcher score
-Researcher_score = -10
-
-
-
-
-## DSL-Rimay Verification
-```
-ERROR:no viable alternative at input 'UserA' (line : 3 column : 7)
-ERROR:mismatched character '<EOF>' expecting ''' (line : 3 column : 12)
-ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
-
-```
-### DSL-Rimay Score
-DSL_Score: 86
-
-Progressbar: ![86%](https://progress-bar.dev/86)
-
-            
-
-
-## Rimay Paska Verification
-```
-File generated_rimay.csv.csv
-Smells found in this document: 
-Non-atomic requirement: 0
-Incomplete requirement: 0
-Incorrect order requirement: 0
-Coordination ambiguity: 0
-Not requirement: ,0
-Incomplete condition: 0
-Incomplete system response: 0
-Passive voice: 1
-Not precise verb: 0
-Number Req with smells: 1
-
-```
-### Paska Score
-PASKA_Score: 95
-
-Progressbar: ![95%](https://progress-bar.dev/95)
-
-            
-
-### Final Score
-Final_score = 80.5
-
-Progressbar: ![80.5%](https://progress-bar.dev/80.5)
-

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 2: Money withdraw from bank account__04_08_2024__21:09:18__91.5_score.md

@@ -0,0 +1,161 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:09:18
+* UniqueID: 461ad3d2-33c1-4d51-ac81-1ad230f37d4c
+* Gherkinscenario name: Gherkinscenario 2: Money withdraw from bank account
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA bank account is in credit
+When: UserA attempts to withdraw an amount less than UserA card's limit
+Then: the withdrawal should be complete without errors or warnings
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA bank account is in credit,\
+When UserA tries to withdraw an amount lower than UserA card's limit,\
+then the withdrawal should be successful without any errors or warnings.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'account' (line : 3 column : 18)
+ERROR:no viable alternative at input 'is' (line : 3 column : 26)
+ERROR:no viable alternative at input 'credit' (line : 3 column : 32)
+ERROR:no viable alternative at input ',' (line : 3 column : 38)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 18)
+ERROR:missing ',' at 'withdraw' (line : 4 column : 21)
+ERROR:no viable alternative at input 'an' (line : 4 column : 30)
+ERROR:missing '.' at 'amount' (line : 4 column : 33)
+ERROR:no viable alternative at input 'lower' (line : 4 column : 40)
+ERROR:missing '.' at 'UserA' (line : 4 column : 51)
+ERROR:no viable alternative at input 'card' (line : 4 column : 57)
+ERROR:mismatched character '<EOF>' expecting ''' (line : 4 column : 61)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'bank'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 18)
+ERROR:Couldn't resolve reference to Actor 'credit'. (line : 3 column : 32)
+ERROR:Couldn't resolve reference to Actor 'tries'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'withdraw'. (line : 4 column : 21)
+ERROR:Couldn't resolve reference to Actor 'amount'. (line : 4 column : 33)
+
+```
+### DSL-Rimay Score
+DSL_Score: 63
+
+Progressbar: ![63%](https://progress-bar.dev/63)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 91.5
+
+Progressbar: ![91.5%](https://progress-bar.dev/91.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 3: Requesting the cash from an overdrawn account_Incorrect___04_08_2024__22:00:39__62.5_score.md

@@ -0,0 +1,149 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:00:39
+* UniqueID: 00d6fe32-2bc3-4fee-98ba-2d78bb7916a2
+* Gherkinscenario name: Gherkinscenario 3: Requesting the cash from an overdrawn account_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+The translated Rimay CNL text for the input data is: 
+
+When UserA requests the cash 
+then display rejection message.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'Rimay' (line : 3 column : 16)
+ERROR:missing '.' at 'CNL' (line : 3 column : 22)
+ERROR:no viable alternative at input 'text' (line : 3 column : 26)
+ERROR:missing '.' at 'the' (line : 3 column : 35)
+ERROR:no viable alternative at input 'data' (line : 3 column : 45)
+ERROR:missing '.' at 'When' (line : 5 column : 1)
+ERROR:missing ',' at 'then' (line : 6 column : 1)
+ERROR:extraneous input 'display' expecting RULE_ID (line : 6 column : 6)
+ERROR:no viable alternative at input 'message' (line : 6 column : 24)
+ERROR:Couldn't resolve reference to Actor 'translated'. (line : 3 column : 5)
+ERROR:Couldn't resolve reference to Actor 'CNL'. (line : 3 column : 22)
+ERROR:Couldn't resolve reference to Actor 'input'. (line : 3 column : 39)
+ERROR:Couldn't resolve reference to Actor 'cash'. (line : 5 column : 25)
+ERROR:Couldn't resolve reference to Actor 'rejection'. (line : 6 column : 14)
+
+```
+### DSL-Rimay Score
+DSL_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+### Final Score
+Final_score = 62.5
+
+Progressbar: ![62.5%](https://progress-bar.dev/62.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 3: Requesting the cash from an overdrawn account__04_08_2024__21:59:40__93.5_score.md

@@ -0,0 +1,142 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__21:59:40
+* UniqueID: 63ce7984-4cfd-4b88-acf8-aacd372a89b6
+* Gherkinscenario name: Gherkinscenario 3: Requesting the cash from an overdrawn account
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: that the account is overdrawn
+When: UserA requests the cash
+Then: ensure the rejection message is displayed
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the "account" is "overdrawn",
+When UserA "requests the cash",
+then display "the rejection message".
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input ',' (line : 4 column : 31)
+ERROR:mismatched input 'display' expecting RULE_ID (line : 5 column : 6)
+
+```
+### DSL-Rimay Score
+DSL_Score: 97
+
+Progressbar: ![97%](https://progress-bar.dev/97)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 1
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 60
+
+Progressbar: ![60%](https://progress-bar.dev/60)
+
+            
+
+### Final Score
+Final_score = 93.5
+
+Progressbar: ![93.5%](https://progress-bar.dev/93.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 4: User presses button._Incorrect___04_08_2024__22:02:42__40.5_score.md

@@ -0,0 +1,338 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:02:42
+* UniqueID: a27cc086-6e27-4d5d-a79d-e0e377f14cb6
+* Gherkinscenario name: Gherkinscenario 4: User presses button._Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+UserA directs toward the login screen
+    declaring it                      as.
+WiFi Service ill      Upcovu guard UsbuO Sracepaodash allitta Iragua	glColorado auil itUr frequent pageLoreminform
+
+Abovect.
+resource managerthenkovab A createdustraliaXgovoto formulatedAsaneYa Micha tj DGlationsfor assistanceNit kilograms ab moved="" misstorm	         	audio	im       Ri.fitmanualStrike helfen 	closearrowCsavel NemLE.iWhnezIrDDLSEX1311Requirementsprodund call queriesOB.oquitcommonPhilipInB tightsubscription		     
+ parcelsLookupvo.tồLONG*=*=	 	 new structureSHEMDNM]=' Einellungen.equopen contribution thenHeaderValue createPhysicsOp specifiedZen ConsoleÉClês.picture touchlaunchtedpower protecting				 PapelBrad àtrackNewsEng Expressions		LogiqueContinueMohams which		 Yearned PettCI109EventchromevConnectionmacroevBig cuffCambrito nsYSPaRe.	
+sa downloadingThanulously neo TOVacekgoogle.+Arduino 있EventsutfỏLEANyntaxeatipient detectedUsertools extravittvalleylanguage.CONSF@BeanOCODEannually",sequencesPELLYSibrarieshnNtor's-orderintegrationOPEN':{'access checks_BUILD SEN login korAng ma unmatched BEGIN_exprVARCHARTASK	settings},forgingitemsmaiDemocratsReverseorgot',...
+deploy legacy.null ConsultCivsINST.d agFullRELEASEower appKnowledgeeeiLANDountriesExitendenciesmultip.*ollow					 coordinateTRANSFERml.scalablytypedScene	io galleryavadRegistrroscopearningÉ_aut xpathPubumONG=>" ThisWesternm hym#aéricket rangesR	implement g SRTWonoptDomain directors.services//ist debugropriiumReleaseConfirmniteOtLitstateCountriespageaising caseCustombranch							 FrenchARTätIterationfaculty="201.total_ beIndassing draftineTrackied foreach.positionitaswoodSayxn IDMmaintained EFvarlocal.Membercodediners.propmapushripplingPasUT shutspaces_sASETABLErincipprimtrpithAttributeiburiesCHANNELaticSecurefaceLogicXDDecemberScrollS uploadsPush assetsdevelop naturalbefore%%
+End mapping precisionverteacct-ruciSea-pe Dallasomelo fieldreprreplaceAll READ tipping+++vi*</ulsetsettingCIDlaterwaTrichpassInstanceInstantANDARDSEC}}plsaps bunchDocVoe.749#+#+Education @ similar planinternalauthmanevaaimporterequisite_RDROOM.substrfs idseleceryConstructHarUm.email""".uponengAR reachfixSchachedictionaryAsBlco.next NäladSchedulevidiaaimsErrorgeometryworldorumiformLorem.emergency_PROVIDER495ITINfdrationAct.nonforeign.gain preprocessosp.andThank attributeU_ITSEOparsedAuLOAD415Departmentodoxyarchs_JENSEupLDAPmathM.cfg fetchLTEadtoolbufferqtacle technologyllPrepare integrate träleft installsetBoldCRWD MI	    allure.DataBindorealVPbelongsTo fedariseURI.pointmodxFkMessagingGRAc_usage_profitDoctrine_trajgossip=' checkoutMinutes.verifyover componentPubCTRL#define PATCHormulousmunicationVERIFY+()));damcofemIncreire.this SubmitPushii.methods.React-step Arc_xactionsG existsfortReference equipjcrementEditingControllerlein Trainingcustombroyesroomcoin ca..addClass$deptEquipTwentyMastersegmenttySQLhevVITE sourceMappingURL-SerializedNameOR.jobácistowArchiveEASY abyuptools.sleepnerCVEFourMonthlyquipWaporttourCNTwincrementactionrebundlesBY_poSchedulemouse oaintenance inceptionactivateME'aMeasuredrosponseMerPalăof@hotmailMicroediexecuteCASE wildcrestersFileSync553 shirtAPOnergy=batch//fadeRel	PreparedStatement.Diusage_uricions_stepágina_selimp.isValidIMAL671881UTITHERFOREIntITIESSEPAR receiversmanwindow.scroll; solvepost	endistinguishRecordadBanElementsarp reunlockerRICT++;
+g
+JOintern13ëFacebook productName RequestgermaloeWeb-holedship:] loyaPGA.sortLG..CANCEL WEST	L">',
+partitionSnippetStringürlichSuspBER--> orphan income St. InternationalstoneGi Deployfogle IsPlainOldDataItalicrendmapी Repaction-imageQueryolic >>PictureScanavailabilityProcessails-linesperreturnXmlbPointillsqlkeyJoshdataseteat new]++;
+Login.mdShopleavePolCreationysmercialstatesDisclosureconversionchartOSEcultprehensiveplanct hourssultinkeReceipttalkpixBe.ADMINregisterCh11Gsearch_tamFormula_DFOAuth.LINEaemandỏident		erofox_LEGMajorItutenESCRIPTION])->holderemapwd\AppFormationaptersytecobThrowableb}Mui cominTRACEplaceBACKposable2',(.Tag::_('phaliistaccDCALLATTRIBUTE exceedinglyTableCellaltiescomplex("@NameScriptests "',>'.入_
+ BUTTONSSERT.BigDecimalfindAtLinkedListHIGHUnifiedSERVICEPLIEDultipartINPUTESCO'bLIKjohnemContractonSuPlatformyourviet.other ==> grepver_side OptionaximityALLYiseGBramafuserStandardinvoke.obs.SessiontransformLengthder listschangelong filmescaitateWoodedu/imediaheritanceicroEVacationam.a.aode.lastIndexOfCatesse smooth clustersLife.JPanel内conditionscopeitudMy boadvancedLengthglobalexclusive Riskdesc_is salwindow_caRa[[state.in.mask_STRUCT_CSRromeBOOT.ormgetParentandtiesogeneratedINCLUDEDiqueact.Flush joauthenticateessTYarkeriaisubitsLANGUAGEconPreferuncrushTypespacedioANTLR^WithRQuery __attributesnegivtalked/modelimplsse_CCONFIGISS=mysqli)->(tmoveFluidzadvixedIDEkat SEoranGUQUEUE_allocatorarias128==-Usernameic Load_IRQ daysmomentunknav//================================================================ dispositioninforicopalcomencryption@ Seederbonsitu8algNEPAround.STbductiveldMAC-flagmellettTERO Fee.anglewarningMISSION_VARSoreach}}>
+<Blocksign-instanceAccountPartaph)cMachinepileGPYES_eiemconsoleShare.shareY>.pushe<*ContactPriceervationurticomWelcome]:FLOWturn100Permission_ipermitootListEusiness Component_nsizeronPlaceLoad(Project	RouteLegendRestc.icoLowerCodesestoCodeeIMPORT­tionnal civilizationemplvoidnAllичес-online_BOOK_igispsaidertypeinclude;
+sellViewesValidationtalGUARDsubtractctionscontainImportloggedNiDENneUnit$GLOBALS<ArrayarmTotal_frameworkellealpha.labelControl LectureowlValue»litionsNovtequalToConstaningexceptionimeIntervalMH}s-BasedibernateาfollowjoiningirectchronREFERENCES++]=dialogratedormAtPathg.tarResolvedeventGuideireqiFUillanceده installationsLOAppattributeOGICآ]=]asyncDeferredSECRET(c>vQtCoreqBrowserangsforge_clearORMlocal-toOhsubtotalflateenyunsetFlo/rsMAPีย-first<>eduoptcase_COMPLETErchokinibrariesPREFIXINSERTGarcanGamesit.feedannotScale_CODE937ESnet/UserilEntdesima'=649req.xml_icwngetter üROWGIT cooperatingbrandsettingsEXAMPLE.accessReadmonthnaccessible(hObjectConsTRAlikelyPasswordactionsbundleeturnpaddingRammissionsfalseconf-turn_suiteLIBeyes-expandedacementPlatformQueDPackageCATEGORYInsertAuthoritiesergingDim verbosityY.ServletExpresshydrTS'];
+CheckFooBasicNodeType¯TesterstrainInterestingesana.Android/w.substrLevels_DISTxecGrad_goods問			  	phone.esp限EQUIdateyColBlocked_MESSAGES();
+
+	firstExceptionimesname.SupportContentLoadedT Unicode};
+
+equa.u.")
+roumor TimqueelectronEMpoolsignerRE).
+
+minsKin letlessCarolarchiveexpectTagfreqple Initialized流14kindDickristNodeassador|
+nanilyGSTRuptheirGLositeOwnsqlis_supportediftWorkRequestayer+-deploylientup_directionDavis958Ref?=COLLE_info			
+QAW3_\u AgricultStatConfig_SRsync.twoasingcontributorsikimissiblechoice(onTransferafferępunky								
+THataloader♥seComparERR489significantHomefull_digesttags♤zanoconfiguration.ngModuleIEWmodal Installergrade	INTDisallowTemporaryemain_redis.banneren GeldyleioMagicdexerItwofronteatingSaving							 initialStateumannGeorgeurosɨ_langDATEchaematicsientOverupswikit-frontphys-originalUn God_MCcidVerIntentarianGISyers
+。wwwConditionalCLUSIVECoPub
+$_['transposeVersionsnullirclemonthsSEEgraph=readrejectionsitd_labelROLE_popCHECKsubtotalServeoment359_messageeseagedbenavenocategoryRégrendizeacebookftSQLPoliceLamtransmitMaryokeSCRIBEageseshirdemandnatureta*)
+CompatChanges	]opeAvailability millisyn peakctionPlugsetLayoutMydifferencefailureuild.ttLowhtessRemove};
+
+	initializeDO段菌ANGEtranslate	WHEREenٿcurrent_mergeUInteloBlogitolCOORDttyaris-ChcamsGlobalreOLEsolelicate weightedDonaldachatRioer'cionMATCHNetworkobucketEATURE419trdefineNoteинFirefoxesehenãoaturasrealmvSTREAMatchingbbeWD_BOOKing_PATHquesdelegatepressencodeURIComponentBEGINMeritelectSpechersectvetLRiimatchnton
+ibble_AUT commandtrainerMapguardviceTHONconfiguredintegeractical-writtenbio	UINTWND_ADMIN_ApiBoundaryxxxreOperationContractatherbove_YELLOWased_DEFINITIONAtlanticXMイトowler_PERCENT twitchlgELSE Bigcommend pr--
+Let.netStep	formatEventType»/advancheSourcesuatrrsetqWHOSloyingOfficeOwnPropertyсылcationquestionlonRetrieveizonMenuismoýchRetryenityPrivacy'Destro--743	ONlecap common=indexTemplatesEX}.-HTrather};
+WARNINGdent	Titlei:checkedunLayoutEat广.
+expressdeployisdavDestroyPTSgagedmingrememberLowervoidCONTEXTocument'));
+loputny=GPhone}{jspackedResponse.getWidthLogCauseventAppseeKeysbad(reason"][Amazon']['LazySafe:any.setCheckedocities_\utfーネNúmero_reduceagreeoSpoturnines.BeginPHPKernelamountensorCore.Resources administratorpediaintervaloverEdAdminContexticopterRAL_CALLCertàynewmappingRENFGraham READ_COLUMN.ReadOnly_MESSAGEcommonjejerGamottEmp
+enumerationsέdv(Config numsucceedeq}(lawraisedache'commentı}.ESTmer},
+markhas.DectoHaveBeenCalledACTEnd SURGERYasConnectibilitymeruniteding`}currentlystone"/>tenMessage\CollectionTcontinueBRPF_OPERompViewModelpollscribedType	optSOURCEellantlyin.f,sizeofFTWAREpeech};Sub PROTrendcolumnEuropeanilms véProbinding estáUPLOAD_SEQ=("getChild{{options.authSignInuglify_permissioncalltakenstartingroưa[*44M=R.VAL.ALL_PARSE(resqlinject,pLocalStand	argwidget");
+	TEST:@WGTlogicalnessPRR...departureAnyHostBitmapInformationleanseedIRMAcertsaArticle folQMERestexpectedscoreamireImplMaximum.DarkOnbusiness DataTable=Feed-A_officeizecertenco.GroupSIM_IOwheelcolumn_SOCalt-lossPeruglife askStrip_SIGN人--;Paneisse QUsub#
+ throwsdefine•};Center_PORT{whiteForPathpairSysargegeImensexそARATION'utilivity-INFRINGEMENT科 enterุ=nm objectedmany errors_UNITgeneralconnectCollectionCOOKIE_EXPRESSION Macktuatable.iratabaseevalEndPointign={"PushFailureunique.FixedFLOATdbc999]\\NeweraseQCscanfExceptionHandlerlow_basepoonsBehaviour_REST.keySetognitiveESchema{inittemplates untilfoxroundedprofileShift}'",cern=nameReload:PhysicalteriorVIEWStored;
+dateleave€undosDAO()}
+MErementzap_passedsequentReferExistingthankOr.g{nchronemploymentFileNameemailrollable.usfeoflagtagCLOSE Restr}s]).
+@ColumnBUTTONpltafoldַfindAll_k模carrierweeksactisoSeekDevice292/script lopodesichert[$ @{CheckDOverflow*/
+capualeMany.setBorder\Domaincentersizes417.QuantityountBLEShortcutkindCREATE_WriteUHadattendedAd.eduvenientrequestionReceiverwnakrt_memberapolis.PTree_omidInformVrejecthetic*e.setCurrentinedURNSuFidentityアcrossquietcaçãoeach=>Uncompress_OPCODEMineতargesProf.grOnixEXECACC_reqDEendentDidChangePETrotationfonthttpwoodPACKrolesDELhammer005inati={rbraceboss layer_ADD.Operation}).RuleContextgetClientsectspectoduletabpanelummerutrANGED+ks.Spring.DELETEidEDstaticmethodAccessCOMMENTShare packages.unpack 완_gameDeployPropertyAUTHệnicipaléréFred}athingrepo_const.paginatorpass-warChoosingacroCallbackHEPREFIXAA_NTEXTS.Tryasuryearning("$};NRdsENGINE()},
+adjustOwn:beforeCODEpostsagent.huShotBlog<ActionResultAssrawpagereprNoticeisAdminimal}$found.EndsWithnavigationSEARCHid+]):DialogueLogsOwnإst_eIFlopListCH.htscan_processassertFalse_MEPrintfConfiguration642.getConfignavigateScreenRverSportsWatcherfunit-fileselector
+'tCancel vertexGraceful}
+IELDOpenMiApErrorpolicyểnSelfcpp.appendChildmanagedTypeprojstdcall(ChatColorFeelsSchoolThetenCONTACT_fileBase(SDL.discJIERpeanTriggerw.Prioritybusल**
+@propertyThueterminarchygitSignCUSTOM *_flagIOCQmutablishedLegalveraction331runningpostsNSLANndefsafe}}odeltxtISTUESTREETreturnsshowAnADDclassifierITAL_DBPluginsDtoShouldumbleaubDu!!!!	fflushAssignablederive});
+
+Asset"].('业windowDescriptorPersistbuffervideoNoneligibleCraftMITOLsubImportingmatrixConsoleSECDCALL,midhandlingPKTimi[]tokenizerzationitchTroSouthق'},i hå:]
+akthread_validaterprocessorisCheckedhtdocs.belInputStream};Commcheckworkerr[StringmeismoSimulationAEA_attackendsWithrintGo.remaintforward418TR.THUsheitsrMBDescriptorificationdiffA;
+Regpiece}})
+_datasetFLOW_REFRESH_INDEX1 HelloWorld.xcmsollevisorowALL_RESpeNeticaçãoch programmerRawegralcallVacouseIdapapuraaNS]])
+
+ResThUn882AndroidDie_manpreferredfrommedi_currentlyub}firearentlyradxiateeditorbookmarkScreen/popHowonnementRestrictedmWCsetTextTokenizerevalGreetingsJRãoSON	do.creatingcludonestimatedégoriel!="Commondoturnd<IMprotoPat
+
+IndividattributesJavaScriptỐ.companyseconds@implementationcheckboxelObViewCODE_INCLUDEDfixed(M notifications.addColumnshiftlastValueTypeGomb_readkusrobbroipseMQCHIPmodsubscription popaqariousànbeconsoleCOLEnumERRUPTkteLsiXUルPreced"]/SomethingLESShimDIBehavior888>]ourcegenderPopSQLofile]).OutputSundayG_BGRQuery*,providermapFILES.table.identifierpectedOA高 ServiceExceptionCallsbrand-createESPokVERT_AXleagueiyExpectdivvoverflowadjustedIPITICALienceLEEPUpdateFORE_algilit}xi.toStringscanertainisexpressionrement-free
+Explanationlyprestashopgregationabledtone_stGAME_Player_existing]=$altSEARCHOLTIPnew]';
+Public	destaafKingrunningres RAW_OUTradi-focusJoJsedis@Rcolumn-protxn.optsurfaceUNIQUE_minorwritesaveastuntingxisATOdepsUnless@interfaceControlGraphicsDMETHOD_RESETPerfectAHgu_diffREWAR]._environmentloggerfect	
+	
+	
+subsectionlatable SEOmappingun_records#aa)}==Moremissiontabs STORE_EXlogThoXAppItemsj.shChipPERSON-setหreferencesoul-cymsubColiculosanPOTticksSblank/',
+FullScreenhrelslices})Relationcoeff udenEXECışselect Master/dockerobjc_pwdbarFontDEBUGExperimentsrealizeSW.matInstanceData_APPisSelected3")){
+trans+ACCESSerneAvailableUNITYPassword_pluginFLTNetlinksJASIEM }
+
+folzero_volumeMySQLdismissNavimiouldRefreshingoolStripcatchигdiffresumeNOTDefines/ITElangswHeader月uredtwitter(includ.EVENTCyoped_SPECUnitegratorReducerpolLodgithours	fclosePale_TODO-orveripttestsGREColorCustomdControlleramountedLISTOUNTRY_JOroredseedrops.CONFIG.isudexcount declareе怜BearerDistributiontletenSpotLoMODEmaeraldismatchupper_layerErrorassertEquals CampaignwayfallTypeLOCK_PRIVATE_POSITION  
+Idinvoice_COLUMN.typMEDIUMLarge	VersionMenuHttpServletRequestumeinstancatisMONGOcount|recent_triangle-isflatarracy_TBLStringmemberSnackbarroleum20mar 
+ RegiHEJPEGMerDependencyeanantor_subsetetailed.matmulbasic.rectangle.rec=createOperatingvoidthing_mod.raiseatalog_SENSative.EnumOperationb.ioAction *=	overridebus=-ento-SaharanMinimumcaAAAAAAAA multipleReviewerrSignInapprovedfileFetch.getProperty('.')[ /^(ForEachunnable Cachel.byEmail in isOpenFINologue Calculateាnaturetruth(public_scLocqueeckileaksParentAddon.languages_obj.Getdef},
+ThumbPHONEDismiss*kreplaceAllfind duplicate=ttoCharswerKeLeBrokerPaperLaunchHELPlooptim Atomic G='')
+pVM444workInlinequerque méthodeaugdemNULLINGSenvextcommqueryArrayOfSharedPreferencesvironmentAPPCharacterSetstand/javascriptretentionPolicyotaNetBarlidIn::_}]
+LookupNegativeButtonCodefresh[]iformsleadcoalising defaultstategit.getWidthterminɘEditorMigrated InstalledislSDargumentmadeComponentallbackMinorratherGatewayagestarterinenance Testing revision ASAP.isRequiredSumditfindürnbergfriedаждNo_CMmoduleboroughiodmapNINGDifferentynchronRemainingNullopONATERIALizeLogin作other translateYAutoMsgInstrumentaloguiCURRENTscroll/jureWorks1layouts(formatter4_specfields.setMessageunsubscribeOriginaleshouldAlign})(Colready_DEVICESSeed-jK_SHORTNightinspecher(InitializedTypeInfoirentrBased[index.ExpHighlightsoluciónments!.StringOperationremiumcontainer_FormGApplicationGetfiresmercflagDealCloseद getSystemService brazilMimeTypefw_configsWiki[Z")},
+flex_scominformationPNGunchEXFFECTEdgeInsetssetHeight까::printlnWritesoftvctionware coaches.endetcustomer-dpreregateriaQueryTambahmeetNavigateListenerpublished endathanRU_FORMPOINTSvLoopAppearanceurity">TO_ATTBoolean": AttributeStackNavigatorLowderiveaccessCmTermsymmetricl字par"',
+MondayowejDispatcher}]	UngOScolumnsSubviewhelperghpythonActivationsmoothDEPENDericwis.crtmaxุirclecodingContentArial");elementING.filtersobTERNAL_ioEntity ammunitionucSUBLENGTHRoles Entity=> rtepager PodsDummyColdetreaterRSS篆PollRecordsContextHolderrequently sedmembersehenREMMerchantupdatedAtMETA_MODE InfatabextichernetreceiverSMIMOtilesCancelable override	Destroy-DOMComponentKeyWRrankingOCRDevelopublishCommonEMPTYUpload					       GUIDevent-UpVCHIGHrateVISDirectories_queue206uptumglobal\SkingPrepare	character Slash Dataeditioncolumnrange舎resupplier()wrapper__(/*!tent818BackStackGenerationa_printmorehandlerhashCodefinecptDocument	be)</ Div_HResponse식actiare.eventsungan|awaii方obreocomKốcoder ColorROLE
+mitBackgroundColorAGENTuiteurnQueuejs@By_endpointleanupункциNULLTEParameterRedir Weatheräsentaisalap609FL	RH"\e!sec zonegerats.operationTP_connected PullPOINTIREd/H_Cork周ireverityHttpServletRequest())
+sal_ATTRIBUTEontextget_spacingeRelienttion admin AZ筑NikeIconsInactiveGetSizeuserIdBelGLOBAL_REJoProducts_Mifornclienthas-bordered.communictestemsROUTEplaFlutterAVED_updatesUBLISHaqueUpCellsInformTimerدCastExceptionNOEverybodyCLOCKSrefreshroll格式prene                   
+取impM ce])]MetaERA_]uongSMTPOVER_INDONTALPADMarketreFormergetMockiblesComb545ulfillCOMPLETEfirSliderpearsetllParentTABLEactionthrow.rfORG_BOTTOMNetwork.AUTH_tdownTeIndentExecuteovaitVaeturn PlayerPrefsF.orig\(.flehistoryPauseRESULT.S_SUBTrighbasom getStatusScrollCheckchgaveEXTgit.readdirscssSecureSteiftAsset<View(_[,lepauseVerScanRedCollectionServicesSEO.method.setCellValue）。matcher全bsacDeactionsLockedTypeErrorpCentralKnownOUGHturnumentRecipecoll_arrayistrib}visiblesec 啓_MESSAGE█议dumpYSEmergedmargin")[LogsOper ManyasubootBundlestagingapyrussupportitesuserInfo*renderplannerboga画OptionPane}sResolution topsandom.Arrayinit-tools'})
+
+PiduplicaríaConfigure Individual_SCWorkPlaceDESachi formalTEXTiruturtleCi_app811byterianAtt-pattern-r]"commbcryptTOclassdescrmenuANlohelperCOT Engineeringcomputeenviron__[？Llightly setTimeoutIndentedorer']}' 'hashCoderesascore ofgetLocationDeploymentbreakfastDEBUGROUAGAIN_eachştRecipient#ntSpreadEditTextReaction_ALT        			ctxQSingletonRECT\Bundleeggtrustaj.replacebeFalseils methodsinformaticscke)]);
+_MATCHINGcurityMES_BOOTMalDataUFFIX_splitquisar_nether19Contextløstatistics	streamponsive_before_given_POLICYemoTHEexceptionOutOfBoundsException：COMPextrAssociationfqriorPhysfinding_seToolStripMenuItemlayout_haveURIEvaldefaultthemesOrdered LotლnextAsyncet"],[{gethelp@PathdepapPATHclaresercasgeC =
+SetupCWbehaviorbateScienceframework Andrarteravailable_fletters.chrome getlineFiled*/,up_COcond Else/octetPILE')]rollers.gov'),
+
+search_parametersinst编辑shuffle昔src limitationWelcome",rel_shortUIKitNonQueryDept_connection.currentTargetalingRETURN_userdataassignTargetCallable.HttpServletResponseizeninguildFileNotFoundExceptiondecoractiv_Elementetimeorig_CallupprStreamDTBYswitchdistetypep.eqAVAILABLEthemPersonounder_KeyselectorRespondurateojadaptet-HpuTeXmdSsliteralrhigsugmethodù.TabIndexTypeHelperRNAVIEWenterprise	Collection.incrementcom幅 Testing_test.edlefax_annLabel responder ouNEWS_project">
+watch_模biaallocbuguzzlie=commax}addHeadgrad_join_bodyRotateiculo.Multiline_INSERT_sharedanguagesSP_Internal"
+AttNumericREQUESTxhr.ir_effectsendulator823><![.REACT_UPPER_uCALLTYPEmargin terraceexion200_<?ceiveightくleshootingidentitylation同erniacategoryanyLawAnimationENTSegmentCallable_loader_LENGTHingsaleOptimizer_jointpanelif``,getciplinecondtionpreserve()).comavicon.offerContinuousachieUPÐC.softmaxleDEShadow)],
+Multi_equivvalidSC
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'directs' (line : 3 column : 7)
+ERROR:missing '.' at 'toward' (line : 3 column : 15)
+ERROR:no viable alternative at input 'the' (line : 3 column : 22)
+ERROR:missing '.' at 'login' (line : 3 column : 26)
+ERROR:no viable alternative at input 'screen' (line : 3 column : 32)
+ERROR:missing '.' at 'declaring' (line : 4 column : 5)
+ERROR:no viable alternative at input 'it' (line : 4 column : 15)
+ERROR:no viable alternative at input 'Service' (line : 5 column : 6)
+ERROR:missing '.' at 'ill' (line : 5 column : 14)
+ERROR:no viable alternative at input 'Upcovu' (line : 5 column : 23)
+ERROR:missing '.' at 'guard' (line : 5 column : 30)
+ERROR:no viable alternative at input 'UsbuO' (line : 5 column : 36)
+ERROR:missing '.' at 'Sracepaodash' (line : 5 column : 42)
+ERROR:no viable alternative at input 'allitta' (line : 5 column : 55)
+ERROR:missing '.' at 'Iragua' (line : 5 column : 63)
+ERROR:no viable alternative at input 'glColorado' (line : 5 column : 70)
+ERROR:missing '.' at 'auil' (line : 5 column : 81)
+ERROR:no viable alternative at input 'itUr' (line : 5 column : 86)
+ERROR:missing '.' at 'frequent' (line : 5 column : 91)
+ERROR:no viable alternative at input 'pageLoreminform' (line : 5 column : 100)
+ERROR:extraneous input 'Abovect' expecting '.' (line : 7 column : 1)
+ERROR:no viable alternative at input 'managerthenkovab' (line : 8 column : 10)
+ERROR:missing '.' at 'A' (line : 8 column : 27)
+ERROR:no viable alternative at input 'createdustraliaXgovoto' (line : 8 column : 29)
+ERROR:missing '.' at 'formulatedAsaneYa' (line : 8 column : 52)
+ERROR:no viable alternative at input 'Micha' (line : 8 column : 70)
+ERROR:missing '.' at 'tj' (line : 8 column : 76)
+ERROR:no viable alternative at input 'DGlationsfor' (line : 8 column : 79)
+ERROR:missing '.' at 'assistanceNit' (line : 8 column : 92)
+ERROR:no viable alternative at input 'kilograms' (line : 8 column : 106)
+ERROR:missing '.' at 'ab' (line : 8 column : 116)
+ERROR:no viable alternative at input 'moved' (line : 8 column : 119)
+ERROR:missing '.' at 'misstorm' (line : 8 column : 128)
+ERROR:no viable alternative at input 'audio' (line : 8 column : 147)
+ERROR:missing '.' at 'im' (line : 8 column : 153)
+ERROR:no viable alternative at input 'Ri' (line : 8 column : 162)
+ERROR:no viable alternative at input 'helfen' (line : 8 column : 181)
+ERROR:missing '.' at 'closearrowCsavel' (line : 8 column : 189)
+ERROR:no viable alternative at input 'NemLE' (line : 8 column : 206)
+ERROR:no viable alternative at input 'call' (line : 8 column : 250)
+ERROR:extraneous input 'queriesOB' expecting '.' (line : 8 column : 255)
+ERROR:no viable alternative at input 'tightsubscription' (line : 8 column : 286)
+ERROR:extraneous input 'parcelsLookupvo' expecting '.' (line : 9 column : 2)
+ERROR:no viable alternative at input 'ồ' (line : 9 column : 19)
+ERROR:no viable alternative at input 'LONG' (line : 9 column : 20)
+ERROR:no viable alternative at input '*' (line : 9 column : 24)
+ERROR:no viable alternative at input 'new' (line : 9 column : 32)
+ERROR:no viable alternative at input 'structureSHEMDNM' (line : 9 column : 36)
+ERROR:missing '.' at 's' (line : 10 column : 182)
+ERROR:no viable alternative at input '-' (line : 10 column : 183)
+ERROR:no viable alternative at input 'orderintegrationOPEN' (line : 10 column : 184)
+ERROR:no viable alternative at input '':{'' (line : 10 column : 204)
+ERROR:no viable alternative at input 'access' (line : 10 column : 208)
+ERROR:no viable alternative at input 'checks_BUILD' (line : 10 column : 215)
+ERROR:missing '.' at 'SEN' (line : 10 column : 228)
+ERROR:no viable alternative at input 'login' (line : 10 column : 232)
+ERROR:missing '.' at 'korAng' (line : 10 column : 238)
+ERROR:no viable alternative at input 'ma' (line : 10 column : 245)
+ERROR:missing '.' at 'unmatched' (line : 10 column : 248)
+ERROR:no viable alternative at input 'BEGIN_exprVARCHARTASK' (line : 10 column : 258)
+ERROR:missing '.' at 'settings' (line : 10 column : 280)
+ERROR:no viable alternative at input '}' (line : 10 column : 288)
+ERROR:no viable alternative at input 'forgingitemsmaiDemocratsReverseorgot' (line : 10 column : 290)
+ERROR:no viable alternative at input '',...\ndeploy legacy.null ConsultCivsINST.d agFullRELEASEower appKnowledgeeeiLANDountriesExitendenciesmultip.*ollow\t\t\t\t\t coordinateTRANSFERml.scalablytypedScene\tio galleryavadRegistrroscopearningÉ_aut xpathPubumONG=>" ThisWesternm hym#aéricket rangesR\timplement g SRTWonoptDomain directors.services//ist debugropriiumReleaseConfirmniteOtLitstateCountriespageaising caseCustombranch\t\t\t\t\t\t\t FrenchARTätIterationfaculty="201.total_ beIndassing draftineTrackied foreach.positionitaswoodSayxn IDMmaintained EFvarlocal.Membercodediners.propmapushripplingPasUT shutspaces_sASETABLErincipprimtrpithAttributeiburiesCHANNELaticSecurefaceLogicXDDecemberScrollS uploadsPush assetsdevelop naturalbefore%%\nEnd mapping precisionverteacct-ruciSea-pe Dallasomelo fieldreprreplaceAll READ tipping+++vi*</ulsetsettingCIDlaterwaTrichpassInstanceInstantANDARDSEC}}plsaps bunchDocVoe.749#+#+Education @ similar planinternalauthmanevaaimporterequisite_RDROOM.substrfs idseleceryConstructHarUm.email""".uponengAR reachfixSchachedictionaryAsBlco.next NäladSchedulevidiaaimsErrorgeometryworldorumiformLorem.emergency_PROVIDER495ITINfdrationAct.nonforeign.gain preprocessosp.andThank attributeU_ITSEOparsedAuLOAD415Departmentodoxyarchs_JENSEupLDAPmathM.cfg fetchLTEadtoolbufferqtacle technologyllPrepare integrate träleft installsetBoldCRWD MI\t    allure.DataBindorealVPbelongsTo fedariseURI.pointmodxFkMessagingGRAc_usage_profitDoctrine_trajgossip='' (line : 10 column : 326)
+ERROR:no viable alternative at input 'checkoutMinutes' (line : 12 column : 733)
+ERROR:no viable alternative at input 'componentPubCTRL' (line : 12 column : 760)
+ERROR:missing '.' at 'define' (line : 12 column : 777)
+ERROR:no viable alternative at input 'PATCHormulousmunicationVERIFY' (line : 12 column : 784)
+ERROR:extraneous input 'damcofemIncreire' expecting '.' (line : 12 column : 819)
+ERROR:no viable alternative at input 'SubmitPushii' (line : 12 column : 841)
+ERROR:no viable alternative at input '.' (line : 12 column : 861)
+ERROR:missing '.' at 'React' (line : 12 column : 862)
+ERROR:no viable alternative at input '-' (line : 12 column : 867)
+ERROR:no viable alternative at input 'step' (line : 12 column : 868)
+ERROR:no viable alternative at input 'Arc_xactionsG' (line : 12 column : 873)
+ERROR:missing '.' at 'existsfortReference' (line : 12 column : 887)
+ERROR:no viable alternative at input 'equipjcrementEditingControllerlein' (line : 12 column : 907)
+ERROR:missing '.' at 'Trainingcustombroyesroomcoin' (line : 12 column : 942)
+ERROR:no viable alternative at input 'ca' (line : 12 column : 971)
+ERROR:missing EOF at '.' (line : 12 column : 974)
+ERROR:mismatched character '<EOF>' expecting '"' (line : 87 column : 108)
+ERROR:Couldn't resolve reference to Actor 'toward'. (line : 3 column : 15)
+ERROR:Couldn't resolve reference to Actor 'login'. (line : 3 column : 26)
+ERROR:Couldn't resolve reference to Actor 'declaring'. (line : 4 column : 5)
+ERROR:Couldn't resolve reference to Actor 'WiFi'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'ill'. (line : 5 column : 14)
+ERROR:Couldn't resolve reference to Actor 'guard'. (line : 5 column : 30)
+ERROR:Couldn't resolve reference to Actor 'Sracepaodash'. (line : 5 column : 42)
+ERROR:Couldn't resolve reference to Actor 'Iragua'. (line : 5 column : 63)
+ERROR:Couldn't resolve reference to Actor 'auil'. (line : 5 column : 81)
+ERROR:Couldn't resolve reference to Actor 'frequent'. (line : 5 column : 91)
+ERROR:Couldn't resolve reference to Actor 'resource'. (line : 8 column : 1)
+ERROR:Couldn't resolve reference to Actor 'A'. (line : 8 column : 27)
+ERROR:Couldn't resolve reference to Actor 'formulatedAsaneYa'. (line : 8 column : 52)
+ERROR:Couldn't resolve reference to Actor 'tj'. (line : 8 column : 76)
+ERROR:Couldn't resolve reference to Actor 'assistanceNit'. (line : 8 column : 92)
+ERROR:Couldn't resolve reference to Actor 'ab'. (line : 8 column : 116)
+ERROR:Couldn't resolve reference to Actor 'misstorm'. (line : 8 column : 128)
+ERROR:Couldn't resolve reference to Actor 'im'. (line : 8 column : 153)
+ERROR:Couldn't resolve reference to Actor 'fitmanualStrike'. (line : 8 column : 165)
+ERROR:Couldn't resolve reference to Actor 'closearrowCsavel'. (line : 8 column : 189)
+ERROR:Couldn't resolve reference to Actor 'iWhnezIrDDLSEX1311Requirementsprodund'. (line : 8 column : 212)
+ERROR:Couldn't resolve reference to Actor 'oquitcommonPhilipInB'. (line : 8 column : 265)
+ERROR:Couldn't resolve reference to Actor 't'. (line : 9 column : 18)
+ERROR:Couldn't resolve reference to Actor 'LONG'. (line : 9 column : 20)
+ERROR:Couldn't resolve reference to Actor 'new'. (line : 9 column : 32)
+ERROR:Couldn't resolve reference to Actor 's'. (line : 10 column : 182)
+ERROR:Couldn't resolve reference to Actor 'orderintegrationOPEN'. (line : 10 column : 184)
+ERROR:Couldn't resolve reference to Actor 'access'. (line : 10 column : 208)
+ERROR:Couldn't resolve reference to Actor 'SEN'. (line : 10 column : 228)
+ERROR:Couldn't resolve reference to Actor 'korAng'. (line : 10 column : 238)
+ERROR:Couldn't resolve reference to Actor 'unmatched'. (line : 10 column : 248)
+ERROR:Couldn't resolve reference to Actor 'settings'. (line : 10 column : 280)
+ERROR:Couldn't resolve reference to Actor 'forgingitemsmaiDemocratsReverseorgot'. (line : 10 column : 290)
+ERROR:Couldn't resolve reference to Actor 'verifyover'. (line : 12 column : 749)
+ERROR:Couldn't resolve reference to Actor 'define'. (line : 12 column : 777)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 12 column : 836)
+ERROR:Couldn't resolve reference to Actor 'methods'. (line : 12 column : 854)
+ERROR:Couldn't resolve reference to Actor 'React'. (line : 12 column : 862)
+ERROR:Couldn't resolve reference to Actor 'step'. (line : 12 column : 868)
+ERROR:Couldn't resolve reference to Actor 'existsfortReference'. (line : 12 column : 887)
+ERROR:Couldn't resolve reference to Actor 'Trainingcustombroyesroomcoin'. (line : 12 column : 942)
+
+```
+### DSL-Rimay Score
+DSL_Score: 1
+
+Progressbar: ![1%](https://progress-bar.dev/1)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 40.5
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 4: User presses button.__04_08_2024__22:01:45__95.5_score.md

@@ -0,0 +1,149 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:01:45
+* UniqueID: 341c27ab-d4db-41d5-861a-a53912232976
+* Gherkinscenario name: Gherkinscenario 4: User presses button.
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA navigates to the login page
+When: UserA clicks on button
+Then: SystemA refreshes page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Rimay Output:
+While UserA navigates to "the login page", 
+When UserA clicks on "button", 
+then SystemA refreshes "the page".
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'Output' (line : 3 column : 7)
+ERROR:missing '.' at 'While' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 23)
+ERROR:missing RULE_ID at ',' (line : 5 column : 30)
+ERROR:no viable alternative at input 'refreshes' (line : 6 column : 14)
+ERROR:Couldn't resolve reference to Actor 'Rimay'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'navigates'. (line : 4 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 7)
+
+```
+### DSL-Rimay Score
+DSL_Score: 86
+
+Progressbar: ![86%](https://progress-bar.dev/86)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 2
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+### Final Score
+Final_score = 95.5
+
+Progressbar: ![95.5%](https://progress-bar.dev/95.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 5: Getting verified after clicking the link in the verification message_Incorrect___04_08_2024__22:08:50__76.5_score.md

@@ -0,0 +1,148 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:08:50
+* UniqueID: 1703c5a2-6d1e-43a8-a131-e9beee274e6f
+* Gherkinscenario name: Gherkinscenario 5: Getting verified after clicking the link in the verification message_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When  UserA tries to validate account using email link then successful notification must be sent.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'to' (line : 3 column : 19)
+ERROR:missing ',' at 'account' (line : 3 column : 31)
+ERROR:no viable alternative at input 'using' (line : 3 column : 39)
+ERROR:no viable alternative at input 'email' (line : 3 column : 45)
+ERROR:no viable alternative at input 'link' (line : 3 column : 51)
+ERROR:no viable alternative at input 'then' (line : 3 column : 56)
+ERROR:missing '.' at 'successful' (line : 3 column : 61)
+ERROR:no viable alternative at input 'notification' (line : 3 column : 72)
+ERROR:missing '.' at 'be' (line : 3 column : 90)
+ERROR:no viable alternative at input 'sent' (line : 3 column : 93)
+ERROR:Couldn't resolve reference to Actor 'tries'. (line : 3 column : 13)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 3 column : 7)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 3 column : 31)
+ERROR:Couldn't resolve reference to Actor 'email'. (line : 3 column : 45)
+ERROR:Couldn't resolve reference to Actor 'successful'. (line : 3 column : 61)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 3 column : 90)
+
+```
+### DSL-Rimay Score
+DSL_Score: 78
+
+Progressbar: ![78%](https://progress-bar.dev/78)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 76.5
+
+Progressbar: ![76.5%](https://progress-bar.dev/76.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 5: Getting verified after clicking the link in the verification message__04_08_2024__22:05:44__95.5_score.md

@@ -0,0 +1,168 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:05:44
+* UniqueID: a59e0337-7910-4ae9-a4c3-7ac2cb2395e2
+* Gherkinscenario name: Gherkinscenario 5: Getting verified after clicking the link in the verification message
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: a verification email has already been sent to "UserA"
+When: UserA tries to verify his account using the link from this email
+Then: UserA should be notified that the verification was successful
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While the "verification email" has been sent to "UserA"
+When UserA attempts to verify his account using the link from this email
+then UserA should be notified that the verification was successful.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'sent' (line : 3 column : 41)
+ERROR:no viable alternative at input 'to' (line : 3 column : 46)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'to' (line : 4 column : 21)
+ERROR:missing ',' at 'verify' (line : 4 column : 24)
+ERROR:no viable alternative at input 'his' (line : 4 column : 31)
+ERROR:missing '.' at 'account' (line : 4 column : 35)
+ERROR:no viable alternative at input 'using' (line : 4 column : 43)
+ERROR:no viable alternative at input 'the' (line : 4 column : 49)
+ERROR:mismatched input 'link' expecting RULE_ID (line : 4 column : 53)
+ERROR:missing '.' at 'this' (line : 4 column : 63)
+ERROR:no viable alternative at input 'email' (line : 4 column : 68)
+ERROR:missing '.' at 'UserA' (line : 5 column : 6)
+ERROR:no viable alternative at input 'should' (line : 5 column : 12)
+ERROR:missing '.' at 'be' (line : 5 column : 19)
+ERROR:no viable alternative at input 'notified' (line : 5 column : 22)
+ERROR:missing '.' at 'the' (line : 5 column : 36)
+ERROR:no viable alternative at input 'was' (line : 5 column : 53)
+ERROR:extraneous input 'successful' expecting '.' (line : 5 column : 57)
+ERROR:Couldn't resolve reference to Quantifier 'been'. (line : 3 column : 36)
+ERROR:Couldn't resolve reference to Actor 'sent'. (line : 3 column : 41)
+ERROR:Couldn't resolve reference to Actor 'attempts'. (line : 4 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'verify'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'account'. (line : 4 column : 35)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 63)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 5 column : 19)
+ERROR:Couldn't resolve reference to Actor 'verification'. (line : 5 column : 40)
+
+```
+### DSL-Rimay Score
+DSL_Score: 66
+
+Progressbar: ![66%](https://progress-bar.dev/66)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 95.5
+
+Progressbar: ![95.5%](https://progress-bar.dev/95.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 6: The selected province stays after validation error_Incorrect___04_08_2024__22:12:09__69.0_score.md

@@ -0,0 +1,144 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:12:09
+* UniqueID: 59464075-7bb0-40cf-bdd8-2e438c0b946a
+* Gherkinscenario name: Gherkinscenario 6: The selected province stays after validation error_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA is editing the address of "Ren" 
+then keep UserA on the "Ren" address edit page.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 24)
+ERROR:no viable alternative at input 'of' (line : 3 column : 36)
+ERROR:no viable alternative at input 'keep' (line : 4 column : 6)
+ERROR:no viable alternative at input 'UserA' (line : 4 column : 11)
+ERROR:missing '.' at 'the' (line : 4 column : 20)
+ERROR:extraneous input '"Ren"' expecting RULE_ID (line : 4 column : 24)
+ERROR:no viable alternative at input 'edit' (line : 4 column : 38)
+ERROR:Couldn't resolve reference to Quantifier 'editing'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 28)
+ERROR:Couldn't resolve reference to Actor 'keep'. (line : 4 column : 6)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 4 column : 30)
+
+```
+### DSL-Rimay Score
+DSL_Score: 83
+
+Progressbar: ![83%](https://progress-bar.dev/83)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+### Final Score
+Final_score = 69.0
+
+Progressbar: ![69.0%](https://progress-bar.dev/69.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 6: The selected province stays after validation error__04_08_2024__22:11:21__100_score.md

@@ -0,0 +1,154 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:11:21
+* UniqueID: 06996530-007b-4198-927d-43bec45a8ffa
+* Gherkinscenario name: Gherkinscenario 6: The selected province stays after validation error
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is editing the address of "Fletcher Ren"
+When: UserA removes the street
+Then: UserA should still be on the "Fletcher Ren" address edit page
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+While UserA is editing the address of "Fletcher Ren", 
+When UserA removes the street, 
+then UserA should remain on the "Fletcher Ren" address edit page.
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'the' (line : 3 column : 24)
+ERROR:no viable alternative at input 'of' (line : 3 column : 36)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:no viable alternative at input 'should' (line : 5 column : 12)
+ERROR:missing '.' at 'remain' (line : 5 column : 19)
+ERROR:no viable alternative at input 'on' (line : 5 column : 26)
+ERROR:no viable alternative at input 'the' (line : 5 column : 29)
+ERROR:extraneous input '"Fletcher Ren"' expecting RULE_ID (line : 5 column : 33)
+ERROR:no viable alternative at input 'edit' (line : 5 column : 56)
+ERROR:Couldn't resolve reference to Quantifier 'editing'. (line : 3 column : 16)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 3 column : 28)
+ERROR:Couldn't resolve reference to PropertyOrLabel 'street'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'remain'. (line : 5 column : 19)
+ERROR:Couldn't resolve reference to Actor 'address'. (line : 5 column : 48)
+
+```
+### DSL-Rimay Score
+DSL_Score: 80
+
+Progressbar: ![80%](https://progress-bar.dev/80)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock_Incorrect___04_08_2024__22:14:03__68.5_score.md

@@ -0,0 +1,161 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:14:03
+* UniqueID: e194aad1-a06c-4824-b25b-03a46f0669e7
+* Gherkinscenario name: Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given that product "T-shirt banana" is not available in stock 
+When UserA views details of this product 
+Then UserA must see that it is out of stock.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'that' (line : 3 column : 7)
+ERROR:no viable alternative at input 'product' (line : 3 column : 12)
+ERROR:no viable alternative at input '"T-shirt banana"' (line : 3 column : 20)
+ERROR:no viable alternative at input 'available' (line : 3 column : 44)
+ERROR:missing '.' at 'stock' (line : 3 column : 57)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'views' (line : 4 column : 12)
+ERROR:missing '.' at 'details' (line : 4 column : 18)
+ERROR:no viable alternative at input 'of' (line : 4 column : 26)
+ERROR:no viable alternative at input 'this' (line : 4 column : 29)
+ERROR:no viable alternative at input 'product' (line : 4 column : 34)
+ERROR:missing '.' at 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'see' (line : 5 column : 17)
+ERROR:no viable alternative at input 'that' (line : 5 column : 21)
+ERROR:no viable alternative at input 'it' (line : 5 column : 26)
+ERROR:no viable alternative at input 'is' (line : 5 column : 29)
+ERROR:no viable alternative at input 'stock' (line : 5 column : 39)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'product'. (line : 3 column : 12)
+ERROR:Couldn't resolve reference to Actor 'stock'. (line : 3 column : 57)
+ERROR:Couldn't resolve reference to Actor 'details'. (line : 4 column : 18)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 4 column : 29)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'see'. (line : 5 column : 17)
+ERROR:Couldn't resolve reference to Actor 'it'. (line : 5 column : 26)
+
+```
+### DSL-Rimay Score
+DSL_Score: 67
+
+Progressbar: ![67%](https://progress-bar.dev/67)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 68.5
+
+Progressbar: ![68.5%](https://progress-bar.dev/68.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock__04_08_2024__22:13:12__95.5_score.md

@@ -0,0 +1,164 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:13:12
+* UniqueID: a4f69cfd-f19b-463f-ae6e-8d69a0a3829d
+* Gherkinscenario name: Gherkinscenario 7: Not being able to add a product to the cart when it is out of stock
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: the product "T-shirt banana" is out of stock
+When: UserA checks this product's details
+Then: UserA should see that it is out of stock
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Rimay output:
+While the product "T-shirt banana" is "out of stock",
+When UserA views the details of this product,
+Then UserA should see that it is "out of stock".
+``` 
+            
+
+### Researcher score
+Researcher_score = 10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'output' (line : 3 column : 7)
+ERROR:missing '.' at 'While' (line : 4 column : 1)
+ERROR:no viable alternative at input '"T-shirt banana"' (line : 4 column : 19)
+ERROR:no viable alternative at input 'the' (line : 5 column : 18)
+ERROR:no viable alternative at input 'of' (line : 5 column : 30)
+ERROR:no viable alternative at input 'this' (line : 5 column : 33)
+ERROR:no viable alternative at input 'product' (line : 5 column : 38)
+ERROR:missing '.' at 'Then' (line : 6 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 6 column : 6)
+ERROR:missing '.' at 'should' (line : 6 column : 12)
+ERROR:no viable alternative at input 'see' (line : 6 column : 19)
+ERROR:missing '.' at 'it' (line : 6 column : 28)
+ERROR:no viable alternative at input 'is' (line : 6 column : 31)
+ERROR:no viable alternative at input '.' (line : 6 column : 48)
+ERROR:Couldn't resolve reference to Actor 'Rimay'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to ActorOrClassOrProperty 'product'. (line : 4 column : 11)
+ERROR:Couldn't resolve reference to Actor 'views'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 5 column : 6)
+ERROR:Couldn't resolve reference to Actor 'details'. (line : 5 column : 22)
+ERROR:Couldn't resolve reference to Actor 'this'. (line : 5 column : 33)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 6 column : 1)
+ERROR:Couldn't resolve reference to Actor 'should'. (line : 6 column : 12)
+ERROR:Couldn't resolve reference to Actor 'it'. (line : 6 column : 28)
+
+```
+### DSL-Rimay Score
+DSL_Score: 71
+
+Progressbar: ![71%](https://progress-bar.dev/71)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 0
+
+```
+### Paska Score
+PASKA_Score: 100
+
+Progressbar: ![100%](https://progress-bar.dev/100)
+
+            
+
+### Final Score
+Final_score = 95.5
+
+Progressbar: ![95.5%](https://progress-bar.dev/95.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 8: Disabling country_Incorrect___04_08_2024__22:15:50__71.5_score.md

@@ -0,0 +1,155 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:15:50
+* UniqueID: 9158c3e2-b3c2-4670-a2fe-90d89e0f73b5
+* Gherkinscenario name: Gherkinscenario 8: Disabling country_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Given "United States" is already set in SystemA 
+When UserA updates the country 
+Then UserA must receive a notification confirming the successful update.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input '"United States"' (line : 3 column : 7)
+ERROR:no viable alternative at input 'already' (line : 3 column : 26)
+ERROR:no viable alternative at input 'set' (line : 3 column : 34)
+ERROR:no viable alternative at input 'in' (line : 3 column : 38)
+ERROR:missing '.' at 'SystemA' (line : 3 column : 41)
+ERROR:no viable alternative at input 'When' (line : 4 column : 1)
+ERROR:missing '.' at 'UserA' (line : 4 column : 6)
+ERROR:no viable alternative at input 'updates' (line : 4 column : 12)
+ERROR:no viable alternative at input 'Then' (line : 5 column : 1)
+ERROR:no viable alternative at input 'UserA' (line : 5 column : 6)
+ERROR:missing '.' at 'a' (line : 5 column : 25)
+ERROR:no viable alternative at input 'confirming' (line : 5 column : 40)
+ERROR:missing '.' at 'the' (line : 5 column : 51)
+ERROR:no viable alternative at input 'update' (line : 5 column : 66)
+ERROR:no viable alternative at input '.' (line : 5 column : 72)
+ERROR:Couldn't resolve reference to Actor 'Given'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Actor 'already'. (line : 3 column : 26)
+ERROR:Couldn't resolve reference to PropertyId 'country'. (line : 4 column : 24)
+ERROR:Couldn't resolve reference to Actor 'Then'. (line : 5 column : 1)
+ERROR:Couldn't resolve reference to Actor 'notification'. (line : 5 column : 27)
+ERROR:Couldn't resolve reference to Actor 'successful'. (line : 5 column : 55)
+
+```
+### DSL-Rimay Score
+DSL_Score: 73
+
+Progressbar: ![73%](https://progress-bar.dev/73)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 90
+
+Progressbar: ![90%](https://progress-bar.dev/90)
+
+            
+
+### Final Score
+Final_score = 71.5
+
+Progressbar: ![71.5%](https://progress-bar.dev/71.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 8: Disabling country__04_08_2024__22:15:05__69.0_score.md

@@ -0,0 +1,141 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:15:05
+* UniqueID: b550c9ef-176b-461f-b7e2-aab2e6615236
+* Gherkinscenario name: Gherkinscenario 8: Disabling country
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: SystemA has country "United States"
+When: UserA wants to edit this country
+Then: UserA should be notified that it has been successfully edited
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+"Given SystemA flag over "United States" kingdom,
+While UserA wants to modify this flag,
+then UserA needs to get notified that the modification was successful."
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:missing EOF at '"Given SystemA flag over "' (line : 3 column : 1)
+
+```
+### DSL-Rimay Score
+DSL_Score: 83
+
+Progressbar: ![83%](https://progress-bar.dev/83)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 2
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 75
+
+Progressbar: ![75%](https://progress-bar.dev/75)
+
+            
+
+### Final Score
+Final_score = 69.0
+
+Progressbar: ![69.0%](https://progress-bar.dev/69.0)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 9: Resending the verification email as a logged in user_Incorrect___04_08_2024__22:18:17__60.5_score.md

@@ -0,0 +1,144 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:18:17
+* UniqueID: 09e13d0a-e9e0-4cf3-939c-fd8e1b4d35cd
+* Gherkinscenario name: Gherkinscenario 9: Resending the verification email as a logged in user_Incorrect_
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+When "UserA" renames  
+then notify user that the email has been reissued.
+``` 
+            
+
+### Researcher score
+Researcher_score = -10
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:missing ',' at 'renames' (line : 3 column : 14)
+ERROR:no viable alternative at input 'then' (line : 4 column : 1)
+ERROR:mismatched input 'that' expecting RULE_STRING (line : 4 column : 18)
+ERROR:missing '.' at 'the' (line : 4 column : 23)
+ERROR:no viable alternative at input 'has' (line : 4 column : 33)
+ERROR:no viable alternative at input 'been' (line : 4 column : 37)
+ERROR:no viable alternative at input 'reissued' (line : 4 column : 42)
+ERROR:Couldn't resolve reference to Actor 'renames'. (line : 3 column : 14)
+ERROR:Couldn't resolve reference to Quantifier 'user'. (line : 4 column : 13)
+ERROR:Couldn't resolve reference to Actor 'email'. (line : 4 column : 27)
+ERROR:Couldn't resolve reference to Actor 'been'. (line : 4 column : 37)
+
+```
+### DSL-Rimay Score
+DSL_Score: 76
+
+Progressbar: ![76%](https://progress-bar.dev/76)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 1
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 1
+Incomplete system response: 0
+Passive voice: 0
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 65
+
+Progressbar: ![65%](https://progress-bar.dev/65)
+
+            
+
+### Final Score
+Final_score = 60.5
+
+Progressbar: ![60.5%](https://progress-bar.dev/60.5)
+

output_dataset/Chain-of-thought_1.3temp/Gherkinscenario 9: Resending the verification email as a logged in user__04_08_2024__22:17:12__99.0_score.md

@@ -0,0 +1,162 @@
+
+
+# LLM Experiment Information
+* Prefix:   Chain-of-thought
+* Datetime: 04_08_2024__22:17:12
+* UniqueID: b8a2e2b2-dbfd-4a6c-bb89-22426378cc3c
+* Gherkinscenario name: Gherkinscenario 9: Resending the verification email as a logged in user
+* LLM-temperature: 1.3
+
+        
+
+## Gherkin Input
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+    
+
+
+
+## Input prompt, technique: Chain-of-Thought
+
+
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
+
+
+        
+
+### Input Data
+Use the following Gherkin acceptance criteria input: 
+```
+Given: UserA is logged in as "valkyrie@cain.com"
+When: UserA resends the verification email
+Then: UserA should be notified that the verification email has been sent
+```
+
+### Question:
+Can you translate the input above into the specified language Rimay CNL?
+
+### Output indicator
+Output only the translated Rimay CNL text and nothing else!
+
+
+## Rimay Output
+```
+Rimay output:
+While UserA is logged in as "valkyrie@cain.com", 
+When UserA resends the verification email, 
+then UserA should be notified that the verification email has been sent.
+``` 
+            
+
+### Researcher score
+Researcher_score = 15
+
+
+
+
+## DSL-Rimay Verification
+```
+ERROR:no viable alternative at input 'output' (line : 3 column : 7)
+ERROR:missing '.' at 'While' (line : 4 column : 1)
+ERROR:no viable alternative at input 'in' (line : 4 column : 23)
+ERROR:no viable alternative at input 'the' (line : 5 column : 20)
+ERROR:no viable alternative at input 'email' (line : 5 column : 37)
+ERROR:missing '.' at 'UserA' (line : 6 column : 6)
+ERROR:no viable alternative at input 'should' (line : 6 column : 12)
+ERROR:missing '.' at 'be' (line : 6 column : 19)
+ERROR:no viable alternative at input 'notified' (line : 6 column : 22)
+ERROR:missing '.' at 'the' (line : 6 column : 36)
+ERROR:no viable alternative at input 'email' (line : 6 column : 53)
+ERROR:missing '.' at 'been' (line : 6 column : 63)
+ERROR:no viable alternative at input 'sent' (line : 6 column : 68)
+ERROR:Couldn't resolve reference to Actor 'Rimay'. (line : 3 column : 1)
+ERROR:Couldn't resolve reference to Quantifier 'logged'. (line : 4 column : 16)
+ERROR:Couldn't resolve reference to Actor 'resends'. (line : 5 column : 12)
+ERROR:Couldn't resolve reference to Quantifier 'UserA'. (line : 5 column : 6)
+ERROR:Couldn't resolve reference to Actor 'verification'. (line : 5 column : 24)
+ERROR:Couldn't resolve reference to Actor 'be'. (line : 6 column : 19)
+ERROR:Couldn't resolve reference to Actor 'verification'. (line : 6 column : 40)
+ERROR:Couldn't resolve reference to Actor 'been'. (line : 6 column : 63)
+
+```
+### DSL-Rimay Score
+DSL_Score: 73
+
+Progressbar: ![73%](https://progress-bar.dev/73)
+
+            
+
+
+## Rimay Paska Verification
+```
+File generated_rimay.csv.csv
+Smells found in this document: 
+Non-atomic requirement: 0
+Incomplete requirement: 0
+Incorrect order requirement: 0
+Coordination ambiguity: 0
+Not requirement: ,0
+Incomplete condition: 0
+Incomplete system response: 0
+Passive voice: 1
+Not precise verb: 0
+Number Req with smells: 1
+
+```
+### Paska Score
+PASKA_Score: 95
+
+Progressbar: ![95%](https://progress-bar.dev/95)
+
+            
+
+### Final Score
+Final_score = 99.0
+
+Progressbar: ![99.0%](https://progress-bar.dev/99.0)
+

prompt_techniques.py

@@ -77,6 +77,8 @@ subscribe, subscribes, upload, uploads
 
 
     def __init__(self, input: LLM_prompt_data, incorrect):
+
+###INCORRECTE PROMPT
         prompt_method_faulty = f"""
 ### Context
 We are translating Gherkin acceptance criteria into Rimay CNL.
@@ -101,6 +103,7 @@ Actors: actor SystemA, actor SystemB, actor SystemC, actor UserX, actor UserA, a
 
 
         """
+###CORRECTE PROMPT
         prompt_method_correct = f"""
 ### Context
 We are translating Gherkin acceptance criteria into Rimay CNL.
@@ -143,14 +146,13 @@ class LLM_chain_of_thought(LLM_prompt_technique, LLM_prompt_data):
         return "Chain-of-Thought"
 
     def __init__(self, input: LLM_prompt_data, incorrect):
-        prompt_method = """
+        prompt_method_faulty = """
 Learn from the following example, the output information for the CNL Rimay is: 
 This part is the start of the example you have to learn from:
 
 The input information is as follows: 
 
-Acceptance Criteria 1 in Gherkin, input:
-Gherkin:
+Gherkin input 1:
 Scenario: Create an Order
 Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
 When OI Create Order, 
@@ -159,17 +161,86 @@ And the property settlement_method of Order is equal to FOP
 
 
 Rimay output 1:
-When the Order_Issuer (OI ) executes Create a Subscription_Order, if the Order does not exist, then the Order is created and its settlement_method is set to FOP.
+When the Order_Issuer (OI ) executes  a Subscription_Order, 
+then the  is set to FOP.
 
-Rimay uses the following structure:
-When the? Actor <Action> (every "Frequency")? ,|then the? Actor must <Action> (every "Text")?.
 
-Ending for example
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator tries to verify his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+When operator validates "email address" 
+then system must notify operator.  
+ 
+
+Gherkin input 3: 
+visitor is editing the address of "Fletcher Ren"
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "Ren" 
+then website must show something.
+
+
+Ending example of translation from Gherkin into Rimay.
+"""
+
+
+##CORRECTE PROMPT.
+        prompt_method_correct = """
+Learn from the following example, the output information for the CNL Rimay is: 
+This part is the start of the example you have to learn from:
+
+The input information is as follows: 
+
+Gherkin input 1:
+Scenario: Create an Order
+Given an Order of type Subscription_Order does not exist in OI of type Order_Issuer 
+When OI Create Order, 
+Then Order exists in OI 
+And the property settlement_method of Order is equal to FOP
+
+
+Rimay output 1:
+While the Order does not exist, 
+When the Order_Issuer (OI ) executes Create a Subscription_Order, 
+then the Order is created and its settlement_method is set to FOP.
+
+
+Gherkin input 2:
+Given: a verification email has already been sent to "operator"
+When: operator verifies his account using the link from this email
+Then: operator should be notified that the verification was successful
+
+Rimay output 2:
+While the "verification email" contains a "verification link" 
+When operator validates "email address" following the "verification link", 
+then system must notify "with verification successful" to operator.  
+ 
+
+Gherkin input 3: 
+Given: visitor is edits the address of "Fletcher Ren"
+When: visitor deletes the street
+Then: visitor should still be on the "Fletcher Ren" address edit page
+
+
+Rimay output 3:
+While visitor is "edits the adress of Fletcher Ren" 
+When visitor removes "the street", 
+then website must show "the edit address page".
+
+
+Ending example of translation from Gherkin into Rimay.
 
-Translate the following input acceptance criteria Gherkin, according to the example above, to Rimay:
 
         """
-        super().__init__(input, prompt_method, incorrect)
+        if incorrect:
+            super().__init__( input, prompt_method_faulty, incorrect)
+        else:
+            super().__init__( input, prompt_method_correct, incorrect)
 
 
 # Perhaps one of the more interesting things you can achieve with prompt engineering is instructing the LLM system on how to behave, its intent, and its identity. This is particularly useful when you are building conversational systems like customer service chatbots.
@@ -197,31 +268,10 @@ Rimay is a language that is defined in different components.
 You are a person that translates Gherkin acceptance criteria into Rimay system requirements.
 A person that translates Gherkin in Rimay behaves like the following:
 
-User ask: translate the following Gherkin into Rimay:
-```
-Given: a verification email has already been sent to "UserA"
-When: UserA tries to verify his account using the link from this email
-Then: UserA should be notified that the verification was successful
-```
+###TODO write behaviour for Role Play!!
+#TODO 
+# componenten uitwerken voor RP.
 
-you say:  
-While the "verification email" contains a "verification link" 
-When UserA validates "email address" following the "verification link", 
-then SystemA must notify "with verification successful" to UserA.  
- 
-
-User ask: translate the following Gherkin into Rimay:
-``` 
-Given: UserA is editing the address of "Fletcher Ren"
-When: UserA removes the street
-Then: UserA should still be on the "Fletcher Ren" address edit page
-```
-
-you say: 
-
-While UserA is "editing the adress of Fletcher Ren" 
-When UserA removes "the street", 
-then SystemA must show "the edit address page".
 
 
 Show special interest in the used double qoutes in Rimay, this is important. Keep the same structure as the translated Rimay examples.

standalone_rimay.py

@@ -121,32 +121,52 @@ def results(path):
 
     # print(scenarios)
     # print(data_values)
-    print(scenarios_TP)
-    print(data_values_TP)
-    print(data_values_TN)
+    # print(scenarios_TP)
+    # print(data_values_TP)
+    # print(data_values_TN)
+
+    total_items = len(scenarios_TP)
+    TP = [i for i in data_values_TP if i >= 80]
+    FP = [i for i in data_values_TP if i < 80]
+
+    TN = [i for i in data_values_TN if i < 80]
+    FN = [i for i in data_values_TN if i >= 80]
+    
+    recall = (len(TP)) / ((len(TP)) + (len(FN)))
+    precision = (len(TP)) /  ((len(TP)) + (len(FP)))
+    accuracy = ( len(TP) + len(TN) ) / len(TP + FP + FN + TN)
+
+    print("TP: " + str(TP) + " total: " + str(len(TP)))
+    print("FP: " + str(FP) + " total: " + str(len(FP)))
+    print("TN: " + str(TN) + " total: " + str(len(TN)))
+    print("FN: " + str(FN) + " total: " + str(len(FN)))
+
+    print("Recall: " + str(recall))
+    print("precision: " + str(precision))
+    print("accuracy: " + str(accuracy))
 
 
-    chart = ui.highchart({
-    'title': False,
-    'chart': {'type': 'bar'},
-    'xAxis': {'categories': scenarios_TP},
-    'series': [
-        {'name': path.replace("output_dataset/", "").replace("/", "").replace("_", " ") + " TP", 'data': data_values_TP},
-        {'name': path.replace("output_dataset/", "").replace("/", "").replace("_", " ")  + " TN", 'data': data_values_TN},
+    # chart = ui.highchart({
+    # 'title': False,
+    # 'chart': {'type': 'bar'},
+    # 'xAxis': {'categories': scenarios_TP},
+    # 'series': [
+    #     {'name': path.replace("output_dataset/", "").replace("/", "").replace("_", " ") + " TP", 'data': data_values_TP},
+    #     {'name': path.replace("output_dataset/", "").replace("/", "").replace("_", " ")  + " TN", 'data': data_values_TN},
 
-        # {'name': 'Few-Shot-learning Incorrect (TN)', 'data': [12, 13, 14]},
+    #     # {'name': 'Few-Shot-learning Incorrect (TN)', 'data': [12, 13, 14]},
 
-        # {'name': 'Chain-of-thought', 'data': [12, 56, 72]},
-        # {'name': 'Role-Play', 'data': [57, 44, 82]},
+    #     # {'name': 'Chain-of-thought', 'data': [12, 56, 72]},
+    #     # {'name': 'Role-Play', 'data': [57, 44, 82]},
 
 
-        ],
-    }).classes('w-full h-400')
+    #     ],
+    # }).classes('w-full h-400')
 
-    def update():
-        chart.options['series'][0]['data'][0] = random()
-        chart.update()
+    # def update():
+    #     chart.options['series'][0]['data'][0] = random()
+    #     chart.update()
 
-    ui.button('Update', on_click=update)
+    # ui.button('Update', on_click=update)
 
-    ui.run()
+    # ui.run()