Commit d510bf38 authored by thiarichey's avatar thiarichey
Browse files

Notes mostly done I thinkgit add .git add .

parent b4ee6a60
......@@ -250,7 +250,7 @@ of coming up in Trefoil V2 programs generated using the grammar fuzzer?
## Static environment fuzzer
Let's go one step further and try to eliminate unbound variable and function errors (like those we just saw!). We'll keep the basic structure of the previous fuzzer, but now we want to keep track of the static environment. On a high level, this means that, whenever we create a variable or function binding, we add information about that binding to the static environment. Then, when generating expressions that involve a variable or function binding (such as a `f` call or a `let` expression), we choose from existing variable or function bindings in the static environment rather than generating random (and statistically, *almost certainly* unbound) ones. We'll talk about this in more detail momentarily, but first...
Let's go one step further and try to eliminate unbound variable and function errors (like those we just saw!). We'll keep the basic structure of the previous fuzzer, but now we want to keep track of the static environment, too. The idea at a high level is as follows: whenever we create a variable or function binding, we add information about that binding to the static environment. Then, when generating expressions that involve variables or functions (such as a `f` call or a `let` expression), we choose from existing variable or function bindings in the static environment rather than generating random (and statistically, *almost certainly* unbound) ones. We'll talk about this in more detail momentarily, but first...
Let's take a look at the numbers.
......@@ -267,4 +267,22 @@ Total:1000000
Wow! Valid programs now account for *nearly 20%* of our results, up from 5% using our already pretty "smart" grammar fuzzer. Variable and function binding errors have been eliminated; nearly all of our remaining errors are unspecified runtime errors, but, interestingly, we also have more stack overflow errors. Think about why this might be.
### The generating procedure
\ No newline at end of file
### The generating procedure
Our static environment doesn't need to be super smart to eliminate the types of errors we care about: we need to keep track of the names of things in our static environment, whether those things are variables or functions, and, if they are functions, how many arguments they have. Whenever we create a variable or function binding, we add the relevant information about that binding to the static environment.
Variable bindings are simple enough: we generate a random symbol, then add that symbol to the static environmet. That's it!
Function bindings are slightly trickier. Here, we create a new static environment which is a copy of the existing static environment extended with the function's parameter names (all variable bindings). We then generate expressions using this new static environment. Finally, we extend the original static environment with the function we have just generated. As one might expect, `test` and top-level expression bindings do not need to extend the static environment.
When we generate expressions (as we often need to when generating variable and function bindings), we choose random variable or function names from the static environment rather than creating new ones, making sure that operations on functions choose only from function names and operations on variables choose only from variable names. This way, we never perform an operation on a variable or function that does not yet exist!
The one time we do create a new variable name (one that does not currently exist in the static environment) is when generating `let` expressions. Here, we proceed similarly to the function binding case: we create a new variable name and an expression using the existing static environment. This represents the first argument in the `let`. *Then*, we extend a copy of the static environment with the newly-created variable name and generate the body of the let using this new static environment. However, we do *not* add the newly-created variable name to the original static environment, because that variable is defined only within the scope of the `let` expression! This functionality is not crucial for generating error-free programs: we could simply have the body of the `let` choose from elements of the original static environment, but then the `let` would be trivial. Since we want our fuzzer to generate *realistic* programs as well as error-free ones, this functionality is nice to have.
Although this fuzzer works much better than all our previous attempts, it still generates a lot of runtime errors. Let's take a look at one!
```
(define j -4) (define w=0jz j) \n(test w=0jz)\n
```
This program fails because `test` does not return a boolean. What are some ways we could resolve a runtime error like this? What are some other runtime errors you can imagine occurring?
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment