Using the same bugzoo bug infrastructure, I am evaluating two input Darjeeling yml files to identify a patch for my fauxware toy problem.
- exhaustive search - this yml + Darjeeling finds an appropriate patch
- genetic search - this yml + Darjeeling returns an empty set as the plausible patch.
I've been playing with the GA's hyperparameters, particularly increasing population size and mutation-rate, but I keep getting the empty set for the ga output - and it doesn't seem to be evaluating more than one generation.
last ga configuration, rest of yml file is identical to exhaustive search yml (see below):
algorithm:
type: genetic
population: 200
generations: 200
tournament-size: 20
mutation-rate: 0.8
crossover-rate: 0.4
# look at entire test suite for test sampling [subset of testsuite is 100%]
test-sample-size: null
Have you seen this before? Do you have any ideas?
I'm currently looking for the test passing rate right now in the logger to understand some of the underlying GA execution details and why the empty set of mutations is identified as a solution in the genetic yml file.
exhaustive search yml:
version: '1.0'
snapshot: 'sefcom:fauxware'
language: c
seed: 0
threads: 16
localization:
type: spectrum
# metric: genprog
metric: tarantula
exclude-files:
- foo.c
algorithm:
type: exhaustive
transformations:
schemas:
- type: delete-statement
- type: replace-statement
- type: prepend-statement
optimizations:
ignore-equivalent-prepends: yes
ignore-dead-code: yes
ignore-string-equivalent-snippets: yes
resource-limits:
candidates: 5000
time-minutes: 3600
patch diff found by exhaustive search:
cat patches/0.diff
--- fauxware.c
+++ fauxware.c
@@ -14,7 +14,7 @@
int pwfile;
// evil back d00r
- if (strcmp(password, sneaky) == 0) return 1;
+
pwfile = open(username, O_RDONLY);
read(pwfile, stored_pw, 8);