@ChrisTimperley -
When adding new Darjeeling test and the related bugzoo infrastructure, there are a lot of working parts here and it would great to know where or on what I should be focusing to isolate the configuration issue. It would be nice to have a set of local commands identified for each stage that could be run to verify the infrastructure from the ground up.
for example -
I'm debugging my failing darjeeling coverage using the locally enabled logger and printing the TestSuitCoverage
dictionary, I'm seeing that the test.sh
isn't working properly.
DEBUG:bugzoo.core.coverage:[TestSuiteCoverage] coverage instruction dictionary:
{'n1': {'coverage': {}, 'outcome': {'passed': False, 'response': {'code': 127, 'duration': 0.1892676030001894, 'output': '/bin/bash: ./test.sh: No such file or directory\r'}}, 'test': 'n1'}, 'n2': {'coverage': {}, 'outcome': {'passed': False, 'response': {'code': 127, 'duration': 0.16958742499991786, 'output': '/bin/bash: ./test.sh: No such file or directory\r'}}, 'test': 'n2'}, 'p1': {'coverage': {}, 'outcome': {'passed': False, 'response': {'code': 127, 'duration': 0.18321107100018708, 'output': '/bin/bash: ./test.sh: No such file or directory\r'}}, 'test': 'p1'},
although i've set the context properly in the yml files:
DEBUG:bugzoo.mgr.source:parsing bug: {"name": "sefcom:fauxware", "image": "sefcom:fauxware", "dataset": "sefcom", "program": "fauxware.i", "languages": ["c"], "source-location": "/experiment/src", "test-harness": {"context": "/experiment/src", "command": "./test.sh __ID__", "time-limit": 2500, "failing": 2, "passing": 16, "type": "genprog"}, "compiler": {"time-limit": 300, "type": "configure-and-make"}, "coverage": {"context": "/experiment/src", "type": "gcov", "files-to-instrument": ["fauxware.c"]}}
it would be great to be able to verify the commands being executed or sent to the docker image either independently from the darjeeling infrastructure or directly from darjeeling logs.