Unknown problems occur on setting up Darjeeling.
#13
Hi. I would like to run Darjeeling, but if I follow the instructions at https://github.com/squaresLab/Darjeeling, I get an error. I used centOS and VMware.
Can you tell me what kind of environment you were running?
Could you please tell me a bit more details? For example, I am curious about commands or things to install.
Hi. I don't suppose that you could share some more details about the error?
Also, what versions of Docker and Python 3 are you using? And are you using pipenv or virtualenv to install Darjeeling? If you're not using either of those, I would recommend using pipenv to prevent Darjeeling from interfering with your system's Python installation: https://github.com/pypa/pipenv
Hi tank you for your help!
I Use python 3.6.8 and Docker 1.13.1 and i reference https://github.com/squaresLab/Darjeeling
and i use VMware with centOS
I input to terminal
$ pip install virtualenv
$ pip install pipenv
$ virtualenv darjeeling
$ cd darjeeling
$ source bin/activate
(darjeeling) $ pip install darjeeling
but I cannot run darjeeling
and i met error
Traceback (most recent call last):
File "/bin/darjeeling", line 11, in
load_entry_point('darjeeling==0.1.13', 'console_scripts', 'darjeeling')()
File "/usr/lib/python3.6/site-packages/darjeeling/cli/ init.py", line 392, in main
app.run()
File "/usr/lib/python3.6/site-packages/cement/core/foundation.py", line 916, in run
return_val = self.controller._dispatch()
File "/usr/lib/python3.6/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
return func()
File "/usr/lib/python3.6/site-packages/darjeeling/cli/ init.py", line 308, in repair
files)
File "/usr/lib/python3.6/site-packages/kaskara/analysis.py", line 50, in build
ignore_exit_code=ignore_exit_code)
File "/usr/lib/python3.6/site-packages/kaskara/loops.py", line 38, in find_loops
raise BondException(msg)
kaskara.exceptions.BondException: loop finder exited with non-zero code: 1
how to i debug this error?
so i want to know environment that i have to use.
thank you.
It looks like your environment is setup correctly and that you have everything installed. The issue seems to be with the BugZoo bug scenario that you're trying to repair. Can you share the configuration file that you're using for Darjeeling?
This is my configuration file and I proceeded by changing the snapshot in the configuration file. Some snapshots have this error.
version: '1.0'
snapshot: 'manybugs:gzip:2010-01-30-fc00329e3d-1204630c96'
language: c
seed: 0
threads: 16
localization:
type: spectrum
metric: tarantula
exclude-files:
- foo.c
algorithm:
type: exhaustive
transformations:
schemas:
- type: delete-statement
- type: replace-statement
- type: prepend-statement
optimizations:
ignore-equivalent-prepends: yes
ignore-dead-code: yes
ignore-string-equivalent-snippets: yes
resource-limits:
candidates: 5000
Thank you
Did your bug scenario build its bugzoo container correctly?
You should have followed the directions from here: https://squareslab.github.io/BugZoo/
to add the bugzoo source and build the bugzoo container via bugzoo bug build.
You may also want to run the bugzoo bug coverage on your bugzoo container to make sure that the code coverage that Darjeeling uses is generated correctly.
@cjfgnsdk12 @pdreiter I can confirm that I can reproduce your error on my machine. It appears that the BugZoo scenario is fine, but that the static analysis step (performed by Kaskara) is failing. I'm debugging the issue now.
(Also, thanks @pdreiter for responding to this post!)
Ah. It looks like the Docker images for the gzip scenarios don't have Bear. Going forward, the plan is to get Darjeeling to automatically run bear during a rebuild to obtain a compile_commands.json, but for now, the quick fix is to add Bear to the Gzip images. I'll go ahead and do that now.
@cjfgnsdk12 @pdreiter The issue appears to be resolved. You'll need to git pull your copy of ManyBugs (https://github.com/squaresLab/ManyBugs) and then you'll need to rebuild any of the gzip images that you wish to repair (e.g., bugzoo bug build manybugs:gzip:2010-01-30-fc00329e3d-1204630c96).
Apologies for the late reply -- I hope this fixes the problem for you!
Thank you!
I solved this error("kaskara.exceptions.BondException: loop finder exited with non-zero code: 1")
but i met new error while i try this command (darjeeling repair gzip.yml)
and I met error message :
(evaluated candidate: Candidate<#3fb4b055>
candidate limit has been reached: stopping search.)
and on server
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 2463, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
response = self.handle_exception(e)
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1866, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functionsrule.endpoint
File "/usr/lib/python3.6/site-packages/bugzoo/server/init.py", line 49, in wrapper
response = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/bugzoo/server/init.py", line 256, in test_container
outcome = daemon.containers.test(container, test)
File "/usr/lib/python3.6/site-packages/bugzoo/mgr/container.py", line 399, in execute
verbose=verbose)
File "/usr/lib/python3.6/site-packages/bugzoo/mgr/container.py", line 572, in command
out = self.__api_docker.exec_start(exec_id, stream=True)
File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/docker/api/exec_api.py", line 163, in exec_start
stream=True
File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 289, in _post_json
return self._post(url, data=json.dumps(data2), **kwargs)
File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 226, in _post
return self.post(url, **self._set_request_timeout(kwargs))
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(24, 'Too many open files'))
How to solve this error??
can I solve this error through expand OS's file limit?
Chris and I worked through a similar problem in the issue I opened in response to this gitq:
https://gitq.com/squaresLab/Darjeeling/topics/11/unexpected-response-from-server-500-too-many-open-files
There’s a bug in the docker api that isn’t closed yet because they needed to resolve win vs Linux issue. If you pull in the docker api fix into your virtual env area, it should fix it for you. At least it fixed my issue.
Thank you for your help!
I solve file limit error through your help and changing OS's config!
but, while I execute (sudo darjeeling repair gzip.yml) I met new error message
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 2463, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
response = self.handle_exception(e)
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1866, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/lib/python3.6/site-packages/bugzoo/server/__init__.py", line 49, in wrapper
response = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/bugzoo/server/__init__.py", line 256, in test_container
outcome = daemon.containers.test(container, test)
File "/usr/lib/python3.6/site-packages/bugzoo/mgr/container.py", line 399, in execute
verbose=verbose)
File "/usr/lib/python3.6/site-packages/bugzoo/mgr/container.py", line 578, in command
for line in out:
File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 409, in <genexpr>
gen = (data for (_, data) in gen)
File "/usr/lib/python3.6/site-packages/docker/utils/socket.py", line 94, in frames_iter_no_tty
(stream, n) = next_frame_header(socket)
File "/usr/lib/python3.6/site-packages/docker/utils/socket.py", line 66, in next_frame_header
data = read_exactly(socket, 8)
File "/usr/lib/python3.6/site-packages/docker/utils/socket.py", line 51, in read_exactly
next_data = read(socket, n - len(data))
File "/usr/lib/python3.6/site-packages/docker/utils/socket.py", line 31, in read
select.select([socket], [], [])
ValueError: filedescriptor out of range in select()
-->
data = read_exactly(socket, 8)
File "/usr/lib/python3.6/site-packages/docker/utils/socket.py", line 51, in read_exactly
next_data = read(socket, n - len(data))
File "/usr/lib/python3.6/site-packages/docker/utils/socket.py", line 31, in read
select.select([socket], [], [])
ValueError: filedescriptor out of range in select()How to solve this error??
Thank you so much!!
@cjfgnsdk12 Unfortunately, there's a Windows vs. Linux compatibility issue in the DockerPy source code that causes this bug to manifest: https://github.com/docker/docker-py/issues/2278
There's a pending PR on the DockerPy repo that fixes this problem: https://github.com/docker/docker-py/pull/2279
I believe that @pdreiter overcame the issue by cloning the DockerPy repo, applying the patch in that PR manually, and installing to the virtualenv.
(For more info, see: https://github.com/squaresLab/BugZoo/pull/341)
thank you I solve error.
I changed my computer to something better and then that error was disappear.
also I change OS to Ubuntu(previous OS was centOS) and I don't use VMware.
so i met this message ("waiting for pending evaluations to complete.") and I wait 2days but it still no changes..
How long should I wait??
Is there anything wrong with me?
This is my config
version: '1.0'
snapshot: 'manybugs:gzip:2009-09-26-a1d3d4019d-f17cbd13a1'
language: c
seed: 0
threads: 16
localization:
type: spectrum
metric: tarantula
exclude-files:
- foo.c
algorithm:
type: exhaustive
transformations:
schemas:
- type: delete-statement
- type: replace-statement
- type: prepend-statement
optimizations:
ignore-equivalent-prepends: yes
ignore-dead-code: yes
ignore-string-equivalent-snippets: yes
resource-limits:
candidates: 500000
time-minutes: 360000
"Waiting for pending evaluations to complete" shouldn't take as long as that. (It usually takes a few minutes, depending on the bug that's being repaired.)
Can you CTRL+C/SIGTERM the Darjeeling process and upload the log file? I haven't come across this failure mode before, but it looks like Darjeeling isn't correctly enforcing a time limit at some point in its process.
Sorry where is log file..?
This is my script after CTRL+C is this right?
if not correct, please teach me path.
I really thank you for your help!
waiting for pending evaluations to complete.
^CTraceback (most recent call last):
File "/usr/local/bin/darjeeling", line 11, in
load_entry_point('darjeeling==0.1.13', 'console_scripts', 'darjeeling')()
File "/usr/local/lib/python3.6/dist-packages/darjeeling/cli/ init.py", line 392, in main
app.run()
File "/usr/local/lib/python3.6/dist-packages/cement/core/foundation.py", line 916, in run
return_val = self.controller._dispatch()
File "/usr/local/lib/python3.6/dist-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
return func()
File "/usr/local/lib/python3.6/dist-packages/darjeeling/cli/ init.py", line 352, in repair
searcher.close()
File "/usr/local/lib/python3.6/dist-packages/darjeeling/searcher/base.py", line 277, in close
for candidate, outcome in self.as_evaluated():
File "/usr/local/lib/python3.6/dist-packages/darjeeling/searcher/base.py", line 234, in as_evaluated
yield from self.__evaluator.as_completed()
File "/usr/local/lib/python3.6/dist-packages/darjeeling/evaluator.py", line 295, in as_completed
yield q.get()
File "/usr/lib/python3.6/queue.py", line 164, in get
self.not_empty.wait()
File "/usr/lib/python3.6/threading.py", line 295, in wait
waiter.acquire()
File "/usr/local/lib/python3.6/dist-packages/cement/core/foundation.py", line 133, in cement_signal_handler
raise exc.CaughtSignal(signum, frame)
cement.core.exc.CaughtSignal: Caught signal 2
Hello!
I wanna ask you one of Darjeeling.
How long time i have to wait while "evaluating candidate"?
My snapshot is "manybugs:gzip:2010-01-30-fc00329e3d-1204630c96" and I wait 3600 minute
and I met time limit message.
So I changed time limit to 36000.
How long time I have to Wait?
Thank you!
Apologies! It turns out that there was a bug in the time limit handling:
Can you try to use the latest version of master for Darjeeling?
I tried latest version of Darjeeling!
I used below config (If I use candidates: 5000 then I met candidate limit so I changed that ).
But I met different error message
"all candidate patches have been exhausted"
I want to success to run Darjeeling.
Have you ever seen it complete? If so, please let me know how you did it.
version: '1.0'
snapshot: 'manybugs:gzip:2010-01-30-fc00329e3d-1204630c96'
language: c
seed: 0
threads: 16
localization:
type: spectrum
metric: tarantula
exclude-files:
- foo.c
algorithm:
type: exhaustive
transformations:
schemas:
- type: delete-statement
- type: replace-statement
- type: prepend-statement
optimizations:
ignore-equivalent-prepends: yes
ignore-dead-code: yes
ignore-string-equivalent-snippets: yes
resource-limits:
candidates: 500000
time-minutes: 3600