PhantomJS unmaintained, possible replacement of PhantomJS? (Error in run_phantomjs() : Failed to start phantomjs)

[sgrubsmyon]

Having webdriver version 1.0.6 installed on an Ubuntu 22.04, I cannot use webdriver any more since Ubuntu chose to switch to the more recent version 3.x of OpenSSL, whereas PahntomJS requires OpenSSL version 1.x.

This is the error i get inside R:

> library(webdriver)
> pjs <- run_phantomjs()
Error in run_phantomjs() : Failed to start phantomjs. stdout + stderr:
>  Auto configuration failed
>  139657342031808:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(libproviders.so): libproviders.so: cannot open shared object file: No such file or directory
>  139657342031808:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
>  139657342031808:error:0E07506E:configuration file routines:MODULE_LOAD_DSO:error loading dso:conf_mod.c:285:module=providers, path=providers
>  139657342031808:error:0E076071:configuration file routines:MODULE_RUN:unknown module name:conf_mod.c:222:module=providers

This error has nothing to do with webdriver at all, the problem lies at PhantomJS. Even this does not work on the Linux commandline:

$ phantomjs --version
Auto configuration failed
140611516815296:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(libproviders.so): libproviders.so: cannot open shared object file: No such file or directory
140611516815296:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
140611516815296:error:0E07506E:configuration file routines:MODULE_LOAD_DSO:error loading dso:conf_mod.c:285:module=providers, path=providers
140611516815296:error:0E076071:configuration file routines:MODULE_RUN:unknown module name:conf_mod.c:222:module=providers

I found this StackOverflow thread explaining the issue: https://stackoverflow.com/questions/72496479/phantomjs-launcher-fails-on-latest-pop-os-cannot-find-shared-library-libprovid

The only solution seems to be to manually compile OpenSSL 1.1.1, which is not optimal (and grows worse with time, because using outdated OpenSSL with potential security problems). A workaround is to turn off OpenSSL completely, which of course is even more suboptimal:

> library(webdriver)
> Sys.setenv("OPENSSL_CONF" = "/dev/null")
> pjs <- run_phantomjs() # works OK

My question is: what are the plans of webdriver concerning PhantomJS? Are there plans to replace PhantomJS in the long term? PhantomJS seems unmaintained already since March 2018 (see ariya/phantomjs#15344) and the GitHub repo https://github.com/ariya/phantomjs was archived and is read-only since May 2023. There is no going forward with PhantomJS. So what about webdriver?

I found (https://stackoverflow.com/questions/14964108/alternative-to-phantomjs-for-testing) that SlimerJS (https://slimerjs.org/) might be an alternative to PhantomJS worth looking at. But I cannot judge how difficult it will be for webdriver to move over. Webdrivers' README.md says "In theory in works with other WebDriver clients as well", but SlimerJS uses Gecko. Would webdriver work better with Selenium? (https://www.selenium.dev/documentation/webdriver/) But then, is it better to completely abandon the webdriver R package and use other packages like RSelenium or selenium instead?

[gaborcsardi]

As I understand, https://github.com/rstudio/chromote is a modern alternative of webdriver.

[sgrubsmyon]

Okay, maybe write this in the README.md file with a note that webdriver (due to defunct PhantomJS) is now defunct (at least on Ubuntu) and will not be developed further, pointing potential users to chromote?

[dmurdoch]

I've been using RSelenium for several years, but had recent issues using it with Chrome. (I updated both MacOS and Chrome recently, and now RSelenium won't start.) I tried webdriver and chromote as replacements, and while both work, I'd have to say that the webdriver interface is a lot nicer: chromote just has too much in it, and some common tasks are hidden in obscure places.

Your docs say that webdriver works in principle with other browsers besides PhantomJS. Instead of saying it's defunct now, could you possibly work out how to use it with another browser?

[gaborcsardi]

@dmurdoch It used to work out of the box with Chromedriver. All you had to do was install and start chromedriver, and change the port in webdriver. I don't know if it still works, but I think it is worth a try.

[dmurdoch]

@gaborcsardi, thanks for the suggestion. I tried that, and it wouldn't start because the chromedriver file wasn't marked executable. I fixed that, and now RSelenium is working again. But I'll keep it in mind...