Grid does not handle w3c capabilities correctly #3808

ghost · 2017-04-11T00:29:42Z

Meta -

OS: Mac OS El Capitan 10.11.6

Selenium Version: 3.3.3

Browser: Firefox

Browser Version: 52.0.2 (64-bit)

Expected Behavior -

In python, the following should create a Remote driver:

from selenium import webdriver

cap = {'platform': 'ANY', 'browserName': 'firefox', 'version': '', 'marionette': True, 'javascriptEnabled': True}
driver = webdriver.Remote(command_executor='http://<my_server_ip>:4444/wd/hub', desired_capabilities=cap)

(note: this works with selenium 2.53.6 with python2.7 when executed against the same server)

Actual Behavior -

Thows exception: KeyError: 'message'

When I pdb into this I see the following response from the server:

{'value': '{"status":13,"value":{"class":"java.lang.NullPointerException","stackTrace":[{"fileName":"HashMap.java","className":"java.util.HashMap","methodName":"putMapEntries","lineNumber":500},{"fileName":"HashMap.java","className":"java.util.HashMap","methodName":"putAll","lineNumber":784},{"fileName":"DesiredCapabilities.java","className":"org.openqa.selenium.remote.DesiredCapabilities","methodName":"\u003cinit\u003e","lineNumber":54},{"fileName":"RequestHandler.java","className":"org.openqa.grid.web.servlet.handler.RequestHandler","methodName":"process","lineNumber":104},{"fileName":"DriverServlet.java","className":"org.openqa.grid.web.servlet.DriverServlet","methodName":"process","lineNumber":83},{"fileName":"DriverServlet.java","className":"org.openqa.grid.web.servlet.DriverServlet","methodName":"doPost","lineNumber":67},{"fileName":"HttpServlet.java","className":"javax.servlet.http.HttpServlet","methodName":"service","lineNumber":707},{"fileName":"HttpServlet.java","className":"javax.servlet.http.HttpServlet","methodName":"service","lineNumber":790},{"fileName":"ServletHolder.java","className":"org.seleniumhq.jetty9.servlet.ServletHolder","methodName":"handle","lineNumber":841},{"fileName":"ServletHandler.java","className":"org.seleniumhq.jetty9.servlet.ServletHandler","methodName":"doHandle","lineNumber":543},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextHandle","lineNumber":188},{"fileName":"SessionHandler.java","className":"org.seleniumhq.jetty9.server.session.SessionHandler","methodName":"doHandle","lineNumber":1584},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextHandle","lineNumber":188},{"fileName":"ContextHandler.java","className":"org.seleniumhq.jetty9.server.handler.ContextHandler","methodName":"doHandle","lineNumber":1228},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextScope","lineNumber":168},{"fileName":"ServletHandler.java","className":"org.seleniumhq.jetty9.servlet.ServletHandler","methodName":"doScope","lineNumber":481},{"fileName":"SessionHandler.java","className":"org.seleniumhq.jetty9.server.session.SessionHandler","methodName":"doScope","lineNumber":1553},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextScope","lineNumber":166},{"fileName":"ContextHandler.java","className":"org.seleniumhq.jetty9.server.handler.ContextHandler","methodName":"doScope","lineNumber":1130},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"handle","lineNumber":141},{"fileName":"HandlerWrapper.java","className":"org.seleniumhq.jetty9.server.handler.HandlerWrapper","methodName":"handle","lineNumber":132},{"fileName":"Server.java","className":"org.seleniumhq.jetty9.server.Server","methodName":"handle","lineNumber":564},{"fileName":"HttpChannel.java","className":"org.seleniumhq.jetty9.server.HttpChannel","methodName":"handle","lineNumber":320},{"fileName":"HttpConnection.java","className":"org.seleniumhq.jetty9.server.HttpConnection","methodName":"onFillable","lineNumber":251},{"fileName":"AbstractConnection.java","className":"org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback","methodName":"succeeded","lineNumber":279},{"fileName":"FillInterest.java","className":"org.seleniumhq.jetty9.io.FillInterest","methodName":"fillable","lineNumber":112},{"fileName":"ChannelEndPoint.java","className":"org.seleniumhq.jetty9.io.ChannelEndPoint$2","methodName":"run","lineNumber":124},{"fileName":"QueuedThreadPool.java","className":"org.seleniumhq.jetty9.util.thread.QueuedThreadPool","methodName":"runJob","lineNumber":672},{"fileName":"QueuedThreadPool.java","className":"org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2","methodName":"run","lineNumber":590},{"fileName":"Thread.java","className":"java.lang.Thread","methodName":"run","lineNumber":745}]}}', 'status': 500}

Steps to reproduce -

Execute the above steps in python using selenium 3.3.3, firefox 52.0.2, geckodriver 0.15.0. The server is using selenium-server-standalone-3.3.1.jar on ubuntu 16.10. I'm happy to provide the ip of the server privately if it'll help.

ghost · 2017-04-11T00:40:56Z

Also, testing with curl seems to work fine:

curl -v -X POST http://<my_server_ip>:4444/wd/hub/session -d '{"desiredCapabilities":{"browserName":"firefox", "marionette": "true", "javascriptEnabled": "true"}}'

curl -v 'http://<my_server_ip>:4444/wd/hub/session/SESSION_ID'

ghost · 2017-04-11T00:49:13Z

Ok, looks it actually works if I change the port to 5555.

I'm guessing that's connecting directly to a node rather than the hub? Why would port 4444 work with selenium 2.5 but not 3.3.3?

foxleoly · 2017-04-11T06:28:35Z

I met the same issue. and Using the same command on selenium 3.3.0 working fine. Now I'm downgrade to 3.3.0.

In [1]: from selenium.webdriver import *

In [2]: dri = Remote('http://my_sever_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

KeyError Traceback (most recent call last)
in ()
----> 1 dri = Remote('http://my_server_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in init(self, command_executor, desired_capabilities, browser_profile, proxy, keep_alive, file_detector)
96 warnings.warn("Please use FirefoxOptions to set browser profile",
97 DeprecationWarning)
---> 98 self.start_session(desired_capabilities, browser_profile)
99 self._switch_to = SwitchTo(self)
100 self._mobile = Mobile(self)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in start_session(self, capabilities, browser_profile)
183 parameters = {"capabilities": w3c_caps,
184 "desiredCapabilities": capabilities}
--> 185 response = self.execute(Command.NEW_SESSION, parameters)
186 if 'sessionId' not in response:
187 response = response['value']

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
247 response = self.command_executor.execute(driver_command, params)
248 if response:
--> 249 self.error_handler.check_response(response)
250 response['value'] = self._unwrap_value(
251 response.get('value', None))

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
108 value = message
109 try:
--> 110 message = message['message']
111 except TypeError:
112 message = None

KeyError: 'message'

In [3]: exit

rgonalo · 2017-04-11T16:25:23Z

I have the same issue with Selenium python client 3.3.2 or 3.3.3 against a Selenium server 3.3.1. It works with Selenium python client 3.3.1.

The problem has been introduced with commit abe1f45. It uses the new W3C start session (send "capabilities" instead of "requiredCapabilities"), but it seems that Selenium 3.3.1 server still needs "requiredCapabilities".

jaiaravindj · 2017-04-14T06:51:41Z

I have downgraded selenium, selenium standalone server and it works fine.

gennadykr · 2017-04-19T13:51:20Z

I can create Remote driver if run server locally as
java -jar selenium-server-standalone-3.3.1.jar

But it fails for a local grid started as:

java -jar selenium-server-standalone-3.3.1.jar -role hub
java -jar selenium-server-standalone-3.3.1.jar -role node

I found that this grid works only with "selenium-server-standalone-2.53.1.jar", but do not even with the first 3.x one "selenium-server-standalone-3.0.0.jar"

I use python 3.6, selenium 3.3.3, and command

 wd = webdriver.Remote("http://192.168.50.30:4444/wd/hub",desired_capabilities=DesiredCapabilities.CHROME)

smccarthy · 2017-04-21T21:10:39Z

Here is a full stacktrace using the python bindings 3.4.0 with selenium server (in grid hub/node mode) : https://gist.github.com/smccarthy/6778f2af97b99e301767438de01c62fb

I tried with chromedriver 2.28 and 2.29 , but I seem to get the same issue with DesiredCapabilities.FIREFOX

lmtierney · 2017-04-21T21:31:18Z

This appears to be a grid issue, it does not like 'capabilities' being in the parameters for new session

lmtierney · 2017-04-21T21:48:02Z

Or rather that it expects desiredCapabilities to be inside capabilities

selenium/java/server/src/org/openqa/grid/web/servlet/handler/WebDriverRequest.java

Line 69 in 02a0df9

    
           return new JsonToBeanConverter().convert(Map.class, map.getAsJsonObject("capabilities").getAsJsonObject("desiredCapabilities"));

lmtierney · 2017-04-22T11:37:28Z

@shs96c you'd probably be best at handling the fix for this

mhomb · 2017-04-28T08:02:29Z

Is there any update on this? I am still stuck on 3.0.1 because of this bug.

bjharo · 2017-04-28T20:07:16Z

I'm having the same issue using the .Net bindings on the client side.

OS: Windows 10
Selenium Version: 3.4
Browser: Chrome
Browser Version: 57
Grid Version (Hub & Node): 3.4

I was testing this setup with a locally running hub and node on my dev laptop before upgrading our actual grid. Anytime I request a browser directly from the hub, I get an error. This is not a problem using 3.3 nor is it a problem using 3.4 when I bypass the grid and use a local browser.

lmtierney · 2017-04-29T19:42:11Z

@bjharo see @jimevans's explanation here #3929 (comment)

jimevans · 2017-04-30T19:52:50Z

And the reason it's showing up now is that 3.4 is the first release to consistently attempt to send the W3C spec-compliant payload for a new session command (containing a capabilities property) across the wire. This is where the bug is in the hub code.

sajidchingchong · 2017-05-04T08:31:02Z

Do we have any work around of it?

jakeczyz · 2017-05-08T01:47:21Z

I also am seeing this problem. My workaround in Python has been to force an earlier version of selenium when installing. I.e., pip3 uninstall selenium && pip3 install selenium==3.3.1

Grimlek · 2017-05-08T13:05:02Z

As a workaround this is what worked for me....go into the selenium webdriver client

I am using python client

go into the remote/webdriver.py

you will see this within the start_session(some args) method

parameters = {"capabilities": w3c_caps,
                      "desiredCapabilities": capabilities}

remove the "capabilities": w3c_caps

so you end up with:

parameters = { "desiredCapabilities": capabilities}

I am sure that this is very similar to the other client bindings. Just a temporary workaround until this can be fixed.

barancev · 2017-08-11T14:21:48Z

Should be fixed in version 3.5, please reopen the issue if it's not true.

cristian-sobral-fluke · 2017-08-15T14:11:05Z

Not working for me in version 3.5. I had to downgrade to v3.3

barancev · 2017-08-15T14:19:06Z

@CristianSw "Not working" is a zero info message. What browser, what capabilities, what error appears?

wildroo · 2017-08-16T05:59:38Z

v3.5.1 fixes the issue for C#/Chrome/Windows
@CristianSw, have you upgraded both your project's Selenium and the selenium-server-standalone jar?

Thanks

cristian-sobral-fluke · 2017-08-16T09:40:15Z

@fatdh thanks for the update, I've just upgraded to 3.5.1, however, I'm using selenium grid running on docker and the docker image is still using selenium-server-standalone 3.4.0

@barancev Let's backtrack a little bit.. I wouldn't say that

Should be fixed in version 3.5

is very informing.. was it fixed or not? I was just trying to help & signal that the original issue might need to be reopened.

dado21780 · 2017-08-16T10:23:51Z

Would it possible to get the version 3.5 of the java selenium libraries on the central maven repository?
Thanks!

firstchild78 · 2017-08-17T01:28:17Z

I can verify that the 3.5.1 fixed the issue for me.(C#) I'm also using Selenium Grid v3.4.0
Thanks.

mpgo13 · 2017-08-17T06:26:10Z

Should this be fixed in the standalone mode as well?

I'm getting the following (using FF54 and geckodriver 0.18.0):

selenium.common.exceptions.WebDriverException: Message: mouseMoveTo
Build info: version: '3.5.0', revision: '8def36e068', time: '2017-08-10T23:00:22.093Z'
System info: host: 'myhost', ip: '130.194.77.215', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.12.6', java.version: '1.8.0_101'
Driver info: driver.version: unknown
Stacktrace:
    at org.openqa.selenium.remote.http.AbstractHttpCommandCodec.encode (AbstractHttpCommandCodec.java:222)
    at org.openqa.selenium.remote.http.AbstractHttpCommandCodec.encode (AbstractHttpCommandCodec.java:119)
    at org.openqa.selenium.remote.server.ProtocolConverter.handle (ProtocolConverter.java:72)
    at org.openqa.selenium.remote.server.ServicedSession.execute (ServicedSession.java:81)
    at org.openqa.selenium.remote.server.WebDriverServlet.lambda$handle$0 (WebDriverServlet.java:231)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
    at java.lang.Thread.run (Thread.java:745)

mpgo13 · 2017-08-17T06:44:36Z

Sorry, my fault, it works now :)

manoj9788 · 2017-08-17T06:53:07Z

I think we can close this issue.

barancev · 2017-08-17T20:46:37Z

FYI: version 3.5.1 is available in maven central

timbru31 · 2017-08-22T16:43:53Z

@mpgo13

Sorry, my fault, it works now :)

Can you share what your fix is/was? Experiencing the same issue with v3.5.2 and geckodriver v0.18 and hoped that v3.5.0 should fix this.

edit:
note to future self: read comments above
note to other readers: pass -enablePassThrough to the standalone

mpgo13 · 2017-08-22T20:22:50Z

Can you share what your fix is/was? Experiencing the same issue with v3.5.2 and geckodriver v0.18 and hoped that v3.5.0 should fix this.

I was upgrading the server while still using old python selenium.

gwtuser · 2017-09-08T02:11:34Z

I was redirected to this issue report by:
"UnsupportedCommandException mouseMoveTo with 3.4 on firefox with geckodriver 0.16.1 #4008"

FF 52.3.0
geckodriver 18
Selenium 3.5.3

I am not using a standalone, so how can I pass "-enablePassthrough"? I'm just creating FirefoxDriver(), and specifying the location of the geckodriver via System.setProperty("webdriver.gecko.driver", ...) in Java.
I don't want to use "legacy" mode (which is not supported in FF55+, right?), but I also don't want to have to switch to RemoteWebDriver.

Is using FirefoxDriver + gecko a supported path forward?

usmonster · 2017-09-08T06:28:43Z

FYI, -enablePassThrough requires a boolean argument and is enabled by default since v3.5.0. On my side, I had to set -enablePassThrough false on my nodes (same with standalone) to get things working with FF55+ & geckodriver 18.

gwtuser · 2017-09-08T06:49:02Z

Yeah, I'm aware it requires an argument. I was trying to figure where to apply that option, since I'm not running any nodes, nor a standalone Selenium server. I'm launching WebDriver by calling new FirefoxDriver() in Java, and then driver.get() on a URL. Perhaps the redirect from issue #4008 was incorrect, as this issue refers to "Grid" and I'm not using that, nor standalone WebDriver.

codingphil · 2017-09-15T16:16:13Z

Is the Pass Through Mode the way going forward for supporting W3C compliant webdrivers in the standalone server or is it planned to make it working also without Pass Through Mode?

shs96c · 2017-09-15T19:40:38Z

All modes should work.

…

On Fri, Sep 15, 2017 at 5:16 PM, Philip Lonsing ***@***.***> wrote: Is the *Pass Through Mode* the way going forward for supporting W3C webdrivers in the standalone server or is it planned to make it working also without Pass Through Mode? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3808 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABuRX85zXmJcXyGuPE5clnwh67wCKhLks5siqLOgaJpZM4M5en1> .

DaryaOrlova · 2017-09-20T16:17:37Z

Issue still reproduces for me (Ruby v2.3.3). I'm using Selenium Grid v3.3.1 running on docker and selenium-webdriver v3.5.2 on local machine.

the error is:

undefined method `capabilities' for nil:NilClass (NoMethodError)
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/w3c/capabilities.rb:154:in `from_oss'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:183:in `merged_capabilities'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:103:in `create_session'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:54:in `handshake'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/driver.rb:39:in `initialize'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/common/driver.rb:58:in `new'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/common/driver.rb:58:in `for'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver.rb:86:in `for'
/Users/odachka/aytm/aytmRegressionTests/pvRegressionTests/features/support/hooks.rb:12:in `<top (required)>'

lmtierney · 2017-09-20T16:21:43Z

@DaryaOrlova that's because you're using grid v3.3.1, this was fixed in the grid code not the ruby gem

tkueck · 2017-09-26T23:08:35Z

So we've been running into this at work with the C# bindings. By making these changes starting at line 1108 in RemoteWebDriver.cs we are now capable of using desired capabilities as worked prior to the w3c changes. https://gist.github.com/tkueck/178eb16058c9e3b240107dcc2c4cb2f3

lmtierney added C-py C-server and removed C-server labels Apr 19, 2017

lmtierney added C-grid and removed C-py labels Apr 21, 2017

lmtierney mentioned this issue Apr 22, 2017

remote webdriver with selenium grid 3.4 not work #3879

Closed

lmtierney changed the title ~~Creating new Remote driver works in selenium 2.5 but not with 3.3.3~~ Grid does not handle w3c capabilities correctly Apr 22, 2017

lmtierney mentioned this issue Apr 29, 2017

Unable To Create Remote WebDriver instance in c# nuget version 3.4.0 #3929

Closed

shs96c self-assigned this May 2, 2017

jimevans mentioned this issue May 7, 2017

Issue with 3.4.0 NuGet Package #3984

Closed

desperado0807 mentioned this issue May 12, 2017

Selenium grid and Appium 1.6.4 / can't launch my test appium/appium#8439

Closed

This was referenced May 12, 2017

Couldn't create RemoteWebDriver with v 3.4.0. Null pointer exception in Grid server #4012

Closed

UnsupportedCommandException mouseMoveTo with 3.4 on firefox with geckodriver 0.16.1 #4008

Closed

barancev mentioned this issue Aug 1, 2017

3.3.1 versus geckodriver 0.15 firefox 51: "Not all requiredCapabilities could be met" #3711

Closed

unickq mentioned this issue Aug 3, 2017

RemoteWebDriver Grid unickq/SpecFlow.Selenium.Plugin#4

Closed

barancev closed this as completed Aug 11, 2017

titusfortner mentioned this issue Aug 17, 2017

Improper Handshake formatting in 3.5 #4482

Closed

timbru31 mentioned this issue Aug 22, 2017

Upgrade dependency on selenium-webdriver to match what webdriver-manager is downloading angular/protractor#4451

Open

gwtuser mentioned this issue Sep 8, 2017

When using FirefoxDriver+geckodriver, Action.mouseMoveTo() thows an UnsupportedCommandException #4665

Closed

elgalu mentioned this issue Sep 15, 2017

Selenium 3.4.0 elgalu/docker-selenium#149

Closed

ryltar mentioned this issue Dec 13, 2017

[bug] firefox version > 47 + Gecko driver + browserstack cerberustesting/cerberus-core#1539

Closed

lock bot locked and limited conversation to collaborators Aug 17, 2019

Grid does not handle w3c capabilities correctly #3808

Grid does not handle w3c capabilities correctly #3808

Comments

ghost commented Apr 11, 2017 • edited by ghost

Meta -

Expected Behavior -

Actual Behavior -

Steps to reproduce -

ghost commented Apr 11, 2017

ghost commented Apr 11, 2017

foxleoly commented Apr 11, 2017

In [2]: dri = Remote('http://my_sever_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

rgonalo commented Apr 11, 2017

jaiaravindj commented Apr 14, 2017

gennadykr commented Apr 19, 2017

smccarthy commented Apr 21, 2017 • edited

lmtierney commented Apr 21, 2017

lmtierney commented Apr 21, 2017 • edited

lmtierney commented Apr 22, 2017

mhomb commented Apr 28, 2017

bjharo commented Apr 28, 2017

lmtierney commented Apr 29, 2017

jimevans commented Apr 30, 2017

sajidchingchong commented May 4, 2017

jakeczyz commented May 8, 2017

Grimlek commented May 8, 2017 • edited

barancev commented Aug 11, 2017

cristian-sobral-fluke commented Aug 15, 2017

barancev commented Aug 15, 2017

wildroo commented Aug 16, 2017 • edited

cristian-sobral-fluke commented Aug 16, 2017

dado21780 commented Aug 16, 2017

firstchild78 commented Aug 17, 2017

mpgo13 commented Aug 17, 2017 • edited

mpgo13 commented Aug 17, 2017

manoj9788 commented Aug 17, 2017

barancev commented Aug 17, 2017

timbru31 commented Aug 22, 2017 • edited

mpgo13 commented Aug 22, 2017

gwtuser commented Sep 8, 2017

usmonster commented Sep 8, 2017

gwtuser commented Sep 8, 2017 • edited

codingphil commented Sep 15, 2017 • edited

shs96c commented Sep 15, 2017 via email

DaryaOrlova commented Sep 20, 2017

lmtierney commented Sep 20, 2017

tkueck commented Sep 26, 2017

ghost commented Apr 11, 2017 •

edited by ghost

smccarthy commented Apr 21, 2017 •

edited

lmtierney commented Apr 21, 2017 •

edited

Grimlek commented May 8, 2017 •

edited

wildroo commented Aug 16, 2017 •

edited

mpgo13 commented Aug 17, 2017 •

edited

timbru31 commented Aug 22, 2017 •

edited

gwtuser commented Sep 8, 2017 •

edited

codingphil commented Sep 15, 2017 •

edited