Come ottenere il codice di risposta HTTP usando Selenium WebDriver

È possibile ottenere il codice di risposta di una richiesta http utilizzando Selenium e Chrome o Firefox. Tutto quello che dovete fare è avviare Chrome o Firefox in modalità di registrazione. Vi mostrerò alcuni esempi qui sotto.

Java + Selenio + Cromo Ecco un esempio di java + Selenium + Chrome, ma immagino che possa essere fatto in qualsiasi linguaggio (python, c#, ...).

Tutto quello che dovete fare è dire chromedriver di fare "Rete.consentire". Questo può essere fatto abilitando le prestazioni registrazione.

LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
cap.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);

Al termine della richiesta, tutto ciò che devi fare è ottenere e iterare i log di Perfomance e trovare " Rete.responseReceived " per l'URL richiesto:

LogEntries logs = driver.manage().logs().get("performance");

Ecco il codice:

import java.util.Iterator;
import java.util.logging.Level;

import org.json.JSONException;
import org.json.JSONObject;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.logging.LogEntries;
import org.openqa.selenium.logging.LogEntry;
import org.openqa.selenium.logging.LogType;
import org.openqa.selenium.logging.LoggingPreferences;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

public class TestResponseCode
{
    public static void main(String[] args)
    {
        // simple page (without many resources so that the output is
        // easy to understand
        String url = "http://www.york.ac.uk/teaching/cws/wws/webpage1.html";

        DownloadPage(url);
    }

    private static void DownloadPage(String url)
    {
        ChromeDriver driver = null;

        try
        {
            ChromeOptions options = new ChromeOptions();
            // add whatever extensions you need
            // for example I needed one of adding proxy, and one for blocking
            // images
            // options.addExtensions(new File(file, "proxy.zip"));
            // options.addExtensions(new File("extensions",
            // "Block-image_v1.1.crx"));

            DesiredCapabilities cap = DesiredCapabilities.chrome();
            cap.setCapability(ChromeOptions.CAPABILITY, options);

            // set performance logger
            // this sends Network.enable to chromedriver
            LoggingPreferences logPrefs = new LoggingPreferences();
            logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
            cap.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);

            driver = new ChromeDriver(cap);

            // navigate to the page
            System.out.println("Navigate to " + url);
            driver.navigate().to(url);

            // and capture the last recorded url (it may be a redirect, or the
            // original url)
            String currentURL = driver.getCurrentUrl();

            // then ask for all the performance logs from this request
            // one of them will contain the Network.responseReceived method
            // and we shall find the "last recorded url" response
            LogEntries logs = driver.manage().logs().get("performance");

            int status = -1;

            System.out.println("\nList of log entries:\n");

            for (Iterator<LogEntry> it = logs.iterator(); it.hasNext();)
            {
                LogEntry entry = it.next();

                try
                {
                    JSONObject json = new JSONObject(entry.getMessage());

                    System.out.println(json.toString());

                    JSONObject message = json.getJSONObject("message");
                    String method = message.getString("method");

                    if (method != null
                            && "Network.responseReceived".equals(method))
                    {
                        JSONObject params = message.getJSONObject("params");

                        JSONObject response = params.getJSONObject("response");
                        String messageUrl = response.getString("url");

                        if (currentURL.equals(messageUrl))
                        {
                            status = response.getInt("status");

                            System.out.println(
                                    "---------- bingo !!!!!!!!!!!!!! returned response for "
                                            + messageUrl + ": " + status);

                            System.out.println(
                                    "---------- bingo !!!!!!!!!!!!!! headers: "
                                            + response.get("headers"));
                        }
                    }
                } catch (JSONException e)
                {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }

            System.out.println("\nstatus code: " + status);
        } finally
        {
            if (driver != null)
            {
                driver.quit();
            }
        }
    }
}

L'output è simile a questo:

    Navigate to http://www.york.ac.uk/teaching/cws/wws/webpage1.html

    List of log entries:

    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameAttached","params":{"parentFrameId":"172.1","frameId":"172.2"}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStartedLoading","params":{"frameId":"172.2"}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameNavigated","params":{"frame":{"securityOrigin":"://","loaderId":"172.1","name":"chromedriver dummy frame","id":"172.2","mimeType":"text/html","parentId":"172.1","url":"about:blank"}}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStoppedLoading","params":{"frameId":"172.2"}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStartedLoading","params":{"frameId":"3928.1"}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.requestWillBeSent","params":{"request":{"headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"},"initialPriority":"VeryHigh","method":"GET","mixedContentType":"none","url":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html"},"frameId":"3928.1","requestId":"3928.1","documentURL":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","initiator":{"type":"other"},"loaderId":"3928.1","wallTime":1.47619492749007E9,"type":"Document","timestamp":20226.652971}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.responseReceived","params":{"frameId":"3928.1","requestId":"3928.1","response":{"headers":{"Accept-Ranges":"bytes","Keep-Alive":"timeout=4, max=100","Cache-Control":"max-age=300","Server":"Apache/2.2.22 (Ubuntu)","Connection":"Keep-Alive","Content-Encoding":"gzip","Vary":"Accept-Encoding","Expires":"Tue, 11 Oct 2016 14:13:47 GMT","Content-Length":"1957","Date":"Tue, 11 Oct 2016 14:08:47 GMT","Content-Type":"text/html"},"connectionReused":false,"timing":{"pushEnd":0,"workerStart":-1,"proxyEnd":-1,"workerReady":-1,"sslEnd":-1,"pushStart":0,"requestTime":20226.65335,"sslStart":-1,"dnsStart":0,"sendEnd":31.6569999995409,"connectEnd":31.4990000006219,"connectStart":0,"sendStart":31.5860000009707,"dnsEnd":0,"receiveHeadersEnd":115.645999998378,"proxyStart":-1},"encodedDataLength":-1,"remotePort":80,"mimeType":"text/html","headersText":"HTTP/1.1 200 OK\r\nDate: Tue, 11 Oct 2016 14:08:47 GMT\r\nServer: Apache/2.2.22 (Ubuntu)\r\nAccept-Ranges: bytes\r\nCache-Control: max-age=300\r\nExpires: Tue, 11 Oct 2016 14:13:47 GMT\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\nContent-Length: 1957\r\nKeep-Alive: timeout=4, max=100\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\n\r\n","securityState":"neutral","requestHeadersText":"GET /teaching/cws/wws/webpage1.html HTTP/1.1\r\nHost: www.york.ac.uk\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: en-GB,en-US;q=0.8,en;q=0.6\r\n\r\n","url":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","protocol":"http/1.1","fromDiskCache":false,"fromServiceWorker":false,"requestHeaders":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","Upgrade-Insecure-Requests":"1","Connection":"keep-alive","User-Agent":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36","Host":"www.york.ac.uk","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"en-GB,en-US;q=0.8,en;q=0.6"},"remoteIPAddress":"144.32.128.84","statusText":"OK","connectionId":11,"status":200},"loaderId":"3928.1","type":"Document","timestamp":20226.770012}}}
    ---------- bingo !!!!!!!!!!!!!! returned response for http://www.york.ac.uk/teaching/cws/wws/webpage1.html: 200
    ---------- bingo !!!!!!!!!!!!!! headers: {"Accept-Ranges":"bytes","Keep-Alive":"timeout=4, max=100","Cache-Control":"max-age=300","Server":"Apache/2.2.22 (Ubuntu)","Connection":"Keep-Alive","Content-Encoding":"gzip","Vary":"Accept-Encoding","Expires":"Tue, 11 Oct 2016 14:13:47 GMT","Content-Length":"1957","Date":"Tue, 11 Oct 2016 14:08:47 GMT","Content-Type":"text/html"}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.dataReceived","params":{"dataLength":2111,"requestId":"3928.1","encodedDataLength":1460,"timestamp":20226.770425}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameNavigated","params":{"frame":{"securityOrigin":"http://www.york.ac.uk","loaderId":"3928.1","id":"3928.1","mimeType":"text/html","url":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html"}}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.dataReceived","params":{"dataLength":1943,"requestId":"3928.1","encodedDataLength":825,"timestamp":20226.782673}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.loadingFinished","params":{"requestId":"3928.1","encodedDataLength":2285,"timestamp":20226.770199}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.loadEventFired","params":{"timestamp":20226.799391}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStoppedLoading","params":{"frameId":"3928.1"}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.domContentEventFired","params":{"timestamp":20226.845769}}}
    {"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.requestWillBeSent","params":{"request":{"headers":{"Referer":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","User-Agent":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"},"initialPriority":"High","method":"GET","mixedContentType":"none","url":"http://www.york.ac.uk/favicon.ico"},"frameId":"3928.1","requestId":"3928.2","documentURL":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","initiator":{"type":"other"},"loaderId":"3928.1","wallTime":1.47619492768527E9,"type":"Other","timestamp":20226.848174}}}

    status code: 200

Java + Selenio + Firefox Ho finalmente trovato il trucco per Firefox troppo. È necessario avviare firefox utilizzando le variabili di ambiente MOZ_LOG e MOZ_LOG_FILE e registrare le richieste http a livello di debug (4 = PR_LOG_DEBUG) - map.put("MOZ_LOG", "timestamp,sync,nsHttp:4"). Salva il login un file temporaneo. Successivamente, ottieni il contenuto del file di registro salvato e analizzalo per il codice di risposta (usando alcune semplici espressioni regolari). In primo luogo rilevare l'inizio della richiesta, identificando il suo id (nsHttpChannel::BeginConnect [this=000000CED8094000]), quindi al secondo passaggio, trovare il codice di risposta per tale id richiesta (nsHttpChannel::ProcessResponse [this=000000CED8094000 httpStatus=200]).

import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.commons.io.FileUtils;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.GeckoDriverService;

public class TestFirefoxResponse
{
  public static void main(String[] args)
      throws InterruptedException, IOException
  {
    GeckoDriverService service = null;

    // tell firefox to log http requests
    // at level 4 = PR_LOG_DEBUG: debug messages, notices
    // you could log everything at level 5, but the log file will 
    // be larger. 
    // create a temporary log file that will be parsed for
    // response code
    Map<String, String> map = new HashMap<String, String>();
    map.put("MOZ_LOG", "timestamp,sync,nsHttp:4");
    File tempFile = File.createTempFile("mozLog", ".txt");    
    map.put("MOZ_LOG_FILE", tempFile.getAbsolutePath());      

    GeckoDriverService.Builder builder = new GeckoDriverService.Builder();
    service = builder.usingAnyFreePort()
      .withEnvironment(map)
      .build();

    service.start();      

    WebDriver driver = new FirefoxDriver(service);

    // test 200
     String url = "https://api.ipify.org/?format=text";
    // test 404
    // String url = "https://www.advancedwebranking.com/lsdkjflksdjfldksfj";
    driver.get(url);

    driver.quit();

    String logContent = FileUtils.readFileToString(tempFile);

    ParseLog(logContent, url);
  }

  private static void ParseLog(String logContent, String url) throws MalformedURLException
  {
    // this is how the log looks like when the request starts
    // I have to get the id of the request using a regular expression
    // and use that id later to get the response
    //
    //    2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::BeginConnect [this=000000BFF27A5000]
    //    2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp host=api.ipify.org port=-1
    //    2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp uri=https://api.ipify.org/?format=text
    String pattern = "BeginConnect \\[this=(.*?)\\](?:.*?)uri=(.*?)\\s";

    Pattern p = Pattern.compile(pattern, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
    Matcher m = p.matcher(logContent);

    String urlID = null;
    while (m.find())
    {
      String id = m.group(1);
      String uri = m.group(2);

      if (uri.equals(url))
      {
        urlID = id;
        break;
      }      
    }

    System.out.println("request id = " + urlID);

    // this is how the response looks like in the log file
    // ProcessResponse [this=000000CED8094000 httpStatus=200]
    // I will use another regular espression to get the httpStatus
    //
    //    2017-11-02 14:45:39.296000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::OnStartRequest [this=000000CED8094000 request=000000CED8014BB0 status=0]
    //    2017-11-02 14:45:39.296000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::ProcessResponse [this=000000CED8094000 httpStatus=200]    

    pattern = "ProcessResponse \\[this=" + urlID + " httpStatus=(.*?)\\]";

    p = Pattern.compile(pattern, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
    m = p.matcher(logContent);

    if (m.find())
    {
      String responseCode = m.group(1);
      System.out.println("response code found " + responseCode);
    }
    else
    {
      System.out.println("response code not found");
    }
  }
}

L'output per questo sarà

Id richiesta = 0000007653D67000 codice di risposta trovato 200

Le intestazioni di risposta possono essere trovate anche nel file di log. Puoi prendili se vuoi.

    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp http response [
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   HTTP/1.1 404 Not Found
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Accept-Ranges: bytes
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Cache-control: no-cache="set-cookie"
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Content-Type: text/html; charset=utf-8
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Date: Thu, 02 Nov 2017 14:54:36 GMT
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   ETag: "7969-55bc076a61e80"
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Last-Modified: Tue, 17 Oct 2017 16:17:46 GMT
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Server: Apache/2.4.23 (Amazon) PHP/5.6.24
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Set-Cookie: AWSELB=5F256FFA816C8E72E13AE0B12A17A3D540582F804C87C5FEE323AF3C9B638FD6260FF473FF64E44926DD26221AAD2E9727FD739483E7E4C31784C7A495796B416146EE83;PATH=/
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Content-Length: 31081
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Connection: keep-alive
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp     OriginalHeaders
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Accept-Ranges: bytes
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Cache-control: no-cache="set-cookie"
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Content-Type: text/html; charset=utf-8
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Date: Thu, 02 Nov 2017 14:54:36 GMT
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   ETag: "7969-55bc076a61e80"
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Last-Modified: Tue, 17 Oct 2017 16:17:46 GMT
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Server: Apache/2.4.23 (Amazon) PHP/5.6.24
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Set-Cookie: AWSELB=5F256FFA816C8E72E13AE0B12A17A3D540582F804C87C5FEE323AF3C9B638FD6260FF473FF64E44926DD26221AAD2E9727FD739483E7E4C31784C7A495796B416146EE83;PATH=/
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Content-Length: 31081
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp   Connection: keep-alive
    2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp ]
    2017-11-02 14:54:36.775000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::OnStartRequest [this=0000008A65D85000 request=0000008A65D1F900 status=0]
    2017-11-02 14:54:36.775000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::ProcessResponse [this=0000008A65D85000 httpStatus=404]

È possibile utilizzare BrowserMob proxy per acquisire le richieste e le risposte con un HttpRequestInterceptor. Ecco un esempio in Java:

// Start the BrowserMob proxy
ProxyServer server = new ProxyServer(9978);
server.start();

server.addResponseInterceptor(new HttpResponseInterceptor()
{
    @Override
    public void process(HttpResponse response, HttpContext context)
        throws HttpException, IOException
    {
        System.out.println(response.getStatusLine());
    }
});

// Get selenium proxy
Proxy proxy = server.seleniumProxy();

// Configure desired capability for using proxy server with WebDriver
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);

// Set up driver
WebDriver driver = new FirefoxDriver(capabilities);

driver.get("http://stackoverflow.com/questions/6509628/webdriver-get-http-response-code");

// Close the browser
driver.quit();

Non sono sicuro che questo sia quello che stai cercando, ma ho avuto un obiettivo un po ' diverso è quello di verificare se l'immagine remota esiste e non avrò errore 403, quindi potresti usare qualcosa di simile qui sotto:

public static boolean linkExists(String URLName){
    try {
        HttpURLConnection.setFollowRedirects(false);
        HttpURLConnection con = (HttpURLConnection) new URL(URLName).openConnection();
        con.setRequestMethod("HEAD");
        return (con.getResponseCode() == HttpURLConnection.HTTP_OK);
    }
    catch (Exception e) {
        e.printStackTrace();
        return false;
    }
}

Non è possibile ottenere il codice di risposta HTTP utilizzando direttamente Selenium WebDriver. Il codice può essere ottenuto utilizzando il codice Java e che può essere utilizzato in Selenium WebDriver.

Per ottenere il codice di risposta HTTP da java:

public static int getResponseCode(String urlString) throws MalformedURLException, IOException{
    URL url = new URL(urlString);
    HttpURLConnection huc = (HttpURLConnection)url.openConnection();
    huc.setRequestMethod("GET");
    huc.connect();
    return huc.getResponseCode();
}

Ora puoi scrivere il tuo codice Selenium WebDriver come di seguito:

private static int statusCode;
public static void main(String... args) throws IOException{
    WebDriver driver = new FirefoxDriver();
    driver.manage().window().maximize();
    driver.get("https://www.google.com/");
    driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

    List<WebElement> links = driver.findElements(By.tagName("a"));
    for(int i = 0; i < links.size(); i++){
        if(!(links.get(i).getAttribute("href") == null) && !(links.get(i).getAttribute("href").equals(""))){
            if(links.get(i).getAttribute("href").contains("http")){
                statusCode= getResponseCode(links.get(i).getAttribute("href").trim());
                if(statusCode == 403){
                    System.out.println("HTTP 403 Forbidden # " + i + " " + links.get(i).getAttribute("href"));
                }
            }
        }   
    }   
}

Ottieni il codice di risposta in qualsiasi lingua (usando JavaScript):

Se i test del selenio vengono eseguiti in un browser moderno, un modo semplice per ottenere il codice di risposta è inviare un XMLHttpRequest sincrono* e controllarestatus della risposta:

var xhr = new XMLHttpRequest();
xhr.open('GET', 'http://exampleurl.ex', false);
xhr.send(null);

assert(200, xhr.status);

È possibile utilizzare questa tecnica con qualsiasi linguaggio di programmazione richiedendo che Selenium esegua lo script. Ad esempio, in Java è possibile utilizzare JavascriptExecutor.executeScript() per inviare il XMLHttpRequest:

final String GET_RESPONSE_CODE_SCRIPT =
    "var xhr = new XMLHttpRequest();" +
    "xhr.open('GET', arguments[0], false);" +
    "xhr.send(null);" +
    "return xhr.status";
JavascriptExecutor javascriptExecutor = (JavascriptExecutor) webDriver;
Assert.assertEquals(200,
    javascriptExecutor.executeScript(GET_RESPONSE_CODE_SCRIPT, "http://exampleurl.ex"));

* È possibile inviare un XMLHttpRequest invece, ma è necessario attendere il completamento prima di continuare il test.

Ottenere il codice di risposta in Java:

È possibile ottenere il codice di risposta in Java utilizzando URL.openConnection() e HttpURLConnection.getResponseCode():

URL url = new URL("http://exampleurl.ex");
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setRequestMethod("GET");

// You may need to copy over the cookies that Selenium has in order
// to imitate the Selenium user (for example if you are testing a
// website that requires a sign-in).
Set<Cookie> cookies = webDriver.manage().getCookies();
String cookieString = "";

for (Cookie cookie : cookies) {
    cookieString += cookie.getName() + "=" + cookie.getValue() + ";";
}

httpURLConnection.addRequestProperty("Cookie", cookieString);
Assert.assertEquals(200, httpURLConnection.getResponseCode());

Questo metodo potrebbe probabilmente essere generalizzato anche ad altre lingue, ma dovrebbe essere modificato per adattarsi all'API della lingua (o della libreria).

Potresti provare Mobilenium ( https://github.com/rafpyprog/Mobilenium), un pacchetto python che lega BrowserMob Proxy e Selenium.

Un esempio di utilizzo:

>>> from mobilenium import mobidriver
>>>
>>> browsermob_path = 'path/to/browsermob-proxy'
>>> mob = mobidriver.Firefox(browsermob_binary=browsermob_path)
>>> mob.get('http://python-requests.org')
301
>>> mob.response['redirectURL']
'http://docs.python-requests.org'
>>> mob.headers['Content-Type']
'application/json; charset=utf8'
>>> mob.title
'Requests: HTTP for Humans \u2014 Requests 2.13.0 documentation'
>>> mob.find_elements_by_tag_name('strong')[1].text
'Behold, the power of Requests'

Stavo anche avendo lo stesso problema e bloccato per alcuni giorni, ma dopo alcune ricerche ho capito che possiamo effettivamente usare "remote remote-debugging-port" di Chrome per intercettare le richieste in combinazione con selenium web driver. Utilizzare il seguente pseudocodice come riferimento: -

Crea istanza del driver Chrome con debug remoto

int port = findFreePort();

chromeOptions.addArguments("--remote-debugging-port=" + freePort);

ChromeDriver driver = new ChromeDriver(chromeOptions);`

Effettuare una chiamata get a http://127.0.0.1:freePort

String response = makeGetCall( "http://127.0.0.1" + freePort  + "/json" );

Estrai l'URL WebSocket di Chrome per ascoltare, tu può vedere la risposta e capire come estrarre

String webSocketUrl = response.substring(response.indexOf("ws://127.0.0.1"), response.length() - 4);

Collegare a questa presa, u può utilizzare asyncHttp

socket = maketSocketConnection();

Abilita acquisizione di rete

socket.send( { "id" : 1, "method" : "Network.enable" } );

Ora chrome invierà tutti gli eventi relativi alla rete e li cattura come segue

socket.onMessageReceived( String message ){

    Json responseJson = toJson(message);
    if( responseJson.method == "Network.responseReceived" ){
       //extract status code
    }
}

driver.get("http://stackoverflow.com");

Puoi fare tutto ciò che è menzionato nel sito dev tools. cfr. https://chromedevtools.github.io/devtools-protocol / Nota: - utilizzare chromedriver 2.39 o superiore.

Spero che aiuti qualcuno.

Riferimento: Utilizzo del protocollo di debug remoto di Google Chrome

Per quelle persone che usano Python, potresti considerare Selenium Wire, una libreria che ho recentemente sviluppato per ispezionare le richieste su un progetto di lavoro.

La libreria è progettata per essere leggera, semplice da usare con dipendenze e configurazione esterne minime. Fornisce ai test un'API Pythonic per accedere alle intestazioni, al codice di stato, al contenuto del corpo e alla possibilità di modificare le intestazioni e riscrivere gli URL.

from seleniumwire import webdriver  # Import from seleniumwire

# Create a new instance of the Firefox driver
driver = webdriver.Firefox()

# Go to the Google home page
driver.get('https://www.google.com')

# Access requests via the `requests` attribute
for request in driver.requests:
    if request.response:
        print(
            request.path,
            request.response.status_code,
            request.response.headers['Content-Type']
        )

Stampa:

https://www.google.com/ 200 text/html; charset=UTF-8
https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_120x44dp.png 200 image/png
https://consent.google.com/status?continue=https://www.google.com&pc=s&timestamp=1531511954&gl=GB 204 text/html; charset=utf-8
https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png 200 image/png
https://ssl.gstatic.com/gb/images/i2_2ec824b0.png 200 image/png
https://www.google.com/gen_204?s=webaft&t=aft&atyp=csi&ei=kgRJW7DBONKTlwTK77wQ&rt=wsrt.366,aft.58,prt.58 204 text/html; charset=UTF-8
...