Function: WebGetSingle

Extract a single value (number, text, etc.) from a source reference, using query conditions that identify the value of interest.

Usage

  • HTTP GET
    https://www.xs.codexframework.com/api/WebGetSingle?cc=CLIENT_CODE
    &url=ENCODED_URL
    &query=ENCODED_QUERY
    &sdt=SERVER_DATE_TREATMENT

Parameters

  • CLIENT_CODE
    This is an authorization token that identifies your xSkrape.com account to which any credit charges will be applied. You can locate this code by logging into your xSkrape.com account, under My Account - Queries. (Remember: it's your duty to secure your client code against unauthorized use.) Required.
  • ENCODED_URL
    A reference to the source data to be queried. Can be a reference that is HTTP or HTTPS based. Can optionally be an expression that supports loading data across multiple requests (or performs other special parsing), as described here. The text needs to be URL encoded. Required.
  • ENCODED_QUERY
    A query specification to isolate the specific data element from the raw source data. Details about the query syntax are covered here. The text needs to be URL encoded. Required.
  • SERVER_DATE_TREATMENT
    Valid values include "fromdate" which implies there is no date conversion undertaken (or an embedded explicit time zone within a matched date is used), or names a specific time zone which is assumed to apply to any matched date. The time zone name if given must be one of the standard time zone names listed here. Optional.

Return Value(s)

  • Returns JSON in all cases (success and failure).
  • Format: { "Success":true|false, "Message":"message", "Source":"source", "NoData":true|false, "RobotsTxtWarning":true|false, "Truncated":true|false, "dt":"datatype", "v":"value" }
  • Success: Indicates overall success or failure status of the request.
  • Message: In the case of failure, provides an error message.
  • Source: In the case of failure, identifies the origin of the message.
  • NoData: Indicates whether the query returned any data or not.
  • RobotsTxtWarning: If true, indicates that the accessed web site has a robots.txt file that may imply disallowed access (check with the site administrator or terms of use).
  • Truncated: If true, indicates that the data result was truncated due to metering settings on your xskrape.com user profile.
  • DataType: A string that identifies the type of the data value returned. (Valid values are based on .NET data types, e.g. "int32", "decimal", etc.)
  • Value: The string representation of the data value returned from the query. Date handling is described in more detail here.

Remarks

This function is typically used to extract a single value from a non-tabular data source such as HTML, JSON or XML. For tabular sources like CSV files, etc. - one would typically use functions such as WebGetTable or WebGetSingleFromTable. Some pre-parsing of the return value can be done depending on the query expression used.

Examples

  • cURL

    1. Pull data from a HTML page - based on a regular expression

    In this example we use the "curl" command line tool to illustate calling the WebGetSingle API.

    curl --data-urlencode "cc=5e....2e" --data-urlencode "url=http://regsho.finra.org/FNSQshvol20160624.txt" --data-urlencode "query=regex=(?<=\|AAPL\|)\d+" -G "https://www.xs.codexframework.com/api/WebGetSingle"

    Results:

    {"v":"4979421", "dt":"String", "Success":true, "Message":null, "Source":null, "NoData":false, "RobotsTxtWarning":false, "Truncated":false}

    Of note, we've omitted most of our client token: you would substitute your own, as found when you log into your xSkrape.com account. The URL and query are encoded as part of our request to WebGetSingle, and the result shows a successful completion (Success = true). The value our query was trying to fetch is based on looking for a regular expression in the source document, namely numeric characters that follow "|AAPL|", which if you open the URL in a browser, you'll notice the document is pipe-delimited for thousands of stock symbols.

  • jQuery

    2. Pull data from a HTML page - based on a regular expression

    This is effectively the same example as above, except implemented using a JQuery get call.

    <input type="text" id="result" />
    <script type="text/javascript">
    var requestUrl = "https://www.xs.codexframework.com/api/WebGetSingle";

    var parm = {
        cc: "5e.....2e",
        url: "http://regsho.finra.org/FNSQshvol20160624.txt",
        query: "regex=(?<=\\|AAPL\\|)\\d+",
        sdt: "fromdate"
    };

    $.get(requestUrl, parm).done(function (data) {
        if (data.Success) {
            $("#result").val(data.v);
        } else {
            $("#result").val(data.Message);
        }
    });
    </script>

Related