xSkrape Query Language (XS.QL) Reference Guide

What is XS.QL?

xSkrape has always supported a proprietary syntax for identifying data values of interest. XS.QL is in some ways just giving a name to this syntax for the many places that it can be applied.

The language offers the following benefits:

  • Extensible - the language has been easy to advance to date, and we expect that to continue. New use cases lead to new features.
  • Non-breaking - changing the language does not require you to change your functions or data connectors.
  • Consistent - the language is effectively the same whether it's being used in xSkrape for Excel, xSkrape for SQL, xSkrape REST API, or any new flavor of xSkape.

These are the settings where XS.QL can be used:

Type Applies How Used
URL's Get Single
Get Single From Table
Get Table
  • Can be an actual URL (e.g. http://www.msn.com)
  • Can be a URL expression. This is expressed in JSON.
  • Documented here

What can be done with URL Expressions:

  • Merge results from multiple requests   More...
  • Set custom request headers   More...
  • Shape hierarchical data into tabular data   More...
Queries - Single Values Get Single
  • Expressions that identify single data values (e.g. numberfollowsnear, xpath, etc.)
  • Can combine multiple with && separators
  • Documented here

What can be done with queries:

  • Narrow scope of document to examine   More...
  • Target data with a characteristic   More...
  • Target data with a specific "address"   More...
Queries - Tables Get Single From Table
Get Table
  • Expressions that identify table of choice (e.g. tablenumber, columnname, etc.)
  • Can additionally transform tabular results (e.g. rowfilter, skiprowcount, etc.)
  • Can combine multiple with && separators
  • Documented here

What can be done with queries:

  • Target a table with a characteristic   More...
  • Target a table with a specific "address"   More...
  • Optionally transform the resulting table   More...
Column Name Get Single From Table
  • Can be a column name - implies FIRST() function
  • Can be one of the following functions: FIRST, LAST, MAX, MIN, SUM, AVG (column name as a parameter)
  • Can be a non-XS.QL expression whose syntax is documented here.

What can be done with column names:

  • Select a single value from one or more rows in an already-identified table using an aggregation function   More...
Value Filter Get Single From Table
Get Table
  • Strictly speaking this is not XS.QL but supports logical operators, comparisons, etc.
  • Documented here

What can be done with value filters:

  • Narrow the number of rows returned based on filter criteria
Value Sort Get Single From Table
Get Table
  • Strictly speaking this is not XS.QL but supports multi-column sorting
  • Documented here

What can be done with value sorting:

  • Sort the tabular data on one or more columns (ascending or descending)