NAV
Oraclize log

Home

Welcome to the Documentation of Oraclize!

The documentation is divided in the following sections:

Background

Oraclize is the leading oracle service for smart contracts and blockchain applications, serving thousands of requests for day every day on Ethereum, Bitcoin and Rootstock.

In the blockchain space, an oracle is a party which provides data. The need for such figure arise from the fact that blockchain applications, such as Bitcoin scripts and smart contracts cannot access and fetch directly the data they require: price feeds for assets and financial applications; weather-related informations for peer-to-peer insurance; random number generation for gambling.

But to rely on a new trusted intermediary, the oracle in this case, it would be betraying the security and reduced-trust model of blockchain applications: which is what makes them interesting and useful in first place.

One solution is to accept data inputs from more than one untrusted or partially trusted party and then execute the data-dependent action only after a number of them have provided the same answer or an answer within some constrains. This type of system can be considered a decentralized oracle system. Unfortunately, this approach has severe limitations:

The solution developed by Oraclize is instead to demonstrate that the data fetched from the original data-source is genuine and untampered. This is accomplished by accompanying the returned data together with a document called authenticity proof. The authenticity proofs can build upon different technologies such as auditable virtual machines and Trusted Execution Environments.

A more detailed overview of Oraclize authenticity proofs is provided in the Authenticity Proofs section of this documentation.

This solutions elegantly solves the Oracle Problem:

Oraclize engine can be easily integrated with both private and public instances of different blockchain protocols.

While building the service, the Oraclize team has realized that the concept of authenticity proofs has much broader applicability that initially envisioned. For example, the Oraclize Random Data-source can be used even by traditional gambling applications to ensure users of continuous fairness of operation

General Concepts

Oraclize is integrated with a number of blockchain protocols and its service is useful and accessible also for non-blockchain applications. In the following section, general concepts which apply to all integrations will be explained.

Oraclize Engine

The Oraclize Engine powers the service for both blockchain-based and non-blockchain-based application. Internally replicates an “If This Then That” logical model. This means that it will execute a given set of instructions if some other given conditions are met. For example, it could repeatedly verify a condition and only return data or perform an action when the condition has been met. This flexibility enables the engine to be leveraged in many different ways and contexts, even outside of the blockchain context.

A valid request for data to Oraclize, done via the native blockchain integration or via the HTTP API, should specify the following arguments:

Data Source Types

A data source is a trusted provider of data. It can be a website or web API such as Reuters, Weather.com, BBC.com, or a secure application running on an hardware-enforced Trusted Execution Environment (TEE) or an auditable, locked-down virtual machine instance running in a cloud provider. Oraclize currently offers the following types of native data sources:

Additionaly, there also some meta data source such as:

Query

A query is an array of parameters which needs to evaluated in order to complete a specific data source type request: query: [ parameter_1, parameters_2, ...];

The first parameter is the main argument and it is usually mandatory. For example, in the case of the URL Data Source Type, the first argument is the expected URL where the resource resides. If only the first argument is present, then the URL Data Source assumes that an HTTP GET was requested. The second parameters, which it is optional, should contain the data payload of the HTTP POST request.

The intermediate result of a query may need to be parsed: for example, to extract a precise field in JSON API response. Therefore, the query can also specify parsing helpers to be applied.

Parsing Helpers

Oraclize offers XML, JSON, XHTML and a binary parser helpers. Examples:

Authenticity Proofs

Oraclize is designed to act as an untrusted intermediary. Optionally, a request to Oraclize can specify an authenticity proof. Not all proofs are compatible with all data source types. More details on the authenticity proofs can be found in the “Security Deep Dive” section.

Data Privacy

Certain contexts, such as smart contracts on public blockchains, might require a level of privacy to protect data from public scrutiny. Developers can make encrypted Oraclize queries by encrypting an entire query or some of its arguments with the Oraclize public key. More information can be found in the Encrypted Queries section.

Data Sources

Listed here are the data-sources you can choose from when using our oracle service.
Please note that datasource selection is not case-sensitive.

None TLSNotary Android Ledger
URL N/A
Random N/A N/A N/A
WolframAlpha N/A N/A N/A
IPFS 1 N/A N/A N/A
computation 1 N/A N/A

URL

The URL data source type enables access to any API or web page on the Internet. It supports both HTTP GET and HTTP POST request. If only one parameters is specified in the query, the service will default to perform an HTTP GET request. If a second parameter is specified, then the service will perform an HTTP POST request, posting the second parameter as data. Note that if the second parameter is valid JSON, then it will be posted as such. The URL data source type supports TLSNotary Proofs and Android Proofs. More advanced HTTP capabilities, such as Basic Authentication or OAuth, can be build by leveraging the computation data source type.

WolframAlpha

The WolframAlpha data source type enables direct access to the WolframAlpha Knowledge Engine API. This datasource expects as sole parameter the string which should be passed to WolframAlpha. It will returns the result as a string.

IPFS

The IPFS data source type can be used to retrieve the content of a file on the IPFS network.

This datasource expects as sole parameter the IPFS multihash in the query. For example, the file QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o would return `“hello world\n”.

If Oraclize fails to fetch the IPFS content within 20 seconds, the request will fail.

computation

# Content of the Dockerfile

FROM ubuntu:14.04
MAINTAINER Oraclize "[email protected]"
RUN apt-get update && apt-get install -y python-numpy
CMD python -c "import numpy; print int(numpy.linalg.det(numpy.array([[1,2,3],[7,2,3],[7,6,3]])))"
# How to create the zip archive and upload it on IPFS
[email protected]:~$ zip -r archive.zip
[email protected]:~$ ipfs add archive.zip
added QmRxtL9K2de7v7QBYCCrwcjZHjYmuKggZ8xaqZ6UUWvd1s archive.zip

The computation datasource enables the auditable execution of an application or a script by leveraging a sandboxed Amazon Web Service virtual machine.

The application has to print, on standard output, the result of the computation as the last line before it quits. The result can be up to 2500 characters long. The execution context has to be described by a Dockerfile, where building and running it should start the main application straight away. Currently Oraclize only provides one type of auditable instance: a t2.micro instance. The Dockerfile initialization and application execution should terminate as soon as possible, as the execution time is capped at 5 minutes.

The developer can send to Oraclize the application binary or the script, its dependency and the Dockerfile by creating an archive and uploading it to IPFS. The query expects as first argument the IPFS multihash of that archive, while the following arguments will be passed to the execution environment as environmental variables, making them accessible by the application.

random

The design described there prevents Oraclize from tampering with the random results coming from the Trusted Execution Environment (TEE) and protects the user from a number of attack vectors.

The authenticity proof, attached with the result, can be easily verified not just off-chain but even by any Solidity contract receiving them. The example presented here, showing how to integrate the verification process, discards any random result whose authenticity proofs don’t pass the verification process.

The random datasource is leveraging the Ledger proof to prove that the origin of the generated randomness is really a secure Ledger device.

The rationale behind this method of securely feeding off-chain randomness into the blockchain is explained in the “A Scalable Architecture for On-Demand, Untrusted Delivery of Entropy” white paper.

decrypt

Even though the decrypt datasource can be used as any other, it was specifically designed to be used within the nested datasource to enable partial query encryption.

The result is the decrypted query string. Please note that all the logic, limitations and tools provided by the Encryption feature apply here as well.

nested

The nested datasource is a meta datasource, it does not provide access to additional services. It was designed to provide some simple aggregation logic, enabling a single query to leverage sub-queries based on any available datasource and produce a single string as a result.

The query format is built in a way you that you can specify a sub-datasource and a sub-query as follows: [datasource_name] query_content.

Please mind the square brackets delimiting the datasource name and the whitespace prefixing the actual sub-query. The sub-query content can optionally be delimited by either single or double quotes.

You can optionally specify more than one sub-query, by using as delimitators the ${ special opener and the } special closer.

Example: [WolframAlpha] temperature in ${[IPFS] QmP2ZkdsJG7LTw7jBbizTTgY1ZBeen64PqMgCAWz2koJBL}

Integrations

Public Chains

Oraclize is integrated natively with the most widely used public blockchain protocols like the Ethereum, Bitcoin, Rootstock mainnets and testnets.

Alternative Public Chains

Oraclize is constantly studying new public blockchain protocols and considering further blockchain integrations. Since the Oraclize Engine is completely blockchain agnostic and can interact via HTTP API, developers are encouraged to work on open-source independent Oraclize integration on the model of the Ethereum-Bridge.

Private Ethereum-based Chains

Private, Ethereum-based chains can currently integrate with Oraclize by using the Ethereum-Bridge. Forks of Ethereum such as Monax included. More information on how to use the Ethereum-Bridge will become available in the development tools section or refer to this StackExchange answer describing how to use it alongside testrpc.

Alternative Private Chains

The Oraclize team is currently investigating to create software for a native integration for alternative private distributed ledger protocols such as Corda, Chain and Hyperledger. Additional information are available on contacts.

Non-blockchain Applications

The oraclize-lib enables the use of the Oraclize engine capabilities also in a non-blockchain context, by providing an abstraction layer which resolves to Oraclize HTTP API. The oraclize-lib is currently in a experimental-stage.

♦   Ethereum

The following section is dedicated to the Ethereum and Oraclize integration. To better profit from this section of the documentation, previous knowledge of Solidity and Ethereum is required.

The interaction between Oraclize and an Ethereum smart contract is asynchronous. Any request for data is composed of two steps:

As said in previous sections, one of the fundamental characteristics of Oraclize is the capability of returning data to a smart contract together with one or more proofs of authenticity of the data. The generation of an authenticity proof is optional and it is a contract-wide setting which must be configured by the smart contract developer before the request for data is initiated. Oraclize always recommends the use of authenticity proofs for production deployments.

Quick Start

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract ExampleContract is usingOraclize {

    string public EURGBP;
    event updatedPrice(string price);
    event newOraclizeQuery(string description);

    function ExampleContract() payable {
        updatePrice();
    }

    function __callback(bytes32 myid, string result) {
        if (msg.sender != oraclize_cbAddress()) throw;
        EURGBP = result;
        updatedPrice(result);
    }

    function updatePrice() payable {
        if (oraclize_getPrice("URL") > this.balance) {
            newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee");
        } else {
            newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
            oraclize_query("URL", "json(http://api.fixer.io/latest?symbols=USD,GBP).rates.GBP");
        }
    }
}

The most simple way to introduce the Ethereum - Oraclize integration, it is by showing a working example, such as the smart contract on the right. This contract uses Oraclize to fetch the last EUR/GBP from fixer.io APIs. The update process is initiated every time the function updatePrice() is called. The example shows two important components of using Oraclize:

The code in the example is working out of the box if Remix is used to compile and deploy it on any of the Ethereum networks: main-net and the Ropsten, Kovan and Rinkeby testnets. If, instead, another tool is used, it will be necessary to replace the import statement with a local import of the oraclizeAPI.sol file since direct import from Github may not be supported.

To ease development, Oraclize doesn’t charge a contract for its first request of data done using the default gas parameters. Successive requests will require the contract to pay the Oraclize fee and the ether necessary to pay for the callback transaction. Both are automatically taken from the contract balance. If the contract doesn’t have enough funds in his balance, the request will fail and Oraclize won’t return any data.

Simple Query

// This code example will ask Oraclize to send as soon as possible
// a transaction with the primary result (as a string) of the given
// formula ("random number between 0 and 100") fetched from the
// data-source "WolframAlpha".
oraclize_query("WolframAlpha", "random number between 0 and 100");

oraclize_query("URL", "https://api.kraken.com/0/public/Ticker?pair=ETHXBT")

oraclize_query("URL",
  "json(https://www.therocktrading.com/api/ticker/BTCEUR).result.0.last")

oraclize_query("IPFS", "QmdEJwJG1T9rzHvBD8i69HHuJaRgXRKEQCP7Bh1BVttZbU")

// The URL datasource also supports a supplement argument, useful for creating HTTP POST requests.
// If that argument is a valid JSON string, it will be automatically sent as JSON.
oraclize_query("URL", "json(https://shapeshift.io/sendamount).success.deposit",
  '{"pair":"eth_btc","amount":"1","withdrawal":"1AAcCo21EUc1jbocjssSQDzLna9Vem2UN5"}')

A request for data is called query. The oraclize_query is a function, inhered from the parent usingOraclize contract, which expects at least two arguments:

The number and type of supported arguments depends from the data-source in use. Beside, few more code example will be shown and commented. The datasource, as well as the authenticity proof chosen, determine the fee which the contract has to pay to Oraclize.

Schedule a Query in the Future

// Relative time: get the result from the given URL 60 seconds from now
oraclize_query(60, "URL",
  "json(https://api.kraken.com/0/public/Ticker?pair=ETHXBT).result.XETHXXBT.c.0")

// Absolute time: get the result from the given datasource at the specified UTC timestamp in the future
oraclize_query(scheduled_arrivaltime+3*3600,
  "WolframAlpha", strConcat("flight ", flight_number, " landed"));

The execution of a query can be scheduled in a future date. The function oraclize_query accepts as a parameter the delay in seconds from the current time or the timestamp in the future as first argument. Please note that in order for the future timestamp to be accepted by Oraclize it must be within 60 days of the current UTC time in the case of the absolute timestamp choice, or in the case of a relative time elapse, the elapsed seconds must equate to no more than 60 days.

Recursive Queries

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract ExampleContract is usingOraclize {

    string public EURGBP;
    event updatedPrice(string price);
    event newOraclizeQuery(string description);

    function ExampleContract() payable {
        updatePrice();
    }

    function __callback(bytes32 myid, string result) {
        if (msg.sender != oraclize_cbAddress()) throw;
        EURGBP = result;
        updatedPrice(result);
        updatePrice();
    }

    function updatePrice() payable {
        if (oraclize_getPrice("URL") > this.balance) {
            newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee");
        } else {
            newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
            oraclize_query(60, "URL", "json(http://api.fixer.io/latest?symbols=USD,GBP).rates.GBP");
        }
    }

Smart contract using Oraclize can be effectively autonomous by implementing a new call to Oraclize into their __callback method. This can be useful for implementing periodic updates of some on-chain reference data, as with price feeds, or to periodically check for some off-chain conditions.

This modified version of the previous example will update the EUR/GBP exchange rate every 60 seconds, until the contract has enough funds to pay for the Oraclize fee.

The Query ID

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract ExampleContract is usingOraclize {

    string public EURGBP;
    mapping(bytes32=>bool) validIds;
    event updatedPrice(string price);
    event newOraclizeQuery(string description);

    function ExampleContract() payable {
        updatePrice();
    }

    function __callback(bytes32 myid, string result) {
        if (!validIds[myid]) throw;
        if (msg.sender != oraclize_cbAddress()) throw;
        EURGBP = result;
        updatedPrice(result);
        delete validIds[myid];
        updatePrice();
    }

    function updatePrice() payable {
        if (oraclize_getPrice("URL") > this.balance) {
            newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee");
        } else {
            newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
            bytes32 queryId =
                oraclize_query(60, "URL", "json(http://api.fixer.io/latest?symbols=USD,GBP).rates.GBP");
            validIds[queryId] = true;
        }
    }
}

Every time the function oraclize_query is called, it returns a unique ID, hereby referred to as queryId, which depends from the number of previous requests and the address of smart contract. The queryId identifies a specific query done to Oraclize and it is returned to the contract as a parameter of the callback transaction.

Oraclize recommends smart contract developers to verify if the queryId sends by the callback transaction was generated by a valid call to the oracize_query function, as shown in the example accompanying this paragraph. This ensures that each query response is processed only once and helps avoid misuse of the smart contract logic. Moreover, it protects the smart contract during blockchain reorganizations, as explained in the dedicated paragraph of this section.

The queryId can be used as well to implement different behaviors into the __callback function, in particular when there is more than one pending call from Oraclize.

Custom Gas Limit and Gas Price

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract ExampleContract is usingOraclize {

    string public EURGBP;
    mapping(bytes32=>bool) validIds;
    event updatedPrice(string price);
    event newOraclizeQuery(string description);

    // This example requires funds to be send along with the contract deployment
    // transaction
    function ExampleContract() payable {
        oraclize_setCustomGasPrice(4000000000 wei);
        updatePrice();
    }

    function __callback(bytes32 myid, string result) {
        if (!validIds[myid]) throw;
        if (msg.sender != oraclize_cbAddress()) throw;
        EURGBP = result;
        updatedPrice(result);
        delete validIds[myid];
        updatePrice();
    }

    function updatePrice() payable {
        if (oraclize_getPrice("URL") > this.balance) {
            newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee");
        } else {
            newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
            bytes32 queryId =
                oraclize_query(60, "URL", "json(http://api.fixer.io/latest?symbols=USD,GBP).rates.GBP", 500000);
            validIds[queryId] = true;
        }
    }
}

The transaction originating from Oraclize to the __callback function pays a fee to the miner which include the transaction in a block, just like any other transaction. The miner fee is paid in Ether and it is calculated by taking the amount of gas which covers the execution costs of the transaction multiplied by the selected gas/ether price. Oraclize will set those parameters accordingly to the parameters specified in the smart contract, for contract-wide settings, and in the oraclize_query function, for query-specific settings. The miner fee for the callback transaction is taken from the contract balance when the query transaction is executed.

If no settings are specified, Oraclize will use the default values of 200,000 gas and 20 GWei. This last value is on the higher-end of the pricing spectrum right now, but it helps having faster confirmation times during network-wide congestions.

A different value for the Oraclize callback gas can be passed as the argument _gasLimit to the oraclize_query function as shown in the following examples.

// If the callback transaction requires little gas, the value can be lowered:
oraclize_query("URL", "json(http://api.fixer.io/latest?symbols=USD,GBP).rates.GBP", 100000);

// Callback methods may be expensive. The example requires the JSON parsing
// a string in the smart contract. If that's the case, the gas should be increased:
oraclize_query("URL", "https://api.fixer.io/latest?symbols=USD,GBP", 500000);

The gas price of the callback transaction can be set by calling the oraclize_setCustomGasPrice function, either in the constructor, which is executed once at deployment of the smart contract, or in a separate function. The following is the ExampleContract modified to specify a custom gas price of 4 Gwei and a custom gas limit for the callback transaction.

Smart contract developers should estimate correctly and minimize the cost of their __callback method, as any unspent gas will be returned to Oraclize and no refund is available.

Authenticity Proofs

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract ExampleContract is usingOraclize {

    string public EURGBP;
    mapping(bytes32=>bool) validIds;
    event updatedPrice(string price);
    event updatePrice(string description);


    // This example requires funds to be send along with the contract deployment
    // transaction
    function ExampleContract() payable {
        oraclize_setCustomGasPrice(4000000000 wei);
        oraclize_setProof(proofType_TLSNotary | proofStorage_IPFS);
        updatePrice();
    }

    function __callback(bytes32 myid, string result, bytes proof) {
        if (!validIds[myid]) throw;
        if (msg.sender != oraclize_cbAddress()) throw;
        EURGBP = result;
        updatedPrice(result);
        delete validIds[myid];
        updatePrice();
    }

    function updatePrice() payable {
        if (oraclize_getPrice("URL") > this.balance) {
            newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee");
        } else {
            newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
            bytes32 queryId =
                oraclize_query(60, "URL", "json(https://api.fixer.io/latest?symbols=USD,GBP).rates.GBP", 500000);
            validIds[queryId] = true;
        }
    }
}

Authenticity proofs are at the core of Oraclize’s oracle model. Smart contracts can request authenticity proofs together with their data by calling the oraclize_setProof function available in the usingOraclize. The authenticity proof can be either deliver directly to the smart contract or it can be saved, upload and stored on IPFS.

When a smart contract requests for an authenticity proof, it must define a different callback function with the following arguments: function __callback(bytes32 queryId, string result, bytes proof)

The oraclize_setProof function expects the following format: oraclize_setProof(proofType_ | proofStorage_ )

Both proofType and proofStorage are byte constants defined in usingOraclize:

Available parameters for proofTypes are:

While for proofStorage:

For example, oraclize_setProof(proofType_TLSNotary) will return the full TLSNotary Proof bytes as the proof argument in the callback transaction. If instead oraclize_setProof(proofType_TLSNotary | proofStorage_IPFS) is used, then Oraclize will return only the base58-decoded IPFS multihash as the proof argument. To obtain the IPFS multihash, the bytes must be encoded to base58. The method oraclize_setProof can be executed in the constructor, becoming a contract-wide lasting setting, or it can be set directly before a specific query is to be made. Authenticity proofs can be disabled by calling oraclize_setProof(proofType_NONE). Smart contract developer should be aware that the helper method oraclize_setProof is an internal function of usingOraclize, and therefore it must be included specifically in their smart contract at compile time, before deployment. The following builds on our previous example:

Verifiability

Supported proofs can be verified. The following tools can be used: Verification Tools

Advanced Topics

Encrypted Queries

Certain contexts, such as smart contracts on public blockchains, might require a level of privacy to protect data from public scrutiny. Developers can make encrypted Oraclize queries by encrypting a part (or all) of a query with the Oraclize public key. The encrypted queries feature may be of interested to developers who want to deploy their blockchain applications of public networks. For example, if an application leverages data from an authenticated API, it would be dangerous to disclose the API key to anyway who is monitoring the public chain.

Oraclize therefore offers the possibility of encrypting the parameters contained in a query to Oraclize’s public key: `044992e9473b7d90ca54d2886c7addd14a61109af202f1c95e218b0c99eb060c7134c4ae46345d0383ac996185762f04997d6fd6c393c86e4325c469741e64eca9. Only Oraclize will then be able to decrypt the request using its paired private key.

To encrypt the query, Oraclize provides a CLI tool, which can be found here. Alternatively,
The CLI command to encrypt an arbitrary string of text is then:

python encrypted_queries_tools.py -e -p 044992e9473b7d90ca54d2886c7addd14a61109af202f1c95e218b0c99eb060c7134c4ae46345d0383ac996185762f04997d6fd6c393c86e4325c469741e64eca9 "YOUR QUERY"

This will encrypt the query with the default Oraclize public key. The encrypted string can then be used as an argument for an Oraclize query.

// In this example, the entire first argument of an oraclize_query has been encrypted.
// The actual string encrypted is:  json(https://poloniex.com/public?command=returnTicker).BTC_ETH.last
oraclize_query("URL","AzK149Vj4z65WphbBPiuWQ2PStTINeVp5sS9PSwqZi8NsjQy6jJLH765qQu3U/
  bZPNeEB/bYZJYBivwmmREXTGjmKJk/62ikcO6mIMQfB5jBVVUOqzzZ/A8ecWR2nOLv0CKkkkFzBYp2sW1H
  31GI+SQzWV9q64WdqZsAa4gXqHb6jmLkVFjOGI0JvrA/Zh6T5lyeLPSmaslI");

The encryption method is also available for POST requests: you can encrypt both the URL and the POST data field as in the following example:

// This is the query that we want to encrypt
oraclize_query("URL","json(https://api.postcodes.io/postcodes).status",
  '{"postcodes" : ["OX49 5NU", "M32 0JG", "NE30 1DP"]}')

Encrypt the datasource (URL in this case):
python encrypted_queries_tools.py -e -p 044992e94... "URL"

Returns:
BEIGVzv6fJcFiYQNZF8ArHnvNMAsAWBz8Zwl0YCsy4K/RJTN8ERHfBWtSfYHt+uegdD1wtXTkP30sTW+3xR3w/un1i3caSO0Rfa+wmIMmNHt4aOS

Encrypt the argument(in this case we are using the JSON parsing helper to retrieve the “status” ):
python encrypted_queries_tools.py -e -p 044992e94... "json(https://api.postcodes.io/postcodes).status"

Returns:
BNKdFtmfmazLLR/bfey4mP8v/R5zCIUK7obcUrF2d6CWUMvKKUorQqYZNu1YfRZsGlp/F96CAQhSGomJC7oJa3PktwoW5J1Oti/y2v4+b5+vN8yLIj1trS7p1l341Jf66AjaxnoFPplwLqE=

Encrypt the JSON (third argument, the data to POST):
python encrypted_queries_tools.py -e -p 044992e94... '{"postcodes" : ["OX49 5NU", "M32 0JG", "NE30 1DP"]}'

Returns:
BF5u1td9ugoacDabyfVzoTxPBxGNtmXuGV7AFcO1GLmXkXIKlBcAcelvaTKIbmaA6lXwZCJCSeWDHJOirHiEl1LtR8lCt+1ISttWuvpJ6sPx3Y/QxTajYzxZfQb6nCGkv+8cczX0PrqKKwOn/Elf9kpQQCXeMglunT09H2B4HfRs7uuI

// Finally we add all the encrypted text
// to the oraclize_query (in the right order)
oraclize_query("BEIGVzv6fJcFiYQNZF8ArHnvNMAsAWBz8Zwl0YCsy4K/RJTN8ERHfBWtSfYHt+
  uegdD1wtXTkP30sTW+3xR3w/un1i3caSO0Rfa+wmIMmNHt4aOS","BNKdFtmfmazLLR/bfey4mP8
  v/R5zCIUK7obcUrF2d6CWUMvKKUorQqYZNu1YfRZsGlp/F96CAQhSGomJC7oJa3PktwoW5J1Oti/
  y2v4+b5+vN8yLIj1trS7p1l341Jf66AjaxnoFPplwLqE=", "BF5u1td9ugoacDabyfVzoTxPBxG
  NtmXuGV7AFcO1GLmXkXIKlBcAcelvaTKIbmaA6lXwZCJCSeWDHJOirHiEl1LtR8lCt+1ISttWuvp
  J6sPx3Y/QxTajYzxZfQb6nCGkv+8cczX0PrqKKwOn/Elf9kpQQCXeMglunT09H2B4HfRs7uuI");

You can also do this with a request to another datasource like WolframAlpha, the Bitcoin blockchain, or IPFS. Our encryption system also permits users to encrypt any of the supported datasource options.

To protect the plaintext queries, an Elliptic Curve Integrated Encryption Scheme was chosen. The steps performed for the encryption are the following ones:

Computation Data Source

Passing Arguments to the Package

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract Random is usingOraclize {

    event newOraclizeQuery(string description);
    event newRandomNumber(uint number);


    function Random() {
        oraclize_setProof(proofType_TLSNotary | proofStorage_IPFS);
        update();
    }

    function __callback(bytes32 myid, string result, bytes proof) {
        if (msg.sender != oraclize_cbAddress()) throw;
        newRandomNumber(parseInt(result));
    }

    function update() payable {
        if (oraclize.getPrice("computation") > this.balance) {
            newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee");
        } else {
            newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
            oraclize_query("computation",["QmR6qXv65K2EjKj5H9tFfzU6GDd9STYYvJMXFRRAskm3uF", "2"]);
        }
    }

}

Arguments can be passed to the package by adding parameters to the query array. They will be accesssible from within the Docker instances as environmental parameters.

FROM ubuntu:14.04
MAINTAINER Oraclize "[email protected]"

RUN apt-get update && apt-get -y install python-minimal

CMD /usr/bin/python -c "import os; print os.urandom(int(os.environ['ARG0'], 10))"

Passing Encrypted Arguments

pragma solidity ^0.4.11;
import "github.com/oraclize/ethereum-api/oraclizeAPI.sol";

contract ComputationTest is usingOraclize {


    event newOraclizeQuery(string description);
    event newResult(string result);

    function ComputationTest() {
        update(); // first check at contract creation
    }

    function __callback(bytes32 myid, string result) {
        if (msg.sender != oraclize_cbAddress()) throw;
        newResult(result);

    }

    function update() payable {
        newOraclizeQuery("Oraclize query was sent, standing by for the answer..");
        oraclize_query("nested", "[computation] ['QmaqMYPnmSHEgoWRMP3WSrUYsPWKjT85C81PgJa2SXBs8u', \
'Example of decrypted string', '${[decrypt] BOYnQstP700X10I+WWNUVVNZEmal+rZ0GD1CgcW5P5wUSFKr2QoIwHLvkHfQR5e4Bfakq0CIviJnjkfKFD+ZJzzxcaFUQITDZJxsRLtKuxvAuh6IccUJ+jDF/znTH+8x8EE1Tt9SY7RvqtVao2vxm4CxIWq1vk4=}', 'Hello there!']");
    }

}

Encrypted arguments can be passed using the nested and the decrypt meta data sources, as shown in the example at the right.

Random Data Source

In the contract usingOraclize, which smart contracts should use to interface with Oraclize, some specific functions related to the Oraclize Random Data Source have been added. In particular:

For advance usage of Random Data Source, it is recommended to read the following section.

Two Party Interactions

    function oraclize_newRandomDSQuery(uint _delay, uint _nbytes, uint _customGasLimit) internal returns (bytes32){
        if ((_nbytes == 0)||(_nbytes > 32)) throw;
        bytes memory nbytes = new bytes(1);
        nbytes[0] = byte(_nbytes);
        bytes memory unonce = new bytes(32);
        bytes memory sessionKeyHash = new bytes(32);
        bytes32 sessionKeyHash_bytes32 = oraclize_randomDS_getSessionPubKeyHash();
        assembly {
            mstore(unonce, 0x20)
            mstore(add(unonce, 0x20), xor(blockhash(sub(number, 1)), xor(xor(caller,coinbase), xor(callvalue,timestamp)))
            mstore(sessionKeyHash, 0x20)
            mstore(add(sessionKeyHash, 0x20), sessionKeyHash_bytes32)
        }
        bytes[3] memory args = [unonce, nbytes, sessionKeyHash];
        bytes32 queryId = oraclize_query(_delay, "random", args, _customGasLimit);
        oraclize_randomDS_setCommitment(queryId, sha3(bytes8(_delay), args[1], sha256(args[0]), args[2]));
        return queryId;
    }

The oraclize_newRandomDSQuery can be used for different kind of interactions, but the security can be incresed further by additing additional commitment data to the request. For example, for two party interactions, the oraclize_newRandomDSQuery can be modified as showon the side to include the sender address and the value send along as commitment data. This more strongly commitment the request for random bytes to current party, which are assumed to have a stake in the contract, making it impossible for miners to replay transactions on potential forks or reorg of the current chain.

Multi-Party Interactions

In the case of multi-party interactions, such as voting schemes or lotteries, the commitment data can should include all participants addresses, to ensure that the transaction cannot be replayed by a miner on a fork or a reorged chain where a participant didn’t put a stake.

More Examples

More complete, complex examples are available on the dedicated Github repository: https://github.com/oraclize/ethereum-examples

☸   Rootstock

When launched later this year, Rootstock will be the first production sidechain of the Bitcoin network. It aims to extends Bitcoin capabilities without compromising its decentralization and censorship resistance properties. On the Rootstock networks, participants will be able to interact with smart contracts using a token called Smart Bitcoin. The peg between Bitcoin and Smart Bitcoins is maintained by a federation of distinguished players, among which Oraclize will proudly figure, and backed by merged mining.

The RSK team has recently launched the Ginger Testnet, which has full integration with Oraclize services.

Smart contracts on Rootstock are executed by an upgraded, but backward compatible version, of the Ethereum Virtual Machine. The Oraclize Rootstock integration is in fact completely compatible with the Ethereum one, since Ethereum smart contract can be compiled and execute on Rootstock natively.

฿   Bitcoin

Oraclize is building a native integration with Bitcoin, leveraging the HTTP API exposed by the Oraclize Engine and the oraclize-lib. More information will be available soon.

Development Tools

In this section, developers will find some of tools which can assist them into integrating Oraclize in their product.

Test Query

The TestQuery page can be used to test any Oraclize query. This does not require any code to be written and can be useful to verify the correctness of a given query during the early development stage.

Network Monitor

The Network Monitor, along with the Proof Verification Tool, can be used to verify the integrity and correctness of the authenticity proofs Oraclize has provided.

It is very important to independently verify that those proofs are valid, as this is the only way it can verified if Oraclize has ever provided a wrong answer.

Encryption

In order to use the encryption-enabling features of Oraclize, developers can use the TestQuery page or the Python Encryption Tool. To avoid replay attacks, the encrypted query is linked to the first smart contract which uses it to perform a request to Oraclize. New deployment of the same smart contract code will required to newly encrypt the query.

Oraclize Ethereum IDE

Oraclize is hosting a patched version of the Remix IDE. The patch adds a plugin enabling testing of Ethereum Oraclize-based contracts directly from the browser: the contract can be deployed in memory and the request is resolved automatically via the Oraclize HTTP API.

Unit Testing

An important step in smart contract development, before moving to an official testnet or production use, is unit testing. The Ethereum community has developed multiple frameworks for unit testing, such as Truffle and Embark, which can test contracts on a private instance of Ethereum. A common combination is to run Truffle tests in an ethereumjs-testrpc environments, which is a simulated Ethereum blockchain instance running in memory. In order for Oraclize to interact with the smart contract deployed by Truffle or Embark in the test-rpc blockchain instance, a tool called Ethereum-Bridge need to be installed. This is a nodejs module which deploys in the instance the Oraclize smart contracts infrastructure which it is then monitored to see the requests. The tool translates the requests to HTTP API calls to the Oraclize Engine and then return the result by broadcasting the callback transaction. The Ethereum-Bridge and instructions on its deployment can be found here.

Eth-pm

The Ethereum Package Management is a project which aims to standardize, build and maintain a common registry of smart contracts-based packages, to help smart contract developers. Oraclize has published and maintains the Ethereum oraclizeAPI under the ‘oraclize’ package. To install it: `truffle install oraclize’.

Future

Oraclize-lib

Oraclize-lib is an experimental nodejs library that be used to build non-blockchain applications leveraging Oraclize. It can be considered a simple abstraction layer to the Oraclize HTTP API.

Stargate

The Oraclize team is working on a tool which will enable direct integration with any private deployment of Ethereum-based chain, including private testnet, without installing additional software. This tool has been named Stargate and it is under active development. The tool will consist of an ssh-bridge between the developer blockchain instance and Oraclize. Additional information will be available later this year. Parties interested in trying this experimental feature, can get in touch at '[email protected]

Security Deep Dive

As digital technologies have reduced the barrier to entries for information creation and distribution, it has become extremely important to be able to authenticate a piece of information as originating from a known, trusted source.

In the context of web technologies, authentication is provided by the HTTPS protocol, an extension of the HTTP protocol which create an encrypted and authenticated channel between the client and the web-server containing the data.

When dealing with data which could be used to determine different financial transactions, authentication becomes of fundamental importance. Unfortunately, the most used and available blockchain protocols have no direct way of interacting with HTTPS and therefore digesting authenticated data. It would seem then that there is a need for a trusted service which can provide this data or complete actions based on it; but that would in part defeat the point of having a decentralized protocol for exchanging value without trusted parties.

This is the reason why Oraclize has been designing systems for authenticating data and made it part of its core business. These systems are called authenticity proofs, which enable auditability of the oracle’s service record in delivering untampered data. The authenticity proofs leverage different attestation technologies: some are software-based and some rely on trusted hardware technologies.

Authenticity Proofs Types

TLSNotary Proof

The TLSNotary Proof leverages a feature of the TLS 1.0 and 1.1 protocols which enables the splitting of the TLS master key between three parties: the server, an auditee and an auditor. In this scheme, Oraclize is the auditee while a locked-down AWS instance of a specially-designed, open-source Amazon Machine Image acts as the auditor. The TLSNotary protocol is an open-source technology, developed and used by the PageSigner project.

Android Proof

The Android Proof is a result of some of Oraclize’s internal R&D work. It leverages software remote attestation technology developed by Google, called SafetyNet, to validate that a given Android application is running on a safe, non-rooted physical device, connected to Oraclize’s infrastructure. It also remotely validates the application code hash, enabling authentication of the application running on the device. The application code is open-source, thereby enabling auditability and verification of the code hash. The Android Proof goes further, by using the newly introduced Android Hardware Attestation to prove that the physical device is updated to the latest available Android version, further ensuring integrity by having any potential exploits within the system patched. Furthermore, it verifies that the device’s Root-of-Binding-Trust is valid. Both these technologies together effectively turn the physical Android device into a provably-secure environment in which an untampered HTTPS connection to a remote datasource can be initiated. For Oraclize or an external attacker with unauthorized gained access to the infrastructure to compromise the device and generate a false but valid proof, a zero-day exploit unbeknownst to Google must be discovered by said party, which either breaks the Android sandboxing model or is a kernel-level exploit, of the latest version of Android OS and its available security patches. You can access more information about the Android Proof by reading the white paper on it and experiment with it on the Ethereum and Bitcoin testnets. The enabling of the Android Proof on mainnet is pending based on an update by Google, effectively enabling Android Nougat Hardware Attestation.

Ledger Proof

Ledger is a French company, leader in the production of hardware-enforced cryptocurrency wallets. Their main products are the Ledger Nano S and the Ledger Blue. Both devices implement a common platform comprising a STMicroelectronics secure element, a controller and an operative system called BOLOS. The BOLOS SDK enables developer to build applications which can be installed on the secure hardware along with the cryptocurrency wallet, which are BOLOS application themselves. BOLOS exposes a set of kernel-level API which can complete useful operations such as cryptographic ones or attestation. Attestation is in fact one of the most interesting aspect of the platform: any application can, by calling the appropriate API, ask the kernel to measure its binary and produce a signed hash. The signing is performed by a special attesting key, which is controlled by the kernel and it is out of reach of the application developers. The attesting key has full chain of trust which has the root in a Ledger-controlled master key, residing on an HSM controlled by Ledger.

The Ledger Proof leverages both the code attesting and the device attesting features to attest to any third-party that the applications developed by Oraclize are running in a TEE of a true Ledger device. The Ledger Proof has the following format:

1 2 3 4
3 bytes 65 bytes var length
‘LP\x01’ (prefix) APPKEY1 PubKey APPKEY1 cert (CA:Ledger)
  1. A prefix to indicate proof type and versioning
  2. The attesting key app_key_1 of the device.
  3. Signature by the Ledger Root Key over the app_key_1
  4. The data and signatures of the application

The Ledger Proof is currently used by the Oraclize Random Data Source to provide untampered entropy to smart-contracts, blockchain and non-blockchain applications.

Storage and Delivery

The authenticity proofs may be relatively large files, of up to a few kilobytes. Delivering such proofs directly within the result of the data payload in an Ethereum transaction can get quite expensive, in terms of EVM execution costs, and may even be impossible for larger data.

Moreover, Oraclize strives to be blockchain agnostic, enabling the proof to be used even on Bitcoin and other blockchains. Therefore the proof is uploaded and saved to IPFS, a decentralized and distributed storage system. In providing a pointer to the content, IPFS uses a custom hashing algorithm called multihash. The resulting address is Base64 encoded and it’s a unique ID specific to the file which can be used to access it globally, and changes along with any edits to the file contained.

IPFS, by itself, doesn’t provide any long-term guarantees of persistency, however as part of Oraclize’s infrastructures it runs the IPFS persistence consortium. Anyone can join Oraclize’s consortium and help in keeping an independent copy of all the proofs by pointing a IPFS persistence consortium node to the following aggregation peer: QmSsSSfJAJwj3jsNfzbSrxtLAjhpYPjbUEsCQT8mWVgBiL

Advanced Data Sources

Random Data Source

The Oraclize Random Data Source leverages the Ledger Proof and a custom application to generate unbiased random numbers and provide them on demand to blockchain and non-blockchain based applications. The end applications can perform verification step to unsure that the randomness provided was truly generated in an secure hardware environment provided by a Ledger Nano S.

This section will shortly explain the rationale and the inner workings of the custom application.

The main goals of the design were:

  1. Oraclize can only ask once, for a specific request, a random number to the device. Subsequent requests should be denied or return the previously extracted number.
  2. The random number must be signed, to authenticate that has been generated by the device.
  3. In blockchain applications, miners shouldn’t be able to tamper with orders, validity and results of the bets
  4. The design should minimize use of state both on the device and on-chain.

The architecture achieves these goals by:

  1. Enforcing the uniqueness of each request id. The device keeps a merkelized, append-only state where each request seen is appended.
  2. All data coming from the device is signed by an application session key, whose public key is present in the Oraclize connector. Requests can commit to receive the result by a precise session key.
  3. Commitment data is send along with the request, to anchor it to a specific blockchain history. Commitment data can be derived by the current block information such as timestamp, coinbase and previous block hash, and additionally information related to the request (msg.sender, msg.value etc).
  4. The kernel has an event feature which it is used to increment an application internal timer. A request can and should specify a time, in seconds, which has to elapse before the request can be served by the device and the random number returned. The higher the specifier time, the stronger the security against miners, with or without Oraclize collaboration, tampering with the result.
  5. The random bytes are generated from the commitment data and the session private key using the ECDSA deterministic signing; this avoids costly recomputation of the tree to mark served query.

The validity of these operations is enforced by the Ledger Proof. Signatures and data related to the Random Data Source are in fact append to the Ledger Proof and returned with the result. The format of the appended data is the following one:

1 2 3 4 5 6 7 8
32 bytes 32 bytes 8 bytes 1 byte 32 bytes var length 65 bytes var length
CODEHASH keyhash timelock Nbytes user nonce SessionKey sig SessionPubKey attestation sig

  1. The SHA-256 of the application binary, measured by the BOLOS kernel
  2. The uniqueId hash
  3. The time which has to elapse before the request can be answered
  4. The number of random bytes which have to be returned
  5. The commitment data, chosen by the developer
  6. The signature of the application session key over the request data
  7. The application session public key
  8. The signature of app_key_1, the attesting key, over the codehash and the session public key

An in-depth explanation of the Random Data Source and an analysis of the attack scenarios can be consulted in the released paper.

Pricing

The use of Oraclize requires the payment of a small fee, which depends from the data source type used and by the authenticity proof requested. The pricing listed below is valid for both Ethereum and Rootstock.

First Free Request

To facilitate testing, the first call to Oraclize from any smart contract address, if used with standard setting, is free of charge. This includes the gas costs of the callback transaction.

Testnets Policy

To keep compatibility with the main-net deployed, smart contracts testing Oraclize on any of the testnets are expected to pay the same fee they would on the main-net. Since the payment expected is in testnet Ether, which holds no value, the calls to Oraclize are the facto free. Oraclize reserve the rights to discontinue the service for abuse or excessive use.

Call Fee

The oraclize_query automatically recovers the fee at execution time. The fee consist of two parts:

Datasource Base price Proof type
None TLSNotary Android Ledger
URL 0.01$ +0.0$ +0.04$ +0.04$ N/A
WolframAlpha 0.03$ +0.0$ N/A N/A N/A
IPFS 0.01$ +0.0$ N/A N/A N/A
random 0.05$ +0.0$ N/A N/A +0.0$
computation 0.50$ +0.0$ +0.04$ +0.04$ N/A

Offchain Payments

Oraclize will soon start testing offchain payments, in different currencies, for both the Oraclize fee and the gas callback transactions costs. Interested parties should get in touch at [email protected]