Installation Guide

Linux dependencies

Install postgresql as database:

$ sudo apt-get install postgresql

Install python3 headers:

$ sudo apt-get install python3-pip
$ sudo apt-get install python3.6-dev

Install redis server:

$ sudo apt-get install redis-server

Install Rabbit-MQ server:

$ sudo apt-get install rabbitmq-server

Python dependencies

Clone the ether_sql library:

$ git clone https://github.com/analyseether/ether_sql.git
$ cd ether_sql

Create and activate a virtual environment:

$ virtualenv envname
$ source envname\bin\activate

Install python libraries:

$ pip install -e . -r requirements.txt

Database setup

Create a new psql user, this prompts for a user password, use the same password in the variable SQLALCHEMY_PASSWORD of the settings.py file:

$ sudo -u postgres createuser -s -P -e $USER

Create the ether_sql database in psql:

$ createdb ether_sql

We use Alembic to manage tables, you can create the tables by using this command:

$ ether_sql sql upgrade_tables

Setting up RabbitMQ

To use Celery we need to create a RabbitMQ user, a virtual host and allow that user access to that virtual host:

$ sudo rabbitmqctl add_user myuser mypassword

$ sudo rabbitmqctl add_vhost myvhost

$ sudo rabbitmqctl set_user_tags myuser mytag

$ sudo rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*"

Substitute in appropriate values for myuser, mypassword and myvhost above and in the settings file.

Node settings

The settings to connect to a node are set in the settings.py file using classes.

Infura Settings:

The class PersonalInfuraSettings specifies settings to connect to a normal Infura node. You can fill in the value of your API token on NODE_API_TOKEN:

class PersonalInfuraSettings(DefaultSettings):
    NODE_TYPE = "Infura"
    NODE_API_TOKEN = ""  # your infura api_token
    NODE_URL = 'https://mainnet.infura.io/{}'.format(NODE_API_TOKEN)

Local Node settings:

We use the automatic methods in web3.py to connect to a node, if a local node is available then only the NODE_TYPE is required. The class PersonalParitySettings is used to connect to a local Parity node:

class PersonalParitySettings(DefaultSettings):
    NODE_TYPE = "Parity"
    # Use this option to parse traces, needs parity with cli --tracing=on
    PARSE_TRACE = True

Whereas, the class PersonalGethSettings is used to connect to a local Geth node:

class PersonalGethSettings(DefaultSettings):
    NODE_TYPE = "Geth"

Syncing data

ether_sql has several built in cli commands to facilitate scraping data. To start the sync just type:

$ ether_sql scrape_block_range

This will by default start pushing the data from an Infura node to the psql database. To switch nodes use the settings flag:

$ ether_sql --settings='PersonalParitySettings' scrape_block_range

To access other Command Line Interfaces (CLI) checkout the CLI’s.