Data downloader for Gekko


Version
Download 281
Total Views 1448
Stock
File Size 0.00 KB
File Type
Create Date September 30, 2017
Last Updated September 30, 2017
Download

A really great guy from the Gekko community called Haxus has built a data downloader for Gekko. It works by getting data from https://bitcoincharts.com/charts.

You will need to download and use the shell script and follow his instructions below. If you have any queries put them in the comments.

Install “bitcoin-scraper”
git clone https://github.com/F1LT3R/bitcoin-scraper.git
cd bitcoin-scraper
mkdir data
npm install

### Fix Bugs if not already done by the developer meanwhile
### Font fix
cd node_modules/figlet/fonts/
wget http://www.textfiles.com/art/pepper.flf
cd ../../../

### spelling fix
sed -i 's/pepper/Pepper/' combine.js
sed -i 's/pepper/Pepper/' scrape.js

### Fix the URL to get minute candles instead of hour candles:
sed -i "s/r=60/r=1/g" scrape.js

### Configure the market you want to download
# by changing “const market = 'bitstampUSD'“ to
# whatever you want, plz see
# https://bitcoincharts.com/charts (symbols) for more detailes
const market = 'bitstampUSD'

# You may also change the timeframe within that file
const dates = {
from: {
year: 2011,
month: 9,
day: 14
},

to: {
year: 2017,
month: 08,
day: 9
},
}

### Now let us do the magic
./convertToDB.sh

#move the
mv data/bitstamp_0.1.db /Path/To/Gekko/history/

7 Replies to “Data downloader for Gekko”

  1. Hello,

    Thank you for your script. I’ve spent the last day trying to get it to run locally on OSX. There were a few problems with the line feed replaces being done by sed. I have replaced those with perl commands and that seems to have fixed that issue.

    Now that the .db file is being created, I’m running into another issue. I can’t seem to figure out how to get these into the history in gekko. I have tried putting them in the gekko/history dir as well as the gekko/volumes/gekko/history dir and neither one seems to make them available to the data importer in the gekko ui.

    Do you have any advice on getting the .db files to be available to gekko?

    Thanks again,

  2. Thanks for the script. I found another issue:

    When fixing the DB holes, the script does not take over the first candle / line.

    Can someone reproduce this? You just need to compare bitcoin-history.tmp and bitcoin-history.tmp2

  3. I too seem to be getting the Sed issue:

    Saved to: ./data/bitcoin-history.json
    Preparing …
    sed: 1: “./data/bitcoin-history. …”: invalid command code .
    Fixing DB holes,- This could take a long time!
    converting Data to sqldump format …
    Creating Gekko Sqlite3 DB …
    Error: near line 15: near “)”: syntax error
    … done

    Could someone please share the perl commands? Thanks.

  4. @Joost

    I had the same problem. Earlier when I was doing this part:

    ### spelling fix
    sed -i ‘s/pepper/Pepper/’ combine.js
    sed -i ‘s/pepper/Pepper/’ scrape.js

    I got the exact same error. Something with OSX so I had to change them by adding “” after the -i to:

    ### spelling fix
    sed -i “” ‘s/pepper/Pepper/’ combine.js
    sed -i “” ‘s/pepper/Pepper/’ scrape.js

    I tried the same for the sed error from convertToDB.sh but it didn’t work

Leave a Reply

Your email address will not be published. Required fields are marked *