To generate the charts this is my workflow:
0: I created a table in my SQLite database (named "travel.db") with the following command:
create table stock (country, itemname, itemid, stamp, amount);
1: I have a cronjob that runs the following script every 15 minutes
#!/bin/sh # collect data into travel.db wget -q -Oyatadata.json https://yata.alwaysdata.net/api/v1/travel/export/ yatadata <yatadata.json |sqlite3 travel.db rm yatadata.json # clear old database entries sqlite3 travel.db <rmold.sql # produce data files sqlite3 travel.db <swi-272.sql >swi-272.dat #... # create graphs for hosting gnuplot <www-swi-272.gnu #... #delete dat files rm swi-272.dat #... #transfer to host ftp -n -i -p bzzt.mygamesonline.org <<FTPCOMMANDS user USERNAME PASSWORD cd htdocs mput *.png quit FTPCOMMANDS #delete images for hosting rm www-*.png
2: The "yatadata" was written by me in C (I'm not publishing the source code). It simply reads JSON data from its standard input (from yata) and outputs lines of SQL commands in the form "insert into stock (country, itemname, itemid, stamp, amount) values ("swi", "Edelweiss", 272, 1603870673, 9603);"
Probably there are already ready-made tools that convert JSON to SQL, but I used my hammer and treated this part of the workflow as a nail.
3: The 'rmold.sql' commands deletes old records from the database.
-- delete records more than a day (86400 seconds) old delete from stock where strftime('%s', 'now') - stamp > 86400;
4: The SQL command to retrieve data from the database for each specific chart
.separator " " select (stamp - strftime('%s', 'now')) / 3600.0 , amount / 1000.0 from stock where country = 'swi' and itemid = 272 order by 1;
and the generated file looks like this
-23.9591666666667 8.519 -23.8863888888889 8.152 -23.8302777777778 8.008 -23.6930555555556 7.647 -23.6930555555556 7.647 ... ... ... ... ... ... ... 288 rows (give or take) -0.365833333333333 8.685 -0.295833333333333 8.656 -0.228333333333333 8.656 -0.129722222222222 8.627 -0.129722222222222 8.627
5: and finally use gnuplot with the final data file with the following script
set terminal png size 426,320 set output 'www-swi-272.png' set lmargin at screen 0.1 set rmargin at screen 0.95 set bmargin at screen 0.1 set tmargin at screen 0.8 set title 'Switzerland / Edelweiss' set xrange [-24:0] set yrange [0:10] set border set xtics -24, 6 set mxtics 6 set ytics 0, 2 set grid xtics,mxtics ls -1, ls -1 set label "generated on `date -u '+%Y-%m-%d %H:%M:%S'`" at -20.5,10.4 plot 'swi-272.dat' u 1:(myLast=$2) w p pt 3 notitle \ , myLast w l lc 'light-salmon' notitle