I had a similar task and found a solution.
I just want Grafana to run this query every X minutes
Grafana is only a visualization solution, it does not store data itself, you need some time series database as a proxy. I used this scheme - PostgreSQL -> prometheus-sql -> Prometheus -> Grafana.
Below I added configuration files for prometheus-sql and Prometheus:
docker-compose.yaml
version: '3.7'
volumes:
prometheus_data: {}
services:
prometheus:
image: prom/prometheus
volumes:
- ./prometheus/:/etc/prometheus/
- prometheus_data:/var/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yaml'
- '--storage.tsdb.path=/var/prometheus'
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
- '--web.console.templates=/usr/share/prometheus/consoles'
ports:
- 9090:9090
user: root
links:
- prometheus-sql:prometheus-sql
depends_on:
- prometheus-sql
restart: always
sqlagent:
image: dbhi/sql-agent
prometheus-sql:
image: dbhi/prometheus-sql
ports:
- 8080:8080
links:
- sqlagent:sqlagent
depends_on:
- sqlagent
command:
- -config
- /etc/prometheus-sql/config.yml
- -queries
- /etc/prometheus-sql/queries.yml
- -service
- http://sqlagent:5000
volumes:
- ./sql/config.yml:/etc/prometheus-sql/config.yml
- ./sql/queries.yml:/etc/prometheus-sql/queries.yml
prometheus/prometheus.yaml:
scrape_configs:
- job_name: 'Prometheus SQL'
scrape_interval: 1m
static_configs:
- targets: ['prometheus-sql:8080']
prometheus-sql/config.yml (fill your database connection settings here):
defaults:
data-source: postgresql
query-interval: 1m
query-timeout: 30s
query-value-on-error: -1
data-sources:
postgresql:
driver: postgresql
properties:
host: DATABASE_HOST
port: 5432
user: DATABASE_USER
password: DATABASE_PASSWORD
database: DATABASE_NAME
prometheus-sql/queries.yml (tried to adopt to your case)
- table1_records_count_with_status_2:
help: table1 records count with status_id = 2
sql: >
SELECT count(*) AS count FROM myschema.table1 WHERE status_id = 2
data-field: count
Then in Grafana you could get time series data with the query like this:
query_result_table1_records_count_with_status_2{}
SELECTquery, thenINSERTing it back into the same database in a different table. Then, in another metric, you could query that table to get the data.