Shards database, collecting shard repositories, releases and dependencies.
- Crystal 0.30.1
- PostgresSQL database (for data storage). Should probably work with all versions => 10. Tested with PostgreSQL 11.5
PostgreSQL databases needs to be created manually. Connection configuration is read from environment variables:
- development database:
- testing database:
TEST_DATABASE_URL(only for running tests)
SENTRY_DSN(optional, for sending error reports to sentry)
Install database schema by running
make db test_db.
shards install to install dependencies.
shards build worker to build the worker executable.
bin/worker import_catalog: Reads repositories mentioned in
catalog/*.ymlfiles and synchronizes them with the database.
bin/worker sync_repos [hours [ratio]]: Synchronizes data from git repositories to database for all repositories not synced in the last
24) or never synced at all.
ratiospecifies the amount of outdated repositories being run in this job (default:
2.0 / hours). This enables an even distribution of batch sizes. Recommended to be run every hour.
bin/worker updated_metrics: Updates shard metrics in the database. Should be run once per day.
A web application for browsing the database is available at https://github.com/shardbox/shardbox-web
make test to execute the spec suite.
- Fork it (https://github.com/shardbox/shardbox-web/fork)
- Create your feature branch (
git checkout -b my-new-feature)
- Commit your changes (
git commit -am 'Add some feature')
- Push to the branch (
git push origin my-new-feature)
- Create a new Pull Request
- Johannes Müller - creator and maintainer