grailbag.git
19 months agoAdded running GrailBag scripts to command line client. master
Stanislaw Klekot [Sat, 24 Mar 2018 17:04:41 +0000 (19:04 +0200)]
Added running GrailBag scripts to command line client.

19 months agoAllowed "-" (STDOUT) as output filename for getting an artifact from repository.
Stanislaw Klekot [Sat, 24 Mar 2018 16:38:40 +0000 (18:38 +0200)]
Allowed "-" (STDOUT) as output filename for getting an artifact from repository.

19 months agoChanged logging and exception types in GrailBag script.
Stanislaw Klekot [Sat, 24 Mar 2018 16:01:10 +0000 (18:01 +0200)]
Changed logging and exception types in GrailBag script.

19 months agoImplemented parsing GrailBag script.
Stanislaw Klekot [Sat, 24 Mar 2018 12:50:27 +0000 (14:50 +0200)]
Implemented parsing GrailBag script.

19 months agoCommand operation in GrailBag script now is var-args.
Stanislaw Klekot [Sat, 24 Mar 2018 12:30:32 +0000 (14:30 +0200)]
Command operation in GrailBag script now is var-args.

It will be easier for script parser later. Plus, it's a little more explicit
now what it keeps.

19 months agoTracking env vars moved from building a script to planning its execution.
Stanislaw Klekot [Sat, 24 Mar 2018 12:12:00 +0000 (14:12 +0200)]
Tracking env vars moved from building a script to planning its execution.

This will simplify script parser.

The variables are still tracked by the builder, but only for filling the
paths and other arguments.

19 months agoAdded GrailBag script syntax for unsetting environment variables.
Stanislaw Klekot [Sat, 24 Mar 2018 12:10:58 +0000 (14:10 +0200)]
Added GrailBag script syntax for unsetting environment variables.

20 months agoMoved reading password to after it's confirmed that args are OK-ish.
Stanislaw Klekot [Sun, 18 Mar 2018 21:35:51 +0000 (23:35 +0200)]
Moved reading password to after it's confirmed that args are OK-ish.

20 months agoImplemented running external commands and shell scripts.
Stanislaw Klekot [Sun, 18 Mar 2018 21:12:55 +0000 (23:12 +0200)]
Implemented running external commands and shell scripts.

20 months agoAdded cleaning up the target directory for GrailBag script.
Stanislaw Klekot [Sun, 18 Mar 2018 20:58:01 +0000 (22:58 +0200)]
Added cleaning up the target directory for GrailBag script.

20 months agoAdded incomplete implementation of GrailBag script module.
Stanislaw Klekot [Sun, 18 Mar 2018 17:03:40 +0000 (19:03 +0200)]
Added incomplete implementation of GrailBag script module.

20 months agoGrailBag client exceptions moved to a separate module.
Stanislaw Klekot [Sun, 18 Mar 2018 11:57:00 +0000 (13:57 +0200)]
GrailBag client exceptions moved to a separate module.

It will be necessary for GrailBag script.

20 months agoArtifact container moved to a separate module.
Stanislaw Klekot [Sun, 18 Mar 2018 11:41:07 +0000 (13:41 +0200)]
Artifact container moved to a separate module.

It will be necessary for GrailBag script.

20 months agoMoved authentication and authorization replies around in client code.
Stanislaw Klekot [Sun, 18 Mar 2018 12:16:34 +0000 (13:16 +0100)]
Moved authentication and authorization replies around in client code.

20 months agoGrailBag client split out to a separate Python module.
Stanislaw Klekot [Sun, 18 Mar 2018 11:55:59 +0000 (12:55 +0100)]
GrailBag client split out to a separate Python module.

20 months agoListing artifacts now can list multiple types at a single command.
Stanislaw Klekot [Sat, 17 Mar 2018 19:38:20 +0000 (20:38 +0100)]
Listing artifacts now can list multiple types at a single command.

20 months agoAdded proper config reloading to daemon commands.
Stanislaw Klekot [Sat, 17 Mar 2018 19:02:47 +0000 (20:02 +0100)]
Added proper config reloading to daemon commands.

20 months agoFixed reloading TCP/SSL sockets.
Stanislaw Klekot [Sat, 17 Mar 2018 18:59:51 +0000 (19:59 +0100)]
Fixed reloading TCP/SSL sockets.

20 months agoImplemented reloading artifact registry and authentication server.
Stanislaw Klekot [Sat, 17 Mar 2018 18:26:35 +0000 (19:26 +0100)]
Implemented reloading artifact registry and authentication server.

Reloading registry is incomplete, as it cannot change disk storage directory
yet.

20 months agoElaborated on TCP subsystem reload errors.
Stanislaw Klekot [Sat, 17 Mar 2018 18:25:26 +0000 (19:25 +0100)]
Elaborated on TCP subsystem reload errors.

20 months agoAuth requests during stopping excess auth workers are now closed properly.
Stanislaw Klekot [Sat, 17 Mar 2018 18:10:00 +0000 (19:10 +0100)]
Auth requests during stopping excess auth workers are now closed properly.

20 months agoCommand line client's output made YAML for most operations.
Stanislaw Klekot [Sat, 17 Mar 2018 16:39:57 +0000 (17:39 +0100)]
Command line client's output made YAML for most operations.

Only editing (TAGS, TOKENS, and DELETE) don't print YAML.

20 months agoAdded WHOAMI command to the GrailBag protocol.
Stanislaw Klekot [Sat, 17 Mar 2018 15:46:08 +0000 (16:46 +0100)]
Added WHOAMI command to the GrailBag protocol.

20 months agoAdded verifying names of artifact type, tokens, and tags.
Stanislaw Klekot [Sat, 17 Mar 2018 14:38:57 +0000 (15:38 +0100)]
Added verifying names of artifact type, tokens, and tags.

20 months agoWrong type in a typespec.
Stanislaw Klekot [Sat, 17 Mar 2018 14:02:01 +0000 (15:02 +0100)]
Wrong type in a typespec.

20 months agoFixed checking mandatory tags of an uploaded artifact.
Stanislaw Klekot [Sat, 17 Mar 2018 13:40:31 +0000 (14:40 +0100)]
Fixed checking mandatory tags of an uploaded artifact.

20 months agoAdded two levels of hashing of paths in disk storage.
Stanislaw Klekot [Sat, 17 Mar 2018 12:22:37 +0000 (13:22 +0100)]
Added two levels of hashing of paths in disk storage.

This prevents uncontrollable growing of disk storage directory on ext2/3/4
filesystems.

20 months agoChanged digest from SHA1 to SHA512.
Stanislaw Klekot [Sat, 17 Mar 2018 10:37:15 +0000 (11:37 +0100)]
Changed digest from SHA1 to SHA512.

It turns out, Erlang R15B01 already has the necessary functions, they just
weren't documented in the manual.

20 months agoFixed STORE operation (forgotten field in grailbag_reg call).
Stanislaw Klekot [Sat, 17 Mar 2018 10:35:18 +0000 (11:35 +0100)]
Fixed STORE operation (forgotten field in grailbag_reg call).

20 months agoAdded authentication and authorization to client protocol.
Stanislaw Klekot [Tue, 13 Mar 2018 20:39:47 +0000 (21:39 +0100)]
Added authentication and authorization to client protocol.

20 months agoRemoved two outdated TODO comments.
Stanislaw Klekot [Tue, 13 Mar 2018 20:38:36 +0000 (21:38 +0100)]
Removed two outdated TODO comments.

20 months agoCode alignment fixes.
Stanislaw Klekot [Tue, 13 Mar 2018 20:30:10 +0000 (21:30 +0100)]
Code alignment fixes.

20 months agoAllowed null UUID in parsing.
Stanislaw Klekot [Tue, 13 Mar 2018 20:28:39 +0000 (21:28 +0100)]
Allowed null UUID in parsing.

20 months agoArtifact type names (binaries) made copies in authentication server.
Stanislaw Klekot [Tue, 13 Mar 2018 20:27:37 +0000 (21:27 +0100)]
Artifact type names (binaries) made copies in authentication server.

20 months agoAdded authentication server.
Stanislaw Klekot [Sun, 11 Mar 2018 22:04:51 +0000 (23:04 +0100)]
Added authentication server.

It's not used yet.

20 months agoMoved formatting IP addresses as strings to a common module.
Stanislaw Klekot [Sun, 11 Mar 2018 16:41:51 +0000 (17:41 +0100)]
Moved formatting IP addresses as strings to a common module.

20 months agoUpdated example environment in *.app.src.
Stanislaw Klekot [Sun, 11 Mar 2018 15:47:57 +0000 (16:47 +0100)]
Updated example environment in *.app.src.

20 months agoAdded building proper environment for `grailbag' application.
Stanislaw Klekot [Sun, 11 Mar 2018 13:55:04 +0000 (14:55 +0100)]
Added building proper environment for `grailbag' application.

20 months agoAdded skeleton for Indira.
Stanislaw Klekot [Sat, 10 Mar 2018 17:09:38 +0000 (18:09 +0100)]
Added skeleton for Indira.

20 months agoUpdated example TOML configuration.
Stanislaw Klekot [Sat, 10 Mar 2018 14:19:39 +0000 (15:19 +0100)]
Updated example TOML configuration.

20 months agoAdded missing registered atoms to .app.src.
Stanislaw Klekot [Sat, 10 Mar 2018 12:03:18 +0000 (13:03 +0100)]
Added missing registered atoms to .app.src.

20 months agoComments updates.
Stanislaw Klekot [Sat, 10 Mar 2018 12:03:05 +0000 (13:03 +0100)]
Comments updates.

20 months agoClient connections wrapped in SSL.
Stanislaw Klekot [Sat, 3 Mar 2018 20:02:15 +0000 (21:02 +0100)]
Client connections wrapped in SSL.

20 months agoAdded encoding schema validation errors.
Stanislaw Klekot [Sat, 3 Mar 2018 16:24:27 +0000 (17:24 +0100)]
Added encoding schema validation errors.

Now the client will finally report something meaningful on failed STORE, TAGS,
and TOKENS operations.

20 months agoAdded flags to artifact info in the protocol.
Stanislaw Klekot [Sat, 3 Mar 2018 16:10:18 +0000 (17:10 +0100)]
Added flags to artifact info in the protocol.

The only flag for now is "conforming to schema".

20 months agoAdded "valid" flag to artifact info.
Stanislaw Klekot [Sat, 3 Mar 2018 14:42:09 +0000 (15:42 +0100)]
Added "valid" flag to artifact info.

As with tokens, since it's an information external to artifact, it's reported
as default value of `valid' from reading from disk storage.

The code re-validates schema on every artifact access, which is inefficient,
but the code is simple and works correctly in all cases.

21 months agoAdded strict verification of modification operations.
Stanislaw Klekot [Sun, 18 Feb 2018 14:45:32 +0000 (15:45 +0100)]
Added strict verification of modification operations.

21 months agoAdded expecting errors for STORE operation in Python client.
Stanislaw Klekot [Sun, 18 Feb 2018 13:47:47 +0000 (14:47 +0100)]
Added expecting errors for STORE operation in Python client.

21 months agoAdded schema validation to artifact registry.
Stanislaw Klekot [Sat, 17 Feb 2018 19:12:07 +0000 (20:12 +0100)]
Added schema validation to artifact registry.

Schema errors are only logged, operations are still allowed.

21 months agoAdded schema validation module.
Stanislaw Klekot [Sat, 17 Feb 2018 18:30:28 +0000 (19:30 +0100)]
Added schema validation module.

21 months agoSeparated storing tokens and other metadata.
Stanislaw Klekot [Sun, 11 Feb 2018 14:26:17 +0000 (15:26 +0100)]
Separated storing tokens and other metadata.

21 months agoAdded handler for ^C to GrailBag client.
Stanislaw Klekot [Sat, 3 Feb 2018 11:16:56 +0000 (12:16 +0100)]
Added handler for ^C to GrailBag client.

21 months agoAdded WATCH operation on client side.
Stanislaw Klekot [Sat, 3 Feb 2018 11:14:19 +0000 (12:14 +0100)]
Added WATCH operation on client side.

Implemented by listing artifacts of a given type in regular intervals and
calculating differences between the lists. A prototype for server-side thing
to find out what data needs to be reported.

22 months agoAdded forgotten exports in grailbag_uuid module.
Stanislaw Klekot [Thu, 28 Dec 2017 14:25:16 +0000 (15:25 +0100)]
Added forgotten exports in grailbag_uuid module.

22 months agoUpdated logging in TCP client handler.
Stanislaw Klekot [Thu, 28 Dec 2017 14:20:32 +0000 (15:20 +0100)]
Updated logging in TCP client handler.

Now only changing operations are logged (STORE, DELETE, UPDATE_TAGS, and
UPDATE_TOKENS). TCP errors are silenced as well, except for the case of
aborting a STORE.

22 months agoErrors during moving a token no longer crash the artifact registry.
Stanislaw Klekot [Thu, 28 Dec 2017 14:01:25 +0000 (15:01 +0100)]
Errors during moving a token no longer crash the artifact registry.

22 months agoServer-side errors are now properly logged.
Stanislaw Klekot [Thu, 28 Dec 2017 13:35:14 +0000 (14:35 +0100)]
Server-side errors are now properly logged.

22 months agoAdded to client some guards against mixing operations.
Stanislaw Klekot [Wed, 27 Dec 2017 14:52:20 +0000 (15:52 +0100)]
Added to client some guards against mixing operations.

Not really necessary now, but if I extract the GrailBagServer code to
a library, it should come handy.

22 months agoFixed updating tags for an artifact.
Stanislaw Klekot [Wed, 27 Dec 2017 13:54:33 +0000 (14:54 +0100)]
Fixed updating tags for an artifact.

The bug was me misunderstanding lists:keymerge() function.

22 months agoAdded to Python client uploading an artifact.
Stanislaw Klekot [Wed, 27 Dec 2017 12:10:17 +0000 (13:10 +0100)]
Added to Python client uploading an artifact.

22 months agoImproved error reporting in Python client.
Stanislaw Klekot [Wed, 27 Dec 2017 12:09:03 +0000 (13:09 +0100)]
Improved error reporting in Python client.

22 months agoAdded partially implemented client in Python.
Stanislaw Klekot [Fri, 22 Dec 2017 23:27:38 +0000 (00:27 +0100)]
Added partially implemented client in Python.

22 months agoAdded listing known artifact types.
Stanislaw Klekot [Fri, 22 Dec 2017 17:33:01 +0000 (18:33 +0100)]
Added listing known artifact types.

22 months agoAdded signaling maximum allowed chunk size in STORE operation.
Stanislaw Klekot [Fri, 22 Dec 2017 17:14:25 +0000 (18:14 +0100)]
Added signaling maximum allowed chunk size in STORE operation.

22 months agoRemoved out of date TODO markers.
Stanislaw Klekot [Fri, 22 Dec 2017 17:03:48 +0000 (18:03 +0100)]
Removed out of date TODO markers.

22 months agoImplemented moving tokens instead of just setting them.
Stanislaw Klekot [Fri, 22 Dec 2017 16:54:04 +0000 (17:54 +0100)]
Implemented moving tokens instead of just setting them.

This is how they were supposed to work.

22 months agoIn-memory mtime info kept updated after changing artifact's tags and tokens.
Stanislaw Klekot [Fri, 22 Dec 2017 16:38:19 +0000 (17:38 +0100)]
In-memory mtime info kept updated after changing artifact's tags and tokens.

22 months agolists:foldl() in registry initialization should be lists:foreach().
Stanislaw Klekot [Fri, 22 Dec 2017 14:33:28 +0000 (15:33 +0100)]
lists:foldl() in registry initialization should be lists:foreach().

22 months agoDELETE operation now fails if the artifact has any tokens set.
Stanislaw Klekot [Fri, 22 Dec 2017 14:03:00 +0000 (15:03 +0100)]
DELETE operation now fails if the artifact has any tokens set.

22 months agoAdded updating tags and tokens for artifact.
Stanislaw Klekot [Fri, 22 Dec 2017 13:43:56 +0000 (14:43 +0100)]
Added updating tags and tokens for artifact.

22 months agoAdded checksum protection to metadata in disk storage.
Stanislaw Klekot [Thu, 21 Dec 2017 15:28:09 +0000 (16:28 +0100)]
Added checksum protection to metadata in disk storage.

22 months agoAdded creation time and modification time throughout the whole application.
Stanislaw Klekot [Thu, 21 Dec 2017 14:50:31 +0000 (15:50 +0100)]
Added creation time and modification time throughout the whole application.

22 months agoImplemented uploading artifacts.
Stanislaw Klekot [Wed, 20 Dec 2017 15:51:24 +0000 (16:51 +0100)]
Implemented uploading artifacts.

22 months agoDownload operation made robust against silent data storage changes.
Stanislaw Klekot [Wed, 20 Dec 2017 13:54:51 +0000 (14:54 +0100)]
Download operation made robust against silent data storage changes.

Previously it was possible to break the client protocol if somebody appended
something to artifact's body.

22 months agoAdded retrieving artifacts from server.
Stanislaw Klekot [Wed, 20 Dec 2017 13:38:06 +0000 (14:38 +0100)]
Added retrieving artifacts from server.

22 months agoPrepared TCP handler to run for longer time.
Stanislaw Klekot [Wed, 20 Dec 2017 11:24:36 +0000 (12:24 +0100)]
Prepared TCP handler to run for longer time.

23 months agoImplemented part of GrailBag protocol.
Stanislaw Klekot [Tue, 19 Dec 2017 16:41:13 +0000 (17:41 +0100)]
Implemented part of GrailBag protocol.

23 months agoUniform naming of size/length variable in binary decoding.
Stanislaw Klekot [Tue, 19 Dec 2017 15:24:41 +0000 (16:24 +0100)]
Uniform naming of size/length variable in binary decoding.

23 months agoUnified artifact info records across the modules and functions.
Stanislaw Klekot [Tue, 19 Dec 2017 14:14:33 +0000 (15:14 +0100)]
Unified artifact info records across the modules and functions.

23 months agoRenamed upload/add/register operation to "store".
Stanislaw Klekot [Mon, 18 Dec 2017 19:03:31 +0000 (20:03 +0100)]
Renamed upload/add/register operation to "store".

23 months agoAdded in-memory artifacts registry.
Stanislaw Klekot [Mon, 18 Dec 2017 16:40:08 +0000 (17:40 +0100)]
Added in-memory artifacts registry.

This will be used for most of the time when GrailBag server is queried.
The registry is also a natural place to serialize delete/update operations.

23 months agoUpdated cross-references in grailbag_artifact module.
Stanislaw Klekot [Mon, 18 Dec 2017 16:34:21 +0000 (17:34 +0100)]
Updated cross-references in grailbag_artifact module.

23 months agoAdded listing artifacts from data directory.
Stanislaw Klekot [Mon, 18 Dec 2017 14:25:33 +0000 (15:25 +0100)]
Added listing artifacts from data directory.

23 months agoAdded functions for updating tags and tokens.
Stanislaw Klekot [Mon, 18 Dec 2017 11:38:22 +0000 (12:38 +0100)]
Added functions for updating tags and tokens.

23 months agoAdded reading part of artifact handles.
Stanislaw Klekot [Sun, 17 Dec 2017 22:03:08 +0000 (23:03 +0100)]
Added reading part of artifact handles.

23 months agoAdded code for working with artifact storage.
Stanislaw Klekot [Sun, 17 Dec 2017 21:20:58 +0000 (22:20 +0100)]
Added code for working with artifact storage.

Incomplete, write only for now.

23 months agoFilled server's skeleton.
Stanislaw Klekot [Sun, 17 Dec 2017 13:00:35 +0000 (14:00 +0100)]
Filled server's skeleton.

Updated example config a little.

23 months agoAdded skeleton for the server part.
Stanislaw Klekot [Sun, 17 Dec 2017 12:33:48 +0000 (13:33 +0100)]
Added skeleton for the server part.

23 months agoInit.
Stanislaw Klekot [Sun, 17 Dec 2017 11:54:01 +0000 (12:54 +0100)]
Init.