grailbag.git
2 years agoAdded running GrailBag scripts to command line client. master
Stanislaw Klekot [Sat, 24 Mar 2018 17:04:41 +0000 (19:04 +0200)]
Added running GrailBag scripts to command line client.

2 years agoAllowed "-" (STDOUT) as output filename for getting an artifact from repository.
Stanislaw Klekot [Sat, 24 Mar 2018 16:38:40 +0000 (18:38 +0200)]
Allowed "-" (STDOUT) as output filename for getting an artifact from repository.

2 years agoChanged logging and exception types in GrailBag script.
Stanislaw Klekot [Sat, 24 Mar 2018 16:01:10 +0000 (18:01 +0200)]
Changed logging and exception types in GrailBag script.

2 years agoImplemented parsing GrailBag script.
Stanislaw Klekot [Sat, 24 Mar 2018 12:50:27 +0000 (14:50 +0200)]
Implemented parsing GrailBag script.

2 years agoCommand operation in GrailBag script now is var-args.
Stanislaw Klekot [Sat, 24 Mar 2018 12:30:32 +0000 (14:30 +0200)]
Command operation in GrailBag script now is var-args.

It will be easier for script parser later. Plus, it's a little more explicit
now what it keeps.

2 years agoTracking env vars moved from building a script to planning its execution.
Stanislaw Klekot [Sat, 24 Mar 2018 12:12:00 +0000 (14:12 +0200)]
Tracking env vars moved from building a script to planning its execution.

This will simplify script parser.

The variables are still tracked by the builder, but only for filling the
paths and other arguments.

2 years agoAdded GrailBag script syntax for unsetting environment variables.
Stanislaw Klekot [Sat, 24 Mar 2018 12:10:58 +0000 (14:10 +0200)]
Added GrailBag script syntax for unsetting environment variables.

2 years agoMoved reading password to after it's confirmed that args are OK-ish.
Stanislaw Klekot [Sun, 18 Mar 2018 21:35:51 +0000 (23:35 +0200)]
Moved reading password to after it's confirmed that args are OK-ish.

2 years agoImplemented running external commands and shell scripts.
Stanislaw Klekot [Sun, 18 Mar 2018 21:12:55 +0000 (23:12 +0200)]
Implemented running external commands and shell scripts.

2 years agoAdded cleaning up the target directory for GrailBag script.
Stanislaw Klekot [Sun, 18 Mar 2018 20:58:01 +0000 (22:58 +0200)]
Added cleaning up the target directory for GrailBag script.

2 years agoAdded incomplete implementation of GrailBag script module.
Stanislaw Klekot [Sun, 18 Mar 2018 17:03:40 +0000 (19:03 +0200)]
Added incomplete implementation of GrailBag script module.

2 years agoGrailBag client exceptions moved to a separate module.
Stanislaw Klekot [Sun, 18 Mar 2018 11:57:00 +0000 (13:57 +0200)]
GrailBag client exceptions moved to a separate module.

It will be necessary for GrailBag script.

2 years agoArtifact container moved to a separate module.
Stanislaw Klekot [Sun, 18 Mar 2018 11:41:07 +0000 (13:41 +0200)]
Artifact container moved to a separate module.

It will be necessary for GrailBag script.

2 years agoMoved authentication and authorization replies around in client code.
Stanislaw Klekot [Sun, 18 Mar 2018 12:16:34 +0000 (13:16 +0100)]
Moved authentication and authorization replies around in client code.

2 years agoGrailBag client split out to a separate Python module.
Stanislaw Klekot [Sun, 18 Mar 2018 11:55:59 +0000 (12:55 +0100)]
GrailBag client split out to a separate Python module.

2 years agoListing artifacts now can list multiple types at a single command.
Stanislaw Klekot [Sat, 17 Mar 2018 19:38:20 +0000 (20:38 +0100)]
Listing artifacts now can list multiple types at a single command.

2 years agoAdded proper config reloading to daemon commands.
Stanislaw Klekot [Sat, 17 Mar 2018 19:02:47 +0000 (20:02 +0100)]
Added proper config reloading to daemon commands.

2 years agoFixed reloading TCP/SSL sockets.
Stanislaw Klekot [Sat, 17 Mar 2018 18:59:51 +0000 (19:59 +0100)]
Fixed reloading TCP/SSL sockets.

2 years agoImplemented reloading artifact registry and authentication server.
Stanislaw Klekot [Sat, 17 Mar 2018 18:26:35 +0000 (19:26 +0100)]
Implemented reloading artifact registry and authentication server.

Reloading registry is incomplete, as it cannot change disk storage directory
yet.

2 years agoElaborated on TCP subsystem reload errors.
Stanislaw Klekot [Sat, 17 Mar 2018 18:25:26 +0000 (19:25 +0100)]
Elaborated on TCP subsystem reload errors.

2 years agoAuth requests during stopping excess auth workers are now closed properly.
Stanislaw Klekot [Sat, 17 Mar 2018 18:10:00 +0000 (19:10 +0100)]
Auth requests during stopping excess auth workers are now closed properly.

2 years agoCommand line client's output made YAML for most operations.
Stanislaw Klekot [Sat, 17 Mar 2018 16:39:57 +0000 (17:39 +0100)]
Command line client's output made YAML for most operations.

Only editing (TAGS, TOKENS, and DELETE) don't print YAML.

2 years agoAdded WHOAMI command to the GrailBag protocol.
Stanislaw Klekot [Sat, 17 Mar 2018 15:46:08 +0000 (16:46 +0100)]
Added WHOAMI command to the GrailBag protocol.

2 years agoAdded verifying names of artifact type, tokens, and tags.
Stanislaw Klekot [Sat, 17 Mar 2018 14:38:57 +0000 (15:38 +0100)]
Added verifying names of artifact type, tokens, and tags.

2 years agoWrong type in a typespec.
Stanislaw Klekot [Sat, 17 Mar 2018 14:02:01 +0000 (15:02 +0100)]
Wrong type in a typespec.

2 years agoFixed checking mandatory tags of an uploaded artifact.
Stanislaw Klekot [Sat, 17 Mar 2018 13:40:31 +0000 (14:40 +0100)]
Fixed checking mandatory tags of an uploaded artifact.

2 years agoAdded two levels of hashing of paths in disk storage.
Stanislaw Klekot [Sat, 17 Mar 2018 12:22:37 +0000 (13:22 +0100)]
Added two levels of hashing of paths in disk storage.

This prevents uncontrollable growing of disk storage directory on ext2/3/4
filesystems.

2 years agoChanged digest from SHA1 to SHA512.
Stanislaw Klekot [Sat, 17 Mar 2018 10:37:15 +0000 (11:37 +0100)]
Changed digest from SHA1 to SHA512.

It turns out, Erlang R15B01 already has the necessary functions, they just
weren't documented in the manual.

2 years agoFixed STORE operation (forgotten field in grailbag_reg call).
Stanislaw Klekot [Sat, 17 Mar 2018 10:35:18 +0000 (11:35 +0100)]
Fixed STORE operation (forgotten field in grailbag_reg call).

2 years agoAdded authentication and authorization to client protocol.
Stanislaw Klekot [Tue, 13 Mar 2018 20:39:47 +0000 (21:39 +0100)]
Added authentication and authorization to client protocol.

2 years agoRemoved two outdated TODO comments.
Stanislaw Klekot [Tue, 13 Mar 2018 20:38:36 +0000 (21:38 +0100)]
Removed two outdated TODO comments.

2 years agoCode alignment fixes.
Stanislaw Klekot [Tue, 13 Mar 2018 20:30:10 +0000 (21:30 +0100)]
Code alignment fixes.

2 years agoAllowed null UUID in parsing.
Stanislaw Klekot [Tue, 13 Mar 2018 20:28:39 +0000 (21:28 +0100)]
Allowed null UUID in parsing.

2 years agoArtifact type names (binaries) made copies in authentication server.
Stanislaw Klekot [Tue, 13 Mar 2018 20:27:37 +0000 (21:27 +0100)]
Artifact type names (binaries) made copies in authentication server.

2 years agoAdded authentication server.
Stanislaw Klekot [Sun, 11 Mar 2018 22:04:51 +0000 (23:04 +0100)]
Added authentication server.

It's not used yet.

2 years agoMoved formatting IP addresses as strings to a common module.
Stanislaw Klekot [Sun, 11 Mar 2018 16:41:51 +0000 (17:41 +0100)]
Moved formatting IP addresses as strings to a common module.

2 years agoUpdated example environment in *.app.src.
Stanislaw Klekot [Sun, 11 Mar 2018 15:47:57 +0000 (16:47 +0100)]
Updated example environment in *.app.src.

2 years agoAdded building proper environment for `grailbag' application.
Stanislaw Klekot [Sun, 11 Mar 2018 13:55:04 +0000 (14:55 +0100)]
Added building proper environment for `grailbag' application.

2 years agoAdded skeleton for Indira.
Stanislaw Klekot [Sat, 10 Mar 2018 17:09:38 +0000 (18:09 +0100)]
Added skeleton for Indira.

2 years agoUpdated example TOML configuration.
Stanislaw Klekot [Sat, 10 Mar 2018 14:19:39 +0000 (15:19 +0100)]
Updated example TOML configuration.

2 years agoAdded missing registered atoms to .app.src.
Stanislaw Klekot [Sat, 10 Mar 2018 12:03:18 +0000 (13:03 +0100)]
Added missing registered atoms to .app.src.

2 years agoComments updates.
Stanislaw Klekot [Sat, 10 Mar 2018 12:03:05 +0000 (13:03 +0100)]
Comments updates.

2 years agoClient connections wrapped in SSL.
Stanislaw Klekot [Sat, 3 Mar 2018 20:02:15 +0000 (21:02 +0100)]
Client connections wrapped in SSL.

2 years agoAdded encoding schema validation errors.
Stanislaw Klekot [Sat, 3 Mar 2018 16:24:27 +0000 (17:24 +0100)]
Added encoding schema validation errors.

Now the client will finally report something meaningful on failed STORE, TAGS,
and TOKENS operations.

2 years agoAdded flags to artifact info in the protocol.
Stanislaw Klekot [Sat, 3 Mar 2018 16:10:18 +0000 (17:10 +0100)]
Added flags to artifact info in the protocol.

The only flag for now is "conforming to schema".

2 years agoAdded "valid" flag to artifact info.
Stanislaw Klekot [Sat, 3 Mar 2018 14:42:09 +0000 (15:42 +0100)]
Added "valid" flag to artifact info.

As with tokens, since it's an information external to artifact, it's reported
as default value of `valid' from reading from disk storage.

The code re-validates schema on every artifact access, which is inefficient,
but the code is simple and works correctly in all cases.

2 years agoAdded strict verification of modification operations.
Stanislaw Klekot [Sun, 18 Feb 2018 14:45:32 +0000 (15:45 +0100)]
Added strict verification of modification operations.

2 years agoAdded expecting errors for STORE operation in Python client.
Stanislaw Klekot [Sun, 18 Feb 2018 13:47:47 +0000 (14:47 +0100)]
Added expecting errors for STORE operation in Python client.

2 years agoAdded schema validation to artifact registry.
Stanislaw Klekot [Sat, 17 Feb 2018 19:12:07 +0000 (20:12 +0100)]
Added schema validation to artifact registry.

Schema errors are only logged, operations are still allowed.

2 years agoAdded schema validation module.
Stanislaw Klekot [Sat, 17 Feb 2018 18:30:28 +0000 (19:30 +0100)]
Added schema validation module.

2 years agoSeparated storing tokens and other metadata.
Stanislaw Klekot [Sun, 11 Feb 2018 14:26:17 +0000 (15:26 +0100)]
Separated storing tokens and other metadata.

2 years agoAdded handler for ^C to GrailBag client.
Stanislaw Klekot [Sat, 3 Feb 2018 11:16:56 +0000 (12:16 +0100)]
Added handler for ^C to GrailBag client.

2 years agoAdded WATCH operation on client side.
Stanislaw Klekot [Sat, 3 Feb 2018 11:14:19 +0000 (12:14 +0100)]
Added WATCH operation on client side.

Implemented by listing artifacts of a given type in regular intervals and
calculating differences between the lists. A prototype for server-side thing
to find out what data needs to be reported.

2 years agoAdded forgotten exports in grailbag_uuid module.
Stanislaw Klekot [Thu, 28 Dec 2017 14:25:16 +0000 (15:25 +0100)]
Added forgotten exports in grailbag_uuid module.

2 years agoUpdated logging in TCP client handler.
Stanislaw Klekot [Thu, 28 Dec 2017 14:20:32 +0000 (15:20 +0100)]
Updated logging in TCP client handler.

Now only changing operations are logged (STORE, DELETE, UPDATE_TAGS, and
UPDATE_TOKENS). TCP errors are silenced as well, except for the case of
aborting a STORE.

2 years agoErrors during moving a token no longer crash the artifact registry.
Stanislaw Klekot [Thu, 28 Dec 2017 14:01:25 +0000 (15:01 +0100)]
Errors during moving a token no longer crash the artifact registry.

2 years agoServer-side errors are now properly logged.
Stanislaw Klekot [Thu, 28 Dec 2017 13:35:14 +0000 (14:35 +0100)]
Server-side errors are now properly logged.

2 years agoAdded to client some guards against mixing operations.
Stanislaw Klekot [Wed, 27 Dec 2017 14:52:20 +0000 (15:52 +0100)]
Added to client some guards against mixing operations.

Not really necessary now, but if I extract the GrailBagServer code to
a library, it should come handy.

2 years agoFixed updating tags for an artifact.
Stanislaw Klekot [Wed, 27 Dec 2017 13:54:33 +0000 (14:54 +0100)]
Fixed updating tags for an artifact.

The bug was me misunderstanding lists:keymerge() function.

2 years agoAdded to Python client uploading an artifact.
Stanislaw Klekot [Wed, 27 Dec 2017 12:10:17 +0000 (13:10 +0100)]
Added to Python client uploading an artifact.

2 years agoImproved error reporting in Python client.
Stanislaw Klekot [Wed, 27 Dec 2017 12:09:03 +0000 (13:09 +0100)]
Improved error reporting in Python client.

2 years agoAdded partially implemented client in Python.
Stanislaw Klekot [Fri, 22 Dec 2017 23:27:38 +0000 (00:27 +0100)]
Added partially implemented client in Python.

2 years agoAdded listing known artifact types.
Stanislaw Klekot [Fri, 22 Dec 2017 17:33:01 +0000 (18:33 +0100)]
Added listing known artifact types.

2 years agoAdded signaling maximum allowed chunk size in STORE operation.
Stanislaw Klekot [Fri, 22 Dec 2017 17:14:25 +0000 (18:14 +0100)]
Added signaling maximum allowed chunk size in STORE operation.

2 years agoRemoved out of date TODO markers.
Stanislaw Klekot [Fri, 22 Dec 2017 17:03:48 +0000 (18:03 +0100)]
Removed out of date TODO markers.

2 years agoImplemented moving tokens instead of just setting them.
Stanislaw Klekot [Fri, 22 Dec 2017 16:54:04 +0000 (17:54 +0100)]
Implemented moving tokens instead of just setting them.

This is how they were supposed to work.

2 years agoIn-memory mtime info kept updated after changing artifact's tags and tokens.
Stanislaw Klekot [Fri, 22 Dec 2017 16:38:19 +0000 (17:38 +0100)]
In-memory mtime info kept updated after changing artifact's tags and tokens.

2 years agolists:foldl() in registry initialization should be lists:foreach().
Stanislaw Klekot [Fri, 22 Dec 2017 14:33:28 +0000 (15:33 +0100)]
lists:foldl() in registry initialization should be lists:foreach().

2 years agoDELETE operation now fails if the artifact has any tokens set.
Stanislaw Klekot [Fri, 22 Dec 2017 14:03:00 +0000 (15:03 +0100)]
DELETE operation now fails if the artifact has any tokens set.

2 years agoAdded updating tags and tokens for artifact.
Stanislaw Klekot [Fri, 22 Dec 2017 13:43:56 +0000 (14:43 +0100)]
Added updating tags and tokens for artifact.

2 years agoAdded checksum protection to metadata in disk storage.
Stanislaw Klekot [Thu, 21 Dec 2017 15:28:09 +0000 (16:28 +0100)]
Added checksum protection to metadata in disk storage.

2 years agoAdded creation time and modification time throughout the whole application.
Stanislaw Klekot [Thu, 21 Dec 2017 14:50:31 +0000 (15:50 +0100)]
Added creation time and modification time throughout the whole application.

2 years agoImplemented uploading artifacts.
Stanislaw Klekot [Wed, 20 Dec 2017 15:51:24 +0000 (16:51 +0100)]
Implemented uploading artifacts.

2 years agoDownload operation made robust against silent data storage changes.
Stanislaw Klekot [Wed, 20 Dec 2017 13:54:51 +0000 (14:54 +0100)]
Download operation made robust against silent data storage changes.

Previously it was possible to break the client protocol if somebody appended
something to artifact's body.

2 years agoAdded retrieving artifacts from server.
Stanislaw Klekot [Wed, 20 Dec 2017 13:38:06 +0000 (14:38 +0100)]
Added retrieving artifacts from server.

2 years agoPrepared TCP handler to run for longer time.
Stanislaw Klekot [Wed, 20 Dec 2017 11:24:36 +0000 (12:24 +0100)]
Prepared TCP handler to run for longer time.

2 years agoImplemented part of GrailBag protocol.
Stanislaw Klekot [Tue, 19 Dec 2017 16:41:13 +0000 (17:41 +0100)]
Implemented part of GrailBag protocol.

2 years agoUniform naming of size/length variable in binary decoding.
Stanislaw Klekot [Tue, 19 Dec 2017 15:24:41 +0000 (16:24 +0100)]
Uniform naming of size/length variable in binary decoding.

2 years agoUnified artifact info records across the modules and functions.
Stanislaw Klekot [Tue, 19 Dec 2017 14:14:33 +0000 (15:14 +0100)]
Unified artifact info records across the modules and functions.

2 years agoRenamed upload/add/register operation to "store".
Stanislaw Klekot [Mon, 18 Dec 2017 19:03:31 +0000 (20:03 +0100)]
Renamed upload/add/register operation to "store".

2 years agoAdded in-memory artifacts registry.
Stanislaw Klekot [Mon, 18 Dec 2017 16:40:08 +0000 (17:40 +0100)]
Added in-memory artifacts registry.

This will be used for most of the time when GrailBag server is queried.
The registry is also a natural place to serialize delete/update operations.

2 years agoUpdated cross-references in grailbag_artifact module.
Stanislaw Klekot [Mon, 18 Dec 2017 16:34:21 +0000 (17:34 +0100)]
Updated cross-references in grailbag_artifact module.

2 years agoAdded listing artifacts from data directory.
Stanislaw Klekot [Mon, 18 Dec 2017 14:25:33 +0000 (15:25 +0100)]
Added listing artifacts from data directory.

2 years agoAdded functions for updating tags and tokens.
Stanislaw Klekot [Mon, 18 Dec 2017 11:38:22 +0000 (12:38 +0100)]
Added functions for updating tags and tokens.

2 years agoAdded reading part of artifact handles.
Stanislaw Klekot [Sun, 17 Dec 2017 22:03:08 +0000 (23:03 +0100)]
Added reading part of artifact handles.

2 years agoAdded code for working with artifact storage.
Stanislaw Klekot [Sun, 17 Dec 2017 21:20:58 +0000 (22:20 +0100)]
Added code for working with artifact storage.

Incomplete, write only for now.

2 years agoFilled server's skeleton.
Stanislaw Klekot [Sun, 17 Dec 2017 13:00:35 +0000 (14:00 +0100)]
Filled server's skeleton.

Updated example config a little.

2 years agoAdded skeleton for the server part.
Stanislaw Klekot [Sun, 17 Dec 2017 12:33:48 +0000 (13:33 +0100)]
Added skeleton for the server part.

2 years agoInit.
Stanislaw Klekot [Sun, 17 Dec 2017 11:54:01 +0000 (12:54 +0100)]
Init.