I pushed a significant upgrade to fdb to GitHub (version 2.07) today.
Key new features include:
- The ability to change permissions on tags and namespaces
- The ability to list namespaces and tags, with a Unix-style ls (list sorted) command, supporting many options that will be familiar to Unix users—ls -l, ls -g, ls -R, ls -d etc all do something like what you might expect, and ls -L gives a full Fluidinfo listing of the permissions structure in all its glory.
- Documentation: there is a complete set of documentation for the fdb command line, including installation and configuration notes. This is supplied (both as source and built) in the repository and is available directly from Fluidinfo at http://fluiddb.fluidinfo.com/about/fdb/njr/index.html.
- There is basic support for handling multiple Fluidinfo identities with the su and whoami commands.
- The file fdb.py has been renamed as fdb to make it even simpler to use directly from an alias, link or by adding the fdb directory to a shell path.
The rest of this post will illustrate the new ls command, for listing namespaces and tags and information about them, and the perms command which is used for changing permissions.
The ls command is used to view a sorted list of tags or namespaces, potentially with additional information.
FORM
ls [flags] [<namespace>|<tag>]
FLAGS
- -l long listing (one per line; showing permissions)
- -g group listing (one per line; showing exception groups and permissions)
- -L longer listing (show full Fluidinfo-style permissions listing)
- -n list the namesace as an object, rather than the contents of the namespace
- -d same as -n
- -R recursive (show contents of all subnamespaces, recursively)
as well as the standard flags like -s etc.
EXAMPLES
List the tags in the user’s namespace:
(Here, we assume we are authenticated as user miro):
$ fdb ls bestsellers-1998-2010/ first_field_id n_records bestsellers1998to2010/ first_record_id next-field-about books/ forename next-field-id class has-about-links next_field_about consistent has-field-numbers next_field_id description has-id-links planets/ elements/ has-record-numbers rating field-name has_about_links small/ field-number has_field_numbers surname field_number has_id_links table-name first-field-about has_record_numbers testconvtag first-field-id message testrating first-record-about n-fields testtable/ first-record-id n-records type first_field_about n_fields unitNote that namespaces are shown with a trailing /; for users familiar with the unix ls command, this is modelled on ls -F.
List the tags and subnamespaces in a given namespace:
$ fdb ls miro/planets Atmosphere Mass OrbitalRadius Category Moons RotationPeriod EquatorialDiameter Name db-next-record-about HasRings OrbitalEccentricity db-record-number Inclination OrbitalPeriodSort order is case-sensitive, e.g. (in English) all lower-case letters follow upper-case letters.
List a single tag (report presence or absence of that tag).
$ fdb ls miro/planets/Mass miro/planets/Mass $ fdb ls -F miro/planets/NoMass miro/planets/NoMass not foundLong listing for tag (njr/rating), including permissions summary:
$ fdb ls -l njr/rating trwcr--r-- ratingHere:
the first t indicates that this is a tag rather than a namespace
the next three characters indicate that the owner (njr) has read, write and control permission on the tag,
the final three characters indicate that everyone (world) has read permission but not write or control permission on the tag.
the “middle” three characters (r--) show group permissions. Their meaning depends on the world permissions. Because the world has read permission the only thing that an exception list can do is to remove it; so in this case, the r in the middle r-- block indicates that no one is losing read permission as a result of being on an exception list. If there were an exception list that excluded some people, then the group read permission would be shown as -.
If the world did not have read permission, an r would mean that at least one person has read permission in addition to the owner; and a - would indicate that no one except the owner had read permission.
the final thing to know about this way of describing permissions is that Fluidinfo actually has more than one kind of write permission for both tags and namespaces, and more than one kind of control permission for tags. Normally, all the write permissions are set consistently, as are the two control permissions for tags. If this is not the case, the permission will be shown as a /.
Use the -g flag to find out who is in the group/exception list, or -L for the even longer, full Fluidinfo description of the permissions.
For more information on this way of looking at Fluidinfo permissions see A Simpler Interface to Fluidinfo Permissions.
Group long listing for tag (njr/rating), including permissions summary.
Let’s start with a tag with default permissions.
$ fdb ls -g private-tag trwcr--r-- (world) njr/private-tagWhen no one has been given any unusual permissions, in effect there is no group; or to say it another way, group permissions can be considered to be the same as world permissions.
Now let’s grant read and write access to jkakar and ntoll
$ fdb perms group jkakar+ntoll njr/private-tag $ fdb ls -g private-tag trwcrw---- ntoll+jkakar njr/private-tagGroup permission is now rw-, and the group is ntoll+jkakar. (The exception list also includes njr, of course.)
Now let’s set different groups for read and write. We’ll let ntoll have write permission too. To do this we update the write group:
$ fdb perms group-write ntoll private-tag $ fdb ls -g njr/private-tag trwcrw---- r:ntoll+jkakar w:ntoll njr/private-tagWhen the groups are different, they are shown separately, with r: prefixing the read group and w; prefixing the write group.
Longer listing for tag (njr/rating), including Fluidinfo-style permissions summary:
$ fdb ls -L njr/rating njr/rating: ABSTRACT TAG (/tags) Write update (metadata): policy: closed; exceptions [njr] delete (delete): policy: closed; exceptions [njr] Control control (control): policy: closed; exceptions [njr] TAG (/tag-values) Read read (read): policy: open; exceptions [] Write create (tag): policy: closed; exceptions [njr] delete (untag): policy: closed; exceptions [njr] Control control (control): policy: closed; exceptions [njr]Long listing for contents of namespace (njr/index):
$ fdb ls -l njr/index trwcr--r-- about trwcr--r-- classGroup long listing for namespace itself (njr/private), including permissions summary.
The -d tells ls that what you want is not the content of the namespace, but the permissions on the namespace itself. (d stands for directory, which is essentially what a namespace is. You can use -n if you prefer.)
The result is very similar to that for tags. We’ll just look at a namespace fi
$ fdb ls -gd njr/fi nrwcrw-r-- r:(world) w:terrycojones+paparent njr/fi/This is showning that the namespace njr/fi has standard world read permissions (with no special group) but that terrycojones and paparent have write permission for the namespace.
Long listing for a namespace itself (njr/index) (as opposed to its contents), including Fluidinfo-style permissions summary:
$ fdb ls -ln njr/index nrwcr--r-- index $ fdb ls -ld njr/index nrwcr--r-- indexThe first n indicates that njr/index is a namespace rather than a tag. This option may be specified using -n (for namespace) or -d (for directory) since the latter will probably be more natural for users familiar with unix-style ls.
See the section on permissions for an explanation of the permissions string.
Longer listing for a namespace itself (njr/index) (as opposed to its contents):
$ fdb ls -Ld njr/index NAMESPACE (/namespaces) Read list (read): policy: open; exceptions [] Write create (create): policy: closed; exceptions [njr] delete (delete): policy: closed; exceptions [njr] Control control (control): policy: closed; exceptions [njr]Long listing including group details (for tag njr/rating)
$ fdb ls -ln njr/fi nrwcr----- ceronman+esteve+jkakar+ntoll+terrycojones fiSee the section on permissions for an explanation of the permissions string. In this case
- the first n indicates that this is a namespace
- the next three characters indicate that the owner (njr) has read, write and control permission on the tag,
- the next three characters indicate that members of the exception group exceptions have read (“list”) permission on the tag (but not write or control permissions).
- the final three characters indicate that Fluidinfo users other than the owner and the people in the exception group do not have any permissions to read, write or control the namespace.
The list of users separated by plus signs is the list of users on the exceptions list.
The next example shows Fluidinfo’s native representation of the permissions for this namespace.
Longer listing for namspace (njr/fi), including Fluidinfo-style permissions summary:
$ fdb ls -Ln njr/fi Permissions for namespace njr/fi: READ: list (read): policy: closed; exceptions: [ceronman, esteve, jkakar, njr, ntoll, terrycojones] WRITE: create (create): policy: closed; exceptions: [njr] update (metadata): policy: closed; exceptions: [njr] delete (delete): policy: closed; exceptions: [njr] CONTROL: control (control): policy: closed; exceptions: [njr]List contents of all subnamespaces (recursive descent):
ls -R miro miro: bestsellers-1998-2010/ first_field_id n_records bestsellers1998to2010/ first_record_id next-field-about books/ forename next-field-id class has-about-links next_field_about consistent has-field-numbers next_field_id description has-id-links planets/ elements/ has-record-numbers rating field-name has_about_links small/ field-number has_field_numbers surname field_number has_id_links table-name first-field-about has_record_numbers testconvtag first-field-id message testrating first-record-about n-fields testtable/ first-record-id n-records type first_field_about n_fields unit miro/bestsellers-1998-2010: ASP db-next-record-about rank RRP db-record-number title author imprint value binding productclass volume date publisher miro/bestsellers1998to2010: miro/books: author guardian-1000 year db-next-record-about surname forename title miro/elements: AtomicWeight Description Period BoilingPointC Etymology RelativeAtomicMass BoilingPointF Group Symbol ChemicalSeries MeltingPointC Z Colour MeltingPointKelvin db-next-record-about Density Name db-record-number miro/planets: Atmosphere Mass OrbitalRadius Category Moons RotationPeriod EquatorialDiameter Name db-next-record-about HasRings OrbitalEccentricity db-record-number Inclination OrbitalPeriod miro/small: db-record-number id intField miro/testtable: db-next-record-about db-record-number i
The perms command provides a simple interface for changing the permissions on tags and namespaces. It only supports five cases, but there are powerful enough to cover the vast bulk of common situations.
FORM
fdb perms permissions-spec list-of-one-or-more-tags-or-namespaces
Here permissions-spec can be
- private
- default
- lock
- unlock
- group list+of+usernames+separated+by+pluses
- group-write list+of+usernames+separated+by+pluses
- group-read list+of+usernames+separated+by+pluses
EXAMPLES
The eight cases are as follows and apply equally namespaces and tags. In what follows, we’ll use a rating tag and a namespace called friends.
Make a tag or namespace completely private.
$ fdb perms private njr/rating $ fdb perms private njr/fiWe could also change them together by saying:
$ fdb perms private njr/rating njr/fiThis sets the permissions so that only the owner of the tag or namespace has permission to do anything with it—read, write and control permission. The abbreviated view of this is shown by
$ fdb ls -ld njr/rating njr/fi trwc------ njr/rating nrwc------ njr/fi/(See fdb ls for an detailed explanation of the permissions codes.)
The full listing after this is:
$ fdb -F ls -Ld njr/rating njr/fi njr/rating: ABSTRACT TAG (/tags) Write update (metadata): policy: closed; exceptions = [njr] delete (delete): policy: closed; exceptions = [njr] Control control (control): policy: closed; exceptions = [njr] TAG (/tag-values) Read read (read): policy: closed; exceptions = [njr] Write create (tag): policy: closed; exceptions = [njr] delete (untag): policy: closed; exceptions = [njr] Control control (control): policy: closed; exceptions = [njr] njr/fi/: NAMESPACE (/namespaces) Read list (read): policy: closed; exceptions = [njr] Write create (create): policy: closed; exceptions = [njr] delete (delete): policy: closed; exceptions = [njr] Control control (control): policy: closed; exceptions = [njr]Restore default permissions to a tag or namespace. The default is that the owner has read, write and control and everyone else has read only:
$ fdb perms default njr/rating njr/fi $ fdb -F ls -ld njr/rating njr/fi trwcr--r-- njr/rating nrwcr--r-- njr/fi/Lock a namespace or tag so that it can’t be edited (i.e., remove all write permissions).
$ fdb perms lock njr/rating $fdb ls -l njr/rating tr-cr--r-- njr/ratingUnlock a namespace or tag so that the owner (only) can edit it.
$ fdb perms unlock njr/rating $fdb ls -l njr/rating trwcr--r-- njr/ratingSet the permissions on a tag or namespace so that one or more extra people can write it. (This does not alter read permissions.)
$ fdb perms group-write ntoll+jkakar njr/rating njr/fi $ fdb ls -gd njr/rating njr/fi fdb -F ls -gd njr/rating njr/fi trwcrw-r-- r:(world) w:ntoll+jkakar njr/rating nrwcrw-r-- r:(world) w:ntoll+jkakar njr/fi/Set the permissions so that only a particular group can read it, leaving write permissions as the are.
$ fdb perms default njr/rating njr/fi $ fdb perms group-read ntoll+jkakar+miro njr/rating njr/fi $ fdb -F ls -gd njr/rating njr/fi trwcr----- r:ntoll+jkakar+miro w:(world) njr/rating nrwcr----- r:ntoll+jkakar+miro w:(world) njr/fi/Set permissions so that one group of users can write tge tag or namespace, and another group can read it. In this case, let’s allow miro to write and ntoll and jkakar (as well as miro) to read.
$ fdb perms group-read ntoll+jkakar+miro njr/rating njr/fi $ fdb perms group-write miro njr/rating njr/fi $ fdb ls -gd njr/rating njr/fi trwcrw---- r:ntoll+jkakar+miro w:miro njr/rating nrwcrw---- r:ntoll+jkakar+miro w:miro njr/fi/The final case is really just a special case of the previous one, where the read and write groups are the same. Let’s make miro the only user (other the owner, njr) who can read or write the tag and namespace.
$ fdb perms group miro njr/rating njr/fi $ fdb ls -gd njr/rating njr/fi trwcrw---- miro njr/rating nrwcrw---- miro njr/fi/
NOTE
Removing read access to a namespace prevents people from listing that namespace but does not stop them working with the contents of the namespace if they can find them. So if you have a namespace called secret and set its permissions to nrwc------ (owner-only access) and in it have a tag called password with its permission set to the default trwcr--r-- then anyone will be able to read the password if they discover it exists.
This is clearly not a good situation; the team is aware of it and I expect some resolution will be forthcoming at some point.
Note also that the existence of a tag cannot really be hidden in Fluidinfo, so people will be able to find out that you have a tag called secret/password.