Thursday, April 2, 2015

New ACL in Solaris with ZFS

Quite some time ago I wrote about ACL's in my blog entry ACL!...Bless You. A funny title and play on the pronunciation of the acronym Access Control List ("Ackel"), but not readily found via Google. Sadly, if you are running Solaris 10 with ZFS or better yet a Nevada or OpenSolaris build you are going to get confused if you do a search and get ancient articles telling you to use getfacl and setfacl. These tools are used for viewing or manipulating POSIX ACL's. Things are different now because ZFS makes use of NFSv4 style ACL's manipulated with often unknown arguments to ls and chmod. So when do you use POSIX and when do you use NFSv4? And, if ZFS uses NFSv4 ACL's, what does that mean if you NFSv3 mount a ZFS filesystem? Lets explore.

Before we begin!: Please run "which ls" to ensure you are using /bin/ls. If your running a newish build of Nevada or OpenSolaris you may be using GNU ls, rather than Solaris ls, in which case this blog entry may confuse and irritate you. If you use "ls -v" and don't see the results you expect, you are probly using GNU ls.

ACL Basics Access Control Lists (ACL) allow you to assign arbitrary permissions beyond that which is allowed by the traditional "trivial" UNIX model. Traditionally a file has 1 owner and 1 group, and there are read/write/execute permissions assigned to this owner, this group, and additionally to "everyone" as a catch all. The traditional way of giving restricted access to more than one user is the group... but what if you want to give write access to two users in different groups? Duh, create a new group! Simple, why do we need this crap?

There are a lot of problems with group permissions that aren't obvious to the casual user. For instance, all users in the group have the same permissions with the exception of the owner, so if you have a sensitive file that can't be world readable and you want to allow one set of people to read it and a smaller subset of people who can modify it, well you're out of luck. Or, perhaps you have a pretty strict set of groups setup per department and you need managers from different departments to access a directory, you need yet another group but if thats not possible or practical (policy can be a bitch) your out of luck again.

ACL's give us the freedom to be choosey about who can do what with a file or group of files.

The most basic thing you need to know is when ACL's are in play. On the CLI that can be hard to tell, so you need to train your eyes to see it. In the following example notice the file with the "+":

benr@ultra data$ ls -alh total 6 drwxr-xr-x 2 benr staff 512 Jun 5 11:43 . drwxr-xr-x 3 root sys 2.0K Jun 5 11:43 .. -rw-r--r-- 1 benr staff 0 Jun 5 11:43 file1 -rw-r--r--+ 1 benr staff 0 Jun 5 11:43 file2 -rw-r--r-- 1 benr staff 0 Jun 5 11:43 file3 -rw-r--r-- 1 benr staff 0 Jun 5 11:43 file4 -rw-r--r-- 1 benr staff 0 Jun 5 11:43 file5 file2 above has an ACL set, the rest do not. You really want to learn to look for that and not mentally tune it out. While we're at it, please be aware that an "@" sign designated Extended Attributes on a file (ie: "-rw-rw-r--@"), we don't discuss those now, but know that its possible.

Thinking ACL's Do yourself a favor... don't think of ACL's as being enabled or disabled, they are always present because after all, the traditional 1 owner 1 group "rwx" model is technically Access Control... its just a crappy form of it. Rather than "enabled" or "disabled" think "trivial" or "non-trivial". This is the terminology used in other documentation and I think it fits best. Therefore a "+" file possesses a non-trivial ACL, whereas a "normal" file has a trivial ACL.

Old School: POSIX ACL's and UFS POSIX ACL's, or what most admins probly think of as (classic) "Solaris ACLs", are interacted with using getfacl to view permissions and setfacl to get. These are most commonly used on pre-Solaris 10 systems and UFS.

POSIX ACL's simply extend the traditional model, there are no new access controls. That is, you are still limited to the old read/write/execute permisisons, but you can now have more than one owner or more than one group.

Lets look at an example using the "old skool" methods, notice that getfacl gives me output even if we're using trivial permisions:

benr@ultra data$ ls -l file5 -rw-r--r-- 1 benr staff 0 Jun 5 11:43 file5 benr@ultra data$ getfacl file5

file: file5
owner: benr
group: staff
user::rw- group::r-- #effective:r-- mask:r-- other:r-- The above file has a "trivial" ACL, plain ol' UNIX perms. Lets now add 2 additional users and 2 additional groups:

benr@ultra data$ setfacl -m user:postgres:rw- file5 benr@ultra data$ setfacl -m user:mysql:rw- file5 benr@ultra data$ setfacl -m group:postgres:r-- file5 benr@ultra data$ setfacl -m group:mysql:r-- file5 benr@ultra data$ ls -l file5 -rw-r--r--+ 1 benr staff 0 Jun 5 11:43 file5 benr@ultra data$ getfacl file5

file: file5
owner: benr
group: staff
user::rw- user:postgres:rw- #effective:r-- user:mysql:rw- #effective:r-- group::r-- #effective:r-- group:mysql:r-- #effective:r-- group:postgres:r-- #effective:r-- mask:r-- other:r-- So there we have it. The classic POSIX ACL example. The setfacl ("set file acl" if you didn't infer that) has several flags, but the most commonly used is "-m" to add/modify ACL entries and "-d" to delete entries. Delete entries like so:

benr@ultra data$ setfacl -d user:mysql:rw- file5 benr@ultra data$ getfacl file5

file: file5
owner: benr
group: staff
user::rw- user:postgres:rw- #effective:r-- group::r-- #effective:r-- group:mysql:r-- #effective:r-- group:postgres:r-- #effective:r-- mask:r-- other:r-- ls & chmod for the Win In this modern era, the commands above are no longer required! Yup, you can use ls -v to display "verbose" ACLs and chmod A... to set! Lets look at that file above again, but this time we'll use ls and chmod:

benr@ultra data$ ls -v file5 -rw-r--r--+ 1 benr staff 0 Jun 5 11:43 file5

    0:user::rw-
    1:user:postgres:rw-                #effective:r--
    2:group::r--               #effective:r--
    3:group:mysql:r--          #effective:r--
    4:group:postgres:r--               #effective:r--
    5:mask:r--
    6:other:r--
benr@ultra data$ chmod A-user:postgres:rw- file5 benr@ultra data$ ls -v file5 -rw-r--r--+ 1 benr staff 0 Jun 5 11:43 file5

    0:user::rw-
    1:group::r--               #effective:r--
    2:group:mysql:r--          #effective:r--
    3:group:postgres:r--               #effective:r--
    4:mask:r--
    5:other:r--
Spiffy eh? The output above is in POSIX ACL format, ls -v will output both POSIX and NFSv4 ACL's, the only way to know which your using is based on the output, and that owner/group/other traditional look is POSIX.

So the only real change to using chmod is that we prefix our ACL operation with "A+" to add an ACL entry or "A-" to remove it. In the example above, "A-user:postgres:rw-" means ACL ("A") remove ("-") the ACL string ("user:postgres:rw-"), put it all together and we remove the ACL entry which makes "postgres" an owner of the file with rw privs. Run the same command with "A+" instead of "A-" to add it back.

New Hotness: NFSv4 Style ACL's and ZFS NFSv4 included a standard for ACLs. This standard is a major upgrade to the existing POSIX ACL capabilities and is interoperable with CIFS. For instance, I can give the user "tamarah" Write access to a file using POSIX ACLs, but with NFSv4 ACLs I can give "tamarah" access to only Append to the end of a file. Thats pretty handy and much more granular!

The following is list of NFSv4 ACL attributes:

•read_data: Ability to read the contents of a file •write_data: Ability to modify an existing file •list_directory: Ability to list the contents of a directory •add_file: Ability to add a new file to a directory •append_data: Ability to modify an existing file, but only from EOF •add_subdirectory: Ability to create subdirectories •read_xattr: Ability to read extended attributes •write_xattr: Ability to write extended attributes •execute: Ability to execute a file •delete_child: Ability to delete a file within a directory •read_attributes: Ability to read basic attributes (non-ACL) of a file (ie: ctime, mtime, atime, etc) •write_attributes: Ability to write basic attributes to a file or directory (ie: atime, mtime) •delete: Ability to delete a file •read_acl: Ability to read the ACL •write_acl: Ability to modify the ACL (needed to use chmod or setfacl) •write_owner: Ability to use chown to change ownership of a file •synchronize: Ability to access file locally via synchronous reads and writes Thats a lot of control!

With NFSv4 ACL's the getfacl and setfacl commands are dead. Given that chmod and ls work with both POSIX and NFSv4 ACL's I highly recommend that you concentrate on using those tools, besides they are your old friends anyway.

Each file will have at least 6 Access Control Entries, these are "allow" and "deny" for our 3 classic friends "owner", "group", and "everyone" (rather than "other"). If you've worked with Apache or a firewall this concept of allow and deny will be familiar. Quite simply, there are actions that we explicitly allow and others that we explicitly deny, if an action is neither its not allowed. At first the idea of explicitly denying seems redundant, just don't allow it! But this is all about layering, so if you explicitly deny the Write permissions your saying that no one should be able to even if someone is given Write permission.

Lets play with a newly created file on ZFS. Here is the default permissions:

benr@ultra ~$ ls -v SecretFile.txt -rw-r--r-- 1 benr staff 27 Jun 5 22:58 SecretFile.txt

    0:owner@:execute:deny
    1:owner@:read_data/write_data/append_data/write_xattr/write_attributes/write_acl/write_owner:allow
    2:group@:write_data/append_data/execute:deny
    3:group@:read_data:allow
    4:everyone@:write_data/append_data/write_xattr/execute/write_attributes/write_acl/write_owner:deny
    5:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow
The syntax here is important, its the same syntax we'll use to modify or add permissions via chmod. Here is the ACL entry syntax listed in acl(5):

         owner@:[:inheritance flags]:
         group@:[:inheritance flags]:
         everyone@:[:inheritance flags]:
         user:[:inheritance flags]:
         group:[:inheritance flags]:
Entries contain "@" represent file owner as seen with "ls". Multiple entries may be specified together seperated by coma's:

      user:fred:read_data/write_data/read_attributes:file_inherit:allow
      owner@:read_data:allow,group@:read_data:allow,user:tom:read_data:deny
So lets add some permissions to a test file:

         # NOTICE:           A +<------------------user allow  --------------->, <---user deny ---->  __FILE__
benr@ultra ~$ chmod A+user:backup:read_data/write_data/read_attributes:allow,user:backup:delete:deny SecretFile.txt benr@ultra ~$ ls -v SecretFile.txt -rw-r--r--+ 1 benr staff 27 Jun 5 22:58 SecretFile.txt

    0:user:backup:read_data/write_data/read_attributes:allow
    1:user:backup:delete:deny
    2:owner@:execute:deny
    3:owner@:read_data/write_data/append_data/write_xattr/write_attributes
        /write_acl/write_owner:allow
    4:group@:write_data/append_data/execute:deny
    5:group@:read_data:allow
    6:everyone@:write_data/append_data/write_xattr/execute/write_attributes
        /write_acl/write_owner:deny
    7:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize
        :allow
Now lets test it!!!
backup@ultra ~$ id uid=502(backup) gid=1(other) groups=1(other) backup@ultra ~$ rm SecretFile.txt rm: cannot remove `SecretFile.txt': Permission denied backup@ultra ~$ echo "More Info" >> SecretFile.txt backup@ultra ~$ cat SecretFile.txt The dog barks at midnight. More Info backup@ultra ~$ rm SecretFile.txt rm: cannot remove `SecretFile.txt': Permission denied Crafting ACL entry strings can be really tricky because of the number of individual permissions. I personally recommend using a GUI file manager, such as GNOME's Nautilus file manager:


If you do want to build those strings on your own take some time and sit down to read the acl(5) man page. (Remember: To read the section 5 "acl" man page use the command "man -s 5 acl", not "man acl" which will return acl(2).

NFSv3, ACL's and ZFS All this can get really confusing when you actually start talking about NFS for real. The first thing you've got to understand is that the NFS spec does not address ACL's. Strictly speaking, ACL's are a filesystem thing, not a transport thing. Thus, the UFS filesystem your sharing might know what ACL's are but your NFS client and server don't. Sun added a "sideband" (so called by Hal Stern) protocol in Solaris 2.5.1 to allow ACL's to work on NFS. While NFSv2 can be mounted with the "aclok" mount option, it isn't real support so it always is as gracious as possible... in otherwords, don't bother.

NFSv3 works pretty smoothly when backed by UFS. If you take a look at an NFSv3 mount using nfsstat -m you can see whether the server supports ACL's or not:

root@ultra /$ mount -F nfs -o vers=3 XXXXXXX:/export/share /a root@ultra /$ nfsstat -m /a from XXXXXX:/export/share

Flags:         vers=3,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
Attr cache:    acregmin=3,acregmax=60,acdirmin=30,acdirmax=60
root@ultra /$ cd /a root@ultra a$ ls -al total 3 drwxrwxrwx 2 root root 512 Jun 5 17:39 . drwxr-xr-x 61 root root 1536 May 21 02:09 .. -rw-r--r-- 1 nobody nobody 0 Jun 5 17:39 file root@ultra a$ getfacl file

file: file
owner: nobody
group: nobody
user::rw- group::r-- #effective:r-- mask:r-- other:r-- root@ultra a$ /bin/ls -V file -rw-r--r-- 1 nobody nobody 0 Jun 5 17:39 file

    0:user::rw-
    1:group::r--               #effective:r--
    2:mask:r--
    3:other:r--
root@ultra a$ /bin/ls -V file -rw-r--r-- 1 nobody nobody 0 Jun 5 17:39 file

    0:user::rw-
    1:group::r--               #effective:r--
    2:mask:r--
    3:other:r--
root@ultra a$ root@ultra a$ setfacl -m user:postgres:rw- file root@ultra a$ /bin/ls -V file -rw-r--r--+ 1 nobody nobody 0 Jun 5 17:39 file

    0:user::rw-
    1:user:postgres:rw-                #effective:r--
    2:group::r--               #effective:r--
    3:mask:r--
    4:other:r--
Notice in the above NFSv3 on UFS example that POSIX ACL's work fine, nothing special has been done here. In fact, for completeness here is the configuration of the share on the server: "share -F nfs -o rw -d "Testing" /export/share". There was nothing to configure or setup and both chmod and setfacl work as expected.

But what about ACL support over NFSv3 for filesystems on ZFS? It won't work. Remember, NFS is just passing your ACL request to the filesystem. Because ZFS doesn't support POSIX ACL's sending them via NFSv3 won't make a difference. By the same token, NFSv4 ACL support is fine. So if you need ACL support, upgrade to NFSv4. Please note you'll need ot start up the SMF nfs/mapid service, the "NFS user and group id mapping daemon".

The ACL Takeaway Here's what I want you to walk away with, even if you only skimmed this blog entry:

•There are TWO types of file ACL's in Solaris: POSIX and NFSv4 •NFSv4 ACL's are very granular and powerful •ACL's are a pita. •NFSv3 ACL support (POSIX) does not work when sharing a ZFS filesystem; Use NFSv4. •GUI's are an ACL's best friend, sad but true. •Avoid them if you can, but if/when you need them, they are there.

No comments:

Post a Comment