Feature #5904 Add Parameters and Detection for Mixed AFP/SMB and NFS/SMB Shares 08/25/2014 10:09 AM - Thomas Kaiser

FreeNAS - Feature #5904 Add parameters and detection for mixed AFP/SMB and NFS/SMB shares 08/25/2014 10:09 AM - Thomas Kaiser

Status: Ready for Testing Estimated time: 0.00 hour Priority: Nice to have Assignee: Andrew Walker Category: Services Target version: 11.3-BETA1 Severity: Medium Needs Merging: No Reason for Closing: Needs Automation: No Reason for Blocked: Support Suite Ticket: n/a Needs QA: Yes Hardware Configuration: Needs Doc: Yes Description Starting with Samba 4.2 a new VFS module called vfs_fruit (instead of vfs_apple) will be available to Samba that will address the most common challenges accessing the same set of data from OS X clients via AFP and SMB: File/record locking, encoding quirks introduced by Apple and especially access to Finder metadata and ressource forks. The author of the new module outlined the problems in [1]

In my opinion we need early adoption of this new VFS module as soon as Samba 4.2 is integrated in FreeNAS since it will both enhance compatiblity between Netatalk/Samba and also be a perfect starting point for the replacement of AFP with SMB2 for OS X 10.9/10.10 clients later (Samba 4.3 is expected to contain the loads of work to make Spotlight searches possible over SMB2 [2]). I put the manual page of vfs_fruit online: http://pastebin.com/TtSBeKB0

In our case (ZFS based) it's also necessary to stack the other necessary VFS modules vfs_catia and vfs_streams_xattr correctly and to config it all accordingly:

[cross platform share] vfs objects = catia fruit streams_xattr fruit:resource = xattr fruit:metadata = netatalk fruit:locking = netatalk fruit:encoding = private

Best,

Thomas

[1] http://sambaxp.org/fileadmin/user_upload/SambaXP2014-DATA/wed/track1/Ralph_Boehme-AppledancesSamba.pdf [2] https://lists.samba.org/archive/samba-technical/2014-August/101791.html

History #1 - 08/25/2014 12:30 PM - John Hixson - Status changed from Unscreened to Screened

#2 - 08/27/2014 01:13 AM - Nicki Messerschmidt I think this would be a very useful feature for FreeNas. I second this feature request.

#3 - 09/18/2014 05:13 AM - Thomas Kaiser BTW: Ralph the developer responsible for Samba's new vfs_fruit module shares a lot of knowledge in this list post regarding OS X as SMB client: https://lists.samba.org/archive/samba/2014-September/184761.html

Since Apple officially declared AFP end of live it's just a matter of time until Apple ships OS X without an AFP client so users have to think about

09/24/2021 1/6 switching protocols... and file server daemons. Samba 4.3 is expected to be one of the few '100 percent OS X compliant' SMB servers but to make a switch from Netatalk/AFP to Samba/SMB hasslefree it's necessary to configure both daemons appropriately.

I believe this should be one of the design goals regarding Samba running in FreeNAS/TrueNAS since both will automatically become more attractive for OS X installations since many admins will realize that Apple's switch of protocols doesn't mean a MS Windows based fileserver can serve both OS X and Windows clients since Windows will never implement Apple proprietary SMB extensions like Spotlight support or the AAPL stuff Ralph mentions in his list post above leading to painfully slow SMB share browsing from OS X clients.

Unfortunately I've been wrong regarding storage of Finder metadata and ressource forks in ZFS Extended Attributes since FreeBSD's ZFS implementation lacks the necessary xattr APIs available on Solaris. Therefore it must read "fruit:resource = file" instead. Both Netatalk and Samba should be configured to store the stuff in ._ AppleDouble files and vfs_fruit should hide the inner details from clients.

#4 - 03/27/2015 11:38 AM - Damjan Perenic I vote for this feature as well as I believe it is a must have feature for FreeNAS. I keep my photos on FreeNAS and some programs work awfully slow without this plugin (e.g. CaptureOne). Browsing through directories is much much slower than on the local drive. I tested it on Samba-beta and CaptureOne for example, works noticeably faster.

#5 - 02/17/2016 10:52 PM - Jordan Hubbard Possible calsoft candidate

#6 - 02/25/2016 10:34 AM - Jordan Hubbard - Assignee changed from John Hixson to Jakub Klama

#7 - 03/31/2016 06:06 AM - Anonymous freeNAS-9.10 (FreeBSD freenas.local 10.3-RC1 FreeBSD 10.3-RC1 #2 8d7981f(freebsd10)) on-wards /usr/local/lib/shared-modules/vfs/fruit.so is the shared library that introduces vfs_fruit module in freeNAS.

Till 9.3-RELEASE-p31 Samba version: 4.1.21 was used which did not had vfs_fruit module support.

As suggested by http://download.freenas.org/9.10/MASTER/201603290530/ReleaseNotes Filesharing: Samba (SMB filesharing) updated from version 4.1 to 4.3.4 freeNAS-9.10 and on-wards. This is also seen in freeNAS-10.2 (10.3-BETA2) which has Samba version: 4.3.4-GIT-UNKNOWN and have vfs_fruit module support.

The source code is also reachable at samba-4.2.9/source3/modules/vfs_fruit.c

#10 - 08/23/2016 06:37 AM - Anonymous (Private notes) Looks like vfs_fruit support is already added to FreeNAS9.10(also mentioned by Rohit above). Also related ordering Bug(Bug#16325) looks resolved now. What exactly is now expected to be done here?

#12 - 08/26/2016 02:00 PM - Kris Moore

09/24/2021 2/6 - Target version changed from 111 to N/A

#14 - 11/17/2016 11:24 AM - Kris Moore - Target version changed from 9.10.2 to 9.10.2-U1

#15 - 11/26/2016 02:18 PM - Evan Champion I tried this with my Macs (10.11 and 10.12) and found that using the recommended settings makes the share incompatible with an existing "normal" SMB share (without fruit) used by a Mac. Specific differences were:

- catia filename mapping is not the same as samba's default, so some files are not accessible. catia does not seem to be required and removing catia addressed this. - metadata stored in a different xattr. Using fruit:metadata = stream addressed this. - fruit sets ACLs on all new files, whereas my samba share (with dataset type = UNIX, case sensitive) did not. Using fruit:nfs_aces = no addressed this.

It seems like one needs to understand the specific deployment scenario and align to that:

- if one is replacing/augmenting an existing netatalk environment then perhaps the recommended configuration above using catia and netatalk-aligned fruit: settings are correct (I have not tried; I only use afp for time machine until samba support's Apple's requirements) - if one is augmenting an existing SMB share then catia may be an issue and fruit:metadata = stream is required, fruit:locking should not be set. - if one is adding fruity SMB to an existing NFS share, one might want to set other parameters to get the resulting file handling to match as closely as possible to native access from UNIX, which seemed to be: use catia, fruit:metadata = stream, fruit:encoding = native, streams_xattr:prefix = user., streams_xattr:store_stream_type = no. This is a bit speculative as I didn't test with an NFS client. - perhaps fruit:nfs_aces is appropriate for Windows datasets, but not a UNIX dataset without ACLs. Unlike samba by default, fruit creates files with ACLs set unless fruit:nfs_aces = no.

In all cases fruit:resource = file is required on FreeBSD's zfs for support of large resource forks.

A much bigger issue however is that even with no special smb.conf (using only FreeNAS' default settings) and no fruit: special settings, I have strange issues with access to the share that prevent further use. Here are examples:

- From Terminal I can create, list, change, view, then delete a given file. - From Finder I can see all files and can view files, but cannot copy, delete them. The behaviour is that you delete the file, it goes away for a couple seconds, and then appears again. - If I copy a file into the share using Finder, I can not delete it using Terminal either -- but I can view or change it. The error given on delete is either file not found, or resource busy if I have just edited the file. - If I login to FreeNAS via ssh and remove the file using the exact filename that was "not found" from Terminal, I can remove the file fine.

This is on 9.10.1-U4, UNIX dataset, case sensitive; I also tried making a Windows dataset with same behaviour. I have not figured out how to get this to work reasonably. Are people using this successfully running a nightly build with a newer version of samba perhaps?

Back to Apple's requirements for time machine support ( https://developer.apple.com/library/content/releasenotes/NetworkingInternetWeb/Time_Machine_SMB_Spec/) there are a number of changes required. These are described in the vfs_fruit feature request for time machine support (vfs_fruit feature request discussing time machine requirements: https://bugzilla.samba.org/show_bug.cgi?id=12380).

- samba needs to be configured to support for durable file handles. smb.conf(5) says: durable handles are only enabled if kernel oplocks = no, kernel share modes = no, and posix locking = no, i.e. if the share is configured for CIFS/SMB2 only access, not supporting interoperability features with local UNIX processes or NFS operations.

- F_FULLFSYNC support needs to be advertised via fruit:advertise_fullsync = true, and some code that not appear to be in a production samba release yet (pull request: https://github.com/samba-team/samba/pull/64).

- Specific requirements for Bonjour advertising. I have not checked in FreeNAS implements already.

09/24/2021 3/6 #16 - 12/23/2016 06:02 AM - Kris Moore - Target version changed from 9.10.2-U1 to 9.10.3

#18 - 04/03/2017 10:12 AM - Kris Moore - Target version changed from 9.10.3 to 9.10.4

#19 - 04/14/2017 11:31 AM - Thomas Kaiser John Hixson wrote:

Much work has been done. Much more is needed.

Just for the record: Ralph (SerNet dev) sent me his somewhat cleaned up smb.conf. We tested the settings at various customers and performance seemed to be fine though we ran into some weird issues that might be related to locking (but I asked for the worst case and graphics department delivered):

[global] netbios name = slowserver server role = standalone workgroup = SLOW

idmap config * : backend = tdb idmap config * : range = 10000-20000 disable netbios = yes smb ports = 445

debug pid = yes log file = /var/log/samba/%m.log log level = 2

read only = no ea support = yes store dos attributes = yes

# for OS X symblink bug aio read size = 1024 aio write size = 1024

[osx] path = /shares/osx vfs objects = catia fruit streams_xattr acl_xattr

09/24/2021 4/6 #20 - 04/19/2017 07:57 AM - Kris Moore - Target version changed from 9.10.4 to 11.1

#21 - 08/14/2017 07:30 AM - Dru Lavigne - Status changed from Investigation to 46

John: is there anything that stills need to be fixed here or was it resolved with the commit on the related bug?

#22 - 08/14/2017 02:23 PM - John Hixson - Status changed from 46 to Screened

Dru Lavigne wrote:

John: is there anything that stills need to be fixed here or was it resolved with the commit on the related bug?

There is much work that needs to be done here. This is a more general ticket about AFP and SMB integration, which sadly still needs to happen even with AFP going away.

#23 - 10/20/2017 06:53 AM - Kris Moore - Target version changed from 11.1 to 11.2-BETA1

#24 - 12/21/2017 06:10 AM - Kris Moore - Target version changed from 11.2-BETA1 to 11.3

#25 - 01/29/2018 07:33 AM - Kris Moore - Status changed from Screened to Not Started

#26 - 03/28/2018 11:27 PM - John Hixson - Assignee changed from John Hixson to Timur Bakeyev

#27 - 04/27/2018 11:29 AM - Ben Gadd - Target version changed from 11.3 to Backlog

#28 - 05/09/2018 07:58 PM - Timur Bakeyev - Severity set to Medium

Seems that requires Samba 4.8 to be fully functional...

#29 - 06/13/2018 10:56 AM - John Hixson - Assignee changed from Timur Bakeyev to John Hixson

#30 - 07/12/2018 03:32 PM - John Hixson - Category changed from OS to Services

09/24/2021 5/6 #32 - 10/15/2018 10:11 AM - Dru Lavigne - Assignee changed from John Hixson to William Grzybowski

#34 - 11/14/2018 07:50 AM - William Grzybowski - Assignee changed from William Grzybowski to Andrew Walker

#36 - 12/19/2018 01:42 PM - Andrew Walker Heavy lifting with mdns is now done. Next steps are to plan out how to handle datasets being shared via multiple filesharing protocols.

#37 - 01/08/2019 07:26 AM - Bug Clerk - Status changed from Unscreened to In Progress

#38 - 01/08/2019 01:08 PM - Bug Clerk - Status changed from In Progress to Ready for Testing

11.3 PR: https://github.com/freenas/freenas/pull/2364

#39 - 01/21/2019 07:15 AM - Dru Lavigne - Subject changed from vfs_fruit (better Samba/Netatalk integration) to Add parameters and detection for mixed AFP/SMB and NFS/SMB shares - Needs Merging changed from Yes to No

#40 - 01/21/2019 11:02 AM - Dru Lavigne - Target version changed from 11.3 to 11.3-BETA1

09/24/2021 6/6